-| `email` | The reported email address for this user | JWT, SAML | MSA, Azure AD | This value is included by default if the user is a guest in the tenant. For managed users (the users inside the tenant), it must be requested through this optional claim or, on v2.0 only, with the OpenID scope. This value isn't guaranteed to be correct, and is mutable over time - never use it for authorization or to save data for a user. For more information, see [Validate the user has permission to access this data](access-tokens.md). If you are the email claim for authorization, we recommend [performing a migration to move to a more secure claim](./migrate-off-email-claim-authorization.md). If you require an addressable email address in your app, request this data from the user directly, using this claim as a suggestion or prefill in your UX. |

-| `email` | | Can be used for both SAML and JWT responses, and for v1.0 and v2.0 tokens. |

- | | `replace_unverified_email_with_upn` (Preview) | In scenarios where email ownership is not verified, the `email` claim returns the user's home tenant UPN instead, unless otherwise stated. For managed users, email is verified if the home tenant owns the email's domain as a custom domain name. For guest users, email is verified if either the home or resource tenants own the email's domain. If the user authenticates using Email OTP, MSA, or Google federation, the `email` claim remains the same. If the user authenticates using Facebook or SAML/WS-Fed IdP federation, the `email` claim isn't returned. The `email` claim isn't guaranteed to be mailbox addressable, regardless of whether it is verified. |

-description: Learn how to migrate your application away from using insecure claims, such as email, for authorization purposes.

-# Migrate away from using email claims for authorization

-This article is meant to provide guidance to developers whose applications are currently using a pattern where the email claim is used for authorization, which can lead to full account takeover by another user. Continue reading to learn more about if your application is impacted, and steps for remediation.

-## How do I know if my application is impacted?

-Microsoft recommends reviewing application source code and determining whether the following patterns are present:

-These patterns are considered insecure, as users without a provisioned mailbox can have any email address set for their Mail (Primary SMTP) attribute. This attribute is **not guaranteed to come from a verified email address**. When an unverified email claim is used for authorization, any user without a provisioned mailbox has the potential to gain unauthorized access by changing their Mail attribute to impersonate another user.

-This risk of unauthorized access has only been found in multi-tenant apps, as a user from one tenant could escalate their privileges to access resources from another tenant through modification of their Mail attribute.

-## Migrate applications to more secure configurations

-You should never use mutable claims (such as `email`, `preferred_username`, etc.) as identifiers to perform authorization checks or index users in a database. These values are re-usable and could expose your application to privilege escalation attacks.

-If your application is currently using a mutable value for indexing users, you should migrate to a globally unique identifier, such as the object ID (referred to as `oid` in the token claims). Doing so ensures that each user is indexed on a value that can't be re-used (or abused to impersonate another user).

-If your application uses email (or any other mutable claim) for authorization purposes, you should read through the [Secure applications and APIs by validating claims](claims-validation.md) and implement the appropriate checks.

-## Short-term risk mitigation

-To mitigate the risk of unauthorized access before updating application code, you can use the `replace_unverified_email_with_upn` property for the optional `email` claim, which replaces (or removes) email claims, depending on account type, according to the following table:

-| **User type** | **Replaced with** |

-||-|

-| On Premise | Home tenant UPN |

-| Cloud Managed | Home tenant UPN |

-| Microsoft Account (MSA) | Email address the user signed up with |

-| Email OTP | Email address the user signed up with |

-| Social IDP: Google | Email address the user signed up with |

-| Social IDP: Facebook | Email claim isn't issued |

-| Direct Fed | Email claim isn't issued |

-Enabling `replace_unverified_email_with_upn` eliminates the most significant risk of cross-tenant privilege escalation by ensuring authorization doesn't occur against an arbitrarily set email value. While enabling this property prevents unauthorized access, it can also break access to users with unverified emails. Internal data suggests the overall percentage of users with unverified emails is low and this tradeoff is appropriate to secure applications in the short term.

-The `replace_unverified_email_with_upn` option is also documented under the documentation for [additional properties of optional claims](active-directory-optional-claims.md#additional-properties-of-optional-claims).

-Enabling `replace_unverified_email_with_upn` should be viewed mainly as a short-term risk mitigation strategy while migrating applications away from email claims, and not as a permanent solution for resolving account escalation risk related to email usage.

-## Next steps

-| `scope` | Required | The value passed for the `scope` parameter in this request should be the resource identifier (application ID URI) of the resource you want, affixed with the `.default` suffix. For the Microsoft Graph example, the value is `https://graph.microsoft.com/.default`. <br/>This value tells the Microsoft identity platform that of all the direct application permissions you have configured for your app, the endpoint should issue a token for the ones associated with the resource you want to use. To learn more about the `/.default` scope, see the [consent documentation](v2-permissions-and-consent.md#the-default-scope). |

-| `scope` | Required | The value passed for the `scope` parameter in this request should be the resource identifier (application ID URI) of the resource you want, affixed with the `.default` suffix. For the Microsoft Graph example, the value is `https://graph.microsoft.com/.default`. <br/>This value informs the Microsoft identity platform that of all the direct application permissions you have configured for your app, it should issue a token for the ones associated with the resource you want to use. To learn more about the `/.default` scope, see the [consent documentation](v2-permissions-and-consent.md#the-default-scope). |

-To view the local administrator password for a Windows device joined to Azure AD, you must be granted the *deviceLocalCredentials.Read.All* permission.

-To view the local administrator password metadata for a Windows device joined to Azure AD, you must be granted the *deviceLocalCredentials.Read* permission.

-The following built-in roles are granted these permissions by default:

-|Built-in role|DeviceLocalCredential.Read.All|DeviceLocalCredential.Read|

-Any roles not listed are granted neither permission.

-* SSO: Once youΓÇÖve setup provisioning for your SAP application, youΓÇÖll want to enable single sign-on for those applications. Azure AD can serve as the identity provider and server as the authentication authority for your SAP applications. Learn more about how you can [configure Azure AD as the corporate identity provider for your SAP applications](https://help.sap.com/docs/IDENTITY_AUTHENTICATION/6d6d63354d1242d185ab4830fc04feb1/058c7b14209f4f2d8de039da4330a1c1.html).

-Custom workflows: When a new employee is hired in your organization, you may need to trigger a workflow within your SAP server.

-* Using the [Entra Identity Governance Lifecycle Workflows](lifecycle-workflow-extensibility.md) in conjunction with the [SAP connector in Azure Logic apps](https://learn.microsoft.com/azure/logic-apps/logic-apps-using-sap-connector), you can trigger custom actions in SAP upon hiring a new employee.

-* Separation of duties: With separation of duties checks now available in preview in Azure AD [entitlement management](https://techcommunity.microsoft.com/t5/azure-active-directory-identity/ensure-compliance-using-separation-of-duties-checks-in-access/ba-p/2466939), customers can now ensure that users don't take on excessive access rights. Admins and access managers can prevent users from requesting additional access packages if theyΓÇÖre already assigned to other access packages or are a member of other groups that are incompatible with the requested access. Enterprises with critical regulatory requirements for SAP apps will have a single consistent view of access controls and enforce separation of duties checks across their financial and other business critical applications and Azure AD-integrated applications. With our [Pathlock](https://pathlock.com/), integration customers can leverage fine-grained separation of duties checks with access packages in Azure AD, and over time will help customers to address Sarbanes Oxley and other compliance requirements.

-### Update extension instance

-|`--plan-name` | **Plan ID** of the extension, found on the Marketplace page in the Azure portal under **Usage Information + Support**. |

-|`--plan-product` | **Product ID** of the extension, found on the Marketplace page in the Azure portal under **Usage Information + Support**. An example of this is the name of the ISV offering used. |

-|`--plan-publisher` | **Publisher ID** of the extension, found on the Marketplace page in the Azure portal under **Usage Information + Support**. |

-| [Authorize access using Google OAuth token](./use-google-as-oauth-token-provider.md) | Shows how to authorize access to your endpoints using Google as an OAuth token provider. |

-| [Get OAuth2 access token from AAD and forward it to the backend](./use-oauth2-for-authorization.md) | Provides and example of using OAuth2 for authorization between the gateway and a backend. It shows how to obtain an access token from AAD and forward it to the backend. |

-description: Azure API management policy sample - Demonstrates how to authorize access to your endpoints using Google as an OAuth token provider.

-# Authorize access using Google OAuth token

-This article shows an Azure API management policy sample that demonstrates how to authorize access to your endpoints using Google as an OAuth token provider. To set or edit a policy code, follow the steps described in [Set or edit a policy](../set-edit-policies.md). To see other examples, see [policy samples](../policy-reference.md).

-## Policy

-Paste the code into the **inbound** block.

-[!code-xml[Main](../../../api-management-policy-samples/examples/Simple Google OAuth validate-jwt.policy.xml)]

-## Next steps

-Learn more about APIM policies:

-+ [Transformation policies](../api-management-transformation-policies.md)

-+ [Policy samples](../policy-reference.md)

-Use [Export-PfxCertificate](/powershell/module/pki/export-pfxcertificate) with the Thumbprint that was returned to export a pfx file from the certificate. Make sure your password is 4 - 12 characters long:

-To configure TLS termination, a TLS/SSL certificate must be added to the listener. This allows the Application Gateway to decrypt incoming traffic and encrypt response traffic to the client. The certificate provided to the Application Gateway must be in Personal Information Exchange (PFX) format, which contains both the private and public keys.

-1. Set the scope for the assignment.

- * The scope will be all resource groups in a subscription or management group or specific resource groups.

-2. Set the parameters for the Flux v2 configuration that will be created.

-> There are built-in policy definitions for these scenarios:

-1. In the "Kubernetes" category, choose the "Configure Kubernetes clusters with specified GitOps configuration using no secrets" built-in policy definition.

-1. Verify **Create a managed identity** is checked, and that the identity will have **Contributor** permissions.

- * For more information, see the [Create a policy assignment quickstart](../../governance/policy/assign-policy-portal.md) and the [Remediate non-compliant resources with Azure Policy article](../../governance/policy/how-to/remediate-resources.md).

-[Set up Azure Monitor for Containers with Azure Arc-enabled Kubernetes clusters](../../azure-monitor/containers/container-insights-enable-arc-enabled-clusters.md).

- You can monitor the progress of the import operation by following the notifications from the Azure portal, or by viewing the events in the [audit log](../azure-monitor/essentials/activity-log.md).

- > Audit log support is not yet available in the Enterprise tiers.

-Yes, you can use the same storage account for persistence across two different caches.

-### Prerequisites and limitations

-| 4.x | 3.10 (Preview)<br/>3.9<br/> 3.8<br/>3.7 |

-npm install @microsoft/applicationinsights-react-js @microsoft/applicationinsights-web --save

- AzAcSnap is a centralized solution for backing up critical database files. It ensures database consistency before performing a storage volume snapshot. As a result, it ensures that the storage volume snapshot can be used for database recovery.

- When you use AzAcSnap with SAP HANA, the records within the backup catalog are kept current with storage snapshots. This capability allows a database administrator to see the backup activity.

- This capability is helpful for non-database volumes that don't need application quiescing before taking a storage snapshot. Examples include SAP HANA log-backup volumes or SAPTRANS volumes.

- This capability provides space-efficient storage volume clones for development and test purposes.

- AzAcSnap leverages storage volume replication to provide options for recovering replicated application-consistent snapshots at a remote site.

-| Standard network features | Generally available (GA) | No |

-| [Azure (Classic deployment model)](/powershell/module/servicemanagement/azure.service/add-azureaccount) commands |`Add-AzureAccount -Environment AzureUSGovernment` |

- You can create new volumes choosing *Standard* or *Basic* network features in supported regions. In regions where the Standard network features aren't supported, the volume defaults to using the Basic network features. For more information, see [Configure network features](configure-network-features.md).

-Azure NetApp Files Standard network features are supported for the following regions:

-| ExpressRoute (ER) FastPath | Yes | No |

-The **Network Features** functionality enables you to indicate whether you want to use VNet features for an Azure NetApp Files volume. With this functionality, you can set the option to ***Standard*** or ***Basic***. You can specify the setting when you create a new NFS, SMB, or dual-protocol volume. See [Guidelines for Azure NetApp Files network planning](azure-netapp-files-network-topologies.md) for details about network features.

-The **Network Features** functionality is not available in Azure Government regions. See [supported regions](azure-netapp-files-network-topologies.md#supported-regions) for a full list.

- You should set **Network Features** to *Basic* if you do not require VNet features.

-* Regardless of the Network Features option you set (*Standard* or *Basic*), an Azure VNet can only have one subnet delegated to Azure NetApp files. See [Delegate a subnet to Azure NetApp Files](azure-netapp-files-delegate-subnet.md#considerations).

-* Currently, you can specify the network features setting only during the creation process of a new volume. You cannot modify the setting on existing volumes.

-* You can create volumes with the Standard network features only if the corresponding [Azure region supports the Standard volume capability](azure-netapp-files-network-topologies.md#supported-regions).

- * If the Standard volume capability is not available for the region, the Network Features field of the Create a Volume page defaults to *Basic*, and you cannot modify the setting.

-* The ability to locate storage compatible with the desired type of network features depends on the VNet specified. If you cannot create a volume because of insufficient resources, you can try a different VNet for which compatible storage is available.

-* You can create Basic volumes from Basic volume snapshots and Standard volumes from Standard volume snapshots. Creating a Basic volume from a Standard volume snapshot is not supported. Creating a Standard volume from a Basic volume snapshot is not supported.

-* When restoring a backup to a new volume, the new volume can be configure with Basic or Standard network features.

-* Conversion between Basic and Standard network features in either direction is not currently supported.

-

-## Set the Network Features option

-This section shows you how to set the Network Features option.

-* The SDK for large volumes isn't currently available.

- Version 8 of the AzAcSnap tool is now generally available. [Azure Application Consistent Snapshot Tool](azacsnap-introduction.md) (AzAcSnap) is a command-line tool that enables customers to simplify data protection for third-party databases in Linux environments. AzAcSnap 8 introduces the following new capabilities and improvements:

- * New global settings file (.azacsnaprc) to control behavior of azacsnap

- * Backup (-c backup) and Details (-c details) fixes

-Restore files on VMs by using Windows Storage Spaces | Not supported on the same VM.<br/><br/> Instead, restore the files on a compatible VM.

-Configure standalone Azure VMs in Windows Storage Spaces | Supported.

-If you are creating a new VM for the image, use a first party Azure Marketplace image supported by Batch as the base image for your managed image. Only first party images can be used as a base image. To get a full list of Azure Marketplace image references supported by Azure Batch, see the [List node agent SKUs](/java/api/com.microsoft.azure.batch.protocol.accounts.listnodeagentskus) operation.

- --node-agent-sku-id "batch.node.ubuntu 16.04"

- nodeAgentSkuId: "batch.node.windows amd64");

- node_agent_sku_id="batch.node.ubuntu 18.04"

-## Create a pool from a Shared Image using the Azure portal

- 1. Select style samples as training data. It's recommended that the style samples are all from the same voice talent profile.

-# Quickstart: Get started using ChatGPT (preview) and GPT-4 (preview) with Azure OpenAI Service

-Each API requires input data to be formatted differently, which in turn impacts overall prompt design. The **Chat Completion API** supports the ChatGPT (preview) and GPT-4 (preview) models. These models are designed to take input formatted in a [specific chat-like transcript](../how-to/chatgpt.md) stored inside an array of dictionaries.

-The **Completion API** supports the older GPT-3 models and has much more flexible input requirements in that it takes a string of text with no specific format rules. Technically the ChatGPT (preview) models can be used with either APIs, but we strongly recommend using the Chat Completion API for these models. To learn more, please consult our [in-depth guide on using these APIs](../how-to/chatgpt.md).

-| [GPT-4](#gpt-4-models) | A set of models that improve on GPT-3.5 and can understand as well as generate natural language and code. **These models are currently in preview.**|

-| [GPT-3](#gpt-3-models) | A series of models that can understand and generate natural language. This includes the new [ChatGPT model (preview)](#chatgpt-gpt-35-turbo-preview). |

-## GPT-4 models (preview)

- These models are currently in preview. For access, existing Azure OpenAI customers can [apply by filling out this form](https://aka.ms/oai/get-gpt4).

-### ChatGPT (gpt-35-turbo) (preview)

-| gpt-35-turbo¹ (ChatGPT) (preview) | East US, France Central, South Central US, West Europe | N/A | 4,096 | Sep 2021 |

-| `gpt-4` ^1,² (preview) | East US, France Central, South Central US | N/A | 8,192 | September 2021 |

-| `gpt-4-32k` ^1,² (preview) | East US, France Central, South Central US | N/A | 32,768 | September 2021 |

-¹ The model is in preview and [only available by request](https://aka.ms/oai/get-gpt4).<br>

-description: Learn about the options for how to use the ChatGPT and GPT-4 models (preview)

-# Learn how to work with the ChatGPT and GPT-4 models (preview)

-The Chat Completion API is a new dedicated API for interacting with the ChatGPT and GPT-4 models. **Both sets of models are currently in preview**. This API is the preferred method for accessing these models. **It is also the only way to access the new GPT-4 models**.

-curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/embeddings?api-version=2022-12-01\

-openai.api_version = "2022-12-01"

-## Best Practices

-# How to Configure Azure OpenAI Service with Managed Identities

- curl ${endpoint%/}/openai/deployments/YOUR_DEPLOYMENT_NAME/completions?api-version=2022-12-01 \

-Let's say you'd like to ensure that the text of the ads on your website mention the correct product and company. In other words, you want to ensure the model isn't making things up. You may want to fine-tune a classifier which filters out incorrect ads.

-curl https://YOUR_RESOURCE_NAME.openaiazure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/completions?api-version=2022-12-01\ \

-curl https://YOUR_RESOURCE_NAME.openaiazure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/completions?api-version=2022-12-01\ \

-Azure OpenAI Service provides REST API access to OpenAI's powerful language models including the GPT-3, Codex and Embeddings model series. In addition, the new GPT-4 and ChatGPT (gpt-35-turbo) model series are now available in preview. These models can be easily adapted to your specific task including but not limited to content generation, summarization, semantic search, and natural language to code translation. Users can access the service through REST APIs, Python SDK, or our web-based interface in the Azure OpenAI Studio.

-| Models available | **NEW GPT-4 series (preview)** <br> GPT-3 base series <br>**NEW ChatGPT (gpt-35-turbo) (preview)**<br> Codex series <br> Embeddings series <br> Learn more in our [Models](./concepts/models.md) page.|

-| Fine-tuning | Ada <br> Babbage <br> Curie <br> Cushman* <br> Davinci* <br> \* Currently unavailable. \*\*East US and West Europe Fine-tuning is currently unavailable to new customers. Please use US South Central for US based training|

-GPT-4 models are the latest available models. These models are currently in preview. For access, existing Azure OpenAI customers can [apply by filling out this form](https://aka.ms/oai/get-gpt4).

-| Requests per minute per model* | Davinci-models (002 and later): 120 <br> ChatGPT model (preview): 300 <br> GPT-4 models (preview): 18 <br> All other models: 300 |

-POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/completions?api-version=2022-12-01

-curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/completions?api-version=2022-12-01\

-curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/embeddings?api-version=2022-12-01 \

-Create completions for chat messages with the ChatGPT (preview) and GPT-4 (preview) models. Chat completions are currently only available with `api-version=2023-03-15-preview`.

-curl https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2023-03-15-preview \

-openai.api_version = "2022-12-01"

-url = openai.api_base + "/openai/deployments?api-version=2022-12-01"

-With [Azure Logic Apps](../logic-apps/logic-apps-overview.md) and the built-in HTTP Webhook connector, you can create automated tasks and workflows that subscribe to a service endpoint, wait for specific events, and run based on those events, rather than regularly checking or *polling* that endpoint.

-* Wait for an item to arrive from an [Azure Event Hub](https://github.com/logicappsio/EventHubAPI) before triggering a logic app run.

-This article shows how to use the Webhook trigger and Webhook action so that your logic app can receive and respond to events at a service endpoint.

-A webhook trigger is event-based, which doesn't depend on checking or polling regularly for new items. When you save a logic app that starts with a webhook trigger, or when you change your logic app from disabled to enabled, the webhook trigger *subscribes* to the specified service endpoint by registering a *callback URL* with that endpoint. The trigger then waits for that service endpoint to call the URL, which starts running the logic app. Similar to the [Request trigger](connectors-native-reqres.md), the logic app fires immediately when the specified event happens. The webhook trigger *unsubscribes* from the service endpoint if you remove the trigger and save your logic app, or when you change your logic app from enabled to disabled.

-A webhook action is also event-based and *subscribes* to the specified service endpoint by registering a *callback URL* with that endpoint. The webhook action pauses the logic app's workflow and waits until the service endpoint calls the URL before the logic app resumes running. The webhook action *unsubscribes* from the service endpoint in these cases:

-* When the webhook action successfully finishes

-* If the logic app run is canceled while waiting for a response

-* Before the logic app times out

-1. Sign in to the [Azure portal](https://portal.azure.com). Open your blank logic app in Logic App Designer.

-In the workload profiles architecture, user-defined routes (UDRs) and securing outbound traffic with a firewall are supported. Learn more in the [networking concepts document](./networking.md#user-defined-routes-udrpreview).

-In the Consumption only architecture, custom user-defined routes (UDRs) and ExpressRoutes aren't supported.

-> The subnet associated with a Container App Environment on the Consumption only architecture requires a CIDR prefix of `/23` or larger. On the workload profiles architecture (preview), a `/27` or larger is required.

-The following service tags are required when using NSGs on the Consumption only architecture:

-The following service tags are required when using NSGs on the workload profiles architecture:

-The following IP rules are required when using NSGs on both the Consumption only architecture and the workload profiles architecture:

-# Networking architecture in Azure Container Apps

-Azure Container Apps run in the context of an [environment](environment.md), which is supported by a virtual network (VNet). By default, your Container App Environment is created with a VNet that is automatically generated for you. Generated VNets are inaccessible to you as they're created in Microsoft's tenant. This VNet is publicly accessible over the internet, can only reach internet accessible endpoints, and supports a limited subset of networking capabilities such as ingress IP restrictions and container app level ingress controls.

-The features available depend on your architecture selection.

-## Architecture Selection

-There are two architectures in Container Apps: the Consumption only architecture supports only the [Consumption plan (GA)](./plans.md) and the workload profiles architecture that supports both the [Consumption + Dedicated plan structure (preview)](./plans.md). The two architectures share many of the same networking characteristics. However, there are some key differences.

-| Architecture Type | Description |

-| Workload profiles architecture (preview) | Supports user defined routes (UDR) and egress through NAT Gateway. The minimum required subnet size is /27. <br /> <br /> As workload profiles are currently in preview, the number of supported regions is limited. To learn more, visit the [workload profiles overview](./workload-profiles-overview.md#supported-regions).|

-| Consumption only architecture | Doesn't support user defined routes (UDR) and egress through NAT Gateway. The minimum required subnet size is /23. |

-| Outbound public IP | Used as the "from" IP for outbound connections that leave the virtual network. These connections aren't routed down a VPN. Outbound IPs aren't guaranteed and may change over time. Using a NAT gateway or other proxy for outbound traffic from a Container App environment is only supported on the workload profile architecture. |

- - When you're using Consumption workload profiles for your container app, IP address assignment behaves the same as when running on the Consumption only architecture. As your app scales, a new IP address is allocated for each new replica.

-In addition, if you're using the Azure CLI with the Consumption only architecture and the [platformReservedCidr](vnet-custom-internal.md#networking-parameters) range is defined, both subnets must not overlap with the IP range defined in `platformReservedCidr`.

-In addition, Container Apps on the workload profiles architecture reserve the following addresses:

-User Defined Routes (UDR) and controlled egress through NAT Gateway are supported in the workload profiles architecture, which is in preview. In the Consumption only architecture, these features aren't supported.

-UDR is only supported on the workload profiles architecture. The following application and network rules must be added to the allowlist for your firewall depending on which resources you are using.

-You can use NAT Gateway to simplify outbound connectivity for your outbound internet traffic in your virtual network on the workload profiles architecture. NAT Gateway is used to provide a static public IP address, so when you configure NAT Gateway on your Container Apps subnet, all outbound traffic from your container app is routed through the NAT Gateway's static public IP address.

-With the workload profiles architecture (preview), you can fully secure your ingress/egress networking traffic. To do so, you should use the following features:

- 1. **Non-custom domains**: If you don't plan to use custom domains, create a private DNS zone that resolves the Container Apps environment's default domain to the static IP address of the Container Apps environment. You can use [Azure Private DNS](../dns/private-dns-overview.md) or your own DNS server. If you use Azure Private DNS, create a Private DNS Zone named as the Container App EnvironmentΓÇÖs default domain (`<UNIQUE_IDENTIFIER>.<REGION_NAME>.azurecontainerapps.io`), with an `A` record. The A record contains the name `*<DNS Suffix>` and the static IP address of the Container Apps environment.

-#### Consumption only architecture

-#### Workload profiles architecture

-| Cores | Environment | 40 | Yes | Maximum number of cores an environment can accommodate. Calculated by the sum of cores requested by each active replica of all revisions in an environment. |

-> [Free trial](https://azure.microsoft.com/offers/ms-azr-0044p) and [Azure for Students](https://azure.microsoft.com/free/students/) subscriptions are limited to one environment per subscription globally.

-> This feature is in preview and is only supported for the workload profiles architecture. User defined routes only work with an internal Azure Container Apps environment.

-For more information on networking concepts in Container Apps, see [Networking Architecture in Azure Container Apps](./networking.md).

-* **Internal environment**: An internal container app environment on the workload profiles architecture that's integrated with a custom virtual network. When you create an internal container app environment, your container app environment has no public IP addresses, and all traffic is routed through the virtual network. For more information, see the [guide for how to create a container app environment on the workload profiles architecture](./workload-profiles-manage-cli.md).

-> You can use an existing virtual network, but a dedicated subnet with a CIDR range of `/23` or larger is required for use with Container Apps when using the Consumption only Architecture. When using the Workload Profiles Architecture, a `/27` or larger is required. To learn more about subnet sizing, see the [networking architecture overview](./networking.md#subnet).

-> Network subnet address prefix requires a minimum CIDR range of `/23` for use with Container Apps when using the Consumption only Architecture. When using the Workload Profiles Architecture, a `/27` or larger is required. To learn more about subnet sizing, see the [networking architecture overview](./networking.md#subnet).

-| `platform-reserved-cidr` | The address range used internally for environment infrastructure services. Must have a size between `/23` and `/12` when using the [Consumption only architecture](./networking.md)|

-| `VnetConfigurationPlatformReservedCidr` | The address range used internally for environment infrastructure services. Must have a size between `/23` and `/12` when using the [Consumption only architecture](./networking.md) |

-For more information on networking concepts in Container Apps, see [Networking Architecture in Azure Container Apps](./networking.md).

-### DDoS Mitigation FlowLogs

-| **Category** | For notifications, this will be `DDoSProtectionNotifications`.|

-| **OperationName** | For notifications, this will be `DDoSProtectionNotifications`. |

-| **Message** | Details of the attack. |

-| **PublicIpAddress** | Your public IP address. |

-# Defender for Cloud support for commercial/government clouds

-In the support table, **NA** indicates that the feature is not available.

-**Feature/Plan** | **Azure** | **Azure Government** | **Azure China**<br/><br/>**21Vianet**

-**FOUNDATIONAL CSPM FEATURES** | | |

-[Continuous export](./continuous-export.md) | GA | GA | GA

-[Workflow automation](./workflow-automation.md) | GA | GA | GA

-[Recommendation exemption rules](./exempt-resource.md) | Public preview | NA | NA

-[Alert suppression rules](./alerts-suppression-rules.md) | GA | GA | GA

-[Alert email notifications](./configure-email-notifications.md) | GA | GA | GA

-[Agent/extension deployment](monitoring-components.md) | GA | GA | GA

-[Asset inventory](./asset-inventory.md) | GA | GA | GA

-[Azure Workbooks support](./custom-dashboards-azure-workbooks.md) | GA | GA | GA

-**DEFENDER FOR CLOUD PLANS** | | |

-**[Agentless discovery for Kubernetes](concept-agentless-containers.md)** | Public preview | NA | NA

-**[Agentless vulnerability assessments for container images.](concept-agentless-containers.md)**<br/><br/> Including registry scanning (up to 20 unique images per billable resources) | Public preview | NA | NA

-**[Defender CSPM](concept-cloud-security-posture-management.md)** | GA | NA | NA

-**[Defender for APIs](defender-for-apis-introduction.md)** | Public preview | NA | NA

-**[Defender for App Service](defender-for-app-service-introduction.md)** | GA | NA | NA

-**[Defender for Azure Cosmos DB](concept-defender-for-cosmos.md)** | Public preview | NA | NA

-**[Defender for Azure SQL database servers](defender-for-sql-introduction.md)**<br/><br/> Partial GA in Vianet21<br/> - A subset of alerts/vulnerability assessments is available.<br/>- Behavioral threat protection isn't available.| GA | GA | GA

-**[Defender for Containers](defender-for-containers-introduction.md)**<br/><br/>Support for Arc-enabled Kubernetes clusters (and therefore AWS EKS too) is in public preview and not available on Azure Government.<br/>Run-time visibility of vulnerabilities in container images is also a preview feature. | GA | GA | GA

-[Defender extension for Azure Arc-enabled Kubernetes clusters/servers/data services](defender-for-kubernetes-azure-arc.md). Requires Defender for Containers/Defender for Kubernetes. | Public preview | NA | NA

-**[Defender for DNS](defender-for-dns-introduction.md)** | GA | GA | GA

-**[Defender for Key Vault](./defender-for-key-vault-introduction.md)** | GA | NA | NA

-**[Defender for Kubernetes](./defender-for-kubernetes-introduction.md)**<br/><br/> Defender for Kubernetes is deprecated and replaced by Defender for Containers. Support for Azure Arc-enabled clusters is in public preview and not available in government clouds. [Learn more](defender-for-kubernetes-introduction.md). | GA | GA | GA

-**[Defender for open-source relational databases](defender-for-databases-introduction.md)** | GA | NA | NA

-**[Defender for Resource Manager](./defender-for-resource-manager-introduction.md)** | GA | GA | GA

-**DEFENDER FOR SERVERS FEATURES** | | |

-[Just-in-time VM access](./just-in-time-access-usage.md) | GA | GA | GA

-[File integrity monitoring](./file-integrity-monitoring-overview.md) | GA | GA | GA

-[Adaptive application controls](./adaptive-application-controls.md) | GA | GA | GA

-[Adaptive network hardening](./adaptive-network-hardening.md) | GA | GA | NA

-[Docker host hardening](./harden-docker-hosts.md) | GA | GA | GA

-[Integrated Qualys scanner](./deploy-vulnerability-assessment-vm.md) | GA | NA | NA

-[Compliance dashboard/reports](./regulatory-compliance-dashboard.md)<br/><br/> Compliance standards might differ depending on the cloud type.| GA | GA | GA

-[Defender for Endpoint integration](./integration-defender-for-endpoint.md) | GA | GA | NA

-[Connect AWS account](./quickstart-onboard-aws.md) | GA | NA | NA

-[Connect GCP project](./quickstart-onboard-gcp.md) | GA | NA | NA

-**[Defender for Storage](./defender-for-storage-introduction.md)**<br/><br/> Some threat protection alerts for Defender for Storage are in public preview. | GA | GA (activity monitoring) | NA

-**[Defender for SQL servers on machines](./defender-for-sql-introduction.md)** | GA | GA | NA

-**[Kubernetes workload protection](kubernetes-workload-protections.md)** | GA | GA | GA

-**[Microsoft Sentinel bi-directional alert synchronization](../sentinel/connect-azure-security-center.md)** | Public preview | NA | NA

-To learn more about the specific Defender for Cloud features available on Windows and Linux, see:

-description: Learn about the container and Kubernetes services that you can protect with Defender for Containers.

-# Defender for Containers feature availability

-These tables show the features that are available, by environment, for Microsoft Defender for Containers. For more information about Defender for Containers, see [Microsoft Defender for Containers](defender-for-containers-introduction.md).

-## Azure (AKS)

-| Domain | Feature | Supported Resources | Linux release state ^{[1](#footnote1)} | Windows release state ^{[1](#footnote1)} | Agentless/Agent-based | Pricing Tier | Azure clouds availability |

-|--|--|--|--|--|--|--|--|

-| Compliance | Docker CIS | VM, Virtual Machine Scale Set | GA | - | Log Analytics agent | Defender for Servers Plan 2 | Commercial clouds<br><br> National clouds: Azure Government, Azure China 21Vianet |

-| Vulnerability Assessment ^{[2](#footnote2)} | Registry scan - OS packages | ACR, Private ACR | GA | Preview | Agentless | Defender for Containers | Commercial clouds<br><br> National clouds: Azure Government, Azure China 21Vianet |

-| Vulnerability Assessment ^{[3](#footnote3)} | Registry scan - language specific packages | ACR, Private ACR | Preview | - | Agentless | Defender for Containers | Commercial clouds<br><br> National clouds: Azure Government, Azure China 21Vianet |

-| Vulnerability Assessment | View vulnerabilities for running images | AKS | GA | Preview | Defender profile | Defender for Containers | Commercial clouds |

-| Hardening | Control plane recommendations | ACR, AKS | GA | Preview | Agentless | Free | Commercial clouds<br><br> National clouds: Azure Government, Azure China 21Vianet |

-| Hardening | Kubernetes data plane recommendations | AKS | GA | - | Azure Policy | Free | Commercial clouds<br><br> National clouds: Azure Government, Azure China 21Vianet |

-| Runtime protection| Threat detection (control plane)| AKS | GA | GA | Agentless | Defender for Containers | Commercial clouds<br><br> National clouds: Azure Government, Azure China 21Vianet |

-| Runtime protection| Threat detection (workload) | AKS | GA | - | Defender profile | Defender for Containers | Commercial clouds |

-| Discovery and provisioning | Discovery of unprotected clusters | AKS | GA | GA | Agentless | Free | Commercial clouds<br><br> National clouds: Azure Government, Azure China 21Vianet |

-| Discovery and provisioning | Collection of control plane threat data | AKS | GA | GA | Agentless | Defender for Containers | Commercial clouds<br><br> National clouds: Azure Government, Azure China 21Vianet |

-| Discovery and provisioning | Auto provisioning of Defender profile | AKS | GA | - | Agentless | Defender for Containers | Commercial clouds<br><br> National clouds: Azure Government, Azure China 21Vianet |

-| Discovery and provisioning | Auto provisioning of Azure policy add-on | AKS | GA | - | Agentless | Free | Commercial clouds<br><br> National clouds: Azure Government, Azure China 21Vianet |

-^{<a name="footnote1"></a>1} Specific features are in preview. The [Azure Preview Supplemental Terms](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include other legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

-^{<a name="footnote2"></a>2} VA can detect vulnerabilities for these [OS packages](#registries-and-images).

-^{<a name="footnote3"></a>3} VA can detect vulnerabilities for these [language specific packages](#registries-and-images).

-### Additional environment information

-#### Registries and images

-#### Kubernetes distributions and configurations

-> For additional requirements for Kuberenetes workload protection, see [existing limitations](../governance/policy/concepts/policy-for-kubernetes.md#limitations).

-#### Network restrictions

-##### Private link

-| Domain | Feature | Supported Resources | Linux release state ^{[1](#footnote1)} | Windows release state ^{[1](#footnote1)} | Agentless/Agent-based | Pricing tier |

-^{<a name="footnote1"></a>1} Specific features are in preview. The [Azure Preview Supplemental Terms](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include other legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

-### Additional environment information

-#### Images

-#### Kubernetes distributions and configurations

-#### Network restrictions

-##### Private link

-##### Outbound proxy support

-| Domain | Feature | Supported Resources | Linux release state ^{[1](#footnote1)} | Windows release state ^{[1](#footnote1)} | Agentless/Agent-based | Pricing tier |

-^{<a name="footnote1"></a>1} Specific features are in preview. The [Azure Preview Supplemental Terms](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include other legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

-### Additional information

-#### Kubernetes distributions and configurations

-#### Network restrictions

-##### Private link

-##### Outbound proxy support

-| Domain | Feature | Supported Resources | Linux release state ^{[1](#footnote1)} | Windows release state ^{[1](#footnote1)} | Agentless/Agent-based | Pricing tier |

-| Vulnerability Assessment ^{[2](#footnote2)} | Registry scan - OS packages | ACR, Private ACR | GA | Preview | Agentless | Defender for Containers |

-| Vulnerability Assessment ^{[3](#footnote3)} | Registry scan - language specific packages | ACR, Private ACR | Preview | - | Agentless | Defender for Containers |

-| Runtime protection ^{[4](#footnote4)} | Threat detection (workload)| Arc enabled K8s clusters | Preview | - | Defender extension | Defender for Containers |

-^{<a name="footnote1"></a>1} Specific features are in preview. The [Azure Preview Supplemental Terms](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include other legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

-^{<a name="footnote2"></a>2} VA can detect vulnerabilities for these [OS packages](#registries-and-images-1).

-^{<a name="footnote3"></a>3} VA can detect vulnerabilities for these [language specific packages](#registries-and-images-1).

-^{<a name="footnote4"></a>4} Runtime protection can detect threats for these [Supported host operating systems](#supported-host-operating-systems).

-### Additional information

-#### Registries and images

-description: Learn about the environments where you can protect servers and virtual machines with Defender for Servers.

-# Support matrices for Defender for Servers

-This article provides information about the environments where you can protect servers and virtual machines with Defender for Servers and the endpoint protections that you can use to protect them.

-## Supported features for virtual machines and servers<a name="vm-server-features"></a>

-The following tables show the features that are supported for virtual machines and servers in Azure, Azure Arc, and other clouds.

-### Windows machines

-| **Feature** | **Azure Virtual Machines and [Virtual Machine Scale Sets with Flexible orchestration](../virtual-machine-scale-sets/virtual-machine-scale-sets-orchestration-modes.md#scale-sets-with-flexible-orchestration)** | **Azure Arc-enabled machines** | **Defender for Servers required** |

-### Linux machines

-| **Feature** | **Azure Virtual Machines and [Virtual Machine Scale Sets with Flexible orchestration](../virtual-machine-scale-sets/virtual-machine-scale-sets-orchestration-modes.md#scale-sets-with-flexible-orchestration)** | **Azure Arc-enabled machines** | **Defender for Servers required** |

-### Multicloud machines

-> [!TIP]

->To experiment with features that are only available with enhanced security features enabled, you can enroll in a 30-day trial. For more information, see the [pricing page](https://azure.microsoft.com/pricing/details/defender-for-cloud/).

-<a name="endpoint-supported"></a>

-## Supported endpoint protection solutions

-The following table provides a matrix of supported endpoint protection solutions and whether you can use Microsoft Defender for Cloud to install each solution for you.

-For information about when recommendations are generated for each of these solutions, see [Endpoint Protection Assessment and Recommendations](endpoint-protection-recommendations-technical.md).

-### Microsoft.Communication.ChatMemberAddedToThreadWithUser event

-```json

-[{

- "id": "4abd2b49-d1a9-4fcc-9cd7-170fa5d96443",

- "topic": "/subscriptions/{subscription-id}/resourceGroups/{group-name}/providers/Microsoft.Communication/communicationServices/{communication-services-resource-name}",

- "subject": "thread/{thread-id}/memberAdded/{rawId}/recipient/{rawId}",

- "data": {

- "time": "2020-09-18T00:47:13.1867087Z",

- "addedBy": "8:acs:5354158b-17b7-489c-9380-95d8821ff76b_00000005-3e5f-1bc6-f40f-343a0d0003f1",

- "memberAdded": {

- "displayName": "John Smith",

- "memberId": "8:acs:5354158b-17b7-489c-9380-95d8821ff76b_00000005-3e5f-1bc6-f40f-343a0d0003fe"

- },

- "createTime": "2020-09-18T00:46:41.559Z",

- "version": 1600390033176,

- "recipientId": "8:acs:5354158b-17b7-489c-9380-95d8821ff76b_00000005-3e5f-1bc6-f40f-343a0d0003f0",

- "transactionId": "pVIjw/pHEEKUOUJ2DAAl5A.1.1.1.1.1818361951.1.1",

- "threadId": "19:6d20c2f921cd402ead7d1b31b0d030cd@thread.v2"

- },

- "eventType": "Microsoft.Communication.ChatMemberAddedToThreadWithUser",

- "dataVersion": "1.0",

- "metadataVersion": "1",

- "eventTime": "2020-09-18T00:47:13.2342692Z"

-}]

-```

-### Microsoft.Communication.ChatMemberRemovedFromThreadWithUser event

-```json

-[{

- "id": "b3701976-1ea2-4d66-be68-4ec4fc1b4b96",

- "topic": "/subscriptions/{subscription-id}/resourceGroups/{group-name}/providers/Microsoft.Communication/communicationServices/{communication-services-resource-name}",

- "subject": "thread/{thread-id}/memberRemoved/{rawId}/recipient/{rawId}",

- "data": {

- "time": "2020-09-18T00:47:51.1461742Z",

- "removedBy": "8:acs:5354158b-17b7-489c-9380-95d8821ff76b_00000005-3e5f-1bc6-f40f-343a0d0003f1",

- "memberRemoved": {

- "displayName": "John",

- "memberId": "8:acs:5354158b-17b7-489c-9380-95d8821ff76b_00000005-3e5f-1bc6-f40f-343a0d0003fe"

- },

- "createTime": "2020-09-18T00:46:41.559Z",

- "version": 1600390071131,

- "recipientId": "8:acs:5354158b-17b7-489c-9380-95d8821ff76b_00000005-3e5f-1bc6-f40f-343a0d0003f0",

- "transactionId": "G9Y+UbjVmEuxAG3O4bEyvw.1.1.1.1.1819803816.1.1",

- "threadId": "19:6d20c2f921cd402ead7d1b31b0d030cd@thread.v2"

- },

- "eventType": "Microsoft.Communication.ChatMemberRemovedFromThreadWithUser",

- "dataVersion": "1.0",

- "metadataVersion": "1",

- "eventTime": "2020-09-18T00:47:51.2244511Z"

-}]

-```

-This metric displays a count of the total number of active flows on the ExpressRoute Gateway. Through split at instance level, you can see active flow count per gateway instance. For more information, see [understand network flow limits](../virtual-network/virtual-machine-network-throughput.md#network-flow-limits).

-# Planned maintenance for ExpressRoute

-|Filtering inter-hub traffic in secure virtual hub deployments|Secured Virtual Hub to Secured Virtual Hub communication filtering isn't yet supported. However, hub to hub communication still works if private traffic filtering via Azure Firewall isn't enabled.|Investigating|

-|Branch to branch traffic with private traffic filtering enabled|Branch to branch traffic isn't supported when private traffic filtering is enabled. |Investigating.<br><br>Don't secure private traffic if branch to branch connectivity is critical.|

-> [!NOTE]

-> This is the configuration deployed when securing connectivity from the Azure Portal with Azure Firewall Manager

-6. Select **Save**.

-7. Select **OK** on the **Warning** dialog.

-## Public preview features

-The following features are in public preview:

-| Feature | Description |

-| - | |

-| Routing Intent and Policies enabling Inter-hub security | This feature allows you to configure internet-bound, private or inter-hub traffic flow through Azure Firewall. For more information, see [Routing Intent and Policies](../virtual-wan/how-to-routing-policies.md). |

-### Policy Analytics (preview)

-Policy Analytics provides insights, centralized visibility, and control to Azure Firewall. IT teams today are challenged to keep Firewall rules up to date, manage existing rules, and remove unused rules. Any accidental rule updates can lead to a significant downtime for IT teams.

-description: Learn about Azure Firewall Policy Analytics (preview)

-# Azure Firewall Policy Analytics (preview)

-> [!IMPORTANT]

-> This feature is currently in PREVIEW.

-> See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

-Enabling Policy Analytics on a Firewall Policy associated with a single firewall is billed per policy as described on the [Azure Firewall Manager pricing](https://azure.microsoft.com/pricing/details/firewall-manager/) page. Enabling Policy Analytics on a Firewall Policy associated with more than one firewall is offered at no added cost.

-## Prerequisites

-### Firewall with no diagnostics settings configured

-1. Once all prerequisites are met, select **Policy analytics (preview)** in the table of contents.

-6. Go to the Firewall attached to the policy and enter the **Diagnostic settings** page. You'll see the **FirewallPolicySetting** added there as part of the policy analytics feature.

-7. Select **Edit Setting**, and ensure the **Resource specific** toggle is checked, and the highlighted tables are checked. In the previous example, all logs are written to the log analytics workspace.

-### Firewall with Diagnostics settings already configured

-1. Ensure that the Firewall attached to the policy is logging to **Resource Specific** tables, and that the following three tables are also selected:

- - AZFWApplicationRuleAggregation

- - AZFWNetworkRuleAggregation

- - AZFWNatRuleAggregation

-2. Next, select **Policy Analytics (preview)** in the table of contents. Once inside the feature, select **Configure Workspaces**.

-3. Now, select **Enable Policy Analytics**.

-4. Next, choose a log analytics workspace. The log analytics workspace should be the same as the Firewall attached to the policy.

-5. Select **Save** after you choose the log analytics workspace.

- During the save process, you might see the following error message: **Failed to update Diagnostic Settings**

- You can disregard this error message if the policy was successfully updated.

-> An *Origin* and a *origin group* in this article refers to the backend and backend pool of the Azure Front Door (classic) configuration.

-To determine the health and proximity of each backend for a given Azure Front Door environment, each Front Door environment periodically sends a synthetic HTTP/HTTPS request to each of your configured origins. Azure Front Door then uses these responses from the probe to determine the "best" origin to route your client requests.

-> Since each Azure Front Door edge POP emits health probes to your origins, the health probe volume for your origins can be quite high. The number of probes depends on your customer's traffic location and your health probe frequency. If the Azure Front Door edge POP doesnΓÇÖt receive real traffic from your end users, the frequency of the health probe from the edge POP is decreased from the configured frequency. If there is customer traffic to all the Azure Front Door edge POP, the health probe volume can be high depending on your health probes frequency.

-> An example to roughly estimate the health probe volume per minute to your origin when using the default probe frequency of 30 seconds. The probe volume on each of your origin is equal to the number of edge POPs times two requests per minute. The probing requests will be less if there is no traffic sent to all of the edge POPs. For a list of edge locations, see [edge locations by region](edge-locations-by-region.md) for Azure Front Door. There could be more than one POP in each edge location.

-> [!NOTE]

-> Azure Front Door HTTP/HTTPS probes are sent with `User-Agent` header set with value: `Edge Health Probe`.

-Azure Front Door supports sending probes over either HTTP or HTTPS protocols. These probes are sent over the same TCP ports configured for routing client requests, and cannot be overridden.

-1. **GET:** The GET method means retrieve whatever information (in the form of an entity) is identified by the Request-URI.

-2. **HEAD:** The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. For new Front Door profiles, by default, the probe method is set as HEAD.

-> [!NOTE]

-> For lower load and cost on your backends, Front Door recommends using HEAD requests for health probes.

-| Determining Health | A 200 OK status code indicates the backend is healthy. Everything else is considered a failure. If for any reason (including network failure) a valid HTTP response isn't received for a probe, the probe is counted as a failure.|

-| Measuring Latency | Latency is the wall-clock time measured from the moment immediately before we send the probe request to the moment when we receive the last byte of the response. We use a new TCP connection for each request, so this measurement isn't biased towards backends with existing warm connections. |

-## How Front Door determines backend health

-Azure Front Door uses the same three-step process below across all algorithms to determine health.

-1. Exclude disabled backends.

-1. Exclude backends that have health probes errors:

- * This selection is done by looking at the last _n_ health probe responses. If at least _x_ are healthy, the backend is considered healthy.

- * _n_ is configured by changing the SampleSize property in load-balancing settings.

- * _x_ is configured by changing the SuccessfulSamplesRequired property in load-balancing settings.

-1. For the sets of healthy backends in the backend pool, Front Door additionally measures and maintains the latency (round-trip time) for each backend.

-> If a single endpoint is a member of multiple backend pools, Azure Front Door optimizes the number of health probes sent to the backend to reduce the load on the backend. Health probe requests will be sent based on the lowest configured sample interval. The health of the endpoint in all pools will be determined by the responses from same health probes.

-If health probes fail for every backend in a backend pool, then Front Door considers all backends unhealthy and routes traffic in a round robin distribution across all of them.

-Once any backend returns to a healthy state, then Front Door will resume the normal load-balancing algorithm.

-If you have a single backend in your backend pool, you can choose to disable the health probes reducing the load on your application backend. Even if you have multiple backends in the backend pool but only one of them is in enabled state, you can disable health probes.

-description: Learn how to use HBase HBCK2 Tool

-# How to use Apache HBase HBCK2 Tool

-Learn how to use HBase HBCK2 Tool.

-## HBCK2 Overview

-HBCK2 is currently a simple tool that does one thing at a time only. In hbase-2.x, the Master is the final arbiter of all state, so a general principle for most HBCK2 commands is that it asks the Master to affect all repair. A Master must be up and running, before you can run HBCK2 commands. While HBCK1 performed analysis reporting your cluster GOOD or BAD, HBCK2 is less presumptuous. In hbase-2.x, the operator figures what needs fixing and then uses tooling including HBCK2 to do fixup.

-## HBCK2 vs HBCK1

-HBCK2 is the successor to HBCK, the repair tool that shipped with hbase-1.x (A.K.A HBCK1). Use HBCK2 in place of HBCK1 making repairs against hbase-2.x clusters. HBCK1 shouldn't be run against a hbase-2.x install. It may do damage. Its write-facility (-fix) has been removed. It can report on the state of a hbase-2.x cluster but its assessments are inaccurate since it doesn't understand the internal workings of a hbase-2.x. HBCK2 doesn't work the way HBCK1 used to, even for the case where commands are similarly named across the two versions.

-## Obtaining HBCK2

-You can find the release under the HBase distribution directory. See the [HBASE Downloads Page](https://dlcdn.apache.org/hbase/hbase-operator-tools-1.2.0/hbase-operator-tools-1.2.0-bin.tar.gz).

-An HBCK Report page added to the Master in 2.1.6 at `/hbck.jsp`, which shows output from two inspections run by the master on an interval. One is the output by the `CatalogJanitor` whenever it runs. If overlaps or holes in, `hbase:meta`, the `CatalogJanitor` lists what it has found. Another background 'chore' process added to compare `hbase:meta` and filesystem content; if any anomaly, it makes note in its HBCK Report section.

-To run the CatalogJanitor, execute the command in hbase shell: `catalogjanitor_run`

-To run hbck chore, execute the command in hbase shell: `hbck_chore_run`

-## Running HBCK2

-We can run the hbck command by launching it via the $HBASE_HOME/bin/hbase script. By default, running bin/hbase hbck, the built-in HBCK1 tooling is run. To run HBCK2, you need to point at a built HBCK2 jar using the -j option as in:

-This command with no options or arguments passed prints the HBCK2 help.

-## HBCK2 Commands

-> Test these commands on a test cluster to understand the functionality before running in production environment

-**assigns [OPTIONS] <ENCODED_REGIONNAME/INPUTFILES_FOR_REGIONNAMES>... | -i <INPUT_FILE>...**

-**Options:**

-`-o,--override` - override ownership by another procedure

-`-i,--inputFiles` - takes one or more encoded region names

-A 'raw' assign that can be used even during Master initialization (if the -skip flag is specified). Skirts Coprocessors. Pass one or more encoded region names. de00010733901a05f5a2a3a382e27dd4 is an example of what a user-space encoded region name looks like. For example:

-Returns the PID(s) of the created AssignProcedure(s) or -1 if none. If `-i or --inputFiles` is specified, pass one or more input file names. Each file contains encoded region names, one per line. For example:

-**unassigns [OPTIONS] <ENCODED_REGIONNAME>...| -i <INPUT_FILE>...**

-**Options:**

-`-o,--override` - override ownership by another procedure

-`-i,--inputFiles` - takes ones or more input files of encoded names

-A 'raw' unassign that can be used even during Master initialization (if the -skip flag is specified). Skirts Coprocessors. Pass one or more encoded region names. de00010733901a05f5a2a3a382e27dd4 is an example of what a user override space encoded region name looks like. For example:

-Returns the PID(s) of the created UnassignProcedure(s) or -1 if none. If `-i or --inputFiles` is specified, pass one or more input file names. Each file contains encoded region names, one per line. For example:

-**Options:**

-`-o,--override` - override if procedure is running/stuck

-`-r,--recursive` - bypass parent and its children. SLOW! EXPENSIVE!

-`-w,--lockWait` - milliseconds to wait before giving up; default=1

-`-i,--inputFiles` - takes one or more input files of PIDs

-Pass one (or more) procedure 'PIDs to skip to procedure finish. Parent of bypassed procedure skips to the finish. Entities are left in an inconsistent state and require manual fixup May need Master restart to clear locks still held. Bypass fails if procedure has children. Add 'recursive' if all you have is a parent PID to finish parent and children. *This is SLOW, and dangerous so use selectively. Doesn't always work*.

-**Options:**

-`i,--inputFiles` takes one or more input files of namespace or table names

-To be used when regions missing from `hbase:meta` but directories are present still in HDFS. This command is an only a check method, designed for reporting purposes and doesn't perform any fixes, providing a view of which regions (if any) would get readded to `hbase:meta`, grouped by respective table/namespace. To effectively readd regions in meta, run addFsRegionsMissingInMeta. This command needs `hbase:meta` to be online. For each namespace/table passed as parameter, it performs a diff between regions available in `hbase:meta` against existing regions dirs on HDFS. Region dirs with no matches are printed grouped under its related table name. Tables with no missing regions show a 'no missing regions' message. If no namespace or table is specified, it verifies all existing regions. It accepts a combination of multiple namespace and tables. Table names should include the namespace portion, even for tables in the default namespace, otherwise it assumes as a namespace value. An example triggering missing regions report for tables 'table_1' and 'table_2', under default namespace:

-An example triggering missing regions report for table 'table_1' under default namespace, and for all tables from namespace 'ns1':

-Returns list of missing regions for each table passed as parameter, or for each table on namespaces specified as parameter. If `-i or --inputFiles` is specified, pass one or more input file names. Each file contains `<NAMESPACE|NAMESPACE:TABLENAME>`, one per line. For example:

-**Options**

-`-i,--inputFiles` takes one or more input files of namespace of table names to be used when regions missing from `hbase:meta` but directories are present still in HDFS. **Needs `hbase:meta` to be online**. For each table name passed as parameter, performs diff between regions available in `hbase:meta` and region dirs on HDFS. Then for dirs with no `hbase:meta` matches, it reads the 'regioninfo' metadata file and re-creates given region in `hbase:meta`. Regions are re-created in 'CLOSED' state in the `hbase:meta` table, but not in the Masters' cache, and they aren't assigned either. To get these regions online, run the HBCK2 'assigns' command printed when this command-run completes.

-> [!NOTE]

-> If using hbase releases older than 2.3.0, a rolling restart of HMasters is needed prior to executing the set of 'assigns' output. An example adding missing regions for tables 'tbl_1' in the default namespace, 'tbl_2' in namespace 'n1' and for all tables from namespace 'n2':

-Returns HBCK2 an 'assigns' command with all reinserted regions. If `-i or --inputFiles` is specified, pass one or more input file names. Each file contains `<NAMESPACE|NAMESPACE:TABLENAME>`, one per line. For example:

-**Options**

-`-f, --fix`- fix meta by removing all extra regions found.

-`-i,--inputFiles`- take one or more input files of namespace or table names

-Reports regions present on `hbase:meta`, but with no related directories on the file system. Needs `hbase:meta` to be online. For each table name passed as parameter, performs diff between regions available in `hbase:meta` and region dirs on the given file system. Extra regions would get deleted from Meta if passed the --fix option.

-> Before deciding on use the "--fix" option, it's worth check if reported extra regions are overlapping with existing valid regions. If so, then `extraRegionsInMeta --fix` is indeed the optimal solution. Otherwise, "assigns" command is the simpler solution, as it recreates regions dirs in the filesystem, if not existing.

-An example triggering extra regions report for table 'table_1' under default namespace, and for all tables from namespace 'ns1':

-An example triggering extra regions report for table 'table_1' under default namespace, and for all tables from namespace 'ns1' with the fix option:

-Returns list of extra regions for each table passed as parameter, or for each table on namespaces specified as parameter. If `-i or --inputFiles` is specified, pass one or more input file names. Each file contains `<NAMESPACE|NAMESPACE:TABLENAME>`, one per line. For example:

-**fixMeta**

-> This doesn't work well with HBase 2.1.6. Not recommended to be used on a 2.1.6 HBase Cluster.

-Do a server-side fix of bad or inconsistent state in `hbase:meta`. Master UI has matching, new 'HBCK Report' tab that dumps reports generated by most recent run of catalogjanitor and a new 'HBCK Chore'. **It's critical that `hbase:meta` first be made healthy before making any other repairs**. Fixes 'holes', 'overlaps', etc., creating (empty) region directories in HDFS to match regions added to `hbase:meta`. **Command isn't the same as the old _hbck1_ command named similarly**. Works against the reports generated by the last catalog_janitor and hbck chore runs. If nothing to fix, run is a loop. Otherwise, if 'HBCK Report' UI reports problems, a run of fixMeta clears up`hbase:meta` issues.

-Trying to fix an orphan table by generating a missing table descriptor file. This command has no effect if the table folder is missing or if the `.tableinfo` is present (we don't override existing table descriptors). This command first checks if the TableDescriptor is cached in HBase Master in which case it recovers the `.tableinfo` accordingly. If TableDescriptor isn't cached in master, then it creates a default `.tableinfo` file with the following items:

-If the `.tableinfo` file was generated using default parameters then make sure you check the table / column family properties later (and change them if needed). This method doesn't change anything in HBase, only writes the new `.tableinfo` file to the file system. Orphan tables, for example, ServerCrashProcedures to stick, you might need to fix the error still after you generated the missing table info files.

-**Options**

-`-f, --fix` - fix any replication issues found.

-`-i,--inputFiles` - take one or more input files of table names

-Looks for undeleted replication queues and deletes them if passed the '--fix' option. Pass a table name to check for replication barrier and purge if '--fix'.

-If `-i or --inputFiles` is specified, pass one or more input file names. Each file contains `<TABLENAME>`, one per line. For example:

-**Options**

-`-i,--inputFiles` take one or more input files of encoded region names and states

-

-**Possible region states:**

-* CLOSIN

-* CLOSED

-* SPLITTING

-* FAILED_OPEN

-* FAILED_CLOSE

-* MERGED

-

-> This is a very risky option intended for use as last resort.

-Example scenarios include unassigns/assigns that can't move forward because region is in an inconsistent state in 'hbase:meta'. For example, the 'unassigns' command can only proceed if passed a region in one of the following states: **SPLITTING|SPLIT|MERGING|OPEN|CLOSING**.

- Before manually setting a region state with this command, certify that this region not handled by a running procedure, such as 'assign' or 'split'. You can get a view of running procedures in the hbase shell using the 'list_procedures' command. An example

-setting region 'de00010733901a05f5a2a3a382e27dd4' to CLOSING:

-Returns "0" if region state changed and "1" otherwise.

-If `-i or --inputFiles` is specified, pass one or more input file names.

-Each file contains `<ENCODED_REGIONNAME> <STATE>` one pair per line.

-For example,

-**Options**

-`-i,--inputFiles` take one or more input files of table names and states

-Possible table states: **ENABLED, DISABLED, DISABLING, ENABLING**.

-To read current table state, in the hbase shell run:

-

-A value of x08x00 == ENABLED, x08x01 == DISABLED, etc.

-Can also run a 'describe `<TABLENAME>` at the shell prompt. An example making table name user ENABLED:

-Returns whatever the previous table state was. If `-i or --inputFiles` is specified, pass one or more input file names. Each file contains `<TABLENAME> <STATE>`, one pair per line.

-For example:

-**Options**

-`-i,--inputFiles` take one or more input files of server names

-Schedule `ServerCrashProcedure(SCP)` for list of `RegionServers`. Format server name as `<HOSTNAME>,<PORT>,<STARTCODE>` (See HBase UI/logs).

-

-Example using RegionServer 'a.example.org, 29100,1540348649479'

-Returns the PID(s) of the created ServerCrashProcedure(s) or -1 if no procedure created (see master logs for why not).

-Command support added in hbase versions 2.0.3, 2.1.2, 2.2.0 or newer. If `-i or --inputFiles` is specified, pass one or more input file names. Each file contains `<SERVERNAME>`, one per line. For example:

-## Fixing Problems

-### Some General Principals

-When making repair, **make sure `hbase:meta` is consistent first before you go about fixing any other issue type** such as a filesystem deviance. Deviance in the filesystem or problems with assign should be addressed after the `hbase:meta` has been put in order. If `hbase:meta` has issues, the Master can't make proper placements when adopting orphan filesystem data or making region assignments.

-Other general principals to keep in mind include a Region can't be assigned if it's in CLOSING state (or the inverse, unassigned if in OPENING state) without first transitioning via CLOSED: Regions must always move from CLOSED, to OPENING, to OPEN, and then to CLOSING, CLOSED.

-When making repair, do fixup of a table-at-a-time.

-If a table is DISABLED, you cant' assign a Region. In the Master logs, you see that the Master reports skipped because the table is DISABLED. You can assign a Region because, currently in the OPENING state and you want it in the CLOSED state so it agrees with the table's DISABLED state. In this situation, you may have to temporarily set the table status to ENABLED, so you can do the assign, and then set it back again after the unassign statement. HBCK2 has facility to allow you to do this change. See the HBCK2 usage output.

-### Assigning/Unassigning

-

-Generally, on assign, the Master persists until successful. An assign takes an exclusive lock on the Region. This precludes a concurrent assign or unassign from running. An assign against a locked Region waits until the lock is released before making progress. See the [Procedures & Locks] section for current list of outstanding Locks.

-**Master startup cannot progress, in holding-pattern until region online**

-The Master is unable to continue startup because there's no Procedure to assign `hbase:meta` (or `hbase:namespace`). To inject one, use the HBCK2 tool:

-where **1588230740 is the encoded name of the `hbase:meta` Region**. Pass the '-skip' option to stop HBCK2 doing a version check against the remote master. If the remote master isn't up, the version check prompts a 'Master is initializing response', or 'PleaseHoldException' and drop the assign attempt. The '-skip' command avoid the version check and lands the scheduled assign.

-The same may happen to the `hbase:namespace` system table. Look for the encoded Region name of the `hbase:namespace` Region and do similar to what we did for `hbase:meta`. In this latter case, the Master actually prints a helpful message that looks like

-To schedule an assign for the hbase:namespace table noted in the above log line, you would do:

-passing the encoded name for the namespace region (the encoded name differs per deploy).

-### Missing Regions in `hbase:meta` region/table restore/rebuild

-There have been some unusual cases where table regions have been removed from `hbase:meta` table. Some triage on such cases revealed these were operator-induced. Users would have run the obsolete hbck1 OfflineMetaRepair tool against an HBCK2 cluster. OfflineMetaRepair is a well known tool for fixing `hbase:meta` table related issues on HBase 1.x versions. The original version isn't compatible with HBase 2.x or higher versions, and it has undergone some adjustments so in the extreme, it can now be run via HBCK2.

-In most of these cases, regions end up missing in `hbase:meta` at random, but hbase may still be operational. In such situations, problem can be addressed with the Master online, using the addFsRegionsMissingInMeta command in HBCK2. This command is less disruptive to hbase than a full `hbase:meta` rebuild covered later, and it can be used even for recovering the namespace table region.

-### Extra Regions in `hbase:meta` region/table restore/rebuild

-There can also be situations where table regions have been removed in file system, but still have related entries on `hbase:meta` table. This may happen due to problems on splitting, manual operation mistakes (like deleting/moving the region dir manually), or even meta info data loss issues such as HBASE-21843.

-Such problem can be addressed with the Master online, using the **extraRegionsInMeta --fix** command in HBCK2. This command is less disruptive to hbase than a full `hbase:meta` rebuild covered later. Also useful when this happens on versions that don't support fixMeta hbck2 option (any prior to "2.0.6", "2.1.6", "2.2.1", "2.3.0","3.0.0").

-### Online `hbase:meta` rebuild recipe

-If `hbase:meta` corruption isn't too critical, hbase would still be able to bring it online. Even if namespace region is among the missing regions, it's possible to scan `hbase:meta` during the initialization period, where Master is waiting for namespace to be assigned. To verify this situation, a` hbase:meta` scan command can be executed. If it doesn't time out or shows any errors, the `hbase:meta` is online:

-HBCK2 **addFsRegionsMissingInMeta** can be used if the message doesn't show any errors. It reads region metadata info available on the FS region directories in order to recreate regions in `hbase:meta`. Since it can run with hbase partially operational, it attempts to disable online tables that are affected the reported problem and it's going to readd regions to `hbase:meta`. It can check for specific tables/namespaces, or all tables from all namespaces. An example shows adding missing regions for tables 'tbl_1' in the default namespace, 'tbl_2' in namespace 'n1', and for all tables from namespace 'n2':

-As it operates independently from Master, once it finishes successfully, more steps are required to actually have the readded regions assigned. These messages are listed as

-**addFsRegionsMissingInMeta** outputs an assigns command with all regions that got readded. This command needs to be executed later, so copy and save it for convenience.

-**For HBase versions prior to 2.3.0, after addFsRegionsMissingInMeta finished successfully and output has been saved, restart all running HBase Masters.**

-Once Master's are restarted and `hbase:meta` is already online (check if Web UI is accessible), run assigns command from addFsRegionsMissingInMeta output saved earlier.

-> If namespace region is among the missing regions, you will need to add --skip flag at the beginning of assigns command returned.

-Should a cluster suffer a catastrophic loss of the `hbase:meta` table, a rough rebuild is possible using the following recipe. In outline, we stop the cluster. Run the HBCK2 OfflineMetaRepair tool, which reads directories and metadata dropped into the filesystem makes the best effort at reconstructing a viable `hbase:met` table; restart your cluster. Inject an assign to bring the system namespace table online; and then finally, reassign user space tables you'd like enabled (the rebuilt `hbase:meta` creates a table with all tables offline and no regions assigned).

-> Use it only as a last resort. Not recommended.

-* Run the rebuild `hbase:meta` command from HBCK2. This moves aside the original `hbase:meta` and puts in place a newly rebuilt one. As an example of how to run the tool. It adds the -details flag so the tool dumps info on the regions its found in hdfs:

-* Start up the cluster. It won't up fully. It's stuck because the namespace table isn't online and there's no assign procedure in the procedure store for this contingency. The hbase master log shows this state. Here's an example of what it logs:

- To assign the namespace table region, you can't use the shell. If you use the shell, it fails with a PleaseHoldException because the master isn't yet up (it's waiting for the namespace table to come online before it declares itself ΓÇÿupΓÇÖ). You have to use the HBCK2 assigns command. To assign, you need the namespace encoded name. It shows in the log quoted. That is, 725a0fe6c2c869d3d0a9ed82bfa80fa3 in this case. You have to pass the -skip command to ΓÇÿskipΓÇÖ the master version check (without it, your HBCK2 invocation elicits the PleaseHoldException because the master isn't yet up). Here's an example adding an assign of the namespace table:

- If the invocation comes back with ΓÇÿConnection refusedΓÇÖ, is the Master up? The Master will shut down after a while if it canΓÇÖt initialize itself. Just restart the cluster/master and rerun the assigns command.

-* When the assigns run successfully, you see it emit the likes of the following. The ‘48’ on the end is the PID of the assign procedure schedule. If the PID returned is ‘-1’, then the master startup hasn't progressed sufficently… retry. Or, the encoded regionname is incorrect. Check.

-* Check the master logs. The master should have come up. You see successful completion of PID=48. Look for a line like this to verify successful master launch:

- The rebuild of `hbase:meta` adds the user tables in DISABLED state and the regions in CLOSED mode. Re-enable tables via the shell to bring all table regions back online. Do it one-at-a-time or see the enable all ".*" command to enable all tables in one shot.

- The rebuild meta is missing edits and may need subsequent repair and cleaning using facility outlined higher up in this TSG.

-HBCK2 can check for hanging references and corrupt files. You can ask it to sideline bad files, which may be needed to get over humps where regions won't online or reads are failing. See the filesystem command in the HBCK2 listing. Pass one or more tablename (or 'none' to check all tables). It reports bad files. Pass the `--fix` option to effect repairs.

-### Procedure Start-over

-At an extreme, as a last resource, if the Master is distraught and all attempts at fixup only turn up undoable locks or Procedures that can't finish, and/or the set of MasterProcWALs is growing without bound. It's possible to wipe the Master state clean. Just move aside the `/hbase/MasterProcWALs/` directory under your HBase install and restart the Master process. It comes back as a tabular format without memory.

-If at the time of the erasure, all Regions were happily assigned or off lined, then on Master restart, the Master should pick up and continue as though nothing happened. But if there were Regions-In-Transition at the time, then the operator has to intervene to bring outstanding assigns/unassigns to their terminal point. Read the `hbase:meta` `info:state` columns as described to figure what needs assigning/unassigning. Having erased all history moving aside the MasterProcWALs, none of the entities should be locked so you 'Improved free to bulk assign/unassign.

-### Retrieve metadata cache validation for (study, series, or instance)

-### Retrieve Rendered Image (For Instance or Frame)

-### Retrieve metadata cache validation for (study, series, or instance)

-### Retrieve Rendered Image (For Instance or Frame)

-In this article, you learned about the MedTech service frequently asked questions (FAQs)

-description: Use Azure RTOS embedded software to connect an NXP MIMXRT1050-EVKB device to Azure IoT and send telemetry.

-# Quickstart: Connect an NXP MIMXRT1050-EVKB Evaluation kit to IoT Central

-**Applies to**: [Embedded device development](about-iot-develop.md#embedded-device-development)<br>

-**Total completion time**: 30 minutes

-[](https://github.com/azure-rtos/getting-started/tree/master/NXP/MIMXRT1050-EVKB/)

-In this quickstart, you use Azure RTOS to connect an NXP MIMXRT1050-EVKB Evaluation kit (from now on, NXP EVK) to Azure IoT.

-You'll complete the following tasks:

-* Install a set of embedded development tools for programming an NXP EVK in C

-* Build an image and flash it onto the NXP EVK

-* Use Azure IoT Central to create cloud components, view properties, view device telemetry, and call direct commands

-## Prerequisites

-* A PC running Windows 10

-* [Git](https://git-scm.com/downloads) for cloning the repository

-* Hardware

- * The [NXP MIMXRT1050-EVKB](https://www.nxp.com/design/development-boards/i-mx-evaluation-and-development-boards/i-mx-rt1050-evaluation-kit:MIMXRT1050-EVK) (NXP EVK)

- * USB 2.0 A male to Micro USB male cable

- * Wired Ethernet access

- * Ethernet cable

-## Prepare the development environment

-To set up your development environment, first you clone a GitHub repo that contains all the assets you need for the quickstart. Then you install a set of programming tools.

-### Clone the repo for the quickstart

-Clone the following repo to download all sample device code, setup scripts, and offline versions of the documentation. If you previously cloned this repo in another quickstart, you don't need to do it again.

-To clone the repo, run the following command:

-```shell

-git clone --recursive https://github.com/azure-rtos/getting-started.git

-```

-### Install the tools

-The cloned repo contains a setup script that installs and configures the required tools. If you installed these tools in another embedded device quickstart, you don't need to do it again.

-> [!NOTE]

-> The setup script installs the following tools:

-> * [CMake](https://cmake.org): Build

-> * [ARM GCC](https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm): Compile

-> * [Termite](https://www.compuphase.com/software_termite.htm): Monitor serial port output for connected devices

-To install the tools:

-1. From File Explorer, navigate to the following path in the repo and run the setup script named *get-toolchain.bat*:

- *getting-started\tools\get-toolchain.bat*

-1. After the installation, open a new console window to recognize the configuration changes made by the setup script. Use this console to complete the remaining programming tasks in the quickstart. You can use Windows CMD, PowerShell, or Git Bash for Windows.

-1. Run the following code to confirm that CMake version 3.14 or later is installed.

- ```shell

- cmake --version

- ```

-## Prepare the device

-To connect the NXP EVK to Azure, you'll modify a configuration file for Wi-Fi and Azure IoT settings, rebuild the image, and flash the image to the device.

-### Add configuration

-1. Open the following file in a text editor:

- *getting-started\NXP\MIMXRT1050-EVKB\app\azure_config.h*

-1. Set the Azure IoT device information constants to the values that you saved after you created Azure resources.

- |Constant name|Value|

- |-|--|

- |`IOT_DPS_ID_SCOPE` |{*Your ID scope value*}|

- |`IOT_DPS_REGISTRATION_ID` |{*Your Device ID value*}|

- |`IOT_DEVICE_SAS_KEY` |{*Your Primary key value*}|

-1. Save and close the file.

-### Build the image

-1. In your console or in File Explorer, run the script *rebuild.bat* at the following path to build the image:

- *getting-started\NXP\MIMXRT1050-EVKB\tools\rebuild.bat*

-2. After the build completes, confirm that the binary file was created in the following path:

- *getting-started\NXP\MIMXRT1050-EVKB\build\app\mimxrt1050_azure_iot.bin*

-### Flash the image

-1. On the NXP EVK, locate the **Reset** button, the Micro USB port, and the Ethernet port. You use these components in the following steps. All three are highlighted in the following picture:

- :::image type="content" source="media/quickstart-devkit-nxp-mimxrt1050-evkb/nxp-1050-evkb-board.png" alt-text="Locate key components on the NXP EVK board":::

-1. Connect the Micro USB cable to the Micro USB port on the NXP EVK, and then connect it to your computer. After the device powers up, a solid green LED shows the power status.

-1. Use the Ethernet cable to connect the NXP EVK to an Ethernet port.

-1. In File Explorer, find the binary file that you created in the previous section.

-1. Copy the binary file *mimxrt1050_azure_iot.bin*

-1. In File Explorer, find the NXP EVK device connected to your computer. The device appears as a drive on your system with the drive label **RT1050-EVK**.

-1. Paste the binary file into the root folder of the NXP EVK. Flashing starts automatically and completes in a few seconds.

- > [!NOTE]

- > During the flashing process, a red LED blinks rapidly on the NXP EVK.

-### Confirm device connection details

-You can use the **Termite** app to monitor communication and confirm that your device is set up correctly.

-1. Start **Termite**.

- > [!TIP]

- > If you have issues getting your device to initialize or connect after flashing, see [Troubleshooting](troubleshoot-embedded-device-quickstarts.md).

-1. Select **Settings**.

-1. In the **Serial port settings** dialog, check the following settings and update if needed:

- * **Baud rate**: 115,200

- * **Port**: The port that your NXP EVK is connected to. If there are multiple port options in the dropdown, you can find the correct port to use. Open Windows **Device Manager**, and view **Ports** to identify which port to use.

- :::image type="content" source="media/quickstart-devkit-nxp-mimxrt1050-evkb/termite-settings.png" alt-text="Screenshot of serial port settings in the Termite app":::

-1. Select OK.

-1. Press the **Reset** button on the device. The button is labeled on the device and located near the Micro USB connector.

-1. In the **Termite** app, check the following checkpoint values to confirm that the device is initialized and connected to Azure IoT.

- ```output

- Starting Azure thread

- Initializing DHCP

- IP address: 10.0.0.77

- Mask: 255.255.255.0

- Gateway: 10.0.0.1

- SUCCESS: DHCP initialized

- Initializing DNS client

- DNS address: 10.0.0.1

- SUCCESS: DNS client initialized

- Initializing SNTP client

- SNTP server 0.pool.ntp.org

- SNTP IP address: 142.147.92.5

- SNTP time update: May 28, 2021 17:36:33.325 UTC

- SUCCESS: SNTP initialized

- Initializing Azure IoT DPS client

- DPS endpoint: global.azure-devices-provisioning.net

- DPS ID scope: ***

- Registration ID: mydevice

- SUCCESS: Azure IoT DPS client initialized

- Initializing Azure IoT Hub client

- Hub hostname: ***.azure-devices.net

- Device id: mydevice

- Model id: dtmi:azurertos:devkit:gsg;1

- Connected to IoT Hub

- SUCCESS: Azure IoT Hub client initialized

- ```

-Keep Termite open to monitor device output in the following steps.

-## Verify the device status

-To view the device status in IoT Central portal:

-1. From the application dashboard, select **Devices** on the side navigation menu.

-1. Confirm that the **Device status** is updated to **Provisioned**.

-1. Confirm that the **Device template** is updated to **Getting Started Guide**.

- :::image type="content" source="media/quickstart-devkit-nxp-mimxrt1050-evkb/iot-central-device-view-status.png" alt-text="Screenshot of device status in IoT Central":::

-## View telemetry

-With IoT Central, you can view the flow of telemetry from your device to the cloud.

-To view telemetry in IoT Central portal:

-1. From the application dashboard, select **Devices** on the side navigation menu.

-1. Select the device from the device list.

-1. View the telemetry as the device sends messages to the cloud in the **Overview** tab.

-1. The temperature is measured from the MCU wafer.

- :::image type="content" source="media/quickstart-devkit-nxp-mimxrt1050-evkb/iot-central-device-telemetry.png" alt-text="Screenshot of device telemetry in IoT Central":::

- > [!NOTE]

- > You can also monitor telemetry from the device by using the Termite app.

-## Call a direct method on the device

-You can also use IoT Central to call a direct method that you've implemented on your device. Direct methods have a name, and can optionally have a JSON payload, configurable connection, and method timeout. In this section, you call a method that enables you to turn an LED on or off.

-To call a method in IoT Central portal:

-1. Select the **Command** tab from the device page.

-1. In the **State** dropdown, select **True**, and then select **Run**. There will be no change on the device as there isn't an available LED to toggle. You can view the output in Termite to monitor the status of the methods.

- :::image type="content" source="media/quickstart-devkit-nxp-mimxrt1050-evkb/iot-central-invoke-method.png" alt-text="Screenshot of calling a direct method on a device in IoT Central":::

-1. In the **State** dropdown, select **False**, and then select **Run**.

-## View device information

-You can view the device information from IoT Central.

-Select **About** tab from the device page.

-> [!TIP]

-> To customize these views, edit the [device template](../iot-central/core/howto-edit-device-template.md).

-## Troubleshoot and debug

-If you experience issues building the device code, flashing the device, or connecting, see [Troubleshooting](troubleshoot-embedded-device-quickstarts.md).

-For debugging the application, see [Debugging with Visual Studio Code](https://github.com/azure-rtos/getting-started/blob/master/docs/debugging.md).

-## Clean up resources

-If you no longer need the Azure resources created in this quickstart, you can delete them from the IoT Central portal.

-To remove the entire Azure IoT Central sample application and all its devices and resources:

-1. Select **Administration** > **Your application**.

-1. Select **Delete**.

-## Next steps

-In this quickstart, you built a custom image that contains Azure RTOS sample code, and then flashed the image to the NXP EVK device. You also used the IoT Central portal to create Azure resources, connect the NXP EVK securely to Azure, view telemetry, and send messages.

-As a next step, explore the following articles to learn more about using the IoT device SDKs to connect devices to Azure IoT.

-> [!div class="nextstepaction"]

-> [Connect a simulated device to IoT Central](quickstart-send-telemetry-central.md)

-> [!div class="nextstepaction"]

-> [Connect a simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md)

-> [!IMPORTANT]

-> Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements.

-description: Use Azure RTOS embedded software to connect a Renesas RX65N-2MB device to Azure IoT and send telemetry.

-# Quickstart: Connect a Renesas Starter Kit+ for RX65N-2MB to IoT Central

-**Applies to**: [Embedded device development](about-iot-develop.md#embedded-device-development)<br>

-**Total completion time**: 30 minutes

-[](https://github.com/azure-rtos/getting-started/tree/master/Renesas/RSK_RX65N_2MB)

-In this quickstart, you use Azure RTOS to connect the Renesas Starter Kit+ for RX65N-2MB (hereafter, the Renesas RX65N) to Azure IoT.

-You will complete the following tasks:

-* Install a set of embedded development tools for programming a Renesas RX65N in C

-* Build an image and flash it onto the Renesas RX65N

-* Use Azure IoT Central to create cloud components, view properties, view device telemetry, and call direct commands

-## Prerequisites

-* A PC running Windows 10

-* [Git](https://git-scm.com/downloads) for cloning the repository

-* Hardware

- * The [Renesas Starter Kit+ for RX65N-2MB](https://www.renesas.com/products/microcontrollers-microprocessors/rx-32-bit-performance-efficiency-mcus/rx65n-2mb-starter-kit-plus-renesas-starter-kit-rx65n-2mb) (Renesas RX65N)

- * The [Renesas E2 emulator Lite](https://www.renesas.com/software-tool/e2-emulator-lite-rte0t0002lkce00000r)

- * 2 USB 2.0 A male to Mini USB male cables

- * The included 5V power supply

- * Ethernet cable

- * Wired Ethernet access

-* An active Azure subscription. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.

-## Prepare the development environment

-To set up your development environment, first you clone a GitHub repo that contains all the assets you need for the quickstart. Then you install a set of programming tools.

-### Clone the repo for the quickstart

-Clone the following repo to download all sample device code, setup scripts, and offline versions of the documentation. If you previously cloned this repo in another quickstart, you don't need to do it again.

-To clone the repo, run the following command:

-```shell

-git clone --recursive https://github.com/azure-rtos/getting-started.git

-```

-### Install the tools

-The cloned repo contains a setup script that installs and configures the required tools. If you installed these tools in another embedded device quickstart, you don't need to do it again.

-> [!NOTE]

-> The setup script installs the following tools:

-> * [CMake](https://cmake.org): Build

-> * [RX GCC](http://gcc-renesas.com/downloads/get.php?f=rx/8.3.0.202004-gnurx/gcc-8.3.0.202004-GNURX-ELF.exe): Compile

-> * [Termite](https://www.compuphase.com/software_termite.htm): Monitor serial port output for connected devices

-To install the tools:

-1. From File Explorer, navigate to the following path in the repo and run the setup script named *get-toolchain.bat*:

- *getting-started\tools\get-toolchain-rx.bat*

-1. Add the RX compiler to the Windows Path:

- *%USERPROFILE%\AppData\Roaming\GCC for Renesas RX 8.3.0.202004-GNURX-ELF\rx-elf\rx-elf\bin*

-1. After the installation, open a new console window to recognize the configuration changes made by the setup script. Use this console to complete the remaining programming tasks in the quickstart. You can use Windows CMD, PowerShell, or Git Bash for Windows.

-1. Run the following commands to confirm that CMake version 3.14 or later is installed and the RX compiler path is set up correctly.

- ```shell

- cmake --version

- rx-elf-gcc

- ```

-To install the remaining tools:

-* Install [Renesas Flash Programmer](https://www.renesas.com/software-tool/renesas-flash-programmer-programming-gui). The Renesas Flash Programmer contains the drivers and tools needed to flash the Renesas RX65N via the Renesas E2 Lite.

-## Prepare the device

-To connect the Renesas RX65N to Azure, you'll modify a configuration file for Wi-Fi and Azure IoT settings, rebuild the image, and flash the image to the device.

-### Add configuration

-1. Open the following file in a text editor:

- *getting-started\Renesas\RSK_RX65N_2MB\app\azure_config.h*

-1. Set the Azure IoT device information constants to the values that you saved after you created Azure resources.

- |Constant name|Value|

- |-|--|

- |`IOT_DPS_ID_SCOPE` |{*Your ID scope value*}|

- |`IOT_DPS_REGISTRATION_ID` |{*Your Device ID value*}|

- |`IOT_DEVICE_SAS_KEY` |{*Your Primary key value*}|

-1. Save and close the file.

-### Build the image

-1. In your console or in File Explorer, run the script *rebuild.bat* at the following path to build the image:

- *getting-started\Renesas\RSK_RX65N_2MB\tools\rebuild.bat*

-2. After the build completes, confirm that the binary file was created in the following path:

- *getting-started\Renesas\RSK_RX65N_2MB\build\app\rx65n_azure_iot.hex*

-### Connect the device

-> [!NOTE]

-> For more information about setting up and getting started with the Renesas RX65N, see [Renesas Starter Kit+ for RX65N-2MB Quick Start](https://www.renesas.com/document/man/e2studio-renesas-starter-kit-rx65n-2mb-quick-start-guide).

-1. Complete the following steps using the following image as a reference.

-

- :::image type="content" source="media/quickstart-devkit-renesas-rx65n-2mb/renesas-rx65n.jpg" alt-text="Locate reset, power, ethernet, USB, and E1/E2Lite on the Renesas RX65N board":::

-1. Using the 5V power supply, connect the **Power Input** on the Renesas RX65N to an electrical outlet.

-1. Using the Ethernet cable, connect the **Ethernet** on the Renesas RX65N to your router.

-1. Using the first Mini USB cable, connect the **USB Serial** on the Renesas RX65N to your computer.

-1. Using the second Mini USB cable, connect the **E2 Lite USB Serial** on the Renesas E2 Lite to your computer.

-1. Using the supplied ribbon cable, connect the **E1/E2Lite** on the Renesas RX65N to the Renesas E2 Lite.

-### Flash the image

-1. Launch the *Renesas Flash Programmer* application from the Start menu.

-2. Select *New Project...* from the *File* menu, and enter the following settings:

- * **Microcontroller**: RX65x

- * **Project Name**: RX65N

- * **Tool**: E2 emulator Lite

- :::image type="content" source="media/quickstart-devkit-renesas-rx65n-2mb/rfp-new.png" alt-text="Screenshot of Renesas Flash Programmer, New Project":::

-3. Select the *Tool Details* button, and navigate to the *Reset Settings* tab.

-4. Select *Reset Pin as Hi-Z* and press the *OK* button.

- :::image type="content" source="media/quickstart-devkit-renesas-rx65n-2mb/rfp-reset.png" alt-text="Screenshot of Renesas Flash Programmer, Reset Settings":::

-5. Press the *Connect* button and when prompted, check the *Auto Authentication* checkbox and then press *OK*.

- :::image type="content" source="media/quickstart-devkit-renesas-rx65n-2mb/rfp-auth.png" alt-text="Screenshot of Renesas Flash Programmer, Authentication":::

-6. Select the *Browse...* button and locate the *rx65n_azure_iot.hex* file created in the previous section.

-7. Press *Start* to begin flashing. This process will take approximately 10 seconds.

-### Confirm device connection details

-You can use the **Termite** app to monitor communication and confirm that your device is set up correctly.

-> [!TIP]

-> If you have issues getting your device to initialize or connect after flashing, see [Troubleshooting](troubleshoot-embedded-device-quickstarts.md).

-1. Start **Termite**.

-1. Select **Settings**.

-1. In the **Serial port settings** dialog, check the following settings and update if needed:

- * **Baud rate**: 115,200

- * **Port**: The port that your Renesas RX65N is connected to. If there are multiple port options in the dropdown, you can find the correct port to use. Open Windows **Device Manager**, and view **Ports** to identify which port to use.

- :::image type="content" source="media/quickstart-devkit-renesas-rx65n-2mb/termite-settings.png" alt-text="Screenshot of serial port settings in the Termite app":::

-1. Select OK.

-1. Press the **Reset** button on the device.

-1. In the **Termite** app, check the following checkpoint values to confirm that the device is initialized and connected to Azure IoT.

- ```output

- Starting Azure thread

- Initializing DHCP

- IP address: 10.0.0.81

- Mask: 255.255.255.0

- Gateway: 10.0.0.1

- SUCCESS: DHCP initialized

- Initializing DNS client

- DNS address: 10.0.0.1

- SUCCESS: DNS client initialized

- Initializing SNTP client

- SNTP server 0.pool.ntp.org

- SNTP IP address: 104.194.242.237

- SNTP time update: May 28, 2021 22:53:27.54 UTC

- SUCCESS: SNTP initialized

- Initializing Azure IoT DPS client

- DPS endpoint: global.azure-devices-provisioning.net

- DPS ID scope: ***

- Registration ID: mydevice

- SUCCESS: Azure IoT DPS client initialized

- Initializing Azure IoT Hub client

- Hub hostname: ***.azure-devices.net

- Device id: mydevice

- Model id: dtmi:azurertos:devkit:gsg;1

- Connected to IoT Hub

- SUCCESS: Azure IoT Hub client initialized

- ```

-Keep Termite open to monitor device output in the following steps.

-## Verify the device status

-To view the device status in IoT Central portal:

-1. From the application dashboard, select **Devices** on the side navigation menu.

-1. Confirm that the **Device status** is updated to **Provisioned**.

-1. Confirm that the **Device template** is updated to **Getting Started Guide**.

- :::image type="content" source="media/quickstart-devkit-renesas-rx65n-2mb/iot-central-device-view-status.png" alt-text="Screenshot of device status in IoT Central":::

-## View telemetry

-With IoT Central, you can view the flow of telemetry from your device to the cloud.

-To view telemetry in IoT Central portal:

-1. From the application dashboard, select **Devices** on the side navigation menu.

-1. Select the device from the device list.

-1. View the telemetry as the device sends messages to the cloud in the **Overview** tab.

- :::image type="content" source="media/quickstart-devkit-renesas-rx65n-2mb/iot-central-device-telemetry.png" alt-text="Screenshot of device telemetry in IoT Central":::

- > [!NOTE]

- > You can also monitor telemetry from the device by using the Termite app.

-## Call a direct method on the device

-You can also use IoT Central to call a direct method that you've implemented on your device. Direct methods have a name, and can optionally have a JSON payload, configurable connection, and method timeout. In this section, you call a method that enables you to turn an LED on or off.

-To call a method in IoT Central portal:

-1. Select the **Command** tab from the device page.

-1. In the **State** dropdown, select **True**, and then select **Run**. The LED light should turn on.

- :::image type="content" source="media/quickstart-devkit-renesas-rx65n-2mb/iot-central-invoke-method.png" alt-text="Screenshot of calling a direct method on a device in IoT Central":::

-1. In the **State** dropdown, select **False**, and then select **Run**. The LED light should turn off.

-## View device information

-You can view the device information from IoT Central.

-Select **About** tab from the device page.

-> [!TIP]

-> To customize these views, edit the [device template](../iot-central/core/howto-edit-device-template.md).

-## Troubleshoot

-If you experience issues building the device code, flashing the device, or connecting, see [Troubleshooting](troubleshoot-embedded-device-quickstarts.md).

-## Clean up resources

-If you no longer need the Azure resources created in this quickstart, you can delete them from the IoT Central portal.

-To remove the entire Azure IoT Central sample application and all its devices and resources:

-1. Select **Administration** > **Your application**.

-1. Select **Delete**.

-## Next steps

-In this quickstart, you built a custom image that contains Azure RTOS sample code, and then flashed the image to the Renesas RX65N device. You also used the IoT Central portal to create Azure resources, connect the Renesas RX65N securely to Azure, view telemetry, and send messages.

-As a next step, explore the following articles to learn more about using the IoT device SDKs to connect devices to Azure IoT.

-> [!div class="nextstepaction"]

-> [Connect a simulated device to IoT Central](quickstart-send-telemetry-central.md)

-> [!div class="nextstepaction"]

-> [Connect a simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md)

-> [!IMPORTANT]

-> Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements.

-| Built-in | - Azure Automation <br>- Azure Blob Storage <br>- Azure Event Hubs <br>- Azure Service Bus <br>- Azure Queues <br>- Azure Tables <br>- HTTP <br>- HTTP + Webhook <br>- SQL Server <br><br>**Note**: HTTP operations can authenticate connections to Azure Storage accounts behind Azure firewalls with the system-assigned identity. |

-| [Managed identity](#managed-identity-authentication) | **Built-in connectors**: <br><br>- **Consumption**: Azure API Management, Azure App Services, Azure Functions, HTTP, HTTP Webhook <br><br>- **Standard**: Azure Automation, Azure Blob Storage, Azure Event Hubs, Azure Queues, Azure Service Bus, Azure Tables, HTTP, HTTP Webhook, SQL Server <br><br>**Managed connectors**: Azure AD Identity Protection, Azure App Service, Azure Automation, Azure Blob Storage, Azure Container Instance, Azure Cosmos DB, Azure Data Explorer, Azure Data Factory, Azure Data Lake, Azure Event Grid, Azure Event Hubs, Azure IoT Central V2, Azure IoT Central V3, Azure Key Vault, Azure Log Analytics, Azure Queues, Azure Resource Manager, Azure Service Bus, Azure Sentinel, Azure Table Storage, Azure VM, HTTP with Azure AD, SQL Server |

- * The **Logic App (Standard)** resource type supports having the [system-assigned managed identity *and* multiple user-assigned managed identities](create-managed-service-identity.md) enabled at the same time, though you still can only select one identity to use at any time.

- * User-assigned managed identity. Currently, only the system-assigned managed identity is available and automatically enabled.

-To perform real-time inferencing, you must deploy a pipeline as an [online endpoint](concept-endpoints.md#what-are-online-endpoints). The online endpoint creates an interface between an external application and your scoring model. A call to an online endpoint returns prediction results to the application in real time. To make a call to an online endpoint, you pass the API key that was created when you deployed the endpoint. The endpoint is based on REST, a popular architecture choice for web programming projects.

-# What are Azure Machine Learning endpoints?

-Use Azure Machine Learning endpoints to streamline model deployments for both real-time and batch inference deployments. Endpoints provide a unified interface to invoke and manage model deployments across compute types.

-> [!IMPORTANT]

-> Items marked (preview) in this article are currently in public preview.

-> The preview version is provided without a service level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities.

-> For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).

-In this article, you learn about:

-> [!div class="checklist"]

-> * Endpoints

-> * Deployments

-> * Managed online endpoints

-> * Kubernetes online endpoints

-> * Batch inference endpoints

-## What are endpoints and deployments?

-After you train a machine learning model, you need to deploy the model so that others can use it to do inferencing. In Azure Machine Learning, you can use **endpoints** and **deployments** to do so.

-An **endpoint**, in this context, is an HTTPS path that provides an interface for clients to send requests (input data) and receive the inferencing (scoring) output of a trained model. An endpoint provides:

-A **deployment** is a set of resources required for hosting the model that does the actual inferencing.

-A single endpoint can contain multiple deployments. Endpoints and deployments are independent Azure Resource Manager resources that appear in the Azure portal.

-Azure Machine Learning allows you to implement both [online endpoints](#what-are-online-endpoints) and [batch endpoints](#what-are-batch-endpoints).

-### Multiple developer interfaces

-Create and manage batch and online endpoints with multiple developer tools:

-## What are online endpoints?

-**Online endpoints** are endpoints that are used for online (real-time) inferencing. Compared to **batch endpoints**, **online endpoints** contain **deployments** that are ready to receive data from clients and can send responses back in real time.

-The following diagram shows an online endpoint that has two deployments, 'blue' and 'green'. The blue deployment uses VMs with a CPU SKU, and runs version 1 of a model. The green deployment uses VMs with a GPU SKU, and uses version 2 of the model. The endpoint is configured to route 90% of incoming traffic to the blue deployment, while green receives the remaining 10%.

-### Online deployments requirements

-To create an online endpoint, you need to specify the following elements:

-Learn how to deploy online endpoints from the [CLI/SDK](how-to-deploy-online-endpoints.md) and the [studio web portal](how-to-use-managed-online-endpoint-studio.md).

-### Test and deploy locally for faster debugging

-Deploy locally to test your endpoints without deploying to the cloud. Azure Machine Learning creates a local Docker image that mimics the Azure Machine Learning image. Azure Machine Learning will build and run deployments for you locally, and cache the image for rapid iterations.

-### Native blue/green deployment

-Recall, that a single endpoint can have multiple deployments. The online endpoint can do load balancing to give any percentage of traffic to each deployment.

-Traffic allocation can be used to do safe rollout blue/green deployments by balancing requests between different instances.

-> [!TIP]

-> A request can bypass the configured traffic load balancing by including an HTTP header of `azureml-model-deployment`. Set the header value to the name of the deployment you want the request to route to.

-Traffic to one deployment can also be mirrored (or copied) to another deployment. Mirroring traffic (also called shadowing) is useful when you want to test for things like response latency or error conditions without impacting live clients; for example, when implementing a blue/green deployment where 100% of the traffic is routed to blue and 10% is mirrored to the green deployment. With mirroring, the results of the traffic to the green deployment aren't returned to the clients but metrics and logs are collected. Testing the new deployment with traffic mirroring/shadowing is also known as [shadow testing](https://microsoft.github.io/code-with-engineering-playbook/automated-testing/shadow-testing/), and the functionality is currently a __preview__ feature.

-Learn how to [safely rollout to online endpoints](how-to-safely-rollout-online-endpoints.md).

-### Application Insights integration

-All online endpoints integrate with Application Insights to monitor SLAs and diagnose issues.

-However [managed online endpoints](#managed-online-endpoints-vs-kubernetes-online-endpoints) also include out-of-box integration with Azure Logs and Azure Metrics.

-### Security

-### Autoscaling

-Autoscale automatically runs the right amount of resources to handle the load on your application. Managed endpoints support autoscaling through integration with the [Azure monitor autoscale](../azure-monitor/autoscale/autoscale-overview.md) feature. You can configure metrics-based scaling (for instance, CPU utilization >70%), schedule-based scaling (for example, scaling rules for peak business hours), or a combination.

-### Visual Studio Code debugging

-Visual Studio Code enables you to interactively debug endpoints.

-### Private endpoint support

-Optionally, you can secure communication with a managed online endpoint by using private endpoints.

-You can configure security for inbound scoring requests and outbound communications with the workspace and other services separately. Inbound communications use the private endpoint of the Azure Machine Learning workspace. Outbound communications use private endpoints created per deployment.

-For more information, see [Secure online endpoints](how-to-secure-online-endpoint.md).

-## Managed online endpoints vs Kubernetes online endpoints

-There are two types of online endpoints: **managed online endpoints** and **Kubernetes online endpoints**.

-Managed online endpoints help to deploy your ML models in a turnkey manner. Managed online endpoints work with powerful CPU and GPU machines in Azure in a scalable, fully managed way. Managed online endpoints take care of serving, scaling, securing, and monitoring your models, freeing you from the overhead of setting up and managing the underlying infrastructure. The main example in this doc uses managed online endpoints for deployment.

-Kubernetes online endpoint allows you to deploy models and serve online endpoints at your fully configured and managed [Kubernetes cluster anywhere](./how-to-attach-kubernetes-anywhere.md),with CPUs or GPUs.

-The following table highlights the key differences between managed online endpoints and Kubernetes online endpoints.

-| | Managed online endpoints | Kubernetes online endpoints |

-| -- | | -- |

-| **Recommended users** | Users who want a managed model deployment and enhanced MLOps experience | Users who prefer Kubernetes and can self-manage infrastructure requirements |

-| **Node provisioning** | Managed compute provisioning, update, removal | User responsibility |

-| **Node maintenance** | Managed host OS image updates, and security hardening | User responsibility |

-| **Cluster sizing (scaling)** | [Managed manual and autoscale](how-to-autoscale-endpoints.md), supporting additional nodes provisioning | [Manual and autoscale](how-to-kubernetes-inference-routing-azureml-fe.md#autoscaling), supporting scaling the number of replicas within fixed cluster boundaries |

-| **Compute type** | Managed by the service | Customer-managed Kubernetes cluster (Kubernetes) |

-| **Managed identity** | [Supported](how-to-access-resources-from-endpoints-managed-identities.md) | Supported |

-| **Virtual Network (VNET)** | [Supported via managed network isolation](how-to-secure-online-endpoint.md) | User responsibility |

-| **Out-of-box monitoring & logging** | [Azure Monitor and Log Analytics powered](how-to-monitor-online-endpoints.md) (includes key metrics and log tables for endpoints and deployments) | User responsibility |

-| **Logging with Application Insights (legacy)** | Supported | Supported |

-| **View costs** | [Detailed to endpoint / deployment level](how-to-view-online-endpoints-costs.md) | Cluster level |

-| **Cost applied to** | VMs assigned to the deployments | VMs assigned to the cluster |

-| **Mirrored traffic** | [Supported](how-to-safely-rollout-online-endpoints.md#test-the-deployment-with-mirrored-traffic-preview) (preview) | Unsupported |

-| **No-code deployment** | Supported ([MLflow](how-to-deploy-mlflow-models-online-endpoints.md) and [Triton](how-to-deploy-with-triton.md) models) | Supported ([MLflow](how-to-deploy-mlflow-models-online-endpoints.md) and [Triton](how-to-deploy-with-triton.md) models) |

-### Managed online endpoints

-Managed online endpoints can help streamline your deployment process. Managed online endpoints provide the following benefits over Kubernetes online endpoints:

- - Automatically provisions the compute and hosts the model (you just need to specify the VM type and scale settings)

- - Automatically updates and patches the underlying host OS image

- - Automatic node recovery if there's a system failure

- - Monitor model availability, performance, and SLA using [native integration with Azure Monitor](how-to-monitor-online-endpoints.md).

- - Debug deployments using the logs and native integration with Azure Log Analytics.

- :::image type="content" source="media/concept-endpoints/log-analytics-and-azure-monitor.png" alt-text="Screenshot showing Azure Monitor graph of endpoint latency.":::

- - Managed online endpoints let you [monitor cost at the endpoint and deployment level](how-to-view-online-endpoints-costs.md)

-

- :::image type="content" source="media/concept-endpoints/endpoint-deployment-costs.png" alt-text="Screenshot cost chart of an endpoint and deployment.":::

- > [!NOTE]

- > Managed online endpoints are based on Azure Machine Learning compute. When using a managed online endpoint, you pay for the compute and networking charges. There is no additional surcharge.

- >

- > If you use a virtual network and secure outbound (egress) traffic from the managed online endpoint, there is an additional cost. For egress, three private endpoints are created _per deployment_ for the managed online endpoint. These are used to communicate with the default storage account, Azure Container Registry, and workspace. Additional networking charges may apply. For more information on pricing, see the [Azure pricing calculator](https://azure.microsoft.com/pricing/calculator/).

-For a step-by-step tutorial, see [How to deploy online endpoints](how-to-deploy-online-endpoints.md).

-## What are batch endpoints?

-**Batch endpoints** are endpoints that are used to do batch inferencing on large volumes of data over a period of time. **Batch endpoints** receive pointers to data and run jobs asynchronously to process the data in parallel on compute clusters. Batch endpoints store outputs to a data store for further analysis.

-### Batch deployment requirements

-To create a batch deployment, you need to specify the following elements:

-If you're deploying [MLFlow models in batch deployments](batch-inference/how-to-mlflow-batch.md), there's no need to provide a scoring script and execution environment, as both are autogenerated.

-Learn more about how to [deploy and use batch endpoints](batch-inference/how-to-use-batch-endpoint.md).

-### Managed cost with autoscaling compute

-Invoking a batch endpoint triggers an asynchronous batch inference job. Compute resources are automatically provisioned when the job starts, and automatically de-allocated as the job completes. So you only pay for compute when you use it.

-You can [override compute resource settings](batch-inference/how-to-use-batch-endpoint.md#overwrite-deployment-configuration-per-each-job) (like instance count) and advanced settings (like mini batch size, error threshold, and so on) for each individual batch inference job to speed up execution and reduce cost.

-### Flexible data sources and storage

-You can use the following options for input data when invoking a batch endpoint:

-> [!NOTE]

-> - If you're using existing V1 FileDatasets for batch endpoints, we recommend migrating them to V2 data assets. You can then refer to the V2 data assets directly when invoking batch endpoints. Currently, only data assets of type `uri_folder` or `uri_file` are supported. Batch endpoints created with GA CLIv2 (2.4.0 and newer) or GA REST API (2022-05-01 and newer) will not support V1 Datasets.

-> - You can also extract the datastores' URI or path from V1 FileDatasets. For this, you'll use the `az ml dataset show` command with the `--query` parameter and use that information for invoke.

-> - While batch endpoints created with earlier APIs will continue to support V1 FileDatasets, we'll be adding more support for V2 data assets in the latest API versions for better usability and flexibility. For more information on V2 data assets, see [Work with data using SDK v2](how-to-read-write-data-v2.md). For more information on the new V2 experience, see [What is v2](concept-v2.md).

-For more information on supported input options, see [Accessing data from batch endpoints jobs](batch-inference/how-to-access-data-batch-endpoints-jobs.md).

-Specify the storage output location to any datastore and path. By default, batch endpoints store their output to the workspace's default blob store, organized by the Job Name (a system-generated GUID).

-### Security

-> [!NOTE]

-> Creating batch endpoints in a private-link enabled workspace is only supported in the following versions.

-> - CLI - version 2.15.1 or higher.

-> - REST API - version 2022-05-01 or higher.

-> - SDK V2 - version 0.1.0b3 or higher.

-Batch endpoints can be used to perform batch scoring on large amounts of data. Such data can be placed in different places. In this tutorial we'll cover the different places where batch endpoints can read data from and how to reference it.

-* This example assumes that you've a model correctly deployed as a batch endpoint. Particularly, we're using the *heart condition classifier* created in the tutorial [Using MLflow models in batch deployments](how-to-mlflow-batch.md).

-## Supported data inputs

-## Input data from a data asset

- Use the Azure Machine Learning CLI, Azure Machine Learning SDK for Python, or Studio to get the location (region), workspace, and data asset name and version. You will need them later.

- input = Input(type=AssetTypes.URI_FOLDER, path=heart_dataset_unlabeled.id)

- "mnistinput": {

- > Data assets ID would look like `/subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.MachineLearningServices/workspaces/<workspace>/data/<data-asset>/versions/<version>`.

- ```bash

- INVOKE_RESPONSE = $(az ml batch-endpoint invoke --name $ENDPOINT_NAME --input $DATASET_ID)

- > [!TIP]

- > You can also use `--input azureml:/<dataasset_name>@latest` as a way to indicate the input.

-## Input data from data stores

-1. We'll need to upload some sample data to it. This example assumes you've uploaded the sample data included in the repo in the folder `sdk/python/endpoints/batch/heart-classifier/data` in the folder `heart-classifier/data` in the blob storage account. Ensure you have done that before moving forward.

- data_path = "heart-classifier/data"

- "mnistinput": {

- ```bash

- INVOKE_RESPONSE = $(az ml batch-endpoint invoke --name $ENDPOINT_NAME --input $INPUT_PATH)

-## Input data from Azure Storage Accounts

- "mnistinput": {

- "mnistinput": {

- "JobInputType" : "UriFolder",

- If your data is a folder, use `--input-type uri_folder`:

-

- ```bash

- INVOKE_RESPONSE = $(az ml batch-endpoint invoke --name $ENDPOINT_NAME --input-type uri_folder --input $INPUT_DATA)

- If your data is a file, use `--input-type uri_file`:

- ```bash

- INVOKE_RESPONSE = $(az ml batch-endpoint invoke --name $ENDPOINT_NAME --input-type uri_file --input $INPUT_DATA)

-## Security considerations when reading data

-Batch endpoints ensure that only authorized users are able to invoke batch deployments and generate jobs. However, depending on how the input data is configured, other credentials may be used to read the underlying data. Use the following table to understand which credentials are used and any additional requirements.

-Batch endpoints allow you to deploy models to perform long-running inference at scale. To indicate how batch endpoints should use your model over the input data to create predictions, you need to create and specify a scoring script (also known as batch driver script). In this article, you will learn how to use scoring scripts in different scenarios and their best practices.

-__mnist/environment/conda.yml__

-description: Learn how to deploy models using batch endpoints with REST APIs.

-# Deploy models with REST for batch scoring

-Learn how to use the Azure Machine Learning REST API to deploy models for batch scoring.

-The REST API uses standard HTTP verbs to create, retrieve, update, and delete resources. The REST API works with any language or tool that can make HTTP requests. REST's straightforward structure makes it a good choice in scripting environments and for MLOps automation.

-In this article, you learn how to use the new REST APIs to:

-> [!div class="checklist"]

-> * Create machine learning assets

-> * Create a batch endpoint and a batch deployment

-> * Invoke a batch endpoint to start a batch scoring job

-## Prerequisites

-> [!IMPORTANT]

-> The code snippets in this article assume that you are using the Bash shell.

->

-> The code snippets are pulled from the `/cli/batch-score-rest.sh` file in the [Azure Machine Learning Example repository](https://github.com/Azure/azureml-examples).

-## Set endpoint name

-> [!NOTE]

-> Batch endpoint names need to be unique at the Azure region level. For example, there can be only one batch endpoint with the name mybatchendpoint in westus2.

-## Azure Machine Learning batch endpoints

-[Batch endpoints](concept-endpoints.md#what-are-batch-endpoints) simplify the process of hosting your models for batch scoring, so you can focus on machine learning, not infrastructure. In this article, you'll create a batch endpoint and deployment, and invoking it to start a batch scoring job. But first you'll have to register the assets needed for deployment, including model, code, and environment.

-There are many ways to create an Azure Machine Learning batch endpoint, including the Azure CLI, Azure Machine Learning SDK for Python, and visually with the studio. The following example creates a batch endpoint and a batch deployment with the REST API.

-## Create machine learning assets

-First, set up your Azure Machine Learning assets to configure your job.

-In the following REST API calls, we use `SUBSCRIPTION_ID`, `RESOURCE_GROUP`, `LOCATION`, and `WORKSPACE` as placeholders. Replace the placeholders with your own values.

-Administrative REST requests a [service principal authentication token](how-to-manage-rest.md#retrieve-a-service-principal-authentication-token). Replace `TOKEN` with your own value. You can retrieve this token with the following command:

-The service provider uses the `api-version` argument to ensure compatibility. The `api-version` argument varies from service to service. Set the API version as a variable to accommodate future versions:

-### Create compute

-Batch scoring runs only on cloud computing resources, not locally. The cloud computing resource is a reusable virtual computer cluster where you can run batch scoring workflows.

-Create a compute cluster:

-> [!TIP]

-> If you want to use an existing compute instead, you must specify the full Azure Resource Manager ID when [creating the batch deployment](#create-batch-deployment). The full ID uses the format `/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/computes/<your-compute-name>`.

-### Get storage account details

-To register the model and code, first they need to be uploaded to a storage account. The details of the storage account are available in the data store. In this example, you get the default datastore and Azure Storage account for your workspace. Query your workspace with a GET request to get a JSON file with the information.

-You can use the tool [jq](https://stedolan.github.io/jq/) to parse the JSON result and get the required values. You can also use the Azure portal to find the same information:

-### Upload & register code

-Now that you have the datastore, you can upload the scoring script. For more information about how to author the scoring script, see [Understanding the scoring script](batch-inference/how-to-batch-scoring-script.md#understanding-the-scoring-script). Use the Azure Storage CLI to upload a blob into your default container:

-> [!TIP]

-> You can also use other methods to upload, such as the Azure portal or [Azure Storage Explorer](https://azure.microsoft.com/features/storage-explorer/).

-Once you upload your code, you can specify your code with a PUT request:

-### Upload and register model

-Similar to the code, upload the model files:

-Now, register the model:

-### Create environment

-The deployment needs to run in an environment that has the required dependencies. Create the environment with a PUT request. Use a docker image from Microsoft Container Registry. You can configure the docker image with `image` and add conda dependencies with `condaFile`.

-Run the following code to read the `condaFile` defined in json. The source file is at `/cli/endpoints/batch/mnist/environment/conda.json` in the example repository:

-Now, run the following snippet to create an environment:

-## Deploy with batch endpoints

-Next, create a batch endpoint, a batch deployment, and set the default deployment for the endpoint.

-### Create batch endpoint

-Create the batch endpoint:

-### Create batch deployment

-Create a batch deployment under the endpoint:

-### Set the default batch deployment under the endpoint

-There's only one default batch deployment under one endpoint, which will be used when invoke to run batch scoring job.

-## Run batch scoring

-Invoking a batch endpoint triggers a batch scoring job. A job `id` is returned in the response, and can be used to track the batch scoring progress. In the following snippets, `jq` is used to get the job `id`.

-### Invoke the batch endpoint to start a batch scoring job

-#### Getting the Scoring URI and access token

-Get the scoring uri and access token to invoke the batch endpoint. First get the scoring uri:

-Get the batch endpoint access token:

-#### Invoke the batch endpoint with different input options

-It's time to invoke the batch endpoint to start a batch scoring job. If your data is a folder (potentially with multiple files) publicly available from the web, you can use the following snippet:

-```rest-api

-response=$(curl --location --request POST $SCORING_URI \

- \"properties\": {

- \"InputData\": {

- \"mnistinput\": {

- \"JobInputType\" : \"UriFolder\",

- \"Uri\": \"https://pipelinedata.blob.core.windows.net/sampledata/mnist\"

- }

- }

- }

-}")

-JOB_ID=$(echo $response | jq -r '.id')

-JOB_ID_SUFFIX=$(echo ${JOB_ID##/*/})

-```

-Now, let's look at other options for invoking the batch endpoint. When it comes to input data, there are multiple scenarios you can choose from, depending on the input type (whether you are specifying a folder or a single file), and the URI type (whether you are using a path on Azure Machine Learning registered datastore, a reference to Azure Machine Learning registered V2 data asset, or a public URI).

-> [!NOTE]

-> For more information about data URI, see [Azure Machine Learning data reference URI](reference-yaml-core-syntax.md#azure-machine-learning-data-reference-uri).

-Below are some examples using different types of input data.

- - Use the short form to represent the URI:

- ```rest-api

- response=$(curl --location --request POST $SCORING_URI \

- --header "Authorization: Bearer $SCORING_TOKEN" \

- --header "Content-Type: application/json" \

- --data-raw "{

- \"properties\": {

- \"InputData\": {

- \"mnistInput\": {

- \"JobInputType\" : \"UriFolder\",

- \"Uri": \"azureml://datastores/workspaceblobstore/paths/$ENDPOINT_NAME/mnist\"

- }

- }

- }

- }")

-

- JOB_ID=$(echo $response | jq -r '.id')

- JOB_ID_SUFFIX=$(echo ${JOB_ID##/*/})

- ```

- - Or use the long form for the same URI:

- ```rest-api

- response=$(curl --location --request POST $SCORING_URI \

- --header "Authorization: Bearer $SCORING_TOKEN" \

- --header "Content-Type: application/json" \

- --data-raw "{

- \"properties\": {

- \"InputData\": {

- \"mnistinput\": {

- \"JobInputType\" : \"UriFolder\",

- \"Uri\": \"azureml://subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/workspaces/$WORKSPACE/datastores/workspaceblobstore/paths/$ENDPOINT_NAME/mnist\"

- }

- }

- }

- }")

-

- JOB_ID=$(echo $response | jq -r '.id')

- JOB_ID_SUFFIX=$(echo ${JOB_ID##/*/})

- ```

- 1. Create the V2 data asset:

- ```rest-api

- DATA_NAME="mnist"

- DATA_VERSION=$RANDOM

-

- response=$(curl --location --request PUT https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/data/$DATA_NAME/versions/$DATA_VERSION?api-version=$API_VERSION \

- --header "Content-Type: application/json" \

- --header "Authorization: Bearer $TOKEN" \

- --data-raw "{

- \"properties\": {

- \"dataType\": \"uri_folder\",

- \"dataUri\": \"https://pipelinedata.blob.core.windows.net/sampledata/mnist\",

- \"description\": \"Mnist data asset\"

- }

- }")

- ```

- 2. Reference the data asset in the batch scoring job:

- ```rest-api

- response=$(curl --location --request POST $SCORING_URI \

- --header "Authorization: Bearer $SCORING_TOKEN" \

- --header "Content-Type: application/json" \

- --data-raw "{

- \"properties\": {

- \"InputData\": {

- \"mnistInput\": {

- \"JobInputType\" : \"UriFolder\",

- \"Uri": \"azureml://locations/$LOCATION_NAME/workspaces/$WORKSPACE_NAME/data/$DATA_NAME/versions/$DATA_VERSION/\"

- }

- }

- }

- }")

-

- JOB_ID=$(echo $response | jq -r '.id')

- JOB_ID_SUFFIX=$(echo ${JOB_ID##/*/})

- ```

- ```rest-api

- response=$(curl --location --request POST $SCORING_URI \

- --header "Authorization: Bearer $SCORING_TOKEN" \

- --header "Content-Type: application/json" \

- --data-raw "{

- \"properties\": {

- \"InputData\": {

- \"mnistInput\": {

- \"JobInputType\" : \"UriFile\",

- \"Uri": \"https://pipelinedata.blob.core.windows.net/sampledata/mnist/0.png\"

- }

- }

- }

- }")

-

- JOB_ID=$(echo $response | jq -r '.id')

- JOB_ID_SUFFIX=$(echo ${JOB_ID##/*/})

- ```

-> [!NOTE]

-> We strongly recommend using the latest REST API version for batch scoring.

-> - If you want to use local data, you can upload it to Azure Machine Learning registered datastore and use REST API for Cloud data.

-> - If you are using existing V1 FileDataset for batch endpoint, we recommend migrating them to V2 data assets and refer to them directly when invoking batch endpoints. Currently only data assets of type `uri_folder` or `uri_file` are supported. Batch endpoints created with GA CLIv2 (2.4.0 and newer) or GA REST API (2022-05-01 and newer) will not support V1 Dataset.

-> - You can also extract the URI or path on datastore extracted from V1 FileDataset by using `az ml dataset show` command with `--query` parameter and use that information for invoke.

-> - While Batch endpoints created with earlier APIs will continue to support V1 FileDataset, we will be adding further V2 data assets support with the latest API versions for even more usability and flexibility. For more information on V2 data assets, see [Work with data using SDK v2](how-to-read-write-data-v2.md). For more information on the new V2 experience, see [What is v2](concept-v2.md).

-#### Configure the output location and overwrite settings

-The batch scoring results are by default stored in the workspace's default blob store within a folder named by job name (a system-generated GUID). You can configure where to store the scoring outputs when you invoke the batch endpoint. Use `OutputData` to configure the output file path on an Azure Machine Learning registered datastore. `OutputData` has `JobOutputType` and `Uri` keys. `UriFile` is the only supported value for `JobOutputType`. The syntax for `Uri` is the same as that of `InputData`, i.e., `azureml://datastores/<datastore-name>/paths/<path-on-datastore>/<file-name>`.

-Following is the example snippet for configuring the output location for the batch scoring results.

-```rest-api

-response=$(curl --location --request POST $SCORING_URI \

- \"properties\": {

- \"InputData\":

- {

- \"mnistInput\": {

- \"JobInputType\" : \"UriFolder\",

- \"Uri": \"azureml://datastores/workspaceblobstore/paths/$ENDPOINT_NAME/mnist\"

- }

- },

- \"OutputData\":

- {

- \"mnistOutput\": {

- \"JobOutputType\": \"UriFile\",

- \"Uri\": \"azureml://datastores/workspaceblobstore/paths/$ENDPOINT_NAME/mnistOutput/$OUTPUT_FILE_NAME\"

- }

- }

- }

-}")

-JOB_ID=$(echo $response | jq -r '.id')

-JOB_ID_SUFFIX=$(echo ${JOB_ID##/*/})

-```

-> [!IMPORTANT]

-> You must use a unique output location. If the output file exists, the batch scoring job will fail.

-### Check the batch scoring job

-Batch scoring jobs usually take some time to process the entire set of inputs. Monitor the job status and check the results after it's completed:

-> [!TIP]

-> The example invokes the default deployment of the batch endpoint. To invoke a non-default deployment, use the `azureml-model-deployment` HTTP header and set the value to the deployment name. For example, using a parameter of `--header "azureml-model-deployment: $DEPLOYMENT_NAME"` with curl.

-### Check batch scoring results

-For information on checking the results, see [Check batch scoring results](batch-inference/how-to-use-batch-endpoint.md#check-batch-scoring-results).

-## Delete the batch endpoint

-If you aren't going use the batch endpoint, you should delete it with the below command (it deletes the batch endpoint and all the underlying deployments):

-## Next steps

-* Learn [how to deploy your model for batch scoring](batch-inference/how-to-use-batch-endpoint.md).

-* Learn to [Troubleshoot batch endpoints](batch-inference/how-to-troubleshoot-batch-endpoints.md)

-* A model registered in the workspace. In this tutorial, we'll use an MLflow model. Particularly, we are using the *heart condition classifier* created in the tutorial [Using MLflow models in batch deployments](how-to-mlflow-batch.md).

-* You must have an endpoint already created. If you don't, follow the instructions at [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md). This example assumes the endpoint is named `heart-classifier-batch`.

-* You must have a compute created where to deploy the deployment. If you don't, follow the instructions at [Create compute](how-to-use-batch-endpoint.md#create-compute). This example assumes the name of the compute is `cpu-cluster`.

-```azurecli

-MODEL_NAME='heart-classifier-sklpipe'

-az ml model create --name $MODEL_NAME --type "custom_model" --path "model"

-```

-# [Python](#tab/sdk)

-```python

-model_name = 'heart-classifier'

-model = ml_client.models.create_or_update(

- Model(name=model_name, path='model', type=AssetTypes.CUSTOM_MODEL)

-)

-```

- :::code language="yaml" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/custom-outputs-parquet/deployment.yml" range="6-9":::

- # [Python](#tab/sdk)

- ```python

- environment = Environment(

- name="batch-mlflow-xgboost",

- conda_file="environment/conda.yaml",

- image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest",

- )

- ```

-2. Create the deployment

- > This example assumes you have an endpoint created with the name `heart-classifier-batch` and a compute cluster with name `cpu-cluster`. If you don't, please follow the steps in the doc [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md).

- :::code language="yaml" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/custom-outputs-parquet/deployment.yml":::

- :::code language="azurecli" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/custom-outputs-parquet/deploy-and-run.sh" ID="create_batch_deployment_set_default" :::

- # [Python](#tab/sdk)

- ```python

- deployment = BatchDeployment(

- name="classifier-xgboost-parquet",

- description="A heart condition classifier based on XGBoost",

- endpoint_name=endpoint.name,

- model=model,

- environment=environment,

- code_configuration=CodeConfiguration(

- code="code/",

- scoring_script="batch_driver.py",

- ),

- compute=compute_name,

- instance_count=2,

- max_concurrency_per_instance=2,

- mini_batch_size=2,

- output_action=BatchDeploymentOutputAction.SUMMARY_ONLY,

- retry_settings=BatchRetrySettings(max_retries=3, timeout=300),

- logging_level="info",

- )

- ```

- ```python

- ml_client.batch_deployments.begin_create_or_update(deployment)

- ```

-

- > [!IMPORTANT]

- > Notice that now `output_action` is set to `SUMMARY_ONLY`.

-1. Let's create the data asset first. This data asset consists of a folder with multiple CSV files that we want to process in parallel using batch endpoints. You can skip this step is your data is already registered as a data asset or you want to use a different input type.

- Create a data asset definition in `YAML`:

-

- __heart-dataset-unlabeled.yml__

-

- ```yaml

- $schema: https://azuremlschemas.azureedge.net/latest/data.schema.json

- name: heart-dataset-unlabeled

- description: An unlabeled dataset for heart classification.

- type: uri_folder

- path: heart-dataset

- ```

-

- Then, create the data asset:

-

- ```azurecli

- az ml data create -f heart-dataset-unlabeled.yml

- ```

-

- # [Python](#tab/sdk)

-

- ```python

- data_path = "resources/heart-dataset/"

- dataset_name = "heart-dataset-unlabeled"

- heart_dataset_unlabeled = Data(

- path=data_path,

- type=AssetTypes.URI_FOLDER,

- description="An unlabeled dataset for heart classification",

- name=dataset_name,

- )

- ```

-

- Then, create the data asset:

-

- ```python

- ml_client.data.create_or_update(heart_dataset_unlabeled)

- ```

-

- To get the newly created data asset, use:

-

- ```python

- heart_dataset_unlabeled = ml_client.data.get(name=dataset_name, label="latest")

- ```

-

-1. Now that the data is uploaded and ready to be used, let's invoke the endpoint:

- # [Azure CLI](#tab/cli)

-

- ```azurecli

- JOB_NAME = $(az ml batch-endpoint invoke --name $ENDPOINT_NAME --deployment-name $DEPLOYMENT_NAME --input azureml:heart-dataset-unlabeled@latest | jq -r '.name')

- ```

- # [Python](#tab/sdk)

- ```python

- input = Input(type=AssetTypes.URI_FOLDER, path=heart_dataset_unlabeled.id)

- job = ml_client.batch_endpoints.invoke(

- endpoint_name=endpoint.name,

- deployment_name=deployment.name,

- input=input,

- )

- ```

- ```azurecli

- az ml job show --name $JOB_NAME

- ```

- # [Python](#tab/sdk)

- ```python

- ml_client.jobs.get(job.name)

- ```

-```azurecli

-az ml job download --name $JOB_NAME --output-name score --download-path ./

-```

-# [Python](#tab/sdk)

-```python

-ml_client.jobs.download(name=job.name, output_name='score', download_path='./')

-```

-```python

-import pandas as pd

-import glob

-output_files = glob.glob("named-outputs/score/*.parquet")

-score = pd.concat((pd.read_parquet(f) for f in output_files))

-```

-Online endpoints are endpoints that are used for real-time inferencing. There are two types of online endpoints: **managed online endpoints** and **Kubernetes online endpoints**. For more information on endpoints and differences between managed online endpoints and Kubernetes online endpoints, see [What are Azure Machine Learning endpoints?](concept-endpoints.md#managed-online-endpoints-vs-kubernetes-online-endpoints).

-Learn how to use [NVIDIA Triton Inference Server](https://aka.ms/nvidia-triton-docs) in Azure Machine Learning with [online endpoints](concept-endpoints.md#what-are-online-endpoints).

-Triton is multi-framework, open-source software that is optimized for inference. It supports popular machine learning frameworks like TensorFlow, ONNX Runtime, PyTorch, NVIDIA TensorRT, and more. It can be used for your CPU or GPU workloads. No-code deployment for Triton models is supported in both [managed online endpoints and Kubernetes online endpoints](concept-endpoints.md#managed-online-endpoints-vs-kubernetes-online-endpoints).

-In this article, you will learn how to deploy Triton and a model to a [managed online endpoint](concept-endpoints.md#managed-online-endpoints). Information is provided on using the CLI (command line), Python SDK v2, and Azure Machine Learning studio.

-# Image processing with batch deployments

-Batch Endpoints can be used for processing tabular data, but also any other file type like images. Those deployments are supported in both MLflow and custom models. In this tutorial, we will learn how to deploy a model that classifies images according to the ImageNet taxonomy.

-* You must have a batch endpoint already created. This example assumes the endpoint is named `imagenet-classifier-batch`. If you don't have one, follow the instructions at [Use batch endpoints for batch scoring](how-to-use-batch-endpoint.md).

-* You must have a compute created where to deploy the deployment. This example assumes the name of the compute is `cpu-cluster`. If you don't, follow the instructions at [Create compute](how-to-use-batch-endpoint.md#create-compute).

-# [Azure CLI](#tab/azure-cli)

-Batch Endpoint can only deploy registered models so we need to register it. You can skip this step if the model you are trying to deploy is already registered.

- # [Python](#tab/sdk)

- # [Python](#tab/sdk)

- # [Python](#tab/sdk)

- # [Python](#tab/sdk)

- # [Azure Machine Learning SDK for Python](#tab/sdk)

- # [Python](#tab/sdk)

- # [Python](#tab/sdk)

- # [Python](#tab/sdk)

- # [Python](#tab/sdk)

- # [Python](#tab/sdk)

- # [Python](#tab/sdk)

-> * [v2](how-to-import-data-assets.md)

-In this article, learn how to import data into the Azure Machine Learning platform from external sources. A successful import automatically creates and registers an Azure Machine Learning data asset with the name provided during the import. An Azure Machine Learning data asset resembles a web browser bookmark (favorites). You don't need to remember long storage paths (URIs) that point to your most frequently used data. Instead, you can create a data asset, and then access that asset with a friendly name.

-The transferred data is partitioned and securely stored as parquet files in Azure storage, to enable faster processing during training. ADF compute costs only involve the time used for data transfers. Storage costs only involve the time needed to cache the data, because cached data is a copy of the data imported from an external source. That external source is hosted in Azure storage.

-The caching feature involves upfront compute and storage costs. However, it pays for itself, and can save money, because it reduces recurring training compute costs, compared to direct connections to external source data during training. It caches data as parquet files, which makes job training faster and more reliable against connection timeouts for larger data sets. Additionally, the caching feature leads to fewer reruns, and fewer training failures.

-Customers who want the "auto-deletion" of unused imported data assets can now choose to import data into "workspacemanageddatastore," also known as "workspacemanagedstore". Microsoft manages this datastore on behalf of the customer and provides the convenience of automatic data management on certain conditions like - last used time or created time. By default, all the data assets imported into the workspace-managed datastore have an auto-delete setting configured to "not used for 30 days". If a data asset isn't used for 30 days, it will automatically delete. Within that time, you can edit the "auto-delete" settings in the imported data asset. You can increase or decrease the duration (number of days), or you can change the "condition". As of now, created time and unused time are the two conditions supported. If you chose to work with a "managed datastore", you must only point the `path` on your data import to `azureml://datastores/workspacemanagedstore`, and Azure Machine Learning will create one for you. The managed datastore costs the same as a regular ADLS Gen2 datastore, which charges by the amount of data that is stored in it. However, the managed datastore offers the benefit of data management.

-> [!NOTE]

-> - There will be only one `workspacemanagedstore` per workspace that would be created

-> - The managed datastore backfills, or is automatic, when the first import job that refers to the managed datastore is submitted.

-> - Users cannot create a `workspacemanagedstore` using any datastore APIs or methods.

-> - In the import definition, users must refer to the managed datastore in this way: `path: azureml://datastores/manageddatastore`. The system automatically assigns a unique path for storage of the imported data. Unlike customer-owned datastores, or a workspace default blobstore, there is no need to provide the entire path where you want to import data.

-> - Currently, the path on the `workspacemanagedstore` can be accessed only by data import service, and `workspacemanagedstore` cannot be given as a destination in any other process or step

-> - The data path in the `workspacemanagedstore` can be accessed only by AzureML service

-> - To access data from the `workspacemanagedstore`, reference the data asset name and version, similar to any other data asset in your jobs or scripts, or processes submitted to AzureML. AzureML knows how to read data from managed datastore.

-> For a successful data import, please verify that you have installed the latest Azure-ai-ml package (version 1.5.0 or later) for SDK, and the ml extension (version 2.15.1 or later).

-> If you have an older SDK package or CLI extension, please remove the old one and install the new one with the code shown in the tab section. Follow these instructions for SDK and CLI:

-# Supported "paths" include either on regular datastore or managed datastore as shown below:

-# path: azureml://datastores/<data_store_name>/paths/<my_path>/${{name}}

-# or path: azureml://datastores/workspacemanagedstore

-# Supported "paths" include either on regular datastore or managed datastore as shown below:

-# or path: azureml://datastores/workspacemanagedstore

-# Supported "paths" include either on regular datastore or managed datastore as shown below:

-# or path: azureml://datastores/workspacemanagedstore

-# Supported "paths" include either on regular datastore or managed datastore as shown below:

-# or path: azureml://datastores/workspacemanagedstore

-The data import action is an asynchronous action. It can take a long time. After submission of an import data action via the CLI or SDK, the Azure Machine Learning service might need several minutes to connect to the external data source. Then the service would start the data import and handle data caching and registration. The times needed for a data import also depends on the size of the source data set.

-## Additional Capabilities

-description: Learn how to manage imported data assets also known as edit auto-deletion.

-# Manage imported data assets (preview)

-> [!div class="op_single_selector" title1="Select the version of Azure Machine Learning SDK you are using:"]

-> * [v2](how-to-import-data-assets.md)

-In this article, learn how to manage imported data assets from life-cycle point of view. We learn how to modify or update auto-delete settings on the data assets that are imported on to a managed datastore (`workspacemanagedstore`) that Microsoft manages for the customer.

-> [!NOTE]

-> Auto-delete settings capability or lifecycle management is offered currently only on the imported data assets in managed datastore aka `workspacemanagedstore`.

-## Modifying auto delete settings

-You can change the auto-delete setting value or condition

-# [Azure CLI](#tab/cli)

-```cli

-> az ml data update -n <my_imported_ds> -v <version_number> --set auto_delete_setting.value='45d'

-> az ml data update -n <my_imported_ds> -v <version_number> --set auto_delete_setting.condition='created_greater_than'

-```

-# [Python SDK](#tab/Python-SDK)

-```python

-from azure.ai.ml.entities import DataΓÇ»

-from azure.ai.ml.constants import AssetTypesΓÇ»

-name='<my_imported_ds>'

-version='<version_number>'

-type='mltable'

-auto_delete_setting = AutoDeleteSetting(

- condition='created_greater_than', value='45d'

-)ΓÇ»

-my_data=Data(name=name,version=version,type=type, auto_delete_setting=auto_delete_setting)

-ml_client.data.create_or_update(my_data)ΓÇ»

-```

-## Deleting/removing auto delete settings

-You can remove a previously configured auto-delete setting.

-

-# [Azure CLI](#tab/cli)

-```cli

-> az ml data update -n <my_imported_ds> -v <version_number> --remove auto_delete_setting

-```

-# [Python SDK](#tab/Python-SDK)

-```python

-from azure.ai.ml.entities import DataΓÇ»

-from azure.ai.ml.constants import AssetTypesΓÇ»

-name='<my_imported_ds>'

-version='<version_number>'

-type='mltable'

-ΓÇ»

-my_data=Data(name=name,version=version,type=type, auto_delete_setting=None)

-ml_client.data.create_or_update(my_data)ΓÇ»

-```

-## Query on the configured auto delete settings

-You can view and list the data assets with certain conditions or with values configured in the "auto-delete" settings, as shown in this Azure CLI code sample:

-```cli

-> az ml data list --query '[?auto_delete_setting.\"condition\"==''created_greater_than'']'

-> az ml data list --query '[?auto_delete_setting.\"value\"==''30d'']'

-```

-## Next steps

-# Use MLflow models in batch deployments

-In this article, learn how to deploy your [MLflow](https://www.mlflow.org) model to Azure Machine Learning for both batch inference using batch endpoints. Azure Machine Learning supports no-code deployment of models created and logged with MLflow. This means that you don't have to provide a scoring script or an environment.

-For no-code-deployment, Azure Machine Learning

-> For more information about the supported file types in batch endpoints with MLflow, view [Considerations when deploying to batch inference](#considerations-when-deploying-to-batch-inference).

-The information in this article is based on code samples contained in the [azureml-examples](https://github.com/azure/azureml-examples) repository. To run the commands locally without having to copy/paste YAML and other files, clone the repo and then change directories to the `cli/endpoints/batch/deploy-models/heart-classifier-mlflow` if you are using the Azure CLI or `sdk/python/endpoints/batch/deploy-models/heart-classifier-mlflow` if you are using our SDK for Python.

-git clone https://github.com/Azure/azureml-examples --depth 1

-cd azureml-examples/cli/endpoints/batch/deploy-models/heart-classifier-mlflow

-* You must have a MLflow model. If your model is not in MLflow format and you want to use this feature, you can [convert your custom ML model to MLflow format](how-to-convert-custom-model-to-mlflow.md).

-1. First, let's connect to Azure Machine Learning workspace where we are going to work on.

- # [Azure CLI](#tab/cli)

-

- ```azurecli

- az account set --subscription <subscription>

- az configure --defaults workspace=<workspace> group=<resource-group> location=<location>

- ```

-

- # [Python](#tab/sdk)

-

- The workspace is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section, we'll connect to the workspace in which you'll perform deployment tasks.

-

- 1. Import the required libraries:

-

- ```python

- from azure.ai.ml import MLClient, Input

- from azure.ai.ml.entities import BatchEndpoint, BatchDeployment, Model, AmlCompute, Data, BatchRetrySettings

- from azure.ai.ml.constants import AssetTypes, BatchDeploymentOutputAction

- from azure.identity import DefaultAzureCredential

- ```

-

- 2. Configure workspace details and get a handle to the workspace:

-

- ```python

- subscription_id = "<subscription>"

- resource_group = "<resource-group>"

- workspace = "<workspace>"

-

- ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group, workspace)

- ```

- # [Azure CLI](#tab/cli)

- ```azurecli

- MODEL_NAME='heart-classifier'

- az ml model create --name $MODEL_NAME --type "mlflow_model" --path "model"

- ```

- # [Python](#tab/sdk)

- ```python

- model_name = 'heart-classifier'

- model_local_path = "heart-classifier-mlflow/model"

- model = ml_client.models.create_or_update(

- Model(name=model_name, path=model_local_path, type=AssetTypes.MLFLOW_MODEL)

- )

- ```

- # [Azure CLI](#tab/cli)

-

- Create a compute definition `YAML` like the following one:

-

- __cpu-cluster.yml__

-

- ```yaml

- $schema: https://azuremlschemas.azureedge.net/latest/amlCompute.schema.json

- name: cluster-cpu

- type: amlcompute

- size: STANDARD_DS3_v2

- min_instances: 0

- max_instances: 2

- idle_time_before_scale_down: 120

- ```

- Create the compute using the following command:

- ```azurecli

- az ml compute create -f cpu-cluster.yml

- ```

- # [Python](#tab/sdk)

- To create a new compute cluster where to create the deployment, use the following script:

- ```python

- compute_name = "cpu-cluster"

- if not any(filter(lambda m : m.name == compute_name, ml_client.compute.list())):

- compute_cluster = AmlCompute(name=compute_name, description="amlcompute", min_instances=0, max_instances=2)

- ml_client.begin_create_or_update(compute_cluster)

- ```

- ```azurecli

- ENDPOINT_NAME="heart-classifier-batch"

- ```

- # [Python](#tab/sdk)

- ```python

- endpoint_name="heart-classifier-batch"

- ```

- # [Azure CLI](#tab/cli)

- To create a new endpoint, create a `YAML` configuration like the following:

- :::code language="yaml" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/endpoint.yml" :::

- Then, create the endpoint with the following command:

- :::code language="azurecli" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/deploy-and-run.sh" ID="create_batch_endpoint" :::

- # [Python](#tab/sdk)

- To create a new endpoint, use the following script:

- ```python

- endpoint = BatchEndpoint(

- name=endpoint_name,

- description="A heart condition classifier for batch inference",

- )

- ```

- Then, create the endpoint with the following command:

- ```python

- ml_client.batch_endpoints.begin_create_or_update(endpoint)

- ```

- # [Azure CLI](#tab/cli)

- To create a new deployment under the created endpoint, create a `YAML` configuration like the following:

- :::code language="yaml" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/deployment-simple/deployment.yml" :::

- Then, create the deployment with the following command:

- :::code language="azurecli" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/deploy-and-run.sh" ID="create_batch_deployment_set_default" :::

- # [Python](#tab/sdk)

- To create a new deployment under the created endpoint, first define the deployment:

- ```python

- deployment = BatchDeployment(

- name="classifier-xgboost-mlflow",

- description="A heart condition classifier based on XGBoost",

- endpoint_name=endpoint.name,

- model=model,

- compute=compute_name,

- instance_count=2,

- max_concurrency_per_instance=2,

- mini_batch_size=2,

- output_action=BatchDeploymentOutputAction.APPEND_ROW,

- output_file_name="predictions.csv",

- retry_settings=BatchRetrySettings(max_retries=3, timeout=300),

- logging_level="info",

- )

- ```

- Then, create the deployment with the following command:

- ```python

- ml_client.batch_deployments.begin_create_or_update(deployment)

- ```

-

- > [!NOTE]

- > Batch deployments only support deploying MLflow models with a `pyfunc` flavor. To use a different flavor, see [Customizing MLflow models deployments with a scoring script](#customizing-mlflow-models-deployments-with-a-scoring-script)..

- # [Azure CLI](#tab/cli)

- :::code language="azurecli" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/deploy-and-run.sh" ID="update_default_deployment" :::

- # [Python](#tab/sdk)

- ```python

- endpoint = ml_client.batch_endpoints.get(endpoint.name)

- endpoint.defaults.deployment_name = deployment.name

- ml_client.batch_endpoints.begin_create_or_update(endpoint)

- ```

- # [Azure CLI](#tab/cli)

- a. Create a data asset definition in `YAML`:

- __heart-dataset-unlabeled.yml__

- :::code language="yaml" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/heart-dataset-unlabeled.yml" :::

- b. Create the data asset:

- :::code language="azurecli" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/deploy-and-run.sh" ID="register_dataset" :::

- # [Python](#tab/sdk)

- a. Create a data asset definition:

- ```python

- data_path = "data"

- dataset_name = "heart-dataset-unlabeled"

- heart_dataset_unlabeled = Data(

- path=data_path,

- type=AssetTypes.URI_FOLDER,

- description="An unlabeled dataset for heart classification",

- name=dataset_name,

- )

- ```

- b. Create the data asset:

- ```python

- ml_client.data.create_or_update(heart_dataset_unlabeled)

- ```

- c. Refresh the object to reflect the changes:

- ```python

- heart_dataset_unlabeled = ml_client.data.get(name=dataset_name, label="latest")

- ```

- # [Azure CLI](#tab/cli)

- :::code language="azurecli" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/deploy-and-run.sh" ID="start_batch_scoring_job" :::

- > [!NOTE]

- > The utility `jq` may not be installed on every installation. You can get installation instructions in [this link](https://stedolan.github.io/jq/download/).

- # [Python](#tab/sdk)

- ```python

- input = Input(type=AssetTypes.URI_FOLDER, path=heart_dataset_unlabeled.id)

- job = ml_client.batch_endpoints.invoke(

- endpoint_name=endpoint.name,

- input=input,

- )

- ```

-

- > [!TIP]

- > Notice how we are not indicating the deployment name in the invoke operation. That's because the endpoint automatically routes the job to the default deployment. Since our endpoint only has one deployment, then that one is the default one. You can target an specific deployment by indicating the argument/parameter `deployment_name`.

- # [Azure CLI](#tab/cli)

- :::code language="azurecli" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/deploy-and-run.sh" ID="show_job_in_studio" :::

- # [Python](#tab/sdk)

- ```python

- ml_client.jobs.get(job.name)

- ```

- * The file name where the data was read from. In tabular data, use this field to know which prediction belongs to which input data. For any given file, predictions are returned in the same order they appear in the input file so you can rely on the row number to match the corresponding prediction.

- * The prediction associated with the input data. This value is returned "as-is" it was provided by the model's `predict().` function.

-# [Python](#tab/sdk)

-```python

-ml_client.jobs.download(name=job.name, output_name='score', download_path='./')

-```

-```python

-import pandas as pd

-from ast import literal_eval

-with open('named-outputs/score/predictions.csv', 'r') as f:

- pd.DataFrame(literal_eval(f.read().replace('\n', ',')), columns=['file', 'prediction'])

-```

-| `.csv`, `.parquet` | `pd.DataFrame` | `ColSpec`. If not provided, columns typing is not enforced. |

-> Be advised that any unsupported file that may be present in the input data will make the job to fail. You will see an error entry as follows: *"ERROR:azureml:Error processing input file: '/mnt/batch/tasks/.../a-given-file.parquet'. File type 'parquet' is not supported."*.

- :::code language="python" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/deployment-custom/code/batch_driver.py" :::

- > [!TIP]

- > If your model is already registered in the model registry, you can download/copy the `conda.yml` file associated with your model by going to [Azure Machine Learning studio](https://ml.azure.com) > Models > Select your model from the list > Artifacts. Open the root folder in the navigation and select the `conda.yml` file listed. Click on Download or copy its content.

- > [!IMPORTANT]

- > This example uses a conda environment specified at `/heart-classifier-mlflow/environment/conda.yaml`. This file was created by combining the original MLflow conda dependencies file and adding the package `azureml-core`. __You can't use the `conda.yml` file from the model directly__.

- # [Azure CLI](#tab/cli)

-

- No extra step is required for the Azure Machine Learning CLI. The environment definition will be included in the deployment file.

- # [Python](#tab/sdk)

- Let's get a reference to the environment:

- ```python

- environment = Environment(

- name="batch-mlflow-xgboost",

- conda_file="deployment-custom/environment/conda.yaml",

- image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest",

- )

- ```

-1. Let's create the deployment now:

- # [Azure CLI](#tab/cli)

- To create a new deployment under the created endpoint, create a `YAML` configuration like the following:

- :::code language="yaml" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/deployment-custom/deployment.yml" :::

- Then, create the deployment with the following command:

- :::code language="azurecli" source="~/azureml-examples-main/cli/endpoints/batch/deploy-models/heart-classifier-mlflow/deploy-and-run.sh" ID="create_new_deployment_not_default" :::

- # [Python](#tab/sdk)

- To create a new deployment under the created endpoint, use the following script:

- ```python

- deployment = BatchDeployment(

- name="classifier-xgboost-custom",

- description="A heart condition classifier based on XGBoost",

- endpoint_name=endpoint.name,

- model=model,

- environment=environment,

- code_configuration=CodeConfiguration(

- code="deployment-custom/code/",

- scoring_script="batch_driver.py",

- ),

- compute=compute_name,

- instance_count=2,

- max_concurrency_per_instance=2,

- mini_batch_size=2,

- output_action=BatchDeploymentOutputAction.APPEND_ROW,

- output_file_name="predictions.csv",

- retry_settings=BatchRetrySettings(max_retries=3, timeout=300),

- logging_level="info",

- )

- ml_client.batch_deployments.begin_create_or_update(deployment)

- ```

-

-# Text processing with batch deployments

-Batch Endpoints can be used for processing tabular data that contain text. Those deployments are supported in both MLflow and custom models. In this tutorial we will learn how to deploy a model that can perform text summarization of long sequences of text using a model from HuggingFace.

-The information in this article is based on code samples contained in the [azureml-examples](https://github.com/azure/azureml-examples) repository. To run the commands locally without having to copy/paste YAML and other files, clone the repo and then change directories to the [`cli/endpoints/batch/deploy-models/huggingface-text-summarization`](https://github.com/azure/azureml-examples/tree/main/cli/endpoints/batch/deploy-models/huggingface-text-summarization) if you are using the Azure CLI or [`sdk/python/endpoints/batch/deploy-models/huggingface-text-summarization`](https://github.com/azure/azureml-examples/tree/main/sdk/python/endpoints/batch/deploy-models/huggingface-text-summarization) if you are using our SDK for Python.

-# [Azure CLI](#tab/cli)

-git clone https://github.com/Azure/azureml-examples --depth 1

-cd azureml-examples/cli/endpoints/batch/deploy-models/huggingface-text-summarization

-```

-# [Python](#tab/python)

-In a Jupyter notebook:

-```python

-!git clone https://github.com/Azure/azureml-examples --depth 1

-!cd azureml-examples/sdk/python/endpoints/batch/deploy-models/huggingface-text-summarization

-### Connect to your workspace

-First, let's connect to Azure Machine Learning workspace where we're going to work on.

-# [Azure CLI](#tab/cli)

-```azurecli

-az account set --subscription <subscription>

-az configure --defaults workspace=<workspace> group=<resource-group> location=<location>

-```

-# [Python](#tab/python)

-The workspace is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section, we'll connect to the workspace in which you'll perform deployment tasks.

-1. Import the required libraries:

-```python

-from azure.ai.ml import MLClient, Input

-from azure.ai.ml.entities import BatchEndpoint, BatchDeployment, Model, AmlCompute, Data, BatchRetrySettings

-from azure.ai.ml.constants import AssetTypes, BatchDeploymentOutputAction

-from azure.identity import DefaultAzureCredential

-```

-2. Configure workspace details and get a handle to the workspace:

-```python

-subscription_id = "<subscription>"

-resource_group = "<resource-group>"

-workspace = "<workspace>"

-ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group, workspace)

-```

-1. We need to indicate over which environment we are going to run the deployment. In our case, our model runs on `Torch` and it requires the libraries `transformers`, `accelerate`, and `optimum` from HuggingFace. Azure Machine Learning already has an environment with Torch and GPU support available. We are just going to add a couple of dependencies in a `conda.yml` file.

- __environment/conda.yml__

- conda_file="environment/torch200-conda.yml",

- > The environment `torch200-transformers-gpu` we've created requires a CUDA 11.8 compatible hardware device to run Torch 2.0 and Ubuntu 20.04. If your GPU device doesn't support this version of CUDA, you can check the alternative `torch113-conda.yml` conda environment (also available on the repository), which runs Torch 1.3 over Ubuntu 18.04 with CUDA 10.1. However, acceleration using the `optimum` and `accelerate` libraries won't be supported on this configuration.

-This article assumes you're using online endpoints, that is, endpoints that are used for online (real-time) inferencing. There are two types of online endpoints: **managed online endpoints** and **Kubernetes online endpoints**. For more information on endpoints and the differences between managed online endpoints and Kubernetes online endpoints, see [What are Azure Machine Learning endpoints?](concept-endpoints.md#managed-online-endpoints-vs-kubernetes-online-endpoints).

-When deploying a machine learning model to a batch endpoint, you can secure their communication using private networks. This article explains the requirements to use batch endpoint in an environment secured by private networks.

-You can secure the inbound scoring requests from clients to an _online endpoint_. You can also secure the outbound communications between a _deployment_ and the Azure resources it uses. Security for inbound and outbound communication are configured separately. For more information on endpoints and deployments, see [What are endpoints and deployments](concept-endpoints.md#what-are-endpoints-and-deployments).

- * **Queue** - Only needed if you plan to use [Batch endpoints](concept-endpoints.md#what-are-batch-endpoints) or the [ParallelRunStep](./tutorial-pipeline-batch-scoring-classification.md) in an Azure Machine Learning pipeline.

- * **Table** - Only needed if you plan to use [Batch endpoints](concept-endpoints.md#what-are-batch-endpoints) or the [ParallelRunStep](./tutorial-pipeline-batch-scoring-classification.md) in an Azure Machine Learning pipeline.

-We are going to create a pipeline in Azure Data Factory that can invoke a given batch endpoint over some data. The pipeline will communicate with Azure Machine Learning batch endpoints using REST. To know more about how to use the REST API of batch endpoints read [Deploy models with REST for batch scoring](how-to-deploy-batch-with-rest.md).

-* The Logic App we are creating will communicate with Azure Machine Learning batch endpoints using REST. To know more about how to use the REST API of batch endpoints read [Deploy models with REST for batch scoring](how-to-deploy-batch-with-rest.md).

-For more information on concepts for endpoints and deployments, see [What are online endpoints?](concept-endpoints.md#what-are-online-endpoints)

-[Managed online endpoints](concept-endpoints.md#what-are-online-endpoints) help to deploy your ML models in a turnkey manner. Managed online endpoints work with powerful CPU and GPU machines in Azure in a scalable, fully managed way. Managed online endpoints take care of serving, scaling, securing, and monitoring your models, freeing you from the overhead of setting up and managing the underlying infrastructure. Details can be found on [Deploy and score a machine learning model by using an online endpoint](how-to-deploy-online-endpoints.md).

-| `version` | string | Version of the schedule. If omitted, Azure Machine Learning autogenerates a version. | |

-| `create_job` | object or string | **Required.** The definition of the job that triggered by a schedule. **One of `string` or `JobDefinition` is required.**| |

-|`start_time`| string |Describes the start date and time with timezone. If start_time is omitted, the first job will run instantly, and the future jobs trigger based on the schedule, saying start_time will match the job created time. If the start time is in the past, the first job will run at the next calculated run time.|

-|`end_time`| string |Describes the end date and time with timezone. If end_time is omitted, the schedule runs until it's explicitly disabled.|

-|`pattern`|object|Specifies the pattern of the recurrence. If pattern is omitted, the job(s) is triggered according to the logic of start_time, frequency and interval.| |

-|`start_time`| string |Describes the start date and time with timezone. If start_time is omitted, the first job will run instantly and the future jobs trigger based on the schedule, saying start_time will match the job created time. If the start time is in the past, the first job will run at the next calculated run time.|

-|`end_time`| string |Describes the end date and time with timezone. If end_time is omitted, the schedule continues to run until it's explicitly disabled.|

-| `experiment_name` | string | Experiment name to organize the job under. The run record of each job will be organized under the corresponding experiment in the studio's "Experiments" tab. If omitted, it uses schedule name as default value. | |

-| `default_datastore` | string | Name of the datastore to use as the default datastore for the pipeline job. This value must be a reference to an existing datastore in the workspace using the `azureml:<datastore-name>` syntax. Any outputs defined in the `outputs` property of the parent pipeline job or child step jobs are stored in this datastore. If omitted, outputs are stored in the workspace blob datastore. | |

-| `default_compute` | string | Name of the compute target to use as the default compute for all steps in the pipeline. If compute is defined at the step level, it overrides this default compute for that specific step. This value must be a reference to an existing compute in the workspace using the `azureml:<compute-name>` syntax. | |

-| `continue_on_step_failure` | boolean | Whether the execution of steps in the pipeline should continue if one step fails. The default value is `False`, which means that if one step fails, the pipeline execution is stopped, canceling any running steps. | `False` |

-| `path` | string | The path to the data to use as input, specified in a few ways: <br><br> - A local path to the data source file or folder, for example, `path: ./iris.csv`. The data uploads during job submission. <br><br> - A URI of a cloud path to the file or folder to use as the input. Supported URI types are `azureml`, `https`, `wasbs`, `abfss`, `adl`. For more information on how to use the `azureml://` URI format, see [Core yaml syntax](reference-yaml-core-syntax.md). <br><br> - An existing registered Azure Machine Learning data asset to use as the input. To reference a registered data asset, use the `azureml:<data_name>:<data_version>` syntax or `azureml:<data_name>@latest` (to reference the latest version of that data asset), for example, `path: azureml:cifar10-data:1` or `path: azureml:cifar10-data@latest`. | | |

-| `mode` | string | Mode of how the data should be delivered to the compute target. <br><br> For read-only mount (`ro_mount`), the data is consumed as a mount path. A folder mounts as a folder and a file mounts as a file. Azure Machine Learning resolves the input to the mount path. <br><br> In the `download` mode, the data downloads to the compute target. Azure Machine Learning resolves the input to the downloaded path. <br><br> If you only want the URL of the storage location of the data artifact(s), instead of mounting or downloading the data itself, you can use the `direct` mode. This passes in the URL of the storage location as the job input. In this case, you're fully responsible for handling credentials to access the storage. | `ro_mount`, `download`, `direct` | `ro_mount` |

-| `type` | string | The type of job output. For the default `uri_folder` type, the output corresponds to a folder. | `uri_folder` | `uri_folder` |

-| `path` | string | The path to the data to use as input, specified in a few ways: <br><br> - A local path to the data source file or folder, for example, `path: ./iris.csv`. The data uploads during job submission. <br><br> - A URI of a cloud path to the file or folder to use as the input. Supported URI types are `azureml`, `https`, `wasbs`, `abfss`, `adl`. For more information on how to use the `azureml://` URI format, see [Core yaml syntax](reference-yaml-core-syntax.md). <br><br> - An existing registered Azure Machine Learning data asset to use as the input. To reference a registered data asset, use the `azureml:<data_name>:<data_version>` syntax or `azureml:<data_name>@latest` (to reference the latest version of that data asset), for example, `path: azureml:cifar10-data:1` or `path: azureml:cifar10-data@latest`. | | |

-| `mode` | string | Mode of how output file(s) are delivered to the destination storage. For read-write mount mode (`rw_mount`) the output directory is a mounted directory. In the upload mode, the written file(s) upload at the end of the job. | `rw_mount`, `upload` | `rw_mount` |

-### Import data definition (preview)

-Customer can directly use `import_data: ./<data_import>.yaml` or can use the following properties to define the data import definition.

-| Key | Type | Description | Allowed values |

-| | - | -- | -- |

-|`type`| string | **Required.** Specifies the data asset type that you want to import the data as. It can be mltable when importing from a Database source, or uri_folder when importing from a FileSource.|`mltable`, `uri_folder`|

-| `name` | string | **Required.** Data asset name to register the imported data under. | |

-| `path` | string | **Required.** The path to the datastore that takes in the imported data, specified in one of two ways: <br><br> - **Required.** A URI of datastore path. Only supported URI type is `azureml`. For more information on how to use the `azureml://` URI format, see [Core yaml syntax](reference-yaml-core-syntax.md). To avoid an over-write, a unique path for each import is recommended. To do this, parameterize the path as shown in this example - `azureml://datastores/<datastore_name>/paths/<source_name>/${{name}}`. The "datastore_name" in the example can be a datastore that you have created or can be workspaceblobstore. Alternately a "managed datastore" can be selected by referencing as shown: `azureml://datastores/workspacemanagedstore`, where the system automatically assigns a unique path. | Azure Machine Learning://<>|

-| `source` | object | External source details of the imported data source. See [Attributes of the `source`](#attributes-of-source-preview) for the set of source properties. | |

-### Attributes of `source` (preview)

-| Key | Type | Description | Allowed values | Default value |

-| | - | -- | -- | - |

-| `type` | string | The type of external source from where you intend to import data from. Only the following types are allowed at the moment - `Database` or `FileSystem`| `Database`, `FileSystem` | |

-| `query` | string | Define this value only when the `type` defined above is `database` The query in the external source of type `Database` that defines or filters data that needs to be imported.| | |

-| `path` | string | Define this only when the `type` defined above is `FileSystem` The path of the folder in the external source of type `FileSystem` where the file(s) or data that needs to be imported resides.| | |

-| `connection` | string | **Required.** The connection property for the external source referenced in the format of `azureml:<connection_name>` | | |

-## Remarks

-The `az ml schedule` command can be used for managing Azure Machine Learning models.

-## Examples

-Examples are available in the [examples GitHub repository](https://github.com/Azure/azureml-examples/tree/main/cli/schedules). A couple are shown below.

-## YAML: Schedule for a job with recurrence pattern

-## YAML: Schedule for a job with cron expression

-## YAML: Schedule for data import with recurrence pattern (preview)

-```yml

-$schema: https://azuremlschemas.azureedge.net/latest/schedule.schema.json

-name: simple_recurrence_import_schedule

-display_name: Simple recurrence import schedule

-description: a simple hourly recurrence import schedule

-trigger:

- type: recurrence

- frequency: day #can be minute, hour, day, week, month

- interval: 1 #every day

- schedule:

- hours: [4,5,10,11,12]

- minutes: [0,30]

- start_time: "2022-07-10T10:00:00" # optional - default will be schedule creation time

- time_zone: "Pacific Standard Time" # optional - default will be UTC

-import_data: ./my-snowflake-import-data.yaml

-```

-## YAML: Schedule for data import definition inline with recurrence pattern on managed datastore (preview)

-```yml

-$schema: https://azuremlschemas.azureedge.net/latest/schedule.schema.json

-name: inline_recurrence_import_schedule

-display_name: Inline recurrence import schedule

-description: an inline hourly recurrence import schedule

-trigger:

- type: recurrence

- frequency: day #can be minute, hour, day, week, month

- interval: 1 #every day

- schedule:

- hours: [4,5,10,11,12]

- minutes: [0,30]

- start_time: "2022-07-10T10:00:00" # optional - default will be schedule creation time

- time_zone: "Pacific Standard Time" # optional - default will be UTC

-import_data:

- type: mltable

- name: my_snowflake_ds

- path: azureml://datastores/workspacemanagedstore

- source:

- type: database

- query: select * from TPCH_SF1.REGION

- connection: azureml:my_snowflake_connection

-```

-## YAML: Schedule for data import with cron expression (preview)

-```yml

-$schema: https://azuremlschemas.azureedge.net/latest/schedule.schema.json

-name: simple_cron_import_schedule

-display_name: Simple cron import schedule

-description: a simple hourly cron import schedule

-trigger:

- type: cron

- expression: "0 * * * *"

- start_time: "2022-07-10T10:00:00" # optional - default will be schedule creation time

- time_zone: "Pacific Standard Time" # optional - default will be UTC

-import_data: ./my-snowflake-import-data.yaml

-```

-## YAML: Schedule for data import definition inline with cron expression (preview)

-```yml

-$schema: https://azuremlschemas.azureedge.net/latest/schedule.schema.json

-name: inline_cron_import_schedule

-display_name: Inline cron import schedule

-description: an inline hourly cron import schedule

-trigger:

- type: cron

- expression: "0 * * * *"

- start_time: "2022-07-10T10:00:00" # optional - default will be schedule creation time

- time_zone: "Pacific Standard Time" # optional - default will be UTC

-import_data:

- type: mltable

- name: my_snowflake_ds

- path: azureml://datastores/workspaceblobstore/paths/snowflake/${{name}}

- source:

- type: database

- query: select * from TPCH_SF1.REGION

- connection: azureml:my_snowflake_connection

-```

-The current schedule supports the timezones in this table. The key can be used directly in the Python SDK, while the value can be used in the YAML job. The table is organized by UTC(Coordinated Universal Time).

-Azure Machine Learning allows you to implement [online endpoints](concept-endpoints.md#what-are-online-endpoints) for real-time inferencing on client data, and [batch endpoints](concept-endpoints.md#what-are-batch-endpoints) for inferencing on large volumes of data over a period of time.

-| | Reader |

-description: This article describes how to use connectors in Purview workflows

-# Workflow connectors

-## Current workflow connectors

-2. Sign in to Azure SQL Database with your Azure AD account, and assign `db_owner` permissions to the Microsoft Purview managed identity.

-3. Run the following command on your SQL database to create a master key:

-> Currently, if you cannot enable **Allow Azure services and resources to access this workspace** on your Azure Synapse workspaces, when set up scan on Microsoft Purview governance portal, you will hit serverless DB enumeration failure. In this case, to scan serverless DBs, you can use [Microsoft Purview REST API - Scans - Create Or Update](/rest/api/purview/scanningdataplane/scans/create-or-update/) to set up scan. Refer to [this example](#set-up-scan-using-api).

-1. In the **Type** dropdown list, select the types of resources that you want to scan within this source. **SQL Database** is the only type we currently support within an Azure Synapse workspace.

-

-1. In the **Credential** dropdown list, select the credential to connect to the resources within your data source.

-

-1. Within each type, you can select to scan either all the resources or a subset of them by name.

-Here's an example of creating scan for serverless DB using API. Replace the `{place_holder}` and `enum_option_1 | enum_option_2 (note)` value with your actual settings.

-# Deploy S/4HANA infrastructure with Azure Center for SAP solutions (preview)

- 1. For **Application subnet** and **Database subnet**, map the IP address ranges as required. It's recommended to use a different subnet for each deployment.

- 1. For **Username**, enter a username.

- 1. For **SAP Transport Options**, you can choose to **Create a new SAP transport Directory** or **Use an existing SAP transport Directory** or completely skip the creation of transport directory by choosing **Dont include SAP transport directory** option. Currently, only NFS on AFS storage account fileshares are supported.

- 1. If you choose to **Use an existing SAP transport Directory**, select the pre - existing NFS fileshare under **File share name** option. The existing transport fileshare will be only mounted on this SID. The selected fileshare shall be in the same region as that of SAP system being created. Currently, file shares existing in a different region can not be selected. Provide the associated privated endpoint of the storage account where the selected fileshare exists under **Private Endpoint** option.

- 1. You can skip the creation of transport file share by selecting **Dont include SAP transport directory** option. The transport fileshare will neither be created or mounted for this SID.

- 1. For **SAP FQDN**, provide only the domain name for you system such "sap.contoso.com"

- 1. For **Managed identity name**, enter a name for a new identity you want to create or select an existing identity from the drop down menu. If you are selecting an existing identity, it should have **Contributor** role access on the Subscription or on Resource Groups related to this SAP system you are trying to deploy. That is, it requires Contributor access to the SAP application Resource Group, Virtual Network Resource Group and Resource Group which has the existing SSHKEY. If you wish to later install the SAP system using ACSS, we also recommend to give the **Storage Blob Data Reader and Reader** and **Data Access roles** on the Storage Account which has the SAP software media.

-# Get quality checks and insights for a Virtual Instance for SAP solutions (preview)

-The *Quality Insights* Azure workbook in *Azure Center for SAP solutions* provides insights about the SAP system resources. The feature is part of the monitoring capabilities built in to the *Virtual Instance for SAP solutions (VIS)*. These quality checks make sure that your SAP system uses Azure and SAP best practices for reliability and performance.

-In this how-to guide, you'll learn how to use quality checks and insights to get more information about virtual machine (VM) configurations within your SAP system.

-The **Quality checks** feature in Azure Center for SAP solutions runs validation checks for all VIS resources. These quality checks validate the SAP system configurations follow the best practices recommended by SAP and Azure. If a VIS doesn't follow these best practices, you receive a recommendation from Azure Advisor.

-The following checks are run for each VIS:

-> [!NOTE]

-> These quality checks run on all VIS instances at a regular frequency of 12 hours. The corresponding recommendations in Azure Advisor also refresh at the same 12-hour frequency.

-If you take action on one or more recommendations from Azure Center for SAP solutions, wait for the next refresh to see any new recommendations from Azure Advisor.

-# Get SAP installation media (preview)

-# Install SAP software (preview)

-# Manage a Virtual Instance for SAP solutions (preview)

-# Management of Azure Center for SAP solutions resources with Azure RBAC (preview)

-# Monitor SAP system from Azure portal (preview)

-## Unregister Azure Monitor for SAP solutions from VIS

-# What is Azure Center for SAP solutions? (preview)

-# Prepare network for infrastructure deployment (preview)

-# Register existing SAP system (preview)

-# Start and stop SAP systems (preview)

-# View post-deployment cost analysis for SAP system (preview)

-## Built-in roles used in Search

-Built-in roles include generally available and preview roles. If these roles are insufficient, [create a custom role](#create-a-custom-role) instead.

-| Role | Description and availability |

-| - | - |

-| [Owner](../role-based-access-control/built-in-roles.md#owner) | (Generally available) Full access to the search resource, including the ability to assign Azure roles. Subscription administrators are members by default.</br></br> (Preview) This role has the same access as the Search Service Contributor role on the data plane. It includes access to all data plane actions except the ability to query the search index or index documents. |

-| [Contributor](../role-based-access-control/built-in-roles.md#contributor) | (Generally available) Same level of access as Owner, minus the ability to assign roles or change authorization options. </br></br> (Preview) This role has the same access as the Search Service Contributor role on the data plane. It includes access to all data plane actions except the ability to query the search index or index documents. |

-| [Reader](../role-based-access-control/built-in-roles.md#reader) | (Generally available) Limited access to partial service information. In the portal, the Reader role can access information in the service Overview page, in the Essentials section and under the Monitoring tab. All other tabs and pages are off limits. </br></br>This role has access to service information: service name, resource group, service status, location, subscription name and ID, tags, URL, pricing tier, replicas, partitions, and search units. This role also has access to service metrics: search latency, percentage of throttled requests, average queries per second. </br></br>This role doesn't allow access to API keys, role assignments, content (indexes or synonym maps), or content metrics (storage consumed, number of objects). </br></br> (Preview) When you enable the RBAC preview for the data plane, the Reader role has read access across the entire service. This allows you to read search metrics, content metrics (storage consumed, number of objects), and the definitions of data plane resources (indexes, indexers, etc.). The Reader role still won't have access to read API keys or read content within indexes. |

-| [Search Service Contributor](../role-based-access-control/built-in-roles.md#search-service-contributor) | (Generally available) This role is identical to the Contributor role and applies to control plane operations. </br></br>(Preview) When you enable the RBAC preview for the data plane, this role also provides full access to all data plane actions on indexes, synonym maps, indexers, data sources, and skillsets as defined by [`Microsoft.Search/searchServices/*`](../role-based-access-control/resource-provider-operations.md#microsoftsearch). This role doesn't give you access to query search indexes or index documents. This role is for search service administrators who need to manage the search service and its objects, but without the ability to view or access object data. </br></br>Like Contributor, members of this role can't make or manage role assignments or change authorization options. To use the preview capabilities of this role, your service must have the preview feature enabled, as described in this article. |

-| [Search Index Data Contributor](../role-based-access-control/built-in-roles.md#search-index-data-contributor) | (Preview) Provides full data plane access to content in all indexes on the search service. This role is for developers or index owners who need to import, refresh, or query the documents collection of an index. |

-| [Search Index Data Reader](../role-based-access-control/built-in-roles.md#search-index-data-reader) | (Preview) Provides read-only data plane access to search indexes on the search service. This role is for apps and users who run queries. |

-<a name="preview-limitations"></a>

-## Preview capabilities and limitations

-+ Role-based access control for data plane operations, such as creating an index or querying an index, is currently in public preview and available under [supplemental terms of use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).

-+ There are no regional, tier, or pricing restrictions for using Azure RBAC preview, but your search service must be in the Azure public cloud. The preview isn't available in Azure Government, Azure Germany, or Azure China 21Vianet.

-+ If you migrate your Azure subscription to a new tenant, the Azure RBAC preview will need to be re-enabled.

-+ Role-based access control is supported in Azure portal and in the following search clients:

- + [Search REST APIs](/rest/api/searchservice/) (all supported versions)

- + [azure.search.documents (Azure SDK for .NET) version 11.4](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/search/Azure.Search.Documents/CHANGELOG.md)

- + [azure.search.documents (Azure SDK for Python) version 11.3](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/CHANGELOG.md)

- + [azure-search-documents (Azure SDK for Java) beta versions of 11.5 and 11.6](https://github.com/Azure/azure-sdk-for-jav),

- + [@azure/search-documents (Azure SDK for JavaScript), version 11.3 (see change log)](https://www.npmjs.com/package/@azure/search-documents?activeTab=explore).

- | Option | Status | Description |

- |--|--|-|

- | API Key | Generally available (default) | Requires an [admin or query API keys](search-security-api-keys.md) on the request header for authorization. No roles are used. |

- | Role-based access control | Preview | Requires membership in a role assignment to complete the task, described in the next step. It also requires an authorization header. |

- | Both | Preview | Requests are valid using either an API key or role-based access control. |

- + Search Service Contributor (preview for data plane requests)

- + Search Index Data Contributor (preview)

- + Search Index Data Reader (preview)

-| November | **Add search to websites** updated versions of React and Azure SDK client libraries: <ul><li>[C#](tutorial-csharp-overview.md)</li><li>[Python](tutorial-python-overview.md)</li><li>[JavaScript](tutorial-javascript-overview.md) </li></ul> "Add search to websites" is a tutorial series with sample code available in three languages. This series was . If you're integrating client code with a search index, these samples demonstrate an end-to-end approach to integration. |

-### Common troubleshooting steps

-Azure Files supports AES-256 Kerberos encryption for AD DS authentication beginning with the AzFilesHybrid module v0.2.2. AES-256 is the recommended authentication method. If you've enabled AD DS authentication with a module version lower than v0.2.2, you'll need to [download the latest AzFilesHybrid module](https://github.com/Azure-Samples/azure-files-samples/releases) and run the PowerShell below. If you haven't enabled AD DS authentication on your storage account yet, follow this [guidance](./storage-files-identity-ad-ds-enable.md#option-one-recommended-use-azfileshybrid-powershell-module) for enablement.

-# Use managed identities to access Event Hub from an Azure Stream Analytics job

-A managed identity is a managed application registered in Azure Active Directory that represents a given Stream Analytics job. The managed application is used to authenticate to a targeted resource, including Event Hubs that are behind a firewall or virtual network (VNet). For more information about how to bypass firewalls, see [Allow access to Azure Event Hubs namespaces via private endpoints](../event-hubs/private-link-service.md#trusted-microsoft-services).

-This article shows you how to enable Managed Identity for an Event Hubs input or output of a Stream Analytics job through the Azure portal. Before you enabled Managed Identity, you must first have a Stream Analytics job and Event Hub resource.

-## Grant the Stream Analytics job permissions to access the Event Hub

-For the Stream Analytics job to access your Event Hub using managed identity, the service principal you created must have special permissions to the Event Hub.

- | Setting | Value |

- | | |

- | Role | Azure Event Hubs Data Owner |

- | Assign access to | User, group, or service principal |

- | Members | \<Name of your Stream Analytics job> |

- 

-You can also grant this role at the Event Hub Namespace level, which will naturally propagate the permissions to all Event Hubs created under it. That is, all Event Hubs under a Namespace can be used as a managed-identity-authenticating resource in your Stream Analytics job.

-## Create an Event Hub input or output 

-Now that your managed identity is configured, you're ready to add the Event Hub resource as an input or output to your Stream Analytics job. 

-### Add the Event Hub as an input

-1. Select **Add Stream Input > Event Hub**. In the input properties window, search and select your Event Hub and select **Managed Identity** from the *Authentication mode* drop-down menu.

-### Add the Event Hub as an output

-1. Select **Add > Event Hub**. In the output properties window, search and select your Event Hub and select **Managed Identity** from the *Authentication mode* drop-down menu.

-You can use [Azure Database for PostgreSQL](https://azure.microsoft.com/services/postgresql/) as an output for data that is relational in nature or for applications that depend on the content being hosted in a relational database. Azure Stream Analytics jobs write to an existing table in PostgreSQL Database. The table schema must exactly match the fields and their types in your job's output.

-Azure Database for PostgreSQL powered by the PostgreSQL community edition is available in two deployment options:

-* Single Server

-* Flexible Server

-* [Quick start for Azure Database for PostgreSQL ΓÇô Single server](../postgresql/quickstart-create-server-database-portal.md)

-> Managed identities for Azure Database for PostgreSQL output in Azure Stream Analytics is currently not supported.

- - [Windows 64-bit](https://go.microsoft.com/fwlink/?linkid=2068602) *(most common)*

- - [Windows 32-bit](https://go.microsoft.com/fwlink/?linkid=2098960)

- - [Windows on Arm](https://go.microsoft.com/fwlink/?linkid=2098961)

-| Public | 1.2.4159 | [Windows 64-bit](https://go.microsoft.com/fwlink/?linkid=2139369) *(most common)*<br />[Windows 32-bit](https://go.microsoft.com/fwlink/?linkid=2139456)<br />[Windows ARM64](https://go.microsoft.com/fwlink/?linkid=2139370) |

-## Updates for version 1.2.4240 (Insider)

-*Date published: May 4, 2023*

-Download: [Windows 64-bit](https://go.microsoft.com/fwlink/?linkid=2139233), [Windows 32-bit](https://go.microsoft.com/fwlink/?linkid=2139144), [Windows ARM64](https://go.microsoft.com/fwlink/?linkid=2139368)

-Download: [Windows 64-bit](https://go.microsoft.com/fwlink/?linkid=2139369), [Windows 32-bit](https://go.microsoft.com/fwlink/?linkid=2139456), [Windows ARM64](https://go.microsoft.com/fwlink/?linkid=2139370)

-Download: [Windows 64-bit](https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RW10DEa), [Windows 32-bit](https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RW10GYu), [Windows ARM64](https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RW10GYw)

-| **AzureCloud** | All [datacenter public IP addresses](https://www.microsoft.com/download/details.aspx?id=56519). | Both | Yes | Yes |

->[!NOTE]

-> Hub Routing Policies are currently in Managed Preview.

->

->To obtain access to this preview, please reach out to previewinterhub@microsoft.com with the Virtual WAN ID, Subscription ID and Azure Region you wish to configure Routing Policies in. Please expect a response within 24-48 hours with confirmation of feature enablement.

->

-> For more information on how to configure Routing Intent and Policies please view the following [document](how-to-routing-policies.md).

-Customers using Azure Firewall manager to set up policies for public and private traffic now can set up their networks in a much simpler manner using Routing Intent and Routing Policies.

-Routing Intent and Routing policies allow you to specify how the Virtual WAN hub forwards Internet-bound and Private (Point-to-site, Site-to-site, ExpressRoute, Network Virtual Appliances inside the Virtual WAN Hub and Virtual Network) Traffic. There are two types of Routing Policies: Internet Traffic and Private Traffic Routing Policies. Each Virtual WAN Hub may have at most one Internet Traffic Routing Policy and one Private Traffic Routing Policy, each with a Next Hop resource.

-While Private Traffic includes both branch and Virtual Network address prefixes, Routing Policies considers them as one entity within the Routing Intent Concept.

-* **Private Traffic Routing Policy**: When a Private Traffic Routing Policy is configured on a Virtual WAN hub, **all** branch and Virtual Network traffic in and out of the Virtual WAN Hub including inter-hub traffic will be forwarded to the Next Hop Azure Firewall resource that was specified in the Private Traffic Routing Policy.

-Virtual hub **Reset** is available only in the Azure portal. Resetting provides you a way to bring any failed resources such as route tables, hub router, or the virtual hub resource itself back to its rightful provisioning state. Consider resetting the hub prior to contacting Microsoft for support. This operation doesn't reset any of the gateways in a virtual hub.

-* When using Azure Firewall in multiple regions, all spoke virtual networks must be associated to the same route table. For example, having a subset of the VNets going through the Azure Firewall while other VNets bypass the Azure Firewall in the same virtual hub isn't possible.

-Customers using Azure Firewall manager to set up policies for public and private traffic now can set up their networks in a much simpler manner using Routing Intent and Routing Policies.

->[!NOTE]

-> Hub Routing Intent is currently in gated public preview.

-> The preview for Hub Routing Intent impacts routing and route advertisements for **all** connections to the Virtual Hub (Point-to-site VPN, Site-to-site VPN, ExpressRoute, NVA, Virtual Network).

-Routing Intent and Routing policies allow you to specify how the Virtual WAN hub forwards Internet-bound and Private (Point-to-site, Site-to-site, ExpressRoute, Network Virtual Appliances inside the Virtual WAN Hub and Virtual Network) Traffic. There are two types of Routing Policies: Internet Traffic and Private Traffic Routing Policies. Each Virtual WAN Hub may have at most one Internet Traffic Routing Policy and one Private Traffic Routing Policy, each with a single Next Hop resource.

-While Private Traffic includes both branch and Virtual Network address prefixes, Routing Policies considers them as one entity within the Routing Intent Concepts.

->[!NOTE]

-> Inter-region traffic **cannot** be inspected by Azure Firewall or NVA. Additionally, configuring both private and internet routing policies is currently **not** supported in most Azure regions. Doing so will put Gateways (ExpressRoute, Site-to-site VPN and Point-to-site VPN) in a failed state and break connectivity from on-premises branches to Azure. Please ensure you only have one type of routing policy on each Virtual WAN hub. For more information, please contact previewinterhub@microsoft.com.

-* **Internet Traffic Routing Policy**: When an Internet Traffic Routing Policy is configured on a Virtual WAN hub, all branch (User VPN (Point-to-site VPN), Site-to-site VPN, and ExpressRoute) and Virtual Network connections to that Virtual WAN Hub will forward Internet-bound traffic to the Azure Firewall resource, Third-Party Security provider or **Network Virtual Appliance** specified as part of the Routing Policy.

- In other words, when Traffic Routing Policy is configured on a Virtual WAN hub, the Virtual WAN advertises a **default** route to all spokes, Gateways and Network Virtual Appliances (deployed in the hub or spoke). This includes the **Network Virtual Appliance** that is the next hop for the Internet Traffic routing policy.

-* **Private Traffic Routing Policy**: When a Private Traffic Routing Policy is configured on a Virtual WAN hub, **all** branch and Virtual Network traffic in and out of the Virtual WAN Hub including inter-hub traffic will be forwarded to the Next Hop Azure Firewall resource or Network Virtual Appliance resource that was specified in the Private Traffic Routing Policy.

- In other words, when a Private Traffic Routing Policy is configured on the Virtual WAN Hub, all branch-to-branch, branch-to-virtual network, virtual network-to-branch and inter-hub traffic will be sent via Azure Firewall or a Network Virtual Appliance deployed in the Virtual WAN Hub.

-## Preview notes

-This preview is provided without a service-level agreement and isn't recommended for production workloads. Some features might be unsupported or have constrained capabilities. For more information, see [Supplemental terms of use for Microsoft Azure previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).

-To obtain access to the preview, please deploy any Virtual WAN hubs and gateways (Site-to-site VPN Gateways, Point-to-site Gateways and ExpressRouteGateways) and then reach out to previewinterhub@microsoft.com with the Virtual WAN ID, Subscription ID and Azure Region you wish to configure Routing Intent in. Expect a response within 48 business hours (Monday-Friday) with confirmation of feature enablement. Please note that any gateways created after feature enablement will need to be upgraded by the Virtual WAN team.

-## Key considerations

-* You will **not** be able to enable routing policies on your deployments with existing Custom Route tables configured or if there are static routes configured in your Default Route Table.

-* Currently, Private Traffic Routing Policies aren't supported in Hubs with Encrypted ExpressRoute connections (Site-to-site VPN Tunnel running over ExpressRoute Private connectivity).

-* In the gated public preview of Virtual WAN Hub routing policies, inter-hub traffic between hubs in different Azure regions is dropped.

-* Routing Intent and Routing Policies currently must be configured via the custom portal link provided in Step 3 of **Prerequisites**. Routing Intents and Policies aren't supported via Terraform, PowerShell, and CLI.

-## Prerequisites

-1. Create a Virtual WAN. Make sure you create at least two Virtual Hubs if you wish to inspect inter-hub traffic. For instance, you may create a Virtual WAN with two Virtual Hubs in East US. If you only wish to inspect branch-to-branch traffic, you may deploy a single Virtual WAN Hub as opposed to multiple hubs.

-2. Convert your Virtual WAN Hub into a Secured Virtual WAN Hub by deploying an Azure Firewall into the Virtual Hubs in the chosen region. For more information on converting your Virtual WAN Hub to a Secured Virtual WAN Hub, see [How to secure your Virtual WAN Hub](howto-firewall.md).

-3. Deploy any Site-to-site VPN, Point-to-site VPN and ExpressRoute Gateways you'll use for testing. Reach out to **previewinterhub@microsoft.com** with the **Virtual WAN Resource ID** and the **Azure Virtual hub Region** you wish to configure Routing Policies in. To locate the Virtual WAN ID, open Azure portal, navigate to your Virtual WAN resource and select Settings > Properties > Resource ID. For example:

- ```

- /subscriptions/<subscriptionID>/resourceGroups/<resourceGroupName>/providers/Microsoft.Network/virtualWans/<virtualWANname>

- ```

-4. Expect a response within 24-48 hours with confirmation of feature enablement.

-5. Ensure that your Virtual Hubs do **not** have any Custom Route Tables or any routes you may have added into the defaultRouteTable. You will **not** be able to enable routing policies on your deployments with existing Custom Route tables configured or if there are static routes configured in your Default Route Table.

-6. If you're using an **Azure Firewall** deployed in the Virtual WAN Hub, see [Configuring routing policies (through Azure Firewall Manager)](#azurefirewall) to configure routing intent and routing policies. If you're using a **Network Virtual Appliance** deployed in the Virtual WAN Hub, see [Configuring routing policies (through Virtual WAN Portal)](#nva) to configure routing intent and routing policies.

-## <a name="azurefirewall"></a> Configure routing policies (through Azure Firewall Manager)

-1. From the custom portal Link provided in the confirmation email from Step 3 in the **Prerequisites** section, navigate to the Virtual WAN Hub that you want to configure Routing Policies on.

-1. Under Security, select **Secured Virtual hub settings** and then **Manage security provider and route settings for this Secured virtual hub in Azure Firewall Manager**

-1. Select the Hub you want to configure your Routing Policies on from the menu.

-1. Select **Security configuration** under **Settings**

-1. If you want to configure an Internet Traffic Routing Policy, select **Azure Firewall** or the relevant Internet Security provider from the dropdown for **Internet Traffic**. If not, select **None**

-1. If you want to configure a Private Traffic Routing Policy (for branch and Virtual Network traffic) via Azure Firewall, select **Azure Firewall** from the dropdown for **Private Traffic**. If not, select **Bypass Azure Firewall**.

- :::image type="content" source="./media/routing-policies/configure-intents.png"alt-text="Screenshot showing how to configure routing policies."lightbox="./media/routing-policies/configure-intents.png":::

-7. If you want to configure a Private Traffic Routing Policy and have branches or virtual networks advertising non-IANA RFC1918 Prefixes, select **Private Traffic Prefixes** and specify the non-IANA RFC1918 prefix ranges in the text box that comes up. Select **Done**.

- :::image type="content" source="./media/routing-policies/private-prefixes.png"alt-text="Screenshot showing how to edit private traffic prefixes."lightbox="./media/routing-policies/private-prefixes.png":::

-8. Select **Inter-hub** to be **Enabled**. Enabling this option ensures your Routing Policies are applied to the Routing Intent of this Virtual WAN Hub.

-9. Select **Save**. This operation takes around 10 minutes to complete.

-10. Repeat steps 2-8 for other Secured Virtual WAN hubs that you want to configure Routing policies for.

-11. At this point, you're ready to send test traffic. Make sure your Firewall Policies are configured appropriately to allow/deny traffic based on your desired security configurations.

-## <a name="nva"></a> Configure routing policies for network virtual appliances (through Virtual WAN portal)

->[!NOTE]

-> The only Network Virtual Appliance deployed in the Virtual WAN hub compatible with routing intent and routing policies are listed in the [Partners section](about-nva-hub.md) as dual-role connectivity and Next-Generation Firewall solution providers.

-1. From the custom portal link provided in the confirmation email from Step 3 in the **Prerequisites** section, navigate to the Virtual WAN hub that you want to configure routing policies on.

-1. Under Routing, select **Routing Policies**.

- :::image type="content" source="./media/routing-policies/routing-policies-vwan-ui.png"alt-text="Screenshot showing how to navigate to routing policies."lightbox="./media/routing-policies/routing-policies-vwan-ui.png":::

-3. If you want to configure a Private Traffic Routing Policy (for branch and Virtual Network Traffic), select **Network Virtual Appliance** under **Private Traffic** and under **Next Hop Resource** select the Network Virtual Appliance resource you wish to send traffic to.

- :::image type="content" source="./media/routing-policies/routing-policies-private-nva.png"alt-text="Screenshot showing how to configure NVA private routing policies."lightbox="./media/routing-policies/routing-policies-private-nva.png":::

-4. If you want to configure a Private Traffic Routing Policy and have branches or virtual networks using non-IANA RFC1918 Prefixes, select **Additional Prefixes** and specify the non-IANA RFC1918 prefix ranges in the text box that comes up. Select **Done**.

- > [!NOTE]

- > At this point in time, Routing Policies for **Network Virtual Appliances** do not allow you to edit the RFC1918 prefixes. Virtual WAN will propagate the RFC1918 aggregate prefixes to all spoke Virtual networks, Gateways as well as the **Network Virtual Appliances**. Be mindful of the implications about the propagation of these prefixes into your environment and create the appropriate policies inside your **Network Virtual Appliance** to control routing behavior.

- :::image type="content" source="./media/routing-policies/private-prefixes-nva.png"alt-text="Screenshot showing how to configure additional private prefixes for NVA routing policies."lightbox="./media/routing-policies/private-prefixes-nva.png":::

-5. If you want to configure an Internet Traffic Routing Policy, under **Internet traffic** select **Network Virtual Appliance** and under **Next Hop Resource** select the Network Virtual Appliance you want to send internet-bound traffic to.

- :::image type="content" source="./media/routing-policies/public-routing-policy-nva.png"alt-text="Screenshot showing how to configure public routing policies for NVA."lightbox="./media/routing-policies/public-routing-policy-nva.png":::

-6. To apply your routing intent and routing policies configuration, click **Save**.

- :::image type="content" source="./media/routing-policies/save-nva.png"alt-text="Screenshot showing how to save routing policies configurations"lightbox="./media/routing-policies/save-nva.png":::

-7. Repeat for all hubs you would like to configure routing policies for.

-8. At this point, you're ready to send test traffic. Make sure your Firewall Policies are configured appropriately to allow/deny traffic based on your desired security configurations.

-## Routing policy configuration examples

-The following section describes two common scenarios customers of applying Routing Policies to Secured Virtual WAN hubs.

-### All Virtual WAN Hubs are secured (deployed with Azure Firewall or NVA)

-In this scenario, all Virtual WAN hubs are deployed with an Azure Firewall or NVA in them. In this scenario, you may configure an Internet Traffic Routing Policy, a Private Traffic Routing Policy or both on each Virtual WAN Hub.

-Consider the following configuration where Hub 1 and Hub 2 have Routing Policies for both Private and Internet Traffic.

-**Hub 1 configuration:**

-* Private Traffic Policy with Next Hop Hub 1 Azure Firewall or NVA

-* Internet Traffic Policy with Next Hop Hub 1 Azure Firewall or NVA

-**Hub 2 configuration:**

-* Private Traffic Policy with Next Hop Hub 2 Azure Firewall or NVA

-* Internet Traffic Policy with Next Hop Hub 2 Azure Firewall or NVA

-The following are the traffic flows that result from such a configuration.

-> Internet Traffic must egress through the **local** Azure Firewall as the default route (0.0.0.0/0) does **not** propagate across hubs.

-| From | To | Hub 1 VNets | Hub 1 branches | Hub 2 VNets | Hub 2 branches| Internet|

-| -- | -- | - | | | | |

-| Hub 1 VNets | &#8594;| Hub 1 AzFW or NVA| Hub 1 AzFW or NVA | Hub 1 and 2 AzFW or NVA | Hub 1 and 2 AzFW or NVA | Hub 1 AzFW or NVA |

-| Hub 1 Branches | &#8594;| Hub 1 AzFW or NVA| Hub 1 AzFW or NVA | Hub 1 and 2 AzFW or NVA | Hub 1 and 2 AzFW or NVA | Hub 1 AzFW or NVA|

-| Hub 2 VNets | &#8594;| Hub 1 and 2 AzFW or NVA| Hub 1 and 2 AzFW or NVA | Hub 2 AzFW or NVA | Hub 2 AzFW or NVA| Hub 2 AzFW or NVA|

-| Hub 2 Branches | &#8594;| Hub 1 and 2 AzFW or NVA | Hub 1 and 2 AzFW or NVA | Hub 2 AzFW or NVA | Hub 2 AzFW or NVA | Hub 2 AzFW or NVA|

-### Mixture of secured and regular Virtual WAN Hubs

-In this scenario, not all Virtual WAN hubs are deployed with an Azure Firewall or Network Virtual Appliances in them. In this scenario, you may configure an Internet Traffic Routing Policy, a Private Traffic Routing Policy on the secured Virtual WAN Hubs.

-Consider the following configuration where Hub 1 (Normal) and Hub 2 (Secured) are deployed in a Virtual WAN. Hub 2 has Routing Policies for both Private and Internet Traffic.

-**Hub 1 Configuration:**

-* N/A (cannot configure Routing Policies if hub is not deployed with Azure Firewall or NVA)

-**Hub 2 Configuration:**

-* Private Traffic Policy with Next Hop Hub 2 Azure Firewall or NVA

-* Internet Traffic Policy with Next Hop Hub 2 Azure Firewall or NVA

- The following are the traffic flows that result from such a configuration. Branches and Virtual Networks connected to Hub 1 **cannot** access the Internet via Azure Firewall in the Hub because the default route (0.0.0.0/0) does **not** propagate across hubs.

-| From | To | Hub 1 VNets | Hub 1 branches | Hub 2 VNets | Hub 2 branches| Internet |

-| -- | -- | - | | | | |

-| Hub 1 VNets | &#8594;| Direct | Direct | Hub 2 AzFW or NVA| Hub 2 AzFW or NVA | - |

-| Hub 1 Branches | &#8594;| Direct | Direct | Hub 2 AzFW or NVA | Hub 2 AzFW or NVA | - |

-| Hub 2 VNets | &#8594;| Hub 2 AzFW or NVA| Hub 2 AzFW or NVA | Hub 2 AzFW or NVA| Hub 2 AzFW or NVA| Hub 2 AzFW or NVA|

-| Hub 2 Branches | &#8594;| Hub 2 AzFW or NVA | Hub 2 AzFW | Hub 2 AzFW or NVA| Hub 2 AzFW or NVA | Hub 2 AzFW or NVA|

-## Troubleshooting

-The following section describes common issues encountered when you configure Routing Policies on your Virtual WAN Hub. Read the below sections and if your issue is still unresolved, reach out to previewinterhub@microsoft.com for support. Expect a response within 48 business hours (Monday through Friday).

-### Troubleshooting configuration issues

-* Make sure that you have gotten confirmation from previewinterhub@microsoft.com that access to the gated public preview has been granted to your subscription and chosen region. You will **not** be able to configure routing policies without being granted access to the preview.

-* After enabling the Routing Policy feature on your deployment, ensure you **only** use the custom portal link provided as part of your confirmation email. Don't use PowerShell, CLI, or REST API calls to manage your Virtual WAN deployments. This includes creating new Branch (Site-to-site VPN, Point-to-site VPN or ExpressRoute) connections.

- >[!NOTE]

- > If you are using Terraform, routing policies are currently not supported.

-* Ensure that your Virtual Hubs don't have any Custom Route Tables or any static routes in the defaultRouteTable. You will **not** be able to select **Enable interhub** from Firewall Manager on your Virtual WAN Hub if there are Custom Route tables configured or if there are static routes in your defaultRouteTable.

-### Troubleshooting data path

-* Currently, using Azure Firewall to inspect inter-hub traffic is available for Virtual WAN hubs that are deployed in the **same** Azure Region.

-* Currently, Private Traffic Routing Policies aren't supported in Hubs with Encrypted ExpressRoute connections (Site-to-site VPN Tunnel running over ExpressRoute Private connectivity).

-* You can verify that the Routing Policies have been applied properly by checking the Effective Routes of the DefaultRouteTable. If Private Routing Policies are configured, you should see routes in the DefaultRouteTable for private traffic prefixes with next hop Azure Firewall. If Internet Traffic Routing Policies are configured, you should see a default (0.0.0.0/0) route in the DefaultRouteTable with next hop Azure Firewall.

-* If there are any Site-to-site VPN gateways or Point-to-site VPN gateways created **after** the feature has been confirmed to be enabled on your deployment, you'll have to reach out again to previewinterhub@microsoft.com to get the feature enabled.

-* If you're using Private Routing Policies with ExpressRoute, note that your ExpressRoute circuit can't advertise exact address ranges for the RFC1918 address ranges (you can't advertise 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16). Ensure you're advertising more specific subnets (within RFC1918 ranges) as opposed to aggregate supernets. Additionally, if your ExpressRoute circuit is advertising a non-RFC1918 prefix to Azure, please make sure the address ranges that you put in the Private Traffic Prefixes text box are less specific than ExpressRoute advertised routes. For example, if the ExpressRoute Circuit is advertising 40.0.0.0/24 from on-premises, put a /23 CIDR range or larger in the Private Traffic Prefix text box (example: 40.0.0.0/23).

-* Make sure you don't have both private and internet routing policies configured on a single Virtual WAN hub. Configuring both private and internet routing policies on the same hub is currently unsupported and will cause Point-to-site VPN, ExpressRoute and Site-to-site VPN gateways to go into a failed state and interrupt datapath connectivity to Azure.

-### Troubleshooting Azure Firewall

-* If you're using non [IANA RFC1918](https://datatracker.ietf.org/doc/html/rfc1918) prefixes in your branches/Virtual Networks, make sure you have specified those prefixes in the "Private Prefixes" text box in Firewall Manager.

-* If you have specified non RFC1918 addresses as part of the **Private Traffic Prefixes** text box in Firewall Manager, you may need to configure SNAT policies on your Firewall to disable SNAT for non-RFC1918 private traffic. For more information, reference the following [document](../firewall/snat-private-range.md).

-* Configure and view Azure Firewall logs to help troubleshoot and analyze your network traffic. For more information on how to set-up monitoring for Azure Firewall, reference the following [document](../firewall/firewall-diagnostics.md). An overview of the different types of Firewall logs can be found [here](../firewall/logs-and-metrics.md).

-* For more information on Azure Firewall, review [Azure Firewall Documentation](../firewall/overview.md).

-## Frequently asked questions

-### Why can't I edit the defaultRouteTable from the custom portal link provided by previewinterhub@microsoft.com?

-As part of the gated public preview of Routing Policies, your Virtual WAN hub routing is managed entirely by Firewall Manager. Additionally, the managed preview of Routing Policies is **not** supported alongside Custom Routing. Custom Routing with Routing Policies will be supported at a later date.

-However, you can still view the Effective Routes of the DefaultRouteTable by navigating to the **Effective Routes** Tab.

-If you have configured private traffic routing policies on your Virtual WAN hub, the Effective Route Table will only contain routes for RFC1918 supernets and any additional address prefixes that were specified in the Additional Private Traffic Prefixes text box.

-### Can I configure a Routing Policy for Private Traffic and also send Internet Traffic (0.0.0.0/0) via a Network Virtual Appliance in a Spoke Virtual Network?

-This scenario isn't supported in the gated public preview. However, reach out to previewinterhub@microsoft.com to express interest in implementing this scenario.

-### Does the default route (0.0.0.0/0) propagate across hubs?

-No. Currently, branches and Virtual Networks will egress to the internet using an Azure Firewall deployed inside of the Virtual WAN hub the branches and Virtual Networks are connected to. You can't configure a connection to access the Internet via the Firewall in a remote hub.

-### Why do I see RFC1918 aggregate prefixes advertised to my on-premises devices?

-When Private Traffic Routing Policies are configured, Virtual WAN Gateways will automatically advertise static routes that are in the default route table (RFC1918 prefixes: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16) in addition to the explicit branch and Virtual Network prefixes.

-### Why are my Gateways (Site-to-site VPN, Point-to-site VPN, ExpressRoute) in a failed state?

-There is currently a limitation where if Internet and private routing policies are configured concurrently on the same hub, Gateways go into a failed state, meaning your branches can't communicate with Azure. For more information on when this limitation will be lifted, contact previewinterhub@microsoft.com.

-|Managed preview|Routing intent and policies enabling inter-hub security|This feature allows you to configure internet-bound, private, or inter-hub traffic flow through the Azure Firewall. For more information, see [Routing intent and policies](how-to-routing-policies.md).|For access to the preview, contact previewinterhub@microsoft.com|Not compatible with NVA in a spoke, but compatible with BGP peering.<br><br>For additional limitations, see [How to configure Virtual WAN hub routing intent and routing policies](how-to-routing-policies.md#key-considerations).|

-|Managed preview|Checkpoint NGFW|Deployment of Checkpoint NGFW NVA into the Virtual WAN hub|DL-vwan-support-preview@checkpoint.com, previewinterhub@microsoft.com|Same limitations as routing intent.<br><br>Doesn't support internet inbound scenario.|

-|Managed preview|Fortinet NGFW/SD-WAN|Deployment of Fortinet dual-role SD-WAN/NGFW NVA into the Virtual WAN hub|azurevwan@fortinet.com, previewinterhub@microsoft.com|Same limitations as routing intent.<br><br>Doesn't support internet inbound scenario.|

-|1|Virtual hub upgrade to VMSS-based infrastructure: Compatibility with NVA in a hub.|For deployments with an NVA provisioned in the hub, the virtual hub router can't be upgraded to Virtual Machine Scale Sets.| July 2022|The Virtual WAN team is working on a fix that will allow Virtual hub routers to be upgraded to Virtual Machine Scale Sets, even if an NVA is provisioned in the hub. After upgrading, users will have to re-peer the NVA with the hub routerΓÇÖs new IP addresses (instead of having to delete the NVA).|