- Azure AD DS needs a network security group to secure the ports needed for the managed domain and block all other incoming traffic. This network security group acts as an extra layer of protection to lock down access to the managed domain. To view the ports required, see [Network security groups and required ports][network-ports].

- If you use secure LDAP, add a rule to the network security group to allow incoming traffic for *TCP* port *636*. For more information, see [Lock down secure LDAP access over the internet](tutorial-configure-ldaps.md#lock-down-secure-ldap-access-over-the-internet)

-[Authentication methods in Azure Active Directory](concept-authentication-authenticator-app.md)

-* **Code to report fraud during initial greeting**. When users receive a phone call to perform multi-factor authentication, they normally press **#** to confirm their sign-in. To report fraud, the user enters a code before pressing **#**. This code is **0** by default, but you can customize it.

-### Policy details

-If the information in the event isn't enough to understand the sign-in results, or adjust the policy to get desired results, the sign-in diagnostic tool can be used. The sign-in diagnostic can be found under **Basic info** > **Troubleshoot Event**. For more information about the sign-in diagnostic, see the article [What is the sign-in diagnostic in Azure AD](../reports-monitoring/overview-sign-in-diagnostics.md).

-### Conditional Access error codes

-To determine the service dependency, check the sign-ins log for the Application and Resource called by the sign-in. In the following screenshot, the application called is **Azure Portal** but the resource called is **Windows Azure Service Management API**. To target this scenario appropriately all the applications and resources should be similarly combined in Conditional Access policy.

-The following samples show public client mobile applications that access the Microsoft Graph API, or your own web API in the name of the user. These client applications use the Microsoft Authentication Library (MSAL).

-Real-time detections may not show up in reporting for five to 10 minutes. Offline detections may not show up in reporting for 48 hours.

-> Our system may detect that the risk event that contributed to the risk user risk score was a false positives or the user risk was remediated with policy enforcement such as completing multi-factor authentication or secure password change. Therefore our system will dismiss the risk state and a risk detail of ΓÇ£AI confirmed sign-in safeΓÇ¥ will surface and it will no longer contribute to the userΓÇÖs risk.

-Premium detections are visible only to Azure AD Premium P2 customers. Customers without Azure AD Premium P2 licenses still receives the premium detections but they'll be titled "additional risk detected".

-| Atypical travel | Offline | This risk detection type identifies two sign-ins originating from geographically distant locations, where at least one of the locations may also be atypical for the user, given past behavior. Among several other factors, this machine learning algorithm takes into account the time between the two sign-ins and the time it would have taken for the user to travel from the first location to the second, indicating that a different user is using the same credentials. <br><br> The algorithm ignores obvious "false positives" contributing to the impossible travel conditions, such as VPNs and locations regularly used by other users in the organization. The system has an initial learning period of the earliest of 14 days or 10 logins, during which it learns a new user's sign-in behavior. |

-| Anomalous Token | Offline | This detection indicates that there are abnormal characteristics in the token such as an unusual token lifetime or a token that is played from an unfamiliar location. This detection covers Session Tokens and Refresh Tokens. <br><br> **NOTE:** Anomalous token is tuned to incur more noise than other detections at the same risk level. This tradeoff is chosen to increase the likelihood of detecting replayed tokens that may otherwise go unnoticed. Because this is a high noise detection, there's a higher than normal chance that some of the sessions flagged by this detection are false positives. We recommend investigating the sessions flagged by this detection in the context of other sign-ins from the user. If the location, application, IP address, User Agent, or other characteristics are unexpected for the user, the tenant admin should consider this as an indicator of potential token replay. |

-| Malware linked IP address | Offline | This risk detection type indicates sign-ins from IP addresses infected with malware that is known to actively communicate with a bot server. This detection is determined by correlating IP addresses of the user's device against IP addresses that were in contact with a bot server while the bot server was active. <br><br> **[This detection has been deprecated](../fundamentals/whats-new-archive.md#planned-deprecationmalware-linked-ip-address-detection-in-identity-protection)**. Identity Protection will no longer generate new "Malware linked IP address" detections. Customers who currently have "Malware linked IP address" detections in their tenant will still be able to view, remediate, or dismiss them until the 90-day detection retention time is reached.|

-| Suspicious inbox manipulation rules | Offline | This detection is discovered by [Microsoft Defender for Cloud Apps](/cloud-app-security/anomaly-detection-policy#suspicious-inbox-manipulation-rules). This detection profiles your environment and triggers alerts when suspicious rules that delete or move messages or folders are set on a user's inbox. This detection may indicate that the user's account is compromised, that messages are being intentionally hidden, and that the mailbox is being used to distribute spam or malware in your organization. |

-| Impossible travel | Offline | This detection is discovered by [Microsoft Defender for Cloud Apps](/cloud-app-security/anomaly-detection-policy#impossible-travel). This detection identifies two user activities (is a single or multiple sessions) originating from geographically distant locations within a time period shorter than the time it would have taken the user to travel from the first location to the second, indicating that a different user is using the same credentials. |

-| Mass Access to Sensitive Files | Offline | This detection is discovered by [Microsoft Defender for Cloud Apps](/defender-cloud-apps/investigate-anomaly-alerts#unusual-file-access-by-user). This detection profiles your environment and triggers alerts when users access multiple files from Microsoft SharePoint or Microsoft OneDrive. An alert is triggered only if the number of accessed files is uncommon for the user and the files might contain sensitive information|

-| Anonymous IP address | Real-time | This risk detection type indicates sign-ins from an anonymous IP address (for example, Tor browser or anonymous VPN). These IP addresses are typically used by actors who want to hide their login telemetry (IP address, location, device, and so on) for potentially malicious intent. |

-| Azure AD threat intelligence | Offline | This risk detection type indicates user activity that is unusual for the given user or is consistent with known attack patterns based on Microsoft's internal and external threat intelligence sources. |

-| Anomalous user activity | Offline | This risk detection indicates that suspicious patterns of activity have been identified for an authenticated user. The post-authentication behavior for users is assessed for anomalies based on an action or sequence of actions occurring for the account, along with any sign-in risk detected. |

-| Azure AD threat intelligence | Offline | This risk detection type indicates user activity that is unusual for the given user or is consistent with known attack patterns based on Microsoft's internal and external threat intelligence sources. |

-While Microsoft doesn't provide specific details about how risk is calculated, we'll say that each level brings higher confidence that the user or sign-in is compromised. For example, something like one instance of unfamiliar sign-in properties for a user might not be as threatening as leaked credentials for another user.

-Disabled user accounts can be re-enabled. If the credentials of a disabled account are compromised, and the account gets re-enabled, bad actors might use those credentials to gain access. That is why, Identity Protection generates risk detections for suspicious activities against disabled user accounts to alert customers about potential account compromise. If an account is no longer in use and wont be re-enabled, customers should consider deleting it to prevent compromise. No risk detections are generated for deleted accounts.

-#### I have not seen any leaked credential risk events for quite some time?

-1. Login to [Ideagen Home](https://cktenant-homev2-scimtest1.ideagenhomedev.com). Click on the **Administration** icon to show the left hand side menu.

-2. Navigate to **Authentication** page under the **Manage tenant** sub menu.

-3. Scroll down in the Authentication page to **Client Token** section and click on **Regenerate**.

-4. **Copy** and save the Bearer Token. This value will be entered in the Secret Token * field in the Provisioning tab of your Ideagen Cloud application in the Azure portal.

-By using the Dapr extension to provision Dapr on your AKS or Arc-enabled Kubernetes cluster, you eliminate the overhead of downloading Dapr tooling and manually installing and managing the runtime on your AKS cluster. Additionally, the extension offers support for all [native Dapr configuration capabilities][dapr-configuration-options] through simple command-line arguments.

-Global Azure cloud is supported with Arc support on the regions listed by [Azure Products by Region][supported-cloud-regions].

-|Dynamics 365 | Yes | No | ^{[6](#tab1400b)} |

-<a name="filesSP">11</a> - Files in on-premises SharePoint are not supported.

-Connecting to on-premises data sources from an Azure Analysis Services server require an [On-premises gateway](analysis-services-gateway.md). When using a gateway, 64-bit providers are required.

-description: Learn how to use backup and restore to perform disaster recovery in Azure API Management.

-To recover from availability problems that affect the region that hosts your API Management service, be ready to reconstitute your service in another region at any time. Depending on your recovery time objective, you might want to keep a standby service in one or more regions. You might also try to maintain their configuration and content in sync with the active service according to your recovery point objective. The service backup and restore features provide the necessary building blocks for implementing disaster recovery strategy.

-This guide shows how to automate backup and restore operations and how to ensure successful authenticating of backup and restore requests by Azure Resource Manager.

-> [!IMPORTANT]

-> Restore operation doesn't change custom hostname configuration of the target service. We recommend to use the same custom hostname and TLS certificate for both active and standby services, so that, after restore operation completes, the traffic can be re-directed to the standby instance by a simple DNS CNAME change.

->

-> Backup operation does not capture pre-aggregated log data used in reports shown on the **Analytics** blade in the Azure portal.

-## Authenticating Azure Resource Manager requests

-> [!IMPORTANT]

-> The REST API for backup and restore uses Azure Resource Manager and has a different authentication mechanism than the REST APIs for managing your API Management entities. The steps in this section describe how to authenticate Azure Resource Manager requests. For more information, see [Authenticating Azure Resource Manager requests](/rest/api/azure).

-All of the tasks that you do on resources using the Azure Resource Manager must be authenticated with Azure Active Directory using the following steps:

-### Create an Azure Active Directory application

-1. Sign in to the [Azure portal](https://portal.azure.com).

-1. Using the subscription that contains your API Management service instance, navigate to the [Azure portal - App registrations](https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationsListBlade) to register an app in Active Directory.

- > [!NOTE]

- > If the Azure Active Directory default directory isn't visible to your account, contact the administrator of the Azure subscription to grant the required permissions to your account.

-1. Select **+ New registration**.

-1. On the **Register an application** page, set the values as follows:

-

- * Set **Name** to a meaningful name.

- * Set **Supported account types** to **Accounts in this organizational directory only**.

- * In **Redirect URI** enter a placeholder URL such as `https://resources`. It's a required field, but the value isn't used later.

- * Select **Register**.

-### Add permissions

-1. Once the application is created, select **API permissions** > **+ Add a permission**.

-1. Select **Microsoft APIs**.

-1. Select **Azure Service Management**.

- :::image type="content" source="./media/api-management-howto-disaster-recovery-backup-restore/add-app-permission.png" alt-text="Screenshot that shows how to add app permissions.":::

-1. Click **Delegated Permissions** beside the newly added application, and check the box for **Access Azure Service Management as organization users (preview)**.

- :::image type="content" source="./media/api-management-howto-disaster-recovery-backup-restore/delegated-app-permission.png" alt-text="Screenshot that shows adding delegated app permissions.":::

-1. Select **Add permissions**.

-### Configure your app

-Before calling the APIs that generate the backup and restore, you need to get a token. The following example uses the [Microsoft.IdentityModel.Clients.ActiveDirectory](https://www.nuget.org/packages/Microsoft.IdentityModel.Clients.ActiveDirectory) NuGet package to retrieve the token.

-> [!IMPORTANT]

-> The [Microsoft.IdentityModel.Clients.ActiveDirectory](https://www.nuget.org/packages/Microsoft.IdentityModel.Clients.ActiveDirectory) NuGet package and Azure AD Authentication Library (ADAL) have been deprecated. No new features have been added since June 30, 2020. We strongly encourage you to upgrade, see the [migration guide](../active-directory/develop/msal-migration.md) for more details.

-```csharp

-using Microsoft.IdentityModel.Clients.ActiveDirectory;

-using System;

-namespace GetTokenResourceManagerRequests

-{

- class Program

- {

- static void Main(string[] args)

- {

- var authenticationContext = new AuthenticationContext("https://login.microsoftonline.com/{tenant id}");

- var result = authenticationContext.AcquireTokenAsync("https://management.azure.com/", "{application id}", new Uri("{redirect uri}"), new PlatformParameters(PromptBehavior.Auto)).Result;

- if (result == null) {

- throw new InvalidOperationException("Failed to obtain the JWT token");

- }

- Console.WriteLine(result.AccessToken);

- Console.ReadLine();

- }

- }

-}

-Replace `{tenant id}`, `{application id}`, and `{redirect uri}` using the following instructions:

-1. Replace `{tenant id}` with the tenant ID of the Azure Active Directory application you created. You can access the ID by clicking **App registrations** -> **Endpoints**.

- ![Endpoints][api-management-endpoint]

-2. Replace `{application id}` with the value you get by navigating to the **Settings** page.

-3. Replace the `{redirect uri}` with the value from the **Redirect URIs** tab of your Azure Active Directory application.

- Once the values are specified, the code example should return a token similar to the following example:

- ![Token][api-management-arm-token]

- > [!NOTE]

- > The token may expire after a certain period. Execute the code sample again to generate a new token.

-## Accessing Azure Storage

-API Management uses an Azure Storage account that you specify for backup and restore operations. When running a backup or restore operation, you need to configure access to the storage account. API Management supports two storage access mechanisms: an Azure Storage access key (the default), or an API Management managed identity.

-### Configure storage account access key

-For steps, see [Manage storage account access keys](../storage/common/storage-account-keys-manage.md?tabs=azure-portal).

-### Configure API Management managed identity

-> [!NOTE]

-> Using an API Management managed identity for storage operations during backup and restore requires API Management REST API version `2021-04-01-preview` or later.

-1. Enable a system-assigned or user-assigned [managed identity for API Management](api-management-howto-use-managed-service-identity.md) in your API Management instance.

- * If you enable a user-assigned managed identity, take note of the identity's **Client ID**.

- * If you will back up and restore to different API Management instances, enable a managed identity in both the source and target instances.

-1. Assign the identity the **Storage Blob Data Contributor** role, scoped to the storage account used for backup and restore. To assign the role, use the [Azure portal](../active-directory/managed-identities-azure-resources/howto-assign-access-portal.md) or other Azure tools.

-## Calling the backup and restore operations

-The REST APIs are [Api Management Service - Backup](/rest/api/apimanagement/current-ga/api-management-service/backup) and [Api Management Service - Restore](/rest/api/apimanagement/current-ga/api-management-service/restore).

-> [!NOTE]

-> Backup and restore operations can also be performed with PowerShell [_Backup-AzApiManagement_](/powershell/module/az.apimanagement/backup-azapimanagement) and [_Restore-AzApiManagement_](/powershell/module/az.apimanagement/restore-azapimanagement) commands respectively.

-Before calling the "backup and restore" operations described in the following sections, set the authorization request header for your REST call.

-```csharp

-request.Headers.Add(HttpRequestHeader.Authorization, "Bearer " + token);

-```

-### <a name="step1"> </a>Back up an API Management service

-To back up an API Management service issue the following HTTP request:

-#### Access using storage access key

-#### Access using managed identity

-**Access using system-assigned managed identity**

-**Access using user-assigned managed identity**

-Backup is a long-running operation that may take more than a minute to complete. If the request succeeded and the backup process began, you receive a `202 Accepted` response status code with a `Location` header. Make `GET` requests to the URL in the `Location` header to find out the status of the operation. While the backup is in progress, you continue to receive a `202 Accepted` status code. A Response code of `200 OK` indicates successful completion of the backup operation.

-### <a name="step2"> </a>Restore an API Management service

-#### Access using storage access key

-#### Access using managed identity

-**Access using system-assigned managed identity**

-**Access using user-assigned managed identity**

-Restore is a long-running operation that may take up to 30 or more minutes to complete. If the request succeeded and the restore process began, you receive a `202 Accepted` response status code with a `Location` header. Make 'GET' requests to the URL in the `Location` header to find out the status of the operation. While the restore is in progress, you continue to receive a `202 Accepted` status code. A response code of `200 OK` indicates successful completion of the restore operation.

-> [!IMPORTANT]

-> **The SKU** of the service being restored into **must match** the SKU of the backed-up service being restored.

->

-> **Changes** made to the service configuration (for example, APIs, policies, developer portal appearance) while restore operation is in progress **could be overwritten**.

-## Constraints when making backup or restore request

-If the storage account is **[firewall][azure-storage-ip-firewall] enabled** and a storage key is used for access, then the customer must **Allow** the set of [Azure API Management control plane IP addresses][control-plane-ip-address] on their storage account for backup or restore to work. The storage account can be in any Azure region except the one where the API Management service is located. For example, if the API Management service is in West US, then the Azure Storage account can be in West US 2 and the customer needs to open the control plane IP 13.64.39.16 (API Management control plane IP of West US) in the firewall. This is because the requests to Azure Storage are not SNATed to a public IP from compute (Azure API Management control plane) in the same Azure region. Cross-region storage requests will be SNATed to the public IP address.

-The frequency with which you perform service backups affect your recovery point objective. To minimize it, we recommend implementing regular backups and performing on-demand backups after you make changes to your API Management service.

-API Management **Premium** tier also supports [zone redundancy](../availability-zones/migrate-api-mgt.md), which provides resiliency and high availability to a service instance in a specific Azure region (location).

-#cusomerintent: As an Azure service administrator, I want to move my service resources to another Azure region.

-1. [Back up](api-management-howto-disaster-recovery-backup-restore.md#-back-up-an-api-management-service) the existing API Management instance to the storage account.

-1. [Restore](api-management-howto-disaster-recovery-backup-restore.md#-restore-an-api-management-service) the source instance's backup to the new API Management instance.

-1. [Back up](api-management-howto-disaster-recovery-backup-restore.md#-back-up-an-api-management-service) the existing API Management instance to the storage account.

-1. [Restore](api-management-howto-disaster-recovery-backup-restore.md#-restore-an-api-management-service) the source instance's backup to the new API Management instance in the target region.

-This example creates a new Azure Cache for Redis Enterprise E10 cache instance called _Cache1_ in the East US region. Then, the cache is added to a new active geo-replication group called `replicationGroup`:

-`/subscriptions/\<your-subscription-ID>/resourceGroups/\<your-resource-group-name>/providers/Microsoft.Cache/redisEnterprise/\<your-cache-name>/databases/default`

-<!-- love the simple, declarative sentences. I am once again add the full product name -->

-This example creates a new Azure Cache for Redis Enterprise E10 cache instance called "Cache1" in the East US region. Then, the cache is added to a new active geo-replication group called `replicationGroup`:

-`id:"/subscriptions/\<your-subscription-ID>/resourceGroups/\<your-resource-group-name>/providers/Microsoft.Cache/redisEnterprise/\<your-cache-name>/databases/default`

- > By default when packaging for functions, the input is treated as text. If you are interested in consuming the raw bytes of the input (for instance for Blob triggers), you should use [AMLRequest to accept raw data](../machine-learning/how-to-deploy-advanced-entry-script.md#binary-data).

-1. If you don't already have a resource group or app service plan to deploy the service, the these commands demonstrate how to create both:

-*Last updated: February 2022*

-*Last updated: June 2022*

-description: Understanding the common alert schema, why you should use it and how to enable it

-The common alert schema standardizes the consumption experience for alert notifications in Azure today. Historically, the three alert types in Azure today (metric, log, and activity log) have had their own email templates, webhook schemas, etc. With the common alert schema, you can now receive alert notifications with a consistent schema.

-Any alert instance describes **the resource that was affected** and **the cause of the alert**, and these instances are described in the common schema in the following sections:

-The typical integration scenarios we hear from customers involve the routing of the alert instance to the concerned team based on some pivot (for example, resource group), after which the responsible team starts working on it. With the common alert schema, you can have standardized routing logic across alert types by leveraging the essential fields, leaving the context fields as is for the concerned teams to investigate further.

-This means that you can potentially have fewer integrations, making the process of managing and maintaining them a _much_ simpler task. Additionally, future alert payload enrichments (for example, customization, diagnostic enrichment, etc.) will only surface up in the common schema.

-The common alert schema will primarily manifest itself in your alert notifications. The enhancements that you will see are listed below:

-| Email | A consistent and detailed email template, allowing you to easily diagnose issues at a glance. Embedded deep-links to the alert instance on the portal and the affected resource ensure that you can quickly jump into the remediation process. |

-| Webhook/Logic App/Azure Function/Automation Runbook | A consistent JSON structure for all alert types, which allows you to easily build integrations across the different alert types. |

-[Learn more about the schema definitions for Webhooks/Logic Apps/Azure Functions/Automation Runbooks.](./alerts-common-schema-definitions.md)

-> The following actions do not support the common alert schema: ITSM Connector.

-You can opt in or opt out to the common alert schema through Action Groups, on both the portal and through the REST API. The toggle to switch to the new schema exists at an action level. For example, you have to separately opt in for an email action and a webhook action.

-> 1. The following alert types support the common schema by default (no opt-in required):

-> - Smart detection alerts

-> 1. The following alert types currently do not support the common schema:

-> - Alerts generated by [VM insights](../vm/vminsights-overview.md)

-

-1. Open any existing or a new action in an action group.

-1. Select ΓÇÿYesΓÇÖ for the toggle to enable the common alert schema as shown.

-You can also use the [Action Groups API](/rest/api/monitor/actiongroups) to opt in to the common alert schema. While making the [create or update](/rest/api/monitor/actiongroups/createorupdate) REST API call, you can set the flag "useCommonAlertSchema" to 'true' (to opt in) or 'false' (to opt out) for any of the following actions - email/webhook/logic app/Azure Function/automation runbook.

-For example, the following request body made to the [create or update](/rest/api/monitor/actiongroups/createorupdate) REST API will do the following:

-description: Understanding what alert processing rules in Azure Monitor are and how to configure and manage them.

-> The previous name for alert processing rules was **action rules**. The Azure resource type of these rules remains **Microsoft.AlertsManagement/actionRules** for backward compatibility.

-Alert processing rules allow you to apply processing on **fired alerts**. You may be familiar with Azure Monitor alert rules, which are rules that generate new alerts. Alert processing rules are different; they are rules that modify the fired alerts themselves as they are being fired. You can use alert processing rules to add [action groups](./action-groups.md) or remove (suppress) action groups from your fired alerts. Alert processing rules can be applied to different resource scopes, from a single resource to an entire subscription. They can also allow you to apply various filters or have the rule work on a pre-defined schedule.

-## What are alert processing rules useful for?

-Some common use cases for alert processing rules include:

-### Notification suppression during planned maintenance

-Many customers set up a planned maintenance time for their resources, either on a one-off basis or on a regular schedule. The planned maintenance may cover a single resource like a virtual machine, or multiple resources like all virtual machines in a resource group. So, you may want to stop receiving alert notifications for those resources during the maintenance window. In other cases, you may prefer to not receive alert notifications at all outside of your business hours. Alert processing rules allow you to achieve that.

-You could alternatively suppress alert notifications by disabling the alert rules themselves at the beginning of the maintenance window, and re-enabling them once the maintenance is over. In that case, the alerts won't fire in the first place. However, that approach has several limitations:

- * This approach is only practical if the scope of the alert rule is exactly the scope of the resources under maintenance. For example, a single alert rule might cover multiple resources, but only a few of those resources are going through maintenance. So, if you disable the alert rule, you will not be alerted when the remaining resources covered by that rule run into issues.

- * You may have many alert rules that cover the resource. Updating all of them is time consuming and error prone.

- * You might have some alerts that are not created by an alert rule at all, like alerts from Azure Backup.

-

-### Management at scale

-Most customers tend to define a few action groups that are used repeatedly in their alert rules. For example, they may want to call a specific action group whenever any high severity alert is fired. As their number of alert rule grows, manually making sure that each alert rule has the right set of action groups is becoming harder.

-Alert processing rules allow you to specify that logic in a single rule, instead of having to set it consistently in all your alert rules. They also cover alert types that are not generated by an alert rule.

-### Add action groups to all alert types

-> Alert processing rules do not affect [Azure Service Health](../../service-health/service-health-overview.md) alerts.

-## Alert processing rule properties

-An alert processing rule definition covers several aspects:

-### Which fired alerts are affected by this rule?

-**SCOPE**

-Each alert processing rule has a scope. A scope is a list of one or more specific Azure resources, or specific resource group, or an entire subscription. **The alert processing rule will apply to alerts that fired on resources within that scope**.

-**FILTERS**

-You can also define filters to narrow down which specific subset of alerts are affected within the scope. The available filters are:

-* **Alert Context (payload)** - the rule will apply only to alerts that contain any of the filter's strings within the [alert context](./alerts-common-schema-definitions.md#alert-context) section of the alert. This section includes fields specific to each alert type.

-* **Alert rule id** - the rule will apply only to alerts from a specific alert rule. The value should be the full resource ID, for example `/subscriptions/SUB1/resourceGroups/RG1/providers/microsoft.insights/metricalerts/MY-API-LATENCY`.

-You can locate the alert rule ID by opening a specific alert rule in the portal, clicking "Properties", and copying the "Resource ID" value. You can also locate it by listing your alert rules from PowerShell or CLI.

-* **Alert rule name** - the rule will apply only to alerts with this alert rule name. Can also be useful with a "Contains" operator.

-* **Description** - the rule will apply only to alerts that contain the specified string within the alert rule description field.

-* **Monitor condition** - the rule will apply only to alerts with the specified monitor condition, either "Fired" or "Resolved".

-* **Monitor service** - the rule will apply only to alerts from any of the specified monitor services.

-For example, use "Platform" to have the rule apply only to metric alerts.

-* **Resource** - the rule will apply only to alerts from the specified Azure resource.

-For example, you can use this filter with "Does not equal" to exclude one or more resources when the rule's scope is a subscription.

-* **Resource group** - the rule will apply only to alerts from the specified resource groups.

-For example, you can use this filter with "Does not equal" to exclude one or more resource groups when the rule's scope is a subscription.

-* **Resource type** - the rule will apply only to alerts on resource from the specified resource types, such as virtual machines. You can use "Equals" to match one or more specific resources, or you can use contains to match a resource type and all its child resources.

-For example, use `resource type contains "MICROSOFT.SQL/SERVERS"` to match both SQL servers and all their child resources, like databases.

-* **Severity** - the rule will apply only to alerts with the selected severities.

-**FILTERS BEHAVIOR**

-* If you define multiple filters in a rule, all of them apply - there is a logical AND between all filters.

- For example, if you set both `resource type = "Virtual Machines"` and `severity = "Sev0"`, then the rule will apply only for Sev0 alerts on virtual machines in the scope.

-* Each filter may include up to five values, and there is a logical OR between the values.

- For example, if you set `description contains ["this", "that"]`, then the rule will apply only to alerts whose description contains either "this" or "that".

-* **Suppression**

-This action removes all the action groups from the affected fired alerts. So, the fired alerts will not invoke any of their action groups (not even at the end of the maintenance window). Those fired alerts will still be visible when you list your alerts in the portal, Azure Resource Graph, API, PowerShell etc.

-The suppression action has a higher priority over the "apply action groups" action - if a single fired alert is affected by different alert processing rules of both types, the action groups of that alert will be suppressed.

-* **Apply action groups**

-This action adds one or more action groups to the affected fired alerts.

-You may optionally control when will the rule apply. By default, the rule is always active. However, you can select a one-off window for this rule to apply, or have a recurring window such as a weekly recurrence.

-## Configuring an alert processing rule

-You can access alert processing rules by navigating to the **Alerts** home page in Azure Monitor.

-Once there, you can click **Alert processing rules** to see and manage your existing rules, or click **Create** --> **Alert processing rules** to open the new alert processing rule wizard.

-Lets review the new alert processing rule wizard.

-In the first tab (**Scope**), you select which fired alerts are covered by this rule. Pick the **scope** of resources whose alerts will be covered - you may choose multiple resources and resource groups, or an entire subscription. You may also optionally add **filters**, as documented above.

-In the second tab (**Rule settings**), you select which action to apply on the affected alerts. Choose between **Suppression** or **Apply action group**. If you choose the apply action group, you can either select existing action groups by clicking **Add action groups**, or create a new action group.

-In the third tab (**Scheduling**), you select an optional schedule for the rule. By default the rule works all the time, unless you disable it. However, you can set it to work **on a specific time**, or **set up a recurring schedule**.

-Let's see an example of a schedule for a one-off, overnight, planned maintenance. It starts in the evening until the next morning, in a specific timezone:

-Let's see an example of a more complex schedule, covering an "outside of business hours" case. It has a recurring schedule with two recurrences - a daily one from the afternoon until the morning, and a weekly one covering Saturday and Sunday (full days).

-In the fourth tab (**Details**), you give this rule a name, pick where it will be stored, and optionally add a description for your reference. In the fifth tab (**Tags**), you optionally add tags to the rule, and finally in the last tab you can review and create the alert processing rule.

-You can use the Azure CLI to work with alert processing rules. See the `az monitor alert-processing-rules` [page in the Azure CLI docs](/cli/azure/monitor/alert-processing-rule) for detailed documentation and examples.

-1. **Install the Auzre CLI**

- Follow the [Installation instructions for the Azure CLI](/cli/azure/install-azure-cli).

- Alternatively, you can use Azure Cloud Shell, which is an interactive shell environment that you use through your browser. To start a Cloud Shell:

- - Open Cloud Shell by going to [https://shell.azure.com](https://shell.azure.com)

- - Select the **Cloud Shell** button on the menu bar at the upper right corner in the [Azure portal](https://portal.azure.com)

-1. **Sign in**

- If you're using a local installation of the CLI, sign in using the `az login` [command](/cli/azure/reference-index#az-login). Follow the steps displayed in your terminal to complete the authentication process.

-1. **Install the `alertsmanagement` extension**

- In order to use the `az monitor alert-processing-rule` commands, install the `alertsmanagement` preview extension.

- To learn more about Azure CLI extensions, check [Use extension with Azure CLI](/cli/azure/azure-cli-extensions-overview?).

-The [CLI documentation](/cli/azure/monitor/alert-processing-rule#az-monitor-alert-processing-rule-create) include more examples and an explanation of each parameter.

-You can use PowerShell to work with alert processing rules. See the `*-AzAlertProcessingRule` commands [in the PowerShell docs](/powershell/module/az.alertsmanagement) for detailed documentation and examples.

-Use the `Set-AzAlertProcessingRule` command to create alert processing rules.

-For example, to create a rule that adds an action group to all alerts in a subscription, run:

-The [PowerShell documentation](/cli/azure/monitor/alert-processing-rule#az-monitor-alert-processing-rule-create) include more examples and an explanation of each parameter.

-## Managing alert processing rules

-From here, you can enable, disable, or delete alert processing rules at scale by selecting the check box next to them. Clicking on an alert processing rule will open it for editing - you can enable or disable the rule in the fourth tab (**Details**).

-You can view and manage your alert processing rules using the [az monitor alert-processing-rules](/cli/azure/monitor/alert-processing-rule) commands from Azure CLI.

-Before you manage alert processing rules with the Azure CLI, prepare your environment using the instructions provided in [Configuring an alert processing rule](#configuring-an-alert-processing-rule).

-You can view and manage your alert processing rules using the [\*-AzAlertProcessingRule](/powershell/module/az.alertsmanagement) commands from Azure CLI.

-Before you manage alert processing rules with the Azure CLI, prepare your environment using the instructions provided in [Configuring an alert processing rule](#configuring-an-alert-processing-rule).

-```cshtml

- @inject Microsoft.ApplicationInsights.AspNetCore.JavaScriptSnippet JavaScriptSnippet

-```

-```cshtml

- @Html.Raw(JavaScriptSnippet.FullScript)

- </head>

-```

-> JavaScript injection provides a default configuration experience. If you require [configuration](./javascript.md#configuration) beyond setting the connection string, you are required to remove auto-injection as described above and manually add the [JavaScript SDK](./javascript.md#adding-the-javascript-sdk).

-* `RequestTrackingTelemetryModule` - Collects RequestTelemetry from incoming web requests

-* `DependencyTrackingTelemetryModule` - Collects [DependencyTelemetry](./asp-net-dependencies.md) from outgoing http calls and sql calls

-* `PerformanceCollectorModule` - Collects Windows PerformanceCounters

-* `QuickPulseTelemetryModule` - Collects telemetry for showing in Live Metrics portal

-* `AppServicesHeartbeatTelemetryModule` - Collects heart beats (which are sent as custom metrics), about Azure App Service environment where application is hosted

-* `AzureInstanceMetadataTelemetryModule` - Collects heart beats (which are sent as custom metrics), about Azure VM environment where application is hosted

-* `EventCounterCollectionModule` - Collects [EventCounters](eventcounters.md); this module is a new feature and is available in SDK version 2.8.0 and later

-The preceding code sample prevents the sending of telemetry to Application Insights. It doesn't prevent any automatic collection modules from collecting telemetry. If you want to remove a particular auto collection module, see [remove the telemetry module](#configuring-or-removing-default-telemetrymodules).

-By default, only `Warning` logs and more severe logs are automatically captured. To change this behavior, explicitly override the logging configuration for the provider `ApplicationInsights` as shown below.

-The following configuration allows ApplicationInsights to capture all `Information` logs and more severe logs.

-It's important to note that the following example doesn't cause the ApplicationInsights provider to capture `Information` logs. It doesn't capture it because the SDK adds a default logging filter that instructs `ApplicationInsights` to capture only `Warning` logs and more severe logs. ApplicationInsights requires an explicit override.

-* The SDK collects [Event Counters](./eventcounters.md) on Linux because [Performance Counters](./performance-counters.md) are only supported in Windows. Most metrics are the same.

-```csharp

-using Microsoft.ApplicationInsights.Channel;

-using Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel;

- public void ConfigureServices(IServiceCollection services)

- {

- // The following will configure the channel to use the given folder to temporarily

- // store telemetry items during network or Application Insights server issues.

- // User should ensure that the given folder already exists

- // and that the application has read/write permissions.

- services.AddSingleton(typeof(ITelemetryChannel),

- new ServerTelemetryChannel () {StorageFolder = "/tmp/myfolder"});

- services.AddApplicationInsightsTelemetry();

- }

-```

-This SDK requires `HttpContext`; therefore, it doesn't work in any non-HTTP applications, including the .NET Core 3.X Worker Service applications. To enable Application Insights in such applications using the newly released Microsoft.ApplicationInsights.WorkerService SDK, see [Application Insights for Worker Service applications (non-HTTP applications)](worker-service.md).

-* [Dependency Injection in ASP.NET Core](/aspnet/core/fundamentals/dependency-injection)

-### Azure PowerShell

-The `New-AzApplicationInsights` PowerShell command does not currently support creating a workspace-based Application Insights resource. To create a workspace-based resource with PowerShell, you can use the Azure Resource Manager templates below and deploy with PowerShell.

-description: Provides information about Microsoft's support for distributed tracing through our partnership in the OpenCensus project

-# What is Distributed Tracing?

-The advent of modern cloud and [microservices](https://azure.com/microservices) architectures has given rise to simple, independently deployable services that can help reduce costs while increasing availability and throughput. But while these movements have made individual services easier to understand as a whole, they've made overall systems more difficult to reason about and debug.

-In monolithic architectures, we've gotten used to debugging with call stacks. Call stacks are brilliant tools for showing the flow of execution (Method A called Method B, which called Method C), along with details and parameters about each of those calls. This is great for monoliths or services running on a single process, but how do we debug when the call is across a process boundary, not simply a reference on the local stack?

-That's where distributed tracing comes in.

-Distributed tracing is the equivalent of call stacks for modern cloud and microservices architectures, with the addition of a simplistic performance profiler thrown in. In Azure Monitor, we provide two experiences for consuming distributed trace data. The first is our [transaction diagnostics](./transaction-diagnostics.md) view, which is like a call stack with a time dimension added in. The transaction diagnostics view provides visibility into one single transaction/request, and is helpful for finding the root cause of reliability issues and performance bottlenecks on a per request basis.

-Azure Monitor also offers an [application map](./app-map.md) view which aggregates many transactions to show a topological view of how the systems interact, and what the average performance and error rates are.

-## How to Enable Distributed Tracing

-## Enabling via Application Insights through auto-instrumentation or SDKs

-The Application Insights agents and/or SDKs for .NET, .NET Core, Java, Node.js, and JavaScript all support distributed tracing natively. Instructions for installing and configuring each Application Insights SDK are available below:

-With the proper Application Insights SDK installed and configured, tracing information is automatically collected for popular frameworks, libraries, and technologies by SDK dependency auto-collectors. The full list of supported technologies is available in [the Dependency auto-collection documentation](./auto-collect-dependencies.md).

- Additionally, any technology can be tracked manually with a call to [TrackDependency](./api-custom-events-metrics.md) on the [TelemetryClient](./api-custom-events-metrics.md).

-Application Insights now supports distributed tracing through [OpenTelemetry](https://opentelemetry.io/). OpenTelemetry provides a vendor-neutral instrumentation to send traces, metrics, and logs to Application Insights. Initially the OpenTelemetry community took on Distributed Tracing. Metrics and Logs are still in progress. A complete observability story includes all three pillars, but currently our [Azure Monitor OpenTelemetry-based exporter preview offerings for .NET, Python, and JavaScript](opentelemetry-enable.md) only include Distributed Tracing. However, our Java OpenTelemetry-based Azure Monitor offering is GA and fully supported.

-The following pages consist of language-by-language guidance to enable and configure MicrosoftΓÇÖs OpenTelemetry-based offerings. Importantly, we share the available functionality and limitations of each offering so you can determine whether OpenTelemetry is right for your project.

-In addition to the Application Insights SDKs, Application Insights also supports distributed tracing through [OpenCensus](https://opencensus.io/). OpenCensus is an open source, vendor-agnostic, single distribution of libraries to provide metrics collection and distributed tracing for services. It also enables the open source community to enable distributed tracing with popular technologies like Redis, Memcached, or MongoDB. [Microsoft collaborates on OpenCensus with several other monitoring and cloud partners](https://open.microsoft.com/2018/06/13/microsoft-joins-the-opencensus-project/).

-[Python](opencensus-python.md)

-The OpenCensus website maintains API reference documentation for [Python](https://opencensus.io/api/python/trace/usage.html) and [Go](https://godoc.org/go.opencensus.io), as well as various different guides for using OpenCensus.

-description: Get page view and session counts, web client data, Single Page Applications (SPA), and track usage patterns. Detect exceptions and performance issues in JavaScript web pages.

-# Application Insights for web pages

-> We continue to assess the viability of OpenTelemetry for browser scenarios. The Application Insights JavaScript SDK is recommended for the forseeable future, which is fully compatible with OpenTelemetry distributed tracing.

-Find out about the performance and usage of your web page or app. If you add [Application Insights](app-insights-overview.md) to your page script, you get timings of page loads and AJAX calls, counts, and details of browser exceptions and AJAX failures, as well as users and session counts. All of this telemetry can be segmented by page, client OS and browser version, geo location, and other dimensions. You can set alerts on failure counts or slow page loading. And by inserting trace calls in your JavaScript code, you can track how the different features of your web page application are used.

-Application Insights can be used with any web pages - you just add a short piece of JavaScript, Node.js has a [standalone SDK](nodejs.md). If your web service is [Java](java-in-process-agent.md) or [ASP.NET](asp-net.md), you can use the server-side SDKs with the client-side JavaScript SDK to get an end-to-end understanding of your app's performance.

-## Adding the JavaScript SDK

-1. First you need an Application Insights resource. If you don't already have a resource and connection string, follow the [create a new resource instructions](create-new-resource.md).

-2. Copy the [connection string](#connection-string-setup) for the resource where you want your JavaScript telemetry to be sent (from step 1.) You'll add it to the `connectionString` setting of the Application Insights JavaScript SDK.

-3. Add the Application Insights JavaScript SDK to your web page or app via one of the following two options:

- * [npm Setup](#npm-based-setup)

- * [JavaScript Snippet](#snippet-based-setup)

-> `@microsoft/applicationinsights-web-basic - AISKULight` does not support the use of connection strings.

-> [!IMPORTANT]

-> Only use one method to add the JavaScript SDK to your application. If you use the NPM Setup, don't use the Snippet and vice versa.

-> NPM Setup installs the JavaScript SDK as a dependency to your project, enabling IntelliSense, whereas the Snippet fetches the SDK at runtime. Both support the same features. However, developers who desire more custom events and configuration generally opt for NPM Setup whereas users looking for quick enablement of out-of-the-box web analytics opt for the Snippet.

-### npm based setup

-Install via Node Package Manager (npm).

-> **Typings are included with this package**, so you do **not** need to install a separate typings package.

-### Snippet based setup

-If your app doesn't use npm, you can directly instrument your webpages with Application Insights by pasting this snippet at the top of each your pages. Preferably, it should be the first script in your `<head>` section so that it can monitor any potential issues with all of your dependencies and optionally any JavaScript errors. If you're using Blazor Server App, add the snippet at the top of the file `_Host.cshtml` in the `<head>` section.

-Starting from version 2.5.5, the page view event will include a new tag "ai.internal.snippet" that contains the identified snippet version. This feature assists with tracking which version of the snippet your application is using.

-The current Snippet (listed below) is version "5", the version is encoded in the snippet as sv:"#" and the [current version is also available on GitHub](https://go.microsoft.com/fwlink/?linkid=2156318).

-> For readability and to reduce possible JavaScript errors, all of the possible configuration options are listed on a new line in snippet code above, if you don't want to change the value of a commented line it can be removed.

-#### Reporting Script load failures

-This version of the snippet detects and reports failures when loading the SDK from the CDN as an exception to the Azure Monitor portal (under the failures &gt; exceptions &gt; browser). The exception provides visibility into failures of this type so that you're aware your application isn't reporting telemetry (or other exceptions) as expected. This signal is an important measurement in understanding that you have lost telemetry because the SDK didn't load or initialize which can lead to:

-For details on this exception see the [SDK load failure](javascript-sdk-load-failure.md) troubleshooting page.

-Reporting of this failure as an exception to the portal doesn't use the configuration option ```disableExceptionTracking``` from the application insights configuration and therefore if this failure occurs it will always be reported by the snippet, even when the window.onerror support is disabled.

-Reporting of SDK load failures isn't supported on Internet Explorer 8 or earlier. This behavior reduces the minified size of the snippet by assuming that most environments aren't exclusively IE 8 or less. If you have this requirement and you wish to receive these exceptions, you'll need to either include a fetch poly fill or create your own snippet version that uses ```XDomainRequest``` instead of ```XMLHttpRequest```, it's recommended that you use the [provided snippet source code](https://github.com/microsoft/ApplicationInsights-JS/blob/master/AISKU/snippet/snippet.js) as a starting point.

-> If you are using a previous version of the snippet, it is highly recommended that you update to the latest version so that you will receive these previously unreported issues.

-All configuration options have been moved towards the end of the script. This placement avoids accidentally introducing JavaScript errors that wouldn't just cause the SDK to fail to load, but also it would disable the reporting of the failure.

-Each configuration option is shown above on a new line, if you don't wish to override the default value of an item listed as [optional] you can remove that line to minimize the resulting size of your returned page.

-The available configuration options are

-

-| src | string **[required]** | The full URL for where to load the SDK from. This value is used for the "src" attribute of a dynamically added &lt;script /&gt; tag. You can use the public CDN location or your own privately hosted one.

-| name | string *[optional]* | The global name for the initialized SDK, defaults to `appInsights`. So ```window.appInsights``` will be a reference to the initialized instance. Note: if you provide a name value or a previous instance appears to be assigned (via the global name appInsightsSDK) then this name value will also be defined in the global namespace as ```window.appInsightsSDK=<name value>```. The SDK initialization code uses this reference to ensure it's initializing and updating the correct snippet skeleton and proxy methods.

-| ld | number in ms *[optional]* | Defines the load delay to wait before attempting to load the SDK. Default value is 0ms and any negative value will immediately add a script tag to the &lt;head&gt; region of the page, which will then block the page load event until to script is loaded (or fails).

-| useXhr | boolean *[optional]* | This setting is used only for reporting SDK load failures. Reporting will first attempt to use fetch() if available and then fallback to XHR, setting this value to true just bypasses the fetch check. Use of this value is only be required if your application is being used in an environment where fetch would fail to send the failure events.

-| crossOrigin | string *[optional]* | By including this setting, the script tag added to download the SDK will include the crossOrigin attribute with this string value. When not defined (the default) no crossOrigin attribute is added. Recommended values aren't defined (the default); ""; or "anonymous" (For all valid values see [HTML attribute: `crossorigin`](https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/crossorigin) documentation)

-| cfg | object **[required]** | The configuration passed to the Application Insights SDK during initialization.

-### Connection String Setup

-### Sending telemetry to the Azure portal

-By default, the Application Insights JavaScript SDK auto-collects many telemetry items that are helpful in determining the health of your application and the underlying user experience.

- - Stack trace

- - Exception details and message accompanying the error

- - Line & column number of error

- - URL where error was raised

- - Url of dependency source

- - Command & Method used to request the dependency

- - Duration of the request

- - Result code and success status of the request

- - ID (if any) of user making the request

- - Correlation context (if any) where request is made

-Telemetry initializers are used to modify the contents of collected telemetry before being sent from the user's browser. They can also be used to prevent certain telemetry from being sent, by returning `false`. Multiple telemetry initializers can be added to your Application Insights instance, and they're executed in order of adding them.

-The input argument to `addTelemetryInitializer` is a callback that takes a [`ITelemetryItem`](https://github.com/microsoft/ApplicationInsights-JS/blob/master/API-reference.md#addTelemetryInitializer) as an argument and returns a `boolean` or `void`. If returning `false`, the telemetry item isn't sent, else it proceeds to the next telemetry initializer, if any, or is sent to the telemetry collection endpoint.

-Most configuration fields are named such that they can be defaulted to false. All fields are optional except for `connectionString`.

-| connectionString | **Required**<br>Connection string that you obtained from the Azure portal. | string<br/>null |

-| accountId | An optional account ID, if your app groups users into accounts. No spaces, commas, semicolons, equals, or vertical bars | string<br/>null |

-| maxBatchSizeInBytes | Max size of telemetry batch. If a batch exceeds this limit, it's immediately sent and a new batch is started | numeric<br/>10000 |

-| maxBatchInterval | How long to batch telemetry for before sending (milliseconds) | numeric<br/>15000 |

-| enableDebug | If true, **internal** debugging data is thrown as an exception **instead** of being logged, regardless of SDK logging settings. Default is false. <br>***Note:*** Enabling this setting will result in dropped telemetry whenever an internal error occurs. This setting can be useful for quickly identifying issues with your configuration or usage of the SDK. If you don't want to lose telemetry while debugging, consider using `loggingLevelConsole` or `loggingLevelTelemetry` instead of `enableDebug`. | boolean<br/>false |

-| loggingLevelConsole | Logs **internal** Application Insights errors to console. <br>0: off, <br>1: Critical errors only, <br>2: Everything (errors & warnings) | numeric<br/> 0 |

-| loggingLevelTelemetry | Sends **internal** Application Insights errors as telemetry. <br>0: off, <br>1: Critical errors only, <br>2: Everything (errors & warnings) | numeric<br/> 1 |

-| diagnosticLogInterval | (internal) Polling interval (in ms) for internal logging queue | numeric<br/> 10000 |

-| samplingPercentage | Percentage of events that will be sent. Default is 100, meaning all events are sent. Set this option if you wish to preserve your data cap for large-scale applications. | numeric<br/>100 |

-| autoTrackPageVisitTime | If true, on a pageview, the _previous_ instrumented page's view time is tracked and sent as telemetry and a new timer is started for the current pageview. It's sent as a custom metric named `PageVisitTime` in `milliseconds` and is calculated via the Date [now()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/now) function (if available) and falls back to (new Date()).[getTime()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/getTime) if now() is unavailable (IE8 or less). Default is false. | boolean<br/>false |

-| overridePageViewDuration | If true, default behavior of trackPageView is changed to record end of page view duration interval when trackPageView is called. If false and no custom duration is provided to trackPageView, the page view performance is calculated using the navigation timing API. |boolean<br/>

-| maxAjaxCallsPerView | Default 500 - controls how many Ajax calls will be monitored per page view. Set to -1 to monitor all (unlimited) Ajax calls on the page. | numeric<br/> 500 |

-| disable&#8203;CorrelationHeaders | If false, the SDK will add two headers ('Request-Id' and 'Request-Context') to all dependency requests to correlate them with corresponding requests on the server side. | boolean<br/> false |

-| correlationHeader&#8203;ExcludedDomains | Disable correlation headers for specific domains | string[]<br/>undefined |

-| correlationHeader&#8203;ExcludePatterns | Disable correlation headers using regular expressions | regex[]<br/>undefined |

-| correlationHeader&#8203;Domains | Enable correlation headers for specific domains | string[]<br/>undefined |

-| disableFlush&#8203;OnBeforeUnload | If true, flush method won't be called when onBeforeUnload event triggers | boolean<br/> false |

-| enableSessionStorageBuffer | If true, the buffer with all unsent telemetry is stored in session storage. The buffer is restored on page load | boolean<br />true |

-| cookieCfg | Defaults to cookie usage enabled see [ICookieCfgConfig](#icookiemgrconfig) settings for full defaults. | [ICookieCfgConfig](#icookiemgrconfig)<br>(Since 2.6.0)<br/>undefined |

-| ~~isCookieUseDisabled~~<br>disableCookiesUsage | If true, the SDK won't store or read any data from cookies. Disables the User and Session cookies and renders the usage blades and experiences useless. isCookieUseDisable is deprecated in favor of disableCookiesUsage, when both are provided disableCookiesUsage takes precedence.<br>(Since v2.6.0) And if `cookieCfg.enabled` is also defined it will take precedence over these values, Cookie usage can be re-enabled after initialization via the core.getCookieMgr().setEnabled(true). | alias for [`cookieCfg.enabled`](#icookiemgrconfig)<br>false |

-| cookieDomain | Custom cookie domain. This option is helpful if you want to share Application Insights cookies across subdomains.<br>(Since v2.6.0) If `cookieCfg.domain` is defined it will take precedence over this value. | alias for [`cookieCfg.domain`](#icookiemgrconfig)<br>null |

-| cookiePath | Custom cookie path. This option is helpful if you want to share Application Insights cookies behind an application gateway.<br>If `cookieCfg.path` is defined it will take precedence over this value. | alias for [`cookieCfg.path`](#icookiemgrconfig)<br>(Since 2.6.0)<br/>null |

-| isRetryDisabled | If false, retry on 206 (partial success), 408 (timeout), 429 (too many requests), 500 (internal server error), 503 (service unavailable), and 0 (offline, only if detected) | boolean<br/>false |

-| isBeaconApiDisabled | If false, the SDK will send all telemetry using the [Beacon API](https://www.w3.org/TR/beacon) | boolean<br/>true |

-| onunloadDisableBeacon | When tab is closed, the SDK will send all remaining telemetry using the [Beacon API](https://www.w3.org/TR/beacon) | boolean<br/> false |

-| sdkExtension | Sets the sdk extension name. Only alphabetic characters are allowed. The extension name is added as a prefix to the 'ai.internal.sdkVersion' tag (for example, 'ext_javascript:2.0.0'). | string<br/> null |

-| isBrowserLink&#8203;TrackingEnabled | If true, the SDK will track all [Browser Link](/aspnet/core/client-side/using-browserlink) requests. | boolean<br/>false |

-| appId | AppId is used for the correlation between AJAX dependencies happening on the client-side with the server-side requests. When Beacon API is enabled, it canΓÇÖt be used automatically, but can be set manually in the configuration. |string<br/> null |

-| enable&#8203;CorsCorrelation | If true, the SDK will add two headers ('Request-Id' and 'Request-Context') to all CORS requests to correlate outgoing AJAX dependencies with corresponding requests on the server side. | boolean<br/>false |

-| enable&#8203;AutoRoute&#8203;Tracking | Automatically track route changes in Single Page Applications (SPA). If true, each route change will send a new Pageview to Application Insights. Hash route changes (`example.com/foo#bar`) are also recorded as new page views.| boolean<br/>false |

-| enableRequest&#8203;HeaderTracking | If true, AJAX & Fetch request headers is tracked. | boolean<br/> false |

-| enableResponse&#8203;HeaderTracking | If true, AJAX & Fetch request's response headers is tracked. | boolean<br/> false |

-| distributedTracingMode | Sets the distributed tracing mode. If AI_AND_W3C mode or W3C mode is set, W3C trace context headers (traceparent/tracestate) will be generated and included in all outgoing requests. AI_AND_W3C is provided for back-compatibility with any legacy Application Insights instrumented services. See example [here](./correlation.md#enable-w3c-distributed-tracing-support-for-web-apps).| `DistributedTracingModes`or<br/>numeric<br/>(Since v2.6.0) `DistributedTracingModes.AI_AND_W3C`<br />(v2.5.11 or earlier) `DistributedTracingModes.AI` |

-| maxAjaxPerf&#8203;LookupAttempts | The maximum number of times to look for the window.performance timings (if available). This option is sometimes required as not all browsers populate the window.performance before reporting the end of the XHR request and for fetch requests this is added after its complete.| numeric<br/> 3 |

-| ajaxPerfLookupDelay | The amount of time to wait before reattempting to find the window.performance timings for an `ajax` request, time is in milliseconds and is passed directly to setTimeout(). | numeric<br/> 25 ms |

-| enableUnhandled&#8203;PromiseRejection&#8203;Tracking | If true, unhandled promise rejections will be autocollected and reported as a JavaScript error. When disableExceptionTracking is true (don't track exceptions), the config value will be ignored and unhandled promise rejections won't be reported. | boolean<br/> false |

-| enablePerfMgr | When enabled (true) this will create local perfEvents for code that has been instrumented to emit perfEvents (via the doPerf() helper). This option can be used to identify performance issues within the SDK based on your usage or optionally within your own instrumented code. [More details are available by the basic documentation](https://github.com/microsoft/ApplicationInsights-JS/blob/master/docs/PerformanceMonitoring.md). Since v2.5.7 | boolean<br/>false |

-| perfEvtsSendAll | When _enablePerfMgr_ is enabled and the [IPerfManager](https://github.com/microsoft/ApplicationInsights-JS/blob/master/shared/AppInsightsCore/src/JavaScriptSDK.Interfaces/IPerfManager.ts) fires a [INotificationManager](https://github.com/microsoft/ApplicationInsights-JS/blob/master/shared/AppInsightsCore/src/JavaScriptSDK.Interfaces/INotificationManager.ts).perfEvent() this flag determines whether an event is fired (and sent to all listeners) for all events (true) or only for 'parent' events (false &lt;default&gt;).<br />A parent [IPerfEvent](https://github.com/microsoft/ApplicationInsights-JS/blob/master/shared/AppInsightsCore/src/JavaScriptSDK.Interfaces/IPerfEvent.ts) is an event where no other IPerfEvent is still running at the point of this event being created and its _parent_ property isn't null or undefined. Since v2.5.7 | boolean<br />false |

-| idLength | The default length used to generate new random session and user ID values. Defaults to 22, previous default value was 5 (v2.5.8 or less), if you need to keep the previous maximum length you should set this value to 5. | numeric<br />22 |

-## Cookie Handling

-The instance based cookie management also replaces the previous CoreUtils global functions of `disableCookies()`, `setCookie(...)`, `getCookie(...)` and `deleteCookie(...)`. And to benefit from the tree-shaking enhancements also introduced as part of version 2.6.0 you should no longer uses the global functions.

-Cookie Configuration for instance-based cookie management added in version 2.6.0.

-| Name | Description | Type and Default |

-| enabled | A boolean that indicates whether the use of cookies by the SDK is enabled by the current instance. If false, the instance of the SDK initialized by this configuration won't store or read any data from cookies | boolean<br/> true |

-| domain | Custom cookie domain, which is helpful if you want to share Application Insights cookies across subdomains. If not provided uses the value from root `cookieDomain` value. | string<br/>null |

-| path | Specifies the path to use for the cookie, if not provided it will use any value from the root `cookiePath` value. | string <br/> / |

-| getCookie | Function to fetch the named cookie value, if not provided it will use the internal cookie parsing / caching. | `(name: string) => string` <br/> null |

-| setCookie | Function to set the named cookie with the specified value, only called when adding or updating a cookie. | `(name: string, value: string) => void` <br/> null |

-| delCookie | Function to delete the named cookie with the specified value, separated from setCookie to avoid the need to parse the value to determine whether the cookie is being added or removed. If not provided it will use the internal cookie parsing / caching. | `(name: string, value: string) => void` <br/> null |

-### Simplified Usage of new instance Cookie Manager

-By setting `autoTrackPageVisitTime: true`, the time in milliseconds a user spends on each page is tracked. On each new PageView, the duration the user spent on the *previous* page is sent as a [custom metric](../essentials/metrics-custom-overview.md) named `PageVisitTime`. This custom metric is viewable in the [Metrics Explorer](../essentials/metrics-getting-started.md) as a "log-based metric".

-## Enable Distributed Tracing

-In JavaScript correlation is turned off by default in order to minimize the telemetry we send by default. The following examples show standard configuration options for enabling correlation.

-The following sample code shows the configurations required to enable correlation:

-> There are two distributed tracing modes/protocols - AI (Classic) and [W3C TraceContext](https://www.w3.org/TR/trace-context/) (New). In version 2.6.0 and later, they are _both_ enabled by default. For older versions, users need to [explicitly opt-in to WC3 mode](../app/correlation.md#enable-w3c-distributed-tracing-support-for-web-apps).

-By default, this SDK will **not** handle state-based route changing that occurs in single page applications. To enable automatic route change tracking for your single page application, you can add `enableAutoRouteTracking: true` to your setup configuration.

-### Single Page Applications

-For Single Page Applications, reference plugin documentation for plugin specific guidance.

-| Plugins |

-### Advanced Correlation

-When a page is first loading and the SDK hasn't fully initialized, we're unable to generate the Operation ID for the first request. As a result, distributed tracing is incomplete until the SDK fully initializes.

-To remedy this problem, you can include dynamic JavaScript on the returned HTML page. The SDK will use a callback function during initialization to retroactively pull the Operation ID from the `serverside` and populate the `clientside` with it.

-Here's a sample of how to create a dynamic JS using Razor:

-When using an npm based configuration, a location must be determined to store the Operation ID to enable access for the SDK initialization bundle to `appInsights.context.telemetryContext.parentID` so it can populate it before the first page view event is sent.

->The application UX is not yet optimized to show these "first hop" advanced distributed tracing scenarios. However, the data will be available in the requests table for query and diagnostics.

-Browser/client-side data can be viewed by going to **Metrics** and adding individual metrics you're interested in:

-

-You can also view your data from the JavaScript SDK via the Browser experience in the portal.

-Select **Browser** and then choose **Failures** or **Performance**.

-

-

-

-To query your telemetry collected by the JavaScript SDK, select the **View in Logs (Analytics)** button. By adding a `where` statement of `client_Type == "Browser"`, you'll only see data from the JavaScript SDK and any server-side telemetry collected by other SDKs will be excluded.

-

-### Source Map Support

-#### Link to Blob storage account

-You can link your Application Insights resource to your own Azure Blob Storage container to automatically unminify call stacks. To get started, see [automatic source map support](./source-map-support.md).

-1. Select an Exception Telemetry item in the Azure portal to view its "End-to-end transaction details"

-2. Identify which source maps correspond to this call stack. The source map must match a stack frame's source file, but suffixed with `.map`

-3. Drag and drop the source maps onto the call stack in the Azure portal

-

-### Application Insights Web Basic

-For a lightweight experience, you can instead install the basic version of Application Insights

-This version comes with the bare minimum number of features and functionalities and relies on you to build it up as you see fit. For example, it performs no autocollection (uncaught exceptions, AJAX, etc.). The APIs to send certain telemetry types, like `trackTrace`, `trackException`, etc., aren't included in this version, so you'll need to provide your own wrapper. The only API that is available is `track`. A [sample](https://github.com/Azure-Samples/applicationinsights-web-sample1/blob/master/testlightsku.html) is located here.

-For runnable examples, see [Application Insights JavaScript SDK Samples](https://github.com/Azure-Samples?q=applicationinsights-js-demo).

-## Upgrading from the old version of Application Insights

- - To manually refresh the current pageview ID (for example, in SPA apps), use `appInsights.properties.context.telemetryTrace.traceID = Microsoft.ApplicationInsights.Telemetry.Util.generateW3CId()`.

- > [!NOTE]

- > To keep the trace ID unique, where you previously used `Util.newId()`, now use `Util.generateW3CId()`. Both ultimately end up being the operation ID.

-Test in internal environment to verify monitoring telemetry is working as expected. If all works, update your API signatures appropriately to SDK V2 version and deploy in your production environments.

-At just 36 KB gzipped, and taking only ~15 ms to initialize, Application Insights adds a negligible amount of loadtime to your website. Minimal components of the library are quickly loaded when using this snippet. In the meantime, the full script is downloaded in the background.

-While the script is downloading from the CDN, all tracking of your page is queued. Once the downloaded script finishes asynchronously initializing, all events that were queued are tracked. As a result, you won't lose any telemetry during the entire life cycle of your page. This setup process provides your page with a seamless analytics system, invisible to your users.

-## ES3/IE8 Compatibility

-As such we need to ensure that this SDK continues to "work" and doesn't break the JS execution when loaded by an older browser. It would be ideal to not support older browsers, but numerous large customers canΓÇÖt control which browser their end users choose to use.

-This statement does NOT mean that we'll only support the lowest common set of features. We need to maintain ES3 code compatibility and when adding new features, they'll need to be added in a manner that wouldn't break ES3 JavaScript parsing and added as an optional feature.

-[See GitHub for full details on IE8 support](https://github.com/Microsoft/ApplicationInsights-JS#es3ie8-compatibility)

-The Application Insights JavaScript SDK is open-source to view the source code or to contribute to the project visit the [official GitHub repository](https://github.com/Microsoft/ApplicationInsights-JS).

-The `correlationHeaderExcludedDomains` configuration property is an exclude list that disables correlation headers for specific domains. This option is useful when including those headers would cause the request to fail or not be sent due to third-party server configuration. This property supports wildcards.

-An example would be `*.queue.core.windows.net`, as seen in the code sample above.

-Adding the application domain to this property should be avoided as it stops the SDK from including the required distributed tracing `Request-Id`, `Request-Context` and `traceparent` headers as part of the request.

-The server-side needs to be able to accept connections with those headers present. Depending on the `Access-Control-Allow-Headers` configuration on the server-side it's often necessary to extend the server-side list by manually adding `Request-Id`, `Request-Context` and `traceparent` (W3C distributed header).

-If the SDK reports correlation recursively, enable the configuration setting of `excludeRequestFromAutoTrackingPatterns` to exclude the duplicate data. This scenario can occur when using connection strings. The syntax for the configuration setting is `excludeRequestFromAutoTrackingPatterns: [<endpointUrl>]`.

-The name *URL ping test* is a bit of a misnomer. These tests don't use Internet Control Message Protocol (ICMP) to check your site's availability. Instead, they use more advanced HTTP request functionality to validate whether an endpoint is responding. They measure the performance associated with that response. They also add the ability to set custom success criteria, coupled with more advanced features like parsing dependent requests and allowing for retries.

-To create an availability test, you need use an existing Application Insights resource or [create an Application Insights resource](create-new-resource.md).

-

-1. In your Application Insights resource, open the **Availability** pane and selectΓÇ» **Add Classic Test**.

-1. Adjust the settings (described in the following table) to your needs and select **Create**.

- |Setting| Explanation |

- |**URL** | The URL can be any webpage that you want to test, but it must be visible from the public internet. The URL can include a query string. For example, you can exercise your database a little. If the URL resolves to a redirect, you can follow it up to 10 redirects.|

- |**Parse dependent requests**| The test requests images, scripts, style files, and other files that are part of the webpage under test. The recorded response time includes the time taken to get these files. The test fails if any of these resources can't be successfully downloaded within the timeout for the whole test. If the option is not enabled, the test only requests the file at the URL that you specified. Enabling this option results in a stricter check. The test might fail for cases that aren't noticeable from manually browsing through the site.

- |**Enable retries**|When the test fails, it's retried after a short interval. A failure is reported only if three successive attempts fail. Subsequent tests are then performed at the usual test frequency. Retry is temporarily suspended until the next success. This rule is applied independently at each test location. *We recommend this option*. On average, about 80 percent of failures disappear on retry.|

- |**Test frequency**| This setting determines how often the test is run from each test location. With a default frequency of five minutes and five test locations, your site is tested every minute on average.|

- |**Test locations**| The values for this setting are the places from which servers send web requests to your URL. *We recommend a minimum of five test locations*, to ensure that you can distinguish problems in your website from network issues. You can select up to 16 locations.

-|Setting| Explanation |

-| **Test timeout** |Decrease this value to be alerted about slow responses. The test is counted as a failure if the responses from your site have not been received within this period. If you selected **Parse dependent requests**, then all the images, style files, scripts, and other dependent resources must have been received within this period.|

-| **HTTP response** | The returned status code that's counted as a success. The code that indicates that a normal webpage has been returned is 200.|

-| **Content match** | We test that an exact case-sensitive match for a string occurs in every response. It must be a plain string, without wildcards (like "Welcome!"). Don't forget that if your page content changes, you might have to update it. *Content match supports only English characters.* |

-|Setting| Explanation |

-|**Near-realtime (Preview)** | We recommend using alerts that work in near real time. You configure this type of alert after you create your availability test. |

-|**Alert location threshold**| The optimal relationship between alert location threshold and the number of test locations is *alert location threshold = number of test locations - 2*, with a minimum of five test locations.|

-Autoscale is a service that allows you to automatically add and remove resources according to the load on your application.

- 

-When the conditions in the rules are met, one or more autoscale actions are triggered, adding or removing VMs. In addition, you can perform other actions like sending email notifications, or webhooks to trigger processes in other systems..

-* [Azure portal](autoscale-get-started.md)

-* [PowerShell](../powershell-samples.md#create-and-manage-autoscale-settings)

-* [Cross-platform Command Line Interface (CLI)](../cli-samples.md#autoscale)

-* [Azure Monitor REST API](/rest/api/monitor/autoscalesettings)

-Resources generate metrics that are used in autoscale rules to trigger scale events. Virtual machine scale sets use telemetry data from Azure diagnostics agents to generate metrics. Telemetry for Web apps and Cloud services comes directly from the Azure Infrastructure.

-Set up schedule-based rules to trigger scale events. Use schedule-based rules when you see time patterns in your load, and want to scale before an anticipated change in load occurs.

-

-* Metric-based

-Trigger based on a metric value, for example when CPU usage is above 50%.

-* Time-based

-Trigger based on a schedule, for example, every Saturday at 8am.

-* The OR operator is used when scaling out with multiple rules.

-* The AND operator is used when scaling in with multiple rules.

- + Start an [Azure Automation runbook](/azure/automation/overview).

- + Call an [Azure Function](/azure/azure-functions/functions-overview).

- + Trigger an [Azure Logic App](/azure/logic-apps/logic-apps-overview).

-| Rules | rules | A set of time or metric-based conditions that trigger a scale action. You can define one or more rules for both scale in and scale out actions. |

-* [Advanced Autoscale configuration using Resource Manager templates for virtual machine scale sets](autoscale-virtual-machine-scale-sets.md)

-* [Autoscale REST API](/rest/api/monitor/autoscalesettings)

-Autoscale scales horizontally, which is an increase, or decrease of the number of resource instances. For example, in a virtual machine scale set, scaling out means adding more virtual machines Scaling in means removing virtual machines. Horizontal scaling is flexible in a cloud situation as it allows you to run a large number of VMs to handle load.

-In contrast, vertical scaling, keeps the same number of resources constant, but gives them more capacity in terms of memory, CPU speed, disk space and network. Adding or removing capacity in vertical scaling is known as scaling or down. Vertical scaling is limited by the availability of larger hardware, which eventually reaches an upper limit. Hardware size availability varies in Azure by region. Vertical scaling may also require a restart of the virtual machine during the scaling process.

-| Web Apps |[Scaling Web Apps](autoscale-get-started.md) |

-| Cloud Services |[Autoscale a Cloud Service](../../cloud-services/cloud-services-how-to-scale-portal.md) |

-| Virtual Machines: Windows scale sets |[Scaling virtual machine scale sets in Windows](../../virtual-machine-scale-sets/tutorial-autoscale-powershell.md) |

-| Virtual Machines: Linux scale sets |[Scaling virtual machine scale sets in Linux](../../virtual-machine-scale-sets/tutorial-autoscale-cli.md) |

-| Virtual Machines: Windows Example |[Advanced Autoscale configuration using Resource Manager templates for virtual machine scale sets](autoscale-virtual-machine-scale-sets.md) |

-| Azure App Service |[Scale up an app in Azure App service](../../app-service/manage-scale-up.md)|

-| API Management service|[Automatically scale an Azure API Management instance](../../api-management/api-management-howto-autoscale.md)

-| Logic Apps |[Adding integration service environment (ISE) capacity](../../logic-apps/ise-manage-integration-service-environment.md#add-ise-capacity)|

-| Spring Cloud |[Set up autoscale for microservice applications](../../spring-apps/how-to-setup-autoscale.md)|

-| Service Bus |[Automatically update messaging units of an Azure Service Bus namespace](../../service-bus-messaging/automate-update-messaging-units.md)|

-| Logic Apps - Integration Service Environment(ISE) | [Add ISE Environment](../../logic-apps/ise-manage-integration-service-environment.md#add-ise-capacity) |

-| Azure App Service Environment | [Autoscaling and App Service Environment v1](../../app-service/environment/app-service-environment-auto-scale.md) |

-| Azure Stream Analytics | [Autoscale streaming units (Preview)](../../stream-analytics/stream-analytics-autoscale.md) |

-| Azure Machine Learning Workspace | [Autoscale an online endpoint](../../machine-learning/how-to-autoscale-endpoints.md) |

-* [Azure Monitor autoscale common metrics](autoscale-common-metrics.md)

-* [Scale virtual machine scale sets](/azure/virtual-machine-scale-sets/tutorial-autoscale-powershell?toc=/azure/azure-monitor/toc.json)

-* [Autoscale using Resource Manager templates for virtual machine scale sets](/azure/virtual-machine-scale-sets/tutorial-autoscale-powershell?toc=/azure/azure-monitor/toc.json)

-* [Best practices for Azure Monitor autoscale](autoscale-best-practices.md)

-* [Use autoscale actions to send email and webhook alert notifications](autoscale-webhook-email.md)

-* [Autoscale REST API](/rest/api/monitor/autoscalesettings)

-* [Troubleshooting virtual machine scale sets and autoscale](../../virtual-machine-scale-sets/virtual-machine-scale-sets-troubleshoot.md)

-* [Troubleshooting Azure Monitor autoscale](/azure/azure-monitor/autoscale/autoscale-troubleshoot)

-description: Shows how to create an Azure managed application that is intended for members of your organization.

-# Quickstart: Create and publish a managed application definition

-This quickstart provides an introduction to working with [Azure Managed Applications](overview.md). You can create and publish a managed application that's intended for members of your organization.

-To publish a managed application to your service catalog, you must:

-Every managed application definition includes a file named _mainTemplate.json_. In it, you define the Azure resources to deploy. The template is no different than a regular ARM template.

-Create a file named _mainTemplate.json_. The name is case-sensitive.

- "type": "string"

-Create a file named _createUiDefinition.json_ (This name is case-sensitive)

-Add the following starter JSON to the file and save it.

-Add the two files to a _.zip_ file named _app.zip_. The two files must be at the root level of the _.zip_ file. If you put them in a folder, you receive an error when creating the managed application definition that states the required files aren't present.

-Upload the package to an accessible location from where it can be consumed. You'll need to provide a unique name for the storage account.

- -Name "mystorageaccount" `

- --name mystorageaccount \

- --account-name mystorageaccount \

- --account-name mystorageaccount \

- --file "D:\myapplications\app.zip"

-Next, you need the role definition ID of the Azure built-in role you want to grant access to the user, user group, or application. Typically, you use the Owner or Contributor or Reader role. The following command shows how to get the role definition ID for the Owner role:

-If you don't already have a resource group for storing your managed application definition, create one now:

-Now, create the managed application definition resource.

-blob=$(az storage blob url --account-name mystorageaccount --container-name appcontainer --name app.zip --output tsv)

-As an alternative, you can choose to store your managed application definition within a storage account provided by you during creation so that its location and access can be fully managed by you for your regulatory needs.

-### Select your storage account

-You must [create a storage account](../../storage/common/storage-account-create.md) to contain your managed application definition for use with Service Catalog.

-Copy the storage account's resource ID. It will be used later when deploying the definition.

-### Set the role assignment for "Appliance Resource Provider" in your storage account

-For detailed steps, see [Assign Azure roles using the Azure portal](../../role-based-access-control/role-assignments-portal.md).

-Use the following ARM template to deploy your packaged managed application as a new managed application definition in Service Catalog whose definition files are stored and maintained in your own storage account:

- "description": "Managed Application name"

- }

- },

- "storageAccountType": {

- "type": "string",

- "defaultValue": "Standard_LRS",

- "allowedValues": [

- "Standard_LRS",

- "Standard_GRS",

- "Standard_ZRS",

- "Premium_LRS"

- ],

- "metadata": {

- "description": "Storage Account type"

- "description": "Storage account resource ID for where you're storing your definition"

- "_artifactsLocation": {

- "description": "The base URI where artifacts required by this template are located."

- "packageFileUri": "[parameters('_artifactsLocation')]",

- "defLocation": "[parameters('definitionStorageResourceID')]",

- "managedResourceGroupId": "[concat(subscription().id,'/resourceGroups/', concat(parameters('applicationName'),'_managed'))]",

- "applicationDefinitionResourceId": "[resourceId('Microsoft.Solutions/applicationDefinitions',variables('managedApplicationDefinitionName'))]"

- "apiVersion": "2020-08-21-preview",

-The `applicationDefinitions` properties include `storageAccountId` that contains the storage account ID for your storage account. You can verify that the application definition files are saved in your provided storage account in a container titled `applicationDefinitions`.

-> For added security, you can create a managed applications definition and store it in an [Azure storage account blob where encryption is enabled](../../storage/common/storage-service-encryption.md). The definition contents are encrypted through the storage account's encryption options. Only users with permissions to the file can see the definition in Service Catalog.

-> Please note that any resource, including VPN Gateways, associated with Public IP Standard SKU addresses are not currently able to move across subscriptions.

-> | managedInstances / databases | Yes | Yes |

-In this tutorial, you learn how to return a value from your Azure Resource Manager template (ARM template). You use outputs when you need a value from a deployed resource. This tutorial takes **7 minutes** to complete.

-You must have Visual Studio Code with the Resource Manager Tools extension, and either Azure PowerShell or Azure CLI. For more information, see [template tools](template-tutorial-create-first-template.md#get-tools).

-It deploys a storage account, but it doesn't return any information about the storage account. You might need to capture properties from a new resource so they're available later for reference.

-You can use outputs to return values from the template. For example, it might be helpful to get the endpoints for your new storage account.

-It uses the [reference](template-functions-resource.md#reference) function to get the runtime state of the storage account. To get the runtime state of a resource, you pass in the name or ID of a resource. In this case, you use the same variable you used to create the name of the storage account.

-To run this deployment command, you must have the [latest version](/cli/azure/install-azure-cli) of Azure CLI.

-In the output for the deployment command, you'll see an object similar to the following example only if the output is in JSON format:

-> If the deployment failed, use the `verbose` switch to get information about the resources being created. Use the `debug` switch to get more information for debugging.

-You've done a lot in the last six tutorials. Let's take a moment to review what you have done. You created a template with parameters that are easy to provide. The template is reusable in different environments because it allows for customization and dynamically creates needed values. It also returns information about the storage account that you could use in your script.

-If you're stopping now, you might want to clean up the resources you deployed by deleting the resource group.

-1. From the Azure portal, select **Resource group** from the left menu.

-2. Enter the resource group name in the **Filter by name** field.

-3. Select the resource group name.

-In this tutorial, you added a return value to the template. In the next tutorial, you'll learn how to export a template and use parts of that exported template in your template.

- > :::image type="content" source="./media/create-account-portal/avi-create-blade.png" alt-text="Screenshot showing how to create an Azure Video Indexer resource." lightbox="./media/create-account-portal/avi-create-blade.png":::

- 

- 

-1. [Create an Ubuntu 20.04 VM in Azure](/cli/azure/install-azure-cli-linux?pivots=apt).

-1. Install the following packages:

- - `pip3` version `pip-21.3.1.tar.gz`

- - `wheel` version 0.37.1

- - `jq` version 1.6

- - `ansible` version 2.9.27

- - `netaddr` version 0.8.0

- - `zip`

- - `netaddr` version 0.8.0

-1. [Create an Azure Storage account through the Azure portal](../storage/common/storage-account-create.md). Make sure to create the storage account in the same subscription as your SAP system infrastructure.

-1. Create a container within the Azure Storage account named `sapbits`.

- 1. On the storage account's sidebar menu, select **Containers** under **Data storage**.

- 1. Select **+ Container**.

- 1. On the **New container** pane, for **Name**, enter `sapbits`.

- 1. Select **Create**.

-1. Generate a shared access signature (SAS) token for the `sapbits` container.

- 1. In the Azure portal, open the Azure Storage account.

-

- 1. Open the `sapbits` container.

- 1. On the container's sidebar menu, select **Shared access signature** under **Security + networking**.

- 1. On the SAS page, under **Allowed resource types**, select **Container**.

- 1. Configure other settings as necessary.

- 1. Select **Generate SAS and connection string**.

- 1. Copy the **SAS token** value. Make sure to copy the `?` prefix with the token.

-

-Below are the common aliases that have been removed as well as their equivalent commands:

-## What's new in PowerShell Core 6

-For more information about what is new in PowerShell Core 6, reference the [PowerShell docs](/powershell/scripting/whats-new/what-s-new-in-powershell-70) and the [Getting Started with PowerShell Core](https://blogs.msdn.microsoft.com/powershell/2017/06/09/getting-started-with-powershell-core-on-windows-mac-and-linux/) blog post.

-You can request viseme output in SSML. For details, see [how to use viseme element in SSML](speech-synthesis-markup.md#viseme-element).

-|Supported Features | <ul><li>Text Translation</li><li>Document Translation</li><li>Custom Translation</li></ul>|

-Replace the `<your-custom-domain>` parameter with your [custom domain endpoint](document-translation/get-started-with-document-translation.md#what-is-the-custom-domain-endpoint).

-https://<your-custom-domain>.cognitiveservices.azure.us/

-|Supported Feature|<ul><li>[Text Translation](https://docs.azure.cn/cognitive-services/translator/reference/v3-0-reference)</li></ul>|

-### Example API translation request

-> [!div class="nextstepaction"]

-> [Azure China: Translator Text reference](https://docs.azure.cn/cognitive-services/translator/reference/v3-0-reference)

-## Next step

-Our vision is to empower developers and organizations to leverage AI to transform society in positive ways. We encourage responsible AI practices to protect the rights and safety of individuals. To achieve this, Microsoft has implemented a Limited Access policy grounded in our [AI Principles](https://www.microsoft.com/ai/responsible-ai) to support responsible deployment of Azure services.

-Limited Access services require registration, and only customers managed by Microsoft, meaning those who are working directly with Microsoft account teams, are eligible for access. The use of these services is limited to the use case selected at the time of registration. Customers must acknowledge that they have reviewed and agree to the terms of service. Microsoft may require customers to re-verify this information.

-Limited Access services are made available to customers under the terms governing their subscription to Microsoft Azure Services (including the [Service Specific Terms](https://go.microsoft.com/fwlink/?linkid=2018760)). Please review these terms carefully as they contain important conditions and obligations governing your use of Limited Access services.

-Features of these services that are not listed above are available without registration.

-### How do I apply for access?

-Please submit an intake form for each Limited Access service you would like to use:

-### How long will the application process take?

-Review may take 5-10 business days. You will receive an email as soon as your application is reviewed.

-Limited Access services are available only to customers managed by Microsoft. Additionally, Limited Access services are only available for certain use cases, and customers must select their intended use case in their application.

-Please use an email address affiliated with your organization in your application. Applications submitted with personal email addresses will be denied.

-If you are not a managed customer, we invite you to submit an application using the same forms and we will reach out to you about any opportunities to join an eligibility program.

-### What if I donΓÇÖt know whether IΓÇÖm a managed customer? What if I donΓÇÖt know my Microsoft contact or donΓÇÖt know if my organization has one?

-We invite you to submit an intake form for the features youΓÇÖd like to use, and weΓÇÖll verify your eligibility for access.

-### What happens if IΓÇÖm an existing customer and I donΓÇÖt apply?

-Existing customers have until June 30, 2023 to submit an intake form and be approved to continue using Limited Access services after June 30, 2023. We recommend allowing 10 business days for review. Without an approved application, you will be denied access after June 30, 2023.

-The intake forms can be found here:

-### IΓÇÖm an existing customer who applied for access to Custom Neural Voice or Speaker Recognition, do I have to apply to keep using these services?

-WeΓÇÖre always looking for opportunities to improve our Responsible AI program, and Limited Access is an update to our service gating processes. If you have previously applied for and been granted access to Custom Neural Voice or Speaker Recognition, we request that you submit a new intake form to continue using these services beyond June 30, 2023.

-If youΓÇÖre an existing customer using Custom Neural Voice or Speaker Recognition on June 21, 2022, you have until June 30, 2023 to submit an intake form with your selected use case and receive approval to continue using these services after June 30, 2023. We recommend allowing 10 days for application processing. Existing customers can continue using the service until June 30, 2023, after which they must be approved for access. The intake forms can be found here:

-### What if my use case is not on the intake form?

-Limited Access features are only available for the use cases listed on the intake forms. If your desired use case is not listed, please let us know in this [feedback form](https://aka.ms/CogSvcsLimitedAccessFeedback) so we can improve our service offerings.

-### What happens to my data if my application is denied?

-If you are an existing customer and your application for access is denied, you will no longer be able to use Limited Access features after June 30, 2023. Your data is subject to MicrosoftΓÇÖs data retention [policies](https://www.microsoft.com/trust-center/privacy/data-management#:~:text=If%20you%20terminate%20a%20cloud,data%20or%20renew%20your%20subscription.).

-* [How to call conversation summarization](./how-to/conversation-summarization.md)

-The following table shows supported client-side capabilities available in Azure Communication Services SDKs:

-| Capability | Supported |

-| | |

-| Send and receive chat messages | ✔️ |

-| Use typing indicators | ✔️ |

-| Read receipt | ❌ |

-| File sharing | ❌ |

-| Reply to chat message | ❌ |

-| React to chat message | ❌ |

-| Audio and video calling | ✔️ |

-| Share screen and see shared screen | ✔️ |

-| Manage Teams convenient recording | ❌ |

-| Manage Teams transcription | ❌ |

-| Receive closed captions | ❌ |

-| Add and remove meeting participants | ❌ |

-| Raise and lower hand | ❌ |

-| See raised and lowered hand | ❌ |

-| See and set reactions | ❌ |

-| Control Teams third-party applications | ❌ |

-| Interact with a poll or Q&A | ❌ |

-| Set and unset spotlight | ❌ |

-| See PowerPoint Live | ❌ |

-| See Whiteboard | ❌ |

-| Participation in breakout rooms | ❌ |

-| Apply background effects | ❌ |

-| See together mode video stream | ❌ |

-When Teams external users leave the meeting, or the meeting ends, they can no longer send or receive new chat messages and no longer have access to messages sent and received during the meeting.

-# Teams administrator controls

-The following list presents the set of features, which are currently available in the Azure Communication Services Calling SDK for JavaScript.

-| | Admit participant in the Lobby into the Teams meeting | ❌ |

-| | Suppport for early media | ❌ |

-| | Place a call on behalf of user | ❌ |

-| PSTN | Make an Emergency call | ❌ |

-description: Create and manage Dynamics 365 in workflows using Azure Logic Apps.

-# Connect to Dynamics 365 from workflows in Azure Logic Apps

-description: Connect to an FTP server from workflows in Azure Logic Apps.

-This article shows how to access your FTP server from a workflow in Azure Logic Apps with the FTP connector. You can then create automated workflows that run when triggered by events in your FTP server or in other systems and run actions to manage files on your FTP server.

- * Managed connector for Consumption and Standard workflows

-* FTP managed connector triggers might experience missing, incomplete, or delayed results when the "last modified" timestamp is preserved. On the other hand, the FTP *built-in* connector trigger in Standard logic app workflows doesn't have this limitation. For more information, review the FTP connector's [Limitations](/connectors/ftp/#limitations) section.

-description: Automate tasks and processes that monitor, create, manage, send, and receive files for an SFTP server using Azure Logic Apps.

-# Monitor, create, and manage SFTP files in Azure Logic Apps

-## Limits

-description: Automate tasks and workflows that manage global SMS, MMS, and IP messages through your Twilio account using Azure Logic Apps.

-tags: connectors

-# Connect to Twilio from Azure Logic Apps

-With Azure Logic Apps and the Twilio connector,

-you can create automated tasks and workflows

-that get, send, and list messages in Twilio,

-which include global SMS, MMS, and IP messages.

-You can use these actions to perform tasks with

-your Twilio account. You can also have other actions

-use the output from Twilio actions. For example,

-when a new message arrives, you can send the message

-content with the Slack connector. If you're new to logic apps,

-review [What is Azure Logic Apps?](../logic-apps/logic-apps-overview.md)

-## Prerequisites

-* An Azure account and subscription. If you don't have an Azure subscription,

-[sign up for a free Azure account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

-* From [Twilio](https://www.twilio.com/):

- * Your Twilio account ID and

- [authentication token](https://support.twilio.com/hc/en-us/articles/223136027-Auth-Tokens-and-How-to-Change-Them),

- which you can find on your Twilio dashboard

- Your credentials authorize your logic app to create a

- connection and access your Twilio account from your logic app.

- If you're using a Twilio trial account,

- you can send SMS only to *verified* phone numbers.

- * A verified Twilio phone number that can send SMS

- * A verified Twilio phone number that can receive SMS

-* Basic knowledge about

-[how to create logic apps](../logic-apps/quickstart-create-first-logic-app-workflow.md)

-* The logic app where you want to access your Twilio account.

-To use a Twilio action, start your logic app with another trigger,

-for example, the **Recurrence** trigger.

-## Connect to Twilio

-1. Sign in to the [Azure portal](https://portal.azure.com),

-and open your logic app in Logic App Designer, if not open already.

-1. Choose a path:

- * Under the last step where you want to add an action,

- choose **New step**.

- -or-

- * Between the steps where you want to add an action,

- move your pointer over the arrow between steps.

- Choose the plus sign (**+**) that appears,

- and then select **Add an action**.

-

- In the search box, enter "twilio" as your filter.

- Under the actions list, select the action you want.

-1. Provide the necessary details for your connection,

-and then choose **Create**:

- * The name to use for your connection

- * Your Twilio account ID

- * Your Twilio access (authentication) token

-1. Provide the necessary details for your selected action

-and continue building your logic app's workflow.

-## Connector reference

-For technical details about triggers, actions, and limits, which are

-described by the connector's OpenAPI (formerly Swagger) description,

-review the connector's [reference page](/connectors/twilio/).

-## Get support

-* For questions, visit the [Microsoft Q&A question page for Azure Logic Apps](/answers/topics/azure-logic-apps.html).

-* To submit or vote on feature ideas, visit the [Logic Apps user feedback site](https://aka.ms/logicapps-wish).

-## Next steps

-* Learn about other [Logic Apps connectors](../connectors/apis-list.md)

-description: How to migrate logic app workflow JSON definitions to the most recent Workflow Definition Language schema version

-# Migrate logic apps to latest schema version

-To move your existing logic apps to the newest schema,

-follow these steps:

-1. In the [Azure portal](https://portal.azure.com),

-open your logic app in the Logic App Designer.

-2. On your logic app's menu, choose **Overview**.

-On the toolbar, choose **Update Schema**.

- > [!NOTE]

- > When you choose **Update Schema**, Azure Logic Apps

- > automatically runs the migration steps and provides

- > the code output for you. You can use this output for

- > updating your logic app definition. However, make

- > sure you follow best practices as described in the

- > following **Best practices** section.

- 

- The Update Schema page appears and shows

- a link to a document that describes the

- improvements in the new schema.

-## Best practices

-Here are some best practices for migrating your

-logic apps to the latest schema version:

-* Copy the migrated script to a new logic app.

-Don't overwrite the old version until you complete

-your testing and confirm that your migrated app works as expected.

-* Test your logic app **before** putting in production.

-* After you finish migration, start updating your logic

-apps to use the [managed APIs](../connectors/apis-list.md)

-where possible. For example, start using Dropbox v2

-everywhere that you use DropBox v1.

-## Next steps

-* Learn how to [manually migrate your Logic apps](../logic-apps/logic-apps-schema-2016-04-01.md)

-description: Automate tasks that monitor, create, manage, send, and receive files for an SFTP server by using SSH and Azure Logic Apps.

-# Create and manage SFTP files using SSH and Azure Logic Apps

-## Limits

-With hierarchical partition keys, we can partition first on **TenantId**, and then **UserId**. We can even partition further down to another level, such as **SessionId**, as long as the overall depth doesn't exceed three levels. When a physical partition exceeds 50 GB of storage, Cosmos DB will automatically split the physical partition so that roughly half of the data on the will be on one physical partition, and half on the other. Effectively, subpartitioning means that a single TenantId can exceed 20 GB of data, and it's possible for a TenantId's data to span multiple physical partitions.

-| blockSizeInMB | Specify the block size in MB used to write data to ADLS Gen2. Learn more [about Block Blobs](/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs#about-block-blobs). <br/>Allowed value is **between 4 MB and 100 MB**. <br/>By default, ADF automatically determines the block size based on your source store type and data. For non-binary copy into ADLS Gen2, the default block size is 100 MB so as to fit in at most 4.95-TB data. It may be not optimal when your data is not large, especially when you use Self-hosted Integration Runtime with poor network resulting in operation timeout or performance issue. You can explicitly specify a block size, while ensure blockSizeInMB*50000 is big enough to store the data, otherwise copy activity run will fail. | No |

- "build":"node node_modules/@microsoft/azure-data-factory-utilities/lib/index"

- "@microsoft/azure-data-factory-utilities":"^0.1.5"

->If you do not use latest versions of PowerShell and Data Factory module, you may run into deserialization errors while running the commands.

->

-

- 1. Select **Add role assignment**.

- 1. On the **Add role assignment** page, for **Role**, select **Classic Virtual Machine Contributor**. In the **Select** box, paste **ddbf3205-c6bd-46ae-8127-60eb93363864**, and then select **MicrosoftAzureBatch** from the list of search results.

- :::image type="content" source="media/join-azure-ssis-integration-runtime-virtual-network/azure-batch-to-vm-contributor.png" alt-text="Search results on the &quot;Add role assignment&quot; page":::

- 1. Select **Save** to save the settings and close the page.

- :::image type="content" source="media/join-azure-ssis-integration-runtime-virtual-network/save-access-settings.png" alt-text="Save access settings":::

- 1. Confirm that you see **MicrosoftAzureBatch** in the list of contributors.

-1. Make sure that *Microsoft.Batch* is a registered resource provider in Azure subscription that has the virtual network for your Azure-SSIS IR to join. For detailed instructions, see the [Register Azure Batch as a resource provider](azure-ssis-integration-runtime-virtual-network-configuration.md#registerbatch) section.

-1. In [Azure portal](https://portal.azure.com), on the left-hand-side menu, select **Data factories**. If you don't see **Data factories** on the menu, select **More services**, and then in the **INTELLIGENCE + ANALYTICS** section, select **Data factories**.

-1. Select your ADF with Azure-SSIS IR in the list. You see the home page for your ADF. Select the **Author & Monitor** tile. You see ADF UI on a separate tab.

-1. In ADF UI, switch to the **Edit** tab, select **Connections**, and switch to the **Integration Runtimes** tab.

- :::image type="content" source="media/join-azure-ssis-integration-runtime-virtual-network/integration-runtimes-tab.png" alt-text="&quot;Integration runtimes&quot; tab":::

-1. If your Azure-SSIS IR is running, in the **Integration Runtimes** list, in the **Actions** column, select the **Stop** button for your Azure-SSIS IR. You can't edit your Azure-SSIS IR until you stop it.

-1. In the **Integration Runtimes** list, in the **Actions** column, select the **Edit** button for your Azure-SSIS IR.

- :::image type="content" source="media/join-azure-ssis-integration-runtime-virtual-network/integration-runtime-edit.png" alt-text="Edit the integration runtime":::

-1. On the **Integration runtime setup** pane, advance through the **General settings** and **Deployment settings** pages by selecting the **Next** button.

-## Hosting SSISDB in Azure SQL Database server or Managed Instance

-1. Click on your data factory resource to open up its resource blade.

-1. Click on **Author and Monitor** to open up the ADF UX. The ADF UX can also be accessed at adf.azure.com.

- :::image type="content" source="media/lab-data-flow-data-share/portal3.png" alt-text="Portal 3":::

-1. You'll be redirected to the homepage of the ADF UX. This page contains quick-starts, instructional videos, and links to tutorials to learn data factory concepts. To start authoring, click on the pencil icon in left side-bar.

-1. The first linked service you'll configure is an Azure SQL DB. You can use the search bar to filter the data store list. Click on the **Azure SQL Database** tile and click continue.

-1. In the SQL DB configuration pane, enter 'SQLDB' as your linked service name. Enter in your credentials to allow data factory to connect to your database. If you're using SQL authentication, enter in the server name, the database, your user name and password. You can verify your connection information is correct by clicking **Test connection**. Click **Create** when finished.

-1. Repeat the same process to add an Azure Synapse Analytics linked service. In the connections tab, click **New**. Select the **Azure Synapse Analytics** tile and click continue.

-1. In the linked service configuration pane, enter 'SQLDW' as your linked service name. Enter in your credentials to allow data factory to connect to your database. If you're using SQL authentication, enter in the server name, the database, your user name and password. You can verify your connection information is correct by clicking **Test connection**. Click **Create** when finished.

-1. The last linked service needed for this lab is an Azure Data Lake Storage gen2. In the connections tab, click **New**. Select the **Azure Data Lake Storage Gen2** tile and click continue.

-1. In the linked service configuration pane, enter 'ADLSGen2' as your linked service name. If you're using Account key authentication, select your ADLS Gen2 storage account from the **Storage account name** dropdown. You can verify your connection information is correct by clicking **Test connection**. Click **Create** when finished.

-To turn on debug, click the **Data flow debug** slider in the top bar of data flow canvas or pipeline canvas when you have **Data flow** activities. Click **OK** when the confirmation dialog is shown. The cluster will start up in about 5 to 7 minutes. Continue on to *Ingest data from Azure SQL DB into ADLS Gen2 using the copy activity* while it is initializing.

-1. In the factory resources pane, click on the plus icon to open the new resource menu. Select **Pipeline**.

-1. Click on the **Source** tab of the copy activity. To create a new dataset, click **New**. Your source will be the table 'dbo.TripData' located in the linked service 'SQLDB' configured earlier.

-1. Search for **Azure SQL Database** and click continue.

-1. Call your dataset 'TripData'. Select 'SQLDB' as your linked service. Select table name 'dbo.TripData' from the table name dropdown. Import the schema **From connection/store**. Click OK when finished.

-1. Click on the **Sink** tab of the copy activity. To create a new dataset, click **New**.

-1. Search for **Azure Data Lake Storage Gen2** and click continue.

-1. In the select format pane, select **DelimitedText** as you're writing to a csv file. Click continue.

-1. Name your sink dataset 'TripDataCSV'. Select 'ADLSGen2' as your linked service. Enter where you want to write your csv file. For example, you can write your data to file `trip-data.csv` in container `staging-container`. Set **First row as header** to true as you want your output data to have headers. Since no file exists in the destination yet, set **Import schema** to **None**. Click OK when finished.

-1. To verify your copy activity is working correctly, click **Debug** at the top of the pipeline canvas to execute a debug run. A debug run allows you to test your pipeline either end-to-end or until a breakpoint before publishing it to the data factory service.

-1. To monitor your debug run, go to the **Output** tab of the pipeline canvas. The monitoring screen will autorefresh every 20 seconds or when you manually click the refresh button. The copy activity has a special monitoring view, which can be access by clicking the eye-glasses icon in the **Actions** column.

-1. In the side pane that opens, select **Create new data flow** and choose **Mapping data flow**. Click **OK**.

-1. The first thing you want to do is configure your two source transformations. The first source will point to the 'TripDataCSV' DelimitedText dataset. To add a source transformation, click on the **Add Source** box in the canvas.

-1. Name your source 'TripDataCSV' and select the 'TripDataCSV' dataset from the source drop-down. If you remember, you didn't import a schema initially when creating this dataset as there was no data there. Since `trip-data.csv` exists now, click **Edit** to go to the dataset settings tab.

-1. Go to tab **Schema** and click **Import schema**. Select **From connection/store** to import directly from the file store. 14 columns of type string should appear.

-1. Go back to data flow 'JoinAndAggregateData'. If your debug cluster has started (indicated by a green circle next to the debug slider), you can get a snapshot of the data in the **Data Preview** tab. Click **Refresh** to fetch a data preview.

-1. The second source you're adding will point at the SQL DB table 'dbo.TripFares'. Under your 'TripDataCSV' source, there will be another **Add Source** box. Click it to add a new source transformation.

-1. Name this source 'TripFaresSQL'. Click **New** next to the source dataset field to create a new SQL DB dataset.

-1. Select the **Azure SQL Database** tile and click continue. *Note: You may notice many of the connectors in data factory are not supported in mapping data flow. To transform data from one of these sources, ingest it into a supported source using the copy activity*.

-1. Call your dataset 'TripFares'. Select 'SQLDB' as your linked service. Select table name 'dbo.TripFares' from the table name dropdown. Import the schema **From connection/store**. Click OK when finished.

-1. To add a new transformation, click the plus icon in the bottom-right corner of 'TripDataCSV'. Under **Multiple inputs/outputs**, select **Join**.

- Select which columns you wish to match on from each stream via the **Join conditions** dropdown. To add an additional join condition, click on the plus icon next to an existing condition. By default, all join conditions are combined with an AND operator, which means all conditions must be met for a match. In this lab, we want to match on columns `medallion`, `hack_license`, `vendor_id`, and `pickup_datetime`

-1. To enter an aggregation expression, click the blue box labeled **Enter expression**. This will open up the data flow expression builder, a tool used to visually create data flow expressions using input schema, built-in functions and operations, and user-defined parameters. For more information on the capabilities of the expression builder, see the [expression builder documentation](./concepts-data-flow-expression-builder.md).

- To get the average fare, use the `avg()` aggregation function to aggregate the `total_amount` column cast to an integer with `toInteger()`. In the data flow expression language, this is defined as `avg(toInteger(total_amount))`. Click **Save and finish** when you're done.

-1. To add an additional aggregation expression, click on the plus icon next to `average_fare`. Select **Add column**.

- To get the total trip distance, use the `sum()` aggregation function to aggregate the `trip_distance` column cast to an integer with `toInteger()`. In the data flow expression language, this is defined as `sum(toInteger(trip_distance))`. Click **Save and finish** when you're done.

-1. Name your sink 'SQLDWSink'. Click **New** next to the sink dataset field to create a new Azure Synapse Analytics dataset.

-1. Select the **Azure Synapse Analytics** tile and click continue.

-1. Call your dataset 'AggregatedTaxiData'. Select 'SQLDW' as your linked service. Select **Create new table** and name the new table dbo.AggregateTaxiData. Click OK when finished

-1. As we did for the copy activity, click **Debug** to execute a debug run. For debug runs, the data flow activity will use the active debug cluster instead of spinning up a new cluster. This pipeline will take a little over a minute to execute.

-1. You can click a transformation to get additional details on its execution such as partitioning information and new/updated/dropped columns.

-1. Open a new tab and navigate to the Azure portal. Copy the script provided to create a user in the database that you want to share data from. Do this by logging into the EDW database using Query Explorer (preview) using AAD authentication.

- create user [dataprovider-xxxx] from external login;

-1. Click on Snapshot schedule. You can disable the snapshot schedule if you choose.

-You should now have an Azure Data Share invitation in your inbox from Microsoft Azure. Launch Outlook Web Access (outlook.com) and log in using the credentials supplied for your Azure subscription.

-In the e-mail that you should have received, click on "View invitation >". At this point, you're going to be simulating the data consumer experience when accepting a data providers invitation to their data share.

-1. Click on the invitation titled *DataProvider*.

-1. Use AAD authentication to log in to Query editor.

-1. On the left menu, select **Create a resource** > **Data + Analytics** > **Data Factory**.

-

- :::image type="content" source="./media/quickstart-create-data-factory-portal/new-azure-data-factory-menu.png" alt-text="Screenshot showing the Data Factory selection in the New pane.":::

-2. On the **New data factory** page, provide values for the fields that are shown in the following image:

-

- :::image type="content" source="./media/load-azure-data-lake-storage-gen2-from-gen1/new-azure-data-factory.png" alt-text="Screenshot showing the New Data factory page.":::

-

- * **Name**: Enter a globally unique name for your Azure data factory. If you receive the error "Data factory name \"LoadADLSDemo\" is not available," enter a different name for the data factory. For example, use the name _**yourname**_**ADFTutorialDataFactory**. Create the data factory again. For the naming rules for Data Factory artifacts, see [Data Factory naming rules](naming-rules.md).

- * **Subscription**: Select your Azure subscription in which to create the data factory.

- * **Resource Group**: Select an existing resource group from the drop-down list. You also can select the **Create new** option and enter the name of a resource group. To learn about resource groups, see [Use resource groups to manage your Azure resources](../azure-resource-manager/management/overview.md).

- * **Version**: Select **V2**.

- * **Location**: Select the location for the data factory. Only supported locations are displayed in the drop-down list. The data stores that are used by the data factory can be in other locations and regions.

-3. Select **Create**.

-4. After creation is finished, go to your data factory. You see the **Data Factory** home page as shown in the following image:

-

-5. Select **Open** on the **Open Azure Data Factory Studio** tile to launch the Data Integration application in a separate tab.

-When you see significant number of throttling errors from [copy activity monitoring](copy-activity-monitoring.md#monitor-visually), it indicates you have reached the capacity limit of your storage account. ADF will retry automatically to overcome each throttling error to make sure there will not be any data lost, but too many retries impact your copy throughput as well. In such case, you are encouraged to reduce the number of copy activities running cocurrently to avoid significant amounts of throttling errors. If you have been using single copy activity to copy data, then you are encouraged to reduce the DIU.

-1. On the left menu, select **Create a resource** > **Integration** > **Data Factory**:

-

- :::image type="content" source="./media/doc-common-process/new-azure-data-factory-menu.png" alt-text="Data Factory selection in the &quot;New&quot; pane":::

-2. In the **New data factory** page, provide values for following fields:

-

- * **Name**: Enter a globally unique name for your Azure data factory. If you receive the error "Data factory name *YourDataFactoryName* is not available", enter a different name for the data factory. For example, you could use the name _**yourname**_**ADFTutorialDataFactory**. Try creating the data factory again. For the naming rules for Data Factory artifacts, see [Data Factory naming rules](naming-rules.md).

- * **Subscription**: Select your Azure subscription in which to create the data factory.

- * **Resource Group**: Select an existing resource group from the drop-down list, or select the **Create new** option and enter the name of a resource group. To learn about resource groups, see [Using resource groups to manage your Azure resources](../azure-resource-manager/management/overview.md).

- * **Version**: Select **V2**.

- * **Location**: Select the location for the data factory. Only supported locations are displayed in the drop-down list. The data stores that are used by data factory can be in other locations and regions.

-3. Select **Create**.

-4. After creation is complete, go to your data factory. You see the **Data Factory** home page as shown in the following image:

-

- Select **Open** on the **Open Azure Data Factory Studio** tile to launch the Data Integration Application in a separate tab.

-1. On the left menu, select **Create a resource** > **Analytics** > **Data Factory**:

-

- :::image type="content" source="./media/quickstart-create-data-factory-portal/new-azure-data-factory-menu.png" alt-text="Data Factory selection in the &quot;New&quot; pane":::

-2. In the **New data factory** page, provide values for the fields that are shown in the following image:

-

- :::image type="content" source="./media/load-data-into-azure-data-lake-store//new-azure-data-factory.png" alt-text="New data factory page":::

-

- * **Name**: Enter a globally unique name for your Azure data factory. If you receive the error "Data factory name \"LoadADLSG1Demo\" is not available," enter a different name for the data factory. For example, you could use the name _**yourname**_**ADFTutorialDataFactory**. Try creating the data factory again. For the naming rules for Data Factory artifacts, see [Data Factory naming rules](naming-rules.md).

- * **Subscription**: Select your Azure subscription in which to create the data factory.

- * **Resource Group**: Select an existing resource group from the drop-down list, or select the **Create new** option and enter the name of a resource group. To learn about resource groups, see [Using resource groups to manage your Azure resources](../azure-resource-manager/management/overview.md).

- * **Version**: Select **V2**.

- * **Location**: Select the location for the data factory. Only supported locations are displayed in the drop-down list. The data stores that are used by data factory can be in other locations and regions. These data stores include Azure Data Lake Storage Gen1, Azure Storage, Azure SQL Database, and so on.

-3. Select **Create**.

-4. After creation is complete, go to your data factory. You see the **Data Factory** home page as shown in the following image:

-

- Select **Open** on the **Open Azure Data Factory Studio** tile to launch the Data Integration Application in a separate tab.

-3. In the **Source data store** page, click **+ Create new connection**:

-7. In the **Destination data store** page, click **+ Create new connection**, and then select **Azure Data Lake Storage Gen1**, and select **Continue**:

-> [!NOTE]

-> You can skip the creation of a new data factory if you wish to use the pipelines feature within your existing Synapse workspace to load the data. Azure Synapse embeds the functionality of Azure Data Factory within its pipelines feature.

-1. On the left menu, select **Create a resource** > **Data + Analytics** > **Data Factory**:

-2. On the **New data factory** page, provide values for following items:

- * **Name**: Enter *LoadSQLDWDemo* for name. The name for your data factory must be *globally unique. If you receive the error "Data factory name 'LoadSQLDWDemo' is not available", enter a different name for the data factory. For example, you could use the name _**yourname**_**ADFTutorialDataFactory**. Try creating the data factory again. For the naming rules for Data Factory artifacts, see [Data Factory naming rules](naming-rules.md).

- * **Subscription**: Select your Azure subscription in which to create the data factory.

- * **Resource Group**: Select an existing resource group from the drop-down list, or select the **Create new** option and enter the name of a resource group. To learn about resource groups, see [Using resource groups to manage your Azure resources](../azure-resource-manager/management/overview.md).

- * **Version**: Select **V2**.

- * **Location**: Select the location for the data factory. Only supported locations are displayed in the drop-down list. The data stores that are used by data factory can be in other locations and regions. These data stores include Azure Data Lake Store, Azure Storage, Azure SQL Database, and so on.

-3. Select **Create**.

-4. After creation is complete, go to your data factory. You see the **Data Factory** home page as shown in the following image:

- :::image type="content" source="./media/doc-common-process/data-factory-home-page.png" alt-text="Home page for the Azure Data Factory, with the Open Azure Data Factory Studio tile.":::

- Select **Open** on the **Open Azure Data Factory Studio** tile to launch the Data Integration Application in a separate tab.

-1. On the left menu, select **Create a resource** > **Analytics** > **Data Factory**:

-

- :::image type="content" source="./media/quickstart-create-data-factory-portal/new-azure-data-factory-menu.png" alt-text="Data Factory selection in the &quot;New&quot; pane":::

-2. In the **New data factory** page, provide values for the fields that are shown in the following image:

-

- :::image type="content" source="./media/load-office-365-data/new-azure-data-factory.png" alt-text="New data factory page":::

-

- * **Name**: Enter a globally unique name for your Azure data factory. If you receive the error "Data factory name *LoadFromOffice365Demo* is not available", enter a different name for the data factory. For example, you could use the name _**yourname**_**LoadFromOffice365Demo**. Try creating the data factory again. For the naming rules for Data Factory artifacts, see [Data Factory naming rules](naming-rules.md).

- * **Subscription**: Select your Azure subscription in which to create the data factory.

- * **Resource Group**: Select an existing resource group from the drop-down list, or select the **Create new** option and enter the name of a resource group. To learn about resource groups, see [Using resource groups to manage your Azure resources](../azure-resource-manager/management/overview.md).

- * **Version**: Select **V2**.

- * **Location**: Select the location for the data factory. Only supported locations are displayed in the drop-down list. The data stores that are used by data factory can be in other locations and regions. These data stores include Azure Data Lake Store, Azure Storage, Azure SQL Database, and so on.

-3. Select **Create**.

-4. After creation is complete, go to your data factory. You see the **Data Factory** home page as shown in the following image:

-

-5. Select **Open** on the **Open Azure Data Factory Studio** tile to launch the Data Integration Application in a separate tab.

-1. Go to the pipeline > **Source tab**, click **+ New** to create a source dataset.

-3. You are now in the copy activity configuration tab. Click on the **Edit** button next to the Microsoft 365 (Office 365) dataset to continue the data configuration.

-5. Go to the **Connection tab** of the Properties window. Next to the Linked service text box, click **+ New**.

-10. Click on the **Import Schema** tab to import the schema for Message dataset.

-3. Click on **Edit** button next to the Azure Blob Storage dataset to continue the data configuration.

-If this is the first time you are requesting data for this context (a combination of which data table is being access, which destination account is the data being loaded into, and which user identity is making the data access request), you will see the copy activity status as **In Progress**, and only when you click into "Details" link under Actions will you see the status as **RequesetingConsent**. A member of the data access approver group needs to approve the request in the Privileged Access Management before the data extraction can proceed.

-| | Interactive authoring | Copy compute scale | Pipeline & External compute scale |

-1. In the Azure portal, go to **Monitor**. Select **Settings** > **Diagnostics settings**.

-1. Select the data factory for which you want to set a diagnostic setting.

-1. If no settings exist on the selected data factory, you're prompted to create a setting. Select **Turn on diagnostics**.

- :::image type="content" source="media/data-factory-monitor-oms/monitor-oms-image1.png" alt-text="Screenshot that shows creating a diagnostic setting if no settings exist.":::

- If there are existing settings on the data factory, you see a list of settings already configured on the data factory. Select **Add diagnostic setting**.

-The **DUAL STANDBY PAIR / ROLE** informational tile shows the name of your dual standby Azure-SSIS IR pair that works in sync with Azure SQL Database managed instance failover group for business continuity and disaster recovery (BCDR) and the current primary/secondary role of your Azure-SSIS IR. When SSISDB failover occurs, your primary and secondary Azure-SSIS IRs will swap roles (see [Configuring your Azure-SSIS IR for BCDR](./configure-bcdr-azure-ssis-integration-runtime.md)).

-1. In the **Integration Runtimes** list, in the **Actions** column, select the **Edit** button for your Azure-SSIS IR.

- :::image type="content" source="media/join-azure-ssis-integration-runtime-virtual-network/integration-runtime-edit.png" alt-text="Edit the integration runtime":::

-For customers who have various subscriptions, and who want to ensure a single plan is deployed across their tenant for cost control, you can use Azure Policy to [restrict creation of Azure DDoS Protection Standard plans](https://aka.ms/ddosrestrictplan). This policy will block the creation of any DDoS plans, unless the subscription has been previously marked as an exception. This policy will also show a list of all subscriptions that have a DDoS plan deployed but should not, marking them as out of compliance.

-When you auto-provision the Log Analytics agent in Defender for Cloud, you can choose to collect additional security events to the workspace. When you auto-provision the Log Analytics agent in Defender for Cloud, the option to collect additional security events to the workspace isn't available. Defender for Cloud doesn't rely on these security events, but they can be helpful for investigations through Microsoft Sentinel.

-If you want to collect security events when you auto-provision the Azure Monitor Agent, you can create a [Data Collection Rule](/azure-monitor/essentials/data-collection-rule-overview) to collect the required events.

-This Microsoft Defender plan detects anomalous activities indicating unusual and potentially harmful attempts to access or exploit databases.

- - [Azure Arc-enabled SQL Server (preview)](/sql/sql-server/azure-arc/overview)

-Defender for Cloud's auto provisioning settings has a toggle for each type of supported extension. When you enable auto provisioning of an extension, you assign the appropriate **Deploy if not exists** policy. This policy type ensures the extension is provisioned on all existing and future resources of that type.

-> Learn more about Azure Policy effects including deploy if not exists in [Understand Azure Policy effects](../governance/policy/concepts/effects.md).

-Connecting your AWS account is part of the multicloud experience available in Microsoft Defender for Cloud. For related information, see the following page:

-Connecting your GCP project is part of the multicloud experience available in Microsoft Defender for Cloud. For related information, see the following page:

-### ServiceNow Integration API - ΓÇ£/external/v3/integration/ (Preview)

-The below API's can be used with the ServiceNow integration via the ServiceNow's Service Graph Connector for Defender for IoT.

-### devices

-This API returns data about all devices that were updated after the given timestamp.

-#### Request

- - ΓÇ£**timestamp**ΓÇ¥ ΓÇô the time from which updates are required, only later updates will be returned.

- - ΓÇ£**sensorId**ΓÇ¥ - use this parameter to get only devices seen by a specific sensor. The ID should be taken from the results of the Sensors API.

- - ΓÇ£**notificationType**ΓÇ¥ - should be a number, from the following mapping:

- - 0 ΓÇô both updated and new devices (default).

- - 1 ΓÇô only new devices.

- - 2 ΓÇô only updated devices.

- - ΓÇ£**page**ΓÇ¥ - the page number, from the result set (first page is 0, default value is 0)

- - ΓÇ£**size**ΓÇ¥ - the page size (default value is 50)

-#### Response

- - ΓÇ£**u_count**ΓÇ¥ - amount of object in the full result sets, including all pages.

- - ΓÇ£**u_devices**ΓÇ¥ - array of device objects. Each object is defined with the parameters listed in the [device](#device) API.

-### Connections

-This API returns data about all device connections that were updated after the given timestamp.

-#### Request

- - ΓÇ£**timestamp**ΓÇ¥ ΓÇô the time from which updates are required, only later updates will be returned.

- - ΓÇ£**page**ΓÇ¥ - the page number, from the result set (default value is 1)

- - ΓÇ£**size**ΓÇ¥ - the page size (default value is 50)

-#### Response

- - ΓÇ£**u_count**ΓÇ¥ - amount of object in the full result sets, including all pages.

- - ΓÇ£**u_connections**ΓÇ¥ - array of

- - ΓÇ£**u_src_device_id**ΓÇ¥ - the ID of the source device.

- - ΓÇ£**u_dest_device_id**ΓÇ¥ - the ID of the destination device.

- - ΓÇ£**u_connection_type**ΓÇ¥ - one of the following:

- - ΓÇ£**One Way**ΓÇ¥

- - ΓÇ£**Two Way**ΓÇ¥

- - ΓÇ£**Multicast**ΓÇ¥

-### device

-#### Request

- - ΓÇ£**deviceId**ΓÇ¥ ΓÇô the ID of the requested device.

- - ΓÇ£**u_id**ΓÇ¥ - the internal ID of the device.

- - ΓÇ£**u_vendor**ΓÇ¥ - the name of the vendor.

- - ΓÇ£**u_mac_address_objects**ΓÇ¥ - array of

- - ΓÇ£**u_mac_address**ΓÇ¥ - mac address of the device.

- - ΓÇ£**u_ip_address_objects**ΓÇ¥ - array of

- - ΓÇ£**u_ip_address**ΓÇ¥ - IP address of the device.

- - ΓÇ£**u_guessed_mac_addresses**ΓÇ¥ - array of

- - ΓÇ£**u_mac_address**ΓÇ¥ - guessed mac address.

- - ΓÇ£**u_name**ΓÇ¥ - the name of the device.

- - ΓÇ£**u_last_activity**ΓÇ¥ - timestamp of the last time the device was active.

- - ΓÇ£**u_first_discovered**ΓÇ¥ - timestamp of the discovery time of the device.

- - ΓÇ£**u_last_update**ΓÇ¥ - timestamp of the last update time of the device.

- - ΓÇ£**u_vlans**ΓÇ¥ - array of

- - ΓÇ£**u_vlan**ΓÇ¥ - vlan in which the device is in.

- - ΓÇ£**u_device_type**ΓÇ¥ -

- - ΓÇ£**u_name**ΓÇ¥ - the device type

- - ΓÇ£**u_purdue_layer**ΓÇ¥ - the default purdue layer for this device type.

- - ΓÇ£**u_category**ΓÇ¥ - will be one of the following:

- - ΓÇ£**IT**ΓÇ¥

- - ΓÇ£**ICS**ΓÇ¥

- - ΓÇ£**IoT**ΓÇ¥

- - ΓÇ£**Network**ΓÇ¥

- - ΓÇ£**u_operating_system**ΓÇ¥ - the device operating system.

- - ΓÇ£**u_protocol_objects**ΓÇ¥ - array of

- - ΓÇ£**u_protocol**ΓÇ¥ - protocol the device uses.

- - ΓÇ£**u_purdue_layer**ΓÇ¥ - the purdue layer that was manually set by the user.

- - ΓÇ£**u_sensor_ids**ΓÇ¥ - array of

- - ΓÇ£**u_sensor_id**ΓÇ¥ - the ID of the sensor that saw the device.

- - ΓÇ£**u_device_urls**ΓÇ¥ - array of

- - ΓÇ£**u_device_url**ΓÇ¥ the URL to view the device in the sensor.

- - ΓÇ£**u_firmwares**ΓÇ¥ - array of

- - ΓÇ£**u_address**ΓÇ¥

- - ΓÇ£**u_module_address**ΓÇ¥

- - ΓÇ£**u_serial**ΓÇ¥

- - ΓÇ£**u_model**ΓÇ¥

- - ΓÇ£**u_version**ΓÇ¥

- - ΓÇ£**u_additional_data**"

-### Deleted devices

-#### Request

- - ΓÇ£**timestamp**ΓÇ¥ ΓÇô the time from which updates are required, only later updates will be returned.

- - Array of

- - ΓÇ£**u_id**ΓÇ¥ - the ID of the deleted device.

-### sensors

-#### Request

- - Array of

- - ΓÇ£**u_id**ΓÇ¥ - internal sensor ID, to be used in the devices API.

- - ΓÇ£**u_name**ΓÇ¥ - the name of the appliance.

- - ΓÇ£**u_connection_state**ΓÇ¥ - connectivity with the CM state. One of the following:

- - ΓÇ£**SYNCED**ΓÇ¥ - Connection is successful.

- - ΓÇ£**OUT_OF_SYNC**ΓÇ¥ - Management console cannot process data received from Sensor.

- - ΓÇ£**TIME_DIFF_OFFSET**ΓÇ¥ - Time drift detected. management console has been disconnected from Sensor.

- - ΓÇ£**DISCONNECTED**ΓÇ¥ - Sensor not communicating with management console. Check network connectivity.

- - ΓÇ£**u_interface_address**ΓÇ¥ - the network address of the appliance.

- - ΓÇ£**u_version**ΓÇ¥ - string representation of the sensorΓÇÖs version.

- - ΓÇ£**u_alert_count**ΓÇ¥ - number of alerts found by the sensor.

- - ΓÇ£**u_device_count**ΓÇ¥ - number of devices discovered by the sensor.

- - ΓÇ£**u_unhandled_alert_count**ΓÇ¥ - number of unhandled alerts in the sensor.

- - ΓÇ£**u_is_activated**ΓÇ¥ - is the alert activated.

- - ΓÇ£**u_data_intelligence_version**ΓÇ¥ - string representation of the data intelligence installed in the sensor.

- - ΓÇ£**u_remote_upgrade_stage**ΓÇ¥ - the state of the remote upgrade. One of the following:

- - "**UPLOADING**"

- - "**PREPARE_TO_INSTALL**"

- - "**STOPPING_PROCESSES**"

- - "**BACKING_UP_DATA**"

- - "**TAKING_SNAPSHOT**"

- - "**UPDATING_CONFIGURATION**"

- - "**UPDATING_DEPENDENCIES**"

- - "**UPDATING_LIBRARIES**"

- - "**PATCHING_DATABASES**"

- - "**STARTING_PROCESSES**"

- - "**VALIDATING_SYSTEM_SANITY**"

- - "**VALIDATION_SUCCEEDED_REBOOTING**"

- - "**SUCCESS**"

- - "**FAILURE**"

- - "**UPGRADE_STARTED**"

- - "**STARTING_INSTALLATION**"

- - "**INSTALLING_OPERATING_SYSTEM**"

- - ΓÇ£**u_uid**ΓÇ¥ - globally unique identifier of the sensor

- - "**u_is_in_learning_mode**" - Boolean indication as to whether the sensor is in Learn mode or not

-### devicecves

-#### Request

- - ΓÇ£**timestamp**ΓÇ¥ ΓÇô the time from which updates are required, only later updates will be returned.

- - ΓÇ£**page**ΓÇ¥ - Defines the page number, from the result set (first page is 0, default value is 0)

- - ΓÇ£**size**ΓÇ¥ - Defines the page size (default value is 50)

- - ΓÇ£**sensorId**ΓÇ¥ - Shows results from a specific sensor, as defined by the given sensor ID.

- - ΓÇ£**score**ΓÇ¥ - Determines a minimum CVE score to be retrieved. All results will have a CVE score equal to or higher than the given value. Default = **0**.

- - ΓÇ£**deviceIds**ΓÇ¥ - A comma-separated list of device IDs from which you want to show results. For example: **1232,34,2,456**

- - ΓÇ£**u_count**ΓÇ¥ - amount of object in the full result sets, including all pages.

- - ΓÇ£**u_id**ΓÇ¥ - the same as in the specific device API.

- - ΓÇ£**u_name**ΓÇ¥ - the same as in the specific device API.

- - ΓÇ£**u_ip_address_objects**ΓÇ¥ - the same as in the specific device API.

- - ΓÇ£**u_mac_address_objects**ΓÇ¥ - the same as in the specific device API.

- - ΓÇ£**u_last_activity**ΓÇ¥ - the same as in the specific device API.

- - ΓÇ£**u_last_update**ΓÇ¥ - the same as in the specific device API.

- - ΓÇ£**u_cves**ΓÇ¥ - an array of CVEs:

- - ΓÇ£**u_ip_address**ΓÇ¥ - the IP address of the specific interface with the specific firmware on which the CVE was detected.

- - ΓÇ£**u_cve_id**ΓÇ¥- the ID of the CVE

- - ΓÇ£**u_score**ΓÇ¥- the risk score of the CVE

- - ΓÇ£**u_attack_vector**ΓÇ¥ - one of the following:

- - "**ADJACENT_NETWORK**"

- - "**LOCAL**"

- - "**NETWORK**"

- - ΓÇ£**u_description**ΓÇ¥ - description about the CVE.

-For more information, see [ServiceNow Integration API - ΓÇ£/external/v3/integration/ (Preview)](references-work-with-defender-for-iot-apis.md#servicenow-integration-apiexternalv3integration-preview).

->>>>>>> 3e9c47c4758cdb6f63a6873219cab9498206cb2a

-> * Integrating with Microsoft Defender for Endpoint

-> [!IMPORTANT]

-> The **Enterprise IoT network sensor** is currently in **PREVIEW**. See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

-Integrate with [Microsoft Defender for Endpoint](/microsoft-365/security/defender-endpoint/) to extend your security analytics capabilities, providing complete coverage across your Enterprise IoT devices. Defender for Endpoint analytics features include alerts, vulnerabilities, and recommendations for your enterprise devices.

-After you've onboarded a plan for Enterprise IoT and set up your Enterprise IoT network sensor, your device data integrates automatically with Microsoft Defender for Endpoint.

-For more information in Defender for Endpoint documentation, see:

-A type checking and casting function for determining whether an expression has a Boolean value.

-`<expression>`, an expression to check whether it is a Boolean.

-A Boolean value indicating if the type of the specified expression is a Boolean.

-A type checking and casting function to check whether a property is defined.

-This is only supported when the property value is a primitive type. Primitive types include string, Boolean, numeric, or `null`. `DateTime`, object types, and arrays are not supported.

-`<property>`, a property to determine whether it is defined. The property must be of a primitive type.

-A type checking and casting function for determining whether an expression's value is `null`.

-`<expression>`, an expression to check whether it is null.

-A Boolean value indicating if the type of the specified expression is `null`.

-A type checking and casting function for determining whether an expression has a number value.

-`<expression>`, an expression to check whether it is a number.

-A Boolean value indicating if the type of the specified expression is a number.

-A type checking and casting function for determining whether an expression's value is of a JSON object type.

-`<expression>`, an expression to check whether it is of an object type.

-A Boolean value indicating if the type of the specified expression is a JSON object.

-A type checking and casting function to determine whether a twin is of a particular model type. Includes models that inherit from the specified model.

-A type checking and casting function for determining whether an expression's value is of a primitive type (string, Boolean, numeric, or `null`).

-`<expression>`, an expression to check whether it is of a primitive type.

-A Boolean value indicating if the type of the specified expression is one of the primitive types (string, Boolean, numeric, or `null`).

-A type checking and casting function for determining whether an expression has a string value.

-`<expression>`, an expression to check whether it is a string.

-* Single-labeled private DNS zones aren't supported. Your private DNS zone must have two or more labels. For example contoso.com has two labels separated by a dot. A private DNS zone can have a maximum of 34 labels.

- 

-1. Give your logic app a name that's unique in your subscription, then select the same subscription, resource group, and location as your IoT hub.

- 

- "topic": "/SUBSCRIPTIONS/<subscription ID>/RESOURCEGROUPS/<resource group name>/PROVIDERS/MICROSOFT.DEVICES/IOTHUBS/<hub name>",

- "hubName": "egtesthub1",

- 

- 

- >

- > Dropping an external table does **not** delete the data, only the table definition.

-FHIR service supports the $export command that allows you to export the data out of the FHIR service account to a storage account.

-The three steps below are used in configuring export data in the FHIR service:

-## Enable managed identity on the FHIR service

-The first step in configuring the FHIR service for export is to enable system wide managed identity on the service, which will be used to grant the service to access the storage account. For more information about managed identities in Azure, see [About managed identities for Azure resources](../../active-directory/managed-identities-azure-resources/overview.md).

-In this step, browse to your FHIR service in the Azure portal, and select the **Identity** blade. Select the **Status** option to **On** , and then select **Save**. **Yes** and **No** buttons will display. Select **Yes** to enable the managed identity for FHIR service. Once the system identity has been enabled, you'll see a system assigned GUID value.

-## Assign permissions to the FHIR service to access the storage account

-1. Select **Access control (IAM)**.

-1. Select **Add > Add role assignment**. If the **Add role assignment** option is grayed out, ask your Azure administrator to assign you permission to perform this task.

-1. On the **Role** tab, select the [Storage Blob Data Contributor](../../role-based-access-control/built-in-roles.md#storage-blob-data-contributor) role.

-1. On the **Members** tab, select **Managed identity**, and then select **Select members**.

-1. Select your Azure subscription.

-1. Select **System-assigned managed identity**, and then select the FHIR service.

-1. On the **Review + assign** tab, select **Review + assign** to assign the role.

-Now you're ready to select the storage account in the FHIR service as a default storage account for export.

-## Specify the export storage account for FHIR service

-The final step is to assign the Azure storage account that the FHIR service will use to export the data to.

-> If you haven't assigned storage access permissions to the FHIR service, the export operations ($export) will fail.

-To do this, select the **Export** blade in FHIR service and select the storage account. To search for the storage account, enter its name in the text field. You can also search for your storage account by using the available filters **Name**, **Resource group**, or **Region**.

-After you've completed this final step, you're ready to export the data using $export command.

-> Only storage accounts in the same subscription as that for FHIR service are allowed to be registered as the destination for $export operations.

-## Use Azure storage accounts behind firewalls

-FHIR service supports a secure export operation. Choose one of the two options below:

-* Allowing FHIR service as a Microsoft Trusted Service to access the Azure storage account.

-* Allowing specific IP addresses associated with FHIR service to access the Azure storage account.

-This option provides two different configurations depending on whether the storage account is in the same location as, or is in a different location from that of the FHIR service.

-Select a storage account from the Azure portal, and then select the **Networking** blade. Select **Selected networks** under the **Firewalls and virtual networks** tab.

-Select **Microsoft.HealthcareApis/workspaces** from the **Resource type** dropdown list and your workspace from the **Instance name** dropdown list.

-Under the **Exceptions** section, select the box **Allow trusted Microsoft services to access this storage account** and save the setting.

-Next, specify the FHIR service instance in the selected workspace instance for the storage account using the PowerShell command.

-```

-$resourceId = "/subscriptions/$subscription/resourceGroups/$resourcegroup/providers/Microsoft.HealthcareApis/workspaces/$workspacename/fhirservices/$fhirname"

-You can see that the networking setting for the storage account shows **two selected** in the **Instance name** dropdown list. One is linked to the workspace instance and the second is linked to the FHIR service instance.

-Note that you'll need to install "Add-AzStorageAccountNetworkRule" using an administrator account. For more information, see [Configure Azure Storage firewalls and virtual networks](../../storage/common/storage-network-security.md)

-`

-Install-Module Az.Storage -Repository PsGallery -AllowClobber -Force

-`

-You're now ready to export FHIR data to the storage account securely. Note that the storage account is on selected networks and isn't publicly accessible. To access the files, you can either enable and use private endpoints for the storage account, or enable all networks for the storage account to access the data there if possible.

-> [!IMPORTANT]

-> The user interface will be updated later to allow you to select the Resource type for FHIR service and a specific service instance.

-### Allowing specific IP addresses for the Azure storage account in a different region

-Select **Networking** of the Azure storage account from the

-portal.

-Select **Selected networks**. Under the Firewall section, specify the IP address in the **Address range** box. Add IP ranges to

-allow access from the internet or your on-premises networks. You can

-find the IP address in the table below for the Azure region where the

-FHIR service is provisioned.

-### Allowing specific IP addresses for the Azure storage account in the same region

-The configuration process is the same as above except a specific IP

-address range in Classless Inter-Domain Routing (CIDR) format is used instead, 100.64.0.0/10. The reason why the IP address range, which includes 100.64.0.0 ΓÇô 100.127.255.255, must be specified is because the actual IP address used by the service varies, but will be within the range, for each $export request.

-> It is possible that a private IP address within the range of 10.0.2.0/24 may be used instead. In that case, the $export operation will not succeed. You can retry the $export request, but there is no guarantee that an IP address within the range of 100.64.0.0/10 will be used next time. That's the known networking behavior by design. The alternative is to configure the storage account in a different region.

-In this article, you learned about the three steps in configuring export settings that allow you to export data out of FHIR service account to a storage account. For more information about the Bulk Export feature that allows data to be exported from the FHIR service, see

-[  ](media/bulk-import/importer-url-and-body.png#lightbox)

-FHIR&#174; is a registered trademark of [HL7](https://hl7.org/fhir/) and is used with the permission of HL7.

-The `$convert-data` operation is integrated into the FHIR service as a RESTful API action. Calling the `$convert-data` endpoint causes the FHIR service to perform a conversion on health data sent in an API request:

-The health data is delivered to the FHIR service in the body of the `$convert-data` request. If the request is successful, the FHIR service will return a FHIR `Bundle` response with the data converted to FHIR.

-> Results when using the de-identified export will vary based on factors such as data inputted, and functions selected by the customer. Microsoft is unable to evaluate the de-identified export outputs or determine the acceptability for customer's use cases and compliance needs. The de-identified export is not guaranteed to meet any specific legal, regulatory, or compliance requirements.

-The $export command can also be used to export de-identified data from the FHIR server. It uses the anonymization engine from [FHIR tools for anonymization](https://github.com/microsoft/FHIR-Tools-for-Anonymization), and takes anonymization config details in query parameters. You can create your own anonymization config file or use the [sample config file](https://github.com/microsoft/Tools-for-Health-Data-Anonymization/blob/master/docs/FHIR-anonymization.md#sample-configuration-file) for HIPAA Safe Harbor method as a starting point.

-The anonymization engine comes with a sample configuration file to help meet the requirements of HIPAA Safe Harbor Method. The configuration file is a JSON file with four sections: `fhirVersion`, `processingErrors`, `fhirPathRules`, `parameters`.

-* `processingErrors` specifies what action to take for the processing errors that may arise during the anonymization. You can _raise_ or _keep_ the exceptions based on your needs.

-* `fhirPathRules` specifies which anonymization method is to be used. The rules are executed in the order of appearance in the configuration file.

-* `parameters` sets rules for the anonymization behaviors specified in _fhirPathRules_.

-Here's a sample configuration file for R4:

-For more detailed information on each of these four sections of the configuration file, select [here](https://github.com/microsoft/Tools-for-Health-Data-Anonymization/blob/master/docs/FHIR-anonymization.md#configuration-file-format).

-## Using $export command for the de-identified data

- `https://<<FHIR service base URL>>/$export?_container=<<container_name>>&_anonymizationConfig=<<config file name>>&_anonymizationConfigEtag=<<ETag on storage>>`

-> Right now the FHIR service only supports de-identified export at the system level ($export).

-| _\_anonymizationConfig_ |DemoConfig.json|Required for de-identified export |Name of the configuration file. See the configuration file format [here](https://github.com/microsoft/FHIR-Tools-for-Anonymization#configuration-file-format). This file should be kept inside a container named **anonymization** within the same Azure storage account that is configured as the export location. |

-| _\_anonymizationConfigEtag_|"0x8D8494A069489EC"|Optional for de-identified export|This is the Etag of the configuration file. You can get the Etag using Azure Storage Explorer from the blob property|

-> Both raw export as well as de-identified export writes to the same Azure storage account specified as part of export configuration. It is recommended that you use different containers corresponding to different de-identified config and manage user access at the container level.

-In this article, you've learned how to set up and use de-identified export. For more information about how to export FHIR data, see

-FHIR&#174; is a registered trademark of [HL7](https://hl7.org/fhir/) and is used with the permission of HL7.

-Before attempting to use `$export`, make sure that your FHIR service is configured to connect with an ADLS Gen2 storage account. For configuring export settings and creating an ADLS Gen2 storage account, refer to the [Configure settings for export](./configure-export-data.md) page.

-After setting up the FHIR service to connect with an ADLS Gen2 storage account, you can call the `$export` endpoint and the FHIR service will export data into a blob storage container inside the storage account. The example request below exports all resources into a container specified by name (`{{containerName}}`). Note that the container in the ADLS Gen2 account must be created beforehand if you want to specify the `{{containerName}}` in the request.

-In some situations, there's a potential for a job to be stuck in a bad state while attempting to `$export` data from the FHIR service. This can occur especially if the ADLS Gen2 storage account permissions haven't been set up correctly. One way to check the status of your `$export` operation is to go to your storage account's **Storage browser** and see if any `.ndjson` files are present in the export container. If the files aren't present and there are no other `$export` jobs running, then there's a possibility the current job is stuck in a bad state. In this case, you can cancel the `$export` job by calling the FHIR service API with a `DELETE` request. Later you can requeue the `$export` job and try again. Information about canceling an `$export` operation can be found in the [Bulk Data Delete Request](https://hl7.org/fhir/uv/bulkdata/export/https://docsupdatetracker.net/index.html#bulk-data-delete-request) documentation from HL7.

-Currently the FHIR service supports `$export` to ADLS Gen2 storage accounts, with the following limitations:

-| `_typeFilter` | Yes | To request finer-grained filtering, you can use `_typeFilter` along with the `_type` parameter. The value of the `_typeFilter` parameter is a comma-separated list of FHIR queries that further restrict the results. |

-> Only storage accounts in the same subscription as that for the FHIR service are allowed to be registered as the destination for `$export` operations.

-# Bulk-import FHIR data (Preview)

-* Maximum number of files to be imported per operation is 1,000.

-* High Speed Parallel Event Grid that triggers from storage accounts or other event grid resources

-FHIR&#174; is a registered trademark of [HL7](https://hl7.org/fhir/) and is used with the permission of HL7.

-Many functions are available when using **JmesPath** as the expression language. Besides the functions available as part of the JmesPath specification, many more custom functions may also be used. This article describes MedTech service-specific custom functions for use with the MedTech service device mapping during the device message normalization process.

-> Check out the [IoMT Connector Data Mapper](https://github.com/microsoft/iomt-fhir/tree/master/tools/data-mapper) tool for editing, testing, and troubleshooting the MedTech service Device and FHIR destination mappings. Export mappings for uploading to the MedTech service in the Azure portal or use with the [open-source version](https://github.com/microsoft/iomt-fhir) of the MedTech service.

-MedTech service enables IoT devices seamless integration with Fast Healthcare Interoperability Resources (FHIR&#174;) services. This reference architecture is designed to accelerate adoption of Internet of Medical Things (IoMT) projects. This solution uses Azure Databricks for the Machine Learning (ML) compute. However, Azure ML Services with Kubernetes or a partner ML solution could fit into the Machine Learning Scoring Environment.

-4. PHI IoMT payload moves from Azure IoT Hub to the MedTech service. Multiple Azure services are represented by 1 MedTech service icon.

- a. MedTech service request Patient resource from FHIR service.

- b. FHIR service sends Patient resource back to the MedTech service.

- c. IoT Patient Observation is record in FHIR service.

-6. Normalized ungrouped data stream sent to Azure Function (ML Input).

-(FHIR&#174;) is a registered trademark of [HL7](https://hl7.org/fhir/) and is used with the permission of HL7.

-The reference architecture below shows the basic components of using Microsoft cloud services to enable Power BI on top of Internet of Medical Things (IoMT) and Fast Healthcare Interoperability Resources (FHIR&#174;) data.

-MedTech service can ingest IoT data from most IoT devices or gateways whatever the location, data center, or cloud.

-(FHIR&#174;) is a registered trademark of [HL7](https://hl7.org/fhir/) and is used with the permission of HL7.

-When combining MedTech service, a Fast Healthcare Interoperability Resources (FHIR&#174;) service, and Teams, you can enable multiple care solutions.

-Below is the MedTech service to Teams notifications conceptual architecture for enabling the MedTech service, FHIR, and Teams Patient App.

-(FHIR&#174;) is a registered trademark of [HL7](https://hl7.org/fhir/) and is used with the permission of HL7.

-Data from health-related devices or medical devices flows through a path in which the MedTech service transforms data into FHIR, and then data is stored on and accessed from the FHIR service. The health data path follows these steps in this order: ingest, normalize, group, transform, and persist. Health data is retrieved from the device in the first step of ingestion. After the data is received, it's processed, or normalized per a user-selected/user-created schema template called the device mapping. Normalized health data is simpler to process and can be grouped. In the next step, health data is grouped into three Operate parameters. After the health data is normalized and grouped, it can be processed or transformed through FHIR destination mappings, and then saved or persisted on the FHIR service.

-To learn how to create Device and FHIR destination mappings, see

-## Pricing plan

-You can only create applications that use a standard pricing plan as a CSP. To showcase Azure IoT Central to your customer, you can create an application that uses the free pricing plan separately. Learn more about the free and standard pricing plans on the [Azure IoT Central pricing page](https://azure.microsoft.com/pricing/details/iot-central/).

-You can only create applications that use a standard pricing plan as a CSP. To showcase Azure IoT Central to your customer, you can create an application that uses the free pricing plan separately. Learn more about the free and standard pricing plans on the [Azure IoT Central pricing page](https://azure.microsoft.com/pricing/details/iot-central/).

-The *free* plan lets you create an IoT Central application to try for seven days. The free plan:

-You can create a copy of any application, minus any device instances, device data history, and user data. The copy uses a standard pricing plan that you'll be billed for. You can't create an application that uses the free pricing plan by copying an application.

- - You only see storage accounts, Event Hubs namespaces, and Service Bus namespaces in the same subscription as your IoT Central application. If you want to export to a destination outside of this subscription, choose **Enter a connection string** and see step 6.

- - For apps created using the free pricing plan, the only way to configure data export is through a connection string. Apps on the free pricing plan don't have an associated Azure subscription.

-You can create IoT Central application using a 7-day free trial, or use a standard pricing plan.

-* There are no specific prerequisites required to deploy this app.

-* You can use the free pricing plan or use an Azure subscription.

-> * Create a solar panel app for free

-* There are no specific prerequisites required to deploy this app.

-* You can use the free pricing plan or use an Azure subscription.

-* There are no specific prerequisites required to deploy this app.

-* You can use the free pricing plan or use an Azure subscription.

-* There are no specific prerequisites required to deploy this app.

-* You can use the free pricing plan or use an Azure subscription.

-* There are no specific prerequisites required to deploy this app.

-* You can use the free pricing plan or use an Azure subscription.

-* There are no specific prerequisites required to deploy this app.

-* You can use the free pricing plan or use an Azure subscription.

- * **URL**: you can use suggested default URL or enter your friendly unique memorable URL. Next, the default setting is recommended if you already have an Azure Subscription. You can start with 7-day free trial pricing plan and choose to convert to a standard pricing plan at any time before the free trail expires.

-* No specific pre-requisites required to deploy this app

-* Recommended to have Azure subscription, but you can even try without it

-* No specific pre-requisites required to deploy this app.

-* Recommended to have Azure subscription, but you can even try without it.

-* There are no specific prerequisites required to deploy this app.

-* You can use the free pricing plan or use an Azure subscription.

-|Azure Virtual Machines deployment service|[Deploy certificates to VMs from customer-managed Key Vault](/archive/blogs/kv/updated-deploy-certificates-to-vms-from-customer-managed-key-vault).|

-|Azure Resource Manager template deployment service|[Pass secure values during deployment](../../azure-resource-manager/templates/key-vault-parameter.md).|

-|Azure Disk Encryption volume encryption service|Allow access to BitLocker Key (Windows VM) or DM Passphrase (Linux VM), and Key Encryption Key, during virtual machine deployment. This enables [Azure Disk Encryption](../../security/fundamentals/encryption-overview.md).|

-|Azure Backup|Allow backup and restore of relevant keys and secrets during Azure Virtual Machines backup, by using [Azure Backup](../../backup/backup-overview.md).|

-|Exchange Online & SharePoint Online|Allow access to customer key for Azure Storage Service Encryption with [Customer Key](/microsoft-365/compliance/customer-key-overview).|

-|Azure Information Protection|Allow access to tenant key for [Azure Information Protection.](/azure/information-protection/what-is-information-protection)|

-|Azure App Service|App Service is trusted only for [Deploying Azure Web App Certificate through Key Vault](https://azure.github.io/AppService/2016/05/24/Deploying-Azure-Web-App-Certificate-through-Key-Vault.html), for individual app itself, the outbound IPs can be added in Key Vault's IP-based rules|

-|Azure SQL Database|[Transparent Data Encryption with Bring Your Own Key support for Azure SQL Database and Azure Synapse Analytics](/azure/azure-sql/database/transparent-data-encryption-byok-overview).|

-|Azure Storage|[Storage Service Encryption using customer-managed keys in Azure Key Vault](../../storage/common/customer-managed-keys-configure-key-vault.md).|

-|Azure Data Lake Store|[Encryption of data in Azure Data Lake Store](../../data-lake-store/data-lake-store-encryption.md) with a customer-managed key.|

-|Azure Synapse Analytics|[Encryption of data using customer-managed keys in Azure Key Vault](../../synapse-analytics/security/workspaces-encryption.md)|

-|Azure Databricks|[Fast, easy, and collaborative Apache SparkΓÇôbased analytics service](/azure/databricks/scenarios/what-is-azure-databricks)|

-|Azure API Management|[Deploy certificates for Custom Domain from Key Vault using MSI](../../api-management/api-management-howto-use-managed-service-identity.md#use-ssl-tls-certificate-from-azure-key-vault)|

-|Azure Data Factory|[Fetch data store credentials in Key Vault from Data Factory](https://go.microsoft.com/fwlink/?linkid=2109491)|

-|Azure Event Hubs|[Allow access to a key vault for customer-managed keys scenario](../../event-hubs/configure-customer-managed-key.md)|

-|Azure Service Bus|[Allow access to a key vault for customer-managed keys scenario](../../service-bus-messaging/configure-customer-managed-key.md)|

-|Azure Import/Export| [Use customer-managed keys in Azure Key Vault for Import/Export service](../../import-export/storage-import-export-encryption-key-portal.md)

-|Azure Container Registry|[Registry encryption using customer-managed keys](../../container-registry/container-registry-customer-managed-keys.md)

-|Azure Application Gateway |[Using Key Vault certificates for HTTPS-enabled listeners](../../application-gateway/key-vault-certs.md)

-|Azure Front Door Standard/Premium|[Using Key Vault certificates for HTTPS](../../frontdoor/standard-premium/how-to-configure-https-custom-domain.md#prepare-your-key-vault-and-certificate)

-|Azure Front Door Classic|[Using Key Vault certificates for HTTPS](../../frontdoor/front-door-custom-domain-https.md#prepare-your-key-vault-and-certificate)

-|Microsoft Purview|[Using credentials for source authentication in Microsoft Purview](../../purview/manage-credentials.md)

-|Azure Machine Learning|[Secure Azure Machine Learning in a virtual network](../../machine-learning/how-to-secure-workspace-vnet.md)|

-When virtual machine scale sets with [public IPs per instance](../virtual-machine-scale-sets/virtual-machine-scale-sets-networking.md) are created with a load balancer in front, the SKU of the instance IPs is determined by the SKU of the Load Balancer (i.e. Basic or Standard). Note that when using a Standard Load Balancer, the individual instance IPs are all of type Standard "no-zone" (though the Load Balancer frontend could be zonal or zone-redundant).

-* Make sure that you install the latest version, [SAP Connector (NCo 3.0) for Microsoft .NET 3.0.24.0 compiled with .NET Framework 4.0 - Windows 64-bit (x64)](https://support.sap.com/en/product/connectors/msnet.html). Earlier versions of SAP NCo might experience the following issues:

- <ABAPTEXT xmlns="http://Microsoft.LobServices.Sap/2007/03/Rfc/">

- <ABAPTEXT xmlns="http://Microsoft.LobServices.Sap/2007/03/Rfc/">

- <ABAPTEXT xmlns="http://Microsoft.LobServices.Sap/2007/03/Rfc/">

-Learn [where and how to deploy your model to a compute target](how-to-deploy-and-where.md).

-* [Deploy your model to a compute target](how-to-deploy-and-where.md)

-> - While Batch endpoints created with earlier APIs will continue to support V1 FileDataset, we will be adding further V2 data assets support with the latest API versions for even more usability and flexibility. For more information on V2 data assets, see [Work with data using SDK v2 (preview)](how-to-use-data.md). For more information on the new V2 experience, see [What is v2](concept-v2.md).

-> * You can authenticate to models deployed as web services (inference endpoints) using __key__ or __token__-based authentication. Keys are static strings, while tokens are retrieved using an Azure AD security object. For more information, see [Configure authentication for models deployed as a web service](how-to-authenticate-web-service.md).

-* [Consume a Machine Learning model deployed as a web service](how-to-consume-web-service.md)

-For more information, see [Deploy models](how-to-deploy-and-where.md#registermodel) and [Troubleshooting deployments](how-to-troubleshoot-deployment.md).

-For more information, see [Deploy models](how-to-deploy-and-where.md).

-For more information, see [Enable model data collection](how-to-enable-data-collection.md).

-+ [How and where to deploy models](how-to-deploy-and-where.md) with Machine Learning

-+ Create clients that [consume a deployed model](how-to-consume-web-service.md)

-1. Select the **Subscription** link in the middle of the page.

-1. For the **Custom role name**, type **Labeler**.

-1. In the **Description** box, add **Labeler access for data labeling projects**.

-1. Replace these two lines with:

-

- ```json

- "actions": [

- "Microsoft.MachineLearningServices/workspaces/read",

- "Microsoft.MachineLearningServices/workspaces/labeling/projects/read",

- "Microsoft.MachineLearningServices/workspaces/labeling/projects/summary/read",

- "Microsoft.MachineLearningServices/workspaces/labeling/labels/read",

- "Microsoft.MachineLearningServices/workspaces/labeling/labels/write"

- ],

- "notActions": [

- ],

- ```

-### Labeling team lead

-You may want to create a second role for a labeling team lead. A labeling team lead can reject the labeled dataset and view labeling insights. In addition, this role also allows you to perform the role of a labeler.

-To add this custom role, repeat the above steps. Use the name **Labeling Team Lead** and replace the two lines with:

-```json

- "actions": [

- "Microsoft.MachineLearningServices/workspaces/read",

- "Microsoft.MachineLearningServices/workspaces/labeling/labels/read",

- "Microsoft.MachineLearningServices/workspaces/labeling/labels/write",

- "Microsoft.MachineLearningServices/workspaces/labeling/labels/reject/action",

- "Microsoft.MachineLearningServices/workspaces/labeling/projects/read",

- "Microsoft.MachineLearningServices/workspaces/labeling/projects/summary/read"

- ],

- "notActions": [

- "Microsoft.MachineLearningServices/workspaces/labeling/projects/write",

- "Microsoft.MachineLearningServices/workspaces/labeling/projects/delete",

- "Microsoft.MachineLearningServices/workspaces/labeling/export/action"

- ],

-```

-With Azure Machine Learning, you can train your model on a variety of resources or environments, collectively referred to as [__compute targets__](v1/concept-azure-machine-learning-architecture.md#compute-targets). A compute target can be a local machine or a cloud resource, such as an Azure Machine Learning Compute, Azure HDInsight, or a remote virtual machine. You can also create compute targets for model deployment as described in ["Where and how to deploy your models"](how-to-deploy-and-where.md).

-* Once you have a trained model, learn [how and where to deploy models](how-to-deploy-and-where.md).

-1. From the list of options to run your experiment, select **Locally**.

-1. **First time use on Windows only**. When prompted to allow File Share, select **Yes**. When you enable file share it allows Docker to mount the directory containing your script to the container. Additionally, it also allows Docker to store the logs and outputs from your run in a temporary directory on your system.

-1. Log the IP address of the host that they are running on. You use the IP address to connect the debugger to the script.

-and set `pip_packages=['debugpy', 'azureml-sdk==<SDK-VERSION>']`. Change the SDK version to match the one you are using. The following code snippet demonstrates how to create an environment:

-Save the `ip_address` value. It is used in the next section.

-In some cases, you may need to interactively debug the Python code contained in your model deployment. For example, if the entry script is failing and the reason cannot be determined by additional logging. By using VS Code and the debugpy, you can attach to the code running inside the Docker container.

-Local web service deployments require a working Docker installation on your local system. For more information on using Docker, see the [Docker Documentation](https://docs.docker.com/). Note that when working with compute instances, Docker is already installed.

- Once the image has been created and downloaded (this process may take more than 10 minutes, so please wait patiently), the image path (includes repository, name, and tag, which in this case is also its digest) is finally displayed in a message similar to the following:

- This attaches your `score.py` locally to the one in the container. Therefore, any changes made in the editor are automatically reflected in the container

-2. For a better experience, you can go into the container with a new VS code interface. Select the `Docker` extention from the VS Code side bar, find your local container created, in this documentation it's `debug:1`. Right-click this container and select `"Attach Visual Studio Code"`, then a new VS Code interface will be opened automatically, and this interface shows the inside of your created container.

-4. To attach VS Code to debugpy inside the container, open VS Code and use the F5 key or select __Debug__. When prompted, select the __Azure Machine Learning Deployment: Docker Debug__ configuration. You can also select the __Run__ extention icon from the side bar, the __Azure Machine Learning Deployment: Docker Debug__ entry from the Debug dropdown menu, and then use the green arrow to attach the debugger.

- After clicking the green arrow and attaching the debugger, in the container VS Code interface you can see some new information:

-* [Local model deployment](how-to-troubleshoot-deployment-local.md)

-* [Remote model deployment](how-to-troubleshoot-deployment.md)

-> - While Batch endpoints created with earlier APIs will continue to support V1 FileDataset, we will be adding further V2 data assets support with the latest API versions for even more usability and flexibility. For more information on V2 data assets, see [Work with data using SDK v2 (preview)](how-to-use-data.md). For more information on the new V2 experience, see [What is v2](concept-v2.md).

-You can generate the code for automated ML experiments with task types classification, regression, and time-series forecasting.

-> [!WARNING]

-> Computer vision models and natural language processing based models in AutoML do not currently support model's training code generation.

-The following diagram illustrates that you can enable code generation for any AutoML created model from the Azure Machine Learning studio UI or with the Azure Machine Learning SDK. First select a model. The model you selected will be highlighted, then Azure Machine Learning copies the code files used to create the model, and displays them into your notebooks shared folder. From here, you can view and customize the code as needed.

-In this article, you learn how to create and manage Azure Machine Learning workspaces using the Azure CLI. The Azure CLI provides commands for managing Azure resources and is designed to get you working quickly with Azure, with an emphasis on automation. The machine learning extension to the CLI provides commands for working with Azure Machine Learning resources.

-> [!NOTE]

-> Examples in this article refer to both CLI v1 and CLI v2 versions. If no version is specified for a command, it will work with either the v1 or CLI v2.

-* An **Azure subscription**. If you do not have one, try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/free/).

-# [CLI v1](#tab/vnetpleconfigurationsv1cli)

-With the Azure Machine Learning CLI extension v1 (`azure-cli-ml`), only some of the commands communicate with the Azure Resource Manager. Specifically, commands that create, update, delete, list, or show Azure resources. Operations such as submitting a training job communicate directly with the Azure Machine Learning workspace. **If your workspace is [secured with a private endpoint](how-to-configure-private-link.md), that is enough to secure commands provided by the `azure-cli-ml` extension**.

-# [CLI v2](#tab/vnetpleconfigurationsv2cli)

-With the Azure Machine Learning CLI extension v2 ('ml'), all of the commands communicate with the Azure Resource Manager. This includes operational data such as YAML parameters and metadata. If your Azure Machine Learning workspace is public (that is, not behind a virtual network), then there is no additional configuration required. Communications are secured using HTTPS/TLS 1.2.

-If your Azure Machine Learning workspace uses a private endpoint and virtual network and you are using CLI v2, choose one of the following configurations to use:

-* If you are __OK__ with the CLI v2 communication over the public internet, use the following `--public-network-access` parameter for the `az ml workspace update` command to enable public network access. For example, the following command updates a workspace for public network access:

-# [Bring existing resources (CLI v1)](#tab/bringexistingresources1)

-To create a workspace that uses existing resources, you must provide the resource ID for each resource. You can get this ID either via the 'properties' tab on each resource via the Azure portal, or by running the following commands using the Azure CLI.

- * **Azure Storage Account**:

- `az storage account show --name <storage-account-name> --query "id"`

- * **Azure Application Insights**:

- `az monitor app-insights component show --app <application-insight-name> -g <resource-group-name> --query "id"`

- * **Azure Key Vault**:

- `az keyvault show --name <key-vault-name> --query "ID"`

- * **Azure Container Registry**:

- `az acr show --name <acr-name> -g <resource-group-name> --query "id"`

- The returned resource ID has the following format: `"/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/<provider>/<subresource>/<resource-name>"`.

-Once you have the IDs for the resource(s) that you want to use with the workspace, use the base `az workspace create -w <workspace-name> -g <resource-group-name>` command and add the parameter(s) and ID(s) for the existing resources. For example, the following command creates a workspace that uses an existing container registry:

-```azurecli-interactive

-az ml workspace create -w <workspace-name>

- -g <resource-group-name>

- --container-registry "/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/Microsoft.ContainerRegistry/registries/<acr-name>"

-```

-# [Bring existing resources (CLI v2)](#tab/bringexistingresources2)

-To create a new workspace while bringing existing associated resources using the CLI, you will first have to define how your workspace should be configured in a configuration file.

-The Resource ID value looks similar to the following: `"/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/<provider>/<subresource>/<resource-name>"`.

-# [CLI v1](#tab/vnetpleconfigurationsv1cli)

-If you want to restrict access to your workspace to a virtual network, you can use the following parameters as part of the `az ml workspace create` command or use the `az ml workspace private-endpoint` commands.

-```azurecli-interactive

-az ml workspace create -w <workspace-name>

- -g <resource-group-name>

- --pe-name "<pe name>"

- --pe-auto-approval "<pe-autoapproval>"

- --pe-resource-group "<pe name>"

- --pe-vnet-name "<pe name>"

- --pe-subnet-name "<pe name>"

-```

-* `--pe-name`: The name of the private endpoint that is created.

-* `--pe-auto-approval`: Whether private endpoint connections to the workspace should be automatically approved.

-* `--pe-resource-group`: The resource group to create the private endpoint in. Must be the same group that contains the virtual network.

-* `--pe-vnet-name`: The existing virtual network to create the private endpoint in.

-* `--pe-subnet-name`: The name of the subnet to create the private endpoint in. The default value is `default`.

-For more details on how to use these commands, see the [CLI reference pages](/cli/azure/ml(v1)/workspace).

-# [CLI v2](#tab/vnetpleconfigurationsv2cli)

-When using private link, your workspace cannot use Azure Container Registry to build docker images. Hence, you must set the image_build_compute property to a CPU compute cluster name to use for Docker image environment building. You can also specify whether the private link workspace should be accessible over the internet using the public_network_access property.

-By default, metadata for the workspace is stored in an Azure Cosmos DB instance that Microsoft maintains. This data is encrypted using Microsoft-managed keys. Instead of using the Microsoft-managed key, you can also provide your own key. Doing so creates an additional set of resources in your Azure subscription to store your data.

-Below CLI commands provide examples for creating a workspace that uses customer-managed keys for encryption using the CLI v1 and CLI v2 versions.

-# [CLI v1](#tab/vnetpleconfigurationsv1cli)

-Use the `--cmk-keyvault` parameter to specify the Azure Key Vault that contains the key, and `--resource-cmk-uri` to specify the resource ID and uri of the key within the vault.

-To [limit the data that Microsoft collects](./concept-data-encryption.md#encryption-at-rest) on your workspace, you can additionally specify the `--hbi-workspace` parameter.

-```azurecli-interactive

-az ml workspace create -w <workspace-name>

- -g <resource-group-name>

- --cmk-keyvault "<cmk keyvault name>"

- --resource-cmk-uri "<resource cmk uri>"

- --hbi-workspace

-```

-# [CLI v2](#tab/vnetpleconfigurationsv2cli)

-# [CLI v1](#tab/workspaceupdatev1)

-```azurecli-interactive

-az ml workspace show -w <workspace-name> -g <resource-group-name>

-```

-# [CLI v2](#tab/workspaceupdatev2)

-# [CLI v1](#tab/workspaceupdatev1)

-```azurecli-interactive

-az ml workspace update -w <workspace-name> -g <resource-group-name>

-```

-# [CLI v2](#tab/workspaceupdatev2)

-# [CLI v1](#tab/workspacesynckeysv1)

-```azurecli-interactive

-az ml workspace sync-keys -w <workspace-name> -g <resource-group-name>

-```

-# [CLI v2](#tab/workspacesynckeysv2)

-To delete a workspace after it is no longer needed, use the following command:

-# [CLI v1](#tab/workspacedeletev1)

-```azurecli-interactive

-az ml workspace delete -w <workspace-name> -g <resource-group-name>

-```

-# [CLI v2](#tab/workspacedeletev2)

-If you accidentally deleted your workspace, are still able to retrieve your notebooks. Please refer to [this documentation](./how-to-high-availability-machine-learning.md#workspace-deletion).

-* **Monitor model data** for differences between training and serving datasets. Start by [collecting model data from deployed models](how-to-enable-data-collection.md).

-* See how to set up data drift on [models deployed to Azure Kubernetes Service](./how-to-enable-data-collection.md).

-* Set up dataset drift monitors with [event grid](how-to-use-event-grid.md).

-* Refer to the [Load registered model](how-to-deploy-advanced-entry-script.md#load-registered-models) docs. When you register a model directory, don't include your scoring script, your mounted dependencies directory, or `requirements.txt` within that directory.

-For example, if the both the requirements.txt and score script is in **my_folder**, then `AZUREML_EXTRA_REQUIREMENTS_TXT` will need to be set to requirements.txt. No longer will `AZUREML_EXTRA_REQUIREMENTS_TXT` be set to **my_folder/requirements.txt**.

-To learn more about deploying a model, see [How to deploy a model](how-to-deploy-and-where.md).

-* Once you have a trained model, learn [how and where to deploy models](how-to-deploy-and-where.md).

-* __Azure CLI session__: You use an active Azure CLI session to authenticate. The Azure CLI extension for Machine Learning (the `ml` extension or CLI v2) is a command line tool for working with Azure Machine Learning. You can log in to Azure via the Azure CLI on your local workstation, without storing credentials in Python code or prompting the user to authenticate. Similarly, you can reuse the same scripts as part of continuous integration and deployment pipelines, while authenticating the Azure CLI with a service principal identity.

-* Node.js - [How to migrate a Node.js app from ADAL to MSAL](../active-directory/develop/msal-node-migration.md).

-* Python - [ADAL to MSAL migration guide for Python](../active-directory/develop/migrate-python-adal-msal.md).

-* [How to configure authentication for models deployed as a web service](how-to-authenticate-web-service.md).

-* [Consume an Azure Machine Learning model deployed as a web service](how-to-consume-web-service.md).

-* Monitor your production models for [data drift](./how-to-enable-data-collection.md).

-If model deployment fails, you won't see logs in [Azure Machine Learning Studio](https://ml.azure.com/) and `service.get_logs()` will return None.

-For problems when deploying a model from Azure Machine Learning to Azure Container Instances (ACI) or Azure Kubernetes Service (AKS), see [Troubleshoot model deployment](how-to-troubleshoot-deployment.md).

-* If the `ENTRYPOINT` has been changed in the new built image, then the HTTP server and related components needs to be loaded by `runsvdir /var/runit`

-* [Learn how to consume a web service](how-to-consume-web-service.md).

-> - While Batch endpoints created with earlier APIs will continue to support V1 FileDataset, we will be adding further V2 data assets support with the latest API versions for even more usability and flexibility. For more information on V2 data assets, see [Work with data using SDK v2 (preview)](how-to-use-data.md). For more information on the new V2 experience, see [What is v2](concept-v2.md).

-description: 'Learn to how work with data using the Python SDK v2 preview for Azure Machine Learning.'

-# Work with data using SDK v2 preview

-Azure Machine Learning allows you to work with different types of data. In this article, you'll learn about using the Python SDK v2 to work with _URIs_ and _Tables_. URIs reference a location either local to your development environment or in the cloud. Tables are a tabular data abstraction.

-For most scenarios, you'll use URIs (`uri_folder` and `uri_file`). A URI references a location in storage that can be easily mapped to the filesystem of a compute node when you run a job. The data is accessed by either mounting or downloading the storage to the node.

-When using tables, you'll use `mltable`. It's an abstraction for tabular data that is used for AutoML jobs, parallel jobs, and some advanced scenarios. If you're just starting to use Azure Machine Learning, and aren't using AutoML, we strongly encourage you to begin with URIs.

-> [!TIP]

-> If you have dataset assets created using the SDK v1, you can still use those with SDK v2. For more information, see the [Consuming V1 Dataset Assets in V2](#consuming-v1-dataset-assets-in-v2) section.

-## Prerequisites

-* An Azure subscription - If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/free/) today.

-* An Azure Machine Learning workspace.

-* The Azure Machine Learning SDK v2 for Python

-## URIs

-The code snippets in this section cover the following scenarios:

-* Reading data in a job

-* Reading *and* writing data in a job

-* Registering data as an asset in Azure Machine Learning

-* Reading registered data assets from Azure Machine Learning in a job

-These snippets use `uri_file` and `uri_folder`.

-> [!TIP]

-> We recommend using an argument parser to pass folder information into _data-plane_ code. By data-plane code, we mean your data processing and/or training code that you run in the cloud. The code that runs in your development environment and submits code to the data-plane is _control-plane_ code.

->

-> Data-plane code is typically a Python script, but can be any programming language. Passing the folder as part of job submission allows you to easily adjust the path from training locally using local data, to training in the cloud. For example, the following example uses `argparse` to get a `uri_folder`, which is joined with the file name to form a path:

->

-> ```python

-> # train.py

-> import argparse

-> import os

-> import pandas as pd

->

-> parser = argparse.ArgumentParser()

-> parser.add_argument("--input_folder", type=str)

-> args = parser.parse_args()

->

-> file_name = os.path.join(args.input_folder, "MY_CSV_FILE.csv")

-> df = pd.read_csv(file_name)

-> print(df.head(10))

-> # process data

-> # train a model

-> # etc

-> ```

->

-> If you wanted to pass in just an individual file rather than the entire folder you can use the `uri_file` type.

-Below are some common data access patterns that you can use in your *control-plane* code to submit a job to Azure Machine Learning:

-### Use data with a training job

-Use the tabs below to select where your data is located.

-# [Local data](#tab/use-local)

-When you pass local data, the data is automatically uploaded to cloud storage as part of the job submission.

-```python

-from azure.ai.ml import Input, command

-from azure.ai.ml.entities import Data

-from azure.ai.ml.constants import AssetTypes

-my_job_inputs = {

- "input_data": Input(

- path='./sample_data', # change to be your local directory

- type=AssetTypes.URI_FOLDER

- )

-}

-job = command(

- code="./src", # local path where the code is stored

- command='python train.py --input_folder ${{inputs.input_data}}',

- inputs=my_job_inputs,

- environment="AzureML-sklearn-0.24-ubuntu18.04-py37-cpu:9",

- compute="cpu-cluster"

-)

-#submit the command job

-returned_job = ml_client.create_or_update(job)

-#get a URL for the status of the job

-returned_job.services["Studio"].endpoint

-```

-# [ADLS Gen2](#tab/use-adls)

-```python

-from azure.ai.ml import Input, command

-from azure.ai.ml.entities import Data, CommandJob

-from azure.ai.ml.constants import AssetTypes

-# in this example we

-my_job_inputs = {

- "input_data": Input(

- path='abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>',

- type=AssetTypes.URI_FOLDER

- )

-}

-job = command(

- code="./src", # local path where the code is stored

- command='python train.py --input_folder ${{inputs.input_data}}',

- inputs=my_job_inputs,

- environment="AzureML-sklearn-0.24-ubuntu18.04-py37-cpu:9",

- compute="cpu-cluster"

-)

-#submit the command job

-returned_job = ml_client.create_or_update(job)

-#get a URL for the status of the job

-returned_job.services["Studio"].endpoint

-```

-# [Blob](#tab/use-blob)

-```python

-from azure.ai.ml import Input, command

-from azure.ai.ml.entities import Data, CommandJob

-from azure.ai.ml.constants import AssetTypes

-# in this example we

-my_job_inputs = {

- "input_data": Input(

- path='https://<account_name>.blob.core.windows.net/<container_name>/path',

- type=AssetTypes.URI_FOLDER

- )

-}

-job = command(

- code="./src", # local path where the code is stored

- command='python train.py --input_folder ${{inputs.input_data}}',

- inputs=my_job_inputs,

- environment="AzureML-sklearn-0.24-ubuntu18.04-py37-cpu:9",

- compute="cpu-cluster"

-)

-#submit the command job

-returned_job = ml_client.create_or_update(job)

-#get a URL for the status of the job

-returned_job.services["Studio"].endpoint

-```

-### Read and write data in a job

-Use the tabs below to select where your data is located.

-# [Blob](#tab/rw-blob)

-```python

-from azure.ai.ml import Input, command

-from azure.ai.ml.entities import Data, CommandJob, JobOutput

-from azure.ai.ml.constants import AssetTypes

-my_job_inputs = {

- "input_data": Input(

- path='https://<account_name>.blob.core.windows.net/<container_name>/path',

- type=AssetTypes.URI_FOLDER

- )

-}

-my_job_outputs = {

- "output_folder": JobOutput(

- path='https://<account_name>.blob.core.windows.net/<container_name>/path',

- type=AssetTypes.URI_FOLDER

- )

-}

-job = command(

- code="./src", #local path where the code is stored

- command='python pre-process.py --input_folder ${{inputs.input_data}} --output_folder ${{outputs.output_folder}}',

- inputs=my_job_inputs,

- outputs=my_job_outputs,

- environment="AzureML-sklearn-0.24-ubuntu18.04-py37-cpu:9",

- compute="cpu-cluster"

-)

-#submit the command job

-returned_job = ml_client.create_or_update(job)

-#get a URL for the status of the job

-returned_job.services["Studio"].endpoint

-```

-# [ADLS Gen2](#tab/rw-adls)

-```python

-from azure.ai.ml import Input, command

-from azure.ai.ml.entities import Data, CommandJob, JobOutput

-from azure.ai.ml.constants import AssetTypes

-my_job_inputs = {

- "input_data": Input(

- path='abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>',

- type=AssetTypes.URI_FOLDER

- )

-}

-my_job_outputs = {

- "output_folder": JobOutput(

- path='abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>',

- type=AssetTypes.URI_FOLDER

- )

-}

-job = command(

- code="./src", #local path where the code is stored

- command='python pre-process.py --input_folder ${{inputs.input_data}} --output_folder ${{outputs.output_folder}}',

- inputs=my_job_inputs,

- outputs=my_job_outputs,

- environment="AzureML-sklearn-0.24-ubuntu18.04-py37-cpu:9",

- compute="cpu-cluster"

-)

-#submit the command job

-returned_job = ml_client.create_or_update(job)

-#get a URL for the status of the job

-returned_job.services["Studio"].endpoint

-```

-### Register data assets

-```python

-from azure.ai.ml.entities import Data

-from azure.ai.ml.constants import AssetTypes

-# select one from:

-my_path = 'abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>' # adls gen2

-my_path = 'https://<account_name>.blob.core.windows.net/<container_name>/path' # blob

-my_data = Data(

- path=my_path,

- type=AssetTypes.URI_FOLDER,

- description="description here",

- name="a_name",

- version='1'

-)

-ml_client.data.create_or_update(my_data)

-```

-### Consume registered data assets in job

-```python

-from azure.ai.ml import Input, command

-from azure.ai.ml.entities import Data, Input, CommandJob

-from azure.ai.ml.constants import AssetTypes

-registered_data_asset = ml_client.data.get(name='titanic', version='1')

-my_job_inputs = {

- "input_data": Input(

- type=AssetTypes.URI_FOLDER,

- path=registered_data_asset.id

- )

-}

-job = command(

- code="./src",

- command='python read_data_asset.py --input_folder ${{inputs.input_data}}',

- inputs=my_job_inputs,

- environment="AzureML-sklearn-0.24-ubuntu18.04-py37-cpu:9",

- compute="cpu-cluster"

-)

-#submit the command job

-returned_job = ml_client.create_or_update(job)

-#get a URL for the status of the job

-returned_job.services["Studio"].endpoint

-```

-## Table

-An [MLTable](concept-data.md#mltable) is primarily an abstraction over tabular data, but it can also be used for some advanced scenarios involving multiple paths. The following YAML describes an MLTable:

-```yaml

-paths:

- - file: ./titanic.csv

-transformations:

- - read_delimited:

- delimiter: ','

- encoding: 'ascii'

- empty_as_string: false

- header: from_first_file

-```

-The contents of the MLTable file specify the underlying data location (here a local path) and also the transforms to perform on the underlying data before materializing into a pandas/spark/dask data frame. The important part here's that the MLTable-artifact doesn't have any absolute paths, making it *self-contained*. All the information stored in one folder; regardless of whether that folder is stored on your local drive or in your cloud drive or on a public http server.

-To consume the data in a job or interactive session, use `mltable`:

-```python

-import mltable

-tbl = mltable.load("./sample_data")

-df = tbl.to_pandas_dataframe()

-```

-For more information on the YAML file format, see [the MLTable file](how-to-create-register-data-assets.md#the-mltable-file).

-<!-- Commenting until notebook is published. For a full example of using an MLTable, see the [Working with MLTable notebook]. -->

-## Consuming V1 dataset assets in V2

-> [!NOTE]

-> While full backward compatibility is provided, if your intention with your V1 `FileDataset` assets was to have a single path to a file or folder with no loading transforms (sample, take, filter, etc.), then we recommend that you re-create them as a `uri_file`/`uri_folder` using the v2 CLI:

->

-> ```cli

-> az ml data create --file my-data-asset.yaml

-> ```

-Registered v1 `FileDataset` and `TabularDataset` data assets can be consumed in an v2 job using `mltable`. To use the v1 assets, add the following definition in the `inputs` section of your job yaml:

-```yaml

-inputs:

- my_v1_dataset:

- type: mltable

- path: azureml:myv1ds:1

- mode: eval_mount

-```

-The following example shows how to do this using the v2 SDK:

-```python

-from azure.ai.ml import Input, command

-from azure.ai.ml.entities import Data, CommandJob

-from azure.ai.ml.constants import AssetTypes

-registered_v1_data_asset = ml_client.data.get(name='<ASSET NAME>', version='<VERSION NUMBER>')

-my_job_inputs = {

- "input_data": Input(

- type=AssetTypes.MLTABLE,

- path=registered_v1_data_asset.id,

- mode="eval_mount"

- )

-}

-job = command(

- code="./src", #local path where the code is stored

- command='python train.py --input_data ${{inputs.input_data}}',

- inputs=my_job_inputs,

- environment="AzureML-sklearn-0.24-ubuntu18.04-py37-cpu:9",

- compute="cpu-cluster"

-)

-#submit the command job

-returned_job = ml_client.jobs.create_or_update(job)

-#get a URL for the status of the job

-returned_job.services["Studio"].endpoint

-```

-## Next steps

-* [Install and set up Python SDK v2 (preview)](https://aka.ms/sdk-v2-install)

-* [Train models with the Python SDK v2 (preview)](how-to-train-sdk.md)

-* [Tutorial: Create production ML pipelines with Python SDK v2 (preview)](tutorial-pipeline-python-sdk.md)

-description: Learn how to integrate pipeline endpoints with client applications in Azure Machine Learning as part of migrating from Machine Learning Studio (Classic).

-In this article, you learn how to integrate client applications with Azure Machine Learning endpoints. For more information on writing application code, see [Consume an Azure Machine Learning endpoint](how-to-consume-web-service.md).

-In this article, you learned how to find schema and sample code for your pipeline endpoints. For more information on consuming endpoints from the client application, see [Consume an Azure Machine Learning endpoint](how-to-consume-web-service.md).

-| | | | | | |

-| V.Small | DS2 v2 | F2s v2 | E2s v3 | NC4as_T4_v3 |

-**Name**: AzureML-sklearn-1.0-ubuntu20.04-py38-cpu

-Version updates for supported environments, including the base images they reference, are released every two weeks to address vulnerabilities no older than 30 days. Based on usage, some environments may be deprecated (hidden from the product but usable) to support more common machine learning scenarios.

-> [How to deploy models with Azure Machine Learning](how-to-deploy-and-where.md).

-For more information on consuming your web service, see [Consume a model deployed as a webservice](how-to-consume-web-service.md).

-+ Learn about all of the [deployment options for Azure Machine Learning](how-to-deploy-and-where.md).

-+ Learn how to [create clients for the web service](how-to-consume-web-service.md).

-For more information about these components, see [Deploy models with Azure Machine Learning](../how-to-deploy-and-where.md).

-* For **DevOps** or **MLOps**, you can monitor information generated by models deployed as web services to identify problems with the deployments and gather data submitted to the service. For more information, see [Collect model data](../how-to-enable-data-collection.md) and [Monitor with Application Insights](../how-to-enable-app-insights.md).

-* [Tutorial: Train and deploy a model](../tutorial-train-deploy-notebook.md)

-For more information, see [Deploy models](../how-to-deploy-and-where.md#registermodel) and [Troubleshooting deployments](../how-to-troubleshoot-deployment.md).

-For more information, see [How to enable model data collection](../how-to-enable-data-collection.md).

-+ Create clients that [consume a deployed model](../how-to-consume-web-service.md)

-With Azure Machine Learning, you can train your model on various resources or environments, collectively referred to as [__compute targets__](concept-azure-machine-learning-architecture.md#compute-targets). A compute target can be a local machine or a cloud resource, such as an Azure Machine Learning Compute, Azure HDInsight, or a remote virtual machine. You also use compute targets for model deployment as described in ["Where and how to deploy your models"](../how-to-deploy-and-where.md).

-When you use your local computer for **inference**, you must have Docker installed. To perform the deployment, use [LocalWebservice.deploy_configuration()](/python/api/azureml-core/azureml.core.webservice.local.localwebservice#deploy-configuration-port-none-) to define the port that the web service will use. Then use the normal deployment process as described in [Deploy models with Azure Machine Learning](../how-to-deploy-and-where.md).

-* Once you have a trained model, learn [how and where to deploy models](../how-to-deploy-and-where.md).

-See the article on [client applications to consume web services](../how-to-consume-web-service.md) for more example clients in other languages.

-* [Troubleshoot a failed deployment](../how-to-troubleshoot-deployment.md)

-> It is highly advised to debug locally before deploying to the web service, for more information see [Debug Locally](../how-to-troubleshoot-deployment-local.md)

-* [Deployment troubleshooting](../how-to-troubleshoot-deployment.md)

-* [Update the web service](../how-to-deploy-update-web-service.md)

-* [Use TLS to secure a web service through Azure Machine Learning](../how-to-secure-web-service.md)

-* [Consume a ML Model deployed as a web service](../how-to-consume-web-service.md)

-* [Collect data for models in production](../how-to-enable-data-collection.md)

-> We recommend that you debug locally before deploying to the web service. For more information, see [Debug Locally](../how-to-troubleshoot-deployment-local.md)

- - If you want to deploy models to GPU nodes or FPGA nodes (or any specific SKU), then you must create a cluster with the specific SKU. There is no support for creating a secondary node pool in an existing cluster and deploying models in the secondary node pool.

- 1. If it is not found, it looks for a match in the global ACR

- 1. If it is not found, the system builds a new image (which will be cached and pushed to the workspace ACR)

-The front-end component (azureml-fe) that routes incoming inference requests to deployed services automatically scales as needed. Scaling of azureml-fe is based on the AKS cluster purpose and size (number of nodes). The cluster purpose and nodes are configured when you [create or attach an AKS cluster](../how-to-create-attach-kubernetes.md). There is one azureml-fe service per cluster, which may be running on multiple pods.

-When scale-up or scale-down, azureml-fe pods will be restarted to apply the cpu/memory changes. Inferencing requests are not affected by the restarts.

-| `<leaf-domain-label + auto-generated suffix>.<region>.cloudapp.azure.com` | Endpoint domain name, if you autogenerated by Azure Machine Learning. If you used a custom domain name, you do not need this entry. |

-* Query AKS API server to discover other instances of itself (it is a multi-pod service)

-After the model is deployed and service starts, azureml-fe will automatically discover it using AKS API and will be ready to route request to it. It must be able to communicate to model PODs.

-Decisions to scale up/down is based off of utilization of the current container replicas. The number of replicas that are busy (processing a request) divided by the total number of current replicas is the current utilization. If this number exceeds `autoscale_target_utilization`, then more replicas are created. If it is lower, then replicas are reduced. By default, the target utilization is 70%.

-For information on authenticating from a client application, see the [Consume an Azure Machine Learning model deployed as a web service](../how-to-consume-web-service.md).

-To enable token authentication, set the `token_auth_enabled=True` parameter when you are creating or updating a deployment. The following example enables token authentication using the SDK:

-* [Deployment troubleshooting](../how-to-troubleshoot-deployment.md)

-* [Consume a ML Model deployed as a web service](../how-to-consume-web-service.md)

-* [Collect data for models in production](../how-to-enable-data-collection.md)

-For more information on creating a client application, see [Create client to consume deployed web service](../how-to-consume-web-service.md).

-* Monitor your production models for [data drift](../how-to-enable-data-collection.md).

-* [Troubleshoot a failed deployment](../how-to-troubleshoot-deployment.md)

-* [Create client applications to consume web services](../how-to-consume-web-service.md)

-* [Update web service](../how-to-deploy-update-web-service.md)

-* [Use TLS to secure a web service through Azure Machine Learning](../how-to-secure-web-service.md)

-* [Collect data for models in production](../how-to-enable-data-collection.md)

-* [Troubleshoot a failed deployment](../how-to-troubleshoot-deployment.md)

-* [Create client applications to consume web services](../how-to-consume-web-service.md)

-* [Monitor your Azure Machine Learning models with Application Insights](../how-to-enable-app-insights.md)

-* [Collect data for models in production](../how-to-enable-data-collection.md)

-> Azure Application Insights only logs payloads of up to 64kb. If this limit is reached, you may see errors such as out of memory, or no information may be logged. If the data you want to log is larger 64kb, you should instead store it to blob storage using the information in [Collect Data for models in production](../how-to-enable-data-collection.md).