+- You must be eligible for or have an active assignment to the *Permissions Management Administrator* role as a user in that tenant.

+ 1. If you aren't already authenticated, sign in as a *Permissions Management Administrator* user.

+ 1. If needed, activate the *Permissions Management Administrator* role in your Azure AD tenant.

+ Title: View personal and organization information in Microsoft Entra Permissions Management

+description: How to view personal and organization information in the Account settings dashboard in Microsoft Entra Permissions Management.

+This information can't be modified because the user information is pulled from Azure AD only **User Session Time(min)**.

+- From the Permissions Management home page, select the down arrow to the right of the **User** (your initials) menu, then select **Account Settings**.

+ The **Personal Information** box displays your **First Name**, **Last Name**, and the **Email Address** used to register your account with Permissions Management.

+1. From the Permissions Management home page, select the down arrow to the right of the **User** (your initials) menu, then select **Account Settings**.

+1. To change duration of the **User Session Timeout (min)**, select **Edit** (the pencil icon), then enter the number of minutes before you want a user session to time out.

+1. Select the checkmark to confirm your new setting.

+ Title: Filter and query user activity in Microsoft Entra Permissions Management

+description: How to filter and query user activity in Microsoft Entra Permissions Management.

+ You can open a maximum number of six query tab pages at the same time. A message appears when you've reached the maximum.

+ - **Task Name**: The name of the task performed by the identity.

+ - **Schedule**: Displays how often a report is generated. You can schedule a one-time report or a monthly report.

+ Title: View data about the activity in your authorization system

+description: How to view data about the activity in your authorization system in the Microsoft Entra Permissions Management Dashboard.

+The Permissions Management **Dashboard** provides an overview of the authorization system and account activity being monitored. Use this dashboard to view data collected from your Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) authorization systems.

+1. From the Permissions Management home page, select **Dashboard**.

+description: How to view current billable resources in your authorization system in Microsoft Entra Permissions Management.

+Here's the current list of resources per cloud provider. This list is subject to change as cloud providers add more services in the future.

+- To view an inventory of created resources and licensing information for your authorization system, see [Display an inventory of created resources and licenses for your authorization system](product-data-inventory.md)

+ Title: Define and manage users, roles, and access levels in Microsoft Permissions Management

+description: How to define and manage users, roles, and access levels in the Permissions Management User management dashboard.

+Inviting a user to Permissions Management adds the user to the system and allows system administrators to assign permissions to those users. Follow the steps to invite a user to Permissions Management.

+- To view user management information, see [Manage users with the User management dashboard](ui-user-management.md).

+- To create group-based permissions, see [Create group-based permissions](how-to-create-group-based-permissions.md).

+1. Under **Template details**, enter *CustomAuthenticationExtensionsAPI* for the **New Function** property.

+| Add 50,000 users to at least two groups | Not currently available |

+## Next steps

+- Learn how to [Use role-based access control in your web application](how-to-web-app-role-based-access-control.md).

+ Title: Use role-based access control in your Node.js web application

+description: Learn how to configure groups and user roles in your customer's tenant, so you can receive them as claims in a security token for your Node.js application

+# Use role-based access control in your Node.js web application

+Role-based access control (RBAC) is a mechanism to enforce authorization in applications. Azure Active Directory (Azure AD) for customers allows you to define application roles for your application and assign those roles to users and groups. The roles you assign to a user or group define their level of access to the resources and operations in your application. When Azure AD for customers issues a security token for an authenticated user, it includes the names of the roles you've assigned the user or group in the security token's roles claim.

+You can also configure your Azure AD for customers tenant to return the group memberships of the user. Developers can then use security groups to implement RBAC in their applications, where the memberships of the user in specific groups are interpreted as their role memberships.

+Once you assign users and groups to roles, the *roles* claim is emitted in your security token. However, to emit the *groups* membership claim in security tokens, you need additional configuration in your customer's tenant.

+In this article, you learn how to receive user roles or group membership or both as claims in a security token for your Node.js web app.

+## Prerequisites

+- A security group in your customer's tenant. If you've not done so, [create one](../../roles/groups-create-eligible.md#azure-portal).

+- If you've not done so, complete the steps in [Using role-based access control for applications](how-to-use-app-roles-customers.md) article. This article shows you how to create roles for your application, how to assign users and groups to those roles, how to add members to a group and how to add a group claim to a to security token. Learn more about [ID tokens](../../develop/id-tokens.md) and [access tokens](../../develop/access-tokens.md).

+- If you've not done so, complete the steps in [Sign in users in your own Node.js web application](how-to-web-app-node-sign-in-overview.md)

+## Receive groups and roles claims in your Node.js web app

+Once you configure your customer's tenant, you can retrieve your *roles* and *groups* claims in your client app. The *roles* and *groups* claims are both present in the ID token and the access token, but your client app only needs to check for these claims in the ID token to implement authorization in the client side. The API app can also retrieve these claims when it receives the access token.

+You check your *roles* claim value as shown in the following code snippet example:

+```javascript

+const msal = require('@azure/msal-node');

+const { msalConfig, TENANT_SUBDOMAIN, REDIRECT_URI, POST_LOGOUT_REDIRECT_URI } = require('../authConfig');

+...

+class AuthProvider {

+...

+ async handleRedirect(req, res, next) {

+ const authCodeRequest = {

+ ...req.session.authCodeRequest,

+ code: req.body.code, // authZ code

+ codeVerifier: req.session.pkceCodes.verifier, // PKCE Code Verifier

+ };

+

+ try {

+ const msalInstance = this.getMsalInstance(this.config.msalConfig);

+ const tokenResponse = await msalInstance.acquireTokenByCode(authCodeRequest, req.body);

+ let roles = tokenResponse.idTokenClaims.roles;

+

+ //Check roles

+ if (roles && roles.includes("Orders.Manager")) {

+ //This user can view the ID token claims page.

+ res.redirect('/id');

+ }

+

+ //User can only view the index page.

+ res.redirect('/');

+ } catch (error) {

+ next(error);

+ }

+ }

+...

+}

+```

+If you assign a user to multiple roles, the `roles` string contains all roles separated by a comma, such as `Orders.Manager,Store.Manager,...`. Make sure you build your application to handle the following conditions:

+- absence of `roles` claim in the token

+- user hasn't been assigned to any role

+- multiple values in the `roles` claim when you assign a user to multiple roles

+You can also check your *groups* claim value as shown in the following code snippet example:

+```javascript

+const tokenResponse = await msalInstance.acquireTokenByCode(authCodeRequest, req.body);

+let groups = tokenResponse.idTokenClaims.groups;

+```

+The groups claim value is the group's *objectId*. If a user is a member of multiple groups, the `groups` string contains all groups separated by a comma, such as `7f0621bc-b758-44fa-a2c6-...,6b35e65d-f3c8-4c6e-9538-...`.

+> [!NOTE]

+> If you assign a user [Azure AD in-built roles](../../roles/permissions-reference.md) or commonly known as directory roles, those roles appear in the *groups* claim of the security token.

+## Handle groups overage

+To ensure that the size of the security token doesnΓÇÖt exceed the HTTP header size limit, Azure AD for customers limits the number of object IDs that it includes in the *groups* claim. The overage limit is **150 for SAML tokens and 200 for JWT tokens**. It's possible to exceed this limit if a user belongs to many groups, and you request for all the groups.

+### Detect group overage in your source code

+If you can't avoid groups overages, then you need to handle it in your code. When you exceed the overage limit, the token doesn't contain the *groups* claim. Instead, the token contains a *_claim_names* claim that contains a *groups* member of the array. So, you need to check the existence of *_claim_names* claim to tell that an overage has occurred. The following code snippet shows you how to detect a groups overage:

+```javascript

+const tokenResponse = await msalInstance.acquireTokenByCode(authCodeRequest, req.body);

+if(tokenResponse.idTokenClaims.hasOwnProperty('_claim_names') && tokenResponse.idTokenClaims['_claim_names'].hasOwnProperty('groups')) {

+ //overage has occurred

+}

+```

+Use the instructions in [Configuring group claims and app roles in tokens](/security/zero-trust/develop/configure-tokens-group-claims-app-roles#group-overages) article to learn how request for the full groups list when groups overage occurs.

+## How to use groups and roles values in your Node.js web app

+In the client app, you can verify whether a signed-in user has the necessary role(s) to access a protected route or call an API endpoint. This can be done by checking the `roles` claim in the ID token. To implement this protection in your app, you can build guards by using a custom middleware.

+In your service app (API app), you can also protect the API endpoints. After you [validate the access token](../../develop/access-tokens.md#validate-tokens) sent by the client app, you can check for the *roles* or *groups* claims in the payload claims of the access token.

+## Do I use App Roles or Groups?

+In this article, you have learned that you can use *App Roles* or *Groups* to implement RBAC in your application. The preferred approach is to use app roles as app roles provide more granular control when managing access/permissions at the application level. For more information on how to choose an approach, see [Choose an approach](../../develop/custom-rbac-for-developers.md#choose-an-approach).

+## Next steps

+- Learn more about [Configuring group claims and app roles in tokens](/security/zero-trust/develop/configure-tokens-group-claims-app-roles).

+## Create a dedicated tenant for your customer scenarios

+When getting started with Azure AD for customers, you first create a tenant that will contain your customer-facing apps, resources, and directory of customer accounts.

+If you've worked with Azure Active Directory, you're already familiar with using an Azure AD tenant that contains your employee directory, internal apps, and other organizational resources. With Azure AD for customers, you create a distinct tenant that follows the standard Azure AD tenant model but is configured for customer scenarios. This tenant contains:

+There are two types of user accounts you can manage in your customer tenant:

+## Add customized sign-in to your customer-facing apps

+Azure AD for customers is intended for businesses that want to make applications available to their customers using the Microsoft Entra platform for identity and access.

+- **Add sign-up and sign-in pages to your apps.** Quickly add intuitive, user-friendly sign-up and sign-up experiences for your customer apps. With a single identity, a customer can securely access all the applications you want them to use.

+- **Add single sign-on (SSO) with social and enterprise identities.** Customers can choose a social, enterprise, or managed identity to sign in with a username and password, email, or one-time passcode.

+- **Add your company branding to the sign-up page.** Customize the look and feel of your sign-up and sign-in experiences, including both the default experience and the experience for specific browser languages.

+- **Easily customize and extend your sign-up flows.** Tailor your identity user flows to your needs. Choose the attributes you want to collect from a customer during sign-up, or add your own custom attributes. If the information your app needs is contained in an external system, create custom authentication extensions to collect and add data to authentication tokens.

+- **Integrate multiple app languages and platforms.** With Microsoft Entra, you can quickly set up and deliver secure, branded authentication flows for multiple app types, platforms, and languages.

+- **Provide self-service account management.** Customers can register for your online services by themselves, manage their profile, delete their account, enroll in a multifactor authentication (MFA) method, or reset their password with no admin or help desk assistance.

+Learn more about [adding sign-in and sign-up to your app](concept-planning-your-solution.md) and [customizing the sign-in look and feel](concept-branding-customers.md).

+|[Azure AD Graph API](https://aka.ms/aadgraphupdate)|Retirement|Jun 30, 2023|

+|[Azure AD PowerShell and MSOnline PowerShell](https://aka.ms/aadgraphupdate)|Deprecation|Mar 30, 2024|

+|[Microsoft Authenticator Lite for Outlook mobile](../../active-directory/authentication/how-to-mfa-authenticator-lite.md)|Feature change|Jun 9, 2023|

+You can have at most one automatic assignment policy in an access package, and the policy can only be created by an administrator. (Catalog owners and access package managers cannot create automatic assignment policies.)

+You'll need to have attributes populated on the users who will be in scope for being assigned access. The attributes you can use in the rules criteria of an access package assignment policy are those attributes listed in [supported properties](../enterprise-users/groups-dynamic-membership.md#supported-properties), along with [extension attributes and custom extension properties](../enterprise-users/groups-dynamic-membership.md#extension-properties-and-custom-extension-properties). These attributes can be brought into Azure AD from [Graph](/graph/api/resources/user), an HR system such as [SuccessFactors](../app-provisioning/sap-successfactors-integration-reference.md), [Azure AD Connect cloud sync](../cloud-sync/how-to-attribute-mapping.md) or [Azure AD Connect sync](../hybrid/how-to-connect-sync-feature-directory-extensions.md). The rules can include up to 5000 users per policy.

+1. Azure AD will evaluate the users in the organization that are in scope of this rule, and create assignments for those users who don't already have assignments to the access package. A policy can include at most 5000 users in its rule. It may take several minutes for the evaluation to occur, or for subsequent updates to user's attributes to be reflected in the access package assignments.

+## Automatic assignment policies

+* Each automatic assignment policy can include at most 5000 users in scope of its rule. Additional users in scope of the rule may not be assigned access.

+ Title: Common sign-in diagnostics AD scenarios

+description: Learn about the scenarios supported by the sign-in diagnostics for Azure AD.

+<a name="supported-scenarios"></a>

+You can use the sign-in diagnostic for Azure Active Directory (Azure AD) to analyze what happened during a sign-in attempt and get recommendations for resolving problems without needing to involve Microsoft support.

+## How to access the Sign-in Diagnostics

+There are three ways to access the Sign-in Diagnostics tool: from the Diagnose and solve problems area, the Azure AD sign-in logs, and when creating a new support request. For more information, see [How to use the sign-in diagnostics](howto-use-sign-in-diagnostics.md).

+## Conditional Access

+Conditional Access policies are used to apply the right access controls when needed to keep your organization secure. Because Conditional Access policies can be used to grant or block access to resources, they often show up in the sign-in diagnostic.

+- [Blocked by Conditional access](../conditional-access/concept-conditional-access-grant.md#block-access)

+ - Your Conditional Access policies prevented the user from signing in.

+- [Failed Conditional access](../conditional-access/troubleshoot-conditional-access.md#select-all-consequences):

+ - It's possible your Conditional Access policies are too strict.

+ - Review your configurations for complete sets of users, groups, and apps.

+ - Make sure you understand the implications of restricting access from certain types of devices.

+- [Multi-factor authentication (MFA) from Conditional access](../conditional-access/concept-conditional-access-grant.md#require-multifactor-authentication):

+ - Your Conditional Access policies triggered the MFA process for the user.

+- [B2B blocked sign-in due to Conditional Access](../external-identities/authentication-conditional-access.md#conditional-access-for-external-users):

+ - You have a Conditional Access policy in place to block external identities from signing in.

+## Multi-factor authentication

+### MFA from other requirements

+If the sign-in diagnostic results showed MFA from a requirement other than Conditional Access, you may have MFA enabled on a per-user basis. We [recommend converting per-user MFA to Conditional Access](recommendation-turn-off-per-user-mfa.md). The sign-in diagnostic provides details around the source of the MFA interruption and the result of the interaction.

+### MFA "proofup"

+Another common scenario occurs when MFA interrupts sign-in attempts. When you run the sign-in diagnostic, information about "proofup" is provided in the diagnostic results. This error appears when users are setting up MFA for the first time and don't complete the setup or their configuration wasn't set up ahead of time.

+

+## Correct & incorrect credentials

+### Successful sign-in

+In some cases, you want to know if sign-in events *aren't* interrupted by Conditional Access or MFA, but they *should* be. The sign-in diagnostic tool provides details about sign-in events that should be interrupted, but aren't.

+### Account locked

+Another common scenario is when a user attempts to sign in with incorrect credentials too many times. This error happens when too many password-based sign-in attempts have occurred with incorrect credentials. The diagnostic results provide information for the administrator to determine where the attempts are coming from and if they're legitimate user sign-in attempts or not. Running the sign-in diagnostic provides details about the apps, the number of attempts, the device used, the operating system, and the IP address. For more information, see [Azure AD Smart Lockout](../authentication/howto-password-smart-lockout.md).

+### Invalid username or password

+If a user tried to sign in using an invalid username or password, the sign-in diagnostic helps the administrator determine the source of the problem. The source could be a user entering incorrect credentials, or a client and/or application(s) that's cached an old password and is resubmitting it. The sign-in diagnostic provides details about the apps, the number of attempts, the device used, the operating system and the IP address.

+## Enterprise apps

+- The service provider (application service, also known as SaaS application) configuration

+Diagnostics for these problems address which side of the problem should be looked at for resolution and what to do

+If the error occurred when a user tried to sign in to an application, the sign-in failed due to a problem with the service provider (application) side of the sign-in flow. Problems detected by the sign-in diagnosis typically must be resolved by changing the configuration or fixing problems on the application service. Resolution for this scenario means you need to sign into the other service and changing some configuration per the diagnostic guidance.

+### Enterprise apps configuration

+Sign-in can fail due to an application configuration issue for the Azure AD side of the application. In these situations, resolution requires reviewing and updating the configuration of the application in the Enterprise Applications page for the application.

+Sign-in events can be interrupted due to security defaults settings. Security defaults enforce best practice security for your organization. One best practice is to require MFA to be configured and used to prevent password sprays, replay attacks, and phishing attempts from being successful.

+When an event doesn't have a contextual analysis in the sign-in diagnostic, an updated error code explanation and relevant content may be shown. The error code insights contain detailed text about the scenario, how to remediate the problem, and any content to read regarding the problem.

+This scenario involves a sign-in event that was blocked or interrupted because the client was attempting to use Legacy (or Basic) Authentication.

+Preventing legacy authentication sign-in is recommended as the best practice for security. Legacy authentication protocols like POP, SMTP, IMAP, and MAPI can't enforce MFA, which makes them preferred entry points for adversaries to attack your organization.

+### B2B blocked sign-in due to Conditional access

+This diagnostic scenario detects a blocked or interrupted sign-in due to the user being from another organization. For example, a B2B sign-in, where a Conditional Access policy requires that the client's device is joined to the resource tenant.

+This diagnostic scenario identifies user specific sign-in issues when the authentication method being used is pass through authentication (PTA) and there's a PTA specific error. Errors due to other problems-even when PTA authentication is being used-will still be diagnosed correctly.

+The diagnostic results show contextual information about the failure and the user signing in. The results could show other reasons why the sign-in failed, and recommended actions the admin can take to resolve the problem. For more information, see [Azure AD Connect: Troubleshoot Pass-through Authentication](../hybrid/tshoot-connect-pass-through-authentication.md).

+### Seamless single sign-on

+Seamless single sign-on integrates Kerberos authentication with cloud authentication. Because this scenario involves two authentication protocols, it can be difficult to understand where a failure point lies when sign-in problems occur. This diagnostic is intended to make these scenarios easier to diagnose and resolve.

+This diagnostic scenario examines the context of the sign-in failure and specific failure cause. The diagnostic results could include contextual information on the sign-in attempt, and suggested actions the admin can take. For more information, see [Troubleshoot Azure Active Directory Seamless single sign-on](../hybrid/tshoot-connect-sso.md).

+- [How to use the sign-in diagnostic](howto-use-sign-in-diagnostics.md)

+- [How to troubleshoot sign-in errors](howto-troubleshoot-sign-in-errors.md)

+ Title: How to troubleshoot sign-in errors

+The Azure Active Directory (Azure AD) sign-in logs enable you to find answers to questions around managing access to the applications in your organization, including:

+In addition, the sign-ins logs can also help you troubleshoot sign-in failures for users in your organization. In this guide, you learn how to isolate a sign-in failure in the sign-ins report, and use it to understand the root cause of the failure. Some common sign-in errors are also described.

+* An Azure AD tenant with a Premium P1/P2 license.

+* A user with the **Global Administrator**, **Security Administrator**, **Security Reader**, or **Reports Reader** role for the tenant.

+* In addition, any user can access their own sign-ins from https://mysignins.microsoft.com.

+## Gather sign-in details

+1. Sign in to the [Azure portal](https://portal.azure.com) using a role of least privilege access.

+1. Go to **Azure AD** > **Sign-ins**.

+1. Use the filters to narrow down the results

+ - Search by username if you're troubleshooting a specific user.

+ - Search by application if you're troubleshooting issues with a specific app.

+ - Select **Failure** from the **Status** menu to display only failed sign-ins.

+1. Select the failed sign-in you want to investigate to open the details window.

+1. Explore the details on each tab. You may want to save a few details for further troubleshooting. These details are highlighted in the screenshot following the list.

+ - Correlation ID

+ - Sign-in error code

+ - Failure reason

+ - Username, User ID, and Sign-in identifier

+ 

+

+## Troubleshoot sign-in errors

+With sign-in details gathered, you should explore the results and troubleshoot the issue.

+### Failure reason and additional details

+The **Failure reason** and **Additional Details** may provide you with the details and next steps to resolve the issue. The Failure reason describes the error. The Additional Details provides more details and often tells you how to resolve the issue.

+

+The following failure reasons and details are common:

+- The failure reason **Authentication failed during the strong authentication request** doesn't provide much to troubleshoot, but the additional details field says the user didn't complete the MFA prompt. Have the user sign-in again and complete the MFA prompts.

+- The failure reason **The Federation Service failed to issue an OAuth Primary Refresh Token** provides a good starting point, but the additional details briefly explain how authentication works in this scenario and tell you to make sure that device sync is enabled.

+- A common failure reason is **Error validating credentials due to invalid username or password**. The user entered something incorrectly and needs to try again.

+### Sign-in error codes

+If you need more specifics to research, you can use the **sign-in error code** for further research.

+- Enter the error code into the **[Error code lookup tool](https://login.microsoftonline.com/error)** to get the error code description and remediation information.

+- Search for an error code in the **[sign-ins error codes reference](../develop/reference-aadsts-error-codes.md)**.

+The following error codes are associated with sign-in events, but this list isn't exhaustive:

+- **50058**: User is authenticated but not yet signed in.

+ - This error code appears for sign-in attempts when the user didn't complete the sign-in process.

+ - Because the user didn't sign-in completely, the User field may display an Object ID or a globally unique identifier (GUID) instead of a username.

+ - In some of these situations, the User ID shows up like "00000000-0000-0000".

+- **90025**: An internal Azure AD service hit its retry allowance to sign the user in.

+ - This error often happens without the user noticing and is usually resolved automatically.

+ - If it persists, have the user sign in again.

+- **500121**: User didn't complete the MFA prompt.

+ - This error often appears if the user hasn't completed setting up MFA.

+ - Instruct the user to complete the setup process through to sign-in.

+If all else fails, or the issue persists despite taking the recommended course of action, [open a support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest). For more information, see [how to get support for Azure AD](../fundamentals/how-to-get-support.md).

+* [Sign-ins report overview](concept-sign-ins.md)

+* [How to use the Sign-in diagnostics](howto-use-sign-in-diagnostics.md)

+ Title: How to use the Sign-in diagnostic

+description: Information on how to use the Sign-in diagnostic in Azure Active Directory.

+# Customer intent: As an Azure AD administrator, I want a tool that gives me the right level of insights into the sign-in activities in my system so that I can easily diagnose and solve problems when they occur.

+# What is the Sign-in diagnostic in Azure AD?

+Determining the reason for a failed sign-in can quickly become a challenging task. You need to analyze what happened during the sign-in attempt, and research the available recommendations to resolve the issue. Ideally, you want to resolve the issue without involving others, such as Microsoft support. If you are in a situation like this, you can use the Sign-in diagnostic in Azure AD, a tool that helps you investigate sign-ins in Azure AD.

+This article gives you an overview of what the Sign-in diagnostic is and how you can use it to troubleshoot sign-in related errors.

+## How it works

+In Azure AD, sign-in attempts are controlled by:

+- **Who** performed a sign-in attempt.

+- **How** a sign-in attempt was performed.

+For example, you can configure Conditional Access policies that enable administrators to configure all aspects of the tenant when they sign in from the corporate network. But the same user might be blocked when they sign in to the same account from an untrusted network.

+Due to the greater flexibility of the system to respond to a sign-in attempt, you might end up in scenarios where you need to troubleshoot sign-ins. The Sign-in diagnostic tool enables diagnosis of sign-in issues by:

+- Analyzing data from sign-in events and flagged sign-ins.

+- Displaying information about what happened.

+- Providing recommendations to resolve problems.

+## How to access it

+To use the Sign-in diagnostic, you must be signed into the tenant as a **Global Reader** or **Global Administrator**. With the correct access level, you can start the Sign-in diagnostic from more than one place.

+Flagged sign-in events can also be reviewed from the Sign-in diagnostic. Flagged sign-in events are captured *after* a user has enabled flagging during their sign-in experience. For more information, see [flagged sign-ins](overview-flagged-sign-ins.md).

+### From Diagnose and Solve Problems

+You can start the Sign-in diagnostic from the **Diagnose and Solve Problems** area of Azure AD. From Diagnose and Solve Problems you can review any flagged sign-in events or search for a specific sign-in event. You can also start this process from the Conditional Access Diagnose and Solve Problems area.

+**To search for sign-in events**:

+1. Go to **Azure AD** or **Azure AD Conditional Access** > **Diagnose and Solve Problems**.

+1. Select the **All Sign-In Events** tab to start a search.

+1. Enter as many details as possible into the search fields.

+ - **User**: Provide the name or email address of who made the sign-in attempt.

+ - **Application**: Provide the application display name or application ID.

+ - **correlationId** or **requestId**: These details can be found in the error report or the sign-in log details.

+ - **Date and time**: Provide a date and time to find sign-in events that occurred within 48 hours.

+1. Select the **Next** button.

+1. Explore the results and take action as necessary.

+### From the Sign-in logs

+You can start the Sign-in diagnostic from a specific sign-in event in the Sign-in logs. When you start the process from a specific sign-in event, the diagnostics start right away. You aren't prompted to enter details first.

+1. Go to **Azure AD** > **Sign-in logs** and select a sign-in event.

+ - You can filter your list to make it easier to find specific sign-in events.

+1. From the Activity Details window that opens, select the **Launch the Sign-in diagnostic** link.

+ 

+1. Explore the results and take action as necessary.

+### From a support request

+If you're in the middle of creating a support request *and* the options you selected are related to sign-in activity, you'll be prompted to run the Sign-in diagnostics during the support request process.

+1. Go to **Azure AD** > **Diagnose and Solve Problems**.

+1. Select the appropriate fields as necessary. For example:

+ - **Service type**: Azure Active Directory Sign-in and Multi-Factor Authentication

+ - **Problem type**: Multi-Factor Authentication

+ - **Problem subtype**: Unable to sign-in to an application due to MFA

+1. Explore the results and take action as necessary.

+ 

+## How to use the diagnostic Results

+After the Sign-in diagnostic completes its search, a few things appear on the screen:

+- The **Authentication Summary** lists all of the events that match the details you provided.

+ - Select the **View Columns** option in the upper-right corner of the summary to change the columns that appear.

+- The **diagnostic Results** describe what happened during the sign-in events.

+ - Scenarios could include MFA requirements from a Conditional Access policy, sign-in events that may need to have a Conditional Access policy applied, or a large number of failed sign-in attempts over the past 48 hours.

+ - Related content and links to troubleshooting tools may be provided.

+ - Read through the results to identify any actions that you can take.

+ - Because it's not always possible to resolve issues without more help, a recommended step might be to open a support ticket.

+

+ 

+

+- Provide feedback on the results to help improve the feature.

+## Next steps

+- [Sign in diagnostics for Azure AD scenarios](concept-sign-in-diagnostics-scenarios.md)

+- [Learn about flagged sign-ins](overview-flagged-sign-ins.md)

+- [Troubleshoot sign-in errors](howto-troubleshoot-sign-in-errors.md)

+> Azure Active Directory integrates with Cerner Central using the SCIM protocol.

+> Azure Active Directory integrates with LinkedIn Elevate using the SCIM protocol.

+> Azure Active Directory integrates with LinkedIn Sales Navigator using the SCIM protocol.

+> [!NOTE]

+> Shrinking persistent volumes is currently not supported. Trying to patch an existing PVC with a smaller size than the current one leads to the following error message:

+> `The persistentVolumeClaim "pvc-azuredisk" is invalid: spec.resources.requests.storage: Forbidden: field can not be less than previous value.`

+#### Use an Azure AD workload identity

+[aks-keyvault-csi-driver]: csi-secrets-store-driver.md

+ Title: "Setup of Network Observability for Azure Kubernetes Service (AKS) - BYO Prometheus and Grafana"

+description: Get started with AKS Network Observability for your AKS cluster using BYO Prometheus and Grafana.

+# Setup of Network Observability for Azure Kubernetes Service (AKS) - BYO Prometheus and Grafana

+AKS Network Observability is used to collect the network traffic data of your AKS cluster. Network Observability enables a centralized platform for monitoring application and network health. Prometheus collects AKS Network Observability metrics, and Grafana visualizes them. Both Cilium and non-Cilium data plane are supported. In this article, learn how to enable the Network Observability add-on and use BYO Prometheus and Grafana to visualize the scraped metrics.

+For more information about AKS Network Observability, see [What is Azure Kubernetes Service (AKS) Network Observability?](network-observability-overview.md).

+## Prerequisites

+- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+- Installations of BYO Prometheus and Grafana.

+### Install the `aks-preview` Azure CLI extension

+```azurecli-interactive

+# Install the aks-preview extension

+az extension add --name aks-preview

+# Update the extension to make sure you have the latest version installed

+az extension update --name aks-preview

+```

+### Register the `NetworkObservabilityPreview` feature flag

+```azurecli-interactive

+az feature register --namespace "Microsoft.ContainerService" --name "NetworkObservabilityPreview"

+```

+Use [az feature show](/cli/azure/feature#az-feature-show) to check the registration status of the feature flag:

+```azurecli-interactive

+az feature show --namespace "Microsoft.ContainerService" --name "NetworkObservabilityPreview"

+```

+Wait for the feature to say **Registered** before preceding with the article.

+```output

+{

+ "id": "/subscriptions/23250d6d-28f0-41dd-9776-61fc80805b6e/providers/Microsoft.Features/providers/Microsoft.ContainerService/features/NetworkObservabilityPreview",

+ "name": "Microsoft.ContainerService/NetworkObservabilityPreview",

+ "properties": {

+ "state": "Registering"

+ },

+ "type": "Microsoft.Features/providers/features"

+}

+```

+When the feature is registered, refresh the registration of the Microsoft.ContainerService resource provider with [az provider register](/cli/azure/provider#az-provider-register):

+```azurecli-interactive

+az provider register -n Microsoft.ContainerService

+```

+## Create a resource group

+A resource group is a logical container into which Azure resources are deployed and managed. Create a resource group with [az group create](/cli/azure/group#az-group-create) command. The following example creates a resource group named **myResourceGroup** in the **eastus** location:

+```azurecli-interactive

+az group create \

+ --name myResourceGroup \

+ --location eastus

+```

+## Create AKS cluster

+Create an AKS cluster with [az aks create](/cli/azure/aks#az-aks-create) command. The following example creates an AKS cluster named **myAKSCluster** in the **myResourceGroup** resource group:

+# [**Non-Cilium**](#tab/non-cilium)

+Non-Cilium clusters support the enablement of Network Observability on an existing cluster or during the creation of a new cluster.

+## New cluster

+Use [az aks create](/cli/azure/aks#az-aks-create) in the following example to create an AKS cluster with Network Observability and non-Cilium.

+```azurecli-interactive

+az aks create \

+ --name myAKSCluster \

+ --resource-group myResourceGroup \

+ --location eastus \

+ --generate-ssh-keys \

+ --network-plugin azure \

+ --network-plugin-mode overlay \

+ --pod-cidr 192.168.0.0/16 \

+ --enable-network-observability

+```

+## Existing cluster

+Use [az aks update](/cli/azure/aks#az-aks-update) to enable Network Observability on an existing cluster.

+```azurecli-interactive

+az aks update \

+ --resource-group myResourceGroup \

+ --name myAKSCluster \

+ --enable-network-observability

+```

+# [**Cilium**](#tab/cilium)

+Use the following example to create an AKS cluster with Network Observability and Cilium.

+```azurecli-interactive

+az aks create \

+ --name myAKSCluster \

+ --resource-group myResourceGroup \

+ --generate-ssh-keys \

+ --location eastus \

+ --max-pods 250 \

+ --network-plugin azure \

+ --network-plugin-mode overlay \

+ --network-dataplane cilium \

+ --node-count 2 \

+ --pod-cidr 192.168.0.0/16

+```

+## Get cluster credentials

+```azurecli-interactive

+az aks get-credentials -name myAKSCluster --resource-group myResourceGroup

+```

+## Enable Visualization on Grafana

+Use the following example to configure scrape jobs on Prometheus and enable visualization on Grafana for your AKS cluster.

+# [**Non-Cilium**](#tab/non-cilium)

+> [!NOTE]

+> The following section requires installations of Prometheus and Grafana.

+1. Add the following scrape job to your existing Prometheus configuration and restart your Prometheus server:

+ ```yml

+ scrape_configs:

+ - job_name: "network-obs-pods"

+ kubernetes_sd_configs:

+ - role: pod

+ relabel_configs:

+ - source_labels: [__meta_kubernetes_pod_container_name]

+ action: keep

+ regex: kappie(.*)

+ - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]

+ separator: ":"

+ regex: ([^:]+)(?::\d+)?

+ target_label: __address__

+ replacement: ${1}:${2}

+ action: replace

+ - source_labels: [__meta_kubernetes_pod_node_name]

+ action: replace

+ target_label: instance

+ metric_relabel_configs:

+ - source_labels: [__name__]

+ action: keep

+ regex: (.*)

+ ```

+1. In **Targets** of Prometheus, verify the **network-obs-pods** are present.

+1. Sign in to Grafana and import Network Observability dashboard with ID [18814](https://grafana.com/grafana/dashboards/18814/).

+# [**Cilium**](#tab/cilium)

+> [!NOTE]

+> The following section requires installations of Prometheus and Grafana.

+1. Add the following scrape job to your existing Prometheus configuration and restart your prometheus server.

+ ```yml

+ scrape_configs:

+ - job_name: 'kubernetes-pods'

+ kubernetes_sd_configs:

+ - role: pod

+ relabel_configs:

+ - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]

+ action: keep

+ regex: true

+ - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]

+ action: replace

+ regex: (.+):(?:\d+);(\d+)

+ replacement: ${1}:${2}

+ target_label: __address__

+ ```

+1. In **Targets** of prometheus, verify the **kubernetes-pods** are present.

+1. Sign in to Grafana and import dashboards with the following ID [16611-cilium-metrics](https://grafana.com/grafana/dashboards/16611-cilium-metrics/)

+## Clean up resources

+If you're not going to continue to use this application, delete the AKS cluster and the other resources created in this article with the following example:

+```azurecli-interactive

+ az group delete \

+ --name myResourceGroup

+```

+## Next steps

+In this how-to article, you learned how to install and enable AKS Network Observability for your AKS cluster.

+- For more information about AKS Network Observability, see [What is Azure Kubernetes Service (AKS) Network Observability?](network-observability-overview.md).

+- To create an AKS cluster with Network Observability and managed Prometheus and Grafana, see [Setup Network Observability for Azure Kubernetes Service (AKS) Azure managed Prometheus and Grafana](network-observability-managed-cli.md).

+ Title: "Setup of Network Observability for Azure Kubernetes Service (AKS) - Azure managed Prometheus and Grafana"

+description: Get started with AKS Network Observability for your AKS cluster using Azure managed Prometheus and Grafana.

+# Setup of Network Observability for Azure Kubernetes Service (AKS) - Azure managed Prometheus and Grafana

+AKS Network Observability is used to collect the network traffic data of your AKS cluster. Network Observability enables a centralized platform for monitoring application and network health. Prometheus collects AKS Network Observability metrics, and Grafana visualizes them. Both Cilium and non-Cilium data plane are supported. In this article, learn how to enable the Network Observability add-on and use Azure managed Prometheus and Grafana to visualize the scraped metrics.

+For more information about AKS Network Observability, see [What is Azure Kubernetes Service (AKS) Network Observability?](network-observability-overview.md).

+## Prerequisites

+- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+### Install the `aks-preview` Azure CLI extension

+```azurecli-interactive

+# Install the aks-preview extension

+az extension add --name aks-preview

+# Update the extension to make sure you have the latest version installed

+az extension update --name aks-preview

+```

+### Register the `NetworkObservabilityPreview` feature flag

+```azurecli-interactive

+az feature register --namespace "Microsoft.ContainerService" --name "NetworkObservabilityPreview"

+```

+Use [az feature show](/cli/azure/feature#az-feature-show) to check the registration status of the feature flag:

+```azurecli-interactive

+az feature show --namespace "Microsoft.ContainerService" --name "NetworkObservabilityPreview"

+```

+Wait for the feature to say **Registered** before preceding with the article.

+```output

+{

+ "id": "/subscriptions/23250d6d-28f0-41dd-9776-61fc80805b6e/providers/Microsoft.Features/providers/Microsoft.ContainerService/features/NetworkObservabilityPreview",

+ "name": "Microsoft.ContainerService/NetworkObservabilityPreview",

+ "properties": {

+ "state": "Registering"

+ },

+ "type": "Microsoft.Features/providers/features"

+}

+```

+When the feature is registered, refresh the registration of the Microsoft.ContainerService resource provider with [az provider register](/cli/azure/provider#az-provider-register):

+```azurecli-interactive

+az provider register -n Microsoft.ContainerService

+```

+## Create a resource group

+A resource group is a logical container into which Azure resources are deployed and managed. Create a resource group with [az group create](/cli/azure/group#az-group-create) command. The following example creates a resource group named **myResourceGroup** in the **eastus** location:

+```azurecli-interactive

+az group create \

+ --name myResourceGroup \

+ --location eastus

+```

+## Create AKS cluster

+Create an AKS cluster with [az aks create](/cli/azure/aks#az-aks-create). The following example creates an AKS cluster named **myAKSCluster** in the **myResourceGroup** resource group:

+# [**Non-Cilium**](#tab/non-cilium)

+Non-Cilium clusters support the enablement of Network Observability on an existing cluster or during the creation of a new cluster.

+Use [az aks create](/cli/azure/aks#az-aks-create) in the following example to create an AKS cluster with Network Observability and non-Cilium.

+## New cluster

+```azurecli-interactive

+az aks create \

+ --name myAKSCluster \

+ --resource-group myResourceGroup \

+ --location eastus \

+ --generate-ssh-keys \

+ --network-plugin azure \

+ --network-plugin-mode overlay \

+ --pod-cidr 192.168.0.0/16 \

+ --enable-network-observability

+```

+## Existing cluster

+Use [az aks update](/cli/azure/aks#az-aks-update) to enable Network Observability for an existing cluster.

+```azurecli-interactive

+az aks update \

+ --resource-group myResourceGroup \

+ --name myAKSCluster \

+ --enable-network-observability

+```

+# [**Cilium**](#tab/cilium)

+Use [az aks create](/cli/azure/aks#az-aks-create) in the following example to create an AKS cluster with Network Observability and Cilium.

+```azurecli-interactive

+az aks create \

+ --name myAKSCluster \

+ --resource-group myResourceGroup \

+ --generate-ssh-keys \

+ --location eastus \

+ --max-pods 250 \

+ --network-plugin azure \

+ --network-plugin-mode overlay \

+ --network-dataplane cilium \

+ --node-count 2 \

+ --pod-cidr 192.168.0.0/16

+```

+## Azure managed Prometheus and Grafana

+Use the following example to install and enable Prometheus and Grafana for your AKS cluster.

+### Create Azure Monitor resource

+```azurecli-interactive

+az resource create \

+ --resource-group myResourceGroup \

+ --namespace microsoft.monitor \

+ --resource-type accounts \

+ --name myAzureMonitor \

+ --location eastus \

+ --properties '{}'

+```

+### Create Grafana instance

+Use [az grafana create](/cli/azure/grafana#az-grafana-create) to create a Grafana instance. The name of the Grafana instance must be unique. Replace **myGrafana** with a unique name for your Grafana instance.

+```azurecli-interactive

+az grafana create \

+ --name myGrafana \

+ --resource-group myResourceGroup

+```

+### Place the Grafana and Azure Monitor resource IDs in variables

+Use [az grafana show](/cli/azure/grafana#az-grafana-show) to place the Grafana resource ID in a variable. Use [az resource show](/cli/azure/resource#az-resource-show) to place the Azure Monitor resource ID in a variable. Replace **myGrafana** with the name of your Grafana instance.

+```azurecli-interactive

+grafanaId=$(az grafana show \

+ --name myGrafana \

+ --resource-group myResourceGroup \

+ --query id \

+ --output tsv)

+azuremonitorId=$(az resource show \

+ --resource-group myResourceGroup \

+ --name myAzureMonitor \

+ --resource-type "Microsoft.Monitor/accounts" \

+ --query id \

+ --output tsv)

+```

+### Link Azure Monitor and Grafana to AKS cluster

+Use [az aks update](/cli/azure/aks#az-aks-update) to link the Azure Monitor and Grafana resources to your AKS cluster.

+```azurecli-interactive

+az aks update \

+ --name myAKSCluster \

+ --resource-group myResourceGroup \

+ --enable-azuremonitormetrics \

+ --azure-monitor-workspace-resource-id $azuremonitorId \

+ --grafana-resource-id $grafanaId

+```

+## Get cluster credentials

+```azurecli-interactive

+az aks get-credentials -name myAKSCluster --resource-group myResourceGroup

+```

+## Enable visualization on Grafana

+# [**Non-Cilium**](#tab/non-cilium)

+> [!NOTE]

+> The following section requires deployments of Azure managed Prometheus and Grafana.

+1. Use the following example to verify the Azure Monitor pods are running.

+ ```azurecli-interactive

+ kubectl get po -owide -n kube-system | grep ama-

+ ```

+ ```output

+ ama-metrics-5bc6c6d948-zkgc9 2/2 Running 0 (21h ago) 26h

+ ama-metrics-ksm-556d86b5dc-2ndkv 1/1 Running 0 (26h ago) 26h

+ ama-metrics-node-lbwcj 2/2 Running 0 (21h ago) 26h

+ ama-metrics-node-rzkzn 2/2 Running 0 (21h ago) 26h

+ ama-metrics-win-node-gqnkw 2/2 Running 0 (26h ago) 26h

+ ama-metrics-win-node-tkrm8 2/2 Running 0 (26h ago) 26h

+ ```

+1. Use the ID [18814]( https://grafana.com/grafana/dashboards/18814/) to import the dashboard from Grafana's public dashboard repo.

+1. Verify the Grafana dashboard is visible.

+# [**Cilium**](#tab/cilium)

+> [!NOTE]

+> The following section requires deployments of Azure managed Prometheus and Grafana.

+1. Use the following example to create a yaml file named **`ama-cilium-configmap.yaml`**. Copy the code in the example into the file created.

+ ```yaml

+ scrape_configs:

+ - job_name: "cilium-pods"

+ kubernetes_sd_configs:

+ - role: pod

+ relabel_configs:

+ - source_labels: [__meta_kubernetes_pod_container_name]

+ action: keep

+ regex: cilium(.*)

+ - source_labels:

+ [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]

+ separator: ":"

+ regex: ([^:]+)(?::\d+)?

+ target_label: __address__

+ replacement: ${1}:${2}

+ action: replace

+ - source_labels: [__meta_kubernetes_pod_node_name]

+ action: replace

+ target_label: instance

+ - source_labels: [__meta_kubernetes_pod_label_k8s_app]

+ action: keep

+ regex: cilium

+ - source_labels: [__meta_kubernetes_pod_name]

+ action: replace

+ regex: (.*)

+ target_label: pod

+ metric_relabel_configs:

+ - source_labels: [__name__]

+ action: keep

+ regex: (.*)

+ ```

+1. To create the `configmap`, use the following example:

+ ```azurecli-interactive

+ kubectl create configmap ama-metrics-prometheus-config-node \

+ --from-file=./ama-cilium-configmap.yaml \

+ --name kube-system

+ ```

+1. Once the Azure Monitor pods have been deployed on the cluster, port forward to the `ama` pod to verify the pods are being scraped. Use the following example to port forward to the pod:

+ ```azurecli-interactive

+ k port-forward $(k get po -l dsName=ama-metrics-node -oname | head -n 1) 9090:9090

+ ```

+1. In **Targets** of prometheus, verify the **cilium-pods** are present.

+1. Sign in to Grafana and import dashboards with the following ID [16611-cilium-metrics](https://grafana.com/grafana/dashboards/16611-cilium-metrics/).

+## Clean up resources

+If you're not going to continue to use this application, delete

+the AKS cluster and the other resources created in this article with the following example:

+```azurecli-interactive

+ az group delete \

+ --name myResourceGroup

+```

+## Next steps

+In this how-to article, you learned how to install and enable AKS Network Observability for your AKS cluster.

+- For more information about AKS Network Observability, see [What is Azure Kubernetes Service (AKS) Network Observability?](network-observability-overview.md).

+- To create an AKS cluster with Network Observability and BYO Prometheus and Grafana, see [Setup Network Observability for Azure Kubernetes Service (AKS) BYO Prometheus and Grafana](network-observability-byo-cli.md).

+ Title: What is Azure Kubernetes Service (AKS) Network Observability? (Preview)

+description: An overview of network observability for Azure Kubernetes Service (AKS).

+# What is Azure Kubernetes Service (AKS) Network Observability? (Preview)

+Kubernetes is a powerful tool for managing containerized applications. As containerized environments grow in complexity, it can be difficult to identify and troubleshoot networking issues in a Kubernetes cluster.

+Network observability is an important part of maintaining a healthy and performant Kubernetes cluster. By collecting and analyzing data about network traffic, you can gain insights into how your cluster is operating and identify potential problems before they cause outages or performance degradation.

+## Overview of Network Observability add-on in AKS

+> [!IMPORTANT]

+> AKS Network Observability is currently in PREVIEW.

+> See the [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

+Networking Observability add-on operates seamlessly on Non-Cilium and Cilium data-planes. It empowers customers with enterprise-grade capabilities for DevOps and SecOps. This solution offers a centralized way to monitor network issues in your cluster for cluster network administrators, cluster security administrators, and DevOps engineers.

+When the Network Observability add-on is enabled, it allows for the collection and conversion of useful metrics into Prometheus format, which can then be visualized in Grafana. There are two options available for using Prometheus and Grafana in this context: Azure managed [Prometheus](/azure/azure-monitor/essentials/prometheus-metrics-overview) and [Grafana](/azure/azure-monitor/visualize/grafana-plugin) or BYO Prometheus and Grafana.

+* **Azure managed Prometheus and Grafana:** This option involves using a managed service provided by Azure. The managed service takes care of the infrastructure and maintenance of Prometheus and Grafana, allowing you to focus on configuring and visualizing your metrics. This option is convenient if you prefer not to manage the underlying infrastructure.

+* **BYO Prometheus and Grafana:** Alternatively, you can choose to set up your own Prometheus and Grafana instances. In this case, you're responsible for provisioning and managing the infrastructure required to run Prometheus and Grafana. Install and configure Prometheus to scrape the metrics generated by the Network Observability add-on and store them. Similarly, Grafana needs to be set up to connect to Prometheus and visualize the collected data.

+## Metrics

+ Network Observability add-on currently only supports node level metrics in both Linux and Windows platforms. The below table outlines the different metrics generated by the Network Observability add-on.

+| Metric Name | Description | Labels | Linux | Windows |

+|-|-|--|-||

+| **kappie_forward_count** | Total forwarded packet count | Direction, NodeName, Cluster | Yes | Yes |

+| **kappie_forward_bytes** | Total forwarded byte count | Direction, NodeName, Cluster | Yes | Yes |

+| **kappie_drop_count** | Total dropped packet count | Reason, Direction, NodeName, Cluster | Yes | Yes |

+| **kappie_drop_bytes** | Total dropped byte count | Reason, Direction, NodeName, Cluster | Yes | Yes |

+| **kappie_tcp_state** | TCP active socket count by TCP state. | State, NodeName, Cluster | Yes | Yes |

+| **kappie_tcp_connection_remote** | TCP active socket count by remote address. | Address, Port, NodeName, Cluster | Yes | No |

+| **kappie_tcp_connection_stats** | TCP connection statistics. (ex: Delayed ACKs, TCPKeepAlive, TCPSackFailures) | Statistic, NodeName, Cluster | Yes | Yes |

+| **kappie_tcp_flag_counters** | TCP packets count by flag. | Flag, NodeName, Cluster | Yes | Yes |

+| **kappie_ip_connection_stats** | IP connection statistics. | Statistic, NodeName, Cluster | Yes | No |

+| **kappie_udp_connection_stats** | UDP connection statistics. | Statistic, NodeName, Cluster | Yes | No |

+| **kappie_udp_active_sockets** | UDP active socket count | NodeName, Cluster | Yes | No |

+| **kappie_interface_stats** | Interface statistics. | InterfaceName, Statistic, NodeName, Cluster | Yes | Yes |

+## Limitations

+* Pod level metrics aren't supported.

+* The deployment of the Network Observability add-on on Mariner 1.0 is currently unsupported.

+## Scale

+Certain scale limitations apply when you use Azure managed Prometheus and Grafana. For more information, see [Scrape Prometheus metrics at scale in Azure Monitor](/azure/azure-monitor/essentials/prometheus-metrics-scrape-scale)

+## Next steps

+- For more information about Azure Kubernetes Service (AKS), see [What is Azure Kubernetes Service (AKS)?](/azure/aks/intro-kubernetes).

+- To create an AKS cluster with Network Observability and Azure managed Prometheus and Grafana, see [Setup Network Observability for Azure Kubernetes Service (AKS) Azure managed Prometheus and Grafana](network-observability-managed-cli.md).

+- To create an AKS cluster with Network Observability and BYO Prometheus and Grafana, see [Setup Network Observability for Azure Kubernetes Service (AKS) BYO Prometheus and Grafana](network-observability-byo-cli.md).

+ Title: Uninstall the Open Service Mesh (OSM) add-on from your Azure Kubernetes Service (AKS) cluster

+description: How to uninstall the Open Service Mesh on Azure Kubernetes Service (AKS) using Azure CLI.

+* Disable the OSM add-on from your cluster using the [`az aks disable-addon`][az-aks-disable-addon] command and the `--addons` parameter.

+ ```azurecli-interactive

+ az aks disable-addons \

+ --resource-group myResourceGroup \

+ --name myAKSCluster \

+ --addons open-service-mesh

+ ```

+## Remove OSM resources

+* Uninstall the remaining resources on the cluster using the `osm uninstall cluster-wide-resources` command.

+ ```console

+ osm uninstall cluster-wide-resources

+ ```

+ > [!NOTE]

+ > For version 1.1, the command is `osm uninstall mesh --delete-cluster-wide-resources`

+ > [!IMPORTANT]

+ > You must remove these additional resources after you disable the OSM add-on. Leaving these resources on your cluster may cause issues if you enable the OSM add-on again in the future.

+## Next steps

+Learn more about [Open Service Mesh][osm].

+<!-- LINKS - Internal -->

+[az-aks-disable-addon]: /cli/azure/aks#az_aks_disable_addons

+[osm]: ./open-service-mesh-about.md

+ Title: Enable Cloud Controller Manager (preview) on your Azure Kubernetes Service (AKS) cluster

+description: Learn how to enable the Out of Tree cloud provider (preview) on your Azure Kubernetes Service (AKS) cluster.

+# Enable Cloud Controller Manager (preview) on your Azure Kubernetes Service (AKS) cluster

+Previously, cloud provider integration with Kubernetes was *in-tree*, where any changes to cloud specific features would follow the standard Kubernetes release cycle. When issues were fixed or enhancements were rolled out, they would need to be within the Kubernetes community's release cycle.

+The Kubernetes community is now adopting an ***out-of-tree*** model, where cloud providers control releases independently of the core Kubernetes release schedule through the [cloud-provider-azure][cloud-provider-azure] component. As part of this cloud-provider-azure component, we're also introducing a cloud-node-manager component, which is a component of the Kubernetes node lifecycle controller. A DaemonSet in the *kube-system* namespace deploys this component.

+> When you enable the Cloud Controller Manager (preview) on your AKS cluster, it also enables the out-of-tree CSI drivers.

+* The Azure CLI. For more information, see [Install the Azure CLI][install-azure-cli].

+* Kubernetes version 1.20.x or higher.

+1. Install the aks-preview extension using the [`az extension add`][az-extension-add] command.

+ ```azurecli

+ az extension add --name aks-preview

+ ```

+2. Update to the latest version of the extension released using the [`az extension update`][az-extension-update] command.

+ ```azurecli

+ az extension update --name aks-preview

+ ```

+1. Register the `EnableCloudControllerManager` feature flag using the [`az feature register`][az-feature-register] command.

+ ```azurecli-interactive

+ az feature register --namespace "Microsoft.ContainerService" --name "EnableCloudControllerManager"

+ ```

+ It takes a few minutes for the status to show *Registered*.

+2. Verify the registration status using the [`az feature show`][az-feature-show] command.

+ ```azurecli-interactive

+ az feature show --namespace "Microsoft.ContainerService" --name "EnableCloudControllerManager"

+ ```

+3. When the status reflects *Registered*, refresh the registration of the *Microsoft.ContainerService* resource provider using the [`az provider register`][az-provider-register] command.

+ ```azurecli-interactive

+ az provider register --namespace Microsoft.ContainerService

+ ```

+* Create a new AKS cluster with Cloud Controller Manager using the [`az aks create`][az-aks-create] command and include the parameter `EnableCloudControllerManager=True` as an `--aks-custom-header`.

+ ```azurecli-interactive

+ az aks create -n aks -g myResourceGroup --aks-custom-headers EnableCloudControllerManager=True

+ ```

+* Upgrade an existing AKS cluster with Cloud Controller Manager using the [`az aks upgrade`][az-aks-upgrade] command and include the parameter `EnableCloudControllerManager=True` as an `--aks-custom-header`.

+ ```azurecli-interactive

+ az aks upgrade -n aks -g myResourceGroup -k <version> --aks-custom-headers EnableCloudControllerManager=True

+ ```

+* Verify the component deployment using the following `kubectl get po` command.

+ ```azurecli-interactive

+ kubectl get po -n kube-system | grep cloud-node-manager

+ ```

+* For more information on CSI drivers, and the default behavior for Kubernetes versions higher than 1.21, review the [CSI documentation][csi-docs].

+* For more information on the Kubernetes community direction regarding out-of-tree providers, see the [community blog post][community-blog].

+[install-azure-cli]: /cli/azure/install-azure-cli

+[az-extension-add]: /cli/azure/extension#az-extension-add

+[az-extension-update]: /cli/azure/extension#az-extension-update

+[az-aks-create]: /cli/azure/aks#az-aks-create

+[az-aks-upgrade]: /cli/azure/aks#az-aks-upgrade

+description: Learn how to use proximity placement groups to reduce latency for your Azure Kubernetes Service (AKS) cluster workloads.

+# Use proximity placement groups to reduce latency for Azure Kubernetes Service (AKS) clusters

+> [!NOTE]

+> When using proximity placement groups on AKS, colocation only applies to the agent nodes. Node to node and the corresponding hosted pod to pod latency is improved. The colocation doesn't affect the placement of a cluster's control plane.

+When deploying your application in Azure, you can create network latency by spreading virtual machine (VM) instances across regions or availability zones, which may impact the overall performance of your application. A proximity placement group is a logical grouping used to make sure Azure compute resources are physically located close to one another. Some applications, such as gaming, engineering simulations, and high-frequency trading (HFT) require low latency and tasks that can complete quickly. For similar high-performance computing (HPC) scenarios, consider using [proximity placement groups](../virtual-machines/co-location.md#proximity-placement-groups) (PPG) for your cluster's node pools.

+This article requires Azure CLI version 2.14 or later. Run `az --version` to find the version. If you need to install or upgrade, see [Install Azure CLI][azure-cli-install].

+* A proximity placement group can map to only *one* availability zone.

+The first resource you deploy with a proximity placement group attaches to a specific data center. Any extra resources you deploy with the same proximity placement group are colocated in the same data center. Once all resources using the proximity placement group are stopped (deallocated) or deleted, it's no longer attached.

+* You can associate multiple node pools with a single proximity placement group.

+* You can only associate a node pool with a single proximity placement group.

+> While proximity placement groups require a node pool to use only *one* availability zone, the [baseline Azure VM SLA of 99.9%](https://azure.microsoft.com/support/legal/sla/virtual-machines/v1_9/) is still in effect for VMs in a single zone.

+Proximity placement groups are a node pool concept and associated with each individual node pool. Using a PPG resource has no impact on AKS control plane availability, which can impact how you should design your cluster with zones. To ensure a cluster is spread across multiple zones, we recommend using the following design:

+* Provision a cluster with the first system pool using *three* zones and no proximity placement group associated to ensure the system pods land in a dedicated node pool, which spreads across multiple zones.

+* Add extra user node pools with a unique zone and proximity placement group associated to each pool. An example is *nodepool1* in zone one and PPG1, *nodepool2* in zone two and PPG2, and *nodepool3* in zone 3 with PPG3. This configuration ensures that, at a cluster level, nodes are spread across multiple zones and each individual node pool is colocated in the designated zone with a dedicated PPG resource.

+Accelerated networking greatly improves networking performance of virtual machines. Ideally, use proximity placement groups with accelerated networking. By default, AKS uses accelerated networking on [supported virtual machine instances](../virtual-network/accelerated-networking-overview.md?toc=/azure/virtual-machines/linux/toc.json#limitations-and-constraints), which include most Azure virtual machine with two or more vCPUs.

+1. Create an Azure resource group using the [`az group create`][az-group-create] command.

+ ```azurecli-interactive

+ az group create --name myResourceGroup --location centralus

+ ```

+2. Create a proximity placement group using the [`az ppg create`][az-ppg-create] command. Make sure to note the ID value in the output.

+ ```azurecli-interactive

+ az ppg create -n myPPG -g myResourceGroup -l centralus -t standard

+ ```

+ The command produces an output similar to the following example output, which includes the *ID* value you need for upcoming CLI commands.

+ ```output

+ {

+ "availabilitySets": null,

+ "colocationStatus": null,

+ "id": "/subscriptions/yourSubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Compute/proximityPlacementGroups/myPPG",

+ "location": "centralus",

+ "name": "myPPG",

+ "proximityPlacementGroupType": "Standard",

+ "resourceGroup": "myResourceGroup",

+ "tags": {},

+ "type": "Microsoft.Compute/proximityPlacementGroups",

+ "virtualMachineScaleSets": null,

+ "virtualMachines": null

+ }

+ ```

+3. Create an AKS cluster using the [`az aks create`][az-aks-create] command and replace the *myPPGResourceID* value with your proximity placement group resource ID from the previous step.

+ ```azurecli-interactive

+ az aks create \

+ --resource-group myResourceGroup \

+ --name myAKSCluster \

+ --ppg myPPGResourceID

+ ```

+You can add a proximity placement group to an existing cluster by creating a new node pool. You can then optionally migrate existing workloads to the new node pool and delete the original node pool.

+Use the same proximity placement group that you created earlier to ensure agent nodes in both node pools in your AKS cluster are physically located in the same data center.

+* Create a new node pool using the [`az aks nodepool add`][az-aks-nodepool-add] command and replace the *myPPGResourceID* value with your proximity placement group resource ID.

+ ```azurecli-interactive

+ az aks nodepool add \

+ --resource-group myResourceGroup \

+ --cluster-name myAKSCluster \

+ --name mynodepool \

+ --node-count 1 \

+ --ppg myPPGResourceID

+ ```

+* Delete the Azure resource group along with its resources using the [`az group delete`][az-group-delete] command.

+ ```azurecli-interactive

+ az group delete --name myResourceGroup --yes --no-wait

+ ```

+Learn more about [proximity placement groups][proximity-placement-groups].

+[az-ppg-create]: /cli/azure/ppg#az_ppg_create

+For the past release history, see [Kubernetes history](https://github.com/kubernetes/kubernetes/releases).

+With Azure Kubernetes Service (AKS), you can set Azure tags on an AKS cluster and its related resources using Azure Resource Manager and the Azure CLI. You can also use Kubernetes manifests to set Azure tags for certain resources. Azure tags are a useful tracking resource for certain business processes, such as *chargeback*.

+Review the following information before you begin:

+* Tags set on an AKS cluster apply to all resources related to the cluster, but not the node pools. This operation overwrites the values of existing keys.

+* Public IPs, files, and disks can have tags set by Kubernetes through a Kubernetes manifest. Tags set in this way maintain the Kubernetes values, even if you update them later using a different method. When you remove public IPs, files, or disks through Kubernetes, any tags set by Kubernetes are removed. The tags on those resources that Kubernetes doesn't track remain unaffected.

+* The Azure CLI version 2.0.59 or later. To find your version, run `az --version`. If you need to install it or update your version, see [Install Azure CLI][install-azure-cli].

+* Kubernetes version 1.20 or later.

+* Azure tags have keys that are case-insensitive for operations, such as when you're retrieving a tag by searching the key. In this case, a tag with the specified key is updated or retrieved regardless of casing. Tag values are case-sensitive.

+* For shared resources, tags can't determine the split in resource usage on their own.

+## Azure tags and AKS clusters

+When you create or update an AKS cluster with the `--tags` parameter, the following are assigned the Azure tags that you specified:

+* The AKS cluster itself and its related resources:

+ * Route table

+ * Public IP

+ * Load balancer

+ * Network security group

+ * Virtual network

+ * AKS-managed kubelet msi

+ * AKS-managed add-on msi

+ * Private DNS zone associated with the *private cluster*

+ * Private endpoint associated with the *private cluster*

+* The node resource group

+> [!NOTE]

+> Azure Private DNS only supports 15 tags. For more information, see the [tag resources](../azure-resource-manager/management/tag-resources.md).

+## Create or update tags on an AKS cluster

+### Create a new AKS cluster

+> [!IMPORTANT]

+> If you're using existing resources when you create a new cluster, such as an IP address or route table, the `az aks create` command overwrites the set of tags. If you delete the cluster later, any tags set by the cluster are removed.

+1. Create a cluster and assign Azure tags using the [`az aks create`][az-aks-create] command with the `--tags` parameter.

+ > [!NOTE]

+ > To set tags on the initial node pool, the virtual machine scale set, and each virtual machine scale set instance associated with the initial node pool, you can also set the `--nodepool-tags` parameter.

+ ```azurecli-interactive

+ az aks create \

+ --resource-group myResourceGroup \

+ --name myAKSCluster \

+ --tags dept=IT costcenter=9999 \

+ --generate-ssh-keys

+ ```

+2. Verify the tags have been applied to the cluster and its related resources using the [`az aks show`][az-aks-show] command.

+ ```azurecli-interactive

+ az aks show -g myResourceGroup -n myAKSCluster --query '[tags]'

+ ```

+ The following example output shows the tags applied to the cluster:

+ ```output

+ {

+ "clusterTags": {

+ "dept": "IT",

+ "costcenter": "9999"

+ }

+ }

+ ```

+### Update an existing AKS cluster

+> Setting tags on a cluster using the `az aks update` command overwrites the set of tags. For example, if your cluster has the tags *dept=IT* and *costcenter=9999*, and you use `az aks update` with the tags *team=alpha* and *costcenter=1234*, the new list of tags would be *team=alpha* and *costcenter=1234*.

+1. Update the tags on an existing cluster using the [`az aks update`][az-aks-update] command with the `--tags` parameter.

+ ```azurecli-interactive

+ az aks update \

+ --resource-group myResourceGroup \

+ --name myAKSCluster \

+ --tags team=alpha costcenter=1234

+ ```

+2. Verify the tags have been applied to the cluster and its related resources using the [`az aks show`][az-aks-show] command.

+ ```azurecli-interactive

+ az aks show -g myResourceGroup -n myAKSCluster --query '[tags]'

+ ```

+ The following example output shows the tags applied to the cluster:

+ ```output

+ {

+ "clusterTags": {

+ "team": "alpha",

+ "costcenter": "1234"

+ }

+ ```

+## Add tags to node pools

+You can apply an Azure tag to a new or existing node pool in your AKS cluster. Tags applied to a node pool are applied to each node within the node pool and are persisted through upgrades. Tags are also applied to new nodes that are added to a node pool during scale-out operations. Adding a tag can help with tasks such as policy tracking or cost estimation.

+When you create or update a node pool with the `--tags` parameter, the tags you specify are assigned to the following resources:

+* The node pool.

+* The virtual machine scale set and each virtual machine scale set instance associated with the node pool.

+### Create a new node pool

+1. Create a node pool with an Azure tag using the [`az aks nodepool add`][az-aks-nodepool-add] command with the `--tags` parameter.

+ ```azurecli-interactive

+ az aks nodepool add \

+ --resource-group myResourceGroup \

+ --cluster-name myAKSCluster \

+ --name tagnodepool \

+ --node-count 1 \

+ --tags abtest=a costcenter=5555 \

+ --no-wait

+ ```

+2. Verify that the tags have been applied to the node pool using the [`az aks show`][az-aks-show] command.

+ ```azurecli-interactive

+ az aks show -g myResourceGroup -n myAKSCluster --query 'agentPoolProfiles[].{nodepoolName:name,tags:tags}'

+ ```

+ The following example output shows the tags applied to the node pool:

+ ```output

+ [

+ {

+ "nodepoolName": "nodepool1",

+ "tags": null

+ },

+ {

+ "nodepoolName": "tagnodepool",

+ "tags": {

+ "abtest": "a",

+ "costcenter": "5555"

+ }

+ }

+ ]

+ ```

+### Update an existing node pool

+> [!IMPORTANT]

+> Setting tags on a node pool using the `az aks nodepool update` command overwrites the set of tags. For example, if your node pool has the tags *abtest=a* and *costcenter=5555*, and you use `az aks nodepool update` with the tags *appversion=0.0.2* and *costcenter=4444*, the new list of tags would be *appversion=0.0.2* and *costcenter=4444*.

+1. Update a node pool with an Azure tag using the [`az aks nodepool update`][az-aks-nodepool-update] command.

+ ```azurecli-interactive

+ az aks nodepool update \

+ --resource-group myResourceGroup \

+ --cluster-name myAKSCluster \

+ --name tagnodepool \

+ --tags appversion=0.0.2 costcenter=4444 \

+ --no-wait

+ ```

+2. Verify the tags have been applied to the node pool using the [`az aks show`][az-aks-show] command.

+ ```azurecli-interactive

+ az aks show -g myResourceGroup -n myAKSCluster --query 'agentPoolProfiles[].{nodepoolName:name,tags:tags}'

+ ```

+ The following example output shows the tags applied to the node pool:

+ ```output

+ [

+ {

+ "nodepoolName": "nodepool1",

+ "tags": null

+ },

+ {

+ "nodepoolName": "tagnodepool",

+ "tags": {

+ "appversion": "0.0.2",

+ "costcenter": "4444"

+ }

+ }

+ ]

+ ```

+## Add tags using Kubernetes

+> Setting tags on files, disks, and public IPs using Kubernetes updates the set of tags. For example, if your disk has the tags *dept=IT* and *costcenter=5555*, and you use Kubernetes to set the tags *team=beta* and *costcenter=3333*, the new list of tags would be *dept=IT*, *team=beta*, and *costcenter=3333*.

+>

+> Any updates you make to tags through Kubernetes retain the value set through Kubernetes. For example, if your disk has tags *dept=IT* and *costcenter=5555* set by Kubernetes, and you use the portal to set the tags *team=beta* and *costcenter=3333*, the new list of tags would be *dept=IT*, *team=beta*, and *costcenter=5555*. If you then remove the disk through Kubernetes, the disk would have the tag *team=beta*.

+You can apply Azure tags to public IPs, disks, and files using a Kubernetes manifest.

+* For public IPs, use *service.beta.kubernetes.io/azure-pip-tags* under *annotations*. For example:

+ ```yml

+ apiVersion: v1

+ kind: Service

+ metadata:

+ annotations:

+ service.beta.kubernetes.io/azure-pip-tags: costcenter=3333,team=beta

+ spec:

+ ...

+ ```

+* For files and disks, use *tags* under *parameters*. For example:

+ ```yml

+

+ apiVersion: storage.k8s.io/v1

+ ...

+ parameters:

+ ...

+ tags: costcenter=3333,team=beta

+ ...

+ ```

+## Next steps

+Learn more about [using labels in an AKS cluster][use-labels-aks].

+<!-- LINKS - internal -->

+[az-aks-create]: /cli/azure/aks#az-aks-create

+[az-aks-show]: /cli/azure/aks#az-aks-show

+[use-labels-aks]: ./use-labels.md

+[az-aks-nodepool-add]: /cli/azure/aks/nodepool#az-aks-nodepool-add

+[az-aks-nodepool-update]: /cli/azure/aks/nodepool#az-aks-nodepool-update

+[az-aks-update]: /cli/azure/aks#az-aks-update

+| Microsoft.Web/sites/write** | Update Web App settings |

+***only required if you are updating access restrictions through Azure portal.*

+ Title: 'Tutorial: PHP app with MySQL and Redis'

+description: Learn how to get a PHP app working in Azure, with connection to a MySQL database and a Redis cache in Azure. Laravel is used in the tutorial.

+# Tutorial: Deploy a PHP, MySQL, and Redis app to Azure App Service

+This tutorial shows how to create a secure PHP app in Azure App Service that's connected to a MySQL database (using Azure Database for MySQL flexible server). You'll also deploy an Azure Cache for Redis to enable the caching code in your application. Azure App Service is a highly scalable, self-patching, web-hosting service that can easily deploy apps on Windows or Linux. When you're finished, you'll have a Laravel app running on Azure App Service on Linux.

+To follow along with this tutorial, clone or download the sample [Laravel](https://laravel.com/) application from the repository:

+ :::column span="2":::

+ **Step 1.** In the Azure portal:

+ 1. Enter "web app database" in the search bar at the top of the Azure portal.

+ 1. Select the item labeled **Web App + Database** under the **Marketplace** heading.

+ You can also navigate to the [creation wizard](https://portal.azure.com/?feature.customportal=false#create/Microsoft.AppServiceWebAppDatabaseV3) directly.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-create-app-mysql-1.png" alt-text="A screenshot showing how to use the search box in the top tool bar to find the Web App + Database creation wizard." lightbox="./media/tutorial-php-mysql-app/azure-portal-create-app-mysql-1.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 2.** In the **Create Web App + Database** page, fill out the form as follows.

+ 1. *Resource Group* &rarr; Select **Create new** and use a name of **msdocs-laravel-mysql-tutorial**.

+ 1. *Region* &rarr; Any Azure region near you.

+ 1. *Name* &rarr; **msdocs-laravel-mysql-XYZ** where *XYZ* is any three random characters. This name must be unique across Azure.

+ 1. *Runtime stack* &rarr; **PHP 8.2**.

+ 1. *Add Azure Cache for Redis?* &rarr; **Yes**.

+ 1. *Hosting plan* &rarr; **Basic**. When you're ready, you can [scale up](manage-scale-up.md) to a production pricing tier later.

+ 1. **MySQL - Flexible Server** is selected for you by default as the database engine. Azure Database for MySQL is a fully managed MySQL database as a service on Azure, compatible with the latest community editions.

+ 1. Select **Review + create**.

+ 1. After validation completes, select **Create**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-create-app-mysql-2.png" alt-text="A screenshot showing how to configure a new app and database in the Web App + Database wizard." lightbox="./media/tutorial-php-mysql-app/azure-portal-create-app-mysql-2.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 3.** The deployment takes a few minutes to complete. Once deployment completes, select the **Go to resource** button. You're taken directly to the App Service app, but the following resources are created:

+ - **Resource group** &rarr; The container for all the created resources.

+ - **App Service plan** &rarr; Defines the compute resources for App Service. A Linux plan in the *Basic* tier is created.

+ - **App Service** &rarr; Represents your app and runs in the App Service plan.

+ - **Virtual network** &rarr; Integrated with the App Service app and isolates back-end network traffic.

+ - **Private endpoints** &rarr; Access endpoints for the database server and the Redis cache in the virtual network.

+ - **Network interfaces** &rarr; Represents private IP addresses, one for each of the private endpoints.

+ - **Azure Database for MySQL flexible server** &rarr; Accessible only from behind its private endpoint. A database and a user are created for you on the server.

+ - **Azure Cache for Redis** &rarr; Accessible only from behind its private endpoint.

+ - **Private DNS zones** &rarr; Enable DNS resolution of the database server and the Redis cache in the virtual network.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-create-app-mysql-3.png" alt-text="A screenshot showing the deployment process completed." lightbox="./media/tutorial-php-mysql-app/azure-portal-create-app-mysql-3.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 1.** In the App Service page, in the left menu, select **Configuration**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-get-connection-string-1.png" alt-text="A screenshot showing how to open the configuration page in App Service." lightbox="./media/tutorial-php-mysql-app/azure-portal-get-connection-string-1.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 2.**

+ 1. Find app settings that begin with **AZURE_MYSQL_**. They were generated from the new MySQL database by the creation wizard.

+ 1. Also, find app settings that begin with **AZURE_REDIS_**. They were generated from the new Redis cache by the creation wizard. To set up your application, this name is all you need.

+ 1. If you want, you can select the **Edit** button to the right of each setting and see or copy its value.

+ Later, you'll change your application code to use these settings.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-get-connection-string-2.png" alt-text="A screenshot showing how to create an app setting." lightbox="./media/tutorial-php-mysql-app/azure-portal-get-connection-string-2.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 3.** In the **Application settings** tab of the **Configuration** page, create a `CACHE_DRIVER` setting:

+ 1. Select **New application setting**.

+ 1. In the **Name** field, enter *CACHE_DRIVER*.

+ 1. In the **Value** field, enter *redis*.

+ 1. Select **OK**.

+ `CACHE_DRIVER` is already used in the Laravel application code. This setting tells Laravel to use Redis as its cache.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-get-connection-string-3.png" alt-text="A screenshot showing how to see the autogenerated connection string." lightbox="./media/tutorial-php-mysql-app/azure-portal-get-connection-string-3.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 4.** Using the same steps in **Step 3**, create the following app settings:

+ - **MYSQL_ATTR_SSL_CA**: Use */home/site/wwwroot/ssl/DigiCertGlobalRootCA.crt.pem* as the value. This app setting points to the path of the [TLS/SSL certificate you need to access the MySQL server](../mysql/flexible-server/how-to-connect-tls-ssl.md#download-the-public-ssl-certificate). It's included in the sample repository for convenience.

+ - **LOG_CHANNEL**: Use *stderr* as the value. This setting tells Laravel to pipe logs to stderr, which makes it available to the App Service logs.

+ - **APP_DEBUG**: Use *true* as the value. It's a [Laravel debugging variable](https://laravel.com/docs/10.x/errors#configuration) that enables debug mode pages.

+ - **APP_KEY**: Use *base64:Dsz40HWwbCqnq0oxMsjq7fItmKIeBfCBGORfspaI1Kw=* as the value. It's a [Laravel encryption variable](https://laravel.com/docs/10.x/encryption#configuration).

+ 1. In the menu bar at the top, select **Save**.

+ 1. When prompted, select **Continue**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-get-connection-string-4.png" alt-text="A screenshot showing how to save settings in the configuration page." lightbox="./media/tutorial-php-mysql-app/azure-portal-get-connection-string-4.png":::

+ :::column-end:::

+> [!IMPORTANT]

+> The `APP_KEY` value is used here for convenience. For production scenarios, it should be generated specifically for your deployment using `php artisan key:generate --show` in the command line.

+In this step, you'll configure GitHub deployment using GitHub Actions. It's just one of many ways to deploy to App Service, but also a great way to have continuous integration in your deployment process. By default, every `git push` to your GitHub repository kicks off the build and deploy action. You'll make some changes to your codebase with Visual Studio Code directly in the browser, then let GitHub Actions deploy automatically for you.

+ :::column span="2":::

+ **Step 1.** In a new browser window:

+ 1. Sign in to your GitHub account.

+ 1. Navigate to [https://github.com/Azure-Samples/laravel-tasks](https://github.com/Azure-Samples/laravel-tasks).

+ 1. Select **Fork**.

+ 1. Select **Create fork**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-1.png" alt-text="A screenshot showing how to create a fork of the sample GitHub repository." lightbox="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-1.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 2.** In the GitHub page, open Visual Studio Code in the browser by pressing the `.` key.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-2.png" alt-text="A screenshot showing how to open the Visual Studio Code browser experience in GitHub." lightbox="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-2.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 3.** In Visual Studio Code in the browser, open *config/database.php* in the explorer. Find the `mysql` section and make the following changes:

+ 1. Replace `DB_HOST` with `AZURE_MYSQL_HOST`.

+ 1. Replace `DB_DATABASE` with `AZURE_MYSQL_DBNAME`.

+ 1. Replace `DB_USERNAME` with `AZURE_MYSQL_USERNAME`.

+ 1. Replace `DB_PASSWORD` with `AZURE_MYSQL_PASSWORD`.

+ 1. Replace `DB_PORT` with `AZURE_MYSQL_PORT`.

+ Remember that these `AZURE_MYSQL_` settings were created for you by the create wizard.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-3.png" alt-text="A screenshot showing Visual Studio Code in the browser and an opened file with modified MySQL variables." lightbox="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-3.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 4.** In *config/database.php* scroll to the Redis `cache` section and make the following changes:

+ 1. Replace `REDIS_HOST` with `AZURE_REDIS_HOST`.

+ 1. Replace `REDIS_PASSWORD` with `AZURE_REDIS_PASSWORD`.

+ 1. Replace `REDIS_PORT` with `AZURE_REDIS_PORT`.

+ 1. Replace `REDIS_CACHE_DB` with `AZURE_REDIS_DATABASE`.

+ 1. In the same section, add a line with `'scheme' => 'tls',`. This configuration tells Laravel to use encryption to connect to Redis.

+ Remember that these `AZURE_REDIS_` settings were created for you by the create wizard.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-4.png" alt-text="A screenshot showing Visual Studio Code in the browser and an opened file with modified Redis variables." lightbox="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-4.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 5.**

+ 1. Select the **Source Control** extension.

+ 1. In the textbox, type a commit message like `Configure DB & Redis variables`.

+ 1. Select **Commit and Push**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-5.png" alt-text="A screenshot showing the changes being committed and pushed to GitHub." lightbox="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-5.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 6.** Back in the App Service page, in the left menu, select **Deployment Center**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-6.png" alt-text="A screenshot showing how to open the deployment center in App Service." lightbox="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-6.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 7.** In the Deployment Center page:

+ 1. In **Source**, select **GitHub**. By default, **GitHub Actions** is selected as the build provider.

+ 1. Sign in to your GitHub account and follow the prompt to authorize Azure.

+ 1. In **Organization**, select your account.

+ 1. In **Repository**, select **laravel-task**.

+ 1. In **Branch**, select **main**.

+ 1. In the top menu, select **Save**. App Service commits a workflow file into the chosen GitHub repository, in the `.github/workflows` directory.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-7.png" alt-text="A screenshot showing how to configure CI/CD using GitHub Actions." lightbox="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-7.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 8.** In the Deployment Center page:

+ 1. Select **Logs**. A deployment run is already started.

+ 1. In the log item for the deployment run, select **Build/Deploy Logs**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-8.png" alt-text="A screenshot showing how to open deployment logs in the deployment center." lightbox="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-8.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 9.** You're taken to your GitHub repository and see that the GitHub action is running. The workflow file defines two separate stages, build and deploy. Wait for the GitHub run to show a status of **Complete**. It takes about 15 minutes.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-9.png" alt-text="A screenshot showing a GitHub run in progress." lightbox="./media/tutorial-php-mysql-app/azure-portal-deploy-sample-code-9.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 1.** Back in the App Service page, in the left menu, select **SSH**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-generate-db-schema-1.png" alt-text="A screenshot showing how to open the SSH shell for your app from the Azure portal." lightbox="./media/tutorial-php-mysql-app/azure-portal-generate-db-schema-1.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 2.** In the SSH terminal:

+ 1. Run `cd /home/site/wwwroot`. Here are all your deployed files.

+ 1. Run `php artisan migrate --force`. If it succeeds, App Service is connecting successfully to the MySQL database.

+ Only changes to files in `/home` can persist beyond app restarts. Changes outside of `/home` aren't persisted.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-generate-db-schema-2.png" alt-text="A screenshot showing the commands to run in the SSH shell and their output." lightbox="./media/tutorial-php-mysql-app/azure-portal-generate-db-schema-2.png":::

+ :::column-end:::

+[Laravel application lifecycle](https://laravel.com/docs/10.x/lifecycle#lifecycle-overview) begins in the **/public** directory instead. The default PHP container for App Service uses Nginx, which starts in the application's root directory. To change the site root, you need to change the Nginx configuration file in the PHP container (*/etc/nginx/sites-available/default*). For your convenience, the sample repository contains a custom configuration file called *default*. As noted previously, you don't want to replace this file using the SSH shell, because the change is outside of `/home` and will be lost after an app restart.

+ :::column span="2":::

+ **Step 1.**

+ 1. From the left menu, select **Configuration**.

+ 1. Select the **General settings** tab.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-change-site-root-1.png" alt-text="A screenshot showing how to open the general settings tab in the configuration page of App Service." lightbox="./media/tutorial-php-mysql-app/azure-portal-change-site-root-1.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 2.** In the General settings tab:

+ 1. In the **Startup Command** box, enter the following command: *cp /home/site/wwwroot/default /etc/nginx/sites-available/default && service nginx reload*.

+ 1. Select **Save**.

+ The command replaces the Nginx configuration file in the PHP container and restarts Nginx. This configuration ensures that the same change is made to the container each time it starts.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-change-site-root-2.png" alt-text="A screenshot showing how to configure a startup command in App Service." lightbox="./media/tutorial-php-mysql-app/azure-portal-change-site-root-2.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 1.** In the App Service page:

+ 1. From the left menu, select **Overview**.

+ 1. Select the URL of your app. You can also navigate directly to `https://<app-name>.azurewebsites.net`.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-browse-app-1.png" alt-text="A screenshot showing how to launch an App Service from the Azure portal." lightbox="./media/tutorial-php-mysql-app/azure-portal-browse-app-1.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 2.** Add a few tasks to the list.

+ Congratulations, you're running a secure data-driven PHP app in Azure App Service.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-browse-app-2.png" alt-text="A screenshot of the Laravel app running in App Service." lightbox="./media/tutorial-php-mysql-app/azure-portal-browse-app-2.png":::

+ :::column-end:::

+> [!TIP]

+> The sample application implements the [cache-aside](/azure/architecture/patterns/cache-aside) pattern. When you reload the page after making data changes, **Response time** in the webpage shows a much faster time because it's loading the data from the cache instead of the database.

+Azure App Service captures all messages logged to the console to assist you in diagnosing issues with your application. The sample app outputs console log messages in each of its endpoints to demonstrate this capability. By default, Laravel's logging functionality (for example, `Log::info()`) outputs to a local file. Your `LOG_CHANNEL` app setting from earlier makes log entries accessible from the App Service log stream.

+ :::column span="2":::

+ **Step 1.** In the App Service page:

+ 1. From the left menu, select **App Service logs**.

+ 1. Under **Application logging**, select **File System**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-stream-diagnostic-logs-1.png" alt-text="A screenshot showing how to enable native logs in App Service in the Azure portal." lightbox="./media/tutorial-php-mysql-app/azure-portal-stream-diagnostic-logs-1.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 2.** From the left menu, select **Log stream**. You see the logs for your app, including platform logs and logs from inside the container.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-stream-diagnostic-logs-2.png" alt-text="A screenshot showing how to view the log stream in the Azure portal." lightbox="./media/tutorial-php-mysql-app/azure-portal-stream-diagnostic-logs-2.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 1.** In the search bar at the top of the Azure portal:

+ 1. Enter the resource group name.

+ 1. Select the resource group.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-clean-up-resources-1.png" alt-text="A screenshot showing how to search for and navigate to a resource group in the Azure portal." lightbox="./media/tutorial-php-mysql-app/azure-portal-clean-up-resources-1.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 2.** In the resource group page, select **Delete resource group**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-clean-up-resources-2.png" alt-text="A screenshot showing the location of the Delete Resource Group button in the Azure portal." lightbox="./media/tutorial-php-mysql-app/azure-portal-clean-up-resources-2.png":::

+ :::column-end:::

+ :::column span="2":::

+ **Step 3.**

+ 1. Enter the resource group name to confirm your deletion.

+ 1. Select **Delete**.

+ :::column-end:::

+ :::column:::

+ :::image type="content" source="./media/tutorial-php-mysql-app/azure-portal-clean-up-resources-3.png" alt-text="A screenshot of the confirmation dialog for deleting a resource group in the Azure portal." lightbox="./media/tutorial-php-mysql-app/azure-portal-clean-up-resources-3.png"::::

+ :::column-end:::

+- The App Service plan is created in **Basic** tier and can be scaled up or down. See [App Service pricing](https://azure.microsoft.com/pricing/details/app-service/linux/).

+- The Azure Cache for Redis is created in **Basic** tier with the minimum cache size. There's a small cost associated with this tier. You can scale it up to higher performance tiers for higher availability, clustering, and other features. See [Azure Cache for Redis pricing](https://azure.microsoft.com/pricing/details/cache/).

+> [!IMPORTANT]

+> The default TLS policy is set to AppGwSslPolicy20220101 for API versions 2023-02-01 or higher. Visit [TLS policy overview](./application-gateway-ssl-policy-overview.md#default-tls-policy) to know more.

+- If no TLS policy is chosen, a [default TLS policy](application-gateway-ssl-policy-overview.md#default-tls-policy) gets applied based on the API version used to create that resource.

+| **Default** | True<br/>(for API version < 2023-02-01) | False | False | True<br/>(for API version >= 2023-02-01) | False |

+### Default TLS policy

+When no specific SSL Policy is specified in the application gateway resource configuration, a default TLS policy gets applied. The selection of this default policy is based on the API version used to create that gateway.

+- **For API versions 2023-02-01 or higher**, the minimum protocol version is set to 1.2 (version up to 1.3 is supported). The gateways created with these API versions will see a read-only property **defaultPredefinedSslPolicy:[AppGwSslPolicy20220101](application-gateway-ssl-policy-overview.md#predefined-tls-policy)** in the resource configuration. This property defines the default TLS policy to use.

+- **For older API versions < 2023-02-01**, the minimum protocol version is set to 1.0 (versions up to 1.2 are supported) as they use the predefined policy [AppGwSslPolicy20150501](application-gateway-ssl-policy-overview.md#predefined-tls-policy) as default.

+If the default TLS doesnΓÇÖt fit your requirement, choose a different Predefined policy or use a Custom one.

+> [!NOTE]

+> Azure PowerShell and CLI support for the updated default TLS policy is coming soon.

+Since the application gateway resource is deployed inside a virtual network, we also perform a check to verify the permission on the provided virtual network resource. This validation is performed during both creation and management operations. You should check your [Azure role-based access control](../role-based-access-control/role-assignments-list-portal.md) to verify the users (and service principals) that operate application gateways also have at least **Microsoft.Network/virtualNetworks/subnets/join/action** permission on the Virtual Network or Subnet. This is also applies to the [Managed Identities for Application Gateway Ingress Controller](./tutorial-ingress-controller-add-on-new.md#deploy-an-aks-cluster-with-the-add-on-enabled).

+> We suggest using this feature control (AFEC) provision only as interim mitigation until you assign the correct permission. You must prioritize fixing the permissions for all the applicable Users (and Service Principals) and then unregister this AFEC flag to reintroduce the permission verification on the Virtual Network resource. It is recommended not to permanently depend on this AFEC method, as it will be removed in the future.

+If the virtual network Application Gateway is deployed into doesn't reside in the same resource group as the AKS nodes, please ensure the identity used by AGIC has Network Contributor role assigned to the subnet the Application Gateway is deployed into.

+```azurecli-interactive

+# Get application gateway id from AKS addon profile

+appGatewayId=$(az aks show -n myCluster -g myResourceGroup -o tsv --query "addonProfiles.ingressApplicationGateway.config.effectiveApplicationGatewayId")

+# Get Application Gateway subnet id

+appGatewaySubnetId=$(az network application-gateway show --ids $appGatewayId -o tsv --query "gatewayIPConfigurations[0].subnet.id")

+# Get AGIC addon identity

+agicAddonIdentity=$(az aks show -n myCluster -g myResourceGroup -o tsv --query "addonProfiles.ingressApplicationGateway.identity.clientId")

+# Assign network contributor role to AGIC addon identity to subnet that contains the Application Gateway

+az role assignment create --assignee $agicAddonIdentity --scope $appGatewaySubnetId --role "Network Contributor"

+```

+ - "5000:5000"

+ ports:

+ - "5000:5000"

+ - "5000:5000"

+ azure-cognitive-service-id-document:

+ - "5000:5000"

+ - "5000:5000"

+ - "5000:5000"

+ - "5000:5000"

+ - "5000:5000"

+Download for [Windows](https://download.microsoft.com/download/2/6/e/26e2b001-1364-41ed-90b0-1340a44ba409/AzureConnectedMachineAgent.msi) or [Linux](manage-agent.md#installing-a-specific-version-of-the-agent)

+The first release of agent version 1.31 had a known issue affecting customers using proxy servers. The issue is indicated by the `AZCM0026: Network Error` and a message about "no IP addresses found" when connecting a server to Azure Arc using a proxy server. A newer version of agent 1.31 was released on June 14, 2023 that addresses this issue.

+To check if you're running the latest version of the Azure connected machine agent, navigate to the server in the Azure portal or run `azcmagent show` from a terminal on the server itself and look for the "Agent version." The table below shows the version numbers for the first and patched releases of agent 1.31.

+| Package type | Version number with proxy issue | Version number of patched agent |

+| | - | - |

+| Windows | 1.31.02347.1069 | 1.31.02356.1083 |

+| RPM-based Linux | 1.31.02347.957 | 1.31.02356.970 |

+| DEB-based Linux | 1.31.02347.939 | 1.31.02356.952 |

+- Remote access to the server using Windows Admin Center or SSH is not supported over private link at this time.

+| E10 | 12 GB | 4 | 4,000 | 300,000 | 207,000 |

+| E20 | 25 GB | 4 | 4,000 | 680,000 | 480,000 |

+| E50 | 50 GB | 8 | 8,000 | 1,200,000 | 900,000 |

+| E100 | 100 GB | 16 | 10,000 | 1,700,000 | 1,650,000 |

+| F300 | 384 GB | 8 | 3,200 | 500,000 | 390,000 |

+| F700 | 715 GB | 16 | 6,400 | 500,000 | 370,000 |

+| F1500 | 1455 GB | 32 | 12,800 | 530,000 | 390,000 |

+| E10 | 12 GB | 4 | 4,000 | 1,400,000 | 1,000,000 |

+| E20 | 25 GB | 4 | 4,000 | 1,200,000 | 900,000 |

+| E50 | 50 GB | 8 | 8,000 | 2,300,000 | 1,700,000 |

+| E100 | 100 GB | 16 | 10,000 | 3,000,000 | 2,500,000 |

+| F300 | 384 GB | 8 | 3,200 | 1,500,000 | 1,200,000 |

+| F700 | 715 GB | 16 | 6,400 | 1,600,000 | 1,200,000 |

+| F1500 | 1455 GB | 32 | 12,800 | 1,600,000 | 1,110,000 |

+| E10 | 200,000 | 830,000 | 930,000 |

+| E20 | 480,000 | 710,000 | 950,000 |

+| E50 | 900,000 | 1,110,000 | 1,200,000 |

+| E100 | 1,600,000 | 1,120,000 | 1,200,000 |

+| F300 | 390,000 | 640,000 |

+| F700 | 370,000 | 610,000 |

+| F1500 | 390,000 | 670,000 |

+| E10 | 1,00,000 | 1,900,000 | 2,500,000 |

+| E20 | 900,000 | 1,700,000 | 2,300,000 |

+| E50 | 1,700,000 | 3,000,000 | 3,900,000 |

+| E100 | 2,500,000 | 4,400,000 | 4,900,000|

+|::|::| :| :|

+| F300 | 1,200,000 | 2,600,000 |

+| F700 | 1,200,000 | 2,600,000 |

+| F1500 | 1,100,000 | 2,800,000 |

+- [Get started with Azure Functions triggers in Azure Cache for Redis](cache-tutorial-functions-getting-started.md)

+- [Using Azure Functions and Azure Cache for Redis to create a write-behind cache](cache-tutorial-write-behind.md)

+ Title: 'Tutorial: Function - Azure Cache for Redis and Azure Functions'

+description: Learn how to use Azure functions with Azure Cache for Redis.

+# Get started with Azure Functions triggers in Azure Cache for Redis

+The following tutorial shows how to implement basic triggers with Azure Cache for Redis and Azure Functions. This tutorial uses VS Code to write and deploy the Azure Function in C#.

+## Requirements

+- Azure subscription

+- [Visual Studio Code](https://code.visualstudio.com/)

+## Instructions

+### Set up an Azure Cache for Redis instance

+Create a new **Azure Cache for Redis** instance using the Azure portal or your preferred CLI tool. We use a _Standard C1_ instance, which is a good starting point. Use the [quickstart guide](quickstart-create-redis.md) to get started.

+<!--  -->

+The default settings should suffice. We use a public endpoint for this demo, but we recommend you use a private endpoint for anything in production.

+Creating the cache can take a few minutes, so feel move to the next section while creating the cache completes.

+### Set up Visual Studio Code

+If you havenΓÇÖt installed the functions extension for VS Code, search for _Azure Functions_ in the extensions menu, and select **Install**. If you donΓÇÖt have the C# extension installed, install it, too.

+<!--  -->

+Next, go to the **Azure** tab, and sign-in to your existing Azure account, or create a new one:

+Create a new local folder on your computer to hold the project that you're building. In our example, we use ΓÇ£AzureRedisFunctionDemoΓÇ¥.

+In the Azure tab, create a new functions app by clicking on the lightning bolt icon in the top right of the **Workspace** box in the lower left of the screen.

+<!--  -->

+Select the new folder that youΓÇÖve created to start the creation of a new Azure Functions project. You get several on-screen prompts. Select:

+- **C#** as the language

+- **.NET 6.0 LTS** as the .NET runtime

+- **Skip for now** as the project template

+> [!NOTE]

+> If you donΓÇÖt have the .NET Core SDK installed, youΓÇÖll be prompted to do so.

+The new project is created:

+<!--  -->

+### Install necessary NuGet packages

+You need to install two NuGet packages:

+1. [StackExchange.Redis](https://www.nuget.org/packages/StackExchange.Redis/), which is the primary .NET client for Redis.

+1. `Microsoft.Azure.WebJobs.Extensions.Redis`, which is the extension that allows Redis keyspace notifications to be used as triggers in Azure Functions.

+Install these packages by going to the **Terminal** tab in VS Code and entering the following commands:

+```terminal

+dotnet add package StackExchange.Redis

+dotnet add package Microsoft.Azure.WebJobs.Extensions.Redis

+dotnet restore

+```

+### Configure cache

+Go to your newly created Azure Cache for Redis instance. Two steps are needed here.

+First, enable **keyspace notifications** on the cache to trigger on keys and commands. Go to your cache in the Azure portal and select the **Advanced settings** from the Resource menu. Scroll down to the field labeled _notify-keyspace-events_ and enter ΓÇ£KEAΓÇ¥. Then select Save at the top of the window. ΓÇ£KEAΓÇ¥ is a configuration string that enables keyspace notifications for all keys and events. More information on keyspace configuration strings can be found [here](https://redis.io/docs/manual/keyspace-notifications/).

+<!--  -->

+Second, select **Access keys** from the Resource menu and write down/copy the Primary connection string field. We use the access key to connect to the cache.

+<!--  -->

+### Set up the example code

+Go back to VS Code, add a file to the project called `RedisFunctions.cs` Copy and paste the code sample:

+```csharp

+using Microsoft.Extensions.Logging;

+using System.Text.Json;

+namespace Microsoft.Azure.WebJobs.Extensions.Redis.Samples

+{

+ public static class RedisSamples

+ {

+ public const string localhostSetting = "redisLocalhost";

+ [FunctionName(nameof(PubSubTrigger))]

+ public static void PubSubTrigger(

+ [RedisPubSubTrigger(localhostSetting, "pubsubTest")] RedisMessageModel model,

+ ILogger logger)

+ {

+ logger.LogInformation(JsonSerializer.Serialize(model));

+ }

+ [FunctionName(nameof(PubSubTriggerResolvedChannel))]

+ public static void PubSubTriggerResolvedChannel(

+ [RedisPubSubTrigger(localhostSetting, "%pubsubChannel%")] RedisMessageModel model,

+ ILogger logger)

+ {

+ logger.LogInformation(JsonSerializer.Serialize(model));

+ }

+ [FunctionName(nameof(KeyspaceTrigger))]

+ public static void KeyspaceTrigger(

+ [RedisPubSubTrigger(localhostSetting, "__keyspace@0__:keyspaceTest")] RedisMessageModel model,

+ ILogger logger)

+ {

+ logger.LogInformation(JsonSerializer.Serialize(model));

+ }

+ [FunctionName(nameof(KeyeventTrigger))]

+ public static void KeyeventTrigger(

+ [RedisPubSubTrigger(localhostSetting, "__keyevent@0__:del")] RedisMessageModel model,

+ ILogger logger)

+ {

+ logger.LogInformation(JsonSerializer.Serialize(model));

+ }

+ [FunctionName(nameof(ListsTrigger))]

+ public static void ListsTrigger(

+ [RedisListTrigger(localhostSetting, "listTest")] RedisMessageModel model,

+ ILogger logger)

+ {

+ logger.LogInformation(JsonSerializer.Serialize(model));

+ }

+ [FunctionName(nameof(ListsMultipleTrigger))]

+ public static void ListsMultipleTrigger(

+ [RedisListTrigger(localhostSetting, "listTest1 listTest2")] RedisMessageModel model,

+ ILogger logger)

+ {

+ logger.LogInformation(JsonSerializer.Serialize(model));

+ }

+ [FunctionName(nameof(StreamsTrigger))]

+ public static void StreamsTrigger(

+ [RedisStreamTrigger(localhostSetting, "streamTest")] RedisMessageModel model,

+ ILogger logger)

+ {

+ logger.LogInformation(JsonSerializer.Serialize(model));

+ }

+ [FunctionName(nameof(StreamsMultipleTriggers))]

+ public static void StreamsMultipleTriggers(

+ [RedisStreamTrigger(localhostSetting, "streamTest1 streamTest2")] RedisMessageModel model,

+ ILogger logger)

+ {

+ logger.LogInformation(JsonSerializer.Serialize(model));

+ }

+ }

+}

+```

+This tutorial shows multiple different triggers:

+1. _PubSubTrigger_, which is triggered when activity is published to the pub/sub channel named `pubsubTest`

+1. _KeyspaceTrigger_, which is built on the Pub/Sub trigger. Use it to look for changes to the key `keyspaceTest`

+1. _KeyeventTrigger_, which is also built on the Pub/Sub trigger. Use it to look for any use of the`DEL` command.

+1. _ListTrigger_, which looks for changes to the list `listTest`

+1. _ListMultipleTrigger_, which looks for changes to list `listTest1` and `listTest2`

+1. _StreamTrigger_, which looks for changes to the stream `streamTest`

+1. _StreamMultipleTrigger_, which looks for changes to streams `streamTest1` and `streamTest2`

+To connect to your cache, take the connection string you copied from earlier and paste to replace the value of `localhost` at the top of the file, set to `127.0.0.1:6379` by default.

+<!--  -->

+### Build and run the code locally

+Switch to the **Run and debug** tab in VS Code and select the green arrow to debug the code locally. If you donΓÇÖt have Azure Functions core tools installed, you're prompted to do so. In that case, youΓÇÖll need to restart VS Code after installing.

+The code should build successfully, which you can track in the Terminal output.

+To test the trigger functionality, try creating and deleting the _keyspaceTest_ key. You can use any way you prefer to connect to the cache. An easy way is to use the built-in Console tool in the Azure Cache for Redis portal. Bring up the cache instance in the Azure portal, and select **Console** to open it.

+<!--  -->

+After it's open, try the following commands:

+- SET keyspaceTest 1

+- SET keyspaceTest 2

+- DEL keyspaceTest

+- PUBLISH pubsubTest testMessage

+- LPUSH listTest test

+- XADD streamTest * name Clippy

+<!--  -->

+You should see the triggers activating in the terminal:

+<!--  -->

+### Deploy code to an Azure function

+Create a new Azure function by going back to the Azure tab, expanding your subscription, and right clicking on **Function App**. Select **Create a Function App in Azure…(Advanced)**.

+<!--  -->

+You see several prompts on information to configure the new functions app:

+- Enter a unique name

+- Choose **.NET 6** as the runtime stack

+- Choose either **Linux** or **Windows** (either works)

+- Select an existing or new resource group to hold the Function App

+- Choose the same region as your cache instance

+- Select **Premium** as the hosting plan

+- Create a new App Service plan

+- Choose the **EP1** pricing tier.

+- Choose an existing storage account or create a new one

+- Create a new Application Insights resource. We use the resource to confirm the trigger is working.

+> [!IMPORTANT]

+> Redis triggers are not currently supported on consumption functions.

+>

+Wait a few minutes for the new Function App to be created. It appears in the drop down under **Function App** in your subscription. Right click on the new function app and select **Deploy to Function App…**

+<!--  -->

+The app builds and starts deploying. You can track progress in the **Output Window**.

+Once deployment is complete, open your Function App in the Azure portal and select **Log Stream** from the Resource menu. Wait for log analytics to connect, and then use the Redis console to activate any of the triggers. You should see the triggers being logged here.

+<!--  -->

+## Next steps

+- [Serverless event-based architectures with Azure Cache for Redis and Azure Functions (preview)](cache-how-to-functions.md)

+- [Build a write-behind cache using Azure Functions](cache-tutorial-write-behind.md)

+ Title: 'Tutorial: Create a write-behind cache use Azure Cache for Redis and Azure Functions'

+description: Learn how to use Using Azure Functions and Azure Cache for Redis to create a write-behind cache.

+# Using Azure Functions and Azure Cache for Redis to create a write-behind cache

+The objective of this tutorial is to use an Azure Cache for Redis instance as a [write-behind cache](https://azure.microsoft.com/resources/cloud-computing-dictionary/what-is-caching/#types-of-caching). The _write-behind_ pattern in this tutorial shows how writes to the cache trigger corresponding writes to an Azure SQL database.

+We use the [Redis trigger for Azure Functions](cache-how-to-functions.md) to implement this functionality. In this scenario, you see how to use Azure Cache for Redis to store inventory and pricing information, while backing up that information in an Azure SQL Database.

+Every new item or new price written to the cache is then reflected in a SQL table in the database.

+## Requirements

+- Azure account

+- Completion of the previous tutorial, [Get started with Azure Functions triggers in Azure Cache for Redis](cache-tutorial-functions-getting-started.md) with the following resources provisioned:

+ - Azure Cache for Redis instance

+ - Azure Function instance

+ - VS Code environment set up with NuGet packages installed

+## Instructions

+### Create and configure a new Azure SQL Database instance

+The SQL database is the backing database for this example. You can create an Azure SQL database instance through the Azure portal or through your preferred method of automation.

+This example uses the portal.

+First, enter a database name and select **Create new** to create a new SQL server to hold the database.

+Select **Use SQL authentication** and enter an admin sign in and password. Make sure to remember these or write them down. When deploying a SQL server in production, use Azure Active Directory (Azure AD) authentication instead.

+Go to the **Networking** tab, and choose **Public endpoint** as a connection method. Select **Yes** for both firewall rules that appear. This endpoint allows access from your Azure Functions app.

+Select **Review + create** and then **Create** after validation finishes. The SQL database starts to deploy.

+Once deployment completes, go to the resource in the Azure portal, and select the **Query editor** tab. Create a new table called ΓÇ£inventoryΓÇ¥ that holds the data you'll be writing to it. Use the following SQL command to make a new table with two fields:

+- `ItemName`, lists the name of each item

+- `Price`, stores the price of the item

+```sql

+CREATE TABLE inventory (

+ ItemName varchar(255),

+ Price decimal(18,2)

+ );

+```

+Once that command has completed, expand the **Tables** folder and verify that the new table was created.

+### Configure the Redis trigger

+First, make a copy of the same VS Code project used in the previous tutorial. Copy the folder from the previous tutorial under a new name, such as ΓÇ£RedisWriteBehindTriggerΓÇ¥ and open it up in VS Code.

+In this example, weΓÇÖre going to use the [pub/sub trigger](cache-how-to-functions.md#redispubsubtrigger) to trigger on `keyevent` notifications. The following list shows our goals:

+1. Trigger every time a SET event occurs. A SET event happens when either new keys are being written to the cache instance or the value of a key is being changed.

+1. Once a SET event is triggered, access the cache instance to find the value of the new key.

+1. Determine if the key already exists in the ΓÇ£inventoryΓÇ¥ table in the Azure SQL database.

+ 1. If so, update the value of that key.

+ 1. If not, write a new row with the key and its value.

+To do this, copy and paste the following code in redisfunction.cs, replacing the existing code.

+```csharp

+using System.Text.Json;

+using Microsoft.Extensions.Logging;

+using StackExchange.Redis;

+using System;

+using System.Data.SqlClient;

+namespace Microsoft.Azure.WebJobs.Extensions.Redis.Samples

+{

+ public static class RedisSamples

+ {

+ public const string cacheAddress = "<YourRedisConnectionString>";

+ public const string SQLAddress = "<YourSQLConnectionString>";

+ [FunctionName("KeyeventTrigger")]

+ public static void KeyeventTrigger(

+ [RedisPubSubTrigger(ConnectionString = cacheAddress, Channel = "__keyevent@0__:set")] RedisMessageModel model,

+ ILogger logger)

+ {

+ // connect to a Redis cache instance

+ var redisConnection = ConnectionMultiplexer.Connect(cacheAddress);

+ var cache = redisConnection.GetDatabase();

+

+ // get the key that was set and its value

+ var key = model.Message;

+ var value = (double)cache.StringGet(key);

+ logger.LogInformation($"Key {key} was set to {value}");

+ // connect to a SQL database

+ String SQLConnectionString = SQLAddress;

+

+ // Define the name of the table you created and the column names

+ String tableName = "dbo.inventory";

+ String column1Value = "ItemName";

+ String column2Value = "Price";

+

+ // Connect to the database. Check if the key exists in the database, if it does, update the value, if it doesn't, add it to the database

+ using (SqlConnection connection = new SqlConnection(SQLConnectionString))

+ {

+ connection.Open();

+ using (SqlCommand command = new SqlCommand())

+ {

+ command.Connection = connection;

+ //Form the SQL query to update the database. In practice, you would want to use a parameterized query to prevent SQL injection attacks.

+ //An example query would be something like "UPDATE dbo.inventory SET Price = 1.75 WHERE ItemName = 'Apple'"

+ command.CommandText = "UPDATE " + tableName + " SET " + column2Value + " = " + value + " WHERE " + column1Value + " = '" + key + "'";

+ int rowsAffected = command.ExecuteNonQuery(); //The query execution returns the number of rows affected by the query. If the key doesn't exist, it will return 0.

+ if (rowsAffected == 0) //If key doesn't exist, add it to the database

+ {

+ //Form the SQL query to update the database. In practice, you would want to use a parameterized query to prevent SQL injection attacks.

+ //An example query would be something like "INSERT INTO dbo.inventory (ItemName, Price) VALUES ('Bread', '2.55')"

+ command.CommandText = "INSERT INTO " + tableName + " (" + column1Value + ", " + column2Value + ") VALUES ('" + key + "', '" + value + "')";

+ command.ExecuteNonQuery();

+

+ logger.LogInformation($"Item " + key + " has been added to the database with price " + value + "");

+ }

+

+ else {

+ logger.LogInformation($"Item " + key + " has been updated to price " + value + "");

+ }

+ }

+ connection.Close();

+ }

+

+ //Log the time the function was executed

+ logger.LogInformation($"C# Redis trigger function executed at: {DateTime.Now}");

+ }

+ }

+}

+```

+> [!IMPORTANT]

+> This example is simplified for the tutorial. For production use, we recommend that you use parameterized SQL queries to prevent SQL injection attacks.

+>

+You need to update the `cacheAddress` and `SQLAddress` variables with the connection strings for your cache instance and your SQL database. You need to manually enter the password for your SQL database connection string, because the password isn't pasted automatically. You can find the Redis connection string in the **Access Keys** of the Resource menu of the Azure Cache for Redis resource. You can find the SQL database connection string under the **ADO.NET** tab in **Connection strings** on the Resource menu in the SQL database resource.

+You see errors in some of the SQL classes. You need to import the `System.Data.SqlClient` NuGet package to resolve these. Go to the VS Code terminal and use the following command:

+```dos

+dotnet add package System.Data.SqlClient

+```

+### Build and run the project

+Go to the **Run and debug tab** in VS Code and run the project. Navigate back to your Azure Cache for Redis instance in the Azure portal and select the **Console** button to enter the Redis Console. Try using some set commands:

+Back in VS Code, you should see the triggers being registered:

+To validate that the triggers are working, go to the SQL database instance in the Azure portal. Then, select **Query editor** from the Resource menu. Create a **New Query** with the following SQL to view the top 100 items in the inventory table:

+```sql

+SELECT TOP (100) * FROM [dbo].[inventory]

+```

+You should see the items written to your Azure Cache for Redis instance show up here!

+### Deploy to your Azure Functions App

+The only thing left is to deploy the code to the actual Azure Function app. As before, go to the Azure tab in VS Code, find your subscription, expand it, find the Function App section, and expand that. Select and hold (or right-click) your Azure Function app. Then, select **Deploy to Function App…**

+Once the deployment has finished, go back to your Azure Cache for Redis instance and use SET commands to write more values. You should see these show up in your Azure SQL database as well.

+If youΓÇÖd like to confirm that your Azure Function app is working properly, go to the app in the portal and select the **Log stream** from the Resource menu. You should see the triggers executing there, and the corresponding updates being made to your SQL database.

+If you ever would like to clear the SQL database table without deleting it, you can use the following SQL query:

+```sql

+TRUNCATE TABLE [dbo].[inventory]

+```

+## Summary

+This tutorial and [Get started with Azure Functions triggers in Azure Cache for Redis](cache-tutorial-functions-getting-started.md) show how to use Azure Cache for Redis to trigger Azure Function apps, and how to use that functionality to use Azure Cache for Redis as a write-behind cache with Azure SQL Database. Using Azure Cache for Redis with Azure Functions is a powerful combination that can solve many integration and performance problems.

+## Next steps

+- [Serverless event-based architectures with Azure Cache for Redis and Azure Functions (preview)](cache-how-to-functions.md)

+- [Get started with Azure Functions triggers in Azure Cache for Redis](cache-tutorial-functions-getting-started.md)

+| **Connection** | Required. The _**name**_ of the variable that holds the connection string, resolved through environment variables or through function app settings. Defaults to look up on the variable _**KustoConnectionString**_, at runtime this variable is looked up against the environment. Documentation on connection string can be found at [Kusto connection strings](/azure/data-explorer/kusto/api/connection-strings/kusto) for example:`"KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId` |

+| **connection** | Required. The _**name**_ of the variable that holds the connection string, resolved through environment variables or through function app settings. Defaults to look up on the variable _**KustoConnectionString**_, at runtime this variable is looked up against the environment. Documentation on connection string can be found at [Kusto connection strings](/azure/data-explorer/kusto/api/connection-strings/kusto) for example:`"KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId` |

+| **connection** | Required. The _**name**_ of the variable that holds the connection string, resolved through environment variables or through function app settings. Defaults to look up on the variable _**KustoConnectionString**_, at runtime this variable is looked up against the environment. Documentation on connection string can be found at [Kusto connection strings](/azure/data-explorer/kusto/api/connection-strings/kusto) for example:`"KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId` |

+The attribute's constructor takes the **Database** and the attributes **KQLCommand**, KQLParameters, and the Connection setting name. The **KQLCommand** can be a KQL statement or a KQL function. The connection string setting name corresponds to the application setting (in `local.settings.json` for local development) that contains the [Kusto connection strings](/azure/data-explorer/kusto/api/connection-strings/kusto) for example: `"KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId`. Queries executed by the input binding are parameterized and the values provided in the KQLParameters are used at runtime.

+| **Connection** | Required. The _**name**_ of the variable that holds the connection string, resolved through environment variables or through function app settings. Defaults to look up on the variable _**KustoConnectionString**_, at runtime this variable is looked up against the environment. Documentation on connection string can be found at [Kusto connection strings](/azure/data-explorer/kusto/api/connection-strings/kusto) for example:`"KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId` .|

+| **connection** | Required. The _**name**_ of the variable that holds the connection string, resolved through environment variables or through function app settings. Defaults to look up on the variable _**KustoConnectionString**_, at runtime this variable is looked up against the environment. Documentation on connection string can be found at [Kusto connection strings](/azure/data-explorer/kusto/api/connection-strings/kusto) for example:`"KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId` |

+| **connection** | Required. The _**name**_ of the variable that holds the connection string, resolved through environment variables or through function app settings. Defaults to look up on the variable _**KustoConnectionString**_, at runtime this variable is looked up against the environment. Documentation on connection string can be found at [Kusto connection strings](/azure/data-explorer/kusto/api/connection-strings/kusto) for example: `"KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId` |

+The attribute's constructor takes the Database and the attributes TableName, MappingRef, DataFormat and the Connection setting name. The **KQLCommand** can be a KQL statement or a KQL function. The connection string setting name corresponds to the application setting (in `local.settings.json` for local development) that contains the [Kusto connection strings](/azure/data-explorer/kusto/api/connection-strings/kusto) for example:`"KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId`. Queries executed by the input binding are parameterized and the values provided in the KQLParameters are used at runtime.

+Virtual network address translation (NAT) simplifies outbound-only internet connectivity for virtual networks. When configured on a subnet, all outbound connectivity uses your specified static public IP addresses. An NAT can be useful for apps that need to consume a third-party service that uses an allowlist of IP address as a security measure. To learn more, see [What is Azure NAT Gateway?](../virtual-network/nat-gateway/nat-overview.md).

+This tutorial shows you how to use NAT gateways to route outbound traffic from an HTTP triggered function. This function lets you check its own outbound IP address. During this tutorial, you'll:

+The [Simple WFS example] sample shows how to easily query a Web Feature Service (WFS) and renders the returned features on the map.

+<!--

+<iframe height='700' scrolling='no' title='Simple WFS example' src='//codepen.io/azuremaps/embed/MWwvVYY/?height=500&theme-id=0&default-tab=js,result&embed-version=2&editable=true' frameborder='no' allowtransparency='true' allowfullscreen='true'>See the Pen <a href='https://codepen.io/azuremaps/pen/MWwvVYY/'>Simple WFS example</a> by Azure Maps (<a href='https://codepen.io/azuremaps'>@azuremaps</a>) on <a href='https://codepen.io'>CodePen</a>.</iframe>

+->

+The [WFS filter example] sample demonstrates the use of different filters with the WFS client.

+<!--

+<iframe height='500' scrolling='no' title= 'WFS filter examples' src='//codepen.io/azuremaps/embed/NWqvYrV/?height=500&theme-id=0&default-tab=result&embed-version=2&editable=true' frameborder='no' allowtransparency='true' allowfullscreen='true'>See the Pen <a href='https://codepen.io/azuremaps/pen/NWqvYrV/'>WFS filter examples</a> by Azure Maps (<a href='https://codepen.io/azuremaps'>@azuremaps</a>) on <a href='https://codepen.io'>CodePen</a>.</iframe>

+-->

+The [WFS service explorer] sample is a simple tool for exploring WFS services on Azure Maps.

+<!--

+-->

+To access WFS services hosted on non-CORS enabled endpoints, a CORS enabled proxy service can be passed into the `proxyService` option of the WFS client as shown below.

+[Simple WFS example]: https://samples.azuremaps.com/spatial-io-module/simple-wfs-example

+[WFS filter example]: https://samples.azuremaps.com/spatial-io-module/wfs-filter-examples

+[WFS service explorer]: https://samples.azuremaps.com/spatial-io-module/wfs-service-explorer

+| [Microsoft Sentinel](../../sentinel/overview.md) | <ul><li>Windows Security Events: [GA](../../sentinel/connect-windows-security-events.md?tabs=AMA)</li><li>Windows Forwarding Event (WEF): [GA](../../sentinel/data-connectors/windows-forwarded-events.md)</li><li>Windows DNS logs: [Public preview with Azure Monitor Agent](../../sentinel/connect-dns-ama.md)</li><li>Linux Syslog CEF: [Public preview with Azure Monitor Agent](../../sentinel/connect-cef-ama.md#set-up-the-common-event-format-cef-via-ama-connector)</li></ul> | Sentinel DNS extension, if youΓÇÖre collecting DNS logs. For all other data types, you just need the Azure Monitor Agent extension. | See [Gap analysis for Microsoft Sentinel](../../sentinel/ama-migrate.md#gap-analysis-between-agents) for a comparison of the extra data collected by Microsoft Sentinel. |

+ For information about the KQL operators that transformations support, see [Structure of transformation in Azure Monitor](../essentials/data-collection-transformations-structure.md#kql-limitations).

+

+ > [!Note]

+ > The only columns that are available to apply transfroms against are TimeGenerated and RawData. Other columns are added to the table automatically after the transformation and are not availiable at the time of transformation.

+ > The _ResourceId column can't be used in the trasnformation.

+

+# Collect Firewall logs with Azure Monitor Agent (Private Preview [signup here](https://forms.office.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR5HgP8BLvCpLhecdvdpZy8VUQ0VCRVg2STY0UkYyOU9RNkU3Qk80VkFOMS4u))

+## Troubleshooting Tool

+Use the [Asure monitor troubleshooter tool](use-azure-monitor-agent-troubleshooter.md) to look for common issues and share results with Microsoft.

+- The scope of your rule is explicitly referring to the old resource.

+Application Map also features [Intelligent view](#application-map-intelligent-view) to assist with fast service health investigations.

+## Application Map Intelligent view

+If **Intelligent view** doesn't load, set the configured time frame to six days or less.

+ | Java | [Browser SDK Loader](./java-standalone-config.md#browser-sdk-loader-preview) |

+If you're using the *exec* form, add the parameter `-javaagent:"path/to/applicationinsights-agent-3.4.14.jar"` to the parameter list somewhere before the `"-jar"` parameter, for example:

+ENTRYPOINT ["java", "-javaagent:path/to/applicationinsights-agent-3.4.14.jar", "-jar", "<myapp.jar>"]

+If you're using the *shell* form, add the JVM arg `-javaagent:"path/to/applicationinsights-agent-3.4.14.jar"` somewhere before `-jar`, for example:

+ENTRYPOINT java -javaagent:"path/to/applicationinsights-agent-3.4.14.jar" -jar <myapp.jar>

+COPY agent/applicationinsights-agent-3.4.14.jar applicationinsights-agent-3.4.14.jar

+ENTRYPOINT["java", "-javaagent:applicationinsights-agent-3.4.14.jar", "-jar", "app.jar"]

+In this example we have copied the `applicationinsights-agent-3.4.14.jar` and `applicationinsights.json` files from an `agent` folder (you can choose any folder of your machine). These two files have to be in the same folder in the Docker container.

+JAVA_OPTS="$JAVA_OPTS -javaagent:path/to/applicationinsights-agent-3.4.14.jar"

+CATALINA_OPTS="$CATALINA_OPTS -javaagent:path/to/applicationinsights-agent-3.4.14.jar"

+If the file `<tomcat>/bin/setenv.sh` already exists, modify that file and add `-javaagent:path/to/applicationinsights-agent-3.4.14.jar` to `CATALINA_OPTS`.

+set CATALINA_OPTS=%CATALINA_OPTS% -javaagent:path/to/applicationinsights-agent-3.4.14.jar

+set "CATALINA_OPTS=%CATALINA_OPTS% -javaagent:path/to/applicationinsights-agent-3.4.14.jar"

+If the file `<tomcat>/bin/setenv.bat` already exists, modify that file and add `-javaagent:path/to/applicationinsights-agent-3.4.14.jar` to `CATALINA_OPTS`.

+Locate the file `<tomcat>/bin/tomcat8w.exe`. Run that executable and add `-javaagent:path/to/applicationinsights-agent-3.4.14.jar` to the `Java Options` under the `Java` tab.

+Add `-javaagent:path/to/applicationinsights-agent-3.4.14.jar` to the existing `JAVA_OPTS` environment variable in the file `JBOSS_HOME/bin/standalone.conf` (Linux) or `JBOSS_HOME/bin/standalone.conf.bat` (Windows):

+ JAVA_OPTS="-javaagent:path/to/applicationinsights-agent-3.4.14.jar -Xms1303m -Xmx1303m ..."

+Add `-javaagent:path/to/applicationinsights-agent-3.4.14.jar` to the existing `jvm-options` in `JBOSS_HOME/domain/configuration/host.xml`:

+ <option value="-javaagent:path/to/applicationinsights-agent-3.4.14.jar"/>

+-javaagent:path/to/applicationinsights-agent-3.4.14.jar

+Add `-javaagent:path/to/applicationinsights-agent-3.4.14.jar` to the existing `jvm-options` in `glassfish/domains/domain1/config/domain.xml`:

+ -javaagent:path/to/applicationinsights-agent-3.4.14.jar>

+ -javaagent:path/to/applicationinsights-agent-3.4.14.jar

+-javaagent:path/to/applicationinsights-agent-3.4.14.jar

+Add the JVM arg `-javaagent:"path/to/applicationinsights-agent-3.4.14.jar"` somewhere before `-jar`, for example:

+java -javaagent:"path/to/applicationinsights-agent-3.4.14.jar" -jar <myapp.jar>

+If you're using the *exec* form, add the parameter `-javaagent:"path/to/applicationinsights-agent-3.4.14.jar"` to the parameter list somewhere before the `"-jar"` parameter, for example:

+ENTRYPOINT ["java", "-javaagent:path/to/applicationinsights-agent-3.4.14.jar", "-jar", "<myapp.jar>"]

+If you're using the *shell* form, add the JVM arg `-javaagent:"path/to/applicationinsights-agent-3.4.14.jar"` somewhere before `-jar`, for example:

+ENTRYPOINT java -javaagent:"path/to/applicationinsights-agent-3.4.14.jar" -jar <myapp.jar>

+ <version>3.4.14</version>

+By default, Application Insights Java 3.x expects the configuration file to be named `applicationinsights.json`, and to be located in the same directory as `applicationinsights-agent-3.4.14.jar`.

+If you specify a relative path, it's resolved relative to the directory where `applicationinsights-agent-3.4.14.jar` is located.

+If you specify a relative path, it's resolved relative to the directory where `applicationinsights-agent-3.4.14.jar` is located.

+ <version>3.4.14</version>

+## Browser SDK Loader (preview)

+This feature automatically injects the [Browser SDK Loader](https://github.com/microsoft/ApplicationInsights-JS#snippet-setup-ignore-if-using-npm-setup) into your application's HTML pages, including configuring the appropriate Connection String.

+For example, when your java application returns a response like:

+```html

+<!DOCTYPE html>

+<html lang="en">

+ <head>

+ <title>Title</title>

+ </head>

+ <body>

+ </body>

+</html>

+```

+Then it will be automatically modified to return:

+```html

+<!DOCTYPE html>

+<html lang="en">

+ <head>

+ <script type="text/javascript">

+ !function(v,y,T){var S=v.location,k="script"

+ <!-- Removed for brevity -->

+ connectionString: "YOUR_CONNECTION_STRING"

+ <!-- Removed for brevity --> }});

+ </script>

+ <title>Title</title>

+ </head>

+ <body>

+ </body>

+</html>

+```

+The script is aiming at helping customers to track the web user data, and sent the collecting server-side telemetry back to users' Azure portal. Details can be found at [ApplicationInsights-JS](https://github.com/microsoft/ApplicationInsights-JS)

+If you want to enable this feature, add the below configuration option:

+```json

+"preview": {

+ "browserSdkLoader": {

+ "enabled": true

+ }

+}

+```

+`applicationinsights-agent-3.4.14.jar` is located.

+-javaagent:path/to/applicationinsights-agent-3.4.14.jar

+Download the [applicationinsights-agent-3.4.14.jar](https://github.com/microsoft/ApplicationInsights-Java/releases/download/3.4.14/applicationinsights-agent-3.4.14.jar) file.

+Point the JVM to the jar file by adding `-javaagent:"path/to/applicationinsights-agent-3.4.14.jar"` to your application's JVM args.

+ Create a configuration file named `applicationinsights.json`, and place it in the same directory as `applicationinsights-agent-3.4.14.jar` with the following content:

+ <version>3.4.14</version>

+If you use only one part of the combination, autoscale only takes action in a single direction (scale out or in) until it reaches the maximum, or minimum instance counts, as defined in the profile. This situation isn't optimal. Ideally, you want your resource to scale out at times of high usage to ensure availability. Similarly, at times of low usage, you want your resource to scale in so that you can realize cost savings.

+1. To enable forecast-only mode, select it from the dropdown. Define a scale-out trigger based on *Percentage CPU*. Then select **Save**. The same process applies to enable predictive autoscale. To disable predictive autoscale or forecast-only mode, select **Disable** from the dropdown.

+ *To enable predictive autoscale, create a scale out rule based on 'Percentage CPU' metric. Click here to go to the 'Configure' tab to set an autoscale rule.*

+> [!TIP]

+> Get notified when this page is updated by copying and pasting the following URL into your feed reader:

+>

+>  https://aka.ms/azmon/rss

+This command decompiles a _azuredeploy.parameters.json_ parameters file into a _azuredeploy.parameters.bicepparam_ file. `-bicep-file` specifies the path to the Bicep file (relative to the .bicepparam file) that is referenced in the `using` declaration.

+The `generate-params` command builds a parameters file from the given Bicep file, updates if there's an existing parameters file.

+ "v0.18.4",

+ "v0.17.1",

+ "v0.16.2",

+ "v0.16.1",

+ "v0.15.31",

+ "v0.14.85",

+ "v0.14.46",

+ "v0.14.6",

+ "v0.13.1",

+ "v0.12.40",

+ "v0.12.1",

+ "v0.11.1",

+ "v0.10.61",

+ "v0.10.13",

+ "v0.9.1",

+ "v0.8.9",

+ "v0.8.2",

+ "v0.7.4",

+ "v0.6.18",

+ "v0.6.11",

+ "v0.6.1",

+ "v0.5.6",

+ "v0.4.1318",

+ "v0.4.1272",

+ "v0.4.1124",

+ "v0.4.1008",

+ "v0.4.613",

+ "v0.4.451",

+ "v0.4.412",

+ "v0.4.63"

+> [!NOTE]

+> The Bicep parameters file is only supported in Bicep CLI version 0.18.4 or newer.

+You can compile Bicep parameters files into JSON parameters files to deploy with a Bicep file. See [build-params](./bicep-cli.md#build-params).

+To deploy to different environments, you create more than one parameters file. When you name the parameters files, identify their use such as development and production. For example, use _main.dev.bicepparam_ and _main.prod.bicepparam_ to deploy resources.

+You can use inline parameters and a local parameters file in the same deployment operation. For example, you can specify some values in the local parameters file and add other values inline during deployment. If you provide values for a parameter in both the local parameters file and inline, the inline value takes precedence. This feature hasn't been implemented for Bicep parameters file.

+The shared responsibility matrix table shows the high-level responsibilities between a customer and Microsoft for different aspects of the deployment and management of the private cloud and the customer application workloads.

+ - Any other details, including Availability Zone requirements for integrating with other Azure services (e.g. Azure NetApp Files, Azure Blob Storage)

+ - Any other details, including Availability Zone requirements for integrating with other Azure services (e.g. Azure NetApp Files, Azure Blob Storage)

+ Title: Enable client certificate authentication for Azure Web PubSub Service (Preview)

+description: How to enable client certificate authentication for Azure Web PubSub Service (Preview)

+# Enable client certificate authentication for Azure Web PubSub Service (Preview)

+You can restrict access to your Azure Web PubSub Service by enabling different types of authentication for it. One way to do it is to request a client certificate and validate the certificate in event handlers. This mechanism is called TLS mutual authentication or client certificate authentication. This article shows how to set up your Azure Web PubSub Service to use client certificate authentication.

+> [!Note]

+> Enabling client certificate authentication in browser scenarios is generally not recommended. Different browsers have different behaviors when dealing with client certificate request, while you have little control in JavaScript appliations. If you want to enable client certificate authentication, we recommend you in scenarios where you have strong control over TLS settings, for example, in native applications.

+## Prerequisites

+* An Azure account with an active subscription. If you don't have an Azure account, you can [create an account for free](https://azure.microsoft.com/free/).

+* An Azure Web PubSub service (must be Standard tier or above).

+* An Azure Function used to handle connect events.

+* A client certificate. You need to know its SHA-1 thumbprint.

+## Deploy Azure Web PubSub Service

+Suppose you're going to use a function called `func-client-cert` as event handler to process `connect` events. Clients connect to a hub called `echo`. Here are the Bicep/ARM templates to deploy an Azure Web PubSub service with client certificate authentication enabled and event handlers configured.

+We enable client certificate authentication via the property `tls.clientCertEnabled`.

+We configure an event handler for `connect` event so we can validate client thumbprint. Also note that `anonymousConnectPolicy` needs to be set to `allow` so clients no longer need to send access tokens.

+### Bicep

+```bicep

+param name string

+param hubName string = 'echo'

+param eventHandlerUrl string = 'https://func-client-cert.azurewebsites.net/api/echo'

+param location string = resourceGroup().location

+resource awps 'Microsoft.SignalRService/WebPubSub@2023-03-01-preview' = {

+ name: name

+ location: location

+ sku: {

+ name: 'Standard_S1'

+ tier: 'Standard'

+ size: 'S1'

+ capacity: 1

+ }

+ properties: {

+ tls: {

+ clientCertEnabled: true

+ }

+ }

+}

+resource hub 'Microsoft.SignalRService/WebPubSub/hubs@2023-03-01-preview' = {

+ parent: awps

+ name: '${hubName}'

+ properties: {

+ eventHandlers: [

+ {

+ urlTemplate: eventHandlerUrl

+ userEventPattern: '*'

+ systemEvents: [

+ 'connect'

+ ]

+ }

+ ]

+ anonymousConnectPolicy: 'allow'

+ }

+}

+```

+### ARM

+```json

+{

+ "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

+ "contentVersion": "1.0.0.0",

+ "parameters": {

+ "name": {

+ "type": "String"

+ },

+ "hubName": {

+ "defaultValue": "echo",

+ "type": "String"

+ },

+ "eventHandlerUrl": {

+ "defaultValue": "https://func-client-cert.azurewebsites.net/api/echo",

+ "type": "String"

+ },

+ "location": {

+ "defaultValue": "[resourceGroup().location]",

+ "type": "String"

+ }

+ },

+ "resources": [

+ {

+ "type": "Microsoft.SignalRService/WebPubSub",

+ "apiVersion": "2023-03-01-preview",

+ "name": "[parameters('name')]",

+ "location": "[parameters('location')]",

+ "sku": {

+ "name": "Standard_S1",

+ "tier": "Standard",

+ "size": "S1",

+ "capacity": 1

+ },

+ "properties": {

+ "tls": {

+ "clientCertEnabled": true

+ }

+ }

+ },

+ {

+ "type": "Microsoft.SignalRService/WebPubSub/hubs",

+ "apiVersion": "2023-03-01-preview",

+ "name": "[concat(parameters('name'), '/', parameters('hubName'))]",

+ "dependsOn": [

+ "[resourceId('Microsoft.SignalRService/WebPubSub', parameters('name'))]"

+ ],

+ "properties": {

+ "eventHandlers": [

+ {

+ "urlTemplate": "[parameters('eventHandlerUrl')]",

+ "userEventPattern": "*",

+ "systemEvents": [

+ "connect"

+ ]

+ }

+ ],

+ "anonymousConnectPolicy": "allow"

+ }

+ }

+ ]

+}

+```

+## Validate client certificate in event handler

+You can validate incoming client certificate via its SHA-1 thumbprint in the `connect` event. The value is available in `clientCertificates` field. See [CloudEvents HTTP extension for event handler](reference-cloud-events.md#connect).

+Here are sample function codes to implement validation logic.

+### JavaScript

+```javascript

+module.exports = async function (context, req) {

+ // For client connect event

+ if (req.headers && req.headers['ce-type'] == 'azure.webpubsub.sys.connect') {

+ // CLIENT_CERT_THUMBPRINT should be configured as an environment variable of valid client certificate SHA-1 thumbprint

+ var validCertThumbprint = process.env['CLIENT_CERT_THUMBPRINT'];

+ var certThumbprint = null;

+ if (req.body.clientCertificates) {

+ certThumbprint = req.body.clientCertificates[0].thumbprint;

+ }

+ if (certThumbprint != validCertThumbprint) {

+ context.log('Expect client cert:', validCertThumbprint, 'but got:', certThumbprint);

+ context.res = {

+ status: 403

+ };

+ return;

+ }

+ }

+ context.res = {

+ // status: 200, /* Defaults to 200 */

+ headers: {

+ 'WebHook-Allowed-Origin': '*'

+ },

+ };

+}

+```

+## Certificate rotation

+In case you want to rotate the certificate, you can update your event handler code to accept multiple thumbprints.

+## Missing client certificate

+Azure Web PubSub Service doesn't abort TLS handshake when clients don't provide client certificate. It's up to event handler to decide whether to accept or reject a connection without client certificate.

+## Next steps

+* [How to configure event handler](howto-develop-eventhandler.md)

+* [Golang sample](https://github.com/Azure/azure-webpubsub/blob/main/samples/golang/clientWithCert/Readme.md)

+| **Regions** | **Americas** ΓÇô Central US, East US 2, East US, North Central US, South Central US, West US 2, West US 3, West Central US, West US, Canada Central, Canada East, Brazil South <br> **Asia Pacific** ΓÇô Australia Central, Australia Central 2, Australia East, Australia Southeast, Japan East, Japan West, Korea Central, Korea South, East Asia, Southeast Asia, Central India, South India, West India, China East, China East 2, China East 3, China North, China North 2, China North 3 <br> **Europe** ΓÇô West Europe, North Europe, France Central, UK South, UK West, Germany North, Germany West Central, Switzerland North, Switzerland West, Central Switzerland North, Norway East, Norway West, Sweden Central, Sweden South <br> **Africa / ME** - South Africa North, South Africa West, UAE North, UAE Central <BR> **Azure Government regions** | France South, Germany Central, Germany Northeast, US Gov IOWA |

+| **OS versions** | SLES 12 with SP2, SP3, SP4 and SP5; SLES 15 with SP0, SP1, SP2, SP3, and SP4 <br><br> RHEL 7.4, 7.6, 7.7, 7.9, 8.1, 8.2, 8.4, 8.6, and 9.0. | |

+| **HANA versions** | SDC on HANA 1.x, MDC on HANA 2.x SPS 04, SPS 05 Rev <= 59, SPS 06 (validated for encryption enabled scenarios as well), and SPS 07. | |

+| **HANA deployments** | SAP HANA on a single Azure VM - Scale up only. <br><br> For high availability deployments, both the nodes on the two different machines are treated as individual nodes with separate data chains. | Scale-out <br><br> In high availability deployments, backup doesnΓÇÖt fail over to the secondary node automatically. Configuring backup should be done separately for each node. |

+- [Azure Monitor agent for Linux](../azure-monitor/agents/azure-monitor-agent-manage.md)

+ Title: Use Bicep to create an experiment in Azure Chaos Studio Preview

+description: Sample Bicep templates to create Azure Chaos Studio Preview experiments.

+# Use Bicep to create an experiment in Azure Chaos Studio Preview

+This article includes a sample Bicep file to get started in Azure Chaos Studio Preview, including:

+* Onboarding a resource as a target (for example, a Virtual Machine)

+* Enabling capabilities on the target (for example, Virtual Machine Shutdown)

+* Creating a Chaos Studio experiment

+* Assigning the necessary permissions for the experiment to execute

+## Build a Virtual Machine Shutdown experiment

+In this sample, we create a chaos experiment with a single target resource and a single virtual machine shutdown fault. You can modify the experiment by referencing the [fault library](chaos-studio-fault-library.md) and [recommended role assignments](chaos-studio-fault-providers.md).

+### Prerequisites

+This sample assumes:

+* The Chaos Studio resource provider is already registered with your Azure subscription

+* You already have a resource group in a region supported by Chaos Studio

+* A virtual machine is deployed in the resource group

+### Review the Bicep file

+```bicep

+@description('The existing virtual machine resource you want to target in this experiment')

+param targetName string

+@description('Desired name for your Chaos Experiment')

+param experimentName string

+@description('Desired region for the experiment, targets, and capabilities')

+param location string = resourceGroup().location

+// Define Chaos Studio experiment steps for a basic Virtual Machine Shutdown experiment

+param experimentSteps array = [

+ {

+ name: 'Step1'

+ branches: [

+ {

+ name: 'Branch1'

+ actions: [

+ {

+ name: 'urn:csci:microsoft:virtualMachine:shutdown/1.0'

+ type: 'continuous'

+ duration: 'PT10M'

+ parameters: [

+ {

+ key: 'abruptShutdown'

+ value: 'true'

+ }

+ ]

+ selectorId: 'Selector1'

+ }

+ ]

+ }

+ ]

+ }

+]

+// Reference the existing Virtual Machine resource

+resource vm 'Microsoft.Compute/virtualMachines@2023-03-01' existing = {

+ name: targetName

+}

+// Deploy the Chaos Studio target resource to the Virtual Machine

+resource chaosTarget 'Microsoft.Chaos/targets@2022-10-01-preview' = {

+ name: 'Microsoft-VirtualMachine'

+ location: location

+ scope: vm

+ properties: {}

+ // Define the capability -- in this case, VM Shutdown

+ resource chaosCapability 'capabilities' = {

+ name: 'Shutdown-1.0'

+ }

+}

+// Define the role definition for the Chaos experiment

+resource chaosRoleDefinition 'Microsoft.Authorization/roleDefinitions@2022-04-01' existing = {

+ scope: vm

+ // In this case, Virtual Machine Contributor role -- see https://learn.microsoft.com/azure/role-based-access-control/built-in-roles

+ name: '9980e02c-c2be-4d73-94e8-173b1dc7cf3c'

+}

+// Define the role assignment for the Chaos experiment

+resource chaosRoleAssignment 'Microsoft.Authorization/roleAssignments@2020-04-01-preview' = {

+ name: guid(vm.id, chaosExperiment.id, chaosRoleDefinition.id)

+ scope: vm

+ properties: {

+ roleDefinitionId: chaosRoleDefinition.id

+ principalId: chaosExperiment.identity.principalId

+ principalType: 'ServicePrincipal'

+ }

+}

+// Deploy the Chaos Studio experiment resource

+resource chaosExperiment 'Microsoft.Chaos/experiments@2022-10-01-preview' = {

+ name: experimentName

+ location: location // Doesn't need to be the same as the Targets & Capabilities location

+ identity: {

+ type: 'SystemAssigned'

+ }

+ properties: {

+ selectors: [

+ {

+ id: 'Selector1'

+ type: 'List'

+ targets: [

+ {

+ id: chaosTarget.id

+ type: 'ChaosTarget'

+ }

+ ]

+ }

+ ]

+ startOnCreation: false // Change this to true if you want to start the experiment on creation

+ steps: experimentSteps

+ }

+}

+```

+## Deploy the Bicep file

+1. Save the Bicep file as `chaos-vm-shutdown.bicep` to your local computer.

+1. Deploy the Bicep file using either Azure CLI or Azure PowerShell, replacing `exampleRG` with the existing resource group that includes the virtual machine you want to target.

+ # [CLI](#tab/CLI)

+ ```azurecli

+ az deployment group create --resource-group exampleRG --template-file chaos-vm-shutdown.bicep

+ ```

+ # [PowerShell](#tab/PowerShell)

+ ```azurepowershell

+ New-AzResourceGroupDeployment -ResourceGroupName exampleRG -TemplateFile ./chaos-vm-shutdown.bicep

+ ```

+1. When prompted, enter the following values:

+ * **targetName**: the name of an existing Virtual Machine within your resource group that you want to target

+ * **experimentName**: the desired name for your Chaos Experiment

+ * **location**: the desired region for the experiment, targets, and capabilities

+1. The template should deploy within a few minutes. Once the deployment is complete, navigate to Chaos Studio in the Azure portal, select **Experiments**, and find the experiment created by the template. Select it, then **Start** the experiment.

+## Next steps

+* [Learn more about Chaos Studio](chaos-studio-overview.md)

+* [Learn more about chaos experiments](chaos-studio-chaos-experiments.md)

+> [**Learn more about Document Translation operations**](../reference/rest-api-guide.md)

+| Custom text classification | `2022-05-01`, `2022-10-01-preview`, `2023-04-01` | `2022-05-01` | `2022-10-01-preview` |

+| Conversational language understanding | `2022-05-01`, `2022-10-01-preview`, `2023-04-01` | `2023-04-01` | `2022-10-01-preview` |

+| Custom named entity recognition | `2022-05-01`, `2022-10-01-preview`, `2023-04-01` | `2023-04-01` | `2022-10-01-preview` |

+| Orchestration workflow | `2022-05-01`, `2022-10-01-preview`, `2023-04-01` | `2023-04-01` | `2022-10-01-preview` |

+|`{API-VERSION}` | The [version](../../concepts/model-lifecycle.md#api-versions) of the API you are calling. | `2023-04-01` |

+|LUIS authoring APIs and SDK support in .NET, Python, Java, and Node.js |[CLU Authoring REST APIs](https://aka.ms/clu-authoring-apis). | For more information, see the [quickstart article](../quickstart.md?pivots=rest-api) for information on the CLU authoring APIs. [Refactoring](#do-i-have-to-refactor-my-code-if-i-migrate-my-applications-from-luis-to-clu) will be necessary to use the CLU authoring APIs. |

+|LUIS Runtime APIs and SDK support in .NET, Python, Java, and Node.js |[CLU Runtime APIs](https://aka.ms/clu-runtime-api). CLU Runtime SDK support for [.NET](/dotnet/api/overview/azure/ai.language.conversations-readme) and [Python](/python/api/overview/azure/ai-language-conversations-readme?view=azure-python-preview&preserve-view=true). | See [how to call the API](../how-to/call-api.md#use-the-client-libraries-azure-sdk) for more information. [Refactoring](#do-i-have-to-refactor-my-code-if-i-migrate-my-applications-from-luis-to-clu) will be necessary to use the CLU runtime API response. |

+ |`{API-VERSION}` | The [version](../../concepts/model-lifecycle.md#api-versions) of the API you are calling. | `2023-04-01` |

+[CLU authoring APIs](https://aka.ms/clu-authoring-apis): Instead of LUIS's specific CRUD APIs for individual actions such as _add utterance_, _delete entity_, and _rename intent_, CLU offers an [import API](/rest/api/language/2023-04-01/conversational-analysis-authoring/import) that replaces the full content of a project using the same name. If your service used LUIS programmatic APIs to provide a platform for other customers, you must consider this new design paradigm. All other APIs such as: _listing projects_, _training_, _deploying_, and _deleting_ are available. APIs for actions such as _importing_ and _deploying_ are asynchronous operations instead of synchronous as they were in LUIS.

+[CLU runtime APIs](https://aka.ms/clu-runtime-api): The new API request and response includes many of the same parameters such as: _query_, _prediction_, _top intent_, _intents_, _entities_, and their values. The CLU response object offers a more straightforward approach. Entity predictions are provided as they are within the utterance text, and any additional information such as resolution or list keys are provided in extra parameters called `extraInformation` and `resolution`.

+description: Learn about Azure Communication Services Call Automation API.

+Azure Communication Services Call Automation provides developers the ability to build server-based, intelligent call workflows, and call recording for voice and Public Switched Telephone Network(PSTN) channels. The SDKs, available in C#, Java, JavaScript and Python, use an action-event model to help you build personalized customer interactions. Your communication applications can listen to real-time call events and perform control plane actions (like answer, transfer, play audio, start recording, etc.) to steer and control calls based on your business logic.

+| Feature Area | Capability | .NET | Java | JavaScript | Python |

+| -| -- | | -- | - | |

+| Pre-call scenarios | Answer a one-to-one call | ✔️ | ✔️ | ✔️ | ✔️ |

+| | Answer a group call | ✔️ | ✔️ | ✔️ | ✔️ |

+| | Place new outbound call to one or more endpoints | ✔️ | ✔️ | ✔️ | ✔️ |

+| | Redirect* (forward) a call to one or more endpoints | ✔️ | ✔️ | ✔️ | ✔️ |

+| | Reject an incoming call | ✔️ | ✔️ | ✔️ | ✔️ |

+| Mid-call scenarios | Add one or more endpoints to an existing call | ✔️ | ✔️ | ✔️ | ✔️ |

+| | Play Audio from an audio file | ✔️ | ✔️ | ✔️ | ✔️ |

+| | Recognize user input through DTMF | ✔️ | ✔️ | ✔️ | ✔️ |

+| | Remove one or more endpoints from an existing call| ✔️ | ✔️ | ✔️ | ✔️ |

+| | Blind Transfer* a 1:1 call to another endpoint | ✔️ | ✔️ | ✔️ | ✔️ |

+| | Hang up a call (remove the call leg) | ✔️ | ✔️ | ✔️ | ✔️ |

+| | Terminate a call (remove all participants and end call)| ✔️ | ✔️ | ✔️ | ✔️ |

+| | Cancel media operations | ✔️ | ✔️ | ✔️ | ✔️ |

+| Query scenarios | Get the call state | ✔️ | ✔️ | ✔️ | ✔️ |

+| | Get a participant in a call | ✔️ | ✔️ | ✔️ | ✔️ |

+| | List all participants in a call | ✔️ | ✔️ | ✔️ | ✔️ |

+| Call Recording | Start/pause/resume/stop recording | ✔️ | ✔️ | ✔️ | ✔️ |

+First, we need to define which scenarios can trigger an `IncomingCall` event. The primary concept to remember is that a call to an Azure Communication Services identity or Public Switched Telephone Network (PSTN) number triggers an `IncomingCall` event. The following are examples of these resources:

+Given these examples, the following scenarios trigger an `IncomingCall` event sent to Event Grid:

+- As identified in the [calling scenarios](#calling-scenarios) section, your application can be notified even when users make calls between each other. You can then combine this scenario together with the [Call Recording APIs](../voice-video-calling/call-recording.md) to meet compliance needs.

+Here is an example of an Event Grid Webhook subscription where the event type filter is listening only to the `IncomingCall` event.

+Here is an example of an advanced filter on an Event Grid subscription watching for the `data.to.PhoneNumber.Value` string starting with a PSTN phone number of `+18005551212.

+Since the `IncomingCall` notification doesn't have a specific destination other than the Event Grid subscription you've created, you're free to associate any particular number to any endpoint in Azure Communication Services. For example, if you acquired a PSTN phone number of `+14255551212` and want to assign it to a user with an identity of `375f0e2f-e8db-4449-9bf7-2054b02e42b4` in your application, you can maintain a mapping of that number to the identity. When an `IncomingCall` notification is sent matching the phone number in the **to** field, invoke the `Redirect` API and supply the identity of the user. In other words, you maintain the number assignment within your application and route or answer calls at runtime.

+1. Event Grid requires you to prove ownership of your Webhook endpoint before it starts delivering events to that endpoint. This requirement prevents a malicious user from flooding your endpoint with events. If you're facing issues with receiving events, ensure the webhook configured is verified by handling `SubscriptionValidationEvent`. For more information, see this [guide](../../../event-grid/webhook-event-delivery.md).

+2. Upon the receipt of an incoming call event, if your application doesn't respond back with 200Ok to Event Grid in time, Event Grid uses exponential backoff retry to send the again. However, an incoming call only rings for 30 seconds, and acting on a call after that won't work. To avoid retries for expired or stale calls, we recommend setting the retry policy as - Max Event Delivery Attempts to 2 and Event Time to Live to 1 minute. These settings can be found under Additional Features tab of the event subscription. Learn more about retries [here](../../../event-grid/delivery-and-retry.md).

+- Learn about [usage and operational logs](../analytics/logs/call-automation-logs.md) published by call automation.

+- Learn about [usage and operational logs](../analytics/logs/call-automation-logs.md) published by call automation.

+| Call Automation | [npm](https://www.npmjs.com/package/@azure/communication-call-automation) | [NuGet](https://www.nuget.org/packages/Azure.Communication.CallAutomation) | [PyPi](https://pypi.org/project/azure-communication-callautomation/) | [Maven](https://search.maven.org/search?q=a:azure-communication-callautomation) | - | - | - |

+| Azure Resource Manager | [REST](/rest/api/communication/resourcemanager/communication-services)| Service| Provision and manage Communication Services resources|

+|Call Automation|[npm](https://www.npmjs.com/package/@azure/communication-call-automation)|[NuGet](https://www.NuGet.org/packages/Azure.Communication.CallAutomation/)|[PyPi](https://pypi.org/project/azure-communication-callautomation/)|[Maven](https://search.maven.org/artifact/com.azure/azure-communication-callautomation)

+> Call Automation currently doesn't support [Rooms](../../concepts/rooms/room-concept.md) calls.

+> Have you established webhook callback events endpoint? EventProcessor still needs to consume callback events through webhook callback. See **[quickstart](../../quickstarts/call-automation/quickstart-make-an-outbound-call.md)** that describes establishing webhook endpoints.

+|**[Call Automation overview](./concepts/call-automation/call-automation.md)**|Review the Communication Services Call Automation SDK overview.|

+description: In this quickstart, you learn how to make an outbound PSTN call using Azure Communication Services using Call Automation

+Azure Communication Services Call Automation APIs are a powerful way to create interactive calling experiences. In this quick start, we cover a way to make an outbound call and recognize various events in the call.

+>

+> [!div class="nextstepaction"]

+> [Build workflow for outbound calls using the purchased phone numbers](../call-automation/quickstart-make-an-outbound-call.md)

+>

+> [Get started with calling in applications](../voice-video-calling/getting-started-with-calling.md)

+- Learn about [call automation](../../concepts/call-automation/call-automation.md) to build workflows that [route and manage calls](../../how-tos/call-automation/actions-for-call-control.md) to Communication Services.

+[Anjuna](https://www.anjuna.io/) provides SGX platform software to run unmodified containers on AKS. For more information, see Anjuna's [documentation about functionality and sample applications](https://www.anjuna.io/partners/microsoft-azure).

+Get started with a sample Redis Cache and Python Custom Application [here](https://www.anjuna.io/partners/microsoft-azure)

+## Azure Managed Instance for Apache Cassandra

+For some customers, adapting to API for Cassandra can be a challenge due to differences in behaviour and/or configuration, especially for lift-and-shift migrations. [Azure Managed Instance for Apache Cassandra](../../managed-instance-apache-cassandr) is a first-party Azure service for hosting and maintaining pure open-source Apache Cassandra clusters with 100% compatibility.

+## Azure Managed Instance for Apache Cassandra

+For some customers, adapting to API for Cassandra can be a challenge due to differences in behaviour and/or configuration, especially for lift-and-shift migrations. If a feature that is critical for your application is listed as not supported below, consider using [Azure Managed Instance for Apache Cassandra](../../managed-instance-apache-cassandr). This is a first-party Azure service for hosting and maintaining pure open-source Apache Cassandra clusters with 100% compatibility.

+When you issue a SQL query, the results are returned in a segmented fashion if the result set is too large.

+Partners can use **Home** > **Savings plan** in the [Azure portal](https://portal.azure.com/) to purchase savings plans on behalf of their customers.

+As of June 2023, partners can purchase an Azure savings plan through Partner Center. Previously, Azure savings plan was only supported for purchase through the Azure portal. Partners can now purchase Azure savings plan through the Partner Center portal, APIs, or they can continue to use the Azure portal.

+To purchase Azure savings plan using the Partner Center APIs, see [Purchase Azure savings plans](/partner-center/developer/azure-purchase-savings-plan).

+- On-demand Capacity Reservation

+>- When you use managed identity authentication for your staging linked service, learn the needed configurations for [Azure Blob](connector-azure-blob-storage.md#managed-identity) and [Azure Data Lake Storage Gen2](connector-azure-data-lake-storage.md#managed-identity) respectively. You also need to grant permissions to your Azure Synapse Analytics workspace managed identity in your staging Azure Blob Storage or Azure Data Lake Storage Gen2 account. To learn how to grant this permission, see [Grant permissions to workspace managed identity](/azure/synapse-analytics/security/how-to-grant-workspace-managed-identity-permissions).

+>- When you use managed identity authentication for your staging linked service, learn the needed configurations for [Azure Blob](connector-azure-blob-storage.md#managed-identity) and [Azure Data Lake Storage Gen2](connector-azure-data-lake-storage.md#managed-identity) respectively. You also need to grant permissions to your Azure Synapse Analytics workspace managed identity in your staging Azure Blob Storage or Azure Data Lake Storage Gen2 account. To learn how to grant this permission, see [Grant permissions to workspace managed identity](/azure/synapse-analytics/security/how-to-grant-workspace-managed-identity-permissions).

+- **Cause**: Azure Cosmos DB limits the size of a single request to 2 MB. The formula is *request size = single document size \* write batch size*. If your document size is large, the default behavior will result in a request size that's too large.

+- **Resolution**: <br>

+You can tune the write batch size. In the copy activity sink, reduce the *write batch size* value (the default value is 10000). <br>

+If reducing the *write batch size* value to 1 still doesn't work, change your Azure Cosmos DB SQL API from V2 to V3. To complete this configuration, you have two options:

+ - **Option 1**: Change your authentication type to service principal or system-assigned managed identity or user-assigned managed identity.

+ - **Option 2**: If you still want to use account key authentication, follow these steps:

+ 1. Create an Azure Cosmos DB for NoSQL linked service.

+ 2. Update the linked service with the following template.

+

+ ```json

+ {

+ "name": "<CosmosDbV3>",

+ "type": "Microsoft.DataFactory/factories/linkedservices",

+ "properties": {

+ "annotations": [],

+ "type": "CosmosDb",

+ "typeProperties": {

+ "useV3": true,

+ "accountEndpoint": "<account endpoint>",

+ "database": "<database name>",

+ "accountKey": {

+ "type": "SecureString",

+ "value": "<account key>"

+ }

+ }

+ }

+ }

+ ```

+> [!NOTE]

+> Alerts with a **delegated access** indication are triggered due to activity of third-party service providers. learn more about [service providers activity indications](/azure/defender-for-cloud/defender-for-resource-manager-usage).

+> [!NOTE]

+> The Microsoft Defender CSPM plan protects across multicloud workloads. With Defender CSPM generally available (GA), the plan will remain free until billing starts on August 1 2023. Billing will apply for compute, database, and storage resources. Billable workloads will be VMs, Storage Accounts, OSS DBs, and SQL PaaS & Servers on Machines.ΓÇï

+ Microsoft Defender CSPM protects across all your multicloud workloads, but billing only applies for Servers, Databases and Storage accounts at $5/billable resource/month. The underlying compute services for AKS are regarded as servers for billing purposes.

+Microsoft Defender for Resource Manager provides visibility into activity that comes from third party service providers that have delegated access as part of the resource manager alerts. For example, `Azure Resource Manager operation from suspicious proxy IP address - delegated access`.

+`Delegated access` refers to access with [Azure Lighthouse](/azure/lighthouse/overview) or with [Delegated administration privileges](/partner-center/dap-faq).

+Alerts that show `Delegated access` also include a customized description and remediation steps.

+1. Change **Boot Mode** to **UEFI BIOS Mode**, and then select **F10: Save**.

+1. Change **Boot Mode** to **UEFI BIOS Mode**, and then select **F10: Save**.

+- [Create a custom image from a VHD file for DevTest Labs](devtest-lab-create-template.md)

+ Title: Create custom image for Azure DevTest Labs VMs from VHD files

+description: Use a VHD file to create an Azure DevTest Labs virtual machine custom image in the Azure portal.

+# Create a custom image for Azure DevTest Labs virtual machines from VHD files

+In this article, you learn how to create a virtual machine (VM) custom image for Azure DevTest Labs by using a virtual hard drive (VHD) file.

+## Create custom images for Azure DevTest Labs in Azure portal

+## In this article:

+- Two VNets are created: myvne4t and myvnet2.

+- An Azure DNS Private Resolver is created in the first VNet with an inbound endpoint at 10.10.0.4.

+- A DNS forwarding ruleset is created to be used with the private resolver.

+- The DNS forwarding ruleset is linked to the second VNet.

+- Example rules are added to the DNS forwarding ruleset.

+This article does not demonstrate DNS forwarding to an on-premises network. For more information, see [Resolve Azure and on-premises domains](private-resolver-hybrid-dns.md).

+The following figure summarizes the setup used in this article:

+

+> [!IMPORTANT]

+> When you create a private DNS zone, Azure stores the zone data as a global resource. This means that the private zone is not dependent on a single VNet or region. You can link the same private zone to multiple VNets in different regions. If service is interrupted in one VNet, your private zone is still available. For more information, see [Azure Private DNS zone resiliency](private-dns-resiliency.md).

+In this article, two VMs are used in a single VNet linked to your private DNS zone with autoregistration enabled. The setup is summarized in the following figure.

+

+[previous-tutorial]: storage-upload-process-images.md

+> Log compaction feature isn't supported in the **Basic** tier.

+

+## Downloading chart data

+The data underlying any dashboard chart can be exported to a CSV file. This is useful for those who wish to import Defender EASM data into third party tools, or work off a CSV file when remediating any issues. To download chart data, first select the specific chart segment that contains the data you wish to download. Note that chart exports currently support individual chart segments; to download multiple segments from the same chart, you will need to export each individual segment.

+Selecting an individual chart segment will open a drillldown view of the data, listing any assets that comprise the segment count. At the top of this page, select **Download CSV report** to begin your export. If you are exporting a small number of assets, this action will directly download the CSV file to your machine. If you are exporting a large number of assets, this action will create a task manager notification where you can track the status of your export.

+

+### Websites by status

+### Personal identifiable information (PII) posture

+ Title: List of built-in packages for guest configuration

+Cluster replication uses a source-push methodology. An HBase cluster can be a source or a destination, or it can fulfill both roles at once. Replication is asynchronous. The goal of replication is eventual consistency. When the source receives an edit to a column family when replication is enabled, the edit is propagated to all destination clusters. When data replicated from one cluster to another, the source cluster and all clusters that have already consumed the data tracked, to prevent replication loops.

+ > There are a variety of ways to obtain the `ssh` utility. On Linux, Unix, and macOS, it's provided as part of the operating system. If you are using Windows, consider one of the following options:

+ The `icb0d0thtw0ebifqt0g1jycdxd.ex.internal.cloudapp.net` text is the __DNS suffix__ for this virtual network. Save this value, as it's used later.

+ Until now, you can't look up the IP address from the other network without specified DNS server IP address.

+When you replicate a cluster, you must specify the tables that you want to replicate. In this section, you load some data into the source cluster. In the next section, you'll enable replication between the two clusters.

+ 3. **Head**: Ensure this parameter is selected. Clear the other node types.

+ > This walkthrough assumes hn1 as active headnode. Check your cluster to identify the active head node.

+- **Enable replication on all tables between the two clusters**. This scenario doesn't require copying or migrating existing data in the tables, and it doesn't use Phoenix tables. Use the following parameters:

+- **Enable replication on all tables, and replicate Phoenix metadata from source to destination**. Phoenix metadata replication isn't perfect. Use it with caution. Use the following parameters:

+### Set up replication between ESP clusters

+**Prerequisites**

+1. Both ESP clusters should be there in the same realm (domain). Check `/etc/krb5.conf` file default realm property to confirm.

+1. Common user should be there who has read and write access to both the clusters

+ 1. For example, if both clusters have same cluster admin user (For example, `admin@abc.example.com`), that user can be used to run the replication script.

+ 1. If both the clusters using same user group, you can add a new user or use existing user from the group.

+ 1. If both the clusters using different user group, you can add a new user to both use existing user from the groups.

+

+**Steps to Execute Replication script**

+> [!NOTE]

+> Perform the following steps only if DNS is unable to resolve hostname correctly of destination cluster.

+> 1. Copy sink cluster hosts IP & hostname mapping in source cluster nodes /etc/hosts file.

+> 1. Copy head node, worker node and ZooKeeper nodes host and IP mapping from /etc/hosts file of destination(sink) cluster.

+> 1. Add copied entries source cluster /etc/hosts file. These entries should be added to head nodes, worker nodes and ZooKeeper nodes.

+**Step: 1**

+Create keytab file for the user using `ktutil`.

+`$ ktutil`

+1. `addent -password -p admin@ABC.EXAMPLE.COM -k 1 -e RC4-HMAC`

+1. Ask for password to authenticate, provide user password

+1. `wkt /etc/security/keytabs/admin.keytab`

+> [!NOTE]

+> Make sure the keytab file is stored in `/etc/security.keytabs/` folder in the `<username>.keytab` format.

+**Step 2**

+Run script action with `-ku` option

+1. Provide `-ku <username>` on ESP clusters.

+

+|Name|Description|

+|-|--|

+|`-ku, --krb-user` | For ESP clusters, Common Kerberos user, who can authenticate both source and destination clusters|

+> [!NOTE]

+> Implementations in Microsoft Azure Government may still require the inbound service tags in the network security group and user defined routes.

+It isn't mandatory to use private endpoints for these resources. But if you plan to use private endpoints for these resources, you must create the resources and configure the private endpoints and DNS entries before you create the HDInsight cluster. All these resources should be accessible from inside the cluster subnet, either through a private endpoint or otherwise. If you're planning to use a private endpoint, it's recommended to leverage the cluster subnet.

+When you run an HDInsight cluster, the Apache Spark job that writes to Azure storage container becomes slow when there are many files/sub-folders. For example, it takes 20 seconds when writing to a new container, but about 2 minutes when writing to a container that has 200k files.

+This issue is a known Spark related issue. The slowness comes from the `ListBlob` and `GetBlobProperties` operations during Spark job execution.

+To track partitions, Spark has to maintain a `FileStatusCache` that, contains info about directory structure. If you use this cache, Spark can parse the paths and be aware of available partitions. The benefit of tracking partitions is that Spark only touches the necessary files when you read data. To keep this information up-to-date, when you write new data, Spark has to list all files under the directory and update this cache.

+In Spark 2.1, while we don't need to update the cache after every write, Spark will check whether an existing partition column matches with the proposed one in the current write request. So it will also lead to listing operations at the beginning of every write.

+When you create a partitioned data set, it's important to use a partitioning scheme that limits the number of files that Spark has to list to update the `FileStatusCache`.

+For every Nth micro batch where N % 100 == 0 (100 is just an example), move existing data to another directory, which is loaded by Spark.

+You can use Azure IoT Explorer to view and manage the properties of your devices. In the following sections, you use the Plug and Play capabilities that are visible in IoT Explorer to manage and interact with the Microchip E54. These capabilities rely on the device model published for the Microchip E54 in the public model repository. You configured IoT Explorer to search this repository for device models earlier in this quickstart.

+As a next step, explore the following articles to learn more about using the IoT device SDKs to connect general devices, and embedded devices, to Azure IoT.

+> [!div class="nextstepaction"]

+> [Connect a general simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md)

+> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](concepts-using-c-sdk-and-embedded-c-sdk.md)

+To receive messages from IoT Hub, a device should subscribe using `devices/{device-id}/messages/devicebound/#` as a **Topic Filter**. The multi-level wildcard `#` in the Topic Filter is used only to allow the device to receive more properties in the topic name. IoT Hub doesn't allow the usage of the `#` or `?` wildcards for filtering of subtopics. Since IoT Hub isn't a general-purpose pub-sub messaging broker, it only supports the documented topic names and topic filters. A device can only subscribe to five topics at a time.

+For guidance on structuring your labs and images, see [Moving from a physical lab to Azure Lab Services](./concept-migrating-physical-labs.md).

+Learn more about the [supported networking scenarios in Azure Lab Services](./concept-lab-services-supported-networking-scenarios.md), such as content filtering.

+> [Advanced networking](how-to-connect-vnet-injection.md) must be enabled during the creation of your lab plan. It can't be added later.

+When you get an Azure subscription, you can create a new lab plan in Azure Lab Services. For more information about creating a new lab plan, see the tutorial on [how to set up a lab plan](./quick-create-resources.md). If you're using a ArcGIS License Manager on a license server, enable [advanced networking](how-to-connect-vnet-injection.md) when creating your lab plan. You can also use an existing lab plan.

+After setting up your license server, you need to enable [advanced networking](how-to-connect-vnet-injection.md) when you create the lab plan.

+> You must enable [advanced networking](how-to-connect-vnet-injection.md) when creating your lab plan. You can't enable advanced networking for an existing lab plan.

+> [Advanced networking](how-to-connect-vnet-injection.md) must be enabled during the creation of your lab plan. It can't be added later.

+Once you have an Azure subscription, you can create a new lab plan in Azure Lab Services. For more information about creating a new lab plan, see the tutorial on [how to set up a lab plan](./quick-create-resources.md). If you're using a [Network License Manager](https://www.mathworks.com/help/install/administer-network-licenses.html) on a license server, enable [advanced networking](how-to-connect-vnet-injection.md) when creating your lab plan. You can also use an existing lab plan.

+> [Advanced networking](how-to-connect-vnet-injection.md) must be enabled during the creation of your lab plan. It can't be added later.

+After you set up a lab plan, create a separate lab for each PLTW class session that your school offers. We also recommend that you create separate images for each type of PLTW class. For more information about how to structure your labs and images, see [Moving from a Physical Lab to Azure Lab Services](./concept-migrating-physical-labs.md).

+> [Advanced networking](how-to-connect-vnet-injection.md) must be enabled during the creation of your lab plan. It can't be added later.

+> [Advanced networking](how-to-connect-vnet-injection.md) must be enabled during the creation of your lab plan. It can't be added later.

+ > You need to enable [advanced networking](how-to-connect-vnet-injection.md) during the creation of your lab plan. You can't configure the lab plan's virtual network at a later stage.

+ > [Advanced networking](how-to-connect-vnet-injection.md) must be enabled during the creation of your lab plan. It can't be added later.

+ Title: Supported networking scenarios

+description: Learn about the supported networking scenarios and architectures for lab plans in Azure Lab Services.

+# Supported networking scenarios for lab plans in Azure Lab Services

+With Azure Lab Services advanced networking for lab plans you can implement various network architectures and topologies. This article lists different networking scenarios and their support in Azure Lab Services.

+## Networking scenarios

+The following table lists common networking scenarios and topologies and their support in Azure Lab Services.

+| Scenario | Enabled | Details |

+| -- | - | - |

+| Lab-to-lab communication | Yes | Learn more about [setting up lab-to-lab communication](./tutorial-create-lab-with-advanced-networking.md). If lab users need multiple virtual machines, you can [configure nested virtualization](./concept-nested-virtualization-template-vm.md). |

+| Open additional ports to the lab VM | No | You can't open additional ports on the lab VM, even with advanced networking.<br/><br/>A workaround solution is to use PowerShell or the Azure SDK to manually add the NAT rules for every VM in the lab (every private IP address). This solution isn't recommended because of the limit on the number of allowed rules for load balancers, and because it's unclear to lab users what the port number for their lab VM is. |

+| Enable distant license server, such as on-premises, cross-region | Yes | Add a [user defined route (UDR)](/azure/virtual-network/virtual-networks-udr-overview) that points to the license server.<br/><br/>If the lab software requires connecting to the license server by its name instead of the IP address, you need to [configure a customer-provided DNS server](/azure/virtual-network/virtual-networks-name-resolution-for-vms-and-role-instances?tabs=redhat#name-resolution-that-uses-your-own-dns-server) or add an entry to the `hosts` file in the lab template.<br/><br/>If multiple services need access to the license server, using them from multiple regions, or if the license server is part of other infrastructure, you can use the [hub-and-spoke Azure networking best practice](/azure/cloud-adoption-framework/ready/azure-best-practices/hub-spoke-network-topology). |

+| Access to on-premises resources, such as a license server | Yes | You can access on-premises resources with these options: <br/>- Configure [Azure ExpressRoute](/azure/expressroute/expressroute-introduction) or create a [site-to-site VPN connection](/azure/vpn-gateway/tutorial-site-to-site-portal) (bridge the networks).<br/>- Add a public IP to your on-premises server with a firewall that only allows incoming connections from Azure Lab Services.<br/><br/>In addition, to reach the on-premises resources from the lab VMs, add a [user defined route (UDR)](/azure/virtual-network/virtual-networks-udr-overview). |

+| Use a [hub-and-spoke networking model](/azure/cloud-adoption-framework/ready/azure-best-practices/hub-spoke-network-topology) | Yes | This scenario works as expected with lab plans and advanced networking. <br/><br/>A number of configuration changes aren't supported with Azure Lab Services, such as adding a default route on a route table. Learn about the [unsupported virtual network configuration changes](./how-to-connect-vnet-injection.md#4-optional-update-the-networking-configuration-settings). |

+| Access lab VMs by private IP address (private-only labs) | Not recommended | This scenario is functional, but makes it difficult for lab users to connect to their lab VM. In the Azure Lab Services website, lab users can't identify the private IP address of their lab VM. In addition, the connect button points to the public endpoint of the lab VM. The lab creator needs to provide lab users with the private IP address of their lab VMs. After a VM reset, this private IP address might change.<br/><br/>If you implement this scenario, don't delete the public IP address or load balancer associated with the lab. If those resources are deleted, the lab fails to scale or publish. |

+| Protect on-premises resources with a firewall | Yes | Putting a firewall between the lab VMs and a specific resource is supported. |

+| Put lab VMs behind a firewall. For example for content filtering, security, and more. | No | The typical firewall setup doesn't work with Azure Lab Services, unless when connecting to lab VMs by private IP address (see previous scenario).<br/><br/>When you set up the firewall, a default route is added on the route table for the subnet. This default route introduces an asymmetric routing problem, which breaks the RDP/SSH connections to the lab. |

+| Use third party over-the-shoulder monitoring software | Yes | This scenario is supported with advanced networking for lab plans. |

+| Use a custom domain name for labs, for example `lab1.labs.myuniversity.edu.au` | No | This scenario doesnΓÇÖt work because the FQDN is defined upon creation of the lab, based on the public IP address of the lab. Changes to the public IP address aren't propagated to the connect button for the template VM or the lab VMs. |

+| Enable forced-tunneling for labs, where all communication to lab VMs doesn't go over the public internet. This is also known as *fully isolated labs*. | No | This scenario doesnΓÇÖt work out of the box. As soon as you associate a route table with the subnet that contains a default route, lab users lose connectivity to the lab.<br/>To enable this scenario, follow the steps for accessing lab VMs by private IP address. |

+| Enable content filtering | Depends | Supported content filtering scenarios:<br/>- Third-party content filtering software on the lab VM:<br/>&nbsp;&nbsp;&nbsp;&nbsp;1. Lab users should run as nonadmin to avoid uninstalling or disabling the software<br/>&nbsp;&nbsp;&nbsp;&nbsp;2. Ensure that outbound calls to Azure aren't blocked.<br/><br/>- DNS-based content filtering: filtering works with advanced networking and specifying the DNS server on the labΓÇÖs subnet. You can use a DNS server that supports content filtering to do DNS-based filtering.<br/><br/>- Proxy-based content filtering: filtering works with advanced networking if the lab VMs can use a customer-provided proxy server that supports content filtering. It works similarly to the DNS-based solution.<br/><br/>Unsupported content filtering:<br/>- Network appliance (firewall): for more information, see the scenario for putting lab VMs behind a firewall.<br/><br/>When planning a content filtering solution, implement a proof of concept to ensure that everything works as expected end to end. |

+| Use a connection broker, such as Parsec, for high-framerate gaming scenarios | Not recommended | This scenario isnΓÇÖt directly supported with Azure Lab Services and would run into the same challenges as accessing lab VMs by private IP address. |

+| *Cyber field* scenario, consisting of a set of vulnerable VMs on the network for lab users to discover and hack into (ethical hacking) | Yes | This scenario works with advanced networking for lab plans. Learn about the [ethical hacking class type](./class-type-ethical-hacking.md). |

+| Enable using Azure Bastion for lab VMs | No | Azure Bastion isn't supported in Azure Lab Services. |

+## Next steps

+- [Connect a lab plan to virtual network with advanced networking](./how-to-connect-vnet-injection.md)

+- [Tutorial: Set up lab-to-lab communication](./tutorial-create-lab-with-advanced-networking.md)

+- Provide feedback or request new features on the [Azure Lab Services community site](https://feedback.azure.com/d365community/forum/502dba10-7726-ec11-b6e6-000d3a4f032c)

+ Title: Connect a lab plan to a virtual network

+description: Learn how to connect a lab plan to a virtual network with Azure Lab Services advanced networking. Advanced networking uses VNET injection to add lab virtual machines in your virtual network.

+# Connect a lab plan to a virtual network with advanced networking

+This article describes how to connect a lab plan to a virtual network with Azure Lab Services advanced networking. With advanced networking, you have more control over the virtual network configuration of your labs. For example, to connect to on-premises resources such as licensing servers, or to use user-defined routes (UDRs). Learn more about the [supported networking scenarios and topologies for advanced networking](./concept-lab-services-supported-networking-scenarios.md).

+Advanced networking for lab plans replaces [Azure Lab Services virtual network peering](how-to-connect-peer-virtual-network.md) that is used with lab accounts.

+Follow these steps to configure advanced networking for your lab plan:

+1. Delegate the virtual network subnet to Azure Lab Services lab plans. Delegation allows Azure Lab Services to create the lab template and lab virtual machines in the virtual network.

+1. Configure the network security group to allow inbound RDP or SSH traffic to the lab template virtual machine and lab virtual machines.

+1. Create a lab plan with advanced networking to associate it with the virtual network subnet.

+1. (Optional) Configure your virtual network.

+Advanced networking can only be enabled when creating a lab plan. Advanced networking is not a setting that can be updated later.

+The following diagram shows an overview of the Azure Lab Services advanced networking configuration. The lab template and lab virtual machines are assigned an IP address in your subnet, and the network security group allows lab users to connect to the lab VMs by using RDP or SSH.

+> [!NOTE]

+> If your organization needs to perform content filtering, such as for compliance with the [Children's Internet Protection Act (CIPA)](https://www.fcc.gov/consumers/guides/childrens-internet-protection-act), you will need to use 3rd party software. For more information, read guidance on [content filtering in the supported networking scenarios](./concept-lab-services-supported-networking-scenarios.md).

+## Prerequisites

+- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F).

+- Your Azure account has the [Network Contributor](/azure/role-based-access-control/built-in-roles#network-contributor) role, or a parent of this role, on the virtual network.

+- An Azure virtual network and subnet in the same Azure region as where you create the lab plan. Learn how to create a [virtual network](/azure/virtual-network/manage-virtual-network) and [subnet](/azure/virtual-network/virtual-network-manage-subnet).

+- The subnet has enough free IP addresses for the template VMs and lab VMs for all labs (each lab uses 512 IP addresses) in the lab plan.

+## 1. Delegate the virtual network subnet

+To use your virtual network subnet for advanced networking in Azure Lab Services, you need to [delegate the subnet](/azure/virtual-network/subnet-delegation-overview) to Azure Lab Services lab plans. Subnet delegation gives explicit permissions to Azure Lab Services to create service-specific resources, such as lab virtual machines, in the subnet.

+1. Select a dedicated subnet you wish to delegate to Azure Lab Services.

+ > The subnet you use for Azure Lab Services should not already be used for a VNET gateway or Azure Bastion.

+1. In **Delegate subnet to a service**, select *Microsoft.LabServices/labplans*, and then select **Save**.

+1. Verify that *Microsoft.LabServices/labplans* appears in the **Delegated to** column for your subnet.

+## 2. Configure a network security group

+When you connect your lab plan to a virtual network, you need to configure a network security group (NSG) to allow inbound RDP/SSH traffic from the user's computer to the template virtual machine and the lab virtual machines. An NSG contains access control rules that allow or deny traffic based on traffic direction, protocol, source address and port, and destination address and port.

+The rules of an NSG can be changed at any time, and changes are applied to all associated instances. It might take up to 10 minutes for the NSG changes to be effective.

+> [!IMPORTANT]

+> If you don't configure a network security group, you won't be able to access the lab template VM and lab VMs via RDP or SSH.

+The network security group configuration for advanced networking consists of two steps:

+1. Create a network security group that allows RDP/SSH traffic

+1. Associate the network security group with the virtual network subnet

+You can use an NSG to control traffic to one or more virtual machines (VMs), role instances, network adapters (NICs), or subnets in your virtual network. An NSG contains access control rules that allow or deny traffic based on traffic direction, protocol, source address and port, and destination address and port. The rules of an NSG can be changed at any time, and changes are applied to all associated instances.

+For more information about NSGs, visit [what is an NSG](/azure/virtual-network/network-security-groups-overview).

+1. If you don't have a network security group yet, follow these steps to [create a network security group (NSG)](/azure/virtual-network/manage-network-security-group).

+Next, associate the NSG with the virtual network subnet to apply the traffic rules to the virtual network traffic.

+## 3. Create a lab plan with advanced networking

+Now that you've configured the subnet and network security group, you can create the lab plan with advanced networking. When you create a new lab on the lab plan, Azure Lab Services creates the lab template and lab virtual machines in the virtual network subnet.

+To create a lab plan with advanced networking in the Azure portal:

+1. Sign in to the [Azure portal](https://portal.azure.com).

+1. Select **Create a resource** in the upper left-hand corner of the Azure portal, and search for **lab plan**.

+1. In the **Networking** tab, select **Enable advanced networking** to configure the virtual network subnet.

+ If your virtual network doesn't appear in the list, verify that the lab plan is in the same Azure region as the virtual network, that you've [delegated the subnet to Azure Lab Services](#1-delegate-the-virtual-network-subnet), and that your [Azure account has the necessary permissions](#prerequisites).

+ Lab users and lab managers can now connect to their lab virtual machines or lab template virtual machine by using RDP or SSH.

+ When you create a new lab, all virtual machines are created in the virtual network and assigned an IP address within the subnet range.

+## 4. (Optional) Update the networking configuration settings

+It's recommended that you use the default configuration settings for the virtual network and subnet when you use advanced networking in Azure Lab Services.

+For specific networking scenarios, you might need to update the networking configuration. Learn more about the [supported networking architectures and topologies in Azure Lab Services](./concept-lab-services-supported-networking-scenarios.md) and the corresponding network configuration.

+You can modify the virtual network settings after you create the lab plan with advanced networking. However, when you change the [DNS settings on the virtual network](/azure/virtual-network/manage-virtual-network#change-dns-servers), you need to restart any running lab virtual machines. If the lab VMs are stopped, they'll automatically receive the updated DNS settings when they start.

+> [!CAUTION]

+> The following networking configuration changes are not supported after you've configured advanced networking :

+>

+> - Delete the virtual network or subnet associated with the lab plan. This causes the labs to stop working.

+> - Change the subnet address range when there are virtual machines created (template VM or lab VMs).

+> - Change the DNS label on the public IP address. This causes the **Connect** button for lab VMs to stop working.

+> - Change the [frontend IP configuration](/azure/load-balancer/manage#add-frontend-ip-configuration) on the Azure load balancer. This causes the **Connect** button for lab VMs to stop working.

+> - Change the FQDN on the public IP address.

+> - Use a route table with a default route for the subnet (forced-tunneling). This causes users to lose connectivity to their lab.

+> - The use of Azure Firewall or Azure Bastion is not supported.

+- [Attach a compute gallery to a lab plan](how-to-attach-detach-shared-image-gallery.md).

+- [Configure automatic shutdown settings for a lab plan](how-to-configure-auto-shutdown-lab-plans.md).

+- [Add lab creators to a lab plan](add-lab-creator.md).

+For guidance on structuring your labs and images, see [Moving from a physical lab to Azure Lab Services](./concept-migrating-physical-labs.md).

+1. **Configure shared resources**. Optionally, [configure licensing servers](how-to-create-a-lab-with-shared-resource.md). For VMs that require access to a licensing server, create a lab using a lab plan with [advanced networking](how-to-connect-vnet-injection.md). You can reuse the same Azure Compute Gallery and the licensing servers that you use with your lab accounts.

+ 1. [Create](quick-create-resources.md) and [configure lab plans](#configure-a-lab-plan). If you plan to use a license server, don't forget to enable [advanced networking](how-to-connect-vnet-injection.md) when creating your lab plans.

+You can reuse the same Azure Compute Gallery and licensing servers that you use with your lab accounts. Optionally, you can also [configure more licensing servers](./how-to-create-a-lab-with-shared-resource.md) and galleries based on your needs. For VMs that require access to a licensing server, you'll create lab plans with [advanced networking](./how-to-connect-vnet-injection.md) enabled as shown in the next step.

+ - If you plan to use a license server, don't forget to enable [advanced networking](./how-to-connect-vnet-injection.md) when creating your lab plans.

+|[Virtual network peering](./how-to-connect-peer-virtual-network.md#configure-at-the-time-of-lab-account-creation)|Lab plans can reuse the same virtual network as lab accounts. </br> - [Setup advanced networking](./how-to-connect-vnet-injection.md) when you create the lab plan.|

+Azure Lab Services provides a feature called advanced networking. Advanced networking enables you to control the network for labs created using lab plans. You can use advanced networking to implement various scenarios including [connecting to licensing servers](how-to-create-a-lab-with-shared-resource.md), using [hub-spoke model for Azure Networking](/azure/architecture/reference-architectures/hybrid-networking/), or lab to lab communication. Learn more about the [supported networking scenarios in Azure Lab Services](./concept-lab-services-supported-networking-scenarios.md).

+In this article, you'll learn how to connect to data sources located outside of Azure, to make that data available to Azure Machine Learning services. For this data availability, Azure supports connections to these external sources:

+> For a successful data import, please verify that you have installed the latest azure-ai-ml package (version 1.5.0 or later) for SDK, and the ml extension (version 2.15.1 or later).

+>

+> If you have an older SDK package or CLI extension, please remove the old one and install the new one with the code shown in the tab section. Follow the instructions for SDK and CLI as shown here:

+# [Azure CLI](#tab/cli)

+```cli

+az extension remove -n ml

+az extension add -n ml --yes

+az extension show -n ml #(the version value needs to be 2.15.1 or later)

+```

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+Not available.

+# [Azure CLI](#tab/cli)

+name: my-sf-db-connection # add your datastore name here

+az ml connection create --file my_snowflakedb_connection.yaml --set credentials.username="XXXXX" credentials.password="XXXXX"

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+1. Navigate to the [Azure Machine Learning studio](https://ml.azure.com).

+1. Under **Assets** in the left navigation, select **Data**. Next, select the **Data Connection** tab. Then select Create as shown in this screenshot:

+ :::image type="content" source="media/how-to-connection/create-new-data-connection.png" lightbox="media/how-to-connection/create-new-data-connection.png" alt-text="Screenshot showing the start of a new data connection in Azure Machine Learning studio UI.":::

+1. In the **Create connection** pane, fill in the values as shown in the screenshot. Choose Snowflake for the category, and **Username password** for the Authentication type. Be sure to specify the **Target** textbox value in this format, filling in your specific values between the < > characters:

+ **jdbc:snowflake://\<myaccount\>.snowflakecomputing.com/?db=\<mydb\>&warehouse=\<mywarehouse\>&role=\<myrole\>**

+ :::image type="content" source="media/how-to-connection/create-snowflake-connection.png" lightbox="media/how-to-connection/create-snowflake-connection.png" alt-text="Screenshot showing creation of a new Snowflake connection in Azure Machine Learning studio UI.":::

+1. Select Save to securely store the credentials in the key vault associated with the relevant workspace. This connection is used when running a data import job.

+# [Azure CLI](#tab/cli)

+name: my-sqldb-connection

+az ml connection create --file my_sqldb_connection.yaml --set credentials.username="XXXXX" credentials.password="XXXXX"

+# [Python SDK](#tab/python)

+# add the sql servername, port address and database

+# [Studio](#tab/azure-studio)

+1. Navigate to the [Azure Machine Learning studio](https://ml.azure.com).

+1. Under **Assets** in the left navigation, select **Data**. Next, select the **Data Connection** tab. Then select Create as shown in this screenshot:

+ :::image type="content" source="media/how-to-connection/create-new-data-connection.png" lightbox="media/how-to-connection/create-new-data-connection.png" alt-text="Screenshot showing the start of a new data connection in Azure Machine Learning studio UI.":::

+1. In the **Create connection** pane, fill in the values as shown in the screenshot. Choose AzureSqlDb for the category, and **Username password** for the Authentication type. Be sure to specify the **Target** textbox value in this format, filling in your specific values between the < > characters:

+ **Server=tcp:\<myservername\>,\<port\>;Database=\<mydatabase\>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30**

+ :::image type="content" source="media/how-to-connection/how-to-create-azuredb-connection.png" lightbox="media/how-to-connection/how-to-create-azuredb-connection.png" alt-text="Screenshot showing creation of a new Azure DB connection in Azure Machine Learning studio UI.":::

+# [Azure CLI](#tab/cli)

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+1. Navigate to the [Azure Machine Learning studio](https://ml.azure.com).

+1. Under **Assets** in the left navigation, select **Data**. Next, select the **Data Connection** tab. Then select Create as shown in this screenshot:

+ :::image type="content" source="media/how-to-connection/create-new-data-connection.png" lightbox="media/how-to-connection/create-new-data-connection.png" alt-text="Screenshot showing the start of a new data connection in Azure Machine Learning studio UI.":::

+1. In the **Create connection** pane, fill in the values as shown in the screenshot. Choose S3 for the category, and **Username password** for the Authentication type. Be sure to specify the **Target** textbox value in this format, filling in your specific values between the < > characters:

+ **\<target\>**

+ :::image type="content" source="media/how-to-connection/how-to-create-amazon-s3-connection.png" lightbox="media/how-to-connection/how-to-create-amazon-s3-connection.png" alt-text="Screenshot showing creation of a new Amazon S3 connection in Azure Machine Learning studio UI.":::

+- [Import data assets on a schedule](reference-yaml-schedule-data-import.md)

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+### Creating data assets from job outputs

+You can create a data asset from an Azure Machine Learning job by setting the `name` parameter in the output. In this example, you submit a job that copies data from a public blob store to your default Azure Machine Learning Datastore and creates a data asset called `job_output_titanic_asset`.

+# [Azure CLI](#tab/cli)

+Create a job specification YAML file (`<file-name>.yml`):

+```yaml

+$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json

+# path: Set the URI path for the data. Supported paths include

+# local: `./<path>

+# Blob: wasbs://<container_name>@<account_name>.blob.core.windows.net/<path>

+# ADLS: abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>

+# Datastore: azureml://datastores/<data_store_name>/paths/<path>

+# Data Asset: azureml:<my_data>:<version>

+# type: What type of data are you pointing to?

+# uri_file (a specific file)

+# uri_folder (a folder)

+# mltable (a table)

+# mode: Set INPUT mode:

+# ro_mount (read-only mount)

+# download (download from storage to node)

+# mode: Set the OUTPUT mode

+# rw_mount (read-write mount)

+# upload (upload data from node to storage)

+type: command

+command: cp ${{inputs.input_data}} ${{outputs.output_data}}

+compute: azureml:cpu-cluster

+environment: azureml://registries/azureml/environments/sklearn-1.1/versions/4

+inputs:

+ input_data:

+ mode: ro_mount

+ path: azureml:wasbs://data@azuremlexampledata.blob.core.windows.net/titanic.csv

+ type: uri_file

+outputs:

+ output_data:

+ mode: rw_mount

+ path: azureml://datastores/workspaceblobstore/paths/quickstart-output/titanic.csv

+ type: uri_file

+ name: job_output_titanic_asset

+

+```

+Next, submit the job using the CLI:

+```azurecli

+az ml job create --file <file-name>.yml

+```

+# [Python SDK](#tab/python)

+```python

+from azure.ai.ml import command, Input, Output, MLClient

+from azure.ai.ml.constants import AssetTypes, InputOutputModes

+from azure.identity import DefaultAzureCredential

+# Set your subscription, resource group and workspace name:

+subscription_id = "<SUBSCRIPTION_ID>"

+resource_group = "<RESOURCE_GROUP>"

+workspace = "<AML_WORKSPACE_NAME>"

+# connect to the AzureML workspace

+ml_client = MLClient(

+ DefaultAzureCredential(), subscription_id, resource_group, workspace

+)

+# ==============================================================

+# Set the input and output URI paths for the data. Supported paths include:

+# local: `./<path>

+# Blob: wasbs://<container_name>@<account_name>.blob.core.windows.net/<path>

+# ADLS: abfss://<file_system>@<account_name>.dfs.core.windows.net/<path>

+# Datastore: azureml://datastores/<data_store_name>/paths/<path>

+# Data Asset: azureml:<my_data>:<version>

+# As an example, we set the input path to a file on a public blob container

+# As an example, we set the output path to a folder in the default datastore

+# ==============================================================

+input_path = "wasbs://data@azuremlexampledata.blob.core.windows.net/titanic.csv"

+output_path = "azureml://datastores/workspaceblobstore/paths/quickstart-output/titanic.csv"

+# ==============================================================

+# What type of data are you pointing to?

+# AssetTypes.URI_FILE (a specific file)

+# AssetTypes.URI_FOLDER (a folder)

+# AssetTypes.MLTABLE (a table)

+# The path we set above is a specific file

+# ==============================================================

+data_type = AssetTypes.URI_FILE

+# ==============================================================

+# Set the input mode. The most commonly-used modes:

+# InputOutputModes.RO_MOUNT

+# InputOutputModes.DOWNLOAD

+# Set the mode to Read Only (RO) to mount the data

+# ==============================================================

+input_mode = InputOutputModes.RO_MOUNT

+# ==============================================================

+# Set the output mode. The most commonly-used modes:

+# InputOutputModes.RW_MOUNT

+# InputOutputModes.UPLOAD

+# Set the mode to Read Write (RW) to mount the data

+# ==============================================================

+output_mode = InputOutputModes.RW_MOUNT

+# ==============================================================

+# Set a data asset name for the output

+# ==============================================================

+data_asset_name = "job_output_titanic_asset"

+# Set the input and output for the job:

+inputs = {

+ "input_data": Input(type=data_type, path=input_path, mode=input_mode)

+}

+outputs = {

+ "output_data": Output(type=data_type, path=output_path, mode=output_mode, name = data_asset_name)

+}

+# This command job copies the data to your default Datastore

+job = command(

+ command="cp ${{inputs.input_data}} ${{outputs.output_data}}",

+ inputs=inputs,

+ outputs=outputs,

+ environment="azureml://registries/azureml/environments/sklearn-1.1/versions/4",

+ compute="cpu-cluster",

+)

+# Submit the command

+ml_client.jobs.create_or_update(job)

+```

+# [Studio](#tab/azure-studio)

+Not available.

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+In this article, you'll learn how to import data into the Azure Machine Learning platform from external sources. A successful import automatically creates and registers an Azure Machine Learning data asset with the name provided during the import. An Azure Machine Learning data asset resembles a web browser bookmark (favorites). You don't need to remember long storage paths (URIs) that point to your most-frequently used data. Instead, you can create a data asset, and then access that asset with a friendly name.

+A data import creates a cache of the source data, along with metadata, for faster and reliable data access in Azure Machine Learning training jobs. The data cache avoids network and connection constraints. The cached data is versioned to support reproducibility. This provides versioning capabilities for data imported from SQL Server sources. Additionally, the cached data provides data lineage for auditing tasks. A data import uses ADF (Azure Data Factory pipelines) behind the scenes, which means that users can avoid complex interactions with ADF. Behind the scenes, Azure Machine Learning also handles management of ADF compute resource pool size, compute resource provisioning, and tear-down, to optimize data transfer by determining proper parallelization.

+You can import data from Amazon S3, Azure SQL, and Snowflake.

+> For a successful data import, please verify that you installed the latest azure-ai-ml package (version 1.5.0 or later) for SDK, and the ml extension (version 2.15.1 or later).

+>

+> If you have an older SDK package or CLI extension, please remove the old one and install the new one with the code shown in the tab section. Follow the instructions for SDK and CLI as shown here:

+# [Azure CLI](#tab/cli)

+```cli

+az extension remove -n ml

+az extension add -n ml --yes

+az extension show -n ml #(the version value needs to be 2.15.1 or later)

+```

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+Not available.

+## Import from an external database as a table data asset

+# [Python SDK](#tab/python)

+ name="<name>",

+ source=Database(connection="<connection>", query="<query>"),

+# [Studio](#tab/azure-studio)

+> [!NOTE]

+> The example seen here describes the process for a Snowflake database. However, this process covers other external database formats, like Azure SQL, etc.

+1. Navigate to the [Azure Machine Learning studio](https://ml.azure.com).

+1. Under **Assets** in the left navigation, select **Data**. Next, select the **Data Import** tab. Then select Create as shown in this screenshot:

+ :::image type="content" source="media/how-to-import-data-assets/create-new-data-import.png" lightbox="media/how-to-import-data-assets/create-new-data-import.png" alt-text="Screenshot showing creation of a new data import in Azure Machine Learning studio UI.":::

+1. At the Data Source screen, select Snowflake, and then select Next, as shown in this screenshot:

+ :::image type="content" source="media/how-to-import-data-assets/select-source-for-snowflake-data-asset.png" lightbox="media/how-to-import-data-assets/select-source-for-snowflake-data-asset.png" alt-text="Screenshot showing selection of a Snowflake data asset.":::

+1. At the Data Type screen, fill in the values. The **Type** value defaults to **Table (mltable)**. Then select Next, as shown in this screenshot:

+ :::image type="content" source="media/how-to-import-data-assets/select-snowflake-data-asset-type.png" lightbox="media/how-to-import-data-assets/select-snowflake-data-asset-type.png" alt-text="Screenshot that shows selection of a Snowflake data asset type.":::

+1. At the Create data import screen, fill in the values, and select Next, as shown in this screenshot:

+ :::image type="content" source="media/how-to-import-data-assets/create-snowflake-data-import.png" lightbox="media/how-to-import-data-assets/create-snowflake-data-import.png" alt-text="Screenshot that shows details of the data source selection.":::

+1. Fill in the values at the Choose a datastore to output screen, and select Next, as shown in this screenshot. **Workspace managed datastore** is selected by default; the path is automatically assigned by the system when you choose manged datastore. If you select **Workspace managed datastore**, the **Auto delete setting** dropdown appears. It offers a data deletion time window of 30 days by default, and [how to manage imported data assets](./how-to-manage-imported-data-assets.md) explains how to change this value.

+ :::image type="content" source="media/how-to-import-data-assets/choose-snowflake-datastore-to-output.png" lightbox="media/how-to-import-data-assets/choose-snowflake-datastore-to-output.png" alt-text="Screenshot that shows details of the data source to output.":::

+ > [!NOTE]

+ > To choose your own datastore, select **Other datastores**. In this case, you must select the path for the location of the data cache.

+1. You can add a schedule. Select **Add schedule** as shown in this screenshot:

+

+ :::image type="content" source="media/how-to-import-data-assets/create-data-import-add-schedule.png" lightbox="media/how-to-import-data-assets/create-data-import-add-schedule.png" alt-text="Screenshot that shows the selection of the Add schedule button.":::

+

+ A new panel opens, where you can define a **Recurrence** schedule, or a **Cron** schedule. This screenshot shows the panel for a **Recurrence** schedule:

+

+ :::image type="content" source="media/how-to-import-data-assets/create-data-import-recurrence-schedule.png" lightbox="media/how-to-import-data-assets/create-data-import-recurrence-schedule.png" alt-text="A screenshot that shows selection of the Add schedule button.":::

+

+ - **Name**: the unique identifier of the schedule within the workspace.

+ - **Description**: the schedule description.

+ - **Trigger**: the recurrence pattern of the schedule, which includes the following properties.

+ - **Time zone**: the trigger time calculation is based on this time zone; (UTC) Coordinated Universal Time by default.

+ - **Recurrence** or **Cron expression**: select recurrence to specify the recurring pattern. Under **Recurrence**, you can specify the recurrence frequency - by minutes, hours, days, weeks, or months.

+ - **Start**: the schedule first becomes active on this date. By default, the creation date of this schedule.

+ - **End**: the schedule will become inactive after this date. By default, it's NONE, which means that the schedule will always be active until you manually disable it.

+ - **Tags**: the selected schedule tags.

+

+ This screenshot shows the panel for a **Cron** schedule:

+

+ :::image type="content" source="media/how-to-import-data-assets/create-data-import-cron-expression-schedule.png" lightbox="media/how-to-import-data-assets/create-data-import-cron-expression-schedule.png" alt-text="Screenshot that shows selection of the Add schedule button.":::

+ - **Name**: the unique identifier of the schedule within the workspace.

+ - **Description**: the schedule description.

+ - **Trigger**: the recurrence pattern of the schedule, which includes the following properties.

+ - **Time zone**: the trigger time calculation is based on this time zone; (UTC) Coordinated Universal Time by default.

+ - **Recurrence** or **Cron expression**: select recurrence to specify the recurring pattern. **Cron expression** allows you to specify more flexible and customized recurrence pattern.

+ - **Start**: the schedule first becomes active on this date. By default, the creation date of this schedule.

+ - **End**: the schedule will become inactive after this date. By default, it's NONE, meaning that the schedule will remain active until you manually disable it.

+ - **Tags**: the selected schedule tags.

+- **(Required)** `expression` uses a standard crontab expression to express a recurring schedule. A single expression is composed of five space-delimited fields:

+ `MINUTES HOURS DAYS MONTHS DAYS-OF-WEEK`

+ - A single wildcard (`*`), which covers all values for the field. A `*`, in days, means all days of a month (which varies with month and year).

+ - The `expression: "15 16 * * 1"` in the sample above means the 16:15PM on every Monday.

+ - The next table lists the valid values for each field:

+

+ | Field | Range | Comment |

+ |-|-|--|

+ | `MINUTES` | 0-59 | - |

+ | `HOURS` | 0-23 | - |

+ | `DAYS` | - | Not supported. The value is ignored and treated as `*`. |

+ | `MONTHS` | - | Not supported. The value is ignored and treated as `*`. |

+ | `DAYS-OF-WEEK` | 0-6 | Zero (0) means Sunday. Names of days also accepted. |

+ - To learn more about crontab expressions, see [Crontab Expression wiki on GitHub](https://github.com/atifaziz/NCrontab/wiki/Crontab-Expression).

+ > [!IMPORTANT]

+ > `DAYS` and `MONTH` are not supported. If you pass one of these values, it will be ignored and treated as `*`.

+- (Optional) `start_time` specifies the start date and time with the timezone of the schedule. For example, `start_time: "2022-05-10T10:15:00-04:00"` means the schedule starts from 10:15:00AM on 2022-05-10 in the UTC-4 timezone. If `start_time` is omitted, the `start_time` equals the schedule creation time. For a start time in the past, the first job runs at the next calculated run time.

+The next screenshot shows the last screen of this process. Review your choices, and select Create. At this screen, and the other screens in this process, select Back to move to earlier screens to change your choices of values.

+ :::image type="content" source="media/how-to-import-data-assets/create-snowflake-data-import-review-values-and-create.png" lightbox="media/how-to-import-data-assets/create-snowflake-data-import-review-values-and-create.png" alt-text="Screenshot that shows all parameters of the data import.":::

+## Import data from an external file system as a folder data asset

+# [Python SDK](#tab/python)

+ name="<name>",

+ source=FileSystem(connection="<connection>", path="<path_on_source>"),

+# [Studio](#tab/azure-studio)

+1. Navigate to the [Azure Machine Learning studio](https://ml.azure.com).

+1. Under **Assets** in the left navigation, select **Data**. Next, select the Data Import tab. Then select Create as shown in this screenshot:

+ :::image type="content" source="media/how-to-import-data-assets/create-new-data-import.png" lightbox="media/how-to-import-data-assets/create-new-data-import.png" alt-text="Screenshot showing creation of a data import in Azure Machine Learning studio UI.":::

+1. At the Data Source screen, select S3, and then select Next, as shown in this screenshot:

+ :::image type="content" source="media/how-to-import-data-assets/select-source-for-s3-data-asset.png" lightbox="media/how-to-import-data-assets/select-source-for-s3-data-asset.png" alt-text="Screenshot showing selection of an S3 data asset.":::

+1. At the Data Type screen, fill in the values. The **Type** value defaults to **Folder (uri_folder)**. Then select Next, as shown in this screenshot:

+ :::image type="content" source="media/how-to-import-data-assets/select-s3-data-asset-type.png" lightbox="media/how-to-import-data-assets/select-s3-data-asset-type.png" alt-text="Screenshot showing selection of a Snowflake data asset type.":::

+1. At the Create data import screen, fill in the values, and select Next, as shown in this screenshot:

+ :::image type="content" source="media/how-to-import-data-assets/create-s3-data-import.png" lightbox="media/how-to-import-data-assets/create-s3-data-import.png" alt-text="Screenshot showing details of the data source selection.":::

+1. Fill in the values at the Choose a datastore to output screen, and select Next, as shown in this screenshot. **Workspace managed datastore** is selected by default; the path is automatically assigned by the system when you choose managed datastore. If you select **Workspace managed datastore**, the **Auto delete setting** dropdown appears. It offers a data deletion time window of 30 days by default, and [how to manage imported data assets](./how-to-manage-imported-data-assets.md) explains how to change this value.

+ :::image type="content" source="media/how-to-import-data-assets/choose-s3-datastore-to-output.png" lightbox="media/how-to-import-data-assets/choose-s3-datastore-to-output.png" alt-text="Screenshot showing details of the data source to output.":::

+1. You can add a schedule. Select **Add schedule** as shown in this screenshot:

+

+ :::image type="content" source="media/how-to-import-data-assets/create-data-import-add-schedule.png" lightbox="media/how-to-import-data-assets/create-data-import-add-schedule.png" alt-text="Screenshot showing selection of the Add schedule button.":::

+

+ A new panel opens, where you can define a **Recurrence** schedule, or a **Cron** schedule. This screenshot shows the panel for a **Recurrence** schedule:

+

+ :::image type="content" source="media/how-to-import-data-assets/create-data-import-recurrence-schedule.png" lightbox="media/how-to-import-data-assets/create-data-import-recurrence-schedule.png" alt-text="A screenshot showing selection of the Add schedule button.":::

+

+ - **Name**: the unique identifier of the schedule within the workspace.

+ - **Description**: the schedule description.

+ - **Trigger**: the recurrence pattern of the schedule, which includes the following properties.

+ - **Time zone**: the trigger time calculation is based on this time zone; (UTC) Coordinated Universal Time by default.

+ - **Recurrence** or **Cron expression**: select recurrence to specify the recurring pattern. Under **Recurrence**, you can specify the recurrence frequency - by minutes, hours, days, weeks, or months.

+ - **Start**: the schedule first becomes active on this date. By default, the creation date of this schedule.

+ - **End**: the schedule will become inactive after this date. By default, it's NONE, which means that the schedule will remain active until you manually disable it.

+ - **Tags**: the selected schedule tags.

+

+ This screenshot shows the panel for a **Cron** schedule:

+

+ :::image type="content" source="media/how-to-import-data-assets/create-data-import-cron-expression-schedule.png" lightbox="media/how-to-import-data-assets/create-data-import-cron-expression-schedule.png" alt-text="Screenshot showing the selection of the Add schedule button.":::

+ - **Name**: the unique identifier of the schedule within the workspace.

+ - **Description**: the schedule description.

+ - **Trigger**: the recurrence pattern of the schedule, which includes the following properties.

+ - **Time zone**: the trigger time calculation is based on this time zone; (UTC) Coordinated Universal Time by default.

+ - **Recurrence** or **Cron expression**: select recurrence to specify the recurring pattern. **Cron expression** allows you to specify more flexible and customized recurrence pattern.

+ - **Start**: the schedule first becomes active on this date. By default, the creation date of this schedule.

+ - **End**: the schedule will become inactive after this date. By default, it's NONE, meaning that the schedule will remain active until you manually disable it.

+ - **Tags**: the selected schedule tags.

+- **(Required)** `expression` uses a standard crontab expression to express a recurring schedule. A single expression is composed of five space-delimited fields:

+ `MINUTES HOURS DAYS MONTHS DAYS-OF-WEEK`

+ - A single wildcard (`*`), which covers all values for the field. A `*`, in days, means all days of a month (which varies with month and year).

+ - The `expression: "15 16 * * 1"` in the sample above means the 16:15PM on every Monday.

+ - The next table lists the valid values for each field:

+

+ | Field | Range | Comment |

+ |-|-|--|

+ | `MINUTES` | 0-59 | - |

+ | `HOURS` | 0-23 | - |

+ | `DAYS` | - | Not supported. The value is ignored and treated as `*`. |

+ | `MONTHS` | - | Not supported. The value is ignored and treated as `*`. |

+ | `DAYS-OF-WEEK` | 0-6 | Zero (0) means Sunday. Names of days also accepted. |

+ - To learn more about crontab expressions, see [Crontab Expression wiki on GitHub](https://github.com/atifaziz/NCrontab/wiki/Crontab-Expression).

+ > [!IMPORTANT]

+ > `DAYS` and `MONTH` are not supported. If you pass one of these values, it will be ignored and treated as `*`.

+- (Optional) `start_time` specifies the start date and time with the timezone of the schedule. For example, `start_time: "2022-05-10T10:15:00-04:00"` means the schedule starts from 10:15:00AM on 2022-05-10 in the UTC-4 timezone. If `start_time` is omitted, the `start_time` equals the schedule creation time. For a start time in the past, the first job runs at the next calculated run time.

+1. As shown in the next screenshot, review your choices at the last screen of this process, and select Create. At this screen, and the other screens in this process, select Back to move to earlier screens if you'd like to change your choices of values.

+ :::image type="content" source="media/how-to-import-data-assets/choose-s3-datastore-to-output.png" lightbox="media/how-to-import-data-assets/choose-s3-datastore-to-output.png" alt-text="Screenshot showing details of the data source to output.":::

+1. The next screenshot shows the last screen of this process. Review your choices, and select Create. At this screen, and the other screens in this process, select Back to move to earlier screens to change your choices of values.

+ :::image type="content" source="media/how-to-import-data-assets/create-s3-data-import-review-values-and-create.png" lightbox="media/how-to-import-data-assets/create-s3-data-import-review-values-and-create.png" alt-text="Screenshot showing all parameters of the data import.":::

+The data import action is an asynchronous action. It can take a long time. After submission of an import data action via the CLI or SDK, the Azure Machine Learning service might need several minutes to connect to the external data source. Then, the service would start the data import, and handle data caching and registration. The time needed for a data import also depends on the size of the source data set.

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+Not available.

+description: Learn how to manage imported data assets also known as edit autodeletion.

+In this article, you'll learn how to manage imported data assets from a life-cycle perspective. We learn how to modify or update auto delete settings on the data assets imported into a managed datastore (`workspacemanagedstore`) that Microsoft manages for the customer.

+> Auto delete settings capability, or lifecycle management, is currently offered only through the imported data assets in managed datastore, also known as `workspacemanagedstore`.

+You can change the auto delete setting value or condition as shown in these code samples:

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+These steps describe how to modify the auto delete settings of an imported data asset in `workspacemanageddatastore` in the Azure Machine Learning studio:

+1. Navigate to [Azure Machine Learning studio](https://ml.azure.com)

+1. As shown in the next screenshot, under **Assets** in the left navigation, select **Data**. At the **Data assets** tab, select an imported data asset located in the **workspacemanageddatastore**

+ :::image type="content" source="./media/how-to-manage-imported-data-assets/data-assets-list.png" lightbox="./media/how-to-manage-imported-data-assets/data-assets-list.png" alt-text="Screenshot highlighting the imported data asset name in workspace managed datastore in the Data assets tab.":::

+1. As shown in the next screenshot, the details page of the data asset has an **Auto delete setting** property. This property is currently active on the data asset. Verify that you have the correct **Version:** of the data asset selected in the drop-down, and select the pencil icon to edit the property.

+ :::image type="content" source="./media/how-to-manage-imported-data-assets/data-assets-details.png" lightbox="./media/how-to-manage-imported-data-assets/data-assets-details.png" alt-text="Screenshot showing the edit of the auto delete setting.":::

+1. To change the auto delete **Condition** setting, select **Created greater than**, and change **Value** to any numerical value. Then, select **Save** as shown in the next screenshot:

+ :::image type="content" source="./media/how-to-manage-imported-data-assets/edit-managed-data-asset-details.png" lightbox="./media/how-to-manage-imported-data-assets/edit-managed-data-asset-details.png" alt-text="Screenshot that shows the managed data asset auto delete settings choices.":::

+ > [!NOTE]

+ > At this time, the supported values range from 1 day to 3 years.

+1. After a successful edit, you'll return to the data asset detail page. This page shows the updated values in **Auto delete settings** property box, as shown in the next screenshot:

+ :::image type="content" source="./media/how-to-manage-imported-data-assets/new-managed-data-asset-details.png" lightbox="./media/how-to-manage-imported-data-assets/new-managed-data-asset-details.png" alt-text="Screenshot showing the managed data asset auto delete settings.":::

+ > [!NOTE]

+ > The auto delete setting is available only on imported data assets in a workspacemanaged datastore, as shown in the above screenshot.

+If you don't want a specific data asset version to become part of life-cycle management, you can remove a previously configured auto delete setting.

+# [Python SDK](#tab/python)

+# [Studio](#tab/azure-studio)

+These steps describe how to delete or clear the auto delete settings of an imported data asset in `workspacemanageddatastore` in the Azure Machine Learning studio:

+1. Navigate to [Azure Machine Learning studio](https://ml.azure.com)

+1. As shown in this screenshot, under **Assets** in the left navigation, select **Data**. On the **Data assets** tab, select an imported data asset located in the **workspacemanageddatastore**:

+ :::image type="content" source="./media/how-to-manage-imported-data-assets/data-assets-list.png" lightbox="./media/how-to-manage-imported-data-assets/data-assets-list.png" alt-text="Screenshot highlighting the imported data asset name in workspace managed datastore in the Data assets tab.":::

+1. As shown in the next screenshot, the details page of the data asset has an **Auto delete setting** property. This property is currently active on the data asset. Verify that you have the correct **Version:** of the data asset selected in the drop-down, and select the pencil icon to edit the property.

+ :::image type="content" source="./media/how-to-manage-imported-data-assets/data-assets-details.png" lightbox="./media/how-to-manage-imported-data-assets/data-assets-details.png" alt-text="Screenshot showing the edit of the auto delete setting.":::

+1. To delete or clear the auto delete setting, select the **Clear auto delete setting** trash can icon at the bottom of the page, as shown in this screenshot:

+ :::image type="content" source="./media/how-to-manage-imported-data-assets/clear-managed-data-asset-details.png" lightbox="./media/how-to-manage-imported-data-assets/clear-managed-data-asset-details.png" alt-text="Screenshot showing the managed data asset auto delete settings choices.":::

+1. After a successful deletion, you'll return to the data asset detail page. This page shows the **Auto delete settings** property box, which displays **None**, as shown in this screenshot:

+ :::image type="content" source="./media/how-to-manage-imported-data-assets/cleared-managed-data-asset-details.png" lightbox="./media/how-to-manage-imported-data-assets/cleared-managed-data-asset-details.png" alt-text="This screenshot shows the managed data asset auto delete settings.":::

+This Azure CLI code sample shows the data assets with certain conditions, or with values configured in the **auto delete** settings:

+Before you explore the detailed options available to you when accessing data, we show you the relevant code snippets to access data so you can get started quickly.

+In this example, you submit an Azure Machine Learning job that accesses data from a *public* blob storage account. However, you can adapt the snippet to access your own data in a private Azure Storage account, by updating the path (for details on how to specify paths, read [Paths](#paths)). Azure Machine Learning seamlessly handles authentication to cloud storage using Azure Active Directory passthrough. When you submit a job, you can choose:

+> If you use keys or SAS tokens to authenticate, we recommend that you [create an Azure Machine Learning datastore](how-to-datastore.md), because the runtime will automatically connect to storage without exposure of the key/token.

+In this example, you submit an Azure Machine Learning job that writes data to your default Azure Machine Learning Datastore. You can optionally set the `name` value of your data asset to create a data asset in the output.

+ "output_data": Output(type=data_type,

+ path=output_path,

+ mode=output_mode,

+ # optional: if you want to create a data asset from the output,

+ # then uncomment name (name can be set without setting version)

+ # name = "<name_of_data_asset>",

+ # version = "<version>",

+ )

+ # optional: if you want to create a data asset from the output,

+ # then uncomment name (name can be set without setting version)

+ # name: <name_of_data_asset>

+ # version: <version>

+

+When you submit a job, the Azure Machine Learning data runtime controls the data load, from the storage location to the compute target. The Azure Machine Learning data runtime has been optimized for speed and efficiency for machine learning tasks. The key benefits include:

+- Data loads are written in the [Rust language](https://www.rust-lang.org/), a language known for high speed and high memory efficiency. For concurrent data downloads, Rust avoids Python [Global Interpreter Lock (GIL)](https://wikipedia.org/wiki/Global_interpreter_lock) issues.

+- Light weight; Rust has *no* dependencies on other technologies - for example JVM. As a result, the runtime installs quickly, and it doesn't drain extra resources (CPU, Memory) on the compute target.

+- Seamless integration with [fsspec](https://filesystem-spec.readthedocs.io/en/latest/) - a unified pythonic interface to local, remote and embedded file systems and byte storage.

+> We suggest that you leverage the Azure Machine Learning data runtime, instead of creating your own mounting/downloading capability in your training (client) code. In particular, we have seen storage throughput constrained when the client code uses Python to download data from storage due to [Global Interpreter Lock (GIL)](https://wikipedia.org/wiki/Global_interpreter_lock) issues.

+When you provide a data input/output to a job, you must specify a `path` parameter that points to the data location. This table shows the different data locations that Azure Machine Learning supports, and also shows `path` parameter examples:

+- **`ro_mount`:** Mount storage location, as read-only on the local disk (SSD) compute target.

+- **`rw_mount`:** Mount storage location, as read-write on the local disk (SSD) compute target.

+- **`download`:** Download the data from the storage location to the local disk (SSD) compute target.

+- **`eval_mount`/`eval_download`:** *These modes are unique to MLTable.* In some scenarios, an MLTable can yield files that might be located in a different storage account than the storage account that hosts the MLTable file. Or, an MLTable can subset or shuffle the data located in the storage resource. That view of the subset/shuffle becomes visible only if the Azure Machine Learning data runtime actually evaluates the MLTable file. For example, this diagram shows how an MLTable used with `eval_mount` or `eval_download` can take images from two different storage containers, and an annotations file located in a different storage account, and then mount/download to the filesystem of the remote compute target.

+- **`direct`:** You might want to read data directly from a URI through other APIs, rather than go through the Azure Machine Learning data runtime. For example, you may want to access data on an s3 bucket (with a virtual-hostedΓÇôstyle or path-style `https` URL) using the boto s3 client. You can obtain the URI of the input as a *string* with the `direct` mode. You see use of the direct mode in Spark Jobs, because the `spark.read_*()` methods know how to process the URIs. For **non-Spark** jobs, it is *your* responsibility to manage access credentials. For example, you must explicitly make use of compute MSI, or otherwise broker access.

+In download mode, all the input data is copied to the local disk (SSD) of the compute target. The Azure Machine Learning data runtime starts the user training script, once all the data is copied. When the user script starts, it reads data from the local disk, just like any other files. When the job finishes, the data is removed from the disk of the compute target.

+| When training starts, all the data is available on the local disk (SSD) of the compute target, for the training script. No Azure storage / network interaction is required. | The dataset must completely fit on a compute target disk.|

+|After the user script starts, there are no dependencies on storage / network reliability. |The entire dataset is downloaded (if training needs to randomly select only a small portion of a data, then much of the download is wasted).|

+|Azure Machine Learning data runtime can parallelize the download (significant difference on many small files) and max network / storage throughput.|The job waits until all data downloads to the local disk of the compute target. If you submit a deep-learning job, the GPUs idle until data is ready.|

+- The data is small enough to fit on the compute target's disk without interference with other training.

+- The training must jump to random positions of a large file.

+In your job, you can change the above defaults by setting the environment variables - for example:

+- *The number of cores*. The more cores available, the more concurrency and therefore faster download speed.

+> For A100 GPU VMs, the Azure Machine Learning data runtime can saturate the NIC (Network Interface Card) when downloading data to the compute target (~24 Gbit/s): **The theoretical maximum throughput possible**.

+We can see that a larger file, broken up into smaller files, can improve download performance due to parallelism. We recommend that you avoid files that become too small (less than 4 MB) because the time needed for storage request submissions increases, relative to time spent downloading the payload. For more information, read [Many small files problem](#many-small-files-problem).

+|No delay at the start of training (unlike download mode).|Dependency on userΓÇÖs code behavior (if the training code that sequentially reads small files in a single thread mount also requests data from storage, it may not maximize the network or storage throughput).|

+- The data is large, and it wonΓÇÖt fit on the compute target local disk.

+In block-based open mode, each file is split into blocks of a predefined size (except for the last block). A read request from a specified position requests a corresponding block from storage, and returns the requested data immediately. A read also triggers background prefetching of *N* next blocks, using multiple threads (optimized for sequential read). Downloaded blocks are cached in two layer cache (RAM and local disk).

+When working with *millions* of files, avoid a *recursive listing* - for example `ls -R /mnt/dataset/folder/`. A recursive listing triggers many calls to list the directory contents of the parent directory. It then requires a separate recursive call for each directory inside, at all child levels. Typically, Azure Storage allows only 5000 elements to be returned per single list request. As a result, a recursive listing of 1M folders containing 10 files each requires `1,000,000 / 5000 + 1,000,000 = 1,000,200` requests to storage. In comparison, 1,000 folders with 10,000 files would only need 1001 requests to storage for a recursive listing.

+When an Azure Machine Learning job executes with data, the `mode` of an input determines how bytes are read from storage and cached on the compute target local SSD disk. For download mode, all the data caches on disk, before the user code starts its execution. Therefore, factors such as

+- **Data locality to compute**: Your storage and compute target locations should be the same. If your storage and compute target are located in different regions, performance degrades because data must transfer across regions. To learn more about ensuring that your data colocates with compute, read [Colocate data with compute](#colocate-data-with-compute).

+The **rslex.log** file provides details about all the file copying, whether or not you chose the mount or download modes. It also describes the Settings (environment variables) used. To start debugging, check whether you have set the [Optimum mount settings for common scenarios](#optimum-mount-settings-for-common-scenarios).

+In the Azure portal, you can select your Storage account, and then **Metrics**, to see the storage metrics:

+You then plot the **SuccessE2ELatency** with **SuccessServerLatency**. If the **metrics show high SuccessE2ELatency and low SuccessServerLatency**, you have limited available threads, or you run low on resources such as CPU, memory, or network bandwidth, you should:

+> Job monitoring supports only compute resources that Azure Machine Learning manages. Jobs with a runtime of less than 5 minutes will not have enough data to populate this view.

+Azure Machine Learning data runtime doesn't use the last `RESERVED_FREE_DISK_SPACE` bytes of disk space, to keep the compute healthy (the default value is `150MB`). If your disk is full, your code is writing files to disk without declaring the files as an output. Therefore, check your code to make sure that data isn't being written erroneously to temporary disk. If you must write files to temporary disk, and that resource is becoming full, consider:

+Files are generally read in *blocks* of 1-4 MB size. Files smaller than a block are read with a single request (GET file.jpg 0-4MB), and files larger than a block have one request made per block (GET file.jpg 0-4MB, GET file.jpg 4-8 MB). The following table shows that files smaller than a 4-MB block result in more storage requests compared to larger files:

+ Title: Schedule data import (preview)

+description: Learn how to schedule an automated data import that brings in data from external sources.

+# Schedule data import jobs (preview)

+In this article, you'll learn how to programmatically schedule data imports and use the schedule UI to do the same. You can create a schedule based on elapsed time. Time-based schedules can be used to take care of routine tasks, such as importing the data regularly to keep them up-to-date. After learning how to create schedules, you'll learn how to retrieve, update and deactivate them via CLI, SDK, and studio UI.

+## Prerequisites

+- You must have an Azure subscription to use Azure Machine Learning. If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/free/) today.

+# [Azure CLI](#tab/cli)

+- Install the Azure CLI and the `ml` extension. Follow the installation steps in [Install, set up, and use the CLI (v2)](how-to-configure-cli.md).

+- Create an Azure Machine Learning workspace if you don't have one. For workspace creation, see [Install, set up, and use the CLI (v2)](how-to-configure-cli.md).

+# [Python SDK](#tab/python)

+- Create an Azure Machine Learning workspace if you don't have one.

+- The [Azure Machine Learning SDK v2 for Python](/python/api/overview/azure/ai-ml-readme).

+# [Studio](#tab/azure-studio)

+- An Azure Machine Learning workspace. See [Create workspace resources](quickstart-create-resources.md).

+- Understanding of Azure Machine Learning pipelines. See [what are machine learning pipelines](concept-ml-pipelines.md), and how to create pipeline job in [CLI v2](how-to-create-component-pipelines-cli.md) or [SDK v2](how-to-create-component-pipeline-python.md).

+## Schedule data import

+To import data on a recurring basis, you must create a schedule. A `Schedule` associates a data import action, and a trigger. The trigger can either be `cron` that use cron expression to describe the wait between runs or `recurrence` that specify using what frequency to trigger job. In each case, you must first define an import data definition. An existing data import, or a data import that is defined inline, works for this. Refer to [Create a data import in CLI, SDK and UI](how-to-import-data-assets.md).

+## Create a schedule

+### Create a time-based schedule with recurrence pattern

+# [Azure CLI](#tab/cli)

+#### YAML: Schedule for data import with recurrence pattern

+```yml

+$schema: https://azuremlschemas.azureedge.net/latest/schedule.schema.json

+name: simple_recurrence_import_schedule

+display_name: Simple recurrence import schedule

+description: a simple hourly recurrence import schedule

+trigger:

+ type: recurrence

+ frequency: day #can be minute, hour, day, week, month

+ interval: 1 #every day

+ schedule:

+ hours: [4,5,10,11,12]

+ minutes: [0,30]

+ start_time: "2022-07-10T10:00:00" # optional - default will be schedule creation time

+ time_zone: "Pacific Standard Time" # optional - default will be UTC

+import_data: ./my-snowflake-import-data.yaml

+```

+#### YAML: Schedule for data import definition inline with recurrence pattern on managed datastore

+```yml

+$schema: https://azuremlschemas.azureedge.net/latest/schedule.schema.json

+name: inline_recurrence_import_schedule

+display_name: Inline recurrence import schedule

+description: an inline hourly recurrence import schedule

+trigger:

+ type: recurrence

+ frequency: day #can be minute, hour, day, week, month

+ interval: 1 #every day

+ schedule:

+ hours: [4,5,10,11,12]

+ minutes: [0,30]

+ start_time: "2022-07-10T10:00:00" # optional - default will be schedule creation time

+ time_zone: "Pacific Standard Time" # optional - default will be UTC

+import_data:

+ type: mltable

+ name: my_snowflake_ds

+ path: azureml://datastores/workspacemanagedstore

+ source:

+ type: database

+ query: select * from TPCH_SF1.REGION

+ connection: azureml:my_snowflake_connection

+```

+`trigger` contains the following properties:

+- **(Required)** `type` specifies the schedule type, either `recurrence` or `cron`. See the following section for more details.

+Next, run this command in the CLI:

+```cli

+> az ml schedule create -f <file-name>.yml

+```

+# [Python SDK](#tab/python)

+```python

+from azure.ai.ml.data_transfer import Database

+from azure.ai.ml.constants import TimeZone

+from azure.ai.ml.entities import (

+ ImportDataSchedule,

+ RecurrenceTrigger,

+ RecurrencePattern,

+)

+from datetime import datetime

+source = Database(connection="azureml:my_sf_connection", query="select * from my_table")

+path = "azureml://datastores/workspaceblobstore/paths/snowflake/schedule/${{name}}"

+my_data = DataImport(

+ type="mltable", source=source, path=path, name="my_schedule_sfds_test"

+)

+schedule_name = "my_simple_sdk_create_schedule_recurrence"

+schedule_start_time = datetime.utcnow()

+recurrence_trigger = RecurrenceTrigger(

+ frequency="day",

+ interval=1,

+ schedule=RecurrencePattern(hours=1, minutes=[0, 1]),

+ start_time=schedule_start_time,

+ time_zone=TimeZone.UTC,

+)

+import_schedule = ImportDataSchedule(

+ name=schedule_name, trigger=recurrence_trigger, import_data=my_data

+)

+ml_client.schedules.begin_create_or_update(import_schedule).result()

+```

+`RecurrenceTrigger` contains following properties:

+- **(Required)** To provide better coding experience, we use `RecurrenceTrigger` for recurrence schedule.

+# [Studio](#tab/azure-studio)

+When you have a data import with satisfactory performance and outputs, you can set up a schedule to automatically trigger this import.

+ 1. Navigate to [Azure Machine Learning studio](https://ml.azure.com)

+ 1. Under **Assets** in the left navigation, select **Data**. On the **Data import** tab, select the imported data asset to which you want to attach a schedule. The **Import jobs history** page should appear, as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/data-import-list.png" lightbox="./media/how-to-schedule-data-import/data-import-list.png" alt-text="Screenshot highlighting the imported data asset name in the Data imports tab.":::

+ 1. At the **Import jobs history** page, select the latest **Import job name** link, to open the pipelines job details page as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/data-import-history.png" lightbox="./media/how-to-schedule-data-import/data-import-history.png" alt-text="Screenshot highlighting the imported data asset guid in the Import jobs history tab.":::

+ 1. At the pipeline job details page of any data import, select **Schedule** -> **Create new schedule** to open the schedule creation wizard, as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/schedule-entry-button.png" lightbox="./media/how-to-schedule-data-import/schedule-entry-button.png" alt-text="Screenshot of the jobs tab, with the create new schedule button.":::

+ 1. The *Basic settings* of the schedule creation wizard have the properties shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/create-schedule-basic-settings.png" lightbox="./media/how-to-schedule-data-import/create-schedule-basic-settings.png" alt-text="Screenshot of schedule creation wizard showing the basic settings.":::

+ - **Name**: the unique identifier of the schedule within the workspace.

+ - **Description**: the schedule description.

+ - **Trigger**: the recurrence pattern of the schedule, which includes the following properties.

+ - **Time zone**: the trigger time calculation is based on this time zone; (UTC) Coordinated Universal Time by default.

+ - **Recurrence** or **Cron expression**: select recurrence to specify the recurring pattern. Under **Recurrence**, you can specify the recurrence frequency - by minutes, hours, days, weeks, or months.

+ - **Start**: the schedule first becomes active on this date. By default, the creation date of this schedule.

+ - **End**: the schedule will become inactive after this date. By default, it's NONE, which means that the schedule remains active until you manually disable it.

+ - **Tags**: the selected schedule tags.

+ After you configure the basic settings, you can select **Review + Create**, and the schedule will automatically submit the data import based on the recurrence pattern you specified. You can also select **Next**, and navigate through the wizard to select or update the data import parameters.

+> [!NOTE]

+> These properties apply to CLI and SDK:

+- **(Required)** `frequency` specifies the unit of time that describes how often the schedule fires. Can have values of `minute`, `hour`, `day`, `week`, or `month`.

+- **(Required)** `interval` specifies how often the schedule fires based on the frequency, which is the number of time units to wait until the schedule fires again.

+- (Optional) `schedule` defines the recurrence pattern, containing `hours`, `minutes`, and `weekdays`.

+ - When `frequency` equals `day`, a pattern can specify `hours` and `minutes`.

+ - When `frequency` equals `week` and `month`, a pattern can specify `hours`, `minutes` and `weekdays`.

+ - `hours` should be an integer or a list, ranging between 0 and 23.

+ - `minutes` should be an integer or a list, ranging between 0 and 59.

+ - `weekdays` a string or list ranging from `monday` to `sunday`.

+ - If `schedule` is omitted, the job(s) triggers according to the logic of `start_time`, `frequency` and `interval`.

+- (Optional) `start_time` describes the start date and time, with a timezone. If `start_time` is omitted, start_time equals the job creation time. For a start time in the past, the first job runs at the next calculated run time.

+- (Optional) `end_time` describes the end date and time with a timezone. If `end_time` is omitted, the schedule continues to trigger jobs until the schedule is manually disabled.

+- (Optional) `time_zone` specifies the time zone of the recurrence. If omitted, the default timezone is UTC. To learn more about timezone values, see [appendix for timezone values](reference-yaml-schedule.md#appendix).

+### Create a time-based schedule with cron expression

+# [Azure CLI](#tab/cli)

+## YAML: Schedule for a data import with cron expression

+#### YAML: Schedule for data import with cron expression (preview)

+```yml

+$schema: https://azuremlschemas.azureedge.net/latest/schedule.schema.json

+name: simple_cron_import_schedule

+display_name: Simple cron import schedule

+description: a simple hourly cron import schedule

+trigger:

+ type: cron

+ expression: "0 * * * *"

+ start_time: "2022-07-10T10:00:00" # optional - default will be schedule creation time

+ time_zone: "Pacific Standard Time" # optional - default will be UTC

+import_data: ./my-snowflake-import-data.yaml

+```

+#### YAML: Schedule for data import definition inline with cron expression (preview)

+```yml

+$schema: https://azuremlschemas.azureedge.net/latest/schedule.schema.json

+name: inline_cron_import_schedule

+display_name: Inline cron import schedule

+description: an inline hourly cron import schedule

+trigger:

+ type: cron

+ expression: "0 * * * *"

+ start_time: "2022-07-10T10:00:00" # optional - default will be schedule creation time

+ time_zone: "Pacific Standard Time" # optional - default will be UTC

+import_data:

+ type: mltable

+ name: my_snowflake_ds

+ path: azureml://datastores/workspaceblobstore/paths/snowflake/${{name}}

+ source:

+ type: database

+ query: select * from TPCH_SF1.REGION

+ connection: azureml:my_snowflake_connection

+```

+The `trigger` section defines the schedule details and contains following properties:

+- **(Required)** `type` specifies the schedule type is `cron`.

+```cli

+> az ml schedule create -f <file-name>.yml

+```

+The list continues here:

+# [Python SDK](#tab/python)

+```python

+from azure.ai.ml.data_transfer import Database

+from azure.ai.ml.constants import TimeZone

+from azure.ai.ml.entities import CronTrigger, ImportDataSchedule

+source = Database(connection="azureml:my_sf_connection", query="select * from my_table")

+path = "azureml://datastores/workspaceblobstore/paths/snowflake/schedule/${{name}}"

+my_data = DataImport(

+ type="mltable", source=source, path=path, name="my_schedule_sfds_test"

+)

+schedule_name = "my_simple_sdk_create_schedule_cron"

+cron_trigger = CronTrigger(

+ expression="15 10 * * 1",

+ start_time=datetime.utcnow(),

+ end_time="2023-12-03T18:40:00",

+)

+import_schedule = ImportDataSchedule(

+ name=schedule_name, trigger=cron_trigger, import_data=my_data

+)

+ml_client.schedules.begin_create_or_update(import_schedule).result()

+```

+The `CronTrigger` section defines the schedule details and contains following properties:

+- **(Required)** To provide better coding experience, we use `CronTrigger` for recurrence schedule.

+The list continues here:

+# [Studio](#tab/azure-studio)

+When you have a data import with satisfactory performance and outputs, you can set up a schedule to automatically trigger this import.

+ 1. Navigate to [Azure Machine Learning studio](https://ml.azure.com)

+ 1. Under **Assets** in the left navigation, select **Data**. On the **Data import** tab, select the imported data asset to which you want to attach a schedule. The **Import jobs history** page should appear, as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/data-import-list.png" lightbox="./media/how-to-schedule-data-import/data-import-list.png" alt-text="Screenshot highlighting the imported data asset name in the Data imports tab.":::

+ 1. At the **Import jobs history** page, select the latest **Import job name** link, to open the pipelines job details page as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/data-import-history.png" lightbox="./media/how-to-schedule-data-import/data-import-history.png" alt-text="Screenshot highlighting the imported data asset guid in the Import jobs history tab.":::

+ 1. At the pipeline job details page of any data import, select **Schedule** -> **Create new schedule** to open the schedule creation wizard, as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/schedule-entry-button.png" lightbox="./media/how-to-schedule-data-import/schedule-entry-button.png" alt-text="Screenshot of the jobs tab, with the create new schedule button.":::

+ 1. The *Basic settings* of the schedule creation wizard have the properties shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/create-schedule-basic-settings.png" lightbox="./media/how-to-schedule-data-import/create-schedule-basic-settings.png" alt-text="Screenshot of schedule creation wizard showing the basic settings.":::

+ - **Name**: the unique identifier of the schedule within the workspace.

+ - **Description**: the schedule description.

+ - **Trigger**: the recurrence pattern of the schedule, which includes the following properties.

+ - **Time zone**: the trigger time calculation is based on this time zone; (UTC) Coordinated Universal Time by default.

+ - **Recurrence** or **Cron expression**: select recurrence to specify the recurring pattern. **Cron expression** allows you to specify more flexible and customized recurrence pattern.

+ - **Start**: the schedule first becomes active on this date. By default, the creation date of this schedule.

+ - **End**: the schedule will become inactive after this date. By default, it's NONE, which means that the schedule remains active until you manually disable it.

+ - **Tags**: the selected schedule tags.

+ After you configure the basic settings, you can select **Review + Create**, and the schedule will automatically submit the data import based on the recurrence pattern you specified. You can also select **Next**, and navigate through the wizard to select or update the data import parameters.

+- **(Required)** `expression` uses a standard crontab expression to express a recurring schedule. A single expression is composed of five space-delimited fields:

+ `MINUTES HOURS DAYS MONTHS DAYS-OF-WEEK`

+ - A single wildcard (`*`), which covers all values for the field. A `*`, in days, means all days of a month (which varies with month and year).

+ - The `expression: "15 16 * * 1"` in the sample above means the 16:15PM on every Monday.

+ - The next table lists the valid values for each field:

+ | Field | Range | Comment |

+ |-|-|--|

+ | `MINUTES` | 0-59 | - |

+ | `HOURS` | 0-23 | - |

+ | `DAYS` | - | Not supported. The value is ignored and treated as `*`. |

+ | `MONTHS` | - | Not supported. The value is ignored and treated as `*`. |

+ | `DAYS-OF-WEEK` | 0-6 | Zero (0) means Sunday. Names of days also accepted. |

+ - To learn more about crontab expressions, see [Crontab Expression wiki on GitHub](https://github.com/atifaziz/NCrontab/wiki/Crontab-Expression).

+ > [!IMPORTANT]

+ > `DAYS` and `MONTH` are not supported. If you pass one of these values, it will be ignored and treated as `*`.

+- (Optional) `start_time` specifies the start date and time with the timezone of the schedule. For example, `start_time: "2022-05-10T10:15:00-04:00"` means the schedule starts from 10:15:00AM on 2022-05-10 in the UTC-4 timezone. If `start_time` is omitted, the `start_time` equals the schedule creation time. For a start time in the past, the first job runs at the next calculated run time.

+- (Optional) `end_time` describes the end date, and time with a timezone. If `end_time` is omitted, the schedule continues to trigger jobs until the schedule is manually disabled.

+- (Optional) `time_zone`specifies the time zone of the expression. If omitted, the timezone is UTC by default. See [appendix for timezone values](reference-yaml-schedule.md#appendix).

+Limitations:

+- Currently, Azure Machine Learning v2 scheduling doesn't support event-based triggers.

+- Use the Azure Machine Learning SDK/CLI v2 to specify a complex recurrence pattern that contains multiple trigger timestamps. The UI only displays the complex pattern and doesn't support editing.

+- If you set the recurrence as the 31st day of every month, the schedule won't trigger jobs in months with less than 31 days.

+### List schedules in a workspace

+# [Azure CLI](#tab/cli)

+# [Python SDK](#tab/python)

+[!notebook-python[] (~/azureml-examples-main/sdk/python/schedules/job-schedule.ipynb?name=list_schedule)]

+# [Studio](#tab/azure-studio)

+In the studio portal, under the **Jobs** extension, select the **All schedules** tab. That tab shows all your job schedules created by the SDK/CLI/UI, in a single list. In the schedule list, you have an overview of all schedules in this workspace, as shown in this screenshot:

+### Check schedule detail

+# [Azure CLI](#tab/cli)

+```cli

+az ml schedule show -n simple_cron_data_import_schedule

+``````

+# [Python SDK](#tab/python)

+```python

+created_schedule = ml_client.schedules.get(name=schedule_name)

+[created_schedule.name]

+```

+# [Studio](#tab/azure-studio)

+You can select a schedule name to show the schedule details page. The schedule details page contains the following tabs, as shown in this screenshot:

+- **Overview**: basic information for the specified schedule.

+ :::image type="content" source="./media/how-to-schedule-data-import/schedule-detail-overview.png" alt-text="Screenshot of the overview tab in the schedule details page." :::

+- **Job definition**: defines the job that the specified schedule triggers, as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/schedule-detail-job-definition.png" alt-text="Screenshot of the job definition tab in the schedule details page.":::

+### Update a schedule

+# [Azure CLI](#tab/cli)

+```cli

+az ml schedule update -n simple_cron_data_import_schedule --set description="new description" --no-wait

+```

+> [!NOTE]

+> To update more than just tags/description, it is recommended to use `az ml schedule create --file update_schedule.yml`

+# [Python SDK](#tab/python)

+```python

+job_schedule = ml_client.schedules.begin_create_or_update(

+ schedule=job_schedule

+).result()

+print(job_schedule)

+```

+# [Studio](#tab/azure-studio)

+#### Update a data import definition to existing schedule

+To change the import frequency, or to create a new association for the data import job, you can update the import definition of an existing schedule.

+> [!NOTE]

+> To update an existing schedule, the association of the schedule with the old import definition will be removed. A schedule can have only one import job definition. However, multiple schedules can call one data import definition.

+ 1. Navigate to [Azure Machine Learning studio](https://ml.azure.com)

+ 1. Under **Assets** in the left navigation, select **Data**. On the **Data import** tab, select the imported data asset to which you want to attach a schedule. Then, the **Import jobs history** page opens, as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/data-import-list.png" alt-text="Screenshot highlighting the imported data asset name in the Data imports tab.":::

+ 1. At the **Import jobs history** page, select the latest **Import job name** link, to open the pipelines job details page as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/data-import-history.png" alt-text="Screenshot highlighting the imported data asset guid in the Import jobs history tab.":::

+ 1. At the pipeline job details page of any data import, select **Schedule** -> **Updated to existing schedule**, to open the Select schedule wizard, as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/schedule-update-button.png" alt-text="Screenshot of the jobs tab with the schedule button selected, showing the create update to existing schedule button.":::

+ 1. Select an existing schedule from the list, as shown in this screenshot:

+ :::image type="content" source="./media/how-to-schedule-data-import/update-select-schedule.png" alt-text="Screenshot of update select schedule showing the select schedule tab." :::

+ > [!IMPORTANT]

+ > Make sure to select the correct schedule to update. Once you finish the update, the schedule will trigger different data imports.

+ 1. You can also modify the source, query and change the destination path, for future data imports that the schedule triggers.

+ 1. Select **Review + Update** to finish the update process. The completed update will send a notification.

+ 1. You can view the new data import definition in the schedule details page when the update is completed.

+#### Update in schedule detail page

+In the schedule details page, you can select **Update settings** to update both the basic settings and advanced settings, including the job input/output and runtime settings of the schedule, as shown in this screenshot:

+### Disable a schedule

+# [Azure CLI](#tab/cli)

+```cli

+az ml schedule disable -n simple_cron_data_import_schedule --no-wait

+```

+# [Python SDK](#tab/python)

+```python

+job_schedule = ml_client.schedules.begin_disable(name=schedule_name).result()

+job_schedule.is_enabled

+```

+# [Studio](#tab/azure-studio)

+You can disable the current schedule at the schedule details page. You can also disable schedules at the **All schedules** tab.

+### Enable a schedule

+# [Azure CLI](#tab/cli)

+```cli

+az ml schedule enable -n simple_cron_data_import_schedule --no-wait

+```

+# [Python SDK](#tab/python)

+```python

+# Update trigger expression

+job_schedule.trigger.expression = "10 10 * * 1"

+job_schedule = ml_client.schedules.begin_create_or_update(

+ schedule=job_schedule

+).result()

+print(job_schedule)

+```

+# [Studio](#tab/azure-studio)

+On the schedule details page, you can enable the current schedule. You can also enable schedules at the **All schedules** tab.

+## Delete a schedule

+> [!IMPORTANT]

+> A schedule must be disabled before deletion. Deletion is an unrecoverable action. After a schedule is deleted, you can never access or recover it.

+# [Azure CLI](#tab/cli)

+```cli

+az ml schedule delete -n simple_cron_data_import_schedule

+```

+# [Python SDK](#tab/python)

+```python

+# Only disabled schedules can be deleted

+ml_client.schedules.begin_disable(name=schedule_name).result()

+ml_client.schedules.begin_delete(name=schedule_name).result()

+```

+# [Studio](#tab/azure-studio)

+You can delete a schedule from the schedule details page or the all schedules tab.

+## RBAC (Role-based-access-control) support

+Schedules are generally used for production. To prevent problems, workspace admins may want to restrict schedule creation and management permissions within a workspace.

+There are currently three action rules related to schedules, and you can configure them in Azure portal. See [how to manage access to an Azure Machine Learning workspace.](how-to-assign-roles.md#create-custom-role) to learn more.

+| Action | Description | Rule |

+|--|-||

+| Read | Get and list schedules in Machine Learning workspace | Microsoft.MachineLearningServices/workspaces/schedules/read |

+| Write | Create, update, disable and enable schedules in Machine Learning workspace | Microsoft.MachineLearningServices/workspaces/schedules/write |

+| Delete | Delete a schedule in Machine Learning workspace | Microsoft.MachineLearningServices/workspaces/schedules/delete |

+## Next steps

+* Learn more about the [CLI (v2) data import schedule YAML schema](./reference-yaml-schedule-data-import.md).

+* Learn how to [manage imported data assets](how-to-manage-imported-data-assets.md).

+Experiments and jobs (or runs) in Azure Machine Learning can be queried using MLflow. You don't need to install any specific SDK to manage what happens inside of a training job, creating a more seamless transition between local runs and the cloud by removing cloud-specific dependencies. In this article, you'll learn how to query and compare experiments and runs in your workspace using Azure Machine Learning and MLflow SDK in Python.

+### REST API

+Query and searching experiments and runs is also available using the MLflow REST API. See [Using MLflow REST with Azure Machine Learning](https://github.com/Azure/azureml-examples/blob/main/sdk/python/using-mlflow/using-rest-api/using_mlflow_rest_api.ipynb) for an example about how to consume it.

+## Query and search experiments

+Use MLflow to search for experiments inside of your workspace. See the following examples:

+* Get all active experiments:

+ ```python

+ mlflow.search_experiments()

+ ```

+

+ > [!NOTE]

+ > In legacy versions of MLflow (<2.0) use method `mlflow.list_experiments()` instead.

+* Get all the experiments, including archived:

+ ```python

+ from mlflow.entities import ViewType

+

+ mlflow.search_experiments(view_type=ViewType.ALL)

+ ```

+* Get a specific experiment by name:

+ ```python

+ mlflow.get_experiment_by_name(experiment_name)

+ ```

+* Get a specific experiment by ID:

+ ```python

+ mlflow.get_experiment('1234-5678-90AB-CDEFG')

+ ```

+### Searching experiments

+The `search_experiments()` method available since Mlflow 2.0 allows searching experiment matching a criteria using `filter_string`.

+* Retrieve multiple experiments based on their IDs:

+ ```python

+ mlflow.search_experiments(filter_string="experiment_id IN ("

+ "'CDEFG-1234-5678-90AB', '1234-5678-90AB-CDEFG', '5678-1234-90AB-CDEFG')"

+ )

+ ```

+* Retrieve all experiments created after a given time:

+ ```python

+ import datetime

+ dt = datetime.datetime(2022, 6, 20, 5, 32, 48)

+ mlflow.search_experiments(filter_string=f"creation_time > {int(dt.timestamp())}")

+ ```

+* Retrieve all experiments with a given tag:

+ ```python

+ mlflow.search_experiments(filter_string=f"tags.framework = 'torch'")

+ ```

+## Query and search runs

+MLflow allows searching runs inside of any experiment, including multiple experiments at the same time. The method `mlflow.search_runs()` accepts the argument `experiment_ids` and `experiment_name` to indicate on which experiments you want to search. You can also indicate `search_all_experiments=True` if you want to search across all the experiments in the workspace:

+* By experiment name:

+ ```python

+ mlflow.search_runs(experiment_names=[ "my_experiment" ])

+ ```

+* By experiment ID:

+ ```python

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ])

+ ```

+* Search across all experiments in the workspace:

+ ```python

+ mlflow.search_runs(filter_string="params.num_boost_round='100'", search_all_experiments=True)

+ ```

+Notice that `experiment_ids` supports providing an array of experiments, so you can search runs across multiple experiments if required. This may be useful in case you want to compare runs of the same model when it is being logged in different experiments (by different people, different project iterations, etc.).

+> [!IMPORTANT]

+> If `experiment_ids`, `experiment_names`, or `search_all_experiments` are not indicated, then MLflow will search by default in the current active experiment. You can set the active experiment using `mlflow.set_experiment()`

+By default, MLflow returns the data in Pandas `Dataframe` format, which makes it handy when doing further processing our analysis of the runs. Returned data includes columns with:

+- Basic information about the run.

+- Parameters with column's name `params.<parameter-name>`.

+- Metrics (last logged value of each) with column's name `metrics.<metric-name>`.

+All metrics and parameters are also returned when querying runs. However, for metrics containing multiple values (for instance, a loss curve, or a PR curve), only the last value of the metric is returned. If you want to retrieve all the values of a given metric, uses `mlflow.get_metric_history` method. See [Getting params and metrics from a run](#getting-params-and-metrics-from-a-run) for an example.

+* Order runs by the attribute `start_time`:

+ ```python

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ],

+ order_by=["attributes.start_time DESC"])

+ ```

+* Order and retrieve the last run:

+ ```python

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ],

+ max_results=1, order_by=["attributes.start_time DESC"])

+ ```

+* Order runs by metric's values:

+ ```python

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ]).sort_values("metrics.accuracy", ascending=False)

+ ```

+ > [!WARNING]

+ > Using `order_by` with expressions containing `metrics.*` in the parameter `order_by` is not supported by the moment. Please use `order_values` method from Pandas as shown in the next example.

+You can also look for a run with a specific combination in the hyperparameters using the parameter `filter_string`. Use `params` to access run's parameters, `metrics` to access metrics logged in the run, and `attributes` to access run information details. MLflow supports expressions joined by the AND keyword (the syntax does not support OR):

+* Search runs based on a parameter's value:

+ ```python

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ],

+ filter_string="params.num_boost_round='100'")

+ ```

+ > [!WARNING]

+ > Only operators `=`, `like`, and `!=` are supported for filtering `parameters`.

+* Search runs based on a metric's value:

+ ```python

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ],

+ filter_string="metrics.auc>0.8")

+ ```

+* Search runs with a given tag:

+ ```python

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ],

+ filter_string="tags.framework='torch'")

+ ```

+* Search runs created by a given user:

+ ```python

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ],

+ filter_string="attributes.user_id = 'John Smith'")

+ ```

+* Search runs that have failed. See [Filter runs by status](#filter-runs-by-status) for possible values:

+ ```python

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ],

+ filter_string="attributes.status = 'Failed'")

+ ```

+* Search runs created after a given time:

+ ```python

+ import datetime

+ dt = datetime.datetime(2022, 6, 20, 5, 32, 48)

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ],

+ filter_string=f"attributes.creation_time > '{int(dt.timestamp())}'")

+ ```

+ > [!TIP]

+ > Notice that for the key `attributes`, values should always be strings and hence encoded between quotes.

+

+* Search runs having the ID in a given set:

+ ```python

+ mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ],

+ filter_string="attributes.run_id IN ('1234-5678-90AB-CDEFG', '5678-1234-90AB-CDEFG')")

+ ```

+When filtering runs by status, notice that MLflow uses a different convention to name the different possible status of a run compared to Azure Machine Learning. The following table shows the possible values:

+Example:

+mlflow.search_runs(experiment_ids=[ "1234-5678-90AB-CDEFG" ],

+ filter_string="attributes.status = 'Failed'")

+To compare and evaluate the quality of your jobs and models in Azure Machine Learning studio, use the [preview panel](./how-to-enable-preview-features.md) to enable the feature. Once enabled, you can compare the parameters, metrics, and tags between the jobs and/or models you selected.

+| Ordering runs by attributes | **&check;** | **&check;** |

+| Filtering runs by attributes | **&check;** | **&check;** |

+| Filtering runs with string comparators (params, tags, and attributes): `LIKE`/`ILIKE` | **&check;** | **&check;** |

+> - ¹ Check the section [Ordering runs](#ordering-runs) for instructions and examples on how to achieve the same functionality in Azure Machine Learning.

+> If you use `mltable` as your major input data, you need to install 'mltable' library into your environment. See the line 9 of this [conda file](https://github.com/Azure/azureml-examples/blob/main/cli/jobs/parallel/1a_oj_sales_prediction/src/parallel_train/conda.yaml) example.

+ Title: Interactive data wrangling with Apache Spark in Azure Machine Learning

+# Interactive Data Wrangling with Apache Spark in Azure Machine Learning

+Data wrangling becomes one of the most important steps in machine learning projects. The Azure Machine Learning integration, with Azure Synapse Analytics, provides access to an Apache Spark pool - backed by Azure Synapse - for interactive data wrangling using Azure Machine Learning Notebooks.

+- Serverless Spark compute

+Before starting data wrangling tasks, you need familiarity with the process of storing secrets

+in the Azure Key Vault. You also need to know how to handle role assignments in the Azure storage accounts. The following sections review these concepts. Then, we'll explore the details of interactive data wrangling using the Spark pools in Azure Machine Learning Notebooks.

+Azure Machine Learning offers serverless Spark compute, and [attached Synapse Spark pool](./how-to-manage-synapse-spark-pool.md), for interactive data wrangling with Apache Spark in Azure Machine Learning Notebooks. The serverless Spark compute doesn't require creation of resources in the Azure Synapse workspace. Instead, a fully managed serverless Spark compute becomes directly available in the Azure Machine Learning Notebooks. Using a serverless Spark compute is the easiest approach to access a Spark cluster in Azure Machine Learning.

+### Serverless Spark compute in Azure Machine Learning Notebooks

+A serverless Spark compute is available in Azure Machine Learning Notebooks by default. To access it in a notebook, select **Serverless Spark Compute** under **Azure Machine Learning Serverless Spark** from the **Compute** selection menu.

+2. Select **Apache Spark version** from the dropdown menu.

+3. Select **Instance type** from the dropdown menu. The following instance types are currently supported:

+4. Input a Spark **Session timeout** value, in minutes.

+5. Select whether to **Dynamically allocate executors**

+6. Select the number of **Executors** for the Spark session.

+7. Select **Executor size** from the dropdown menu.

+8. Select **Driver size** from the dropdown menu.

+9. To use a conda file to configure a Spark session, check the **Upload conda file** checkbox. Then, select **Browse**, and choose the conda file with the Spark session configuration you want.

+10. Add **Configuration settings** properties, input values in the **Property** and **Value** textboxes, and select **Add**.

+11. Select **Apply**.

+12. Select **Stop session** in the **Configure new session?** pop-up.

+The session configuration changes persist and become available to another notebook session that is started using the serverless Spark compute.

+- To use the serverless Spark compute, select **Serverless Spark Compute** under **Azure Machine Learning Serverless Spark**, from the **Compute** selection menu.

+- To use an attached Synapse Spark pool, select an attached Synapse Spark pool under **Synapse Spark pools**, from the **Compute** selection menu.

+2. [Create Azure Key Vault secrets](./apache-spark-environment-configuration.md#store-azure-storage-account-credentials-as-secrets-in-azure-key-vault) for the service principal tenant ID, client ID and client secret values.

+3. Select **Serverless Spark compute** under **Azure Machine Learning Serverless Spark** from the **Compute** selection menu, or select an attached Synapse Spark pool under **Synapse Spark pools** from the **Compute** selection menu.

+4. To set the service principal tenant ID, client ID and client secret in the configuration, and execute the following code sample.

+5. Import and wrangle data using data URI in format `abfss://<FILE_SYSTEM_NAME>@<STORAGE_ACCOUNT_NAME>.dfs.core.windows.net/<PATH_TO_DATA>` as shown in the code sample, using the Titanic data.

+1. Select **Serverless Spark compute** under **Azure Machine Learning Serverless Spark** from the **Compute** selection menu, or select an attached Synapse Spark pool under **Synapse Spark pools** from the **Compute** selection menu.

+To access data from [Azure Machine Learning Datastore](how-to-datastore.md), define a path to data on the datastore with [URI format](how-to-create-data-assets.md?tabs=cli#create-data-assets) `azureml://datastores/<DATASTORE_NAME>/paths/<PATH_TO_DATA>`. To wrangle data from an Azure Machine Learning Datastore in a Notebooks session interactively:

+1. Select **Serverless Spark compute** under **Azure Machine Learning Serverless Spark** from the **Compute** selection menu, or select an attached Synapse Spark pool under **Synapse Spark pools** from the **Compute** selection menu.

+- [Submit Spark jobs in Azure Machine Learning](./how-to-submit-spark-jobs.md)

+| **Replication and consistency**| You need to be able to configure the full array of [tunable consistency settings](https://cassandra.apache.org/doc/latest/cassandr)) |

+| **Data model**| You are migrating workloads which have a mixture of uniform distribution of data, and skewed data (with respect to both storage and throughput across partition keys) requiring flexibility on vertical scale of nodes. | You are migrating workloads which have a mixture of uniform distribution of data, and skewed data (with respect to both storage and throughput across partition keys) requiring flexibility on vertical scale of nodes. | You are building a new application, or your existing application has a relatively uniform distribution of data with respect to both storage and throughput across partition keys. |

+ > [!NOTE]

+ > If there is a connectivity issue with Azure or if the appliance services are down for more than 90 minutes, the active replication cycles for replicating servers are reset to 0% and the respective cycle runs from the beginning.

+# Manage an Azure Database for MySQL - Flexible Server using Azure portal

+ Title: Create connection monitor - ARMClient

+# Create a connection monitor using the ARMClient

+## Steps to create a connection monitor using ARMClient

+ * sources - Choose from endpoints created above. Azure based source endpoints need to have Azure Network Watcher extension installed and non-Azure based source endpoints need to haveAzure Log Analytics agent installed. To install an agent for your source, see [Install monitoring agents](./connection-monitor-overview.md#install-monitoring-agents).

+description: Learn how to use queries to replace the deprecated fields in the Traffic Analytics schema with the new ones.

+# Use sample queries to replace deprecated fields in traffic analytics schema (March 2020 schema update)

+The [Traffic analytics log schema](./traffic-analytics-schema.md) includes the following new fields:

+The new fields provide information about source and destination IP addresses, and they simplify queries.

+The following three examples show you how to replace the old fields with the new ones.

+## Example 1: `VMIP_s`, `Subscription_g`, `Region_s`, `Subnet_s`, `VM_s`, `NIC_s`, and `PublicIPs_s` fields

+## Example 2: `NSGRules_s` field

+The schema no longer aggregates data across a network security group (NSG). In the updated schema, `NSGList_s` contains only one network security group. Also, `NSGRules` contains only one rule. The complicated formatting has been removed here and in other fields, as shown in the following example.

+## Example 3: `FlowCount_d` field

+Because the schema doesn't club data across the network security group, the `FlowCount_d` is simply:

+- To get answers to frequently asked questions, see [Traffic analytics FAQ](traffic-analytics-faq.yml).

+- To learn more about functionality, see [Traffic analytics overview](traffic-analytics.md).

+> [!IMPORTANT]

+> As the name implies, temporary tablespaces in PostgreSQL are used for temporary objects, as well as other internal database operations, such as sorting. Therefore we do not recommend creating user schema objects in temporary tablespace, as we dont guarantee durability of such objects after Server restarts, HA failovers, etc.

+| 9. | Enable VM management from the Azure portal. </br></br>Enabling this immediately after activating the Azure Stack Edge Pro 2 device occasionally causes an error. Wait one minute and retry. | Navigate to the ASE resource in the Azure portal, go to **Edge services**, select **Virtual machines** and select **Enable**. |

+| 10. | Run the diagnostics tests for the Azure Stack Edge Pro 2 device in the local web UI, and verify they all pass. </br></br>You may see a warning about a disconnected, unused port. You should fix the issue if the warning relates to any of these ports:</br></br>- Port 2 - management</br>- Port 3 - access network</br>- Port 4 - data networks</br></br>For all other ports, you can ignore the warning. </br></br>If there are any errors, resolve them before continuing with the remaining steps. This includes any errors related to invalid gateways on unused ports. In this case, either delete the gateway IP address or set it to a valid gateway for the subnet. | [Run diagnostics, collect logs to troubleshoot Azure Stack Edge device issues](../databox-online/azure-stack-edge-gpu-troubleshoot.md) |

+description: Learn how to configure high-availability (HA) Pacemaker cluster providers for Azure Monitor for SAP solutions.

+#Customer intent: As a developer, I want to create a high-availability Pacemaker cluster so that I can use the resource with Azure Monitor for SAP solutions.

+# Create high-availability cluster provider for Azure Monitor for SAP solutions

+In this how-to guide, you learn how to create a high-availability (HA) Pacemaker cluster provider for Azure Monitor for SAP solutions. You install the HA agent and then create the provider for Azure Monitor for SAP solutions.

+## Install an HA agent

+Before you add providers for HA (Pacemaker) clusters, install the appropriate agent for your environment in each cluster node.

+For SUSE-based clusters, install **ha_cluster_provider** in each node. For more information, see the [HA cluster exporter installation guide](https://github.com/ClusterLabs/ha_cluster_exporter#installation). Supported SUSE versions include SLES for SAP 12 SP3 and later versions.

+For RHEL-based clusters, install **performance co-pilot (PCP)** and the **pcp-pmda-hacluster** subpackage in each node. For more information, see the [PCP HACLUSTER agent installation guide](https://access.redhat.com/articles/6139852). Supported RHEL versions include 8.2, 8.4, and later versions.

+For RHEL-based Pacemaker clusters, also install [PMProxy](https://access.redhat.com/articles/6139852) in each node.

+### Install an HA cluster exporter on RHEL

+1. Install and enable the HA cluster PMDA. Replace `$PCP_PMDAS_DIR` with the path where `hacluster` is installed. Use the `find` command in Linux to find the path.

+1. Data is then collected in the system by PCP. You can export the data by using `pmproxy` at `http://<SERVER-NAME-OR-IP-ADDRESS>:44322/metrics?names=ha_cluster`. Replace `<SERVER-NAME-OR-IP-ADDRESS>` with your server name or IP address.

+To [enable TLS 1.2 or higher](enable-tls-azure-monitor-sap-solutions.md), follow the steps in [this article](https://github.com/ClusterLabs/ha_cluster_exporter#tls-and-basic-authentication).

+## Create a provider for Azure Monitor for SAP solutions

+1. On the resource's menu, under **Settings**, select **Providers**.

+ 

+1. (Optional) Select **Enable secure communication** and choose a certificate type.

+ 

+ 

+When the provider settings validation operation fails with the code `PrometheusURLConnectionFailure`:

+1. Verify that the Prometheus endpoint is reachable from the subnet that you provided when you created the Azure Monitor for SAP solutions resource.

+# Configure SAP HANA provider for Azure Monitor for SAP solutions

+In this how-to guide, you learn how to configure an SAP HANA provider for Azure Monitor for SAP solutions through the Azure portal.

+To [enable TLS 1.2 higher](enable-tls-azure-monitor-sap-solutions.md) for the SAP HANA provider, follow the steps in [this SAP document](https://www.sap.com/documents/2018/11/b865eb91-287d-0010-87a3-c30de2ffd8ff.html).

+- An Azure subscription.

+

+ 

+ 1. Optionally, select **Enable secure communication** and choose the certificate type from the dropdown menu.

+ 1. For **IP address**, enter the IP address or hostname of the server that runs the SAP HANA instance that you want to monitor. If you're using a hostname, make sure there's connectivity within the virtual network.

+ 1. For **Database tenant**, enter the HANA database that you want to connect to. We recommend that you use **SYSTEMDB** because tenant databases don't have all monitoring views. For legacy single-container HANA 1.0 instances, leave this field blank.

+ 1. For **Database username**, enter the dedicated SAP HANA database user. This user needs the **MONITORING** or **BACKUP CATALOG READ** role assignment. For nonproduction SAP HANA instances, use **SYSTEM** instead.

+ 1. For **Database password**, enter the password for the database username. You can either enter the password directly or use a secret inside Azure Key Vault.

+description: This article provides details to configure an IBM Db2 provider for Azure Monitor for SAP solutions.

+# Create IBM Db2 provider for Azure Monitor for SAP solutions

+In this how-to guide, you learn how to create an IBM Db2 provider for Azure Monitor for SAP solutions through the Azure portal.

+- An Azure subscription.

+## Create a user for the Db2 server

+First, create a new user for your Db2 server for use by Azure Monitor for SAP solutions. Then run the following script to provide the new Db2 user with appropriate permissions. Make sure to replace `<username>` with the Db2 username.

+To [enable TLS 1.2 or higher](enable-tls-azure-monitor-sap-solutions.md), follow the steps in [this document](https://assets.cdn.sap.com/sapcom/docs/2018/12/d2922a3b-307d-0010-87a3-c30de2ffd8ff.pdf).

+## Create an IBM Db2 provider

+1. Go to the Azure Monitor for SAP solutions service.

+ 1. (Optional) Select **Enable secure communication** and choose a certificate type from the dropdown list.

+# Configure Linux provider for Azure Monitor for SAP solutions

+In this how-to guide, you learn how to create a Linux OS provider for Azure Monitor for SAP solutions resources.

+- Install the [node exporter latest version](https://prometheus.io/download/#node_exporter) in each SAP host that you want to monitor, either BareMetal or Azure virtual machine (VM). For more information, see the [node exporter GitHub repository](https://github.com/prometheus/node_exporter).

+1. Run `tar xvfz node_exporter-*.*-amd64.tar.gz`.

+1. Run `cd node_exporter-*.*-amd64`.

+1. Run `./node_exporter`.

+## Script to set up the node exporter

+### Set up a cron job to start node exporter on a VM restart

+1. If the target VM is restarted or stopped, node exporter is also stopped. It must be manually started again to continue monitoring.

+1. Run the `sudo crontab -e` command to open a cron file.

+1. Add the command `@reboot cd /path/to/node/exporter && nohup ./node_exporter &` at the end of the cron file. This starts node exporter on a VM reboot.

+ ```shell

+ # If you do not have a crontab file already, create one by running the command: sudo crontab -e

+ sudo crontab -l > crontab_new

+ echo "@reboot cd /path/to/node/exporter && nohup ./node_exporter &" >> crontab_new

+ sudo crontab crontab_new

+ sudo rm crontab_new

+ ```

+To [enable TLS 1.2 or higher](enable-tls-azure-monitor-sap-solutions.md), follow the steps in [this article](https://prometheus.io/docs/guides/tls-encryption/).

+ 1. (Optional) Select **Enable secure communication**, choose a certificate type.

+ 1. If you're using `firewall-cmd`, run `_firewall-cmd_ _--permanent_ _--add-port=9100/tcp_ ` and then run `_firewall-cmd_ _--reload_`.

+ 1. If you're using `ufw`, run `_ufw_ _allow_ _9100/tcp_` and then run `_ufw_ _reload_`.

+1. If the Linux host is an Azure VM, make sure that all applicable network security groups allow inbound traffic at port 9100 from **VirtualNetwork** as the source.

+When the provider settings validation operation fails with the code `PrometheusURLConnectionFailure`:

+ 1. If you're using `firewall-cmd`, run `_firewall-cmd_ _--permanent_ _--add-port=9100/tcp_ ` and then run `_firewall-cmd_ _--reload_`.

+ 1. If you're using `ufw`, run `_ufw_ _allow_ _9100/tcp_` and then run `_ufw_ _reload_`.

+ 1. Go to the folder where you installed the node exporter. The file name resembles `node_exporter-*.*-amd64`.

+ 1. Adding `nohup` and `&` to the preceding command decouples `node_exporter` from the Linux machine command line. If they're not included, `node_exporter` stops when the command line is closed.

+1. Verify that the Prometheus endpoint is reachable from the subnet that you provided when you created the Azure Monitor for SAP solutions resource.

+## Suggestion

+Use this suggestion for troubleshooting

+### Enable the node exporter

+1. Run the `nohup ./node_exporter &` command to enable `node_exporter`.

+1. Adding `nohup` and `&` to the preceding command decouples `node_exporter` from the Linux machine command line. If they're not included, `node_exporter` stops when the command line is closed.

+ Title: Configure SQL Server provider for Azure Monitor for SAP solutions

+description: Learn how to configure a SQL Server provider for use with Azure Monitor for SAP solutions.

+#Customer intent: As a developer, I want to configure a SQL Server provider so that I can use Azure Monitor for SAP solutions for monitoring.

+# Configure SQL Server for Azure Monitor for SAP solutions

+In this how-to guide, you learn how to configure a SQL Server provider for Azure Monitor for SAP solutions through the Azure portal.

+- An Azure subscription.

+## Open a Windows port

+Open the Windows port in the local firewall of SQL Server and the network security group where SQL Server and Azure Monitor for SAP solutions exist. The default port is 1433.

+## Configure SQL Server

+Configure SQL Server to accept sign-ins from Windows and SQL Server:

+1. Open SQL Server Management Studio.

+1. Open **Server Properties** > **Security** > **Authentication**.

+Create a user for Azure Monitor for SAP solutions to sign in to SQL Server by using the following script. Make sure to replace:

+- `<Database to monitor>` with your SAP database's name.

+- `<password>` with the password for your user.

+To enable [TLS 1.2 or higher](enable-tls-azure-monitor-sap-solutions.md), follow the steps in [this article](/sql/database-engine/configure-windows/configure-sql-server-encryption?view=sql-server-ver15&preserve-view=true).

+## Install an Azure Monitor for SAP solutions provider

+1. On the resource menu, under **Settings**, select **Providers**.

+ 1. (Optional) Select **Enable secure communication** and choose a certificate type from the dropdown list.

+ 1. For **SID**, enter the SAP system identifier.

+ 1. Select **Create** to create the provider.

+Over the years, customers running the SAP Business Warehouse (BW) system see an exponential growth in database size, which increases compute cost. To achieve the right balance of cost and performance, customers can use near-line storage (NLS) to migrate historical data.

+The NLS implementation based on SAP IQ is the standard method by SAP to move historical data from a primary database (SAP HANA or AnyDB). The integration of SAP IQ makes it possible to separate frequently accessed data from infrequently accessed data, which makes less resource demand in the SAP BW system.

+This guide provides guidelines for planning, deploying, and configuring SAP BW NLS with SAP IQ on Azure. This guide covers common Azure services and features that are relevant for SAP IQ NLS deployment and doesn't cover any NLS partner solutions.

+This guide doesn't replace SAP's standard documentation on NLS deployment with SAP IQ. Instead, it complements the official installation and administration documentation.

+In an operative SAP BW system, the volume of data increases constantly because of business and legal requirements. The large volume of data can affect the performance of the system and increase the administration effort, which results in the need to implement a data-aging strategy.

+If you want to keep the amount of data in your SAP BW system without deleting, you can use data archiving. The data is first moved to archive or near-line storage and then deleted from the SAP BW system. You can either access the data directly or load it back as required, depending on how the data has been archived.

+SAP BW users can use SAP IQ as a near-line storage solution. The adapter for SAP IQ as a near-line solution is delivered with the SAP BW system. With NLS implemented, frequently used data is stored in an SAP BW online database (SAP HANA or AnyDB). Infrequently accessed data is stored in SAP IQ, which reduces the cost to manage data and improves the performance of the SAP BW system. To ensure consistency between online data and near-line data, the archived partitions are locked and are read-only.

+SAP IQ supports two types of architecture: simplex and multiplex. In a simplex architecture, a single instance of an SAP IQ server runs on a single virtual machine. Files might be located on a host machine or on a network storage device.

+> [!IMPORTANT]

+In Azure, the SAP IQ server must be implemented on a separate virtual machine (VM). We don't recommend installing SAP IQ software on an existing server that already has other database instances running, because SAP IQ uses complete CPU and memory for its own usage. One SAP IQ server can be used for multiple SAP NLS implementations.

+- **Storage**: In Azure, SAP IQ supports premium managed disks (Windows and Linux), Azure shared disks (Windows only), and Azure NetApp Files (Linux only).

+For more up-to-date information based on your SAP IQ release, see the [Product Availability Matrix](https://userapps.support.sap.com/sap/support/pam).

+Sizing of SAP IQ is confined to CPU, memory, and storage. You can find general sizing guidelines for SAP IQ on Azure in [SAP note 1951789](https://launchpad.support.sap.com/#/notes/1951789). The sizing recommendation that you get by following the guidelines needs to be mapped to certified Azure virtual machine types for SAP. [SAP note 1928533](https://launchpad.support.sap.com/#/notes/1928533) provides the list of supported SAP products and Azure VM types.

+If you're already running your SAP systems on Azure, you've probably identified your region. SAP IQ deployment must be in the same region as your SAP BW system for which you're implementing the NLS solution.

+### Deployment options

+To achieve redundancy of SAP systems in an Azure infrastructure, your application needs to be deployed in either flexible scale set, availability zones, or availability sets. Although you can achieve SAP IQ high availability by using the SAP IQ multiplex architecture, the multiplex architecture doesn't meet the requirements of the NLS solution.

+To achieve high availability for the SAP IQ simplex architecture, you need to configure a two-node cluster with a custom solution. The two-node SAP IQ cluster can be deployed in flexible scale set with FD=1, availability zones or availability sets. However, it is advised to configure zone redundant storage when setting up a highly available solution across availability zones.

+Based on SAP IQ sizing, you need to map your requirements to Azure virtual machines. This approach is supported in Azure for SAP products. [SAP note 1928533](https://launchpad.support.sap.com/#/notes/1928533) is a good starting point that lists supported Azure VM types for SAP products on Windows and Linux.

+Beyond the selection of only supported VM types, you also need to check whether those VM types are available in specific regions. You can check the availability of VM types on the [Products available by region](https://azure.microsoft.com/global-infrastructure/services/) webpage. To choose the pricing model, see [Azure virtual machines for SAP workload](planning-guide.md#azure-virtual-machines-for-sap-workload).

+Azure Storage has various storage types available for customers. You can find details about them in the article [What disk types are available in Azure?](../../virtual-machines/disks-types.md).

+Some of the storage types in Azure have limited use for SAP scenarios, but other types are well suited or optimized for specific SAP workload scenarios. For more information, see the [Azure Storage types for SAP workload](planning-guide-storage.md) guide. It highlights the storage options that are suited for SAP.

+ A [managed disk](../../virtual-machines/managed-disks-overview.md) is a block-level storage volume that Azure manages. You can use managed disks for SAP IQ simplex deployment. Various types of managed disks are available, but we recommend that you use [premium SSDs](../../virtual-machines/disks-types.md#premium-ssds) for SAP IQ.

+ [Shared disks](../../virtual-machines/disks-shared.md) are a new feature for Azure managed disks that allow you to attach a managed disk to multiple VMs simultaneously. Shared managed disks don't natively offer a fully managed file system that can be accessed through SMB or NFS. You need to use a cluster manager like a [Windows Server failover cluster](https://github.com/MicrosoftDocs/windowsserverdocs/blob/master/WindowsServerDocs/failover-clustering/failover-clustering-overview.md) (WSFC), which handles cluster node communication and write locking.

+The following table lists the recommendations for each storage type based on the operating system:

+## [Deploy SAP IQ on Windows](#tab/sapiq-windows)

+### Windows server preparation and installation

+### High-availability deployment on Windows

+SAP IQ supports both a simplex and a multiplex architecture. For the NLS solution, only simplex server architecture is available and evaluated. Simplex is a single instance of an SAP IQ server running on a single virtual machine.

+Technically, you can achieve SAP IQ high availability by using a multiplex server architecture, but the multiplex architecture doesn't meet the requirements of the NLS solution. For simplex server architecture, SAP doesn't provide any features or procedures to run SAP IQ in a high-availability configuration.

+### Backup and restore for system deployed on Windows

+- **Full backup**: It makes a complete copy of the database.

+- **Incremental backup**: It copies all transactions since the last backup of any type.

+Depending on your SAP IQ database size, you can schedule your database backup from any of the backup scenarios. But if you're using SAP IQ with the NLS interface delivered by SAP, you might want to automate the backup process for an SAP IQ database. Automation ensures that the SAP IQ database can always be recovered to a consistent state without loss of data that's moved between the primary database and the SAP IQ database. For details on setting up automation for SAP IQ near-line storage, see [SAP note 2741824 - How to setup backup automation for SAP IQ Cold Store/Near-line Storage](https://launchpad.support.sap.com/#/notes/2741824).

+## [Deploy SAP IQ on Linux](#tab/sapiq-linux)

+### Linux server preparation and installation

+### High-availability deployment on Linux

+SAP IQ supports both a simplex and a multiplex architecture. For the NLS solution, only simplex server architecture is available and evaluated. Simplex is a single instance of an SAP IQ server running on a single virtual machine.

+### Backup and restore for system deployed on Linux

+- **Full backup**: It makes a complete copy of the database.

+- **Incremental backup**: It copies all transactions since the last backup of any type.

+Depending on your SAP IQ database size, you can schedule your database backup from any of the backup scenarios. But if you're using SAP IQ with the NLS interface delivered by SAP, you might want to automate the backup process for an SAP IQ database. Automation ensures that the SAP IQ database can always be recovered to a consistent state without loss of data that's moved between the primary database and the SAP IQ database. For details on setting up automation for SAP IQ near-line storage, see [SAP note 2741824 - How to setup backup automation for SAP IQ Cold Store/Near-line Storage](https://launchpad.support.sap.com/#/notes/2741824).

+This section explains the strategy to provide disaster recovery (DR) protection for the SAP IQ NLS solution. It complements the [Set up disaster recovery for SAP](../../site-recovery/site-recovery-sap.md) article, which represents the primary resources for an overall SAP DR approach. The process described in that article is presented at an abstract level. You need to validate the exact steps and thoroughly test your DR strategy.

+For SAP IQ, see [SAP note 2566083](https://launchpad.support.sap.com/#/notes/0002566083), which describes methods to implement a DR environment safely. In Azure, you can also use [Azure Site Recovery](../../site-recovery/site-recovery-overview.md) for an SAP IQ DR strategy. The strategy for SAP IQ DR depends on the way it's deployed in Azure, and it should also be in line with your SAP BW system.

+If you've installed SAP IQ as a standalone system that doesn't have any application-level redundancy or high availability, but the business requires a DR setup, all the disks (Azure-managed disks) attached to the virtual machine will be local.

+You can use [Azure Site Recovery](../../site-recovery/site-recovery-overview.md) to replicate a standalone SAP IQ virtual machine in the secondary region. It replicates the servers and all the attached managed disks to the secondary region so that if a disaster or an outage occurs, you can easily fail over to your replicated environment and continue working. To start replicating the SAP IQ VMs to the Azure DR region, follow the guidance in [Replicate a virtual machine to Azure](../../site-recovery/azure-to-azure-tutorial-enable-replication.md).

+- Whether a standalone SAP IQ instance will suffice for your business requirements.

+If you need a standalone SAP IQ instance on a DR site, you can use [Azure Site Recovery](../../site-recovery/site-recovery-overview.md) to replicate a primary SAP IQ virtual machine in the secondary region. It replicates the servers and all the local attached managed disks to the secondary region, but it won't replicate an Azure shared disk or a network drive like Azure NetApp Files.

+To achieve [high availability for SAP workload on Azure](sap-high-availability-guide-start.md), virtual machines are typically deployed in an [availability set](planning-guide.md#availability-sets), [availability zones](planning-guide.md#availability-zones) or in [flexible scale set](./virtual-machine-scale-set-sap-deployment-guide.md) to protect applications from infrastructure maintenance or failure within region. But the deployment doesnΓÇÖt protect applications from widespread disaster within region. So to protect applications from regional disaster, disaster recovery strategy for the applications should be in place. Disaster recovery is a documented and structured approach that is designed to assist an organization in executing the recovery processes in response to a disaster, and to protect or minimize IT services disruption and promote recovery.

+[](media/disaster-recovery/disaster-recovery-reference-architecture.png#lightbox)

+

+ > [!IMPORTANT]

+ > If SAP system is configured with flexible scale set with FD=1, then you need to use [PowerShell](../../site-recovery/azure-to-azure-powershell.md) to set up Azure Site Recovery for disaster recovery. Currently, it's the only method available to configure disaster recovery for VMs deployed in scale set.

+ > [!NOTE]

+Azure regions often are separated by large distances. Depending on the geopolitical region, the distance between Azure regions might be hundreds of miles, or even several thousand miles, like in the United States. Because of the distance, network traffic between assets that are deployed in two different Azure regions experience significant network roundtrip latency. The latency is significant enough to exclude synchronous data exchange between two SAP HANA instances under typical SAP workloads.

+Azure Virtual Network uses a different IP address range. The IP addresses are deployed in the second Azure region. So, you either need to change the SAP HANA client configuration, or preferably, you need to create steps to change the name resolution. This way, the clients are redirected to the new secondary site's server IP address. For more information, see the SAP article [Client connection recovery after takeover](https://help.sap.com/doc/6b94445c94ae495c83a19646e7c3fd56/2.0.02/en-US/c93a723ceedc45da9a66ff47672513d3.html).

+If you're using the scenario of sharing the DR target with a QA system in one VM, you need to take these considerations into account:

+- The memory requirement of logreplay operation mode without preload isn't deterministic and depends on the columnstore structures loaded. In extreme cases, you might require 50% of the memory of the primary instance. The memory for logreplay operation mode is independent on whether you chose to have the data preloaded set or not.

+

+A small change that you can make in the configuration might be to configure data as preloading. However, given the manual nature of failover and the fact that application layers also need to move to the second region, it might not make sense to preload data.

+## Combine availability within one region and across regions

+In these cases, you can set up what SAP calls an [SAP HANA multi-tier system replication configuration](https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.02/en-US/ca6f4c62c45b4c85a109c7faf62881fc.html) by using HANA system replication. The architecture would look like:

+

+SAP introduced [multi-target system replication](https://help.sap.com/viewer/42668af650f84f9384a3337bcd373692/2.0.03/en-US/0b2c70836865414a8c65463180d18fec.html) with HANA 2.0 SPS3. Multi-target system replication brings some advantages in update scenarios. For example, the DR site (Region 2) isn't impacted when the secondary HA site is down for maintenance or updates.

+

+

+> The operation modes between the different tiers need to be homogeneous. You **can't** use logreplay as operation mode between tier 1 and tier 2 and delta_datashipping to supply tier 3. You can only choose the one or the other operation mode that needs to be consistent for all tiers. Since delta_datashipping is not suitable to give you an RPO=0, the only reasonable operation mode for such a multi-tier configuration remains logreplay. For details about operation modes and some restrictions, see the SAP article [Operation modes for SAP HANA system replication](https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.02/en-US/627bd11e86c84ec2b9fcdf585d24011c.html).

+This article describes several availability scenarios for SAP HANA within one Azure region. Azure has many regions, spread throughout the world. For the list of Azure regions, see [Azure regions](https://azure.microsoft.com/regions/). For deploying SAP HANA on VMs within one Azure region, Microsoft offers deployment of a single VM with a HANA instance. For increased availability, you can deploy two VMs with two HANA instances using either a [flexible scale set](./virtual-machine-scale-set-sap-deployment-guide.md) with FD=1, [availability zones](./high-availability-zones.md) or an [availability set](../../virtual-machines/windows/tutorial-availability-sets.md) that uses HANA system replication for availability.

+Azure regions that provide Availability Zones consist of multiple data centers, each with its own power source, cooling, and network infrastructure. The purpose of offering different zones within a single Azure region is to enable the deployment of applications across two or three available Availability Zones. By distributing your application deployment across zones, any power or networking issues affecting a specific Azure Availability Zone infrastructure wouldn't fully disrupt your application's functionality within the Azure region. While there might be some reduced capacity, such as the potential loss of VMs in one zone, the VMs in the remaining zones would continue to operate without interruption. To set up two HANA instances in separate VMs spanning across different zones, you have the option to deploy VMs using either the [flexible scale set with FD=1](./virtual-machine-scale-set-sap-deployment-guide.md) or [availability zones](./high-availability-zones.md) deployment option.

+For increased availability within a region, it's advised to deploy two VMs with two HANA instances using an [availability set](../../virtual-machines/windows/tutorial-availability-sets.md). An Azure Availability Set is a logical grouping capability that ensures that the VM resources configured within Availability Set are failure-isolated from each other when they're deployed within an Azure datacenter. Azure ensures that the VMs you place within an Availability Set run across multiple physical servers, compute racks, storage units, and network switches. In some Azure documentation, this configuration is referred to as placements in different [update and fault domains](../../virtual-machines/availability.md). These placements usually are within an Azure datacenter. Assuming that power source and network issues would affect the datacenter that you're deploying, all your capacity in one Azure region would be affected.

+In a single-VM scenario, you create an Azure VM for the SAP HANA instance. You use Azure Premium Storage to host the operating system disk and all your data disks. The Azure uptime SLA of 99.9 percent and the SLAs of other Azure components is sufficient for you to fulfill your availability SLAs for your customers. In this scenario, you have no need to use an Azure Availability Set for VMs that run the DBMS layer. In this scenario, you rely on two different features:

+- Azure VM auto restart (also referred to as Azure service healing)

+- SAP HANA auto restart

+- If the host isn't in a healthy state after successful reboot, a redeployment of the VMs that were originally on the now unhealthy node onto a healthy host server is initiated. In this case, the original host is marked as not healthy. It won't be used for further deployments until it's cleared or replaced.

+- If the unhealthy host has problems during the reboot process, an immediate restart of the VMs on a healthy host is triggered.

+With the host and VM monitoring provided by Azure, Azure VMs that experience host issues are automatically restarted on a healthy Azure host.

+> [!IMPORTANT]

+> Azure service healing will not restart Linux VMs where the guest OS is in a kernel panic state. The default settings of the commonly used Linux releases, are not automatically restarting VMs or server where the Linux kernel is in panic state. Instead the default foresees to keep the OS in kernel panic state to be able to attach a kernel debugger to analyze. Azure is honoring that behavior by not automatically restarting a VM with the guest OS in such a state. Assumption is that such occurrences are extremely rare. You could overwrite the default behavior to enable a restart of the VM. To change the default behavior enable the parameter 'kernel.panic' in /etc/sysctl.conf. The time you set for this parameter is in seconds. Common recommended values are to wait for 20-30 seconds before triggering the reboot through this parameter. For more information, see [sysctl.conf](https://gitlab.com/procps-ng/procps/blob/master/sysctl.conf).

+The second feature that you rely on in this scenario is the fact that the HANA service that runs in a restarted VM starts automatically after the VM reboots. You can set up [HANA service auto restart](https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.01/en-US/cf10efba8bea4e81b1dc1907ecc652d3.html) through the watchdog services of the various HANA services.

+You might improve this single-VM scenario by adding a cold failover node to an SAP HANA configuration. In the SAP HANA documentation, this setup is called [host autofailover](https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.01/en-US/ae60cab98173431c97e8724856641207.html). This configuration might make sense in an on-premises deployment situation where the server hardware is limited, and you dedicate a single-server node as the host autofailover node for a set of production hosts. But in Azure, where the underlying infrastructure of Azure provides a healthy target server for a successful VM restart, it doesn't make sense to deploy SAP HANA host autofailover. Because of Azure service healing, there's no reference architecture that foresees a standby node for HANA host autofailover.

+High availability architectures based on standby node or HANA System Replication can be found in following documents. In cases where standby nodes or HANA system replication high availability isn't used in SAP HANA scale-out configurations, you can depend on Azure VMs' service healing capabilities and the automatic restart of the SAP HANA instance once the VM is operational again.

+- RedHat Enterprise Linux

+ - [High availability of SAP HANA scale-out system with HSR on RHEL](./sap-hana-high-availability-scale-out-hsr-rhel.md).

+ - [Deploy a SAP HANA scale-out system with standby node on Azure VMs by using Azure NetApp Files on RHEL](./sap-hana-scale-out-standby-netapp-files-rhel.md).

+- SUSE Linux Enterprise Server

+ - [High availability of SAP HANA scale-out system with HSR on SLES](./sap-hana-high-availability-scale-out-hsr-suse.md).

+ - [Deploy a SAP HANA scale-out system with standby node on Azure VMs by using Azure NetApp Files on SLES](./sap-hana-scale-out-standby-netapp-files-suse.md).

+To ensure the availability of the HANA system within a specific region, you have the option to configure two VMs across the availability zones of the region or within the region. To achieve this objective, you can configure the VMs using flexible scale set, availability zones or availability set deployment option. The base setup in Azure would look like:

+

+To illustrate the different SAP HANA availability scenarios, a few of the layers in the diagram are omitted. The diagram shows only layers that depict VMs, hosts, Availability Sets, and Azure regions. Azure Virtual Network instances, resource groups, and subscriptions don't play a role in the scenarios described in this section.

+One of the most rudimentary setups is to use backups. In particular, you might have transaction log backups shipped from one VM to another Azure VM. You can choose the Azure Storage type. In this setup, you're responsible for scripting the copy of scheduled backups that are conducted on the first VM to the second VM. If you need to use the second VM instances, you must restore the full, incremental/differential, and transaction log backups to the point that you need.

+

+This setup isn't well suited to achieving great Recovery Point Objective (RPO) and Recovery Time Objective (RTO) times. RTO times especially would suffer due to the need to fully restore the complete database by using the copied backups. However, this setup is useful for recovering from unintended data deletion on the main instances. With this setup, at any time, you can restore to a certain point in time, extract the data, and import the deleted data into your main instance. Hence, it might make sense to use a backup copy method in combination with other high-availability functionality.