+* [Manage your Azure AD B2C tenant](tenant-management-manage-administrator.md)

+- **Guest account** - A guest account can only be a Microsoft account or an Azure AD user that can be used to share administration responsibilities such as [managing a tenant](tenant-management-manage-administrator.md).

+ >You can configure the NPS Server to support PAP. If PAP is not an option, you can set OVERRIDE_NUMBER_MATCHING_WITH_OTP = FALSE to fall back to Approve/Deny push notifications.

+ // No token found in the cache or Azure AD insists that a form interactive auth is required (e.g. the tenant admin turned on MFA)

+ logging.info("No suitable token exists in cache. Let's get a new one from Azure AD.")

+Here's an example of the security context for the self-hosted gateway container:

+| &#9679; Windows Server 2022 (including Server Core) <br> &#9679; Windows Server 2019 (including Server Core) <br> &#9679; Windows Server 2016, version 1709 and 1803 (excluding Server Core) <br> &#9679; Windows Server 2012, 2012 R2 <br> &#9679; Windows 10 Enterprise (including multi-session) and Pro | &#9679; Debian GNU/Linux 10 and 11 <br> &#9679; Ubuntu 22.04 LTS <br> &#9679; SUSE Linux Enterprise Server 15.2, and 15.3 <br> &#9679; Red Hat Enterprise Linux Server 7 and 8ΓÇ»|

+

+| &#9679; Windows Server 2022 (including Server Core) <br> &#9679; Windows Server 2019 (including Server Core) <br> &#9679; Windows Server 2016, version 1709 and 1803 (excluding Server Core) <br> &#9679; Windows Server 2012, 2012 R2 <br> &#9679; Windows 10 Enterprise (including multi-session) and Pro| &#9679; Debian GNU/Linux 10 and 11 <br> &#9679; Ubuntu 22.04 LTS <br> &#9679; SUSE Linux Enterprise Server 15.2, and 15.3 <br> &#9679; Red Hat Enterprise Linux Server 7 and 8ΓÇ»|

+| Azure SignalR Service (Premium tier) | [Automatically scale units of an Azure SignalR service](https://learn.microsoft.com/azure/azure-signalr/signalr-howto-scale-autoscale) |

+| Azure Monitor Logs | Send resource logs to Azure Monitor Logs for analysis with other collected log data. | [Collect Azure resource logs in Log Analytics workspace in Azure Monitor](essentials/resource-logs.md#send-to-log-analytics-workspace) |

+| Storage | Send resource logs to Azure Storage for archiving. | [Archive Azure resource logs](essentials/resource-logs.md#send-to-azure-storage) |

+Run the following command to verify that the daemon set was deployed properly:

+ | [ASCAuditLogs](/azure/azure-monitor/reference/tables/ASCAuditLogs) | Azure Sphere audit logs generated by Azure Sphere service and devices. |

+description: Use JSONPath in Azure Monitor workbooks to transform the JSON data results to a different data format.

+Workbooks can query data from many sources. Some endpoints, such as [Azure Resource Manager](../../azure-resource-manager/management/overview.md) or custom endpoints can return results in JSON. If the JSON data returned by the queried endpoint is in a format that you don't want, you can use JSONPath transformation to convert the JSON to a table structure. You can then use the table to plot [workbook visualizations](./workbooks-overview.md#visualizations).

+JSONPath is a query language for JSON that's similar to XPath for XML. Like XPath, JSONPath allows for the extraction and filtration of data out of the JSON structure.

+In this example, the JSON object represents a store's inventory. We're going to create a table of the store's available books listing their titles, authors, and prices.

+ Column IDs are the column headers. Column JSON paths fields represent the path from the root of the table to the column value.

+1. Select **Run Query**.

+## Use regular expressions to covert values

+You may have some data that isn't in a standard format. To use that data effectively, you would want to convert that data into a standard format.

+In this example, the published date is in YYYMMMDD format. The code interprets this value as a numeric value, not text, resulting in right-justified numbers, instead of as a date.

+You can use the **Type**, **RegEx Match** and **Replace With** fields in the result settings to convert the result into true dates.

+|Result setting field |Description |

+|||

+|Type|Allows you to explicitly change the type of the value returned by the API. This field us usually left unset, but you can use this field to force the value to a different type. |

+|Regex Match|Allows you to enter a regular expression to take part (or parts) of the value returned by the API instead of the whole value. This field is usually combined with the **Replace With** field. |

+|Replace With|Use this field to create the new value along with the regular expression. If this value is empty, the default is `$&`, which is the match result of the expression. See string.replace documentation to see other values that you can use to generate other output.|

+To convert YYYMMDD format into YYYY-MM-DD format:

+1. Select the Published row in the grid.

+1. In the **Type** field, select Date/Time so that the column is usable in charts.

+1. In the **Regex Match** field, use this regular expression: `([0-9]{4})([0-9]{2})([0-9]{2})`. This regular expression:

+ - matches a four digit number, then a two digit number, then another two digit number.

+ - The parentheses form capture groups to use in the next step.

+ 1.In the **Replace With**, use this regular expression: `$1-$2-$3. This expression creates a new string with each captured group, with a hyphen between them, turning "12345678" into "1234-56-78").

+1. Run the query again.

+ 

+> [!WARNING]

+> Please be careful while using non-English language in your tags. It can cause decoding progress failure while loading your VM's metadata from IMDS (Instance Metadata Service).

+ In this scenario, `OnConnectedAsync` and `OnDisconnectedAsync` will be triggered on the new server and the old server respectively with an `IConnectionMigrationFeature` set in the `Context`, which can be used to identify if the client connection was being migrated-in or migrated-out. It could be useful especially for stateful scenarios.

+ eventData.Payload[2] as uint? == ReasonForStarvation)

+Azure Web PubSub service works as a data processor service and doesn't store any customer data. Azure Web PubSub service processes customer data within the region the customer deploys the service instance in. If you use Azure Web PubSub service together with other Azure services, like Azure Storage for diagnostics, see [this white paper](https://azure.microsoft.com/resources/achieving-compliant-data-residency-and-security-with-azure/) for guidance about how to keep data residency in Azure regions.

+## Recover a database from .bak file using SSMS

+You can use *Restore as Files* operation to restore the database files in `.bak` format while restoring from the Azure portal. [Learn more](restore-sql-database-azure-vm.md#restore-as-files).

+When the restoration of the `.bak` file to the Azure virtual machine is complete, you can trigger restore using **TSQL commands** through SSMS.

+ΓÇ»

+To restore the database files to the *original path on the source server*, remove the `MOVE` clause from the TSQL restore query.

+ΓÇ»

+**Example**

+ ```azurecli-interactive

+ USE [master]

+ RESTORE DATABASE [<DBName>] FROM  DISK = N'<.bak file path>'

+ ```

+>[!Note]

+>You shouldn’t have the same database files on the target server (restore with replace).  Also, you can [enable instant file initialization on the target server to reduce the file initialization time overhead]( /sql/relational-databases/databases/database-instant-file-initialization?view=sql-server-ver16).

+To relocate the database files from the target restore server, you can frame a TSQL command using the `MOVE` clauses.

+ ```azurecli

+ USE [master]

+ RESTORE DATABASE [<DBName>] FROM  DISK = N'<.bak file path>'  MOVE N'<LogicalName1>' TO N'<TargetFilePath1OnDisk>',  MOVE N'<LogicalName2>' TO N'<TargetFilePath2OnDisk>' GO

+ ```

+**Example**

+ ```azurecli

+ USE [master]

+ RESTORE DATABASE [test] FROM  DISK = N'J:\dbBackupFiles\test.bak' WITH  FILE = 1,  MOVE N'test' TO N'F:\data\test.mdf',  MOVE N'test_log' TO N'G:\log\test_log.ldf',  NOUNLOAD,  STATS = 5

+ GO

+ ```

+If there are more than two files for the database, you can add additional `MOVE` clauses to the restore query. You can also use SSMS for database recovery using `.bak` files. [Learn more](/sql/relational-databases/backup-restore/restore-a-database-backup-using-ssms?view=sql-server-ver16).

+>[!Note]

+>For large database recovery, we recommend you to use TSQL statements. If you want to relocate the specific database files, see the list of database files in the JSON format created during the **Restore as Files** operation.

+At the Recovery Services vault, you can enable Cross Region Restore. Learn about [how to turn on Cross Region Restore](./backup-create-rs-vault.md#set-cross-region-restore).

+ Title: Conceptual whitepapers

+description: This list of conceptual whitepapers describes various Azure Cosmos DB service, development, and data concepts in depth.

+# Azure Cosmos DB conceptual whitepapers

+| [Schema-Agnostic Indexing with Azure Cosmos DB](https://www.vldb.org/pvldb/vol8/p1668-shukla.pdf) | This paper describes Azure Cosmos DB's indexing subsystem. This paper includes a deep dive into various Azure Cosmos DB capabilities. These capabilities include; document representation, query language, document indexing approach, core index support, and early production experiences. |

+| [Azure Cosmos DB and personal data](https://servicetrust.microsoft.com/ViewPage/TrustDocuments?command=Download&downloadType=Document&downloadId=87cc6456-4b23-473c-94d3-6c713b8b8956&docTab=6d000410-c9e9-11e7-9a91-892aae8839ad_FAQ_and_White_Papers) | This paper provides guidance for Azure Cosmos DB customers managing a cloud-based database, an on-premises database, or both. This paper is ideal for customers who want to learn more about the rules related to how we handle and protect their personal data in their database systems. |

+| [Next-gen app development with Azure Cosmos DB](https://azure.microsoft.com/resources/microsoft-azure-cosmos-db-flexible-reliable-cloud-nosql-at-any-scale/) | This paper explores how we uniquely positioned Azure Cosmos DB to address the data requirements of modern apps. It includes three customers spotlights highlighting the API for MongoDB, simplified data management, cost-effective scalability, market-leading performance, and reliability. |

+| [Cloud-Scale Data for Spring Developers](https://azure.github.io/cloud-scale-data-for-devs-guide/) | This paper helps you build cloud-native Java applications in Azure. This paper helps you gain insights about using NoSQL and why you should consider Azure Cosmos DB, our fully managed, distributed NoSQL database service on Azure. |

+## Next steps

+- Learn about [common Azure Cosmos DB use cases](use-cases.md)

+Budgets in Cost Management help you plan for and drive organizational accountability. They help you proactively inform others about their spending to manage costs and monitor how spending progresses over time.

+You can configure alerts based on your actual cost or forecasted cost to ensure that your spending is within your organizational spending limit. Notifications are triggered when the budget thresholds you've created are exceeded. None of your resources is affected, and your consumption isn't stopped. You can use budgets to compare and track spending as you analyze costs.

+See [Limitations](../active-directory/managed-identities-azure-resources/managed-identities-faq.md#limitations) of managed identities, which also apply to managed identities in Azure Data Factory.

+| Prerequisites | - **Foundational CSPM capabilities** - None <br> <br> - **Defender Cloud Security Posture Management (CSPM)** - Agentless scanning requires the **Subscription Owner** to enable the plan. Anyone with a lower level of authorization can enable the Defender CSPM plan but the agentless scanner won't be enabled by default due to lack of permissions. Attack path analysis and security explorer won't populate with vulnerabilities because the agentless scanner is disabled. |

+Defender for cloud offers foundational multicloud CSPM capabilities for free. These capabilities are automatically enabled by default on any subscription or account that has onboarded to Defender for Cloud. The foundational CSPM includes asset discovery, continuous assessment and security recommendations for posture hardening, compliance with Microsoft Cloud Security Benchmark (MCSB), and a [Secure score](secure-score-access-and-track.md) which measure the current status of your organizationΓÇÖs posture.

+The optional Defender CSPM plan, provides advanced posture management capabilities such as [Attack path analysis](#attack-path-analysis), [Cloud security explorer](#cloud-security-explorer), advanced threat hunting, [security governance capabilities](#security-governance-and-regulatory-compliance), and also tools to assess your [security compliance](#security-governance-and-regulatory-compliance) with a wide range of benchmarks, regulatory standards, and any custom security policies required in your organization, industry, or region.

+The following table summarizes each plan and their cloud availability.

+Security governance and regulatory compliance refer to the policies and processes which organizations have in place. These policies ensure that they comply with laws, rules and regulations put in place by external bodies (government) which control activity in a given jurisdiction. Defender for Cloud allows you to view your regulatory compliance through the regulatory compliance dashboard.

+The cloud security graph is a graph-based context engine that exists within Defender for Cloud. The cloud security graph collects data from your multicloud environment and other data sources. For example, the cloud assets inventory, connections and lateral movement possibilities between resources, exposure to internet, permissions, network connections, vulnerabilities and more. The data collected builds a graph representing your multicloud environment.

+Defender for Cloud's contextual security capabilities assist security teams in reducing the risk of impactful breaches. Defender for Cloud uses environmental context to perform a risk assessment of your security issues, identifies the biggest security risks, and distinguishes them from less risky issues.

+Use the cloud security explorer, to proactively identify security risks in your cloud environment by running graph-based queries on the cloud security graph, which is Defender for Cloud's context engine. You can prioritize your security team's concerns, while taking your organization's specific context and conventions into account.

+With the cloud security explorer, you can query all of your security issues and environment context such as assets inventory, exposure to internet, permissions, and lateral movement between resources and across multiple clouds (Azure and AWS).

+Learn more about [the cloud security graph, attack path analysis, and the cloud security explorer](concept-attack-path.md).

+## Prerequisites

+- You must [enable agentless scanning](enable-vulnerability-assessment-agentless.md).

+- You must [enable Defender for CSPM](enable-enhanced-security.md).

+- You must [enable Defender for Containers](defender-for-containers-enable.md), and install the relevant agents in order to view attack paths that are related to containers.

+ When you enable Defender for Containers, you also gain the ability to [query](how-to-manage-cloud-security-explorer.md#build-a-query-with-the-cloud-security-explorer) containers data plane workloads in the security explorer.

+- Required roles and permissions:

+ - Security Reader

+ - Security Admin

+ - Reader

+ - Contributor

+ - Owner

+Check the [cloud availability tables](supported-machines-endpoint-solutions-clouds-servers.md) to see which government and cloud environments are supported.

+The cloud security explorer allows you to build queries that can proactively hunt for security risks in your environments with dynamic and efficient features such as:

+- **Multi-cloud and multi-resource queries** - The entity selection control filters are grouped and combined into logical control categories to assist you in building queries across cloud environments and across resources simultaneously.

+- **Custom Search** - Use the dropdown menus to apply filters to build your query.

+- **Query templates** - Use any of the available pre-built query templates to more efficiently build your query.

+- **Share query link** - Copy and share a link of your query with other people.

+ :::image type="content" source="media/concept-cloud-map/cloud-security-explorer-main-page.png" alt-text="Screenshot of the cloud security explorer page." lightbox="media/concept-cloud-map/cloud-security-explorer-main-page.png":::

+1. Search for and select a resource from the drop-down menu.

+ :::image type="content" source="media/how-to-manage-cloud-security/cloud-security-explorer-select-resource.png" alt-text="Screenshot of the resource drop-down menu." lightbox="media/how-to-manage-cloud-security/cloud-security-explorer-select-resource.png":::

+1. Select **+** to add other filters to your query.

+

+ :::image type="content" source="media/how-to-manage-cloud-security/cloud-security-explorer-query-search.png" alt-text="Screenshot that shows a full query and where to select on the screen to perform the search." lightbox="media/how-to-manage-cloud-security/cloud-security-explorer-query-search.png":::

+1. Add subfilters as needed.

+1. After building your query, select **Search** to run the query.

+ :::image type="content" source="media/how-to-manage-cloud-security/cloud-security-explorer-query-search-populated.png" alt-text="Screenshot that shows where to select search to run the query and results populated." lightbox="media/how-to-manage-cloud-security/cloud-security-explorer-query-search-populated.png":::

+Query templates are pre-formatted searches using commonly used filters. Use one of the existing query templates from the bottom of the page by selecting **Open query**.

+You can modify any template to search for specific results by changing the query and selecting **Search**.

+## Share a query

+Use the query link to share a query with other people. After creating a query, select **Share query link**. The link is copied to your clipboard.

+View the [reference list of attack paths and cloud security graph components](attack-path-reference.md).

+Learn about the [Defender CSPM plan options](concept-cloud-security-posture-management.md#defender-cspm-plan-options).

+- [Enhanced Cloud Security Explorer](#enhanced-cloud-security-explorer)

+- [The built-in policy [Preview]: Private endpoint should be configured for Key Vault has been deprecated](#the-built-in-policy-preview-private-endpoint-should-be-configured-for-key-vault-has-been-deprecated)

+### Enhanced Cloud Security Explorer

+An improved version of the cloud security explorer includes a refreshed user experience that removes query friction dramatically, added the ability to run multicloud and multi-resource queries, and embedded documentation for each query option.

+The Cloud Security Explorer now allows you to run cloud-abstract queries across resources. You can use either the pre-built query templates or use the custom search to apply filters to build your query. Learn [how to manage Cloud Security Explorer](how-to-manage-cloud-security-explorer.md).

+### The built-in policy \[Preview]: Private endpoint should be configured for Key Vault has been deprecated

+The built-in policy [`[Preview]: Private endpoint should be configured for Key Vault`](https://ms.portal.azure.com/#view/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2F5f0bc445-3935-4915-9981-011aa2b46147) has been deprecated and has been replaced with the [`[Preview]: Azure Key Vaults should use private link`](https://ms.portal.azure.com/#view/Microsoft_Azure_Policy/PolicyDetailBlade/definitionId/%2Fproviders%2FMicrosoft.Authorization%2FpolicyDefinitions%2Fa6abeaec-4d90-4a02-805f-6b26c4d3fbe9) policy.

+Learn more about [integrating Azure Key Vault with Azure Policy](../key-vault/general/azure-policy.md#network-access).

+ - `description`: (Optional) The explanation of what the parameter is used for. Can be used to provide

+4. Add the new property, `mail.smtp.ssl.protocols` with a value of `TLSv1.2`.

+The following example creates an HSM named **ContosoMHSM2**, in the resource group **ContosoResourceGroup**, residing in the **West US 3** location, with **the current signed in user** as the only administrator.

+az keyvault security-domain upload --hsm-name ContosoMHSM2 --sd-exchange-key ContosoMHSM2-SDE.cer --sd-file ContosoMHSM-SD.json --sd-wrapping-keys cert_0.key cert_1.key

+You do not need to assign the reservation to specific Azure Database for MariaDB servers. An already running Azure Database for MariaDB or ones that are newly deployed, will automatically get the benefit of reserved pricing. By purchasing a reservation, you are pre-paying for the compute costs for a period of one or three years. As soon as you buy a reservation, the Azure Database for MariaDB compute charges that match the reservation attributes are no longer charged at the pay-as-you go rates. A reservation does not cover software, networking, or storage charges associated with the MariaDB Database server. At the end of the reservation term, the billing benefit expires, and the Azure Database for MariaDB are billed at the pay-as-you go price. Reservations do not auto-renew. For pricing information, see the [Azure Database for MariaDB reserved capacity offering](https://azure.microsoft.com/pricing/details/mariadb/). </br>

+The vCore reservation discount is applied automatically to the number of Azure Database for MariaDB servers that match the Azure Database for MariaDB reserved capacity reservation scope and attributes. You can update the scope of the Azure Database for MariaDB reserved capacity reservation through Azure portal, PowerShell, CLI or through the API. </br></br>

+After you grant the permissions to the UMI, they're enabled for all servers created with the UMI assigned as a server identity.

+### Grant permissions to User assigned managed identity

+The following sample PowerShell script grants the necessary permissions for a UMI. This sample assigns permissions to the UMI `umiservertest`.

+To run the script, you must sign in as a user with a Global Administrator or Privileged Role Administrator role.

+The script grants the `User.Read.All`, `GroupMember.Read.All`, and `Application.Read.ALL` permissions to a UMI to access [Microsoft Graph](/graph/auth/auth-concepts#microsoft-graph-permissions).

+```powershell

+# Script to assign permissions to the UMI "umiservertest"

+import-module AzureAD

+$tenantId = '<tenantId>' # Your Azure AD tenant ID

+Connect-AzureAD -TenantID $tenantId

+# Log in as a user with a "Global Administrator" or "Privileged Role Administrator" role

+# Script to assign permissions to an existing UMI

+# The following Microsoft Graph permissions are required:

+# User.Read.All

+# GroupMember.Read.All

+# Application.Read.ALL

+# Search for Microsoft Graph

+$AAD_SP = Get-AzureADServicePrincipal -SearchString "Microsoft Graph";

+$AAD_SP

+# Use Microsoft Graph; in this example, this is the first element $AAD_SP[0]

+#Output

+#ObjectId AppId DisplayName

+#-- -- --

+#47d73278-e43c-4cc2-a606-c500b66883ef 00000003-0000-0000-c000-000000000000 Microsoft Graph

+#44e2d3f6-97c3-4bc7-9ccd-e26746638b6d 0bf30f3b-4a52-48df-9a82-234910c4a086 Microsoft Graph #Change

+$MSIName = "<managedIdentity>"; # Name of your user-assigned

+$MSI = Get-AzureADServicePrincipal -SearchString $MSIName

+if($MSI.Count -gt 1)

+{

+Write-Output "More than 1 principal found, please find your principal and copy the right object ID. Now use the syntax $MSI = Get-AzureADServicePrincipal -ObjectId <your_object_id>"

+# Choose the right UMI

+Exit

+}

+# If you have more UMIs with similar names, you have to use the proper $MSI[ ]array number

+# Assign the app roles

+$AAD_AppRole = $AAD_SP.AppRoles | Where-Object {$_.Value -eq "User.Read.All"}

+New-AzureADServiceAppRoleAssignment -ObjectId $MSI.ObjectId -PrincipalId $MSI.ObjectId -ResourceId $AAD_SP.ObjectId[0] -Id $AAD_AppRole.Id

+$AAD_AppRole = $AAD_SP.AppRoles | Where-Object {$_.Value -eq "GroupMember.Read.All"}

+New-AzureADServiceAppRoleAssignment -ObjectId $MSI.ObjectId -PrincipalId $MSI.ObjectId -ResourceId $AAD_SP.ObjectId[0] -Id $AAD_AppRole.Id

+$AAD_AppRole = $AAD_SP.AppRoles | Where-Object {$_.Value -eq "Application.Read.All"}

+New-AzureADServiceAppRoleAssignment -ObjectId $MSI.ObjectId -PrincipalId $MSI.ObjectId -ResourceId $AAD_SP.ObjectId[0] -Id $AAD_AppRole.Id

+```

+In the final steps of the script, if you have more UMIs with similar names, you have to use the proper `$MSI[ ]array` number. An example is `$AAD_SP.ObjectId[0]`.

+### Check permissions for user-assigned managed identity

+To check permissions for a UMI, go to the [Azure portal](https://portal.azure.com). In the **Azure Active Directory** resource, go to **Enterprise applications**. Select **All Applications** for **Application type**, and search for the UMI that was created.

+Select the UMI, and go to the **Permissions** settings under **Security**.

+After you grant the permissions to the UMI, they're enabled for all servers created with the UMI assigned as a server identity.

+description: Concepts about migrating your Single server to Azure Database for PostgreSQL Flexible server.

+Azure Database for PostgreSQL powered by the PostgreSQL community edition is available in two deployment modes:

+* Flexible Server

+* Single Server

+Flexible Server is the next generation managed PostgreSQL service in Azure that provides maximum flexibility over your database, built-in cost-optimizations, and offers several improvements over Single Server. We recommend new customers to get started with flexible server and highly recommend existing single server customers to migrate to Flexible server to have the best experience of running PostgreSQL workloads on Azure.

+In this article, we provide compelling reasons for single server customers to migrate to flexible server and a walk-through of tackling this migration in a simple, efficient and a hassle-free way.

+## Why should you choose Flexible Server?

+* **[Superior performance](../flexible-server/overview.md)** - Flexible server runs on Linux VM that is best suited to run PostgreSQL engine as compared to Windows environment, which is the case with Single Server.

+* **[Cost Savings](../flexible-server/how-to-deploy-on-azure-free-account.md)** ΓÇô Flexible server allows you to stop and start server on-demand to lower your TCO. Your compute tier billing is stopped immediately which allows you to have significant cost savings during development, testing and for time-bound predictable production workloads.

+* **[Support for new PG versions](../flexible-server/concepts-supported-versions.md)** - Flexible server currently supports PG version 11 and onwards till version 14. Newer community versions of PostgreSQL will only be supported in flexible server.

+* **Minimized Latency** ΓÇô You can collocate your flexible server in the same availability zone as the application server that results in a minimal latency. This option isn't available in Single server.

+

+* **[Connection Pooling](../flexible-server/concepts-pgbouncer.md)** - Flexible server has a built-in connection pooling mechanism using **pgBouncer** that can support thousands of active connections with low overhead.

+* **[Server Parameters](../flexible-server/concepts-server-parameters.md)** - Flexible server offers a richer set of server parameters when compared to Single server for server configuration and tuning.

+* **[Custom Maintenance Window](../flexible-server/concepts-maintenance.md)** - You can schedule the maintenance window of flexible server for a specific day and time of the week. Single server doesn't have this flexibility.

+* **[High Availability](../flexible-server/concepts-high-availability.md)** - Flexible server supports HA within same availability zone and across availability zones by configuring a warm standby server that is in sync with the primary.

+* **[Security](../flexible-server/concepts-security.md)** - Flexible server offers multiple layers of information protection and encryption to protect your data.

+A feature-set comparison between Single server and Flexible server is available [here](../flexible-server/concepts-compare-single-server-flexible-server.md).

+All the learnings acquired by running customer workloads on Azure database for PostgreSQL - Single Server is incorporated into Flexible server and we recommend using Flexible server for all your new PostgreSQL deployments on Azure.

+

+## How to migrate from Single Server to Flexible Server?

+Let us first look at the methods you can consider performing the migration from Single server to Flexible server.

+**Offline Migration** ΓÇô In an offline migration, all applications connecting to your single server are stopped and the database(s) is copied to flexible server.

+**Online Migration** - In an online migration, applications connecting to your single server aren't stopped while database(s) are copied to flexible server. The initial copy of the databases is followed by replication to keep flexible server in sync with the single server. A cutover is performed when the flexible server is in complete sync with the single server resulting in minimal downtime.

+The following table gives an overview of Offline vs Online migration.

+| Mode | Pros | Cons |

+|:|:|:--|

+| Offline | - Simple, easy and less complex to execute.<br> - Very fewer chances of failure.<br> - No restrictions in terms of database objects it can handle | Downtime to applications. |

+| Online | - Very minimal downtime to application.<br> - Ideal for large databases and for customers having limited downtime requirements.<br>| - Replication used in online migration has multiple restrictions listed in this [doc](https://www.postgresql.org/docs/current/logical-replication-restrictions.html) (e.g Primary Keys needed in all tables) <br> - Tough and much complex to execute than offline migration.ΓÇ»<br> - Greater chances of failure due to complexity of migration.ΓÇ»<br> There is an impact on the source serverΓÇÖs storage and compute if the migration runs for a long time. The impact needs to be monitored closely during migration. |

+>[!IMPORTANT]

+> Offline migration is the recommended way to perform migrations from single server to flexible server. Customers should consider online migrations only if their downtime requirements are not met.ΓÇ»

+The following table lists the different tools available for performing the migration from single server to flexible server.

+| Tool | Mode | Pros | Cons |

+|:|:|:--|:|

+| Single to Flex Migration tool (**Recommended**) | Offline | - Managed migration service.<br> - No complex setup/pre-requisites required<br> - Simple to use portal-based migration experience<br> - Fast offline migration tool<br> - No limitations in terms of size of databases it can handle. | Downtime to applications.|

+| pg_dump and pg_restore | Offline | - Tried and tested tool that has been in use for long time<br> - Suited for databases of size less than 10 GB<br> | - Need prior knowledge of setting up and using this tool<br> - Slow when compared to other tools <br> Significant downtime to your application.|

+| Azure DMS | Online | - Minimal downtime to your application<br> - Free of cost | - Complex setup<br> - High chances of migration failures<br> - Can't handle database of sizes > 1 TB<br> - Can't handle write-intensive workload|

+The next section of the document gives an overview of the Single to Flex Migration tool, its implementation, limitations, and the experience that makes it the recommended tool to perform migrations from single to flexible server.

+>[!NOTE]

+> The Single to Flex Migration tool currently supports only **Offline** migrations. Support for online migrations will be introduced later in the tool.

+## Single to Flexible Migration tool - Overview

+The single to flex migration tool is a hosted solution where we spin up a purpose-built docker container in the target Flexible server VM and drive the incoming migrations. This docker container spins up on-demand when a migration is initiated from a single server and gets decommissioned once the migration is completed. The migration container uses a new binary called [pgcopydb](https://github.com/dimitri/pgcopydb) that provides a fast and efficient way of copying databases from one server to another. Though pgcopydb uses the traditional pg_dump and pg_restore for schema migration, it implements its own data migration mechanism that involves multi-process streaming parts from source to target. Also, pgcopydb bypasses pg_restore way of index building and drives that internally in a way that all indexes are built concurrently. So, the data migration process is quicker with pgcopydb. Following is the process diagram of the new version of the migration tool.

+The following table shows the time for performing offline migrations for databases of various sizes using the single to flex migration tool. The migration was performed using a flexible server with the SKU ΓÇô **Standard_D4ds_v4(4 cores, 16GB Memory, 128GB disk and 500 iops)**

+| 5 GB | 00:03 |

+| 10 GB | 00:08 |

+| 50 GB | 00:35 |

+| 100 GB | 01:00 |

+| 500 GB | 04:00 |

+| 1,000 GB | 07:00 |

+> The above numbers give you an approximation of the time taken to complete the migration. To get a precise value for migrating your server, we strongly recommend taking a **PITR (point in time restore)** of your single server and running it against the single to flex migration tool.

+>[!IMPORTANT]

+> In order to perform faster migrations, pick a higher SKU for your flexible server. You can always change the SKU to match the application needs post migration.

+## Limitations

+* You can have only one active migration to your flexible server.

+* You can select a max of eight databases in one migration attempt. If you've more than eight databases, you must wait for the first migration to be complete before initiating another migration for the rest of the databases. Support for migration of more than eight databases in a single migration will be introduced later.

+* The source and target server must be in the same Azure region. Cross region migrations are not supported.

+* The tool takes care of the migration of data and schema. It doesn't migrate managed service features such as server parameters, connection security details, firewall rules, users, roles and permissions. In the later part of the document, we point you to docs that can help you perform the migration of users, roles and firewall rules from single server to flexible server.

+* The migration tool shows the number of tables copied from source to target server. You need to validate the data in target server post migration.

+* The tool only migrates user databases and not system databases like template_0, template_1, azure_sys and azure_maintenance. ΓÇ»

+## Experience

+Get started with the Single to Flex migration tool by using any of the following methods:

+* [Migrate using the Azure portal](../migrate/how-to-migrate-single-to-flexible-portal.md)

+* [Migrate using the Azure CLI](../migrate/how-to-migrate-single-to-flexible-cli.md)

+## Best practices

+Here, we go through the phases of an overall database migration journey, with guidance on how to use Single to Flex migration tool in the process.

+### Pre migration

+#### Application compatibility

+Single server supports PG version 9.6,10 and 11 while Flexible server supports PG version 11, 12, 13 and 14. Given the differences in supported versions, you might be moving across versions while migrating from single to flexible server. If that is the case, make sure your application works well with the version of flexible server you're trying to migrate to. If there are breaking changes, make sure to fix them on your application before migrating to flexible server. Use this [link](https://www.postgresql.org/docs/14/appendix-obsolete.html) to check for any breaking changes while migrating to the target version.

+#### Database migration planning

+The most important thing to consider for performing offline migration using the single to flex migration tool is the downtime incurred by the application.

+##### How to calculate the downtime?  

+In most cases, the non-prod servers (dev, UAT, test, staging) are migrated using offline migrations. Since these servers have less data than the production servers, the migration completes fast. For migration of production server, you need to know the time it would take to complete the migration to plan for it in advance.

+The time taken for an offline migration to complete is dependent on several factors that includes the number of databases, size of databases, number of tables inside each database, number of indexes, and the distribution of data across tables. It also depends on the SKU of the source and target server, and the IOPS available on the source and target server. Given the many factors that can affect the migration time, it's hard to estimate the total time for the offline migration to complete. The best approach would be to try it on a server restored from the primary server.

+For calculating the total downtime to perform offline migration of production server, the following phases are considered.

+* **Migration of PITR** - The best way to get a good estimate on the time taken to migrate your production database server would be to take a point-in time restore of your production server and run the offline migration on this newly restored server.

+* **Migration of Buffer** - After completing the above step, you can plan for actual production migration during a time period when the application traffic is low. This migration can be planned on the same day or probably a week away. By this time, the size of the source server might have increased. Update your estimated migration time for your production server based on the amount of this increase. If the increase is significant, you can consider doing another test using the PITR server. But for most servers the size increase shouldn't be significant enough.

+* **Validation** - Once the offline migration completes for the production server, you need to verify if the data in flexible server is an exact copy of the single server. Customers can use opensource/thirdparty tools or can do the validation manually. Prepare the validation steps that you would like to do in advance of the actual migration. Validation can include: 

+ * Row count match for all the tables involved in the migration.ΓÇ»

+ * Matching counts for all the database object (tables, sequences, extensions, procedures, indexes)

+ * Comparing max or min IDs of key application related columns

+>[!NOTE]

+> The size of databases is not the right metric for validation.The source server might have bloats/dead tuples which can bump up the size on the source server. Also, the storage containers used in single and flexible servers are completely different. It is completely normal to have size differences between source and target servers. If there is an issue in the first three steps of validation, it indicates a problem with the migration.ΓÇ»

+* **Migration of server settings** - The users, roles/privileges, server parameters, firewall rules (if applicable), tags, alerts need to be manually copied from single server to flexible server. Users and roles are migrated from Single to Flexible server by following the steps listed in this [doc](../single-server/how-to-upgrade-using-dump-and-restore.md).

+* **Changing connection strings** - Post successful validation, application should change their connection strings to point to flexible server. This activity is coordinated with the application team to make changes to all the references of connection strings pointing to single server. Note that in the flexible server the user parameter in the connection string no longer needs to be in the **username@servername** format. You should just use the **user=username** format for this parameter in the connection string

+For example

+Psql -h **mysingleserver**.postgres.database.azure.com -u **user1@mysingleserver** -d db1

+should now be of the format

+Psql -h **myflexserver**.postgres.database.azure.com -u user1 -d db1

+**Total planned downtime** = **Time to migrate PITR** + **time to migrate Buffer** + **time for Validation** + **time to migrate server settings** + **time to switch connection strings to the flexible server.**

+While most frequently a migration runs without a hitch, itΓÇÖs good practice to plan for contingencies if there is additional time required for debugging or if a migration may need to be restarted.

+#### Migration prerequisites

+The following pre-requisites need to be taken care of before using the Single to Flex Migration tool for migration

+##### Network connectivity between Single and Flexible Server

+The following table summarizes the possible network configuration on single and flexible server.

+|Server Type| Public Access | Private Access |

+|:|:-|:--|

+| Single Server | Public access turned **ON** with firewall rules or VNet rules (service endpoints). | Public access turned OFF with private end points configured.|

+| Flexible Server | Public access with firewall rules | Server deployed inside a VNet and delegated subnet.|

+>[!NOTE]

+> Currently private end points are not supported in Flexible server. It will be supported at a later point time.

+The following table summarizes the list of networking scenarios supported by the Single to Flex migration tool.

+

+|Single Server Config| Flexible Server Config | Supported by Migration tool |

+|:|:-|:--|

+| Public Access | Public Access | Yes|

+| Public Access | Private Access | Yes|

+| Private Access | Public Access | No|

+| Private Access | Private Access | Yes|

+**Steps needed to establish connectivity between your Single and Flexible Server**

+* If your single server is public access (case #1 and case #2 in the above table), there's nothing needed from your end. The single to flex migration tool automatically establishes connection between single and flexible server and the migration will go through.

+* If your single server is in private access, then the only supported scenario is when your Flexible server is inside a VNet. If your flexible server is deployed in the same VNet as the private end point of your Single server, connections between single server and flexible server should automatically work provided there is no network security group(NSGs) blocking the connectivity between subnets. If flexible server is deployed in another VNet, [peering should be established between the VNets](../../virtual-network/tutorial-connect-virtual-networks-portal.md) for the connection to work between Single and Flexible server.

+##### Allow list required extensions

+Use the following select command in the Single Server databases to list all the extensions that are being used.

+```

+ select * from pg_extension;

+```

+Search for the **azure.extensions** parameter on the Server Parameters blade on your Flexible server. Select the list of extensions obtained by running the above query on your Single server database to this server parameter and click Save. You should wait for the deployment to complete before proceeding further.

+>[!NOTE]

+> If TIMESCALEDB, PG_PARTMAN or POSTGIS_TIGER_DECODER extensions are used in your single server database, please raise a support request since the Single to Flex migration tool will not handle these extensions.

+Check if the list contains any of the following extensions:

+* PG_CRON

+* PG_HINT_PLAN

+* PG_PARTMAN_BGW

+* PG_PREWARM

+* PG_STAT_STATEMENTS

+* PG_AUDIT

+* PGLOGICAL

+* WAL2JSON

+If yes, then follow the below steps.

+Go to the server parameters blade and search for **shared_preload_libraries** parameter. This parameter indicates the set of extension libraries that are preloaded at the server restart. Pg_cron and pg_stat_statements extensions are selected by default. Select the list of above extensions used by the single server database to this parameter and click on Save.

+The changes to this server parameter would require a server restart to come into effect.

+Use the **Save and Restart** option and wait for the postgresql server to restart.

+### Migration

+Once the pre-migration steps are complete, you're ready to carry out the migration of the production databases of your single server. At this point, you've finalized the day and time of production migration along with a planned downtime for your applications.

+* Create a flexible server with a **General-Purpose** or **Memory Optimized** compute tier. Pick a minimum 4VCore or higher SKU to complete the migration quickly. Burstable SKUs are blocked for use as migration target servers.

+* Don't include HA or geo redundancy option while creating flexible server. You can always enable it with zero downtime once the migration from single server is complete. Don't create any read-replicas yet on the flexible server.

+* Before initiating the migration, stop all the applications that connect to your production server.

+* Checkpoint the source server by running **checkpoint** command and restart the source server.

+This command ensures any remaining applications or connections are disconnected. Additionally, you can run **select * from pg_stat_activity;** after the restart to ensure no applications is connected to the source server.

+Trigger the migration of your production databases using the single to flex migration tool. The migration requires close monitoring, and the monitoring user interface of the migration tool comes in handy. Check the migration status over the period of time to ensure there is progress and wait for the migration to complete.

+### Post migration

+* Once the migration is complete, verify the data on your flexible server and make sure it's an exact copy of the single server.

+* Post verification, enable HA/ backup options as needed on your flexible server.

+* Change the SKU of the flexible server to match the application needs. This change needs a database server restart.

+* Migrate users and roles from single to flexible servers. This step can be done by creating users on flexible servers and providing them with suitable privileges or by using the steps that are listed in this [doc](../single-server/how-to-upgrade-using-dump-and-restore.md).

+* If you've changed any server parameters from their default values in single server, copy those server parameter values in flexible server.

+* Copy other server settings like tags, alerts, firewall rules (if applicable) from single server to flexible server.

+* Make changes to your application to point the connection strings to flexible server.

+* Monitor the database performance closely to see if it requires performance tuning.

+Additionally, you can deploy managed private endpoints for your Azure Key Vault resources if you need to run scans using any authentication options rather than Managed Identities, such as SQL Authentication or Account Key.

+At-scale data processing systems typically store a single table in storage as multiple files. In the Microsoft Purview Data Catalog, this concept is represented by using resource sets. A resource set is a single object in the catalog that represents a large number of assets in storage.

+Microsoft Purview can customize and further enrich your resource set assets through the **Advanced Resource Sets** capability. Advanced resource sets allow Microsoft Purview to understand the underlying partitions of data ingested and enables the creation of [resource set pattern rules](how-to-resource-set-pattern-rules.md) that customize how Microsoft Purview groups resource sets during scanning.

+When Advanced Resource Sets are enabled, Microsoft Purview runs extra aggregations to compute the following information about resource set assets:

+Advanced resource sets are off by default in all new Microsoft Purview instances. Advanced resource sets can be enabled from **Account information** in the management hub. Only those users who are added to the Data Curator role at root collection, can manage Advanced Resource Sets settings.

+> Enabling advanced resource sets will impact the refresh rate of asset and classification insights. When advanced resource sets are on, asset and classification insights will only update twice a day.

+## How resource sets are displayed in the Microsoft Purview Data Catalog

+To customize or override how Microsoft Purview detects which assets are grouped as resource sets and how they're displayed within the catalog, you can define pattern rules in the management center. For step-by-step instructions and syntax, see [resource set pattern rules](how-to-resource-set-pattern-rules.md).

+ 1. [JDK 8 or later](https://www.oracle.com/java/technologies/javase-jdk11-downloads.html) is installed. Restart the machine after you newly install the JDK for it to take effect.

+ 4. [JDK 8 or later](https://www.oracle.com/java/technologies/javase-jdk11-downloads.html) is installed. Restart the machine after you newly install the JDK for it to take effect.

+ 2. [JDK 8 or later](https://www.oracle.com/java/technologies/javase-jdk11-downloads.html) is installed. Restart the machine after you newly install the JDK for it to take effect.

+| [Azure Storage: Managed Disks](https://learn.microsoft.com/azure/virtual-machines/disks-redundancy?source=recommendations) |   |

+1. Specify a name for the profile in the **Profile/Filter Number** field.

+ > [!NOTE]

+ > Vanilla SAP installation requires this additional step: right-click the profile you have created and create a new filter.

+General purpose V2 storage accounts (Hot and Cool tier) | Supported | Usage of GPv2 is recommended because GPv1 doesn't support ZRS (Zonal Redundant Storage).

+Azure Storage firewalls for virtual networks | Supported | If you're using firewall enabled cache storage account or target storage account, ensure you ['Allow trusted Microsoft services'](../storage/common/storage-network-security.md#exceptions).<br></br>Also, ensure that you allow access to at least one subnet of source Vnet.<br></br>Note: Don't restrict virtual network access to your storage accounts used for Site Recovery. You should allow access from 'All networks'.

+Soft delete | Not supported | Soft delete isn't supported because once it is enabled on cache storage account, it increases cost. Azure Site Recovery performs frequent creates/deletes of log files while replicating causing costs to increase.

+Red Hat Enterprise Linux | 6.7, 6.8, 6.9, 6.10, 7.0, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6,[7.7](https://support.microsoft.com/help/4528026/update-rollup-41-for-azure-site-recovery), [7.8](https://support.microsoft.com/help/4564347/), [7.9](https://support.microsoft.com/help/4578241/), [8.0](https://support.microsoft.com/help/4531426/update-rollup-42-for-azure-site-recovery), 8.1, [8.2](https://support.microsoft.com/help/4570609/), [8.3](https://support.microsoft.com/help/4597409/), [8.4](https://support.microsoft.com/topic/883a93a7-57df-4b26-a1c4-847efb34a9e8) (4.18.0-305.30.1.el8_4.x86_64 or higher), [8.5](https://support.microsoft.com/topic/883a93a7-57df-4b26-a1c4-847efb34a9e8) (4.18.0-348.5.1.el8_5.x86_64 or higher), [8.6](https://support.microsoft.com/topic/update-rollup-62-for-azure-site-recovery-e7aff36f-b6ad-4705-901c-f662c00c402b), 8.7 (Azure Site Recovery for 8.7 isn't available in China regions).

+CentOS | 6.5, 6.6, 6.7, 6.8, 6.9, 6.10 </br> 7.0, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, [7.8](https://support.microsoft.com/help/4564347/), [7.9 pre-GA version](https://support.microsoft.com/help/4578241/), 7.9 GA version is supported from 9.37 hot fix patch** </br> 8.0, 8.1, [8.2](https://support.microsoft.com/help/4570609), [8.3](https://support.microsoft.com/help/4597409/), 8.4 (4.18.0-305.30.1.el8_4.x86_64 or later), 8.5 (4.18.0-348.5.1.el8_5.x86_64 or later), 8.6, 8.7 (Azure Site Recovery for 8.7 isn't available in China regions).

+Debian 9 | Includes support for 9.1 to 9.13. Debian 9.0 isn't supported. [Supported kernel versions](#supported-debian-kernel-versions-for-azure-virtual-machines)

+> For Linux versions, Azure Site Recovery doesn't support custom OS kernels. Only the stock kernels that are part of the distribution minor version release/update are supported.

+SUSE Linux Enterprise Server 12 (SP1, SP2, SP3, SP4, SP5) | [9.52](https://support.microsoft.com/topic/update-rollup-65-for-azure-site-recovery-kb5021964-15db362f-faac-417d-ad71-c22424df43e0) | All [stock SUSE 12 SP1,SP2,SP3,SP4,SP5 kernels](https://www.suse.com/support/kb/doc/?id=000019587) are supported. </br></br> No new SUSE Linux Enterprise Server 12 kernels supported in this release. |

+> Multipath software isn't supported.

+Azure RBAC policies | Not supported | Azure role-based access control (Azure RBAC) policies on VMs aren't replicated to the failover VM in target region.

+Extensions | Not supported | Extensions aren't replicated to the failover VM in target region. It needs to be installed manually after failover.

+Resize disk on replicated VM | Resizing up on the source VM is supported. Resizing down on the source VM isn't supported. Resizing should be performed before failover. No need to disable/re-enable replication.<br/><br/> If you change the source VM after failover, the changes aren't captured.<br/><br/> If you change the disk size on the Azure VM after failover, changes aren't captured by Site Recovery, and failback will be to the original VM size.<br/><br/> If resizing to >=4 TB, note Azure guidance on disk caching [here](../virtual-machines/premium-storage-performance.md).

+Disk caching | Disk Caching isn't supported for disks 4 TiB and larger. If multiple disks are attached to your VM, each disk that is smaller than 4 TiB will support caching. Changing the cache setting of an Azure disk detaches and re-attaches the target disk. If it's the operating system disk, the VM is restarted. Stop all applications/services that might be affected by this disruption before changing the disk cache setting. Not following those recommendations could lead to data corruption.

+Disk subscription limits | Up to 3000 protected disks per Subscription | Ensure that the Source or Target subscription doesn't have more than 3000 Azure Site Recovery-protected Disks (Both Data and OS).

+Encryption at host | Not Supported | The VM will get protected, but the failed over VM won't have Encryption at host enabled. [See detailed information](../virtual-machines/disks-enable-host-based-encryption-portal.md) to create a VM with end-to-end encryption using Encryption at host.

+Azure Disk Encryption (ADE) for Windows OS | Supported for VMs with managed disks. | VMs using unmanaged disks aren't supported. <br/><br/> HSM-protected keys aren't supported. <br/><br/> Encryption of individual volumes on a single disk isn't supported. |

+Azure Disk Encryption (ADE) for Linux OS | Supported for VMs with managed disks. | VMs using unmanaged disks aren't supported. <br/><br/> HSM-protected keys aren't supported. <br/><br/> Encryption of individual volumes on a single disk isn't supported. <br><br> Known issue with enabling replication. [Learn more.](./azure-to-azure-troubleshoot-errors.md#enable-protection-failed-as-the-installer-is-unable-to-find-the-root-disk-error-code-151137) |

+Hot add | Supported | Enabling replication for a data disk that you add to a replicated Azure VM is supported for VMs that use managed disks. <br/><br/> Only one disk can be hot added to an Azure VM at a time. Parallel addition of multiple disks isn't supported. |

+Storage Spaces Direct | Supported for crash consistent recovery points. Application consistent recovery points aren't supported. |

+Scale-out File Server | Supported for crash consistent recovery points. Application consistent recovery points aren't supported. |

+DRBD | Disks that are part of a DRBD setup aren't supported. |

+Cool and Hot Storage | Not supported | Virtual machine disks aren't supported on cool and hot storage

+Soft delete | Not supported | Soft delete isn't supported because once it's enabled on a storage account, it increases cost. Azure Site Recovery performs very frequent creates/deletes of log files while replicating causing costs to increase.

+iSCSI disks | Not supported | Azure Site Recovery may be used to migrate or failover iSCSI disks into Azure. However, iSCSI disks aren't supported for Azure to Azure replication and failover/failback.

+NIC | Maximum number supported for a specific Azure VM size | NICs are created when the VM is created during failover.<br/><br/> The number of NICs on the failover VM depends on the number of NICs on the source VM when replication was enabled. If you add or remove a NIC after enabling replication, it doesn't impact the number of NICs on the replicated VM after failover. <br/><br/> The order of NICs after failover isn't guaranteed to be the same as the original order. <br/><br/> You can rename NICs in the target region based on your organization's naming conventions. NIC renaming is supported using PowerShell.

+Internet Load Balancer | Not supported | You can set up public/internet load balancers in the primary region. However, public/internet load balancers aren't supported by Azure Site Recovery in the DR region.

+VPN site-to-site connection to on-premises<br/><br/>(with or without ExpressRoute)| Supported | Ensure that the UDRs and NSGs are configured in such a way that the Site Recovery traffic isn't routed to on-premises. [Learn more](./azure-to-azure-about-networking.md)

+Palo Alto Network Appliance | Not supported | With third-party appliances, there are often restrictions imposed by the provider inside the Virtual Machine. Azure Site Recovery needs agent, extensions, and outbound connectivity to be available. But the appliance doesn't let any outbound activity to be configured inside the Virtual Machine.

+- Deploy disaster recovery by [replicating Azure VMs](./azure-to-azure-quickstart.md).

+Azure Monitor Logs provide a log data platform that collects activity and resource logs, along with other monitoring data. Within Azure Monitor Logs, you use Log Analytics to write and test log queries and interactively analyze log data. You can visualize and query log results, and configure alerts to take actions based on monitored data.

+Using Azure Monitor Logs with Site Recovery is supported for **Azure to Azure** replication and **VMware VM/physical server to Azure** replication.

+> To get the churn data logs and upload rate logs for VMware and physical machines, you need to install a Microsoft monitoring agent on the Process Server. This agent sends the logs of the replicating machines to the workspace. This capability is available only for the 9.30 mobility agent version onwards.

+## Prerequisites

+- At least one machine is protected in a Recovery Services vault.

+3. Select the Azure Monitor Logs subscription and the Log Analytics workspace.

+3. Download the Windows Agent (64-bit) on the Process Server.

+7. Once the installation is complete, go to Log Analytics workspace and select **Legacy agents management**. Go to the **Data** page and select **Windows Performance Counters**.

+8. Select **'+'** to add the following two counters with a sample interval of 300 seconds:

+This query plots a trend graph for a specific disk, **disk0**, of a replicated item, **win-9r7sfh9qlru**, which represents the data change rate (Write Bytes per Second) and data upload rate. You can find the disk name on the **Disks** blade of the replicated item in the recovery services vault. The instance name to be used in the query is the DNS name of the machine followed by _ and the disk name as in this example.

+This query plots a summary table for Azure VMs replicated to a secondary Azure region. It shows the VM name, replication, and protection status, RPO, test failover status, Mobility agent version, any active replication errors, and the source location.

+This query plots a summary table for VMware VMs and physical servers replicated to Azure. It shows the machine name, replication and protection status, RPO, test failover status, Mobility agent version, any active replication errors, and the relevant process server.

+For the alert, set **Threshold value** to `20`.

+For the alert, set **Threshold value** to `1`.

+For the alert, set **Threshold value** to `20`.

+For the alert, set **Threshold value** to `1`.

+For the alert, set **Threshold value** to `20`.

+### Test failover for a single machine exceeds 90 days

+For the alert, set **Threshold value** to `1`.

+ 1. For **Delta table path**, it's used to specify the location and name of your Delta Lake table stored in Azure Data Lake Storage Gen2. You can choose to use one or more path segments to define the path to the delta table and the delta table name. To learn more, see to [Write to Delta Lake table (Public Preview)](./write-to-delta-lake.md).

+ | **Computing** | This processor is used for processing the stream data according to the query logic, for example, aggregating, filtering, grouping with window, etc. To learn more about the stream data computation query functions, see [Azure Stream Analytics Query Language Reference](/stream-analytics-query/stream-analytics-query-language-reference). |

+ | **Merger** | This processor is to receive the crossing-partition stream data, which were outputted from several upstream streaming nodes. The best practice to optimize job performance is to update query to remove the merger processor to make the job become parallel since the merger processor is the bottleneck of the job. The job diagram simulator feature within Visual Studio Code ASA extension can help you simulating your query locally when you optimizing your job query. To learn more, see [Optimize query using job diagram simulator (preview)](./optimize-query-using-job-diagram-simulator.md). |

+ To learn more details of the input and output types, see [Azure Stream Analytics inputs overview](./stream-analytics-define-inputs.md), and [Azure Stream Analytics outputs overview](./stream-analytics-define-outputs.md).

+ Title: Build real-time dashboard with Power BI dataset produced from Stream Analytics no code editor

+description: Learn how to use the no code editor to easily create a Stream Analytics job to produce the Power BI dataset, and use it to build the real-time dashboard. It continuously reads from Event Hubs, and outputs the data into Power BI dataset to build the real-time Power BI dashboard.

+# Build real-time dashboard with Power BI dataset produced from Stream Analytics no code editor

+This article describes how you can use the no code editor to easily create a Stream Analytics job to produce processed data into Power BI dataset. It continuously reads from your Event Hubs, processes and outputs the data into Power BI dataset to build the real-time Power BI dashboard.

+## Prerequisites

+- Your Azure Event Hubs resources must be publicly accessible and not be behind a firewall or secured in an Azure Virtual Network

+- You should have an existing Power BI workspace, and you have the permission to create dataset there.

+- The data in your Event Hubs must be serialized in either JSON, CSV, or Avro format.

+## Develop a Stream Analytics job to create Power BI dataset with selected data

+1. In the [Azure portal](https://portal.azure.com), locate and select the Azure Event Hubs instance.

+1. Select **Features** > **Process Data** and then select **Start** on the **Build the real-time data dashboard with Power BI** card.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/event-hub-process-data-templates.png" alt-text="Screenshot showing the Filter and ingest to ADLS Gen2 card where you select Start." lightbox="./media/no-code-build-power-bi-dashboard/event-hub-process-data-templates.png" :::

+1. Enter a name for the Stream Analytics job, then select **Create**.

+

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/create-new-stream-analytics-job.png" alt-text="Screenshot showing where to enter a job name." lightbox="./media/no-code-build-power-bi-dashboard/create-new-stream-analytics-job.png" :::

+1. Specify the **Serialization type** of your data in the Event Hubs window and the **Authentication method** that the job will use it to connect to the Event Hubs. Then select **Connect**.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/event-hub-configuration.png" alt-text="Screenshot showing the Event Hubs connection configuration." lightbox="./media/no-code-build-power-bi-dashboard/event-hub-configuration.png" :::

+1. When the connection is established successfully and you have data streams flowing into your Event Hubs instance, you immediately see two things:

+ - Fields that are present in the input data. You can choose **Add field** or select the three dot symbol next to a field to remove, rename, or change its type.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/no-code-schema.png" alt-text="Screenshot showing the Event Hubs field list where you can remove, rename, or change the field type." lightbox="./media/no-code-build-power-bi-dashboard/no-code-schema.png" :::

+ - A live sample of incoming data in the **Data preview** table under the diagram view. It automatically refreshes periodically. You can select **Pause streaming preview** to see a static view of the sample input data.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/no-code-sample-input.png" alt-text="Screenshot showing sample data under Data Preview." lightbox="./media/no-code-build-power-bi-dashboard/no-code-sample-input.png" :::

+1. Select the **Manage** tile. In the **Manage fields** configuration panel, choose the fields you want to output. If you want to add all the fields, select **Add all fields**.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/manage-fields-configuration.png" alt-text="Screenshot that shows the manage field operator configuration." lightbox="./media/no-code-build-power-bi-dashboard/manage-fields-configuration.png" :::

+1. Select **Power BI** tile. In the **Power BI** configuration panel, fill in needed parameters and connect.

+ - **Dataset**: it's the Power BI destination where the Azure Stream Analytics job output data is written into.

+ - **Table**: it's the table name in the Dataset where the output data goes to.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/no-code-pbi-configuration.png" alt-text="Screenshot that shows the Power BI output configuration." lightbox="./media/no-code-build-power-bi-dashboard/no-code-pbi-configuration.png" :::

+1. Optionally, select **Get static preview/Refresh static preview** to see the data preview that will be ingested in event hub.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/no-code-output-static-preview.png" alt-text="Screenshot showing the Get static preview/Refresh static preview option." lightbox="./media/no-code-build-power-bi-dashboard/no-code-output-static-preview.png" :::

+1. Select **Save** and then select **Start** the Stream Analytics job.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/no-code-save-start.png" alt-text="Screenshot showing the Save and Start options." lightbox="./media/no-code-build-power-bi-dashboard/no-code-save-start.png" :::

+1. To start the job, specify:

+ - The number of **Streaming Units (SUs)** the job runs with. SUs represents the amount of compute and memory allocated to the job. We recommended that you start with three and then adjust as needed.

+ - **Output data error handling** ΓÇô It allows you to specify the behavior you want when a jobΓÇÖs output to your destination fails due to data errors. By default, your job retries until the write operation succeeds. You can also choose to drop such output events.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/no-code-start-job.png" alt-text="Screenshot showing the Start Stream Analytics job options where you can change the output time, set the number of streaming units, and select the Output data error handling options." lightbox="./media/no-code-build-power-bi-dashboard/no-code-start-job.png" :::

+1. After you select **Start**, the job starts running within two minutes, and the metrics will be open in tab section below.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/job-metrics-after-started.png" alt-text="Screenshot that shows the job metrics after it's started." lightbox="./media/no-code-build-power-bi-dashboard/job-metrics-after-started.png" :::

+ You can also see the job under the Process Data section on the **Stream Analytics jobs** tab. Select **Open metrics** to monitor it or stop and restart it, as needed.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/no-code-list-jobs.png" alt-text="Screenshot of the Stream Analytics jobs tab where you view the running jobs status." lightbox="./media/no-code-build-power-bi-dashboard/no-code-list-jobs.png" :::

+## Build the real-time dashboard in Power BI

+Now, you have the Azure Stream Analytics job running and the data is continuously written into the table in the Power BI dataset you've configured. You can now create the real-time dashboard in Power BI workspace.

+1. Go to the Power BI workspace, which you've configured in above Power BI output tile, and select **+ New** in the top left corner, then choose **Dashboard** to give the new dashboard a name.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/pbi-dashboard-creation.png" alt-text="Screenshot of the pbi dashboard creation." lightbox="./media/no-code-build-power-bi-dashboard/pbi-dashboard-creation.png" :::

+2. Once the new dashboard is created, you'll be led to the new dashboard. Select **Edit**, and choose **+ Add a tile** in the top menu bar. A right pane is open. Select **Custom Streaming Data** to go to next page.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/pbi-dashboard-add-tile.png" alt-text="Screenshot of the pbi dashboard adding tile." lightbox="./media/no-code-build-power-bi-dashboard/pbi-dashboard-add-tile.png" :::

+3. Select the streaming dataset (for example **nocode-pbi-demo-xujx**) which you've configured in Power BI node, and go to next page.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/pbi-dashboard-add-tile-select-dataset.png" alt-text="Screenshot of the pbi dashboard adding tile with selected dataset." lightbox="./media/no-code-build-power-bi-dashboard/pbi-dashboard-add-tile-select-dataset.png" :::

+4. Fill in the tile details, and follow the next step to complete the tile configuration.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/pbi-dashboard-add-tile-details.png" alt-text="Screenshot of the pbi dashboard adding tile with configured details." lightbox="./media/no-code-build-power-bi-dashboard/pbi-dashboard-add-tile-details.png" :::

+5. Then, you can adjust its size and get the continuously updated dashboard as below.

+ :::image type="content" source="./media/no-code-build-power-bi-dashboard/pbi-dashboard-report.png" alt-text="Screenshot of the pbi dashboard report." lightbox="./media/no-code-build-power-bi-dashboard/pbi-dashboard-report.png" :::

+## Next steps

+Learn more about Azure Stream Analytics and how to monitor the job you've created.

+* [Introduction to Azure Stream Analytics](stream-analytics-introduction.md)

+* [Monitor Stream Analytics job with Azure portal](stream-analytics-monitoring.md)

+* [Azure Stream Analytics no code editor](./no-code-stream-processing.md)

+ Title: Build real-time dashboard with Azure Synapse Analytics and Power BI

+description: Use no code editor to compute aggregations and write to Azure Synapse Analytics and build real-time dashboards using Power BI

+# Build real-time Power BI dashboards with Stream Analytics no code editor

+1. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/).

+2. Deploy the TollApp event generator to Azure, use this link to [Deploy TollApp Azure Template](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-stream-analytics%2Fmaster%2FSamples%2FTollApp%2FVSProjects%2FTollAppDeployment%2Fazuredeploy.json). Set the 'interval' parameter to 1. And use a new resource group for this step.

+3. Create an [Azure Synapse Analytics workspace](../synapse-analytics/get-started-create-workspace.md) with a [Dedicated SQL pool](../synapse-analytics/get-started-analyze-sql-pool.md#create-a-dedicated-sql-pool).

+ > [!NOTE]

+ > If you'd like to build the real-time Power BI dashboard directly without capturing the data into database, you can skip step#3 and 4, then go to this guide to [<u>build real-time dashboard with Power BI dataset produced by Stream Analytics job</u>](./no-code-build-power-bi-dashboard.md).

+4. Create a table named `carsummary` using your Dedicated SQL pool. You can do it by running the following SQL script:

+2. Use the Power BI connector for Azure Synapse SQL to connect to your database with **DirectQuery**.

+ You'll then see a chart that can be published. You can configure [automatic page refresh](/power-bi/create-reports/desktop-automatic-page-refresh#authoring-reports-with-automatic-page-refresh-in-power-bi-desktop) and set it to 3 minutes to get a real-time view.

+## More option

+Except the Azure Synapse SQL, you can also use the SQL Database as the no-code editor output to receive the streaming data. And then use Power BI connector to connect the SQL Database with your database with **DirectQuery** as well to build the real-time dashboard.

+It's also a good option to build the real-time dashboard with your streaming data. For more information about the SQL Database output, see [Transform and ingest to SQL Database](./no-code-transform-filter-ingest-sql.md).

+In this tutorial, you created a Stream Analytics job using the no code editor to define aggregations and write results to Azure Synapse Analytics. You then used the Power BI to build a real-time dashboard to see the results produced by the job.

+- [Build real-time dashboard with Power BI dataset](./no-code-build-power-bi-dashboard.md)

+- [Capture data from Event Hubs in Delta Lake format (preview)](./capture-event-hub-data-delta-lake.md)

+Managed identities eliminate the limitations of user-based authentication methods. These limitations include the need to reauthenticate because of password changes or user token expirations that occur every 90 days.

+**Exactly once delivery (preview)** is supported in the ADLS Gen2 as no code editor output. You can enable it in the **Write mode** section in ADLS Gen2 configuration. For more information about this feature, see [Exactly once delivery (preview) in Azure Data Lake Gen2](./blob-storage-azure-data-lake-gen2-output.md#exactly-once-delivery-public-preview)

+**Write to Delta Lake table (preview)** is supported in the ADLS Gen2 as no code editor output. You can access this option in section **Serialization** in ADLS Gen2 configuration. For more information about this feature, see [Write to Delta Lake table (Public Preview)](./write-to-delta-lake.md).

+1. Select the **Manage** tile. In the **Manage fields** configuration panel, choose the fields you want to output. If you want to add all the fields, click **Add all fields**.

+You may consider to optimize it to parallel job (as example below) by rewriting your query or updating input/output configurations with the **job diagram simulator** within Visual Studio Code ASA extension or query editor in Azure portal. To learn more, see [Optimize query using job diagram simulator (preview)](./optimize-query-using-job-diagram-simulator.md).

+| | Spark integration | Azure Data Explorer linked service: Built-in Kusto Spark integration with support for Azure Active Directory pass-through authentication, Synapse Workspace MSI, and Service Principal | Built-in Kusto Spark connector integration with support for Azure Active Directory pass-through authentication, Synapse Workspace MSI, and Service Principal |

+> You can schedule updates from the Overview blade or Machines blade in update management center (preview) page or from the selected VM.

+1. In **Change update settings**, select **+Add machine** to select the machine for which you want to change the update settings.

+1. In **Select resources**, select the machine and select **Add**.

+1. In the **Change update settings** page, you will see the machine classified as per the operating system with the list of following updates that you can select and apply.

+ - **Periodic assessment** - The **periodic Assessment** is set to run every 24 hours. You can either enable or disable this setting.

+ - **Hot patch** - You can enable [hot patching](../automanage/automanage-hotpatch.md) for Windows Server Azure Edition Virtual Machines (VMs). Hot patching is a new way to install updates on supported *Windows Server Azure Edition* virtual machines that doesn't require a reboot after installation. You can use update management center (preview) to install other patches by scheduling patch installation or triggering immediate patch deployment. You can enable, disable or reset this setting.

+ - **Automatic by OS (Windows Automatic Updates)** - When the workload running on the VM doesn't have to meet availability targets, the operating system updates are automatically downloaded and installed. Machines are rebooted as needed.

+ - **Manual updates** - This mode disables Windows automatic updates on VMs. Patches are installed manually or using a different solution.

+ - **Image Default** - Only supported for Linux Virtual Machines, this mode uses the default patching configuration in the image used to create the VM.

+1. After you make the selection, select **Save**.

+1. In **Update management center**, select **Machines** > your **subscription**.

+1. Select the checkbox of your machine from the list and select **Update settings**.

+1. In **Change update settings**, select **+Add machine** to select the machine for which you want to change the update settings.

+1. In **Select resources**, select the machine and select **Add** and follow the procedure from step 5 listed in **From Overview blade** of [Configure settings on single VM](#configure-settings-on-single-vm).

+1. In **Updates (Preview)**, select **Update Settings**.

+> You can schedule updates from the Overview blade or Machines blade.

+1. In **Change update settings**, select the update settings that you want to change for your machines. Follow the procedure from step 3 listed in **From Overview blade** of [Configure settings on single VM](#configure-settings-on-single-vm).

+1. In **Update management center**, select **Machines** > your **subscription**, and select the checkbox for all your machines from the list.