-| [Request tracing](api-management-howto-api-inspector.md) | Γ£ö∩╕Å | Γ¥î⁴ | Γ£ö∩╕Å | Γ£ö∩╕Å | Γ¥î |

-⁴ Tracing is currently unavailable in the v2 tiers.

-* Workspace gateways don't support APIs created from Azure resources such as Azure OpenAI Service, App Service, Function Apps, and so on

- - **ARR affinity**: In a multi-instance deployment, ensure that the client is routed to the same instance for the life of the session. You can set this option to **Off** for stateless applications.

-Note that the default affinity cookie name is *ApplicationGatewayAffinity* and you can change it. If you deploy multiple application gateway instances in the same network topology, you must set unique cookie names for each instance. If you're using a custom affinity cookie name, an additional cookie is added with `CORS` as suffix. For example: *CustomCookieNameCORS*.

-Applicable tiers: **Standard (preview)**, **Premium (preview)**, **Enterprise**, **Enterprise Flash**

-Azure Cache for Redis supports zone redundant configurations in the Standard (preview), Premium (preview), and Enterprise tiers. A [zone redundant cache](cache-how-to-zone-redundancy.md) can place its nodes across different [Azure Availability Zones](../reliability/availability-zones-overview.md) in the same region. It eliminates data center or Availability Zone outage as a single point of failure and increases the overall availability of your cache.

-In this article, you'll learn how to configure a zone-redundant Azure Cache instance using the Azure portal.

-Azure Cache for Redis Standard (Preview), Premium (Premium), and Enterprise tiers provide built-in redundancy by hosting each cache on two dedicated virtual machines (VMs). Even though these VMs are located in separate [Azure fault and update domains](/azure/virtual-machines/availability) and highly available, they're susceptible to data center-level failures. Azure Cache for Redis also supports zone redundancy in its Standard (preview), Premium (preview) and Enterprise tiers. A zone-redundant cache runs on VMs spread across multiple [Availability Zones](../reliability/availability-zones-overview.md). It provides higher resilience and availability.

- | **Location** | Select a location. | Select a [region](https://azure.microsoft.com/regions/) near other services that will use your cache. |

- :::image type="content" source="media/cache-how-to-zone-redundancy/cache-availability-zone.png" alt-text="Screenshot showing the Advanced tab with a red box around Availability zones.:":::

- > Enabling Automatic Zone Allocation is currently NOT supported for Geo Replicated caches or caches with VNET injection.

-1. Availability zones can be selected manually for Premium tier caches. The count of availability zones must always be less than or equal to the Replica count for the cache.

-A Premium cache has one primary and one replica node by default. To configure zone redundancy for more than two Availability Zones, you need to add [more replicas](cache-how-to-multi-replicas.md) to the cache you're creating.

-Yes, updating an existing Standard or Premium cache to use zone redundancy is supported. You can enable it by selecting **Allocate Zones automatically** from the **Advanced settings** on the Resource menu. You cannot disable zone redundancy once you have enabled it.

-When your cache uses zone redundancy configured with multiple Availability Zones, data is replicated from the primary cache node in one zone to the other node(s) in another zone(s). The data transfer charge is the network egress cost of data moving across the selected Availability Zones. For more information, see [Bandwidth Pricing Details](https://azure.microsoft.com/pricing/details/bandwidth/).

-For more information, see

-This event occurs when a function app doesn't have the `FUNCTIONS_WORKER_RUNTIME` application setting, which is required.

-The `FUNCTIONS_WORKER_RUNTIME` application setting indicates the language or language stack on which the function app runs, such as `python`. For more information on valid values, see the [`FUNCTIONS_WORKER_RUNTIME`](../../functions-app-settings.md#functions_worker_runtime) reference.

-While not currently required, you should always specify `FUNCTIONS_WORKER_RUNTIME` for your function apps. When you don't have this setting and the Functions host can't determine the correct language or language stack, you might see exceptions or unexpected behaviors.

-Because `FUNCTIONS_WORKER_RUNTIME` is likely to become a required setting, you should explicitly set it in all of your existing function apps and deployment scripts to prevent any downtime in the future.

-In a production application, add `FUNCTIONS_WORKER_RUNTIME` to the [application settings](../../functions-how-to-use-azure-function-app-settings.md#settings).

-When running locally in Azure Functions Core Tools, also add `FUNCTIONS_WORKER_RUNTIME` to the [local.settings.json file](../../functions-develop-local.md#local-settings-file).

-description: This article tracks FedRAMP and DoD compliance scope for Azure, Dynamics 365, Microsoft 365, and Power Platform cloud services across Azure, Azure Government, and Azure Government Secret cloud environments.

-description: Learn about Trusted Internet Connections (TIC) guidance for Azure IaaS and PaaS services

-description: Comprehensive list of Azure Government cloud solution providers, resellers, and distributors.

-Since the launch of [Azure Government services in the Cloud Solution Provider (CSP) program)](https://azure.microsoft.com/blog/announcing-microsoft-azure-government-services-in-the-cloud-solution-provider-program/), we've worked with the partner community to bring them the benefits of this channel. Our goal is to enable the partner community to resell Azure Government and help them grow their business while providing customers with cloud services they need.

-Below you can find a list of all the authorized Cloud Solution Providers (CSPs), Agreement for Online Services for Government (AOS-G), and Licensing Solution Providers (LSP) that can transact Azure Government. Updates to this list will be made as new partners are onboarded.

-| iOS | ✔️ | ✔️ | ❌ | ❌ | ✔️ | ❌ |

-In this article, you learn how to implement Microsoft Teams spotlight capability with Azure Communication Services Calling SDKs. This capability allows users in the call or meeting to pin and unpin videos for everyone. The maximum limit of pinned videos is seven.

-The following tables define support for Spotlight in Azure Communication Services.

-### Identities & call types

-|Identities | Teams meeting | Room | 1:1 call | Group call | 1:1 Teams interop call | Group Teams interop call |

-|--|||-|||--|

-|Communication Services user | ✔️ | ✔️ | | ✔️ | | ✔️ |

-|Microsoft 365 user | ✔️ | ✔️ | | ✔️ | | ✔️ |

-### Operations

-The following table shows support for individual APIs in Calling SDK to individual identity types.

-|Operations | Communication Services user | Microsoft 365 user |

-|--||-|

-| startSpotlight | ✔️ [1] | ✔️ [1] |

-| stopSpotlight | ✔️ | ✔️ |

-| stopAllSpotlight | ✔️ [1] | ✔️ [1] |

-| getSpotlightedParticipants | ✔️ | ✔️ |

-[1] In Teams meeting scenarios, these APIs are only available to users with role organizer, co-organizer, or presenter.

-### SDKs

-The following table shows support for Spotlight feature in individual Azure Communication Services SDKs.

-| Platforms | Web | Web UI | iOS | iOS UI | Android | Android UI | Windows |

-||--|--|--|--|-|--||

-|Is Supported | ✔️ | ✔️ | ✔️ | | ✔️ | | ✔️ |

-description: Make your Microsoft Teams virtual meetings feel more personal with Teams together mode.

-In this article, you learn how to implement Microsoft Teams Together Mode with Azure Communication Services Calling SDKs. This feature enhances virtual meetings and calls, making them feel more personal. By creating a unified view that places everyone in a shared background, participants can connect seamlessly and collaborate effectively.

-|Identities | Teams meeting | Room | 1:1 call | Group call | 1:1 Teams interop call | Group Teams interop call |

-|--|||-|||--|

-|Operations | Communication Services user | Microsoft 365 user |

-|--||-|

-| Start together mode stream | | ✔️ [1] |

-| Get together mode stream | ✔️ | ✔️ |

-||--|--|--|--|-|--||

-|Is Supported | ✔️ | | | | | | |

-## Together Mode

-In this quickstart you are going to learn how to start a call from Azure Communication Services user to Teams Auto Attendant. You are going to achieve it with the following steps:

-1. Enable federation of Azure Communication Services resource with Teams Tenant.

-2. Select or create Teams Auto Attendant via Teams Admin Center.

-3. Get email address of Auto Attendant via Teams Admin Center.

-4. Get Object ID of the Auto Attendant via Graph API.

-5. Start a call with Azure Communication Services Calling SDK.

-If you'd like to skip ahead to the end, you can download this quickstart as a sample on [GitHub](https://github.com/Azure-Samples/communication-services-javascript-quickstarts/tree/main/voice-apps-calling).

-## Create or select Teams Auto Attendant

-Teams Auto Attendant is system that provides an automated call handling system for incoming calls. It serves as a virtual receptionist, allowing callers to be automatically routed to the appropriate person or department without the need for a human operator. You can select existing or create new Auto Attendant via [Teams Admin Center](https://aka.ms/teamsadmincenter).

-Learn more about how to create Auto Attendant using Teams Admin Center [here](/microsoftteams/create-a-phone-system-auto-attendant?tabs=general-info).

-## Find Object ID for Auto Attendant

-After Auto Attendant is created, we need to find correlated Object ID to use it later for calls. Object ID is connected to Resource Account that was attached to Auto Attendant - open [Resource Accounts tab](https://admin.teams.microsoft.com/company-wide-settings/resource-accounts) in Teams Admin and find email of account.

-All required information for Resource Account can be found in [Microsoft Graph Explorer](https://developer.microsoft.com/en-us/graph/graph-explorer) using this email in the search.

-```console

-https://graph.microsoft.com/v1.0/users/lab-test2-cq-@contoso.com

-```

-In results we'll are able to find "ID" field

-```json

- "userPrincipalName": "lab-test2-cq@contoso.com",

- "id": "31a011c2-2672-4dd0-b6f9-9334ef4999db"

-```

-In this quickstart you are going to learn how to start a call from Azure Communication Services user to Teams Call Queue. You are going to achieve it with the following steps:

-1. Enable federation of Azure Communication Services resource with Teams Tenant.

-2. Select or create Teams Call Queue via Teams Admin Center.

-3. Get email address of Call Queue via Teams Admin Center.

-4. Get Object ID of the Call Queue via Graph API.

-5. Start a call with Azure Communication Services Calling SDK.

-If you'd like to skip ahead to the end, you can download this quickstart as a sample on [GitHub](https://github.com/Azure-Samples/communication-services-javascript-quickstarts/tree/main/voice-apps-calling).

-## Create or select Teams Call Queue

-Teams Call Queue is a feature in Microsoft Teams that efficiently distributes incoming calls among a group of designated users or agents. It's useful for customer support or call center scenarios. Calls are placed in a queue and assigned to the next available agent based on a predetermined routing method. Agents receive notifications and can handle calls using Teams' call controls. The feature offers reporting and analytics for performance tracking. It simplifies call handling, ensures a consistent customer experience, and optimizes agent productivity. You can select existing or create new Call Queue via [Teams Admin Center](https://aka.ms/teamsadmincenter).

-Learn more about how to create Call Queue using Teams Admin Center [here](/microsoftteams/create-a-phone-system-call-queue?tabs=general-info).

-## Find Object ID for Call Queue

-After Call queue is created, we need to find correlated Object ID to use it later for calls. Object ID is connected to Resource Account that was attached to call queue - open [Resource Accounts tab](https://admin.teams.microsoft.com/company-wide-settings/resource-accounts) in Teams Admin and find email.

-All required information for Resource Account can be found in [Microsoft Graph Explorer](https://developer.microsoft.com/en-us/graph/graph-explorer) using this email in the search.

-```console

-https://graph.microsoft.com/v1.0/users/lab-test2-cq-@contoso.com

-```

-In results we'll are able to find "ID" field

-```json

- "userPrincipalName": "lab-test2-cq@contoso.com",

- "id": "31a011c2-2672-4dd0-b6f9-9334ef4999db"

-```

-2. Modify the linked service payload from its legacy format to the new pattern. You can either fill in each field from the user interface after changing the type mentioned above, or update the payload directly through the JSON Editor. Refer to the [Linked service properties](#linked-service-properties) section in this article for the supported connection properties. The following examples show the differences in payload for the legacy and new Snowflake connectors:

- **Legacy Snowflake connector JSON payload:**

- ```json

- {

-ΓÇ» ΓÇ» "name": "Snowflake1",

-ΓÇ» ΓÇ» "type": "Microsoft.DataFactory/factories/linkedservices",

- ΓÇ» ΓÇ» "properties": {

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "annotations": [],

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "type": "Snowflake",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "typeProperties": {

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "authenticationType": "Basic",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "connectionString": "jdbc:snowflake://<fake_account>.snowflakecomputing.com/?user=FAKE_USER&db=FAKE_DB&warehouse=FAKE_DW&schema=PUBLIC",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "encryptedCredential": "<placeholder>"

- ΓÇ» ΓÇ» ΓÇ» ΓÇ» },

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "connectVia": {

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "referenceName": "AzureIntegrationRuntime",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "type": "IntegrationRuntimeReference"

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» }

-ΓÇ» ΓÇ» }

- }

- ```

- **New Snowflake connector JSON payload:**

- ```json

- {

-ΓÇ» ΓÇ» "name": "Snowflake2",

-ΓÇ» ΓÇ» "type": "Microsoft.DataFactory/factories/linkedservices",

- ΓÇ» ΓÇ» "properties": {

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "parameters": {

- ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "schema": {

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "type": "string",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "defaultValue": "PUBLIC"

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» }

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» },

- ΓÇ» ΓÇ» ΓÇ» ΓÇ» "annotations": [],

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "type": "SnowflakeV2",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "typeProperties": {

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "authenticationType": "Basic",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "accountIdentifier": "<FAKE_Account",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "user": "FAKE_USER",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "database": "FAKE_DB",

- ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "warehouse": "FAKE_DW",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "encryptedCredential": "<placeholder>"

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» },

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "connectVia": {

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "referenceName": "AutoResolveIntegrationRuntime",

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» ΓÇ» "type": "IntegrationRuntimeReference"

- ΓÇ» ΓÇ» ΓÇ» ΓÇ» }

-ΓÇ» ΓÇ» }

- }

- ```

-1. Update dataset to use the new linked service. You can either create a new dataset based on the newly created linked service, or update an existing dataset's type property from _SnowflakeTable_ to _SnowflakeV2Table_.

-Microsoft Dev Box encrypts the disk by default to protect your data. You don't need to enable BitLocker, and doing so can prevent you from accessing your dev box.

-# Mandatory fields.

-description: Learn how to stream Azure Digital Twins telemetry to clients using Azure SignalR

-# Optional fields. Don't forget to remove # if you need a field.

-#

-#

-#

-# Integrate Azure Digital Twins with Azure SignalR Service

-In this article, you'll learn how to integrate Azure Digital Twins with [Azure SignalR Service](../azure-signalr/signalr-overview.md).

-The solution described in this article allows you to push digital twin telemetry data to connected clients, such as a single webpage or a mobile application. As a result, clients are updated with real-time metrics and status from IoT devices, without the need to poll the server or submit new HTTP requests for updates.

-## Prerequisites

-Here are the prerequisites you should complete before proceeding:

-* Before integrating your solution with Azure SignalR Service in this article, you should complete the Azure Digital Twins [Connect an end-to-end solution](tutorial-end-to-end.md), because this how-to article builds on top of it. The tutorial walks you through setting up an Azure Digital Twins instance that works with a virtual IoT device to trigger digital twin updates. This how-to article will connect those updates to a sample web app using Azure SignalR Service.

-* You'll need the following values from the tutorial:

- - Event Grid topic

- - Resource group

- - App service/function app name

-

-* You'll need [Node.js](https://nodejs.org/) installed on your machine.

-Be sure to sign in to the [Azure portal](https://portal.azure.com/) with your Azure account, as you'll need to use it in this guide.

-### Download the sample applications

-First, download the required sample apps. You'll need both of the following samples:

-* [Azure Digital Twins end-to-end samples](/samples/azure-samples/digital-twins-samples/digital-twins-samples/): This sample contains an *AdtSampleApp* that holds two Azure functions for moving data around an Azure Digital Twins instance (you can learn about this scenario in more detail in [Connect an end-to-end solution](tutorial-end-to-end.md)). It also contains a *DeviceSimulator* sample application that simulates an IoT device, generating a new temperature value every second.

- - If you haven't already downloaded the sample as part of the tutorial in [Prerequisites](#prerequisites), [navigate to the sample](/samples/azure-samples/digital-twins-samples/digital-twins-samples/) and select the **Browse code** button underneath the title. Doing so will take you to the GitHub repo for the samples, which you can download as a .zip by selecting the **Code** button and **Download ZIP**.

- :::image type="content" source="media/includes/download-repo-zip.png" alt-text="Screenshot of the digital-twins-samples repo on GitHub and the steps for downloading it as a zip." lightbox="media/includes/download-repo-zip.png":::

- This button will download a copy of the sample repo in your machine, as *digital-twins-samples-main.zip*. Unzip the folder.

-* [SignalR integration web app sample](/samples/azure-samples/digitaltwins-signalr-webapp-sample/digital-twins-samples/): This sample React web app will consume Azure Digital Twins telemetry data from an Azure SignalR Service.

- - Navigate to the sample link and use the same download process to download a copy of the sample to your machine, as *digitaltwins-signalr-webapp-sample-main.zip*. Unzip the folder.

-## Solution architecture

-You'll be attaching Azure SignalR Service to Azure Digital Twins through the path below. Sections A, B, and C in the diagram are taken from the architecture diagram of the [end-to-end tutorial prerequisite](tutorial-end-to-end.md). In this how-to article, you'll build section D on the existing architecture, which includes two new Azure functions that communicate with SignalR and client apps.

-## Create Azure SignalR instance

-Next, create an Azure SignalR instance to use in this article by following the instructions in [Create an Azure SignalR Service instance](../azure-signalr/signalr-quickstart-azure-functions-csharp.md#create-an-azure-signalr-service-instance) (for now, only complete the steps in this section).

-Leave the browser window open to the Azure portal, as you'll use it again in the next section.

-## Publish and configure the Azure Functions app

-In this section, you'll set up two Azure functions:

-* *negotiate* - A HTTP trigger function. It uses the *SignalRConnectionInfo* input binding to generate and return valid connection information.

-* *broadcast* - An [Event Grid](../event-grid/overview.md) trigger function. It receives Azure Digital Twins telemetry data through the event grid, and uses the output binding of the SignalR instance you created in the previous step to broadcast the message to all connected client applications.

-Start Visual Studio or another code editor of your choice, and open the code solution in the *digital-twins-samples-main\ADTSampleApp* folder. Then do the following steps to create the functions:

-1. In the *SampleFunctionsApp* project, create a new C# class called *SignalRFunctions.cs*.

-1. Replace the contents of the class file with the following code:

-

- :::code language="csharp" source="~/digital-twins-docs-samples/sdks/csharp/signalRFunction.cs":::

-1. In Visual Studio's **Package Manager Console** window, or any command window on your machine, navigate to the folder *digital-twins-samples-main\AdtSampleApp\SampleFunctionsApp*, and run the following command to install the `SignalRService` NuGet package to the project:

- ```cmd

- dotnet add package Microsoft.Azure.WebJobs.Extensions.SignalRService --version 1.14.0

- ```

- Running this command should resolve any dependency issues in the class.

-1. Publish the function to Azure, using your preferred method.

- For instructions on how to publish the function using **Visual Studio**, see [Develop Azure Functions using Visual Studio](../azure-functions/functions-develop-vs.md#publish-to-azure). For instructions on how to publish the function using **Visual Studio Code**, see [Create a C# function in Azure using Visual Studio Code](../azure-functions/create-first-function-vs-code-csharp.md?tabs=in-process#publish-the-project-to-azure). For instructions on how to publish the function using the **Azure CLI**, see [Create a C# function in Azure from the command line](../azure-functions/create-first-function-cli-csharp.md?tabs=azure-cli%2Cin-process#deploy-the-function-project-to-azure).

-### Configure the function

-Next, configure the function to communicate with your Azure SignalR instance. You'll start by gathering the SignalR instance's connection string, and then add it to the functions app's settings.

-1. Go to the [Azure portal](https://portal.azure.com/) and search for the name of your SignalR instance in the search bar at the top of the portal. Select the instance to open it.

-1. Select **Keys** from the instance menu to view the connection strings for the SignalR service instance.

-1. Select the **Copy** icon to copy the **Primary CONNECTION STRING**.

- :::image type="content" source="media/how-to-integrate-azure-signalr/signalr-keys.png" alt-text="Screenshot of the Azure portal that shows the Keys page for the SignalR instance. The connection string is being copied." lightbox="media/how-to-integrate-azure-signalr/signalr-keys.png":::

-1. Finally, add your Azure SignalR connection string to the function's app settings, using the following Azure CLI command. Also, replace the placeholders with your resource group and app service/function app name from the [tutorial prerequisite](how-to-integrate-azure-signalr.md#prerequisites). The command can be run in [Azure Cloud Shell](https://shell.azure.com), or locally if you have the [Azure CLI installed on your machine](/cli/azure/install-azure-cli):

-

- ```azurecli-interactive

- az functionapp config appsettings set --resource-group <your-resource-group> --name <your-function-app-name> --settings "AzureSignalRConnectionString=<your-Azure-SignalR-ConnectionString>"

- ```

- The output of this command prints all the app settings set up for your Azure function. Look for `AzureSignalRConnectionString` at the bottom of the list to verify it's been added.

- :::image type="content" source="media/how-to-integrate-azure-signalr/output-app-setting.png" alt-text="Screenshot of the output in a command window, showing a list item called 'AzureSignalRConnectionString'.":::

-## Connect the function to Event Grid

-Next, subscribe the *broadcast* Azure function to the Event Grid topic you created during the [tutorial prerequisite](how-to-integrate-azure-signalr.md#prerequisites). This action will allow telemetry data to flow from the thermostat67 twin through the Event Grid topic and to the function. From here, the function can broadcast the data to all the clients.

-To broadcast the data, you'll create an Event subscription from your Event Grid topic to your *broadcast* Azure function as an endpoint.

-In the [Azure portal](https://portal.azure.com/), navigate to your Event Grid topic by searching for its name in the top search bar. Select **+ Event Subscription**.

-On the **Create Event Subscription** page, fill in the fields as follows (fields filled by default aren't mentioned):

-* **EVENT SUBSCRIPTION DETAILS** > **Name**: Give a name to your event subscription.

-* **ENDPOINT DETAILS** > **Endpoint Type**: Select **Azure Function** from the menu options.

-* **ENDPOINT DETAILS** > **Endpoint**: Select the **Select an endpoint** link, which will open a **Select Azure Function** window:

- - Fill in your **Subscription**, **Resource group**, **Function app**, and **Function** (**broadcast**). Some of these fields may autopopulate after selecting the subscription.

- - Select **Confirm Selection**.

-Back on the **Create Event Subscription** page, select **Create**.

-At this point, you should see two event subscriptions in the **Event Grid Topic** page.

-## Configure and run the web app

-In this section, you'll see the result in action. First, configure the sample client web app to connect to the Azure SignalR flow you've set up. Next, you'll start up the simulated device sample app that sends device telemetry data through your Azure Digital Twins instance. After that, you'll view the sample web app to see the simulated device data updating the sample web app in real time.

-### Configure the sample client web app

-Next, you'll configure the sample client web app. Start by gathering the HTTP endpoint URL of the *negotiate* function, and then use it to configure the app code on your machine.

-1. Go to the Azure portal's [Function apps](https://portal.azure.com/#blade/HubsExtension/BrowseResource/resourceType/Microsoft.Web%2Fsites/kind/functionapp) page and select your function app from the list. In the app menu, select **Functions** and choose the **negotiate** function.

- :::image type="content" source="media/how-to-integrate-azure-signalr/functions-negotiate.png" alt-text="Screenshot of the Azure portal function apps, with 'Functions' highlighted in the menu and 'negotiate' highlighted in the list of functions.":::

-1. Select **Get function URL** and copy the value up through **/api** (don't include the last **/negotiate?**). You'll use this value in the next step.

- :::image type="content" source="media/how-to-integrate-azure-signalr/get-function-url.png" alt-text="Screenshot of the Azure portal showing the 'negotiate' function with the 'Get function URL' button and the function URL highlighted.":::

-1. Using Visual Studio or any code editor of your choice, open the unzipped _**digitaltwins-signalr-webapp-sample-main**_ folder that you downloaded in the [Download the sample applications](#download-the-sample-applications) section.

-1. Open the *src/App.js* file, and replace the function URL in `HubConnectionBuilder` with the HTTP endpoint URL of the **negotiate** function that you saved in the previous step:

- ```javascript

- const hubConnection = new HubConnectionBuilder()

- .withUrl('<Function-URL>')

- .build();

- ```

-1. In Visual Studio's **Developer command prompt** or any command window on your machine, navigate to the *digitaltwins-signalr-webapp-sample-main\src* folder. Run the following command to install the dependent node packages:

- ```cmd

- npm install

- ```

-Next, set permissions in your function app in the Azure portal:

-1. In the Azure portal's [Function apps](https://portal.azure.com/#blade/HubsExtension/BrowseResource/resourceType/Microsoft.Web%2Fsites/kind/functionapp) page, select your function app instance.

-1. Scroll down in the instance menu and select **CORS**. On the CORS page, add `http://localhost:3000` as an allowed origin by entering it into the empty box. Check the box for **Enable Access-Control-Allow-Credentials** and select **Save**.

- :::image type="content" source="media/how-to-integrate-azure-signalr/cors-setting-azure-function.png" alt-text="Screenshot of the Azure portal showing the CORS Setting in Azure Function.":::

-### Run the device simulator

-During the end-to-end tutorial prerequisite, you [configured the device simulator](tutorial-end-to-end.md#configure-and-run-the-simulation) to send data through an IoT Hub and to your Azure Digital Twins instance.

-Now, start the simulator project located in *digital-twins-samples-main\DeviceSimulator\DeviceSimulator.sln*. If you're using Visual Studio, you can open the project and then run it with this button in the toolbar:

-A console window will open and display simulated device temperature telemetry messages. These messages are being sent through your Azure Digital Twins instance, where they're then picked up by the Azure functions and SignalR.

-You don't need to do anything else in this console, but leave it running while you complete the next step.

-### See the results

-To see the results in action, start the SignalR integration web app sample. You can do so from any console window at the *digitaltwins-signalr-webapp-sample-main\src* location, by running this command:

-```cmd

-npm start

-```

-Running this command will open a browser window running the sample app, which displays a visual temperature gauge. Once the app is running, you should start seeing the temperature telemetry values from the device simulator that propagate through Azure Digital Twins being reflected by the web app in real time.

-## Clean up resources

-If you no longer need the resources created in this article, follow these steps to delete them.

-Using the Azure Cloud Shell or local Azure CLI, you can delete all Azure resources in a resource group with the [az group delete](/cli/azure/group#az-group-delete) command. Removing the resource group will also remove:

-* The Azure Digital Twins instance (from the end-to-end tutorial)

-* The IoT hub and the hub device registration (from the end-to-end tutorial)

-* The Event Grid topic and associated subscriptions

-* The Azure Functions app, including all three functions and associated resources like storage

-* The Azure SignalR instance

-> [!IMPORTANT]

-> Deleting a resource group is irreversible. The resource group and all the resources contained in it are permanently deleted. Make sure that you do not accidentally delete the wrong resource group or resources.

-```azurecli-interactive

-az group delete --name <your-resource-group>

-```

-Finally, delete the project sample folders that you downloaded to your local machine (*digital-twins-samples-main.zip*, *digitaltwins-signalr-webapp-sample-main.zip*, and their unzipped counterparts).

-## Next steps

-In this article, you set up Azure functions with SignalR to broadcast Azure Digital Twins telemetry events to a sample client application.

-Next, learn more about Azure SignalR Service:

-* [What is Azure SignalR Service?](../azure-signalr/signalr-overview.md)

-Or read more about Azure SignalR Service Authentication with Azure Functions:

-* [Azure SignalR Service authentication](../azure-signalr/signalr-tutorial-authenticate-azure-functions.md)

- dns1 IN A 5.4.3.2

- dns2 IN A 4.3.2.1

- server1 IN A 4.4.3.2

- server2 IN A 5.5.4.3

- ftp IN A 3.3.2.1

- IN A 3.3.3.2

- dns1 3600 IN A 5.4.3.2

- dns2 3600 IN A 4.3.2.1

- ftp 3600 IN A 3.3.2.1

- ftp 3600 IN A 3.3.3.2

- server1 3600 IN A 4.4.3.2

- server2 3600 IN A 5.5.4.3

-In this article, you'll know how to manage legal tags in your Azure Data Manager for Energy instance. A Legal tag is the entity that represents the legal status of data in the Azure Data Manager for Energy instance. Legal tag is a collection of properties that governs how data can be ingested and consumed. A legal tag is required for data to be [ingested](concepts-csv-parser-ingestion.md) into your Azure Data Manager for Energy instance. It's also required for the [consumption](concepts-index-and-search.md) of the data from your Azure Data Manager for Energy instance. Legal tags are defined at a data partition level individually.

-While in Azure Data Manager for Energy instance, [entitlement service](concepts-entitlements.md) defines access to data for a given user(s), legal tag defines the overall access to the data across users. A user may have access to manage the data within a data partition however, they may not be able to do so-until certain legal requirements are fulfilled.

-Run the below curl command in Azure Cloud Bash to create a legal tag for a given data partition of your Azure Data Manager for Energy instance.

-Consider an Azure Data Manager for Energy instance named "medstest" with a data partition named "dp1"

-The Create Legal Tag api, internally appends data-partition-id to legal tag name if it isn't already present. For instance, if request has name as: ```legal-tag```, then the create legal tag name would be ```<instancename>-<data-partition-id>-legal-tag```

-The sample response will have data-partition-id appended to the legal tag name and sample response will be:

-Run the below curl command in Azure Cloud Bash to get the legal tag associated with a data partition of your Azure Data Manager for Energy instance.

-Consider an Azure Data Manager for Energy instance named "medstest" with a data partition named "dp1"

-| **Dallas** | [Equinix DA3](https://www.equinix.com/locations/americas-colocation/united-states-colocation/dallas-data-centers/da3/)<br/>[Equinix DA6](https://www.equinix.com/data-centers/americas-colocation/united-states-colocation/dallas-data-centers/da6) | 1 | &cross; | &check; | Aryaka Networks<br/>AT&T Connectivity Plus<br/>AT&T Dynamic Exchange<br/>AT&T NetBond<br/>Cologix<br/>Cox Business Cloud Port<br/>Equinix<br/>GTT<br/>Intercloud<br/>Internet2<br/>Level 3 Communications<br/>MCM Telecom<br/>Megaport<br/>Momentum Telecom<br/>Neutrona Networks<br/>Orange<br/>PacketFabric<br/>Telmex Uninet<br/>Telia Carrier<br/>Telefonica<br/>Transtelco<br/>Verizon<br/>Vodafone<br/>Zayo |

-| **Los Angeles** | [CoreSite LA1](https://www.coresite.com/data-centers/locations/los-angeles/one-wilshire) | 1 | &cross; | &check; | AT&T Dynamic Exchange<br/>CoreSite<br/>China Unicom Global<br/>Cloudflare<br/> Megaport<br/>Momentum Telecom<br/>Neutrona Networks<br/>NTT<br/>Zayo</br></br> |

-| **Miami** | [Equinix MI1](https://www.equinix.com/locations/americas-colocation/united-states-colocation/miami-data-centers/mi1/) | 1 | &cross; | &check; | AT&T Dynamic Exchange<br/>Claro<br/>C3ntro<br/>Equinix<br/>Megaport<br/>Momentum Telecom<br/>Neutrona Networks<br/>PitChile |

-| **Queretaro (Mexico)** | [KIO Networks QR01](https://www.kionetworks.com/es-mx/) | 4 | &cross; | &check; | Cirion Technologies<br/>Equinix<br/>KIO<br/>MCM Telecom<br/>Megaport<br/>Transtelco |

-| **Tokyo2** | [AT TOKYO](https://www.attokyo.com/) | 2 | Japan East | &check; | AT TOKYO<br/>China Telecom Global<br/>China Unicom Global<br/>Colt<br/>DE-CIX<br/>Digital Realty<br/>Equinix<br/>IPC<br/>IX Reach<br/>Megaport<br/>PCCW Global Limited<br/>Tokai Communications |

-| **Washington DC** | [Equinix DC2](https://www.equinix.com/locations/americas-colocation/united-states-colocation/washington-dc-data-centers/dc2/)<br/>[Equinix DC6](https://www.equinix.com/data-centers/americas-colocation/united-states-colocation/washington-dc-data-centers/dc6) | 1 | East US<br/>East US 2 | &check; | Aryaka Networks<br/>AT&T NetBond<br/>British Telecom<br/>CenturyLink Cloud Connect<br/>Cologix<br/>Colt<br/>Comcast<br/>Coresite<br/>Cox Business Cloud Port<br/>Crown Castle<br/>Digital Realty<br/>Equinix<br/>IPC<br/>Internet2<br/>InterCloud<br/>IPC<br/>Iron Mountain<br/>IX Reach<br/>Level 3 Communications<br/>Lightpath<br/>Megaport<br/>Momentum Telecom<br/>Neutrona Networks<br/>NTT Communications<br/>Orange<br/>PacketFabric<br/>SES<br/>Sprint<br/>Tata Communications<br/>Telia Carrier<br/>Telefonica<br/>Verizon<br/>Zayo |

-| **[Neutrona Networks](https://flo.net/)** | &check; | &check; | Dallas<br/>Los Angeles<br/>Miami<br/>Sao Paulo<br/>Washington DC |

-| **[Transtelco](https://transtelco.net/enterprise-services/)** | &check; | &check; | Dallas<br/>Queretaro(Mexico City)|

-> The portal experience for exporting definitions to GitHub was deprecated in April 2023.

-Get-AzPolicyDefinition -Name 'b2982f36-99f2-4db5-8eff-283140c09693' | ConvertTo-Json -Depth 10

-Azure Resource Graph gives the ability to query at scale with complex filtering, grouping and sorting. Azure Resource Graph supports the policy resources table, which contains policy resources such as definitions, assignments and exemptions. Review our [sample queries.](../samples/resource-graph-samples.md#azure-policy). The Resource Graph explorer portal experience allows downloads of query results to CSV using the ["Download to CSV"](../../resource-graph/first-query-portal.md#download-query-results-as-a-csv-file) toolbar option.

-While we defined the expected state of the resource, we havenn't defined what we want done with non-compliant resources. Azure Policy supports many [effects](../concepts/effect-basics.md). For this tutorial, we define the business requirement as preventing the creation of resources if they aren't compliant with the business rules. To meet this goal, we use the [deny](../concepts/effect-deny.md) effect. We also want the option to suspend the policy for specific assignments. Use the [disabled](../concepts/effect-disabled.md) effect and make the effect a [parameter](../concepts/definition-structure-parameters.md) in the policy definition.

-In this article, you'll learn how to obtain an access token for the Azure API for FHIR using the Azure CLI. When you [provision the Azure API for FHIR](fhir-paas-portal-quickstart.md), you configure a set of users or service principals that have access to the service. If your user object ID is in the list of allowed object IDs, you can access the service using a token obtained using the Azure CLI.

-The Azure API for FHIR uses a `resource` or `Audience` with URI equal to the URI of the FHIR server `https://<FHIR ACCOUNT NAME>.azurehealthcareapis.com`. You can obtain a token and store it in a variable (named `$token`) with the following command:

-In this article, you've learned how to obtain an access token for the Azure API for FHIR using the Azure CLI. To learn how to access the FHIR API using Postman, proceed to the Postman tutorial.

-FHIR&#174; is a registered trademark of [HL7](https://hl7.org/fhir/) and is used with the permission of HL7.

-This article outlines the basic steps to get started with Azure API for FHIR. Azure API for FHIR is a managed, standards-based, compliant API for clinical health data that enables solutions for actionable analytics and machine learning.

-As a prerequisite, you'll need an Azure subscription and have been granted proper permissions to create Azure resource groups and deploy Azure resources. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.

-After youΓÇÖve located the Azure API for FHIR resource, select **Create**.

-When you're working with healthcare data, it's important to ensure that the data is secure, and it can't be accessed by unauthorized users or applications. FHIR servers use [OAuth 2.0](https://oauth.net/2/) to ensure this data security. Azure API for FHIR is secured using [Microsoft Entra ID](../../active-directory/index.yml), which is an example of an OAuth 2.0 identity provider. [Microsoft Entra identity configuration for Azure API for FHIR](././../azure-api-for-fhir/azure-active-directory-identity-configuration.md) provides an overview of FHIR server authorization, and the steps needed to obtain a token to access a FHIR server. While these steps apply to any FHIR server and any identity provider, this article will walk you through Azure API for FHIR as the FHIR server and Microsoft Entra ID as our identity provider. For more information about accessing Azure API for FHIR, see [Access control overview](././../azure-api-for-fhir/azure-active-directory-identity-configuration.md#access-control-overview).

-How Azure API for FHIR validates the access token will depend on implementation and configuration. The article [Azure API for FHIR access token validation](azure-api-fhir-access-token-validation.md) will guide you through the validation steps, which can be helpful when troubleshooting access issues.

-FHIR&#174; is a registered trademark of [HL7](https://hl7.org/fhir/) and is used with the permission of HL7.

-The Fast Healthcare Interoperability Resources (FHIR&#174;) specification defines a set of search parameters for all resources and search parameters that are specific to a resource(s). However, there are scenarios where you might want to search against an element in a resource that isnΓÇÖt defined by the FHIR specification as a standard search parameter. This article describes how you can define your own [search parameters](https://www.hl7.org/fhir/searchparameter.html) to be used in the Azure API for FHIR.

-> Each time you create, update, or delete a search parameter youΓÇÖll need to run a [reindex job](how-to-run-a-reindex.md) to enable the search parameter to be used in production. Below we will outline how you can test search parameters before reindexing the entire FHIR server.

-To create a new search parameter, you `POST` the `SearchParameter` resource to the database. The code example below shows how to add the [US Core Race SearchParameter](http://hl7.org/fhir/us/core/STU3.1.1/SearchParameter-us-core-race.html) to the `Patient` resource.

-> The new search parameter will appear in the capability statement of the FHIR server after you POST the search parameter to the database **and** reindex your database. Viewing the `SearchParameter` in the capability statement is the only way tell if a search parameter is supported in your FHIR server. If you can find the search parameter by searching for the search parameter but cannot see it in the capability statement, you still need to index the search parameter. You can POST multiple search parameters before triggering a reindex operation.

-Important elements of a `SearchParameter`:

-* **url**: A unique key to describe the search parameter. Many organizations, such as HL7, use a standard URL format for the search parameters that they define, as shown above in the US Core race search parameter.

-* **code**: The value stored in **code** is what youΓÇÖll use when searching. For the example above, you would search with `GET {FHIR_URL}/Patient?race=<code>` to get all patients of a specific race. The code must be unique for the resource(s) the search parameter applies to.

-* **base**: Describes which resource(s) the search parameter applies to. If the search parameter applies to all resources, you can use `Resource`; otherwise, you can list all the relevant resources.

-* **expression**: Describes how to calculate the value for the search. When describing a search parameter, you must include the expression, even though it isn't required by the specification. This is because you need either the expression or the xpath syntax and the Azure API for FHIR ignores the xpath syntax.

-While you canΓÇÖt use the search parameters in production until you run a reindex job, there are a few ways to test your search parameters before reindexing the entire database.

-First, you can test your new search parameter to see what values will be returned. By running the command below against a specific resource instance (by inputting their ID), you'll get back a list of value pairs with the search parameter name and the value stored for the specific patient. This will include all of the search parameters for the resource and you can scroll through to find the search parameter you created. Running this command won't change any behavior in your FHIR server.

-For example, to find all search parameters for a patient:

-The result will look like this:

-Once you see that your search parameter is displaying as expected, you can reindex a single resource to test searching with the element. First you'll reindex a single resource:

-Running this, sets the indices for any search parameters for the specific resource that you defined for that resource type. This does make an update to the FHIR server. Now you can search and set the use partial indices header to true, which means that it will return results where any of the resources has the search parameter indexed, even if not all resources of that type have it indexed.

-Continuing with our example above, you could index one patient to enable the US Core Race `SearchParameter`:

-And then search for patients that have a specific race:

-After you have tested and are satisfied that your search parameter is working as expected, run or schedule your reindex job so the search parameters can be used in the FHIR server for production use cases.

-> If you don't know the ID for your search parameter, you can search for it. Using `GET {{FHIR_URL}}/SearchParameter` will return all custom search parameters, and you can scroll through the search parameter to find the search parameter you need. You could also limit the search by name. With the example below, you could search for name using `USCoreRace: GET {{FHIR_URL}}/SearchParameter?name=USCoreRace`.

-The result will be an updated `SearchParameter` and the version will increment.

-If you need to delete a search parameter, use the following:

-In this article, youΓÇÖve learned how to create a search parameter. Next you can learn how to reindex your FHIR server.

-FHIR&#174; is a registered trademark of [HL7](https://hl7.org/fhir/) and is used with the permission of HL7.

-There are scenarios where you may have search or sort parameters in the Azure API for FHIR that haven't yet been indexed. This scenario is relevant when you define your own search parameters. Until the search parameter is indexed, it can't be used in search. This article covers how to run a reindex job to index search parameters that haven't yet been indexed in your FHIR service database.

-Reindex job can be executed against entire FHIR service database and against specific custom search parameter.

-To run reindex job, use the following `POST` call with the JSON formatted `Parameters` resource in the request body:

-If the request is successful, you receive a **201 Created** status code in addition to a `Parameters` resource in the response.

-To run reindex job against specific custom search parameter, use the following `POST` call with the JSON formatted `Parameters` resource in the request body:

-> To check the status of a reindex job or to cancel the job, you'll need the reindex ID. This is the `"id"` carried in the `"parameter"` value returned in the response. In the example above, the ID for the reindex job would be `560c7c61-2c70-4c54-b86d-c53a9d29495e`.

-Once youΓÇÖve started a reindex job, you can check the status of the job using the following call:

-An example response:

-The following information is shown in the above response:

-* `progress`: Reindex job percent complete. Equals `resourcesSuccessfullyReindexed`/`totalResourcesToReindex` x 100.

-* 'resourceReindexProgressByResource (CountReindexed of Count)': Provides reindexed count of the total count, per resource type. In cases where reindexing for a specific resource type is queued, only Count is provided.

-* 'searchParams': Lists url of the search parameters impacted by the reindex job.

-A reindex job can be quite performance intensive. WeΓÇÖve implemented some throttling controls to help you manage how a reindex job will run on your database.

-> It is not uncommon on large datasets for a reindex job to run for days. For a database with 30,000,000 million resources, we noticed that it took 4-5 days at 100K RUs to reindex the entire database.

-Below is a table outlining the available parameters, defaults, and recommended ranges. You can use these parameters to either speedup the process (use more compute) or slow down the process (use less compute). For example, you could run the reindex job on a low traffic time and increase your compute to get it done quicker. Instead, you could use the settings to ensure a low usage of compute and have it run for days in the background.

-| QueryDelayIntervalInMilliseconds | The delay between each batch of resources being kicked off during the reindex job. A smaller number will speed up the job while a higher number will slow it down. | 500 MS (.5 seconds) | 50-500000 |

-If you want to use any of the parameters above, you can pass them into the Parameters resource when you start the reindex job.

-In this article, youΓÇÖve learned how to start a reindex job. To learn how to define new search parameters that require the reindex job, see

-FHIR&#174; is a registered trademark of [HL7](https://hl7.org/fhir/) and is used with the permission of HL7.

-Observability provides visibility into every layer of your Azure IoT Operations configuration. It gives you insight into the actual behavior of issues, which increases the effectiveness of site reliability engineering. Azure IoT Operations offers observability through custom curated Grafana dashboards that are hosted in Azure. These dashboards are powered by Azure Monitor managed service for Prometheus and by Container Insights. This article shows you how to configure the services you need for observability.

-The steps in this section install shared monitoring resources and configure your Arc enabled cluster to emit observability signals to these resources. The shared monitoring resources include Azure Managed Grafana, Azure Monitor Workspace, Azure Managed Prometheus, Azure Log Analytics, and Container Insights.

-1. In your console, go to the local folder where you want to clone the Azure IoT Operations repo:

- > [!NOTE]

- > The repo contains the deployment definition of Azure IoT Operations, and samples that include the sample dashboards used in this article.

-1. Clone the repo to your local machine, using the following command:

- ```shell

- git clone https://github.com/Azure/azure-iot-operations.git

- ```

-1. Browse to the following path in your local copy of the repo:

- *azure-iot-operations\tools\setup-3p-obs-infra*

-1. To deploy the observability components, run the following command. Use the subscription ID and resource group of your Arc-enabled cluster that you want to monitor.

- > [!NOTE]

- > To discover other optional parameters you can set, see the [bicep file](https://github.com/Azure/azure-iot-operations/blob/main/tools/setup-3p-obs-infra/observability-full.bicep). The optional parameters can specify things like alternative locations for cluster resources.

-

- ```azurecli

- az deployment group create \

- --subscription <subscription-id> \

- --resource-group <cluster-resource-group> \

- --template-file observability-full.bicep \

- --parameters grafanaAdminId=$(az ad user show --id $(az account show --query user.name --output tsv) --query=id --output tsv) \

- clusterName=<cluster-name> \

- sharedResourceGroup=<shared-resource-group> \

- sharedResourceLocation=<shared-resource-location> \

- --query=properties.outputs

- ```

-

- The previous command grants admin access for the newly created Grafana instance to the user who runs it. If that access isn't what you want, run the following command instead. You need to set up permissions manually before anyone can access the Grafana instance.

-

- ```azurecli

- az deployment group create \

- --subscription <subscription-id> \

- --resource-group <cluster-resource-group> \

- --template-file observability-full.bicep \

- --parameters clusterName=<cluster-name> \

- sharedResourceGroup=<shared-resource-group> \

- sharedResourceLocation=<shared-resource-location> \

- --query=properties.outputs

- ```

- To set up permissions manually, [add a role assignment](../../managed-grafan#add-a-grafana-role-assignment) to the Grafana instance for any users who should have access. Assign one of the Grafana roles (Grafana Admin, Grafana Editor, Grafana Viewer) depending on the level of access desired.

-If the deployment succeeds, a few pieces of information are printed at the end of the command output. The information includes the Grafana URL and the resource IDs for both the Log Analytics and Azure Monitor resources that were created. The Grafana URL allows you to go to the Grafana instance that you configure in [Deploy dashboards to Grafana](#deploy-dashboards-to-grafana). The two resource IDs enable you to configure other Arc enabled clusters by following the steps in [Add an Arc-enabled cluster to existing observability infrastructure](howto-add-cluster.md).

-1. Copy and paste the following configuration to a new file named *ama-metrics-prometheus-config.yaml*, and save the file:

-

- ```yml

- apiVersion: v1

- data:

- prometheus-config: |2-

- scrape_configs:

- - job_name: e4k

- scrape_interval: 1m

- static_configs:

- - targets:

- - aio-mq-diagnostics-service.azure-iot-operations.svc.cluster.local:9600

- - job_name: nats

- scrape_interval: 1m

- static_configs:

- - targets:

- - aio-dp-msg-store-0.aio-dp-msg-store-headless.azure-iot-operations.svc.cluster.local:7777

- - job_name: otel

- scrape_interval: 1m

- static_configs:

- - targets:

- - aio-otel-collector.azure-iot-operations.svc.cluster.local:8889

- - job_name: aio-annotated-pod-metrics

- kubernetes_sd_configs:

- - role: pod

- relabel_configs:

- - action: drop

- regex: true

- source_labels:

- - __meta_kubernetes_pod_container_init

- - action: keep

- regex: true

- source_labels:

- - __meta_kubernetes_pod_annotation_prometheus_io_scrape

- - action: replace

- regex: ([^:]+)(?::\\d+)?;(\\d+)

- replacement: $1:$2

- source_labels:

- - __address__

- - __meta_kubernetes_pod_annotation_prometheus_io_port

- target_label: __address__

- - action: replace

- source_labels:

- - __meta_kubernetes_namespace

- target_label: kubernetes_namespace

- - action: keep

- regex: 'azure-iot-operations'

- source_labels:

- - kubernetes_namespace

- scrape_interval: 1m

- kind: ConfigMap

- metadata:

- name: ama-metrics-prometheus-config

- namespace: kube-system

- ```

-1. To apply the configuration file you created, run the following command:

- `kubectl apply -f ama-metrics-prometheus-config.yaml`

-Complete the following steps to install the Azure IoT Operations curated Grafana dashboards.

-* **Conversion (optional)**: Modifying the input fields to fit into the output fields. Conversion is required when multiple input fields are combined into a single output field.

- - 'Payload.He said: "No. It's done"'

- In this case, the path is split into the segments `Payload`, `He said: "No`, and `It's done"` (starting with a space).

- - *

- output: *

- - ColorProperties.*

- output: *

- - TextureProperties.*

- output: *

- - *.Max # - $1

- - *.Min # - $2

- output: ColorProperties.*

- conversion: ($1 + $2) / 2

- - *.Max # - $1

- - *.Min # - $2

- - *.Avg # - $3

- - *.Mean # - $4

- output: ColorProperties.*

- - *.Max # - $1

- - *.Min # - $2

- - *.Mid.Avg # - $3

- - *.Mid.Mean # - $4

- output: ColorProperties.*

- - *.Max # - $1

- - *.Min # - $2

- output: ColorProperties.*.Avg

- - *.Max # - $1

- - *.Min # - $2

- output: ColorProperties.*.Diff

- - *.Max # - $1

- - *.Min # - $2

- output: ColorProperties.*

- - *.Max # - $1

- - *.Min # - $2

- output: ColorProperties.*

- - $context(position).*

- output: Employment.*

-To get started with dataflows, you need to configure endpoints. An endpoint is the connection point for the dataflow. You can use an endpoint as a source or destination for the dataflow. Some endpoint types can be used as [both sources and destinations](#endpoint-types-for-use-as-sources-and-destinations), while others are for [destinations only](#endpoint-type-for-destinations-only). A dataflow needs at least one source endpoint and one destination endpoint.

-The following example shows a custom resource definition with all of the configuration options. The required fields are dependent on the endpoint type. Review the sections for each endpoint type for configuration guidance.

-```yaml

-apiVersion: connectivity.iotoperations.azure.com/v1beta1

-kind: DataflowEndpoint

-metadata:

- name: <endpoint-name>

-spec:

- endpointType: <endpointType> # mqtt, kafka, or localStorage

- authentication:

- method: <method> # systemAssignedManagedIdentity, x509Credentials, userAssignedManagedIdentity, or serviceAccountToken

- systemAssignedManagedIdentitySettings: # Required if method is systemAssignedManagedIdentity

- audience: https://eventgrid.azure.net

- ### OR

- # x509CredentialsSettings: # Required if method is x509Credentials

- # certificateSecretName: x509-certificate

- ### OR

- # userAssignedManagedIdentitySettings: # Required if method is userAssignedManagedIdentity

- # clientId: <id>

- # tenantId: <id>

- # audience: https://eventgrid.azure.net

- ### OR

- # serviceAccountTokenSettings: # Required if method is serviceAccountToken

- # audience: my-audience

- mqttSettings: # Required if endpoint type is mqtt

- host: example.westeurope-1.ts.eventgrid.azure.net:8883

- tls: # Omit for no TLS or MQTT.

- mode: <mode> # enabled or disabled

- trustedCaCertificateConfigMap: ca-certificates

- sharedSubscription:

- groupMinimumShareNumber: 3 # Required if shared subscription is enabled.

- groupName: group1 # Required if shared subscription is enabled.

- clientIdPrefix: <prefix>

- retain: keep

- sessionExpirySeconds: 3600

- qos: 1

- protocol: mqtt

- maxInflightMessages: 100

-```

-| Name | Description |

-|-|--|

-| `endpointType` | Type of the endpoint. Values: `mqtt`, `kafka`, `dataExplorer`, `dataLakeStorage`, `fabricOneLake`, or `localStorage`. |

-| `authentication.method` | Method of authentication. Values: `systemAssignedManagedIdentity`, `x509Credentials`, `userAssignedManagedIdentity`, or `serviceAccountToken`. |

-| `authentication.systemAssignedManagedIdentitySettings.audience` | Audience of the service to authenticate against. Defaults to `https://eventgrid.azure.net`. |

-| `authentication.x509CredentialsSettings.certificateSecretName` | Secret name of the X.509 certificate. |

-| `authentication.userAssignedManagedIdentitySettings.clientId` | Client ID for the user-assigned managed identity. |

-| `authentication.userAssignedManagedIdentitySettings.tenantId` | Tenant ID. |

-| `authentication.userAssignedManagedIdentitySettings.audience` | Audience of the service to authenticate against. Defaults to `https://eventgrid.azure.net`. |

-| `authentication.serviceAccountTokenSettings.audience` | Audience of the service account. Optional, defaults to the broker internal service account audience. |

-| `mqttSettings.host` | Host of the MQTT broker in the form of \<hostname\>:\<port\>. Connects to MQTT broker if omitted.|

-| `mqttSettings.tls` | TLS configuration. Omit for no TLS or MQTT broker. |

-| `mqttSettings.tls.mode` | Enable or disable TLS. Values: `enabled` or `disabled`. Defaults to `disabled`. |

-| `mqttSettings.tls.trustedCaCertificateConfigMap` | Trusted certificate authority (CA) certificate config map. No CA certificate if omitted. No CA certificate works for public endpoints like Azure Event Grid.|

-| `mqttSettings.sharedSubscription` | Shared subscription settings. No shared subscription if omitted. |

-| `mqttSettings.sharedSubscription.groupMinimumShareNumber` | Number of clients to use for shared subscription. |

-| `mqttSettings.sharedSubscription.groupName` | Shared subscription group name. |

-| `mqttSettings.clientIdPrefix` | Client ID prefix. Client ID generated by the dataflow is \<prefix\>-id. No prefix if omitted.|

-| `mqttSettings.retain` | Whether or not to keep the retain setting. Values: `keep` or `never`. Defaults to `keep`. |

-| `mqttSettings.sessionExpirySeconds` | Session expiry in seconds. Defaults to `3600`.|

-| `mqttSettings.qos` | Quality of service. Values: `0` or `1`. Defaults to `1`.|

-| `mqttSettings.protocol` | Use MQTT or web sockets. Values: `mqtt` or `websockets`. Defaults to `mqtt`.|

-| `mqttSettings.maxInflightMessages` | The maximum number of messages to keep in flight. For subscribe, it's the receive maximum. For publish, it's the maximum number of messages to send before waiting for an acknowledgment. Default is `100`. |

-## Endpoint types for use as sources and destinations

-The following endpoint types are used as sources and destinations.

-### MQTT

-MQTT endpoints are used for MQTT sources and destinations. You can configure the endpoint, Transport Layer Security (TLS), authentication, and other settings.

-#### MQTT broker

-To configure an MQTT broker endpoint with default settings, you can omit the host field, along with other optional fields. This configuration allows you to connect to the default MQTT broker without any extra configuration in a durable way, no matter how the broker changes.

-```yaml

-apiVersion: connectivity.iotoperations.azure.com/v1beta1

-kind: DataflowEndpoint

-metadata:

- name: mq

-spec:

- endpointType: mqtt

- authentication:

- method: serviceAccountToken

- serviceAccountTokenSettings:

- audience: aio-mq

- mqttSettings:

- {}

-```

-#### Event Grid

-To configure an Azure Event Grid MQTT broker endpoint, use managed identity for authentication.

-kind: DataflowEndpoint

- name: eventgrid

-spec:

- endpointType: mqtt

- authentication:

- method: systemAssignedManagedIdentity

- systemAssignedManagedIdentitySettings:

- audience: "https://eventgrid.azure.net"

- mqttSettings:

- host: example.westeurope-1.ts.eventgrid.azure.net:8883

- tls:

- mode: Enabled

-```

-#### Other MQTT brokers

-For other MQTT brokers, you can configure the endpoint, TLS, authentication, and other settings as needed.

-```yaml

-spec:

- endpointType: mqtt

- authentication:

- ...

- mqttSettings:

- host: example.mqttbroker.com:8883

- tls:

- mode: Enabled

- trustedCaCertificateConfigMap: <your CA certificate config map>

-```

-Under `authentication`, you can configure the authentication method for the MQTT broker. Supported methods include X.509:

-```yaml

-authentication:

- method: x509Credentials

- x509CredentialsSettings:

- certificateSecretName: <your x509 secret name>

-```

-> [!IMPORTANT]

-> When you use X.509 authentication with an Event Grid MQTT broker, go to the Event Grid namespace > **Configuration** and check these settings:

->

-> - **Enable MQTT**: Select the checkbox.

-> - **Enable alternative client authentication name sources**: Select the checkbox.

-> - **Certificate Subject Name**: Select this option in the dropdown list.

-> - **Maximum client sessions per authentication name**: Set to **3** or more.

->

-> The alternative client authentication and maximum client sessions options allow dataflows to use client certificate subject name for authentication instead of `MQTT CONNECT Username`. This capability is important so that dataflows can spawn multiple instances and still be able to connect. To learn more, see [Event Grid MQTT client certificate authentication](../../event-grid/mqtt-client-certificate-authentication.md) and [Multi-session support](../../event-grid/mqtt-establishing-multiple-sessions-per-client.md).

-System-assigned managed identity:

-```yaml

-authentication:

- method: systemAssignedManagedIdentity

- systemAssignedManagedIdentitySettings:

- # Audience of the service to authenticate against

- # Optional; defaults to the audience for Event Grid MQTT Broker

- audience: https://eventgrid.azure.net

-```

-User-assigned managed identity:

-```yaml

-authentication:

- method: userAssignedManagedIdentity

- userAssignedManagedIdentitySettings:

- clientId: <id>

- tenantId: <id>

-```

-Kubernetes SAT:

-```yaml

-authentication:

- method: serviceAccountToken

- serviceAccountTokenSettings:

- audience: <your service account audience>

-```

-You can also configure shared subscriptions, QoS, MQTT version, client ID prefix, keep alive, clean session, session expiry, retain, and other settings.

-```yaml

- endpointType: mqtt

- mqttSettings:

- sharedSubscription:

- groupMinimumShareNumber: 3

- groupName: group1

- qos: 1

- mqttVersion: v5

- clientIdPrefix: dataflow

- keepRetain: enabled

-### Kafka

-Kafka endpoints are used for Kafka sources and destinations. You can configure the endpoint, TLS, authentication, and other settings.

-#### Azure Event Hubs

-To configure an Azure Event Hubs Kafka, we recommend that you use managed identity for authentication.

-```yaml

-apiVersion: connectivity.iotoperations.azure.com/v1beta1

-kind: DataflowEndpoint

-metadata:

- name: kafka

-spec:

- endpointType: kafka

- authentication:

- method: systemAssignedManagedIdentity

- systemAssignedManagedIdentitySettings:

- audience: <your Event Hubs namespace>.servicebus.windows.net

- kafkaSettings:

- host: <your Event Hubs namespace>.servicebus.windows.net:9093

- tls:

- mode: Enabled

- consumerGroupId: mqConnector

-```

-#### Other Kafka brokers

-For example, to configure a Kafka endpoint, set the host, TLS, authentication, and other settings as needed.

-kind: DataflowEndpoint

- name: kafka

- endpointType: kafka

- authentication:

- ...

- kafkaSettings:

- host: example.kafka.com:9093

- tls:

- mode: Enabled

- consumerGroupId: mqConnector

-```

-Under `authentication`, you can configure the authentication method for the Kafka broker. Supported methods include SASL, X.509, system-assigned managed identity, and user-assigned managed identity.

-```yaml

-authentication:

- method: sasl

- saslSettings:

- saslType: PLAIN

- tokenSecretName: <your token secret name>

- # OR

- method: x509Credentials

- x509CredentialsSettings:

- certificateSecretName: <your x509 secret name>

- # OR

- method: systemAssignedManagedIdentity

- systemAssignedManagedIdentitySettings:

- audience: https://<your Event Hubs namespace>.servicebus.windows.net

- # OR

- method: userAssignedManagedIdentity

- userAssignedManagedIdentitySettings:

- clientId: <id>

- tenantId: <id>

-### Configure settings specific to source endpoints

-For Kafka endpoints, you can configure settings specific for using the endpoint as a source. These settings have no effect if the endpoint is used as a destination.

-```yaml

-spec:

- endpointType: kafka

- kafkaSettings:

- consumerGroupId: fromMq

-```

-### Configure settings specific to destination endpoints

-For Kafka endpoints, you can configure settings specific for using the endpoint as a destination. These settings have no effect if the endpoint is used as a source.

-```yaml

-spec:

- endpointType: kafka

- kafkaSettings:

- compression: gzip

- batching:

- latencyMs: 100

- maxBytes: 1000000

- maxMessages: 1000

- partitionStrategy: static

- kafkaAcks: all

- copyMqttProperties: enabled

-```

-> [!IMPORTANT]

-> By default, data flows don't send MQTT message user properties to Kafka destinations. These user properties include values such as `subject` that stores the name of the asset sending the message. To include user properties in the Kafka message, you must update the `DataflowEndpoint` configuration to include `copyMqttProperties: enabled`.

-## Endpoint type for destinations only

-The following endpoint type is used for destinations only.

-### Local storage and Edge Storage Accelerator

-Use the local storage option to send data to a locally available persistent volume, through which you can upload data via Edge Storage Accelerator edge volumes. In this case, the format must be Parquet.

-```yaml

-apiVersion: connectivity.iotoperations.azure.com/v1beta1

-kind: DataflowEndpoint

-metadata:

- name: esa

-spec:

- endpointType: localStorage

- localStorageSettings:

- persistentVolumeClaimRef: <your PVC name>

-```

- name: my-dataflow-profile

- tolerations:

- ...

- diagnostics:

- logFormat: text

- logLevel: info

- metrics:

- mode: enabled

- cacheTimeoutSeconds: 600

- exportIntervalSeconds: 10

- prometheusPort: 9600

- updateIntervalSeconds: 30

- traces:

- mode: enabled

- cacheSizeMegabytes: 16

- exportIntervalSeconds: 10

- openTelemetryCollectorAddress: null

- selfTracing:

- mode: enabled

- frequencySeconds: 30

- spanChannelCapacity: 100

-| Field Name | Description |

-|-|--|

-| `instanceCount` | Number of instances to spread the dataflow across. Optional; automatically determined if not set. Currently in the preview release, set the value to `1`. |

-| `tolerations` | Node tolerations. Optional; see [Kubernetes Taints and Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/). |

-| `diagnostics` | Diagnostics settings. |

-| `diagnostics.logFormat` | Format of the logs. For example, `text`. |

-| `diagnostics.logLevel` | Log level. For example, `info`, `debug`, `error`. Optional; defaults to `info`. |

-| `diagnostics.metrics` | Metrics settings. |

-| `diagnostics.metrics.mode` | Mode for metrics. For example, `enabled`. |

-| `diagnostics.metrics.cacheTimeoutSeconds` | Cache timeout for metrics in seconds. |

-| `diagnostics.metrics.exportIntervalSeconds` | Export interval for metrics in seconds. |

-| `diagnostics.metrics.prometheusPort` | Port for Prometheus metrics. |

-| `diagnostics.metrics.updateIntervalSeconds` | Update interval for metrics in seconds. |

-| `diagnostics.traces` | Traces settings. |

-| `diagnostics.traces.mode` | Mode for traces. For example, `enabled`. |

-| `diagnostics.traces.cacheSizeMegabytes` | Cache size for traces in megabytes. |

-| `diagnostics.traces.exportIntervalSeconds` | Export interval for traces in seconds. |

-| `diagnostics.traces.openTelemetryCollectorAddress` | Address for the OpenTelemetry collector. |

-| `diagnostics.traces.selfTracing` | Self-tracing settings. |

-| `diagnostics.traces.selfTracing.mode` | Mode for self-tracing. For example, `enabled`. |

-| `diagnostics.traces.selfTracing.frequencySeconds`| Frequency for self-tracing in seconds. |

-| `diagnostics.traces.spanChannelCapacity` | Capacity of the span channel. |

-## Default settings

-The default settings for a dataflow profile are:

-* Instances: (null)

-* Log level: Info

-* Node tolerations: None

-* Diagnostic settings: None

- maxInstances: 3

-# Create a dataflow

-A dataflow is the path that data takes from the source to the destination with optional transformations. You can configure the dataflow by using the Azure IoT Operations portal or by creating a *Dataflow* custom resource. Before you create a dataflow, you must [configure dataflow endpoints for the data sources and destinations](howto-configure-dataflow-endpoint.md).

-The following example is a dataflow configuration with an MQTT source endpoint, transformations, and a Kafka destination endpoint:

- profileRef: my-dataflow-profile

- mode: enabled

- - operationType: source

- name: my-source

- endpointRef: mq

- dataSources:

- - thermostats/+/telemetry/temperature/#

- - humidifiers/+/telemetry/humidity/#

- serializationFormat: json

- - operationType: builtInTransformation

- name: my-transformation

- filter:

- - inputs:

- - 'temperature.Value'

- - '"Tag 10".Value'

- expression: "$1*$2<100000"

- map:

- - inputs:

- - '*'

- output: '*'

- - inputs:

- - temperature.Value

- output: TemperatureF

- expression: cToF($1)

- - inputs:

- - '"Tag 10".Value'

- output: 'Tag 10'

- serializationFormat: json

- - operationType: destination

- name: my-destination

- endpointRef: kafka

- dataDestination: factory

-| Name | Description |

-|--|-|

-| `profileRef` | Reference to the [dataflow profile](howto-configure-dataflow-profile.md). |

-| `mode` | Mode of the dataflow: `enabled` or `disabled`. |

-| `operations[]` | Operations performed by the dataflow. |

-| `operationType` | Type of operation: `source`, `destination`, or `builtInTransformation`. |

-## Configure source

-To configure a source for the dataflow, specify the endpoint reference and data source. You can specify a list of data sources for the endpoint. For example, MQTT or Kafka topics. The following definition is an example of a dataflow configuration with a source endpoint and data source:

-apiVersion: connectivity.iotoperations.azure.com/v1beta1

-kind: Dataflow

-metadata:

- name: mq-to-kafka

- namespace: azure-iot-operations

-spec:

- profileRef: example-dataflow

- operations:

- - operationType: source

- sourceSettings:

- endpointRef: mq-source

- dataSources:

- - azure-iot-operations/data/thermostat

-| Name | Description |

-|--||

-| `operationType` | `source` |

-| `sourceSettings` | Settings for the `source` operation. |

-| `sourceSettings.endpointRef` | Reference to the `source` endpoint. |

-| `sourceSettings.dataSources` | Data sources for the `source` operation. Wildcards ( `#` and `+` ) are supported. |

-## Configure transformation

-The transformation operation is where you can transform the data from the source before you send it to the destination. Transformations are optional. If you don't need to make changes to the data, don't include the transformation operation in the dataflow configuration. Multiple transformations are chained together in stages regardless of the order in which they're specified in the configuration.

- - operationType: builtInTransformation

- name: transform1

- builtInTransformationSettings:

- datasets:

- # ...

- filter:

- # ...

- map:

- # ...

-| Name | Description |

-|--||

-| `operationType` | `builtInTransformation` |

-| `name` | Name of the transformation. |

-| `builtInTransformationSettings` | Settings for the `builtInTransformation` operation. |

-| `builtInTransformationSettings.datasets` | Add other data to the source data given a dataset and condition to match. |

-| `builtInTransformationSettings.filter` | Filter the data based on a condition. |

-| `builtInTransformationSettings.map` | Move data from one field to another with an optional conversion. |

-| Name | Description |

-||-|

-| `builtInTransformationSettings.datasets.key` | Dataset used for enrichment (key in DSS). |

-| `builtInTransformationSettings.datasets.expression` | Condition for the enrichment operation. |

-spec:

- operations:

- - operationType: builtInTransformation

- name: transform1

- builtInTransformationSettings:

- datasets:

- - key: assetDataset

- inputs:

- - $source.deviceId # - $1

- - $context(assetDataset).asset # - $2

- expression: $1 == $2

-| Name | Description |

-||-|

-| `builtInTransformationSettings.filter.inputs[]` | Inputs to evaluate a filter condition. |

-| `builtInTransformationSettings.filter.expression` | Condition for the filter evaluation. |

-spec:

- operations:

- - operationType: builtInTransformation

- name: transform1

- builtInTransformationSettings:

- filter:

- - inputs:

- - temperature ? $last # - $1

- expression: "$1 > 20"

-| Name | Description |

-||-|

-| `builtInTransformationSettings.map[].inputs[]` | Inputs for the map operation |

-| `builtInTransformationSettings.map[].output` | Output field for the map operation |

-| `builtInTransformationSettings.map[].expression` | Conversion formula for the map operation |

-spec:

- operations:

- - operationType: builtInTransformation

- name: transform1

- builtInTransformationSettings:

- map:

- - inputs:

- - temperature # - $1

- output: temperatureCelsius

- expression: "($1 - 32) * 5/9"

- - inputs:

- - $context(assetDataset).location

- output: location

-## Configure destination

-To configure a destination for the dataflow, you need to specify the endpoint and a path (topic or table) for the destination.

-| Name | Description |

-|--|-|

-| `destinationSettings.endpointRef` | Reference to the `destination` endpoint |

-| `destinationSettings.dataDestination` | Destination for the data |

-### Configure destination endpoint reference

-To configure the endpoint for the destination, you need to specify the ID and endpoint reference:

-spec:

- operations:

- - operationType: destination

- name: destination1

- destinationSettings:

- endpointRef: eventgrid

-### Configure destination path

-After you have the endpoint, you can configure the path for the destination. If the destination is an MQTT or Kafka endpoint, use the path to specify the topic:

- destinationSettings:

- endpointRef: eventgrid

- dataDestination: factory

-For storage endpoints like Microsoft Fabric, use the path to specify the table name:

- destinationSettings:

- endpointRef: adls

- dataDestination: telemetryTable

-To communicate with the state store, clients must `PUBLISH` requests to the system topic `statestore/FA9AE35F-2F64-47CD-9BFF-08E2B32A0FE8/command/invoke`. Because the state store is part of Azure IoT Operations, it does an implicit `SUBSCRIBE` to this topic on startup.

- Client->>+State Store:Request<BR>PUBLISH statestore/FA9AE35F-2F64-47CD-9BFF-08E2B32A0FE8/command/invoke<BR>Response Topic:client-defined-response-topic<BR>Correlation Data:1234<BR>Payload(RESP3)

-State store clients request the state store monitor a given `keyName` for changes by sending a `KEYNOTIFY` message. Just like all state store requests, clients PUBLISH a QoS1 message with this message via MQTT5 to the state store system topic `statestore/FA9AE35F-2F64-47CD-9BFF-08E2B32A0FE8/command/invoke`.

-clients/statestore/FA9AE35F-2F64-47CD-9BFF-08E2B32A0FE8/{clientId}/command/notify/{keyName}

-clients/statestore/FA9AE35F-2F64-47CD-9BFF-08E2B32A0FE8/636C69656E742D696431/command/notify/534F4D454B4559`

- * `VALUE` and `{MODIFIED-VALUE}`. `VALUE` is a string literal indicating that the next field, `{MODIFIED-VALUE}`, contains the value the key was changed to. This value is only sent in response to keys being modified because of a `SET` and is only included if the `KEYNOTIFY` request included the optional `GET` flag.

-> | `spec:metadata:hostname` | The MQTT broker hostname. Default is `aio-mq-dmqtt-frontend` |

-> | `spec:metadata:tcpPort` | The MQTT broker port number. Default is `8883` |

- { "name": "aio-ca-trust-bundle", "mountPath": "/var/run/certs/aio-mq-ca-cert" }

- value: aio-mq-dmqtt-frontend

- value: 8883

- value: /var/run/certs/aio-mq-ca-cert/ca.crt

- value: aio-mq-dmqtt-frontend

- value: 8883

- value: /var/run/certs/aio-mq-ca-cert/ca.crt

- aio-mq-broker-auth/group: dapr-workload

- audience: aio-mq

- audience: aio-mq

- mountPath: /var/run/certs/aio-mq-ca-cert/

- value: "aio-mq-dmqtt-frontend"

- value: "8883"

- value: "/var/run/certs/aio-mq-ca-cert/ca.crt"

-| `volumes.aio-mq-ca-cert-chain` | The chain of trust to validate the MQTT broker TLS cert |

- aio-mq-broker-auth/group: dapr-workload

- audience: aio-mq

- mountPath: /var/run/certs/aio-mq-ca-cert/

- audience: aio-mq

- mosquitto_sub -L mqtt://aio-mq-dmqtt-frontend/sensor/window_data

-description: Use the Azure CLI to deploy Azure IoT Operations to an Arc-enabled Kubernetes cluster.

-Deploy Azure IoT Operations Preview to a Kubernetes cluster using the Azure CLI. Once you have Azure IoT Operations deployed, then you can manage and deploy other workloads to your cluster.

- * Resources that you can configure in your Azure IoT Operations solution, like assets, MQTT broker, and dataflows.

-* An Azure IoT Operations *instance* is one part of a deployment. It's the parent resource that bundles the suite of services that are defined in [What is Azure IoT Operations Preview?](../overview-iot-operations.md), like MQ, Akri, and OPC UA connector.

-In this article, when we talk about deploying Azure IoT Operations we mean the full set of components that make up a *deployment*. Once the deployment exists, you can view, manage, and update the *instance*.

-* Azure access permissions. At a minimum, have **Contributor** permissions in your Azure subscription. Depending on the deployment feature flag status you select, you might also need **Microsoft/Authorization/roleAssignments/write** permissions for the resource group that contains your Arc-enabled Kubernetes cluster. You can make a custom role in Azure role-based access control or assign a built-in role that grants this permission. For more information, see [Azure built-in roles for General](../../role-based-access-control/built-in-roles/general.md).

- If you *don't* have role assignment write permissions, you can still deploy Azure IoT Operations by disabling some features. This approach is discussed in more detail in the [Deploy](#deploy) section of this article.

- * In the Azure CLI, use the [az role assignment create](/cli/azure/role/assignment#az-role-assignment-create) command to give permissions. For example, `az role assignment create --assignee sp_name --role "Role Based Access Control Administrator" --scope subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyResourceGroup`

- * In the Azure portal, when you assign privileged admin roles to a user or principal, you can restrict access using conditions. For this scenario, select the **Allow user to assign all roles** condition in the **Add role assignment** page.

- :::image type="content" source="./media/howto-deploy-iot-operations/add-role-assignment-conditions.png" alt-text="Screenshot that shows assigning users highly privileged role access in the Azure portal.":::

-* An Azure key vault that has the **Permission model** set to **Vault access policy**. You can check this setting in the **Access configuration** section of an existing key vault. To create a new key vault, use the [az keyvault create](/cli/azure/keyvault#az-keyvault-create) command:

- az keyvault create --enable-rbac-authorization false --name "<KEYVAULT_NAME>" --resource-group "<RESOURCE_GROUP>"

-* Azure CLI installed on your development machine. For more information, see [How to install the Azure CLI](/cli/azure/install-azure-cli). This scenario requires Azure CLI version 2.53.0 or higher. Use `az --version` to check your version and `az upgrade` to update if necessary.

-* An Azure Arc-enabled Kubernetes cluster. If you don't have one, follow the steps in [Prepare your Azure Arc-enabled Kubernetes cluster](./howto-prepare-cluster.md?tabs=wsl-ubuntu).

- If you deployed Azure IoT Operations to your cluster previously, uninstall those resources before continuing. For more information, see [Update Azure IoT Operations](#update-azure-iot-operations).

- Azure IoT Operations should work on any CNCF-conformant kubernetes cluster. Currently, Microsoft only supports K3s on Ubuntu Linux and WSL, or AKS Edge Essentials on Windows.

- Use the Azure IoT Operations extension for Azure CLI to verify that your cluster host is configured correctly for deployment by using the [verify-host](/cli/azure/iot/ops#az-iot-ops-verify-host) command on the cluster host:

-### [Azure CLI](#tab/cli)

-1. Sign in to Azure CLI interactively with a browser even if you already signed in before. If you don't sign in interactively, you might get an error that says *Your device is required to be managed to access your resource* when you continue to the next step to deploy Azure IoT Operations.

- ```azurecli

- az login

- ```

- > [!NOTE]

- > If you're using GitHub Codespaces in a browser, `az login` returns a localhost error in the browser window after logging in. To fix, either:

- >

- > * Open the codespace in VS Code desktop, and then run `az login` in the terminal. This opens a browser window where you can log in to Azure.

- > * Or, after you get the localhost error on the browser, copy the URL from the browser and use `curl <URL>` in a new terminal tab. You should see a JSON response with the message "You have logged into Microsoft Azure!".

-1. Deploy Azure IoT Operations to your cluster. Use optional flags to customize the [az iot ops init](/cli/azure/iot/ops#az-iot-ops-init) command to fit your scenario.

- By default, the `az iot ops init` command takes the following actions, some of which require that the principal signed in to the CLI has elevated permissions:

- * Set up a service principal and app registration to give your cluster access to the key vault.

- * Configure TLS certificates.

- * Configure a secrets store on your cluster that connects to the key vault.

- * Deploy the Azure IoT Operations resources.

- ```azurecli

- az iot ops init --cluster <CLUSTER_NAME> --resource-group <RESOURCE_GROUP> --kv-id <KEYVAULT_SETTINGS_PROPERTIES_RESOURCE_ID>

- ```

- Use the [optional parameters](/cli/azure/iot/ops#az-iot-ops-init-optional-parameters) to customize your deployment, including:

- | Parameter | Value | Description |

- | | -- | -- |

- | `--add-insecure-listener` | | Add an insecure 1883 port config to the default listener. *Not for production use*. |

- | `--broker-config-file` | Path to JSON file | Provide a configuration file for the MQTT broker. For more information, see [Advanced MQTT broker config](https://github.com/Azure/azure-iot-ops-cli-extension/wiki/Advanced-Mqtt-Broker-Config) and [Configure core MQTT broker settings](../manage-mqtt-broker/howto-configure-availability-scale.md). |

- | `--disable-rsync-rules` | | Disable the resource sync rules on the deployment feature flag if you don't have **Microsoft.Authorization/roleAssignment/write** permissions in the resource group. |

- | `--name` | String | Provide a name for your Azure IoT Operations instance. Otherwise, a default name is assigned. You can view the `instanceName` parameter in the command output. |

- | `--no-progress` | | Disables the deployment progress display in the terminal. |

- | `--simulate-plc` | | Include the OPC PLC simulator that ships with the OPC UA connector. |

- | `--sp-app-id`,<br>`--sp-object-id`,<br>`--sp-secret` | Service principal app ID, service principal object ID, and service principal secret | Include all or some of these parameters to use an existing service principal, app registration, and secret instead of allowing `init` to create new ones. For more information, see [Configure service principal and Key Vault manually](howto-manage-secrets.md#configure-service-principal-and-key-vault-manually). |

- | **MQTT broker configuration** | *Optional*: Replace the default settings for the MQTT broker. For more information, see [Configure core MQTT broker settings](../manage-mqtt-broker/howto-configure-availability-scale.md). |

-1. Select **Next: Automation**.

-1. On the **Automation** tab, provide the following information:

- | Parameter | Value |

- | | -- |

- | **Subscription** | Select the subscription that contains your Azure key vault. |

- | **Azure Key Vault** | Select your Azure key vault. Or, select **Create new**.<br><br>Ensure that your key vault has **Vault access policy** as its permission model. To check this setting, select **Manage selected vault** > **Settings** > **Access configuration**. |

- :::image type="content" source="./media/howto-deploy-iot-operations/deploy-automation.png" alt-text="A screenshot that shows the third tab for deploying Azure IoT Operations from the portal.":::

-1. If you didn't prepare your Azure CLI environment as described in the prerequisites, do so now in a terminal of your choice:

- ```azurecli

- az upgrade

- az extension add --upgrade --name azure-iot-ops

- ```

-1. Sign in to Azure CLI interactively with a browser even if you already signed in before. If you don't sign in interactively, you might get an error that says *Your device is required to be managed to access your resource* when you continue to the next step to deploy Azure IoT Operations.

- ```azurecli

- az login

- ```

-1. Copy the [az iot ops init](/cli/azure/iot/ops#az-iot-ops-init) command from the **Automation** tab in the Azure portal and run it in your terminal.

-While the deployment is in progress, you can watch the resources being applied to your cluster.

-* If your terminal supports it, `init` displays the deployment progress.

- :::image type="content" source="./media/howto-deploy-iot-operations/view-deployment-terminal.png" alt-text="A screenshot that shows the progress of an Azure IoT Operations deployment in a terminal.":::

- Once the **Deploy IoT Operations** phase begins, the text in the terminal becomes a link to view the deployment progress in the Azure portal.

- :::image type="content" source="./media/howto-deploy-iot-operations/view-deployment-portal.png" alt-text="A screenshot that shows the progress of an Azure IoT Operations deployment in the Azure portal." lightbox="./media/howto-deploy-iot-operations/view-deployment-portal.png":::

-* Otherwise, or if you choose to disable the progress interface with `--no-progress` added to the `init` command, you can use kubectl commands to view the pods on your cluster:

- ```bash

- kubectl get pods -n azure-iot-operations

- ```

- It can take several minutes for the deployment to complete. Rerun the `get pods` command to refresh your view.

-After the deployment is complete, use [az iot ops check](/cli/azure/iot/ops#az-iot-ops-check) to evaluate IoT Operations service deployment for health, configuration, and usability. The *check* command can help you find problems in your deployment and configuration.

-```azurecli

-az iot ops check

-```

-You can also check the configurations of topic maps, QoS, and message routes by adding the `--detail-level 2` parameter for a verbose view.

-## Manage Azure IoT Operations

-After deployment, you can use the Azure CLI and Azure portal to view and manage your Azure IoT Operations instance.

-### List instances

-#### [Azure CLI](#tab/cli)

-Use the `az iot ops list` command to see all of the Azure IoT Operations instances in your subscription or resource group.

-The basic command returns all instances in your subscription.

-```azurecli

-az iot ops list

-```

-To filter the results by resource group, add the `--resource-group` parameter.

-```azurecli

-az iot ops list --resource-group <RESOURCE_GROUP>

-```

-#### [Azure portal](#tab/portal)

-1. In the [Azure portal](https://portal.azure.com), search for and select **Azure IoT Operations**.

-1. Use the filters to view Azure IoT Operations instances based on subscription, resource group, and more.

-### View instance

-#### [Azure CLI](#tab/cli)

-Use the `az iot ops show` command to view the properties of an instance.

-```azurecli

-az iot ops show --name <INSTANCE_NAME> --resource-group <RESOURCE_GROUP>

-```

-You can also use the `az iot ops show` command to view the resources in your Azure IoT Operations deployment in the Azure CLI. Add the `--tree` flag to show a tree view of the deployment that includes the specified Azure IoT Operations instance.

-```azurecli

-az iot ops show --name <INSTANCE_NAME> --resource-group <RESOURCE_GROUP> --tree

-```

-The tree view of a deployment looks like the following example:

-```bash

-MyCluster

-Γö£ΓöÇΓöÇ extensions

-Γöé Γö£ΓöÇΓöÇ akvsecretsprovider

-Γöé Γö£ΓöÇΓöÇ azure-iot-operations-ltwgs

-Γöé ΓööΓöÇΓöÇ azure-iot-operations-platform-ltwgs

-ΓööΓöÇΓöÇ customLocations

- ΓööΓöÇΓöÇ MyCluster-cl

- Γö£ΓöÇΓöÇ resourceSyncRules

- ΓööΓöÇΓöÇ resources

- Γö£ΓöÇΓöÇ MyCluster-ops-init-instance

- ΓööΓöÇΓöÇ MyCluster-observability

-```

-You can run `az iot ops check` on your cluster to assess health and configurations of individual Azure IoT Operations components. By default, the command checks MQ but you can [specify the service](/cli/azure/iot/ops#az-iot-ops-check-examples) with `--ops-service` parameter.

-#### [Azure portal](#tab/portal)

-You can view your Azure IoT Operations instance in the Azure portal.

-1. In the [Azure portal](https://portal.azure.com), go to the resource group that contains your Azure IoT Operations instance, or search for and select **Azure IoT Operations**.

-1. Select the name of your Azure IoT Operations instance.

-1. On the **Overview** page of your instance, the **Arc extensions** table displays the resources that were deployed to your cluster.

- :::image type="content" source="../get-started-end-to-end-sample/media/quickstart-deploy/view-instance.png" alt-text="Screenshot that shows the Azure IoT Operations instance on your Arc-enabled cluster." lightbox="../get-started-end-to-end-sample/media/quickstart-deploy/view-instance.png":::

-### Update instance tags and description

-#### [Azure CLI](#tab/cli)

-Use the `az iot ops update` command to edit the tags and description parameters of your Azure IoT Operations instance. The values provided in the `update` command replace any existing tags or description

-```azurecli

-az iot ops update --name <INSTANCE_NAME> --resource-group <RESOURCE_GROUP> --desc "<INSTANCE_DESCRIPTION>" --tags <TAG_NAME>=<TAG-VALUE> <TAG_NAME>=<TAG-VALUE>

-```

-To delete all tags on an instance, set the tags parameter to a null value. For example:

-```azurecli

-az iot ops update --name <INSTANCE_NAME> --resource-group --tags ""

-```

-#### [Azure portal](#tab/portal)

-1. In the [Azure portal](https://portal.azure.com), go to the resource group that contains your Azure IoT Operations instance, or search for and select **Azure IoT Operations**.

-1. Select the name of your Azure IoT Operations instance.

-1. On the **Overview** page of your instance, select **Add tags** or **edit** to modify tags on your instance.

-## Uninstall Azure IoT Operations

-The Azure CLI and Azure portal offer different options for uninstalling Azure IoT Operations.

-If you want to delete an entire Azure IoT Operations deployment, use the Azure CLI.

-If you want to delete an Azure IoT Operations instance but keep the related resources in the deployment, use the Azure portal.

-### [Azure CLI](#tab/cli)

-Use the [az iot ops delete](/cli/azure/iot/ops#az-iot-ops-delete) command to delete the entire Azure IoT Operations deployment from a cluster. The `delete` command evaluates the Azure IoT Operations related resources on the cluster and presents a tree view of the resources to be deleted. The cluster should be online when you run this command.

-The `delete` command removes:

-* The Azure IoT Operations instance

-* Arc extensions

-* Custom locations

-* Resource sync rules

-* Resources that you can configure in your Azure IoT Operations solution, like assets, MQTT broker, and dataflows.

-```azurecli

-az iot ops delete --cluster <CLUSTER_NAME> --resource-group <RESOURCE_GROUP>

-```

-### [Azure portal](#tab/portal)

-1. In the [Azure portal](https://portal.azure.com), go to the resource group that contains your Azure IoT Operations instance, or search for and select **Azure IoT Operations**.

-1. Select the name of your Azure IoT Operations instance.

-1. On the **Overview** page of your instance, select **Delete** your instance.

-1. Review the list of resources that are and aren't deleted as part of this operation, then type the name of your instance and select **Delete** to confirm.

- :::image type="content" source="./media/howto-deploy-iot-operations/delete-instance.png" alt-text="A screenshot that shows deleting an Azure IoT Operations instance in the Azure portal.":::

-## Update Azure IoT Operations

-Currently, there's no support for updating an existing Azure IoT Operations deployment. Instead, uninstall and redeploy a new version of Azure IoT Operations.

-1. Use the [az iot ops delete](/cli/azure/iot/ops#az-iot-ops-delete) command to delete the Azure IoT Operations deployment on your cluster.

- ```azurecli

- az iot ops delete --cluster <CLUSTER_NAME> --resource-group <RESOURCE_GROUP>

- ```

-1. Update the CLI extension to get the latest Azure IoT Operations version.

- ```azurecli

- az extension update --name azure-iot-ops

- ```

-1. Follow the steps in this article to deploy the newest version of Azure IoT Operations to your cluster.

- >[!TIP]

- >Add the `--ensure-latest` flag to the `az iot ops init` command to check that the latest Azure IoT Operations CLI version is installed and raise an error if an upgrade is available.

-description: Create, update, and manage secrets that are required to give your Arc-connected cluster access to Azure resources.

-#CustomerIntent: As an IT professional, I want prepare an Azure-Arc enabled Kubernetes cluster with Key Vault secrets so that I can deploy Azure IoT Operations to it.

-Secrets management in Azure IoT Operations Preview uses Azure Key Vault as the managed vault solution on the cloud and uses the secrets store CSI driver to pull secrets down from the cloud and store them on the edge.

-* An Arc-enabled Kubernetes cluster. For more information, see [Prepare your cluster](./howto-prepare-cluster.md).

-## Configure a secret store on your cluster

-Azure IoT Operations supports Key Vault for storing secrets and certificates. The `az iot ops init` Azure CLI command automates the steps to set up a service principal to give access to the key vault and configure the secrets that you need for running Azure IoT Operations.

-For more information, see [Deploy Azure IoT Operations Preview to an Arc-enabled Kubernetes cluster](howto-deploy-iot-operations.md?tabs=cli).

-## Configure service principal and Key Vault manually

-If the Azure account executing the `az iot ops init` command doesn't have permissions to query the Microsoft Graph and create service principals, you can prepare these upfront and use extra arguments when running the CLI command as described in [Deploy Azure IoT Operations Preview to an Arc-enabled Kubernetes cluster](howto-deploy-iot-operations.md?tabs=cli).

-### Configure service principal for interacting with Key Vault via Microsoft Entra ID

-Follow these steps to create a new Application Registration for the Azure IoT Operations application to use to authenticate to Key Vault.

-First, register an application with Microsoft Entra ID:

-1. In the Azure portal search bar, search for and select **Microsoft Entra ID**.

-1. Select **App registrations** from the **Manage** section of the Microsoft Entra ID menu.

-1. Select **New registration**.

-1. On the **Register an application** page, provide the following information:

- | Field | Value |

- | -- | -- |

- | **Name** | Provide a name for your application. |

- | **Supported account types** | Ensure that **Accounts in this organizational directory only (<YOUR_TENANT_NAME> only - Single tenant)** is selected. |

- | **Redirect URI** | Select **Web** as the platform. You can leave the web address empty. |

-1. Select **Register**.

- When your application is created, you're directed to its resource page.

-1. Copy the **Application (client) ID** from the app registration overview page. You'll use this value as an argument when running Azure IoT Operations deployment with the `az iot ops init` command.

-Next, give your application permissions for Key Vault:

-1. On the resource page for your app, select **API permissions** from the **Manage** section of the app menu.

-1. Select **Add a permission**.

-1. On the **Request API permissions** page, scroll down and select **Azure Key Vault**.

-1. Select **Delegated permissions**.

-1. Check the box to select **user_impersonation** permissions.

-1. Select **Add permissions**.

-Create a client secret that is added to your Kubernetes cluster to authenticate to your key vault:

-1. On the resource page for your app, select **Certificates & secrets** from the **Manage** section of the app menu.

-1. Select **New client secret**.

-1. Provide an optional description for the secret, then select **Add**.

-1. Copy the **Value** from your new secret. You'll use this value later when you run `az iot ops init`.

-Retrieve the service principal Object ID:

-1. On the **Overview** page for your app, under the **Essentials** section, select the **Application name** link under **Managed application in local directory**. This opens the Enterprise Application properties. Copy the Object ID to use when you run `az iot ops init`.

-### Create a key vault

-Create a new Azure Key Vault instance and ensure that it has the **Permission Model** set to **Vault access policy**.

-```bash

-az keyvault create --enable-rbac-authorization false --name "<your unique key vault name>" --resource-group "<the name of the resource group>"

-```

-If you have an existing key vault, you can change the permission model by executing the following:

-```bash

-az keyvault update --name "<your unique key vault name>" --resource-group "<the name of the resource group>" --enable-rbac-authorization false

-```

-You'll need the Key Vault resource ID when you run `az iot ops init`. To retrieve the resource ID, run:

-```bash

-az keyvault show --name "<your unique key vault name>" --resource-group "<the name of the resource group>" --query id -o tsv

-```

-### Set service principal access policy in Key Vault

-The newly created service principal needs **secret** `list` and `get` access policy for the Azure IoT Operations to work with the secret store.

-To manage Key Vault access policies, the principal logged in to the CLI needs sufficient Azure permissions. In the Role Based Access Control (RBAC) model, this permission is included in Key Vault contributor or higher roles.

->[!TIP]

->If you used the logged-in CLI principal to create the key vault, then you probably already have the right permissions. However, if you're pointing to a different or existing key vault then you should check that you have sufficient permissions to set access policies.

-Run the following to assign **secret** `list` and `get` permissions to the service principal.

-```bash

-az keyvault set-policy --name "<your unique key vault name>" --resource-group "<the name of the resource group>" --object-id <Object ID copied from Enterprise Application SP in Microsoft Entra ID> --secret-permissions get list

-```

-### Pass service principal and Key Vault arguments to Azure IoT Operations deployment

-When following the guide [Deploy Azure IoT Operations Preview to an Arc-enabled Kubernetes cluster](howto-deploy-iot-operations.md?tabs=cli), pass in additional flags to the `az iot ops init` command in order to use the preconfigured service principal and key vault.

-The following example shows how to prepare the cluster for Azure IoT Operations without fully deploying it by using `--no-deploy` flag. You can also run the command without this argument for a default Azure IoT Operations deployment.

-```bash

-az iot ops init --cluster "<your cluster name>" --resource-group "<the name of the resource group>" \

- --kv-id <Key Vault Resource ID> \

- --sp-app-id <Application registration App ID (client ID) from Microsoft Entra ID> \

- --sp-object-id <Object ID copied from Enterprise Application in Microsoft Entra ID> \

- --sp-secret "<Client Secret from App registration in Microsoft Entra ID>" \

- --no-deploy

-```

-One step that the `init` command takes is to ensure all Secret Provider Classes (SPCs) required by Azure IoT Operations have a default secret configured in key vault. If a value for the default secret does not exist `init` will create one. This step requires that the principal logged in to the CLI has secret `set` permissions. If you want to use an existing secret as the default SPC secret, you can specify it with the `--kv-sat-secret-name` parameter, in which case the logged in principal only needs secret `get` permissions.

-## Add a secret to an Azure IoT Operations component

-Once you have the secret store set up on your cluster, you can create and add Key Vault secrets.

-1. Create your secret in Key Vault with whatever name and value you need. You can create a secret by using the [Azure portal](https://portal.azure.com) or the [az keyvault secret set](/cli/azure/keyvault/secret#az-keyvault-secret-set) command.

-1. On your cluster, identify the secret provider class (SPC) for the component that you want to add the secret to. For example, `aio-default-spc`. Use the following command to list all SPCs on your cluster:

- ```bash

- kubectl get secretproviderclasses -A

- ```

-1. Open the file in your preferred text editor. If you use k9s, type `e` to edit.

-1. Add the secret object to the list under `spec.parameters.objects.array`. For example:

- ```yml

- spec:

- parameters:

- keyvaultName: my-key-vault

- objects: |

- array:

- - |

- objectName: PlaceholderSecret

- objectType: secret

- objectVersion: ""

- ```

-1. Save your changes and apply them to your cluster. If you use k9s, your changes are automatically applied.

-The CSI driver updates secrets by using a polling interval, therefore the new secret isn't available to the pod until the next polling interval. To update a component immediately, restart the pods for the component. For example, to restart the Data Processor component, run the following commands:

-```console

-kubectl delete pod aio-dp-reader-worker-0 -n azure-iot-operations

-kubectl delete pod aio-dp-runner-worker-0 -n azure-iot-operations

-```

-An Azure Arc-enabled Kubernetes cluster is a prerequisite for deploying Azure IoT Operations Preview. This article describes how to prepare an Azure Arc-enabled Kubernetes cluster before you [Deploy Azure IoT Operations Preview to an Arc-enabled Kubernetes cluster](howto-deploy-iot-operations.md) to run your own workloads. This article includes guidance for both Ubuntu, Windows, and cloud environments.

-> If you want to deploy Azure IoT Operations and run a sample workload, see the [Quickstart: Run Azure IoT Operations Preview in Github Codespaces with K3s](../get-started-end-to-end-sample/quickstart-deploy.md).

-To prepare your Azure Arc-enabled Kubernetes cluster, you need:

-* Hardware that meets the [system requirements](/azure/azure-arc/kubernetes/system-requirements).

-* Azure CLI version 2.46.0 or newer installed on your development machine. Use `az --version` to check your version and `az upgrade` to update if necessary. For more information, see [How to install the Azure CLI](/cli/azure/install-azure-cli).

-* The Azure IoT Operations extension for Azure CLI. Use the following command to add the extension or update it to the latest version:

- * Ensure that your machine has a minimum of 10-GB RAM, 4 vCPUs, and 40-GB free disk space.

- * Review the [AKS Edge Essentials requirements and support matrix](/azure/aks/hybrid/aks-edge-system-requirements).

- * Review the [AKS Edge Essentials networking guidance](/azure/aks/hybrid/aks-edge-concept-networking).

-* Azure CLI version 2.46.0 or newer installed on your development machine. Use `az --version` to check your version and `az upgrade` to update if necessary. For more information, see [How to install the Azure CLI](/cli/azure/install-azure-cli).

-* The Azure IoT Operations extension for Azure CLI. Use the following command to add the extension or update it to the latest version:

-* Review the [K3s requirements](https://docs.k3s.io/installation/requirements).

-Azure IoT Operations also works on Ubuntu in Windows Subsystem for Linux (WSL) on your Windows machine. Use WSL for testing and development purposes only.

-To set up your WSL Ubuntu environment:

-1. [Install Linux on Windows with WSL](/windows/wsl/install).

-1. Enable `systemd`:

- ```bash

- sudo -e /etc/wsl.conf

- ```

- Add the following to _wsl.conf_ and then save the file:

- ```text

- [boot]

- systemd=true

- ```

-1. After you enable `systemd`, [re-enable running windows executables from WSL](https://github.com/microsoft/WSL/issues/8843):

- ```bash

- sudo sh -c 'echo :WSLInterop:M::MZ::/init:PF > /usr/lib/binfmt.d/WSLInterop.conf'

- sudo systemctl unmask systemd-binfmt.service

- sudo systemctl restart systemd-binfmt

- sudo systemctl mask systemd-binfmt.service

- ```

-### [Codespaces](#tab/codespaces)

-* An Azure subscription. If you don't have an Azure subscription, [create one for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.

-* A [GitHub](https://github.com) account.

-* Visual Studio Code installed on your development machine. For more information, see [Download Visual Studio Code](https://code.visualstudio.com/download).

-This section provides steps to prepare and Arc-enable clusters in validated environments on Linux and Windows as well as GitHub Codespaces in the cloud.

-[Azure Kubernetes Service Edge Essentials](/azure/aks/hybrid/aks-edge-overview) is an on-premises Kubernetes implementation of Azure Kubernetes Service (AKS) that automates running containerized applications at scale. AKS Edge Essentials includes a Microsoft-supported Kubernetes platform that includes a lightweight Kubernetes distribution with a small footprint and simple installation experience, making it easy for you to deploy Kubernetes on PC-class or "light" edge hardware.

-The [AksEdgeQuickStartForAio.ps1](https://github.com/Azure/AKS-Edge/blob/main/tools/scripts/AksEdgeQuickStart/AksEdgeQuickStartForAio.ps1) script automates the the process of creating and connecting a cluster, and is the recommended path for deploying Azure IoT Operations on AKS Edge Essentials.

- .\AksEdgeQuickStartForAio.ps1 -SubscriptionId "<SUBSCRIPTION_ID>" -TenantId "<TENANT_ID>" -ResourceGroupName "<RESOURCE_GROUP_NAME>" -Location "<LOCATION>" -ClusterName "<CLUSTER_NAME>"

-Azure IoT Operations should work on any CNCF-conformant kubernetes cluster. For Ubuntu Linux, Microsoft currently supports K3s clusters.

-> [!IMPORTANT]

-> If you're using Ubuntu in Windows Subsystem for Linux (WSL), run all of these steps in your WSL environment, including the Azure CLI steps for configuring your cluster.

-1. Run the K3s installation script:

- curl -sfL https://get.k3s.io | sh -

- For full installation information, see the [K3s quick-start guide](https://docs.k3s.io/quick-start).

-### [Codespaces](#tab/codespaces)

-> [!IMPORTANT]

-> Codespaces are easy to set up quickly and tear down later, but they're not suitable for performance evaluation or scale testing. Use GitHub Codespaces for exploration only.

-1. On the machine where you deployed the Kubernetes cluster, or in your WSL environment, sign in with Azure CLI:

-1. Set environment variables for your Azure subscription, location, a new resource group, and the cluster name as it will show up in your resource group.

-1. Set the Azure subscription context for all commands:

- ```azurecli

- az account set -s $SUBSCRIPTION_ID

- ```

-1. Register the required resource providers in your subscription:

- >[!NOTE]

- >This step only needs to be run once per subscription. To register resource providers, you need permission to do the `/register/action` operation, which is included in subscription Contributor and Owner roles. For more information, see [Azure resource providers and types](../../azure-resource-manager/management/resource-providers-and-types.md).

- ```azurecli

- az provider register -n "Microsoft.ExtendedLocation"

- az provider register -n "Microsoft.Kubernetes"

- az provider register -n "Microsoft.KubernetesConfiguration"

- az provider register -n "Microsoft.IoTOperationsOrchestrator"

- az provider register -n "Microsoft.IoTOperations"

- az provider register -n "Microsoft.DeviceRegistry"

- ```

-1. Use the [az group create](/cli/azure/group#az-group-create) command to create a resource group in your Azure subscription to store all the resources:

- ```azurecli

- az group create --location $LOCATION --resource-group $RESOURCE_GROUP --subscription $SUBSCRIPTION_ID

- ```

-1. Use the [az connectedk8s connect](/cli/azure/connectedk8s#az-connectedk8s-connect) command to Arc-enable your Kubernetes cluster and manage it as part of your Azure resource group:

- ```azurecli

- az connectedk8s connect -n $CLUSTER_NAME -l $LOCATION -g $RESOURCE_GROUP --subscription $SUBSCRIPTION_ID

- ```

-1. Get the `objectId` of the Microsoft Entra ID application that the Azure Arc service uses and save it as an environment variable.

- ```azurecli

- export OBJECT_ID=$(az ad sp show --id bc313c14-388c-4e7d-a58e-70017303ee3b --query id -o tsv)

- ```

-1. Use the [az connectedk8s enable-features](/cli/azure/connectedk8s#az-connectedk8s-enable-features) command to enable custom location support on your cluster. This command uses the `objectId` of the Microsoft Entra ID application that the Azure Arc service uses. Run this command on the machine where you deployed the Kubernetes cluster:

- ```azurecli

- az connectedk8s enable-features -n $CLUSTER_NAME -g $RESOURCE_GROUP --custom-locations-oid $OBJECT_ID --features cluster-connect custom-locations

- ```

-### [Codespaces](#tab/codespaces)

-To verify that your Kubernetes cluster is now Azure Arc-enabled, run the following command:

-## Create sites

-A _site_ is a collection of Azure IoT Operations instances. Sites typically group instances by physical location and make it easier for OT users to locate and manage assets. An IT administrator creates sites and assigns Azure IoT Operations instances to them. To learn more, see [What is Azure Arc site manager (preview)?](/azure/azure-arc/site-manager/overview).

-description: Understand the key components in the Akri services architecture and how they relate to each other. Includes some information about the CNCF version of Akri

- - ignite-2023

-# CustomerIntent: As an industrial edge IT or operations user, I want to understand the key components in the Akri services architecture so that I understand how it works to enable device and asset discovery for my edge solution.

-# Akri services architecture

-This article helps you understand the architecture of the Akri services. After you learn about the core components of the Akri services, you can use them to detect devices and assets, and add them to your Kubernetes cluster.

-The Akri services are a Microsoft-managed commercial version of [Akri](https://docs.akri.sh/), an open-source Cloud Native Computing Foundation (CNCF) project.

-## Core components

-The Akri services consist of the following five components:

-## Custom resource definitions

-A custom resource definition (CRD) is a Kubernetes API extension that lets you define new object types. There are two Akri services CRDs:

-### Akri configuration CRD

-The configuration CRD configures the Akri services. You create configurations that describe the resources to discover and the pod to deploy on a node that discovers a resource. To learn more, see [Akri configuration CRD](https://github.com/project-akri/akri/blob/main/deployment/helm/crds/akri-configuration-crd.yaml). The CRD schema specifies the settings all configurations must have, including the following settings:

-### Akri instance CRD

-Each Akri instance represents an individual resource that's visible to the cluster. For example, if there are five IP cameras visible to the cluster, there are five instances. The instance CRD enables Akri services coordination and resource sharing. These instances store internal state and aren't intended for you to edit. To learn more, see [Resource sharing in-depth](https://docs.akri.sh/architecture/resource-sharing-in-depth).

-## Agent

-The Akri agent implements [Kubernetes Device-Plugins](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/) for discovered resources. The Akri Agent performs the following tasks:

-These tasks, combined with the state stored in the instance, enable multiple nodes to share a resource while respecting the limits defined by the `spec.capacity` setting.

-To learn more, see [Agent in-depth](https://docs.akri.sh/architecture/agent-in-depth).

-## Discovery handlers

-A discovery handler finds devices. Examples of device include:

-The discovery handler reports all discovered devices to the agent. There are often protocol implementations for discovering a set of devices, whether a network protocol like OPC UA or a proprietary protocol. Discovery handlers implement the `DiscoveryHandler` service defined in [`discovery.proto`](https://github.com/project-akri/akri/blob/main/discovery-utils/proto/discovery.proto). A discovery handler is required to register with the agent, which hosts the `Registration` service defined in [`discovery.proto`](https://github.com/project-akri/akri/blob/main/discovery-utils/proto/discovery.proto).

-To learn more, see [Custom Discovery Handlers](https://docs.akri.sh/development/handler-development).

-## Controller

-The goals of the Akri controller are to:

-To achieve these goals, the controller:

-These tasks enable the Akri controller to ensure that protocol brokers and Kubernetes services are running on all nodes and exposing the desired resources, while respecting the limits defined by the `spec.capacity` setting.

-For more information, see the documentation for [Controller In-depth](https://docs.akri.sh/architecture/controller-in-depth).

-mosquitto_sub --host aio-mq-dmqtt-frontend --port 8883 --topic "azure-iot-operations/data/#" -v --debug --cafile /var/run/certs/ca.crt -D CONNECT authentication-method 'K8S-SAT' -D CONNECT authentication-data $(cat /var/run/secrets/tokens/mq-sat)

-mosquitto_sub --host aio-mq-dmqtt-frontend --port 8883 --topic "azure-iot-operations/data/#" -V mqttv5 -F %P --cafile /var/run/certs/ca.crt -D CONNECT authentication-method 'K8S-SAT' -D CONNECT authentication-data $(cat /var/run/secrets/tokens/mq-sat)

-description: How to discover and configure OPC UA data sources at the edge automatically by using the Akri services

-# CustomerIntent: As an industrial edge IT or operations user, I want to discover and create OPC UA data sources in my industrial edge environment so that I can reduce manual configuration overhead.

-# Discover OPC UA data sources using the Akri services

-In this article, you learn how to discover OPC UA data sources automatically. After you deploy Azure IoT Operations Preview, you configure the Akri services to discover OPC UA data sources at the edge. The Akri services create custom resources in your Kubernetes cluster that represent the data sources it discovers. The ability to discover OPC UA data sources removes the need to [manually configure them by using the operations experience web UI](howto-manage-assets-remotely.md).

-> [!IMPORTANT]

-> Currently, you can't use Azure Device Registry to manage the assets that the Akri services discover and create.

-The Akri services enable you to detect and create assets in the address space of an OPC UA server. The OPC UA asset detection generates `AssetType` and `Asset` custom resources for [OPC UA Device Integration (DI) specification](https://reference.opcfoundation.org/DI/v104/docs/) compliant assets.

-## Prerequisites

- ```bash

- kubectl get pods -n azure-iot-operations

- ```

- The output includes a line that shows the Akri agent and discovery pods are running:

- ```output

- NAME READY STATUS RESTARTS AGE

- aio-akri-agent-daemonset-hwpc7 1/1 Running 0 17mk0s

- aio-opc-asset-discovery-wzlnj 1/1 Running 0 8m28s

- ```

-## Configure the OPC UA discovery handler

-To configure the OPC UA discovery handler for asset detection, create a YAML configuration file that contains the values described in this section:

-| Name | Mandatory | Datatype | Default | Comment |

-| - | | -- | - | - |

-| `EndpointUrl` | true | String | null | The OPC UA endpoint URL to use for asset discovery |

-| `AutoAcceptUntrustedCertificates` | true ┬╣ | Boolean | false | Should the client autoaccept untrusted certificates? A certificate can only be autoaccepted as trusted if no nonsuppressible errors occurred during chain validation. For example, a certificate with incomplete chain isn't accepted. |

-| `UseSecurity` | true ┬╣ | Boolean | true | Should the client use a secure connection? |

-| `UserName` | false | String | null | The username for user authentication. ┬▓ |

-| `Password` | false | String | null | The password for user authentication. ┬▓ |

-┬╣ The current version of the discovery handler only supports `UseSecurity=false` and requires `autoAcceptUntrustedCertificates=true`.

-┬▓ A temporary implementation until the Akri services can pass Kubernetes secrets.

-The following example demonstrates discovery of an OPC PLC server. You can add the asset parameters for multiple OPC PLC servers.

-1. To create the YAML configuration file, copy and paste the following content into a new file, and save it as `opcua-configuration.yaml`:

- If you're using the simulated PLC server that was deployed with the Azure IoT Operations Quickstart, you don't need to change the `endpointUrl`. If you have your own OPC UA servers running or are using the simulated PLC servers deployed on Azure, add in your endpoint URL accordingly. Discovery endpoint URLs look like `opc.tcp://<FQDN>:50000/`. To find the FQDNs of your OPC PLC servers, go to your deployment in the Azure portal. For each server, copy and paste the **FQDN** value into your endpoint URLs.

- ```yaml

- apiVersion: akri.sh/v0

- kind: Configuration

- metadata:

- name: aio-akri-opcua-asset

- spec:

- discoveryHandler:

- name: opcua-asset

- discoveryDetails: "opcuaDiscoveryMethod:\n - asset:\n endpointUrl: \" opc.tcp://opcplc-000000:50000\"\n useSecurity: false\n autoAcceptUntrustedCertificates: true\n"

- brokerProperties: {}

- capacity: 1

- ```

-1. To apply the configuration, run the following command:

- ```bash

- kubectl apply -f opcua-configuration.yaml -n azure-iot-operations

- ```

-> [!TIP]

-> In a default Azure IoT Operations deployment, the OPC UA discovery handler is already configured to discover the simulated PLC server. If you want to discover assets connected to additional OPC UA servers, you can add them to the configuration file.

-## Verify the configuration

-To confirm that the asset discovery container is configured and running:

-1. Use the following command to check the pod logs:

- ```bash

- kubectl logs <insert aio-opc-asset-discovery pod name> -n azure-iot-operations

- ```

- A log from the `aio-opc-asset-discovery` pod indicates after a few seconds that the discovery handler registered itself with the Akri

- ```2024-08-01T15:04:12.874Z aio-opc-asset-discovery-4nsgs - Akri OPC UA Asset Discovery (1.0.0-preview-20240708+702c5cafeca2ea49fec3fb4dc6645dd0d89016ee) is starting with the process id: 1

- 2024-08-01T15:04:12.948Z aio-opc-asset-discovery-4nsgs - OPC UA SDK 1.5.374.70 from 07/20/2024 07:37:16

- 2024-08-01T15:04:12.973Z aio-opc-asset-discovery-4nsgs - OPC UA SDK informational version: 1.5.374.70+1ee3beb87993019de4968597d17cb54d5a4dc3c8

- 2024-08-01T15:04:12.976Z aio-opc-asset-discovery-4nsgs - Akri agent registration enabled: True

- 2024-08-01T15:04:13.475Z aio-opc-asset-discovery-4nsgs - Hosting starting

- 2024-08-01T15:04:13.547Z aio-opc-asset-discovery-4nsgs - Overriding HTTP_PORTS '8080' and HTTPS_PORTS ''. Binding to values defined by URLS instead 'http://+:8080'.

- 2024-08-01T15:04:13.774Z aio-opc-asset-discovery-4nsgs - Now listening on: http://:8080

- 2024-08-01T15:04:13.774Z aio-opc-asset-discovery-4nsgs - Application started. Press Ctrl+C to shut down.

- 2024-08-01T15:04:13.774Z aio-opc-asset-discovery-4nsgs - Hosting environment: Production

- 2024-08-01T15:04:13.774Z aio-opc-asset-discovery-4nsgs - Content root path: /app

- 2024-08-01T15:04:13.774Z aio-opc-asset-discovery-4nsgs - Hosting started

- 2024-08-01T15:04:13.881Z aio-opc-asset-discovery-4nsgs - Registering with Agent as HTTP endpoint using own IP from the environment variable POD_IP: 10.42.0.245

- 2024-08-01T15:04:14.875Z aio-opc-asset-discovery-4nsgs - Registered with the Akri agent with name opcua-asset for http://10.42.0.245:8080 with type Network and shared True

- 2024-08-01T15:04:14.877Z aio-opc-asset-discovery-4nsgs - Successfully re-registered OPC UA Asset Discovery Handler with the Akri agent

- 2024-08-01T15:04:14.877Z aio-opc-asset-discovery-4nsgs - Press CTRL+C to exit

- ```

- After about a minute, the Akri services issue the first discovery request based on the configuration:

- ```output

- 2024-08-01T15:04:15.280Z aio-opc-asset-discovery-4nsgs [opcuabroker@311 SpanId:6d3db9751eebfadc, TraceId:e5594cbaf3993749e92b45c88c493377, ParentId:0000000000000000 ConnectionId:0HN5I7CQJPJL0 RequestPath:/v0.DiscoveryHandler/Discover RequestId:0HN5I7CQJPJL0:00000001] - Reading message.

- 2024-08-01T15:04:15.477Z aio-opc-asset-discovery-4nsgs [opcuabroker@311 SpanId:6d3db9751eebfadc, TraceId:e5594cbaf3993749e92b45c88c493377, ParentId:0000000000000000 ConnectionId:0HN5I7CQJPJL0 RequestPath:/v0.DiscoveryHandler/Discover RequestId:0HN5I7CQJPJL0:00000001] - Received discovery request from ipv6:[::ffff:10.42.0.241]:48638

- 2024-08-01T15:04:15.875Z aio-opc-asset-discovery-4nsgs [opcuabroker@311 SpanId:6d3db9751eebfadc, TraceId:e5594cbaf3993749e92b45c88c493377, ParentId:0000000000000000 ConnectionId:0HN5I7CQJPJL0 RequestPath:/v0.DiscoveryHandler/Discover RequestId:0HN5I7CQJPJL0:00000001] - Start asset discovery

- 2024-08-01T15:04:15.882Z aio-opc-asset-discovery-4nsgs [opcuabroker@311 SpanId:6d3db9751eebfadc, TraceId:e5594cbaf3993749e92b45c88c493377, ParentId:0000000000000000 ConnectionId:0HN5I7CQJPJL0 RequestPath:/v0.DiscoveryHandler/Discover RequestId:0HN5I7CQJPJL0:00000001] - Discovering OPC UA opc.tcp://opcplc-000000:50000 using Asset Discovery

- 2024-08-01T15:04:15.882Z aio-opc-asset-discovery-4nsgs [opcuabroker@311 SpanId:6d3db9751eebfadc, TraceId:e5594cbaf3993749e92b45c88c493377, ParentId:0000000000000000 ConnectionId:0HN5I7CQJPJL0 RequestPath:/v0.DiscoveryHandler/Discover RequestId:0HN5I7CQJPJL0:00000001] - Selected AutoAcceptUntrustedCertificates mode: False

- ```

- After the discovery is complete, the discovery handler sends the result back to the Akri services to create an Akri instance custom resource with asset information and observable variables. The discovery handler repeats the discovery every 10 minutes to detect any changes on the server.

-1. To view the discovered Akri instances, run the following command:

- ```bash

- kubectl get akrii -n azure-iot-operations

- ```

- The output from the previous command looks like the following example. You might need to wait for a few seconds for the Akri instance to be created:

- ```output

- NAME CONFIG SHARED NODES AGE

- akri-opcua-asset-dbdef0 akri-opcua-asset true ["k3d-k3s-default-server-0"] 24h

- ```

- The connector for OPC UA supervisor watches for new Akri instance custom resources of type `opc-ua-asset`, and generates the initial asset types and asset custom resources for them. You can modify asset custom resources by adding settings such as extended publishing for more data points, or connector for OPC UA observability settings.

-1. To confirm that the Akri instance properly connected to the connector for OPC UA, run the following command. Replace the placeholder with the name of the Akri instance that was included in the output of the previous command:

- ```bash

- kubectl get akrii <AKRI_INSTANCE_NAME> -n azure-iot-operations -o json

- ```

- The command output includes a section that looks like the following example. The snippet shows the Akri instance `brokerProperties` values and confirms that the connector for OPC UA is connected.

- ```json

- "spec": {

-

- "brokerProperties": {

- "ApplicationUri": "Boiler #2",

- "AssetEndpointProfile": "{\"spec\":{\"uuid\":\"opc-ua-broker-opcplc-000000-azure-iot-operation\"……

- ```

-description: How to configure the OPC PLC simulator to work with the connector for OPC UA. The simulator generates sample data for testing and development purposes.

-# CustomerIntent: As a developer, I want to configure an OPC PLC simulator in my industrial edge environment to test the process of managing OPC UA assets connected to the simulator.

-# Configure the OPC PLC simulator to work with the connector for OPC UA

-In this article, you learn how to configure and connect the OPC PLC simulator. The simulator simulates an OPC UA server with multiple nodes that generate random data and anomalies. You can configure user defined nodes. The OPC UA simulator lets you test the process of managing OPC UA assets with the [operations experience](howto-manage-assets-remotely.md) web UI or [the Akri services](overview-akri.md).

-## Prerequisites

-A deployed instance of Azure IoT Operations Preview. To deploy Azure IoT Operations for demonstration and exploration purposes, see [Quickstart: Run Azure IoT Operations Preview in Github Codespaces with K3s](../get-started-end-to-end-sample/quickstart-deploy.md). If you deploy Azure IoT Operations as described, the installation includes the OPC PLC simulator.

-## Deploy the OPC PLC simulator

-This section shows how to deploy the OPC PLC simulator if you didn't include it when you first deployed Azure IoT Operations.

-The following step lowers the security level for the OPC PLC so that it accepts connections from the connector for OPC UA or any client without an explicit peer certificate trust operation.

-> [!IMPORTANT]

-> Don't use the following example in production, use it for simulation and test purposes only.

-Run the following code to update the connector for OPC UA deployment and apply the new settings:

-```bash

-az k8s-extension update \

- --version 0.3.0-preview \

- --name opc-ua-broker \

- --release-train preview \

- --cluster-name <cluster-name> \

- --resource-group <azure-resource-group> \

- --cluster-type connectedClusters \

- --auto-upgrade-minor-version false \

- --config opcPlcSimulation.deploy=true \

- --config opcPlcSimulation.autoAcceptUntrustedCertificates=true

-```

-The OPC PLC simulator runs as a separate pod in the `azure-iot-operations` namespace. The pod name looks like `opcplc-000000-7b6447f99c-mqwdq`.

-## Configure mutual trust between the connector for OPC UA and the OPC PLC

-To learn more about mutual trust in OPC UA, see [OPC UA certificates infrastructure for the connector for OPC UA](overview-opcua-broker-certificates-management.md).

-The application instance certificate of the OPC PLC simulator is a self-signed certificate managed by [cert-manager](https://cert-manager.io/) and stored in the `aio-opc-ua-opcplc-default-application-cert-000000` Kubernetes secret.

-To configure mutual trust between the connector for OPC UA and the OPC PLC simulator:

-1. Get the certificate and push it to Azure Key Vault:

- ```bash

- kubectl -n azure-iot-operations get secret aio-opc-ua-opcplc-default-application-cert-000000 -o jsonpath='{.data.tls\.crt}' | \

- base64 -d | \

- xargs -0 -I {} \

- az keyvault secret set \

- --name "opcplc-crt" \

- --vault-name <your-azure-key-vault-name> \

- --value {} \

- --content-type application/x-pem-file

- ```

-1. Add the certificate to the `aio-opc-ua-broker-trust-list` custom resource in the cluster. Use a Kubernetes client such as `kubectl` to configure the `opcplc.crt` secret in the `SecretProviderClass` object array in the cluster.

- The following example shows a complete `SecretProviderClass` custom resource that contains the simulator certificate in a PEM encoded file with the .crt extension:

- ```yml

- apiVersion: secrets-store.csi.x-k8s.io/v1

- kind: SecretProviderClass

- metadata:

- name: aio-opc-ua-broker-trust-list

- namespace: azure-iot-operations

- spec:

- provider: azure

- parameters:

- usePodIdentity: 'false'

- keyvaultName: <your-azure-key-vault-name>

- tenantId: <your-azure-tenant-id>

- objects: |

- array:

- - |

- objectName: opcplc-crt

- objectType: secret

- objectAlias: opcplc.crt

- ```

- > [!NOTE]

- > The time it takes to project Azure Key Vault certificates into the cluster depends on the configured polling interval.

-The connector for OPC UA trust relationship with the OPC PLC simulator is now established and you can create an `AssetEndpointProfile` to connect to your OPC PLC simulator.

-## Optionally configure your `AssetEndpointProfile` without mutual trust established

-Optionally, you can configure an asset endpoint profile without establishing mutual trust between the connector for OPC UA and the OPC PLC simulator. If you understand the risks, you can turn off authentication for testing purposes.

-> [!CAUTION]

-> Don't configure for no authentication in production or pre-production environments. Exposing your cluster to the internet without authentication can lead to unauthorized access and even DDOS attacks.

-To allow your asset endpoint profile to connect to an OPC PLC server without establishing mutual trust, use the `additionalConfiguration` setting to modify the `AssetEndpointProfile` configuration.

-Patch the asset endpoint with `autoAcceptUntrustedServerCertificates=true`:

-```bash

-ENDPOINT_NAME=<name-of-you-endpoint-here>

-kubectl patch AssetEndpointProfile $ENDPOINT_NAME \

-```

-## Related content

-A deployed instance of Azure IoT Operations Preview. To deploy Azure IoT Operations for demonstration and exploration purposes, see [Quickstart: Run Azure IoT Operations Preview in Github Codespaces with K3s](../get-started-end-to-end-sample/quickstart-deploy.md).

-First, configure the secrets for the username and password in Azure Key Vault and project them into the connected cluster by using a `SecretProviderClass` object.

-1. Configure the username and password in Azure Key Vault. In the following example, use the `username` and `password` as secret references for the asset endpoint configuration in the operations experience web UI.

- Replace the placeholders for username and password with the credentials used to connect to the OPC UA server.

- To configure the username and password, run the following code:

- ```bash

- # Create username Secret in Azure Key Vault

- az keyvault secret set \

- --name "username" \

- --vault-name "<your-azure-key-vault-name>" \

- --value "<your-opc-ua-server-username>" \

- --content-type "text/plain"

- # Create password Secret in Azure Key Vault

- az keyvault secret set \

- --name "password" \

- --vault-name "<your-azure-key-vault-name>" \

- --value "<your-opc-ua-server-password>" \

- --content-type "text/plain"

- ```

-1. Configure the `aio-opc-ua-broker-user-authentication` custom resource in the cluster. Use a Kubernetes client such as `kubectl` to configure the `username` and `password` secrets in the `SecretProviderClass` object array in the cluster.

- The following example shows a complete `SecretProviderClass` custom resource after you add the secrets:

- ```yml

- apiVersion: secrets-store.csi.x-k8s.io/v1

- kind: SecretProviderClass

- metadata:

- name: aio-opc-ua-broker-user-authentication

- namespace: azure-iot-operations

- spec:

- provider: azure

- parameters:

- usePodIdentity: 'false'

- keyvaultName: <azure-key-vault-name>

- tenantId: <azure-tenant-id>

- objects: |

- array:

- - |

- objectName: username

- objectType: secret

- objectVersion: ""

- - |

- objectName: password

- objectType: secret

- objectVersion: ""

- ```

- > [!NOTE]

- > The time it takes to project Azure Key Vault certificates into the cluster depends on the configured polling interval.

-In the operations experience, select the **Username & password** option when you configure the Asset endpoint. Enter the names of the references that store the username and password values. In this example, the names of the references are `username` and `password`.

-A deployed instance of Azure IoT Operations Preview. To deploy Azure IoT Operations for demonstration and exploration purposes, see [Quickstart: Run Azure IoT Operations Preview in Github Codespaces with K3s](../get-started-end-to-end-sample/quickstart-deploy.md).

- ```azcli

- # Upload my-server.der OPC UA server's certificate as secret to Azure Key Vault

- --name "my-server-der" \

- --file ./my-server.der \

- ```azcli

- # Upload my-server.crt OPC UA server's certificate as secret to Azure Key Vault

- --name "my-server-crt" \

- --file ./my-server.crt \

- ```yml

- ```yml

-1. Get the CA certificate public key encode in DER or PEM format. These certificates are typically stored in files with either the .der or .crt extension. Get the CA's certificate revocation list (CRL). This list is typically in a file with the .crl. Check the documentation for your OPC UA server for details.

- ```azcli

- --name "my-server-ca-der" \

- --file ./my-server-ca.der \

- --name "my-server-crl" \

- --file ./my-server-ca.crl \

- ```azcli

- --name "my-server-ca-crt" \

- --file ./my-server-ca.crt \

- --name "my-server-crl" \

- --file ./my-server-ca.crl \

- ```yml

- ```yml

-If your OPC UA server uses a certificate issued by a certificate authority (CA), but you don't want to trust all certificates issued by the CA, complete the following steps:

-1. Besides the certificate itself, connector for OPC UA needs the CA certificate to properly validate the issuer chain of the OPC UA server's certificate. Add the CA certificate and its certificate revocation list (CRL) to a separate list called `aio-opc-ua-broker-issuer-list`.

- ```azcli

- --name "my-server-ca-der" \

- --file ./my-server-ca.der \

- --name "my-server-crl" \

- --file ./my-server-ca.crl \

- ```azcli

- --name "my-server-ca-crt" \

- --file ./my-server-ca.crt \

- --name "my-server-crl" \

- --file ./my-server-ca.crl \

- ```yml

- ```yml

- In PowerShell, you can complete the same task with the following command:

-For production environments, you can configure the connector for OPC UA to use an enterprise grade application instance certificate. Typically, an enterprise certificate authority (CA) issues this certificate and you need the CA certificate to your configuration. Often, there's a hierarchy of CAs and you need to add the complete validation chain of CAs to your configuration.

-| _enterprise-grade-ca-1.crl_ | The CA's certificate revocation list (CRL) file. |

- ```azcli

- --name "opcuabroker-certificate-der" \

- --file ./opcuabroker-certificate.der \

- --name "opcuabroker-certificate-pem" \

- --file ./opcuabroker-certificate.pem \

- --name "enterprise-grade-ca-1-der" \

- --file ./enterprise-grade-ca-1.der \

- --name "enterprise-grade-ca-1-crl" \

- --file ./enterprise-grade-ca-1.crl \

- ```yml

- ```yml

- ```azcli

-These assets, tags, and events map inbound data from OPC UA servers to friendly names that you can use in the MQTT broker and data processor pipelines.

-After you sign in, the web UI displays a list of sites. Each site is a collection of Azure IoT Operations instances where you can configure and manage your assets. A site typically represents a physical location where a you have physcial assets deployed. Sites make it easier for you to locate and manage assets. Your [IT administrator is responsible for grouping instances in to sites](/azure/azure-arc/site-manager/overview). Any Azure IoT Operations instances that aren't assigned to a site appear in the **Unassigned instances** node. Select the site that you want to use:

-When the OPC PLC simulator is running, dataflows from the simulator, to the connector, to the OPC UA broker, and finally to the MQTT broker.

- | ns=3;s=FastUInt10 | temperature | None |

- | ns=3;s=FastUInt100 | Tag 10 | None |

-description: Understand how the Akri services enable you to discover devices and assets at the edge, and expose them as resources on your cluster.

- - ignite-2023

-# CustomerIntent: As an industrial edge IT or operations user, I want to to understand how the Akri services enable me to discover devices and assets at the edge, and expose them as resources on a Kubernetes cluster.

-# What are the Akri services?

-The Akri services host the discovery handlers that enable you to detect devices and assets at the edge, and expose them as resources on a Kubernetes cluster. Use the Akri services to simplify the process of projecting leaf devices such as OPC UA devices, cameras, IoT sensors, and peripherals into your cluster. The Akri services use the devices' own protocols to project leaf devices into your cluster. For administrators who attach or remove devices from a cluster, this capability reduces the amount of coordination and manual configuration required.

-The Akri services are also extensible. You can use them as shipped, or you can add custom discovery and provisioning capabilities by adding protocol handlers, brokers, and behaviors.

-The Akri services are a Microsoft-managed commercial version of [Akri](https://docs.akri.sh/), an open-source Cloud Native Computing Foundation (CNCF) project.

-## Leaf device integration challenges

-It's common to run Kubernetes directly on infrastructure. But to integrate non-Kubernetes IoT leaf devices into a Kubernetes cluster requires a unique solution.

-IoT leaf devices present the following challenges, They:

-## Core capabilities

-To address the challenge of integrating non-Kubernetes IoT leaf devices, the Akri services have several core capabilities:

-### Device discovery

-Akri services deployments can include fixed-network discovery handlers. Discovery handlers enable assets from known network endpoints to find leaf devices as they appear on device interfaces or local subnets. Examples of network endpoints include OPC UA servers at a fixed IP address, and network scanning discovery handlers.

-### Dynamic provisioning

-Another capability of the Akri services is dynamic device provisioning.

-With the Akri services, you can dynamically provision devices such as:

-### Compatibility with Kubernetes

-The Akri services use standard Kubernetes primitives that let you apply your existing expertise and knowledge. Small devices connected to an Akri-configured cluster can appear as Kubernetes resources, just like memory or CPUs. The Akri services controller enables the cluster operator to start brokers, jobs, or other workloads for individual connected devices or groups of devices. These device configurations and properties remain in the cluster so that if there's node failure, other nodes can pick up any lost work.

-## Discover OPC UA assets

-The Akri services are a turnkey solution that lets you discover and create assets connected to an OPC UA server at the edge. The Akri services discover devices at the edge and maps them to assets in your cluster. The assets send telemetry to upstream connectors. The Akri services let you eliminate the painstaking process of manually configuring and onboarding the assets to your cluster.

-## Key features

-The following list shows the key features of the Akri

- - **Device network scanning**. This capability is useful for finding devices in smaller, remote locations such as a replacement camera in a store. The ONVIF and OPC UA localhost protocols currently support device network scanning discovery.

- - **Device connecting**. This capability is typically used in larger industrial scenarios such as factory environments where the network is typically static and network scanning isn't permitted. The `udev` and OPC UA local discovery server protocols currently support device connecting discovery.

- - **Device attach**. The Akri services also support custom logic for mapping or connecting devices. There are [open-source templates](https://docs.akri.sh/development/handler-development) to accelerate customization.

-### Features supported

-The Akri services support the following features:

-| [CNCF Akri Features](https://docs.akri.sh/) | Supported |

-| - | :-: |

-| Dynamic discovery of devices at the edge (supported protocols: OPC UA, ONVIF, udev) | ✅ |

-| Schedule devices with minimal latency using Akri's information on node affinity on the cluster | ✅ |

-| View Akri metrics and logs locally through Prometheus and Grafana | ✅ |

-| Secrets and credentials management | ✅ |

-| M:N device to broker ratio through configuration-level resource support | ✅ |

-| Observability on Akri deployments through Prometheus and Grafana dashboards | ✅ |

-| Akri services features | Supported |

-|--|::|

-| Installation through the Akri services Arc cluster extension | ✅ |

-| Deployment through the orchestration service | ✅ |

-| Onboard devices as custom resources to an edge cluster | ✅ |

-| View the Akri services metrics and logs through Azure Monitor | ❌ |

-| Akri services configuration by using the operations experience web UI | ❌ |

-| The Akri services detect and create assets that can be ingested into the Azure Device Registry | ❌ |

-| ISVs can build and sell custom protocol handlers for Azure IoT Operations solutions | ❌ |

-## Related content

-To learn more about the Akri services, see:

-To learn more about the open-source CNCF Akri, see the following resources:

-Assets are a core element of an Azure IoT Operations solution.

-An *asset* in an industrial edge environment is any item of value that you want to manage, monitor, and collect data from. An asset can be a machine, a software component, an entire system, or a physical object of value such as a field of crops, or a building. These assets are examples that exist in manufacturing, retail, energy, healthcare, and other sectors.

-An *asset* in Azure IoT Operations is a logical entity that you create to represent a real asset. An Azure IoT Operations asset can emit telemetry and events. You use these logical asset instances to manage the real assets in your industrial edge environment.

-> [!TIP]

-> Assets maybe related to IoT devices. While all IoT devices are assets, not all assets are devices. An *IoT device* is a physical object connected to the internet to collect, generate, and communicate data. IoT devices typically contain embedded components to perform specific functions. They can manage or monitor other things in their environment. Examples of IoT devices include crop sensors, smart thermostats, connected security cameras, wearable devices, and monitoring devices for manufacturing machinery or vehicles.

-Each of these services is explained in greater detail in the following sections.

-The operations experience web UI lets operations teams perform these tasks in a simplified web interface. The operations experience uses the other services described previously, to complete these tasks. You can also use the Azure IoT Operations CLI to manage assets by using the `az iot ops asset` command.

-## Manage assets as Azure resources in a centralized registry

-In an industrial edge environment with many assets, it's useful for IT and operations teams to have a single registry for devices and assets. Azure Device Registry Preview provides this capability, and projects your industrial assets as Azure resources. Teams that use Device Registry together with the operations experience, have a consistent deployment and management experience across cloud and edge environments.

-The following screenshot shows an example thermostat asset in the operations experience:

-The following screenshot shows the example thermostat asset in the Azure portal:

-The following screenshot shows the example thermostat asset as a Kubernetes custom resource:

-The following features are supported in Azure Device Registry:

-| Feature | Supported |

-| -- | :-: |

-| Asset resource management by using Azure API | ✅ |

-| Asset resource management by using operations experience | ✅ |

-| Asset synchronization to Kubernetes cluster running Azure IoT Operations | ✅ |

-| Asset as Azure resource (with capabilities such as Azure resource groups and tags) | ✅ |

-## Discover edge assets

-A common task in complex edge solutions is to discover assets and automatically add them to your Kubernetes cluster. The Akri services provide this capability. For administrators who attach or remove assets from the cluster, the Akri services reduce the amount of coordination and manual configuration.

-The Akri services include fixed-network discovery handlers. Discovery handlers enable assets from known network endpoints to find leaf devices as they appear on device interfaces or local subnets. Examples of network endpoints include OPC UA servers at a fixed IP address, and network scanning discovery handlers.

-The Akri services are installed as part of Azure IoT Operations and you can configure them alongside the OPC UA simulation PLC server. The OPC UA discovery handler starts automatically and inspects the OPC UA simulation PLC server's address space. The discovery handler reports assets back to the Akri services and triggers deployment of the `AssetEndpointProfile` and `Asset` custom resources into the cluster.

-To project the trusted certificates from Azure Key Vault into the Kubernetes cluster, you must configure a `SecretProviderClass` custom resource. This custom resource contains a list of all the secret references associated with the trusted certificates. The connector for OPC UA uses the custom resource to map the trusted certificates into connector for OPC UA containers and make them available for validation. The default name for the `SecretProviderClass` custom resource that handles the trusted certificates list is *aio-opc-ua-broker-trust-list*.

-> [!NOTE]

-> The time it takes to project Azure Key Vault certificates into the cluster depends on the configured polling interval.

-If your OPC UA server's application instance certificate is signed by an intermediate certificate authority, but you don't want to automatically trust all the certificates issued by the certificate authority, you can use an issuer certificate list to manage the trust relationship.

-An _issuer certificate list_ stores the certificate authority certificates that the connector for OPC UA trusts. If the application certificate of an OPC UA server is signed by an intermediate certificate authority, the connector for OPC UA validates the full chain of certificate authorities up to the root. The issuer certificate list should contain the certificates of all the certificate authorities in the chain to ensure that the connector for OPC UA can validate the OPC UA servers.

-| Handling of OPC UA Global Discovery Service (GDS) | Unsupported | ❌ |

-The connector for OPC UA includes an OPC UA simulation server that you can use to test your applications. To learn more, see [Configure an OPC PLC simulator to work with the connector for OPC UA](howto-configure-opc-plc-simulator.md).

-The OPC UA discovery handler:

-> [!NOTE]

-> Asset detection by Akri only works for OPC UA servers that don't require user or transport authentication.

-To learn more about Akri, see [What are the Akri services?](overview-akri.md).

-Complete [Quickstart: Run Azure IoT Operations Preview in GitHub Codespaces with K3s](quickstart-deploy.md) before you begin this quickstart.

-To sign in to the operations experience, you need a work or school account in the tenant where you deployed Azure IoT Operations. If you're currently using a Microsoft account (MSA), you need to create a Microsoft Entra ID with at least contributor permissions for the resource group that contains your **Kubernetes - Azure Arc** instance. To learn more, see [Known Issues > Create Entra account](../troubleshoot/known-issues.md#known-issues-azure-iot-operations-preview).

-The data that OPC UA servers expose can have a complex structure and can be difficult to understand. Azure IoT Operations provides a way to model OPC UA assets as tags, events, and properties. This modeling makes it easier to understand the data and to use it in downstream processes such as the MQTT broker and data processor pipelines.

-> [!IMPORTANT]

-> You must use a work or school account to sign in to the operations experience. To learn more, see [Known Issues > Create Entra account](../troubleshoot/known-issues.md#known-issues-azure-iot-operations-preview).

-## Configure the simulator

-These quickstarts use the **OPC PLC simulator** to generate sample data. To enable the quickstart scenario, you need to configure your asset endpoint to connect without mutual trust established. This configuration is not recommended for production or pre-production environments:

-1. To configure the asset endpoint for the quickstart scenario, run the following command:

- ```console

- kubectl patch AssetEndpointProfile opc-ua-connector-0 -n azure-iot-operations --type=merge -p '{"spec":{"additionalConfiguration":"{\"applicationName\":\"opc-ua-connector-0\",\"security\":{\"autoAcceptUntrustedServerCertificates\":true}}"}}'

- ```

- > [!CAUTION]

- > Don't use this configuration in production or pre-production environments. Exposing your cluster to the internet without proper authentication might lead to unauthorized access and even DDOS attacks.

- To learn more, see [Deploy the OPC PLC simulator](../discover-manage-assets/howto-configure-opc-plc-simulator.md) section.

-1. To enable the configuration changes to take effect immediately, first find the name of your `aio-opc-supervisor` pod by using the following command:

- ```console

- kubectl get pods -n azure-iot-operations

- ```

- The name of your pod looks like `aio-opc-supervisor-956fbb649-k9ppr`.

-1. Restart the `aio-opc-supervisor` pod by using a command that looks like the following example. Use the `aio-opc-supervisor` pod name from the previous step:

- ```console

- kubectl delete pod aio-opc-supervisor-956fbb649-k9ppr -n azure-iot-operations

- ```

-After you define an asset, a connector for OPC UA pod discovers it. The pod uses the asset endpoint that you specify in the asset definition to connect to an OPC UA server. You can use `kubectl` to view the discovery pod that was created when you added the asset endpoint. The pod name looks like `aio-opc-opc.tcp-1-8f96f76-kvdbt`:

-```console

-kubectl get pods -n azure-iot-operations

-```

-When the OPC PLC simulator is running, dataflows from the simulator, to the connector for OPC UA, and finally to the MQTT broker.

-> [!TIP]

-> Data from an asset with a name that starts with _boiler-_ is from an asset that was automatically discovered. This is not the same asset as the thermostat asset you created.

-## Discover OPC UA data sources by using Akri services

-In the previous section, you saw how to add assets manually. You can also use Akri services to automatically discover OPC UA data sources and create Akri instance custom resources that represent the discovered devices. Currently, Akri services can't detect and create assets that can be ingested into the Azure Device Registry Preview. Therefore, you can't currently manage assets discovered by Akri in the Azure portal.

-When you deploy Azure IoT Operations, the deployment includes the Akri discovery handler pods. To verify these pods are running, run the following command:

-```console

-kubectl get pods -n azure-iot-operations | grep akri

-```

-The output from the previous command looks like the following example:

-```output

-aio-akri-otel-collector-5c775f745b-g97qv 1/1 Running 3 (4h15m ago) 2d23h

-aio-akri-agent-daemonset-mp6v7 1/1 Running 3 (4h15m ago) 2d23h

-```

-Use the following command to verify that the discovery pod is running:

-```console

-kubectl get pods -n azure-iot-operations | grep discovery

-```

-The output from the previous command looks like the following example:

-```output

-aio-opc-asset-discovery-wzlnj 1/1 Running 0 19m

-```

-To configure the Akri services to discover OPC UA data sources, create an Akri configuration that references your OPC UA source. Run the following command to create the configuration:

-```console

-kubectl apply -f https://raw.githubusercontent.com/Azure-Samples/explore-iot-operations/main/samples/quickstarts/akri-opcua-asset.yaml

-```

-The following snippet shows the YAML file that you applied:

-> [!IMPORTANT]

-> There's currently a known issue where the configuration for the asset endpoint contains an invalid setting. To work around this issue, you need to remove the `"securityMode":"none"` setting from the configuration for the `opc-ua-broker-opcplc-000000-50000` asset endpoint. To learn more, see [Connector for OPC UA](../troubleshoot/known-issues.md#akri-services).

-To verify the configuration, run the following command to view the Akri instances that represent the OPC UA data sources discovered by Akri services. You might need to wait a few minutes for the configuration to be available:

-```console

-kubectl get akrii -n azure-iot-operations

-```

-The output from the previous command looks like the following example.

-```output

-NAME CONFIG SHARED NODES AGE

-akri-opcua-asset-dbdef0 akri-opcua-asset true ["k3d-k3s-default-server-0"] 45s

-```

-Now you can use these resources in the local cluster namespace.

-To confirm that the Akri services are connected to the connector for OPC UA, copy and paste the name of the Akri instance from the previous step into the following command:

-```console

-kubectl get akrii <AKRI_INSTANCE_NAME> -n azure-iot-operations -o json

-```

-The command output looks like the following example. This example excerpt from the output shows the Akri instance `brokerProperties` values and confirms that it's connected the connector for OPC UA.

-```json

-"spec": {

- "brokerProperties": {

- "ApplicationUri": "Boiler #2",

- "AssetEndpointProfile": "{\"spec\":{\"uuid\":\"opc-ua-broker-opcplc-000000-azure-iot-operation\"……

-```

-If you won't use this deployment further, delete the Kubernetes cluster where you deployed Azure IoT Operations and remove the Azure resource group that contains the cluster.

-The services deployed in this quickstart include:

-* [MQTT broker](../manage-mqtt-broker/overview-iot-mq.md)

-* [Connector for OPC UA](../discover-manage-assets/overview-opcua-broker.md) with simulated thermostat asset to start generating data

-* [Akri services](../discover-manage-assets/overview-akri.md)

-* [Azure Device Registry Preview](../discover-manage-assets/overview-manage-assets.md#manage-assets-as-azure-resources-in-a-centralized-registry)

-* [Observability](../configure-observability-monitoring/howto-configure-observability.md)

-The following quickstarts in this series build on this one to define sample assets, data processing pipelines, and visualizations. If you want to deploy Azure IoT Operations to a cluster such as AKS Edge Essentials in order to run your own workloads, see [Prepare your Azure Arc-enabled Kubernetes cluster](../deploy-iot-ops/howto-prepare-cluster.md?tabs=aks-edge-essentials) and [Deploy Azure IoT Operations Preview to an Arc-enabled Kubernetes cluster](../deploy-iot-ops/howto-deploy-iot-operations.md).

-This series of quickstarts is intended to help you get started with Azure IoT Operations as quickly as possible so that you can evaluate an end-to-end scenario. In a true development or production environment, these tasks would be performed by multiple teams working together and some tasks might require elevated permissions.

-If you want to rerun this quickstart with a cluster that already has Azure IoT Operations deployed to it, refer to the steps in [Clean up resources](#clean-up-resources) to uninstall Azure IoT Operations before continuing.

-1. Create an Azure Key Vault to manage secrets for your cluster.

-1. Configure your cluster with a secrets store and service principal to communicate with cloud resources.

-## Deploy Azure IoT Operations Preview

-In this section, you use the [az iot ops init](/cli/azure/iot/ops#az-iot-ops-init) command to configure your cluster so that it can communicate securely with your Azure IoT Operations components and key vault, then deploy Azure IoT Operations.

-1. Create a key vault. For this scenario, use the same name and resource group as your cluster. Keyvault names have a maximum length of 24 characters, so the following command truncates the `CLUSTER_NAME`environment variable if necessary.

- az keyvault create --enable-rbac-authorization false --name ${CLUSTER_NAME:0:24} --resource-group $RESOURCE_GROUP

- > You can use an existing key vault for your secrets, but verify that the **Permission model** is set to **Vault access policy**. You can check this setting in the Azure portal in the **Access configuration** section of an existing key vault. Or use the [az keyvault show](/cli/azure/keyvault#az-keyvault-show) command to check that `enableRbacAuthorization` is false.

- az iot ops init --simulate-plc --cluster $CLUSTER_NAME --resource-group $RESOURCE_GROUP --kv-id $(az keyvault show --name ${CLUSTER_NAME:0:24} -o tsv --query id)

- If you get an error that says *Your device is required to be managed to access your resource*, run `az login` again and make sure that you sign in interactively with a browser.

- >[!TIP]

- >If you've run `az iot ops init` before, it automatically created an app registration in Microsoft Entra ID for you. You can reuse that registration rather than creating a new one each time. To use an existing app registration, add the optional parameter `--sp-app-id <APPLICATION_CLIENT_ID>`.

-While the deployment is in progress, you can watch the resources being applied to your cluster. You can use kubectl commands to observe changes on the cluster or, since the cluster is Arc-enabled, you can use the Azure portal.

-It can take several minutes for the deployment to complete. Continue running the `get pods` command to refresh your view.

-1. On the **Overview** page of your instance, the **Arc extensions** table displays the resources that were deployed to your cluster.

-If you want to delete the Azure IoT Operations deployment but want to keep your cluster, use the [az iot ops delete](/cli/azure/iot/ops#az-iot-ops-delete) command.

- ```azurecli

- az iot ops delete --cluster $CLUSTER_NAME --resource-group $RESOURCE_GROUP

- ```

-If you want to delete all of the resources you created for this quickstart, delete the Kubernetes cluster where you deployed Azure IoT Operations and remove the Azure resource group that contained the cluster.

-You also need a Microsoft Fabric subscription. In your subscription, you need access to a **premium workspace** with **Contributor** or above permissions.

-Start by navigating to the [Real-Time Intelligence experience in Microsoft Fabric](https://msit.powerbi.com/home?experience=kusto).

-Follow the steps in [Create an eventstream in Microsoft Fabric](/fabric/real-time-intelligence/event-streams/create-manage-an-eventstream?pivots=enhanced-capabilities) to create a new eventstream from the Real-Time Intelligence capabilities.

-After the eventstream is created, you'll see the main editor where you can start adding sources to the eventstream.

-Follow the steps in [Add Azure Event Hubs source to an eventstream](/fabric/real-time-intelligence/event-streams/add-source-azure-event-hubs?pivots=enhanced-capabilities) to add the event source. Keep the following notes in mind:

-* When it's time to select a **Data format**, choose *Json* (it might be selected already by default).

-* Make sure to complete all the steps in the article through selecting **Publish** on the ribbon.

-1. Next, create a KQL table in your database. Call it *OPCUA* and use the following columns.

- | Temperature | decimal |

- | Pressure | decimal |

- .create table ['OPCUA'] ingestion json mapping 'opcua_mapping' '[{"column":"Temperature", "Properties":{"Path":"$.temperature.Value"}},{"column":"Pressure", "Properties":{"Path":"$.[\'Tag 10\'].Value"}},{"column":"Timestamp", "Properties":{"Path":"$[\'EventProcessedUtcTime\']"}}]'

- ```

-Follow the steps in [Add a KQL Database destination to an eventstream](/fabric/real-time-intelligence/event-streams/add-destination-kql-database?pivots=enhanced-capabilities#direct-ingestion-mode) to add the destination. Keep the following notes in mind:

-* Use direct ingestion mode.

-* On the **Configure** step, select the *OPCUA* table that you created earlier.

-* On the **Inspect** step, open the **Advanced** options. Under **Mapping**, select **Existing mapping** and choose *opcua_mapping*.

-

-In this section, you'll create a new [Real-Time Dashboard](/fabric/real-time-intelligence/dashboard-real-time-create) to visualize your quickstart data. The dashboard will automatically allow filtering by timestamp, and will display visual summaries of temperature and pressure data.

-Then, follow the steps in the [Add data source](/fabric/real-time-intelligence/dashboard-real-time-create#add-data-source) section to add your database as a data source. Keep the following notes in mind:

-* In the **Data sources** pane, your database will be under **OneLake data hub**.

-Next, add a tile to your dashboard to show a line chart of temperature and pressure over time for the selected time range.

-1. Enter the following KQL query for the tile. This query applies a built-in filter parameter from the dashboard selector for time range, and pulls the resulting records with their timestamp, temperature, and pressure.

- | project Timestamp, Temperature, Pressure

- * **Tile name**: *Temperature and pressure over time*

- * **Visual type**: *Line chart*

- * **Data**:

- * **Y columns**: *Temperature (decimal)* and *Pressure (decimal)* (already inferred by default)

- * **X columns**: *Timestamp (datetime)* (already inferred by default)

- * **Y Axis**:

- * **Label**: *Units*

- * **X Axis**:

- * **Label**: *Timestamp*

-Next, create some tiles to display the maximum values of temperature and pressure.

-1. Enter the following KQL query for the tile. This query applies a built-in filter parameter from the dashboard selector for time range, and takes the highest temperature value from the resulting records.

-

- * **Tile name**: *Max temperature*

- * **Visual type**: *Stat*

- * **Data**:

- * **Value column**: *Temperature (decimal)* (already inferred by default)

-

-

-1. Replace *Temperature* in the KQL query with *Pressure*, so that it matches the query below.

- | top 1 by Pressure desc

- | summarize by Pressure

- **Run** the query to verify that a maximum pressure can be found.

- * **Tile name**: *Max pressure*

- * **Data**:

- * **Value column**: *Pressure (decimal)* (already inferred by default)

-If you're not going to continue to use this deployment, delete the Kubernetes cluster where you deployed Azure IoT Operations. In Azure, remove the Azure resource group that contains the cluster and your event hub. If you used Codespaces for these quickstarts, delete your Codespace from GitHub.

-You can also delete your Microsoft Fabric workspace and/or all the resources within it associated with this quickstart, including the eventstream, Eventhouse, and Real-Time Dashboard.

-To create an Event Hubs namespace and an event hub, run the following Azure CLI commands in your Codespaces terminal. These commands create the Event Hubs namespace in the same resource group as your Kubernetes cluster:

-```azurecli

-az eventhubs namespace create --name ${CLUSTER_NAME:0:24} --resource-group $RESOURCE_GROUP --location $LOCATION

-# AIO Arc extension name

-AIO_EXTENSION_NAME=$(az k8s-extension list --resource-group $RESOURCE_GROUP --cluster-name $CLUSTER_NAME --cluster-type connectedClusters -o tsv --query "[?extensionType=='microsoft.iotoperations'].name")

-az deployment group create \

- --name assign-RBAC-roles \

- --resource-group $RESOURCE_GROUP \

- --template-file samples/quickstarts/event-hubs-config.bicep \

- --parameters aioExtensionName=$AIO_EXTENSION_NAME \

- --parameters clusterName=$CLUSTER_NAME \

- --parameters eventHubNamespaceName=${CLUSTER_NAME:0:24}

-To create and configure a dataflow in your cluster, run the following commands in your Codespaces terminal. This dataflow forwards messages from the MQTT topic to the event hub you created without making any changes:

-sed 's/<NAMESPACE>/'"${CLUSTER_NAME:0:24}"'/' samples/quickstarts/dataflow.yaml > dataflow.yaml

-If you're not going to continue to use this deployment, delete the Kubernetes cluster where you deployed Azure IoT Operations and remove the Azure resource group that contains the cluster.

-You can also delete your Microsoft Fabric workspace.

- az provider register -n "Microsoft.IoTOperationsOrchestrator"

-1. Enable a firewall rule for port 8883:

- New-NetFirewallRule -DisplayName "MQTT broker" -Direction Inbound -Protocol TCP -LocalPort 8883 -Action Allow

-1. Run the following command and make a note of the IP address for the service called `aio-mq-dmqtt-frontend`:

- kubectl get svc aio-mq-dmqtt-frontend -n azure-iot-operations -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

-1. Enable port forwarding for port 8883. Replace `<aio-mq-dmqtt-frontend IP address>` with the IP address you noted in the previous step:

- netsh interface portproxy add v4tov4 listenport=8883 listenaddress=0.0.0.0 connectport=8883 connectaddress=<aio-mq-dmqtt-frontend IP address>

- az provider register -n "Microsoft.IoTOperationsOrchestrator"

-MQTT broker supports multiple authentication methods for clients, and you can configure each listener to have its own authentication system with *BrokerAuthentication* resources.

-Azure IoT Operations Preview deploys a default BrokerAuthentication resource named `authn` linked with the default listener named `listener` in the `azure-iot-operations` namespace. It's configured to only use Kubernetes Service Account Tokens (SATs) for authentication. To inspect it, run:

-The output shows the default BrokerAuthentication resource, with metadata removed for brevity:

- - method: ServiceAccountToken

- serviceAccountToken:

- audiences:

- - aio-mq

-To change the configuration, modify the `authenticationMethods` setting in this BrokerAuthentication resource or create new brand new BrokerAuthentication resource with a different name. Then, deploy it using `kubectl apply`.

-## Relationship between BrokerListener and BrokerAuthentication

-The following rules apply to the relationship between BrokerListener and BrokerAuthentication:

-* Each BrokerListener can have multiple ports. Each port can be linked to a BrokerAuthentication resource.

-* Each BrokerAuthentication can support multiple authentication methods at once

- custom:

- serviceAccountToken:

- - method: x509Credentials

- x509Credentials:

-To learn more about each of the authentication options, see the following sections:

-## X.509 client certificate

-### Prerequisites

-Both EC and RSA keys are supported, but all certificates in the chain must use the same key algorithm. If you're importing your own CA certificates, ensure that the client certificate uses the same key algorithm as the CAs.

-### Import trusted client root CA certificate

-A trusted root CA certificate is required to validate the client certificate. To import a root certificate that can be used to validate client certificates, first import the certificate PEM as ConfigMap under the key `client_ca.pem`. Client certificates must be rooted in this CA for MQTT broker to authenticate them.

-```console

-$ kubectl describe configmap client-ca -n azure-iot-operations

-MIIBmzCCAUGgAwIBAgIQVAZ2I0ydpCut1imrk+fM3DAKBggqhkjOPQQDAjAsMRAw

-...

-t2xMXcOTeYiv2wnTq0Op0ERKICHhko2PyCGGwnB2Gg==

-### Certificate attributes

-X509 attributes can be specified in the *BrokerAuthentication* resource. For example, every client that has a certificate issued by the root CA `CN = Contoso Root CA Cert, OU = Engineering, C = US` or an intermediate CA `CN = Contoso Intermediate CA` receives the attributes listed.

-apiVersion: mqttbroker.iotoperations.azure.com/v1beta1

-kind: BrokerAuthentication

-metadata:

- name: authn

- namespace: azure-iot-operations

- - method: x509Credentials

- x509Credentials:

-Authorization rules can be applied to clients using X.509 certificates with these attributes.

-### Enable X.509 client authentication

-Finally, once the trusted client root CA certificate and the certificate-to-attribute mapping are imported, enable X.509 client authentication by adding `x509` as one of the authentication methods as part of a BrokerAuthentication resource linked to a TLS-enabled listener. For example:

-```yaml

-spec:

- authenticationMethods:

- - method: x509Credentials

- x509Credentials:

- trustedClientCaCert: client-ca

- attributes:

- secretName: x509-attributes

-```

-Modify the `authenticationMethods` setting in a BrokerAuthentication resource to specify `ServiceAccountToken` as a valid authentication method. The `audiences` specifies the list of valid audiences for tokens. Choose unique values that identify the MQTT broker service. You must specify at least one audience, and all SATs must match one of the specified audiences.

- serviceAccountToken:

- - aio-mq

- - my-audience

-Here, the `serviceAccountName` field in the pod configuration must match the service account associated with the token being used. Also, The `serviceAccountToken.audience` field in the pod configuration must be one of the `audiences` configured in the BrokerAuthentication resource.

-mosquitto_pub --host aio-mq-dmqtt-frontend --port 8883 --message "hello" --topic "world" --debug --cafile /var/run/certs/ca.crt -D CONNECT authentication-method 'K8S-SAT' -D CONNECT authentication-data $(cat /var/run/secrets/tokens/mq-sat)

-Modify the `authenticationMethods` setting in a BrokerAuthentication resource to specify `custom` as a valid authentication method. Then, specify the parameters required to communicate with a custom authentication server.

- - custom:

- # Trusted CA certificate for validating custom authentication server certificate.

- # Required unless the server certificate is publicly-rooted.

- caCert: custom-auth-ca

-Authorization policies determine what actions the clients can perform on the broker, such as connecting, publishing, or subscribing to topics. Configure MQTT broker to use one or multiple authorization policies with the *BrokerAuthorization* resource.

-You can set to one *BrokerAuthorization* for each listener. Each *BrokerAuthorization* resource contains a list of rules that specify the principals and resources for the authorization policies.

-> To have the *BrokerAuthorization* configuration apply to a listener, at least one *BrokerAuthentication* must also be linked to that listener.

-## Configure BrokerAuthorization for listeners

-The specification of a *BrokerAuthorization* resource has the following fields:

-| Field Name | Required | Description |

-| | | |

-| authorizationPolicies | Yes | This field defines the settings for the authorization policies, such as *enableCache* and *rules*.|

-| enableCache | No | Whether to enable caching for the authorization policies. If set to `true`, the broker caches the authorization results for each client and topic combination to improve performance and reduce latency. If set to `false`, the broker evaluates the authorization policies for each client and topic request, to ensure consistency and accuracy. This field is optional and defaults to `false`. |

-| rules | No | A list of rules that specify the principals and resources for the authorization policies. Each rule has these subfields: *principals* and *brokerResources*. |

-| principals | Yes | This subfield defines the identities that represent the clients, such as *usernames*, *clientids*, and *attributes*.|

-| usernames | No | A list of usernames that match the clients. The usernames are case-sensitive and must match the usernames provided by the clients during authentication. |

-| clientids | No | A list of client IDs that match the clients. The client IDs are case-sensitive and must match the client IDs provided by the clients during connection. |

-| attributes | No | A list of key-value pairs that match the attributes of the clients. The attributes are case-sensitive and must match the attributes provided by the clients during authentication. |

-| brokerResources | Yes | This subfield defines the objects that represent the actions or topics, such as *method* and *topics*. |

-| method | Yes | The type of action that the clients can perform on the broker. This subfield is required and can be one of these values: **Connect**: This value indicates that the clients can connect to the broker. **Publish**: This value indicates that the clients can publish messages to topics on the broker. **Subscribe**: This value indicates that the clients can subscribe to topics on the broker. |

-| topics | No | A list of topics or topic patterns that match the topics that the clients can publish or subscribe to. This subfield is required if the method is *Subscribe* or *Publish*. |

-The following example shows how to create a *BrokerAuthorization* resource that defines the authorization policies for a listener named *my-listener*.

- enableCache: true

- - temperature-sensor

- - humidity-sensor

-To create rules based on properties from a client's certificate, its root CA, or intermediate CA, define the X.509 attributes in the *BrokerAuthorization* resource. For more information, see [Certificate attributes](howto-configure-authentication.md#certificate-attributes).

-kubectl annotate serviceaccount mqtt-client aio-mq-broker-auth/group=authz-sat

-Attribute annotations must begin with `aio-mq-broker-auth/` to distinguish them from other annotations.

- - "orders"

-To disable authorization, set `authorizationEnabled: false` in the BrokerListener resource. When the policy is set to allow all clients, all [authenticated clients](./howto-configure-authentication.md) can access all operations.

-```yaml

-apiVersion: mqttbroker.iotoperations.azure.com/v1beta1

-kind: BrokerListener

-metadata:

- name: "my-listener"

- namespace: azure-iot-operations

-spec:

- brokerRef: "my-broker"

- authenticationEnabled: false

- authorizationEnabled: false

- port: 1883

-```

-The **Broker** resource is the main resource that defines the overall settings for MQTT broker. It also determines the number and type of pods that run the *Broker* configuration, such as the frontends and the backends. You can also use the *Broker* resource to configure its memory profile. Self-healing mechanisms are built in to the broker and it can often automatically recover from component failures. For example, a node fails in a Kubernetes cluster configured for high availability.

-To configure the scaling settings MQTT broker, you need to specify the `mode` and `cardinality` fields in the specification of the *Broker* custom resource. For more information on setting the mode and cardinality settings using Azure CLI, see [az iot ops init](/cli/azure/iot/ops#az-iot-ops-init).

-The `mode` field can be one of these values:

- mode: distributed

-The following table lists the properties of the broker advanced settings that include client configurations, encryption of internal traffic, certificate rotation, and node tolerations.

-| Name | Type | Description |

-|-|--|--|

-| clients | ClientConfig | Configurations related to all clients |

-| clients.maxKeepAliveSeconds | `integer` | Upper bound of a client's keep alive, in seconds |

-| clients.maxMessageExpirySeconds | `integer` | Upper bound of message expiry interval, in seconds |

-| clients.maxReceiveMaximum | `integer` | Upper bound of receive maximum that a client can request in the CONNECT packet |

-| clients.maxSessionExpirySeconds | `integer` | Upper bound of session expiry interval, in seconds |

-| clients.subscriberQueueLimit | `SubscriberQueueLimit` | The limit on the number of queued messages for a subscriber |

-| clients.subscriberQueueLimit.length | `integer` | The maximum length of the queue before messages are dropped |

-| clients.subscriberQueueLimit.strategy | `SubscriberMessageDropStrategy` | The strategy for dropping messages from the queue |

-| clients.subscriberQueueLimit.strategy.DropOldest | `string` | The oldest message is dropped |

-| clients.subscriberQueueLimit.strategy.None | `string` | Messages are never dropped |

-| encryptInternalTraffic | Encrypt | The setting to enable or disable encryption of internal traffic |

-| encryptInternalTraffic.Disabled | `string` | Disable internal traffic encryption |

-| encryptInternalTraffic.Enabled | `string` | Enable internal traffic encryption |

-| internalCerts | CertManagerCertOptions | Certificate rotation and private key configuration |

-| internalCerts.duration | `string` | Lifetime of certificate. Must be specified using a *Go* *time.Duration* format (h, m, s). For example, 240h for 240 hours and 45m for 45 minutes. |

-| internalCerts.privateKey | `CertManagerPrivateKey` | Configuration of certificate private key |

-| internalCerts.renewBefore | `string` | Duration before renewing a certificate. Must be specified using a *Go* *time.Duration* format (h, m, s). For example, 240h for 240 hours and 45m for 45 minutes. |

-| internalCerts.privateKey.algorithm | PrivateKeyAlgorithm | Algorithm for private key |

-| internalCerts.privateKey.rotationPolicy | PrivateKeyRotationPolicy | Cert-manager private key rotation policy |

-| internalCerts.privateKey.algorithm.Ec256 | `string`| Algorithm - EC256 |

-| internalCerts.privateKey.algorithm.Ec384 | `string`| Algorithm - EC384 |

-| internalCerts.privateKey.algorithm.Ec521 | `string`| Algorithm - EC521 |

-| internalCerts.privateKey.algorithm.Ed25519 | `string`| Algorithm - Ed25519|

-| internalCerts.privateKey.algorithm.Rsa2048 | `string`| Algorithm - RSA2048|

-| internalCerts.privateKey.algorithm.Rsa4096 | `string`| Algorithm - RSA4096|

-| internalCerts.privateKey.algorithm.Rsa8192 | `string`| Algorithm - RSA8192|

-| internalCerts.privateKey.rotationPolicy.Always | `string`| Always rotate key |

-| internalCerts.privateKey.rotationPolicy.Never | `string`| Never rotate key |

-| tolerations | NodeTolerations | The details of tolerations that are applied to all *Broker* pods |

-| tolerations.effect | `string` | Toleration effect |

-| tolerations.key | `string` | Toleration key |

-| tolerations.operator | `TolerationOperator` | Toleration operator. For example, "Exists" or "Equal". |

-| tolerations.value | `string` | Toleration value |

-| tolerations.operator.Equal | `string` | Equal operator |

-| tolerations.operator.Exists | `string` | Exists operator |

- tolerations:

- effect: string

- key: string

- operator: Equal

- value: string

-You can configure diagnostics using the *Broker* custom resource definition (CRD). The following table shows the properties of the broker diagnostic settings and all default values.

-| Name | Format | Default | Description |

-| | - | - | |

-| logs.exportIntervalSeconds | integer | 30 | How often to export the logs to the open telemetry collector |

-| logs.exportLogLevel | string | error | The level of logs to export |

-| logs.level | string | info | The log level. For example, `debug`, `info`, `warn`, `error`, `trace` |

-| logs.openTelemetryCollectorAddress | string | | The open telemetry collector endpoint where to export |

-| metrics.exportIntervalSeconds | integer | 30 | How often to export the metrics to the open telemetry collector |

-| metrics.mode | MetricsEnabled | Enabled | The toggle to enable/disable metrics. |

-| metrics.openTelemetryCollectorAddress| string | | The open telemetry collector endpoint where to export |

-| metrics.prometheusPort | integer | 9600 | The prometheus port to expose the metrics |

-| metrics.stalenessTimeSeconds | integer | 600 | The time used to determine if a metric is stale and drop from the metrics cache |

-| metrics.updateIntervalSeconds | integer | 30 | How often to refresh the metrics |

-| selfcheck.intervalSeconds | integer | 30 | The self check interval |

-| selfcheck.mode | SelfCheckMode | Enabled | The toggle to enable/disable self check |

-| selfcheck.timeoutSeconds | integer | 15 | The timeout for self check |

-| traces.cacheSizeMegabytes | integer | 16 | The cache size in megabytes |

-| traces.exportIntervalSeconds | integer | 30 | How often to export the metrics to the open telemetry collector |

-| traces.mode | TracesMode | Enabled | The toggle to enable/disable traces |

-| traces.openTelemetryCollectorAddress | string | | The open telemetry collector endpoint where to export |

-| traces.selfTracing | SelfTracing | | The self tracing properties |

-| traces.spanChannelCapacity | integer | 1000 | The span channel capacity |

- mode: auto

- exportIntervalSeconds: 220

- exportLogLevel: nym

- level: debug

- openTelemetryCollectorAddress: acfqqatmodusdbzgomgcrtulvjy

- stalenessTimeSeconds: 463

- mode: Enabled

- exportIntervalSeconds: 246

- openTelemetryCollectorAddress: vyasdzsemxfckcorfbfx

- prometheusPort: 60607

- updateIntervalSeconds: 15

- intervalSeconds: 106

- timeoutSeconds: 70

- cacheSizeMegabytes: 97

- exportIntervalSeconds: 114

- openTelemetryCollectorAddress: oyujxiemzlqlcsdamytj

- intervalSeconds: 179

- spanChannelCapacity: 47152

-diskBackedMessageBufferSettings:

-diskBackedMessageBufferSettings:

- diskBackedMessageBufferSettings:

-To customize the network access and security use the **BrokerListener** resource. A listener corresponds to a network endpoint that exposes the broker to the network. You can have one or more BrokerListener resources for each *Broker* resource, and thus multiple ports with different access control each.

-Each listener can have its own authentication and authorization rules that define who can connect to the listener and what actions they can perform on the broker. You can use *BrokerAuthentication* and *BrokerAuthorization* resources to specify the access control policies for each listener. This flexibility allows you to fine-tune the permissions and roles of your MQTT clients, based on their needs and use cases.

-The *BrokerListener* resource has these fields:

-| Field Name | Required | Description |

-||-|-|

-| brokerRef | Yes | The name of the broker resource that this listener belongs to. This field is required and must match an existing *Broker* resource in the same namespace. |

-| ports[] | Yes | The listener can listen on multiple ports. List of ports that the listener accepts client connections. |

-| ports.authenticationRef | No | Reference to client authentication settings. Omit to disable authentication. To learn more about authentication, see [Configure MQTT broker authentication](howto-configure-authentication.md). |

-| ports.authorizationRef | No | Reference to client authorization settings. Omit to disable authorization. |

-| ports.nodePort | No | Kubernetes node port. Only relevant when this port is associated with a NodePort listener. |

-| ports.port | Yes | TCP port for accepting client connections. |

-| ports.protocol | No | Protocol to use for client connections. Values: `Mqtt`, `Websockets`. Default: `Mqtt` |

-| ports.tls | No | TLS server certificate settings for this port. Omit to disable TLS. |

-| ports.tls.automatic | No | Automatic TLS server certificate management with cert-manager. [Configure TLS with automatic certificate management](howto-configure-tls-auto.md)|

-| ports.tls.automatic.duration | No | Lifetime of certificate. Must be specified using a *Go* time format (h\|m\|s). For example, 240h for 240 hours and 45m for 45 minutes. |

-| ports.tls.automatic.issuerRef | No | cert-manager issuer reference. |

-| ports.tls.automatic.issuerRef.group | No | cert-manager issuer group. |

-| ports.tls.automatic.issuerRef.kind | No | cert-manager issuer kind. Values: `Issuer`, `ClusterIssuer`. |

-| ports.tls.automatic.issuerRef.name | No | cert-manager issuer name. |

-| ports.tls.automatic.privateKey | No | Type of certificate private key. |

-| ports.tls.automatic.privateKey.algorithm | No | Algorithm for the private key. Values: `Ec256`, `Ec384`, `ec521`, `Ed25519`, `Rsa2048`, `Rsa4096`, `Rsa8192`. |

-| ports.tls.automatic.privateKey.rotationPolicy | No | Size of the private key. Values: `Always`, `Never`. |

-| ports.tls.automatic.renewBefore | No | When to begin certificate renewal. Must be specified using a *Go* time format (h\|m\|s). For example, 240h for 240 hours and 45m for 45 minutes. |

-| ports.tls.automatic.san | No | Additional Subject Alternative Names (SANs) to include in the certificate. |

-| ports.tls.automatic.san.dns | No | DNS SANs. |

-| ports.tls.automatic.san.ip | No | IP address SANs. |

-| ports.tls.automatic.secretName | No | Secret for storing server certificate. Any existing data will be overwritten. This is a reference to the secret through an identifying name, not the secret itself. |

-| ports.tls.automatic.secretNamespace | No | Certificate Kubernetes namespace. Omit to use current namespace. |

-| ports.tls.manual | No | Manual TLS server certificate management through a defined secret. For more information, see [Configure TLS with manual certificate management](howto-configure-tls-manual.md).|

-| ports.tls.manual.secretName | Yes | Kubernetes secret containing an X.509 client certificate. This is a reference to the secret through an identifying name, not the secret itself. |

-| ports.tls.manual.secretNamespace | No | Certificate K8S namespace. Omit to use current namespace. |

-| serviceName | No | The name of Kubernetes service created for this listener. Kubernetes creates DNS records for this `serviceName` that clients should use to connect to MQTT broker. This subfield is optional and defaults to `aio-mq-dmqtt-frontend`. |

-| serviceType | No | The type of the Kubernetes service created for this listener. This subfield is optional and defaults to `clusterIp`. Must be either `loadBalancer`, `clusterIp`, or `nodePort`. |

-When you deploy Azure IoT Operations Preview, the deployment also creates a *BrokerListener* resource named `listener` in the `azure-iot-operations` namespace. This listener is linked to the default Broker resource named `broker` that's also created during deployment. The default listener exposes the broker on port 8883 with TLS and SAT authentication enabled. The TLS certificate is [automatically managed](howto-configure-tls-auto.md) by cert-manager. Authorization is disabled by default.

- serviceName: aio-mq-dmqtt-frontend

- - authenticationRef: authn

- port: 8883

- automatic:

- apiGroup: cert-manager.io

-The default BrokerListener uses the service type *ClusterIp*. You can have only one listener per service type. If you want to add more ports to service type *ClusterIp*, you can update the default listener to add more ports. For example, you could add a new port 1883 with no TLS and authentication off with the following kubectl patch command:

- serviceType: loadBalancer

- automatic:

- mode: Automatic

-> During initial deployment, Azure IoT Operations is installed with a default Issuer for TLS server certificates. You can use this issuer for development and testing. For more information, see [Default root CA and issuer with Azure IoT Operations](#default-root-ca-and-issuer-with-azure-iot-operations-preview). The steps below are only required if you want to use a different issuer.

- serviceName: my-new-tls-listener # Avoid conflicts with default service name 'aio-mq-dmqtt-frontend'

- port: 8884 # Avoid conflicts with default port 8883

- tls:

- automatic:

- issuerRef:

- name: my-issuer

- kind: Issuer

- - aio-mq-dmqtt-frontend.azure-iot-operations.svc.cluster.local

-## Default root CA and issuer with Azure IoT Operations Preview

-To help you get started, Azure IoT Operations is deployed with a default "quickstart" root CA and issuer for TLS server certificates. You can use this issuer for development and testing.

-* The CA certificate is self-signed and not trusted by any clients outside of Azure IoT Operations. The subject of the CA certificate is `CN = Azure IoT Operations Quickstart Root CA - Not for Production` and it expires in 30 days from the time of installation.

-* The root CA certificate is stored in a Kubernetes secret called `aio-ca-key-pair-test-only`.

-* The public portion of the root CA certificate is stored in a ConfigMap called `aio-ca-trust-bundle-test-only`. You can retrieve the CA certificate from the ConfigMap and inspect it with kubectl and openssl.

- ```bash

- kubectl get configmap aio-ca-trust-bundle-test-only -n azure-iot-operations -o json | jq -r '.data["ca.crt"]' | openssl x509 -text -noout

- ```

- ```Output

- Certificate:

- Data:

- Version: 3 (0x2)

- Serial Number:

- <SERIAL-NUMBER>

- Signature Algorithm: ecdsa-with-SHA256

- Issuer: CN = Azure IoT Operations Quickstart Root CA - Not for Production

- Validity

- Not Before: Nov 2 00:34:31 2023 GMT

- Not After : Dec 2 00:34:31 2023 GMT

- Subject: CN = Azure IoT Operations Quickstart Root CA - Not for Production

- Subject Public Key Info:

- Public Key Algorithm: id-ecPublicKey

- Public-Key: (256 bit)

- pub:

- <PUBLIC-KEY>

- ASN1 OID: prime256v1

- NIST CURVE: P-256

- X509v3 extensions:

- X509v3 Basic Constraints: critical

- CA:TRUE

- X509v3 Key Usage:

- Certificate Sign

- X509v3 Subject Key Identifier:

- <SUBJECT-KEY-IDENTIFIER>

- Signature Algorithm: ecdsa-with-SHA256

- [SIGNATURE]

- ```

-* By default, there's already a CA issuer configured in the `azure-iot-operations` namespace called `aio-ca-issuer`. It's used as the common CA issuer for all TLS server certificates for IoT Operations. MQTT broker uses an issuer created from the same CA certificate to issue TLS server certificates for the default TLS listener on port 8883. You can inspect the issuer with the following command:

- ```bash

- kubectl get issuer aio-ca-issuer -n azure-iot-operations -o yaml

- ```

- ```Output

- apiVersion: cert-manager.io/v1

- kind: Issuer

- metadata:

- annotations:

- meta.helm.sh/release-name: azure-iot-operations

- meta.helm.sh/release-namespace: azure-iot-operations

- creationTimestamp: "2023-11-01T23:10:24Z"

- generation: 1

- labels:

- app.kubernetes.io/managed-by: Helm

- name: aio-ca-issuer

- namespace: azure-iot-operations

- resourceVersion: "2036"

- uid: <UID>

- spec:

- ca:

- secretName: aio-ca-key-pair-test-only

- status:

- conditions:

- - lastTransitionTime: "2023-11-01T23:10:59Z"

- message: Signing CA verified

- observedGeneration: 1

- reason: KeyPairVerified

- status: "True"

- type: Ready

- ```

- port: 8885 # Avoid port conflict with default listener at 8883

- manual:

- secretName: server-cert-secret

- # Otherwise use the long hostname: aio-mq-dmqtt-frontend.azure-iot-operations.svc.cluster.local

- audience: aio-mq # Must match audience in BrokerAuthentication

- mosquitto_pub --host aio-mq-dmqtt-frontend --port 8883 --message "hello" --topic "world" --debug --cafile /var/run/certs/ca.crt -D CONNECT authentication-method 'K8S-SAT' -D CONNECT authentication-data $(cat /var/run/secrets/tokens/mq-sat)

- mosquitto_sub --host aio-mq-dmqtt-frontend --port 8883 --topic "world" --debug --cafile /var/run/certs/ca.crt -D CONNECT authentication-method 'K8S-SAT' -D CONNECT authentication-data $(cat /var/run/secrets/tokens/mq-sat)

-If you are connecting to the broker from a different namespace, you must use the full service hostname `aio-mq-dmqtt-frontend.azure-iot-operations.svc.cluster.local`. You must also add the DNS name to the server certificate by including a subject alternative name (SAN) DNS field to the *BrokerListener* resource. For more information, see [Configure server certificate parameters](howto-configure-tls-auto.md#optional-configure-server-certificate-parameters).

-For example, to create a K3d cluster with mapping the MQTT broker's default MQTT port 8883 to localhost:8883:

-k3d cluster create --port '8883:8883@loadbalancer'

- port: 8883

- targetPort: 8883

- kubectl get service aio-mq-dmqtt-frontend --namespace azure-iot-operations

- aio-mq-dmqtt-frontend LoadBalancer 10.43.107.11 XXX.XX.X.X 8883:30366/TCP 14h

-> In this example, the mosquitto client uses username and password to authenticate with the broker along with the root CA cert to verify the broker's TLS certificate chain. Here, the `--insecure` flag is required because the default TLS certificate issued to the load balancer is only valid for the load balancer's default service name (aio-mq-dmqtt-frontend) and assigned IPs, not localhost.

-1. To access the broker, forward the broker listening port 8883 to the host.

- kubectl port-forward --namespace azure-iot-operations service/aio-mq-dmqtt-frontend 8883:mqtts-8883

-1. Use 127.0.0.1 to connect to the broker at port 8883 with the same authentication and TLS configuration as the example without port forwarding.

-1. Assume that the broker's service is exposed to an external IP using a load balancer. For example if you patched the default load balancer `aio-mq-dmqtt-frontend`, get the external IP address for the service.

- kubectl get service aio-mq-dmqtt-frontend --namespace azure-iot-operations

- aio-mq-dmqtt-frontend LoadBalancer 10.43.107.11 192.168.0.4 8883:30366/TCP 14h

-1. Set up port forwarding to the `aio-mq-dmqtt-frontend` service on the external IP address `192.168.0.4` and port `8883`:

- netsh interface portproxy add v4tov4 listenport=8883 connectport=8883 connectaddress=192.168.0.4

- New-NetFirewallRule -DisplayName "AIO MQTT Broker" -Direction Inbound -Protocol TCP -LocalPort 8883 -Action Allow

-description: Use MQTT broker to publish and subscribe to messages. Destinations include other MQTT brokers, Azure IoT Data Processor, and Azure cloud services.

-* The _operations experience_ is a web UI that provides a unified experience for operational technologists to manage assets and data processor pipelines in an Azure IoT Operations deployment. An IT administrator can use [Azure Arc site manager (preview)](/azure/azure-arc/site-manager/overview) to group Azure IoT Operations instances by physical location and make it easier for OT users to find instances.

-> During public preview, there's no support for upgrading an existing Azure IoT Operations deployment to a newer version. Instead, remove Azure IoT Operations from your cluster and then deploy the latest version. For more information, see [Update Azure IoT Operations](deploy-iot-ops/howto-deploy-iot-operations.md#update-azure-iot-operations).

-The [connector for OPC UA](discover-manage-assets/overview-opcua-broker.md) manages the connection to OPC UA servers and other leaf devices. The connector for OPC UA publishes data from the OPC UA servers and the devices discovered by _Akri services_ to MQTT broker topics.

-The [Akri services](discover-manage-assets/overview-akri.md) help you discover and connect to other types of devices and assets.

-* Data processor pipelines subscribe to MQTT topics to retrieve messages for processing.

-In Azure IoT operations v0.6.0, the data processor is replaced by [dataflows](./connect-to-cloud/overview-dataflow.md). Dataflows provide enhanced data transformation and data contextualization capabilities within Azure IoT Operations.

-> If you want to continue using the data processor, you must deploy Azure IoT Operations v0.5.1 with the additional flag to include data processor component. It's not possible to deploy the data processor with Azure IoT Operations v0.6.0. The Azure IoT operations CLI extension that includes the flag for deploying the data processor is version 0.5.1b1. This version requires Azure CLI v2.46.0 or greater. The data processor documentation is currently available on the previous versions site: [Azure IoT Operations data processor](/previous-versions/azure/iot-operations/process-data/overview-data-processor).

-<!-- TODO: Fix the previous versions link before we publish -->

-In the 0.6.x public preview release, Azure IoT Operations supports clusters that are Arc-enabled in the following regions:

-* East US

-* East US 2

-* West US

-* West US 2

-* West Europe

-* North Europe

->[!NOTE]

->West US 3 was supported in previous versions of Azure IoT Operations, but isn't supported in version 0.6.x.

-### Akri services

-This component helps you discover and connect to devices and assets.

-### Data processor

-This component lets you aggregate, enrich, normalize, and filter the data from your devices and assets. The data processor is a pipeline-based data processing engine that lets you process data at the edge before you send it to the other services either at the edge or in the cloud

-This component manages the connection to OPC UA servers and other leaf devices. The connector for OPC UA publishes data from the OPC UA servers and the devices discovered by _Akri services_ to MQTT broker topics.

-This web UI provides a unified experience for operational technologists to manage assets and data processor pipelines in an Azure IoT Operations deployment.

-description: Available observability metrics for Akri services to monitor the health and performance of your solution.

- - ignite-2023

-# CustomerIntent: As an IT admin or operator, I want to be able to monitor and visualize data

-# on the health of my industrial assets and edge environment.

-# Metrics for Akri services

-Akri services provide a set of observability metrics that you can use to monitor and analyze the health of your solution. This article lists the available metrics, and describes the meaning and usage details of each metric.

-## Available metrics

-| Metric name | Definition |

-| -- | - |

-| instance_count | The number of OPC UA assets that Akri services detects and adds as a custom resource to the cluster at the edge. |

-| discovery_response_result | The success or failure of every discovery request that the Agent sends to the Discovery Handler.|

-| discovery_response_time | The time in seconds from the point when Akri services apply the configuration, until the Agent makes the first discovery request.|

-## Related content

-description: Known issues for the MQTT broker, Layered Network Management, connector for OPC UA, OPC PLC simulator, data processor, and operations experience web UI.

-## MQTT broker

- As a workaround, first fix the connection problem. Then either restart all the pods in the cluster with pod names that start with "aio-opc-opc.tcp", or delete the `AssetEndpointProfile` and deploy it again.

- As a workaround, add the server's certificate to the trusted certificates store as described in [Configure the trusted certificates list](../discover-manage-assets/howto-configure-opcua-certificates-infrastructure.md#configure-the-trusted-certificates-list).

- Or, you can [Optionally configure your AssetEndpointProfile without mutual trust established](../discover-manage-assets/howto-configure-opc-plc-simulator.md#optionally-configure-your-assetendpointprofile-without-mutual-trust-established). This workaround should not be used in production environments.

-> Don't use this configuration in production or pre-production environments. Exposing your cluster to the internet without proper authentication might lead to unauthorized access and even DDOS attacks.

- ```yaml

- apiVersion: connectivity.iotoperations.azure.com/v1beta1

- kind: DataflowEndpoint

- metadata:

- name: kafka-target

- namespace: azure-iot-operations

- spec:

- endpointType: kafkaSettings

- kafkaSettings:

- host: "<NAMESPACE>.servicebus.windows.net:9093"

- batching:

- latencyMs: 0

- maxMessages: 100

- tls:

- mode: Enabled

- copyMqttProperties: enabled

- authentication:

- method: SystemAssignedManagedIdentity

- systemAssignedManagedIdentitySettings:

- audience: https://<NAMESPACE>.servicebus.windows.net

- ```

-## Akri services

-When Akri services generate an asset endpoint for a discovered asset, the configuration may contain an invalid setting that prevents the asset from connecting to the MQTT broker. To resolve this issue, edit the `AssetEndpointProfile` configuration and remove the `"securityMode":"none"` setting from thr `additionalConfiguration` property. For example, the configuration for the `opc-ua-broker-opcplc-000000-50000` asset endpoint generated in the quickstarts should look like the following example:

-```yml

-apiVersion: deviceregistry.microsoft.com/v1beta1

-kind: AssetEndpointProfile

-metadata:

- creationTimestamp: "2024-08-05T11:41:21Z"

- generation: 2

- name: opc-ua-broker-opcplc-000000-50000

- namespace: azure-iot-operations

- resourceVersion: "233018"

- uid: f9cf479f-7a77-49b5-af88-18d509e9cdb0

-spec:

- additionalConfiguration: '{"applicationName":"opc-ua-broker-opcplc-000000-50000","keepAliveMilliseconds":10000,"defaults":{"publishingIntervalMilliseconds":1000,"queueSize":1,"samplingIntervalMilliseconds":1000},"subscription":{"maxItems":1000,"lifetimeMilliseconds":360000},"security":{"autoAcceptUntrustedServerCertificates":true,"securityPolicy":"http://opcfoundation.org/UA/SecurityPolicy#None"},"session":{"timeoutMilliseconds":60000,"keepAliveIntervalMilliseconds":5000,"reconnectPeriodMilliseconds":500,"reconnectExponentialBackOffMilliseconds":10000}}'

- targetAddress: "\topc.tcp://opcplc-000000:50000"

- transportAuthentication:

- ownCertificates: []

- userAuthentication:

- mode: Anonymous

- uuid: opc-ua-broker-opcplc-000000-50000

-```

-A sporadic issue might cause the `aio-opc-asset-discovery` pod to restart with the following error in the logs: `opcua@311 exception="System.IO.IOException: Failed to bind to address http://unix:/var/lib/akri/opcua-asset.sock: address already in use.`.

-To work around this issue, use the following steps to update the **DaemonSet** specification:

-1. Locate the **target** custom resource provided by `orchestration.iotoperations.azure.com` with a name that ends with `-ops-init-target`:

- ```console

- kubectl get targets -n azure-iot-operations

- ```

-1. Edit the target configuration and find the `spec.components.aio-opc-asset-discovery.properties.resource.spec.template.spec.containers.env` parameter. For example:

- ```console

- kubectl edit target solid-zebra-97r6jr7rw43vqv-ops-init-target -n azure-iot-operations

- ```

-1. Add the following environment variables to the `spec.components.aio-opc-asset-discovery.properties.resource.spec.template.spec.containers.env` configuration section:

- ```yml

- - name: ASPNETCORE_URLS

- value: http://+8443

- - name: POD_IP

- valueFrom:

- fieldRef:

- fieldPath: "status.podIP"

- ```

-1. Save your changes. The final specification looks like the following example:

- ```yml

- apiVersion: orchestrator.iotoperations.azure.com/v1

- kind: Target

- metadata:

- name: <cluster-name>-target

- namespace: azure-iot-operations

- spec:

- displayName: <cluster-name>-target

- scope: azure-iot-operations

- topologies:

- ...

- version: 1.0.0.0

- components:

- ...

- - name: aio-opc-asset-discovery

- type: yaml.k8s

- properties:

- resource:

- apiVersion: apps/v1

- kind: DaemonSet

- metadata:

- labels:

- app.kubernetes.io/part-of: aio

- name: aio-opc-asset-discovery

- spec:

- selector:

- matchLabels:

- name: aio-opc-asset-discovery

- template:

- metadata:

- labels:

- app.kubernetes.io/part-of: aio

- name: aio-opc-asset-discovery

- spec:

- containers:

- - env:

- - name: ASPNETCORE_URLS

- value: http://+8443

- - name: POD_IP

- valueFrom:

- fieldRef:

- fieldPath: status.podIP

- - name: DISCOVERY_HANDLERS_DIRECTORY

- value: /var/lib/akri

- - name: AKRI_AGENT_REGISTRATION

- value: 'true'

- image: >-

- edgeappmodel.azurecr.io/opcuabroker/discovery-handler:0.4.0-preview.3

- imagePullPolicy: Always

- name: aio-opc-asset-discovery

- ports: ...

- resources: ...

- volumeMounts: ...

- volumes: ...

- ```

-## Operations experience web UI

-To sign in to the operations experience, you need a Microsoft Entra ID account with at least contributor permissions for the resource group that contains your **Kubernetes - Azure Arc** instance. You can't sign in with a Microsoft account (MSA). To create an account in your Azure tenant:

-1. Sign in to the [Azure portal](https://portal.azure.com/) with the same tenant and user name that you used to deploy Azure IoT Operations.

-1. In the Azure portal, go to the **Microsoft Entra ID** section, select **Users > +New user > Create new user**. Create a new user and make a note of the password, you need it to sign in later.

-1. In the Azure portal, go to the resource group that contains your **Kubernetes - Azure Arc** instance. On the **Access control (IAM)** page, select **+Add > Add role assignment**.

-1. On the **Add role assignment page**, select **Privileged administrator roles**. Then select **Contributor** and then select **Next**.

-1. On the **Members** page, add your new user to the role.

-1. Select **Review and assign** to complete setting up the new user.

-You can now use the new user account to sign in to the [Azure IoT Operations](https://iotoperations.azure.com) portal.

-[Azure CLI version 2.46.0 or higher](/cli/azure/install-azure-cli) is required and the [Azure IoT Operations extension](/cli/azure/iot/ops) installed.

-## Data processor pipeline deployment troubleshooting

-If your data processor pipeline deployment status is showing as **Failed**, use the following commands to find the pipeline error codes.

-To list the data processor pipeline deployments, run the following command:

-```bash

-kubectl get pipelines -A

-```

-The output from the pervious command looks like the following example:

-```text

-NAMESPACE NAME AGE

-azure-iot-operations passthrough-data-pipeline 2d20h

-azure-iot-operations reference-data-pipeline 2d20h

-azure-iot-operations contextualized-data-pipeline 2d20h

-```

-To view detailed information for a pipeline, run the following command:

-```bash

-kubectl describe pipelines passthrough-data-pipeline -n azure-iot-operations

-```

-The output from the previous command looks like the following example:

-```text

-...

-Status:

- Provisioning Status:

- Error

- Code: <ErrorCode>

- Message: <ErrorMessage>

- Status: Failed

-Events: <none>

-```

-If you see the following message when you try to access the **Pipelines** tab in the Azure IoT Operations (preview) portal:

-_Data Processor not found in the current deployment. Please re-deploy with the additional argument to include the data processor._

-You need to deploy Azure IoT Operations with the optional data processor component included. To do this, you need to add the `--include-dp` argument when you run the [az iot ops init](/cli/azure/iot/ops#az-iot-ops-init) command. You must use the `--include-dp` argument to include the data processor component when you first deploy Azure IoT Operations. You can't add this optional component to an existing deployment.

-> [!TIP]

-> If you want to delete the Azure IoT Operations deployment but plan on reinstalling it on your cluster, use the [az iot ops delete](/cli/azure/iot/ops?az-iot-ops-delete) command.

-1. Verify the regions are supported for public preview. Public preview supports eight regions. For more information, see [Quickstart: Run Azure IoT Operations Preview in Github Codespaces with K3s](../get-started-end-to-end-sample/quickstart-deploy.md#connect-a-kubernetes-cluster-to-azure-arc).

-# Tutorial: Create a multiple virtual machine inbound NAT rule using the Azure portal

-> [Create a cross-region load balancer using the Azure portal](tutorial-cross-region-portal.md)

-In this article, you will learn how to:

-1. Click on **Discover** > **Are your machines virtualized?**

-2. Click **Generate key** to start the creation of the required Azure resources. Do not close the Discover page during the creation of resources.

-3. Copy the generated key. You will need the key later to complete the registration of the scale-out appliance.

-In **Download Azure Migrate appliance**, click **Download**. You need to download the PowerShell installer script to deploy the scale-out appliance on an existing server running Windows Server 2019 or Windows Server 2022 and with the required hardware configuration (32-GB RAM, 8 vCPUs, around 80 GB of disk storage and internet access, either directly or through a proxy).

-5. Select from the scenario, cloud, configuration and connectivity options to deploy the desired appliance. For instance, the selection shown below sets up a **scale-out appliance** to initiate concurrent replications on servers running in your VMware environment to an Azure Migrate project with **default _(public endpoint)_ connectivity** on **Azure public cloud**.

- 2. The appliance will verify the key and start the auto-update service, which updates all the services on the appliance to their latest versions. When the auto-update has run, you can select **View appliance services** to see the status and versions of the services running on the appliance server.

- :::image type="content" source="./media/tutorial-discover-vmware/device-code.png" alt-text="Screenshot that shows where to copy the device code and log in.":::

- 5. After you successfully log in, return to the browser tab that displays the appliance configuration manager. If the Azure user account that you used to log in has the required permissions for the Azure resources that were created during key generation, appliance registration starts.

-To complete the registration of the scale-out appliance, click **import** to get the necessary configuration files from the primary appliance.

-1. Login (remote desktop) to the primary appliance and execute the following PowerShell commands:

-1. In the pop-up window opened in the previous step, select the location of the copied configuration zip file and click **Save**.

- Once the files have been successfully imported, the registration of the scale-out appliance will complete and it will show you the timestamp of the last successful import. You can also see the registration details by clicking **View details**.

-At this point, you should revalidate that the scale-out appliance is able to connect to your vCenter server. Click **revalidate** to validate vCenter Server connectivity from scale-out appliance.

-The Migration and modernization tool will take care of distributing the virtual machines between the primary and scale-out appliance for replication. Once the replication is done, you can migrate the virtual machines.

-A chamber with a public IP connector configured to allow users who's IP is listed after the first entry of the allowlist can't access the chamber either through the desktop or data pipeline using AzCopy. If the allowlist on a public IP connector contains overlapping networks, in some instances the preprocessor might fail to detect the overlapping networks before attempting to commit them to the active NSG. Failures aren't reported back to the user. Other NSG rules elsewhere - either before or after the interfering rule - might not be processed, defaulting to the "deny all" rule. Access to the connector might be blocked unexpectedly for users that previously had access and appear elsewhere in the list. Access is blocked for all connector interactions including desktop, data pipeline upload, and data pipeline download. The connector still responds to port queries, but doesn't allow interactions from an IP or IP range shown in the connector networking allowlist.

-IP address requirements are different between Oracle Database@Azure and in OCI. In the [Requirements for IP Address Space](https://docs.oracle.com/iaas/exadatacloud/doc/ecs-network-setup.html#ECSCM-GUID-D5C577A1-BC11-470F-8A91-77609BBEF1EA) documentation for Exadata in , the following differences with the requirements of Oracle Database@Azure must be considered:

-* **Move the resource to a new resource group.**

- 1. Follow the steps to access the resource blade.

- 1. Select the link to the resource from the **Name** field in the table.

- 1. From the resource's overview page, select the **Move** link on the Resource group field.

- 1. From the **Move resources** page, use the drop-down field for **Resource group** to select an existing resource group.

- 1. To create and use a new resource group, select the **Create new** link below the Resource group field. Enter a new resource group name in the **Name** field. Select the **OK** button to save your new resource group and use it. Select the **Cancel** button to return without creating a new resource group.

-* **Move the resource to a new subscription.**

- > [!NOTE]

- > You must have access to another Microsoft Azure subscription, and that subscription must have been setup for access to OracleDB@Azure. If both of these conditions are not met, you will not be able to successfully move the resource to another subscription.

- 1. Follow the steps to access the resource blade.

- 1. Select the link to the resource from the Name field in the table.

- 1. From the resource's overview page, select the **Move** link on the Subscription field.

- 1. From the **Move resources** page, use the drop-down field for **Subscription** to select an existing subscription.

- 1. You can also simultaneously move the resource group for the resource. To do this, note the steps in the **Move the resource to a new resource group** tasks. **Add, manage, or delete tags for the resource.**

- 1. Follow the steps to access the resource blade.

- 1. Select the link to the resource from the Name field in the table.

- 1. From the resource's overview page, select the **Edit** link on the **Tags** field.

- 1. To create a new tag, enter values in the **Name** and **Value** fields.

- 1. To edit an existing tag, change the value in the existing tag's **Value** field.

- 1. To delete an existing tag, select the **Trashcan** icon at the right-side of the tag.

- 1. Manage the resource from within the OCI console.

-description: Provides specs on Microsoft ground stations and outlines partner ground station network.

-description: Learn more about a contact resource and how to schedule a contact.

-description: Learn how to configure a contact profile