+Face Liveness detection can be used to determine if a face in an input video stream is real (live) or fake (spoofed). It's an important building block in a biometric authentication system to prevent imposters from gaining access to the system using a photograph, video, mask, or other means to impersonate another person.

+The goal of liveness detection is to ensure that the system is interacting with a physically present live person at the time of authentication. Such systems are increasingly important with the rise of digital finance, remote access control, and online identity verification processes.

+The Azure AI Face liveness detection solution successfully defends against various spoof types ranging from paper printouts, 2d/3d masks, and spoof presentations on phones and laptops. Liveness detection is an active area of research, with continuous improvements being made to counteract increasingly sophisticated spoofing attacks over time. Continuous improvements will be rolled out to the client and the service components over time as the overall solution gets more robust to new types of attacks.

+| -- |--|

+## Prerequisites

+## Set up frontend applications and app servers to perform liveness detection

+We provide SDKs in different languages for frontend applications and app servers. See the following instructions to set up your frontend applications and app servers.

+### Download SDK for frontend application

+Once you have access to the SDK, follow instructions in the [azure-ai-vision-sdk](https://github.com/Azure-Samples/azure-ai-vision-sdk) GitHub repository to integrate the UI and the code into your native mobile application. The liveness SDK supports Java/Kotlin for Android mobile applications, Swift for iOS mobile applications and JavaScript for web applications:

+Once you've added the code into your application, the SDK handles starting the camera, guiding the end-user in adjusting their position, composing the liveness payload, and calling the Azure AI Face cloud service to process the liveness payload.

+### Download Azure AI Face client library for app server

+#### Create environment variables

+## Perform liveness detection

+1. The SDK then starts the camera, guides the user to position correctly, and then prepares the payload to call the liveness detection service endpoint.

+## Perform liveness detection with face verification

+### Select a good reference image

+#### Technical requirements

+#### Composition requirements

+### Set up the orchestration of liveness with verification.

+## Clean up resources

+* The [tutorials](./Tutorials/liveness.md) are longer guides that show you how to use this service as a component in broader business solutions.

+Mistral AI offers two categories of models in the [Azure AI Studio](https://ai.azure.com). These models are available in the [model catalog](model-catalog-overview.md):

+* __Premium models__: Mistral Large and Mistral Small. These models can be deployed as serverless APIs with pay-as-you-go token-based billing.

+* __Open models__: Mixtral-8x7B-Instruct-v01, Mixtral-8x7B-v01, Mistral-7B-Instruct-v01, and Mistral-7B-v01. These models can be deployed to managed computes in your own Azure subscription.

+You can browse the Mistral family of models in the model catalog by filtering on the Mistral collection.

+- An [AI Studio hub](../how-to/create-azure-ai-resource.md). The serverless API model deployment offering for eligible models in the Mistral family is only available with hubs created in these regions:

+ - East US

+ - East US 2

+ - North Central US

+ - South Central US

+ - West US

+ - West US 3

+ - Sweden Central

+ For a list of regions that are available for each of the models supporting serverless API endpoint deployments, see [Region availability for models in serverless API endpoints](deploy-models-serverless-availability.md).

+1. Select the project in which you want to deploy your model. To use the serverless API model deployment offering, your project must belong to one of the regions listed in the [prerequisites](#prerequisites).

+## Reference for Mistral family of models deployed as a service

+### Mistral Chat API

+#### Request example

+#### Response example

+| **Sample Type** | **Sample Notebook** |

+|-|-|

+| CLI using CURL and Python web requests | [webrequests.ipynb](https://aka.ms/mistral-large/webrequests-sample) |

+| OpenAI SDK (experimental) | [openaisdk.ipynb](https://aka.ms/mistral-large/openaisdk) |

+| LangChain | [langchain.ipynb](https://aka.ms/mistral-large/langchain-sample) |

+| Mistral AI | [mistralai.ipynb](https://aka.ms/mistral-large/mistralai-sample) |

+| LiteLLM | [litellm.ipynb](https://aka.ms/mistral-large/litellm-sample) |

+- [Region availability for models in serverless API endpoints](deploy-models-serverless-availability.md)

+If you configured your virtual network with a list of custom DNS servers, name resolution in App Service will use up to five custom DNS servers. If your virtual network is using custom DNS servers and you're using private endpoints, you should read [this article](../private-link/private-endpoint-dns.md) carefully. You also need to consider that your custom DNS servers are able to resolve any public DNS records used by your app. Your DNS configuration needs to either forward requests to a public DNS server, include a public DNS server like Azure DNS in the list of custom DNS servers or specify an alternative server at the app level.

+> This article is applicable only for PowerShell 5.1. PowerShell 7+ versions do not support Workflows, and outdated runbooks cannot be updated. We recommend you to use PowerShell 7.2 textual runbooks for advanced features such as parallel job execution. [Learn more](../automation-runbook-types.md#limitations) about limitations of PowerShell Workflow runbooks.

+This tutorial shows you how to implement Key Vault references in your code. It builds on the web app introduced in the ASP.NET core quickstart listed in the prerequisites below. Before you continue, complete this [quickstart](./quickstart-aspnet-core-app.md).

+Finish the quickstart: [Create an ASP.NET Core app with App Configuration](./quickstart-aspnet-core-app.md).

+**Create VMware resource from the specified Arc for Server machine in the vCenter**

+az connectedvmware vm create-from-machines --resource-group contoso-rg --name contoso-vm --vcenter-id /subscriptions/999998ee-cd13-9999-b9d4-55ca5c25496d/resourceGroups/allhands-demo/providers/microsoft.connectedvmwarevsphere/VCenters/ContosovCentervcenters/contoso-vcenter

+az connectedvmware vm create-from-machines --resource-group contoso-rg --vcenter-id /subscriptions/999998ee-cd13-9999-b9d4-55ca5c25496d/resourceGroups/allhands-demo/providers/microsoft.connectedvmwarevsphere/VCenters/ContosovCentervcenters/contoso-vcenter

+az connectedvmware vm create-from-machines --subscription contoso-sub --vcenter-id /subscriptions/999998ee-cd13-9999-b9d4-55ca5c25496d/resourceGroups/allhands-demo/providers/microsoft.connectedvmwarevsphere/VCenters/ContosovCentervcenters/contoso-vcenter

+Azure Functions provides integrated support for developing, deploying, and managing containerized function apps on [Azure Container Apps](../container-apps/overview.md). Use Azure Container Apps to host your function app containers when you need to run your event-driven functions in Azure in the same environment as other microservices, APIs, websites, workflows, or any container hosted programs. Container Apps hosting lets you run your functions in a fully managed, Kubernetes-based environment with built-in support for open-source monitoring, mTLS, Dapr, and Kubernetes Event-driven Autoscaling (KEDA).

+1. Select **Enter a URL** and enter a valid URL pointing to your hosted file. Hosted files must be a valid spatial dataset with a `.json`, `.geojson`, `.wkt`, `.kml` or `.shp` extension. After the link to the hosted file is added to the reference layer, the URL appears in the **Enter a URL** field. To remove the data from the visual simply delete the URL.

+ :::image type="content" source="./media/power-bi-visual/reference-layer-hosted.png" alt-text="Screenshot showing the reference layers section when using the 'Enter a URL' input control.":::

+1. Alternatively, you can create a dynamic URL using Data Analysis Expressions ([DAX]) based on fields, variables or other programmatic elements. By utilizing DAX, the URL will dynamically change based on filters, selections, or other user interactions and configurations. For more information, see [Expression-based titles in Power BI Desktop].

+ :::image type="content" source="./media/power-bi-visual/reference-layer-hosted-dax.png" alt-text="Screenshot showing the reference layers section when using DAX for the URL input.":::

+[DAX]: /dax/

+[Expression-based titles in Power BI Desktop]: /power-bi/create-reports/desktop-conditional-format-visual-titles

+| API |`api.applicationinsights.io`<br/>`api1.applicationinsights.io`<br/>`api2.applicationinsights.io`<br/>`api3.applicationinsights.io`<br/>`api4.applicationinsights.io`<br/>`api5.applicationinsights.io`<br/>`dev.applicationinsights.io`<br/>`dev.applicationinsights.microsoft.com`<br/>`dev.aisvc.visualstudio.com`<br/>`www.applicationinsights.io`<br/>`www.applicationinsights.microsoft.com`<br/>`www.aisvc.visualstudio.com`<br/>`api.loganalytics.io`<br/>`*.api.loganalytics.io`<br/>`dev.loganalytics.io`<br>`docs.loganalytics.io`<br/>`www.loganalytics.io`<br/>`api.loganalytics.azure.com` |20.37.52.188 <br/> 20.37.53.231 <br/> 20.36.47.130 <br/> 20.40.124.0 <br/> 20.43.99.158 <br/> 20.43.98.234 <br/> 13.70.127.61 <br/> 40.81.58.225 <br/> 20.40.160.120 <br/> 23.101.225.155 <br/> 52.139.8.32 <br/> 13.88.230.43 <br/> 52.230.224.237 <br/> 52.242.230.209 <br/> 52.173.249.138 <br/> 52.229.218.221 <br/> 52.229.225.6 <br/> 23.100.94.221 <br/> 52.188.179.229 <br/> 52.226.151.250 <br/> 52.150.36.187 <br/> 40.121.135.131 <br/> 20.44.73.196 <br/> 20.41.49.208 <br/> 40.70.23.205 <br/> 20.40.137.91 <br/> 20.40.140.212 <br/> 40.89.189.61 <br/> 52.155.118.97 <br/> 52.156.40.142 <br/> 23.102.66.132 <br/> 52.231.111.52 <br/> 52.231.108.46 <br/> 52.231.64.72 <br/> 52.162.87.50 <br/> 23.100.228.32 <br/> 40.127.144.141 <br/> 52.155.162.238 <br/> 137.116.226.81 <br/> 52.185.215.171 <br/> 40.119.4.128 <br/> 52.171.56.178 <br/> 20.43.152.45 <br/> 20.44.192.217 <br/> 13.67.77.233 <br/> 51.104.255.249 <br/> 51.104.252.13 <br/> 51.143.165.22 <br/> 13.78.151.158 <br/> 51.105.248.23 <br/> 40.74.36.208 <br/> 40.74.59.40 <br/> 13.93.233.49 <br/> 52.247.202.90 |80,443 |

+If the `audioOutputLevel` value is low, it's likely that the volume sent by the sender is also low.

+The value of `audioOutputLevel` ranges from 0 - 65536. In practice, values lower than 60 can be considered quiet, and values lower than 150 are often considered low volume.

+If the `audioOutputLevel` value appears normal, the issue may be related to system volume settings or speaker issues on the receiver's side.

+For example, if the user uses Windows, they should check the volume mixer settings and apps volume settings.

+## References

+### Troubleshooting process

+Below is a flow diagram of the troubleshooting process for this issue.

+1. When a user reports experiencing low audio volume, the first thing to check is whether the volume of the incoming audio is low. The application can obtain this information by checking `audioOutputLevel` in the media stats.

+2. If the `audioOutputLevel` value is constantly low, it indicates that the volume of audio sent by the speaking participant is low. In this case, ask the user to verify if the speaking participant has issues with their microphone device or input volume settings.

+3. If the `audioOutputLevel` value isn't always low, the user may still experience low audio volume issue due to system volume settings. Ask the user to check their system volume settings.

+4. If the user's system volume is set to a low value, the user should increase the volume in the settings.

+5. In some systems that support app-specific volume settings, the audio volume output from the app may be low even if system volume isn't low. In this case, the user should check their volume setting of the app within the system.

+6. If the volume setting of the app in the system is low, the user should increase it.

+7. If you still can't determine why the audio output volume is low, ask the user to check their speaker device or select another audio output device. The issue may be due to a device problem and not related to the software or operating system. Not all platforms support speaker enumeration in the browser. For example, you can't select an audio output device through the JavaScript API in the Safari browser or in Chrome on Android. In these cases, you should configure the audio output device in the system settings.

+## References

+### Troubleshooting process

+If a user can't hear sound during a call, one possibility is that the speaking participant has an issue with their microphone.

+If the speaking participant is using your application, you can follow this flow diagram to troubleshoot the issue.

+1. First, check if a microphone is available. The application can obtain this information by invoking `DeviceManager.getMicrophone` API or by detecting a `noMicrophoneDevicesEnumerated` UFD Bad event.

+2. If no microphone device is available, prompt the user to plug in a microphone.

+3. If a microphone is available but there's no outgoing audio, consider other possibilities such as permission issues, device issues, or network problems.

+4. If permission is denied, refer to [The speaking participant doesn't grant the microphone permission](./microphone-permission.md) for more information.

+5. If permission is granted, consider whether the issue is due to an external problem, such as `microphoneMuteUnexpectedly` UFD.

+6. The `microphoneMuteUnexpectedly` UFD Bad event is triggered when the browser mutes the audio input track. The application can monitor this UFD but isn't able to detect the reason at JavaScript layer. You can still provide instructions in the app and ask if the user is using hardware mute button on their headset.

+7. If the user releases the hardware mute and the `microphoneMuteUnexpectedly` UFD recovers, the issue is resolved.

+8. If the user isn't using the hardware mute, ask the user to unplug and replug the microphone, or to select another microphone. Ensure the user hasn't muted the microphone at the system level.

+9. No outgoing audio issue can also happen when there's a `microphoneNotFunctioning` UFD Bad event.

+10. If there's no `microphoneNotFunctioning` UFD Bad event, consider other possibilities, such as network issues.

+11. If there's a `networkReconnect` Bad UFD, outgoing audio may be temporarily lost due to a network disconnection. Refer to [There's a network issue in the call](./network-issue.md) for detailed information.

+12. If there are no microphone-related events and no network-related events, create a support ticket for ACS team to investigate the issue. Refer to [Reporting an issue](../general-troubleshooting-strategies/report-issue.md).

+13. If a `microphoneNotFunctioning` UFD Bad event occurs, and the user has no outgoing audio, they can try to recover the stream by using ACS [mute](/javascript/api/azure-communication-services/@azure/communication-calling/call?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-call-mute) and [unmute](/javascript/api/azure-communication-services/@azure/communication-calling/call?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-call-unmute).

+14. If the `microphoneNotFunctioning` UFD doesn't recover after the user performs ACS mute and unmute, there might be an issue with the microphone device. Ask the user to unplug and replug the microphone or select another microphone.

+Your application should always call the [askDevicePermission](/javascript/api/azure-communication-services/@azure/communication-calling/devicemanager?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-devicemanager-askdevicepermission) API after the `CallClient` is initialized.

+The application can also show a warning message if the user denies the permission, so the user can fix it before joining a call.

+It's also important to listen for the [microphonePermissionDenied](../references/ufd/microphone-permission-denied.md) UFD event. Display a warning message if the user revokes the permission during the call. By doing so, the user is aware of the issue and can adjust their browser or system settings accordingly.

+## References

+### Troubleshooting process

+If a user can't hear sound during a call, one possibility is that the speaking participant hasn't granted microphone permission.

+If the speaking participant is using your application, you can follow this flow diagram to troubleshoot the issue.

+1. Check if there's a `microphonePermissionDenied` Bad UFD event for the speaking participant. This usually indicates that the user has denied the permission or that the permission isn't requested.

+2. If a `microphonePermissionDenied` Bad UFD event occurs, verify whether the app has called `askDevicePermission` API.

+3. The app must call `askDevicePermission` if this API hasn't been invoked before the user joins the call. The app can offer a smoother user experience by determining the current state of permissions. For instance, it can display a message instructing the user to adjust their permissions if necessary.

+4. If the app has called `askDevicePermission` API, but the user still gets a `microphonePermissionDenied` Bad UFD event. The user has to reset or grant the microphone permission in the browser. If they have confirmed that the permission is granted in the browser, they should check if the OS is blocking mic access to the browser.

+5. If there's no `microphonePermissionDenied` Bad UFD, we need to consider other possibilities. For the speaking participant, there might be other potential reasons for issues with outgoing audio, such as network reconnection, or device issues.

+6. If there's a `networkReconnect` Bad UFD, the outgoing audio may be temporarily lost due to a network disconnection. See [There's a network issue in the call](./network-issue.md) for detailed information.

+7. If no `networkReconnect` Bad UFD occurs, there might be a problem on the speaking participant's microphone. See [The speaking participant's microphone has a problem](./microphone-issue.md) for detailed information.

+## References

+### Troubleshooting process

+If a user can't hear sound during a call, one possibility is that the speaking participant or the receiving end has network issues.

+Below is a flow diagram of the troubleshooting process for this issue.

+1. First, check if there's a `networkReconnect` UFD. The user may experience audio loss during the network reconnection.

+2. The UFD can happen on either the sender's end or the receiver's end. In both cases, packets don't flow, so the user can't hear the audio.

+3. If there's no `networkReconnect` UFD, consider other potential causes, such as permission issues or device problems.

+4. If the permission is denied, refer to [The speaking participant doesn't grant the microphone permission](./microphone-permission.md) for more information.

+5. The issue could also be due to device problems, refer to [The speaking participant's microphone has a problem](./microphone-issue.md).

+- The audio output device is functioning properly

+## References

+### Troubleshooting process

+Below is a flow diagram of the troubleshooting process for this issue.

+1. When a user reports experiencing poor audio quality during a call, the first thing to check is the source of the issue. It could be coming from the sender's side or the receiver's side. If other participants on different networks also have similar issues, it's very possible that the issue comes from the sender's side.

+2. Check if there's `networkSendQuality` UFD Bad event on the sender's side.

+3. If there's no `networkSendQuality` UFD Bad event on the sender's side, the poor audio could be due to device issues or audio distortion caused by the browser's audio processing module. Ask the user to collect diagnostic audio recordings from the browser. Refer to [How to collect diagnostic audio recordings](../references/how-to-collect-diagnostic-audio-recordings.md)

+4. If there's a `networkSendQuality` UFD Bad event, the poor audio quality might be due to the sender's network issues. Check the sender's network.

+5. If the user experiences poor audio quality but no other participants have the same issue, and there are only two participants in the call, still check the sender's network.

+6. If the user experiences poor audio quality but no other participants have the same issue in a group call, the issue might be due to the receiver's network. Check for a `networkReceiveQuality` UFD Bad event on the receiver's end.

+7. If there's a `networkReceiveQuality` UFD Bad event, check the receiver's network.

+8. If you can't find a `networkReceiveQuality` UFD Bad event, check if other media stats metrics on the receiver's end are poor, such as packetsLost, jitter, etc.

+9. If you can't determine why the audio quality on the receiver's end is poor, create a support ticket for the ACS team to investigate.

+If the user uses Windows, they should also check the volume mixer settings and apps volume settings.

+If you're using Web Audio API in your application, you might also check if there's `AudioRenderer error with rendering audio code: 3` in the log.

+This error occurs when there are too many AudioContext instances open at the same time, particularly if the application doesn't properly close the AudioContext or

+if there's an AudioContext creation associated with the UI component refresh logic.

+## References

+### Troubleshooting process

+If a user can't hear sound during a call, one possibility is that the participant has an issue with their speaker.

+The speaker issue isn't easily detected and usually requires users to check their system settings or their audio output devices.

+Below is a flow diagram of the troubleshooting process for this issue.

+1. When a user reports that they can't hear audio, the first thing we need to check is whether the incoming audio is silent. The application can obtain this information by checking `audioOutputLevel` in the media stats.

+2. If the `audioOutputLevel` value is constantly 0, it indicates that the incoming audio is silent. In this case, ask the user to verify if the speaking participant is muted or experiencing other issues, such as permission issues, device problems, or network issues.

+3. If the `audioOutputLevel` value isn't always 0, the user may still be unable to hear audio due to system volume settings. Ask the user to check their system volume settings.

+4. If the user's system volume is set to 0 or very low, the user should increase the volume in the settings.

+5. In some systems that support app-specific volume settings, the audio volume output from the app may be low even if system volume isn't low. In this case, the user should check their volume setting of the app within the system.

+6. If the volume setting of the app in the system is 0 or very low, the user should increase it.

+7. In certain cases, the audio element in the browser may fail to play or decode the audio, you can find an error message `AudioRenderer error with rendering audio code: 3` in the log.

+8. A common case for the AudioRenderer error is that the app uses the Web Audio API but doesn't release AudioContext objects properly. Browsers have a limit on the number of AudioContext instances that can be open simultaneously.

+9. If you still can't determine why the user can't hear sound during the call, ask the user to check their speaker device or select another audio output device. Note that not all platforms support speaker enumeration in the browser. For example, you can't select an audio output device through the JavaScript API in the Safari browser or in Chrome on Android. In these cases, you should configure the audio output device in the system settings.

+If a user consistently experiences this issue and you're unable to determine the cause, you may consider filing a support ticket for further assistance.

+### Check the duration of stream acquisition

+The stream acquisition is part of the call setup flow. You can get this information from webrtc-internals page.

+To access the page, open a new tab and enter edge://webrtc-internals (Edge) or chrome://webrtc-internals (Chrome).

+Once you're on the webrtc-internals page, you can calculate the duration of the stream acquisition by comparing the timestamp of the getUserMedia call and the result. If the duration is abnormally long, you may need to check the devices.

+### Check the duration of HTTP requests

+You can also check the Network tab of the Developer tools to see the size of requests and how long they take to finish.

+If the issue is due to the long duration of the signaling request, you should be able to see some requests taking very long time from the network trace.

+If you need to file a support ticket, we may request the browser HAR file.

+To learn how to collect a HAR file, see [Capture a browser trace for troubleshooting](../../../../../azure-portal/capture-browser-trace.md).

+ Title: References - How to collect call info

+description: Learn how to collect call info.

+# How to collect call info

+When you report an issue, providing important call information can help us quickly locate the problematic area and gain a deeper understanding of the issue.

+* ACS resource ID

+* call ID

+* participant ID

+## How to get ACS resource ID

+You can get this information from [https://portal.azure.com](https://portal.azure.com).

+## How to get call ID and participant ID

+The participant ID is important when there are multiple users in the same call.

+```typescript

+// call ID

+call.id

+// local participant ID

+call.info.participantId

+```

+The [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API doesn't resolve immediately, as there are multiple underlying asynchronous operations involved in the video subscription process and thus this API response is an asynchronous response.

+If your application disposes of the render object while the video subscription is in progress, the [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API throws an error.

+When there's a `create view timeout` issue, the [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API throws an error.

+The [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API detects no incoming video frames and considers this situation a subscription failure, therefore, it throws a createView timeout error.

+A feasible but less elegant approach for handling this createView timeout error is to continuously retry invoking the [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API until it succeeds.

+A workaround approach for handling this createView timeout error is to continuously retry invoking [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API until it succeeds when this network event is happening.

+If your application subscribes to the video stream from the server (for example, using [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API), the server forwards the sender's video packets to the receiver.

+If the number of active video subscriptions exceeds the limit, the [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API throws an error.

+If the video becomes unavailable before the [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API finishes, the [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API throws an error.

+If you subscribe to a video that is unavailable, the [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API throws an error.

+However, if there's asynchronous processing in the application layer, that might cause some delay before invoking [`createView`](/javascript/api/azure-communication-services/@azure/communication-calling/videostreamrenderer?view=azure-communication-services-js&preserve-view=true#@azure-communication-calling-videostreamrenderer-createview) API.

+1. To enable virtual network integration for your logic app, and add your logic app to the previously created subnet, follow these steps:

+ 1. On the logic app menu resource, under **Settings**, select **Networking**.

+ 1. In the **Outbound traffic configuration** section, next to **Virtual network integration**, select **Not configured** > **Add virtual network integration**.

+ 1. On the **Add virtual network integration** pane that opens, select your Azure subscription and your virtual network.

+ 1. From the **Subnet** list, select the subnet where you want to add your logic app.

+ 1. When you're done, select **Connect**, and return to the **Networking** page.

+ The **Virtual network integration** property is now set to the selected virtual network and subnet, for example:

+ :::image type="content" source="media/enable-stateful-affinity-built-in-connectors/enable-virtual-network-integration.png" alt-text="Screenshot shows Azure portal, Standard logic app resource, Networking page with selected virtual network and subnet.":::

+| Element | Value |

+||--|

+1. On the **Scale out** page, in the **App Scale out** section, follow these steps:

+ 1. Set **Enforce Scale Out Limit** to **Yes**, which shows the **Maximum Scale Out Limit**.

+ 1. Set **Always Ready Instances** to the same number as **Maximum Scale Out Limit** and **Maximum Burst**, which appears in the **Plan Scale out** section, for example:

+ :::image type="content" source="media/enable-stateful-affinity-built-in-connectors/scale-in-settings.png" alt-text="Screenshot shows Azure portal, Standard logic app resource, Scale out page, and Always Ready Instances number set to match Maximum Burst and Maximum Scale Out Limit.":::

+In .NET 8.0.4 and later, ASP.NET Core apps that use [data protection](/aspnet/core/security/data-protection/introduction) are automatically configured to keep protected data accessible to all replicas as the application scales. When your app begins to scale, a key manager handles the writing and sharing keys across multiple revisions. As the app is deployed, the environment variable `autoConfigureDataProtection` is automatically set `true` to enable this feature. For more information on this auto configuration, see [this GitHub pull request](https://github.com/Azure/azure-rest-api-specs/pull/28001).

+Artifact cache addresses the challenge of pull limits imposed by public registries. We recommend users authenticate their cache rules with their upstream source credentials. Then pull images from the local ACR, to help mitigate rate limits.

+>[!WARNING]

+> We recommend customers to [create a credential set](container-registry-artifact-cache.md#create-new-credentials) when sourcing content from Docker hub.

+>[!WARNING]

+> We recommend customers to [create a credential set](container-registry-artifact-cache.md#create-new-credentials) when sourcing content from Docker hub.

+ > [!IMPORTANT]

+ > Mirroring in Microsoft Fabric is now available in preview for NoSql API. This feature provides all the capabilities of Azure Synapse Link with better analytical performance, ability to unify your data estate with Fabric OneLake and open access to your data in OneLake with Delta Parquet format. If you are considering Azure Synapse Link, we recommend that you try mirroring to assess overall fit for your organization. To get started with mirroring, click [here](/fabric/database/mirrored-database/azure-cosmos-db?context=/azure/cosmos-db/context/context).

+To get started with Azure Synapse Link, please visit [ΓÇ£Getting started with Azure Synapse LinkΓÇ¥](synapse-link.md)

+Analytical workloads typically involve aggregations and sequential scans of selected fields. The data analytical store is stored in a column-major order, allowing values of each field to be serialized together, where applicable. This format reduces the IOPS required to scan or compute statistics over specific fields. It dramatically improves the query response times for scans over large data sets.

+The row store persists the above data in a serialized format, per row, on the disk. This format allows for faster transactional reads, writes, and operational queries, such as, "Return information about Product 1". However, as the dataset grows large and if you want to run complex analytical queries on the data it can be expensive. For example, if you want to get "the sales trends for a product under the category named 'Equipment' across different business units and months", you need to run a complex query. Large scans on this dataset can get expensive in terms of provisioned throughput and can also impact the performance of the transactional workloads powering your real-time applications and services.

+Azure Cosmos DB transactional store uses horizontal partitioning to elastically scale the storage and throughput without any downtime. Horizontal partitioning in the transactional store provides scalability & elasticity in auto-sync to ensure data is synced to the analytical store in near real time. The data sync happens regardless of the transactional traffic throughput, whether it's 1000 operations/sec or 1 million operations/sec, and it doesn't impact the provisioned throughput in the transactional store.

+You can use the following syntax example, with the same documents of the Spark example above:

+You can implement transformations using `cast`, `convert` or any other T-SQL function to manipulate your data. You can also hide complex datatype structures by using views.

+##### Working with MongoDB `_id` field

+MongoDB `_id` field is fundamental to every collection in MongoDB and originally has a hexadecimal representation. As you can see in the table above, full fidelity schema will preserve its characteristics, creating a challenge for its visualization in Azure Synapse Analytics. For correct visualization, you must convert the `_id` datatype as below:

+###### Working with MongoDB `_id` field in Spark

+###### Working with MongoDB `_id` field in SQL

+* Full Fidelity schema data types map mentioned above isn't valid for NoSQL API accounts that use JSON datatypes. As an example, `float` and `integer` values are represented as `num` in analytical store.

+> When `analytical TTL` is set to a value larger than `transactional TTL` value, your container will have data that only exists in analytical store. This data is read only and currently we don't support document level `TTL` in analytical store. If your container data may need an update or a delete at some point in time in the future, don't use `analytical TTL` bigger than `transactional TTL`. This capability is recommended for data that won't need updates or deletes in the future.

+ * If any geo-region of the database account is configured for zone-redundancy, it is allocated in Zone-redundant Storage (ZRS) accounts. You need to enable Availability Zones on a region of their Azure Cosmos DB database account to have analytical data of that region stored in Zone-redundant Storage. ZRS offers durability for storage resources of at least 99.9999999999% (12 9's) over a given year.

+For more information about Azure Storage durability, see [this link.](/azure/storage/common/storage-redundancy)

+* Continuous backup mode for Synapse Link enabled accounts is in public preview. Currently, you can't migrate to continuous backup if you disabled Synapse Link on any of your collections in a Cosmos DB account.

+Data in Azure Cosmos DB analytical store can be queried simultaneously from the different analytics runtimes supported by Azure Synapse Analytics. Azure Synapse Analytics supports Apache Spark and serverless SQL pool with Azure Cosmos DB analytical store.

+ Title: Analytics and BI

+description: Review Azure Cosmos DB options to enable large-scale analytics and BI reporting on your operational data.

+# Analytics and Business Intelligence (BI) on your Azure Cosmos DB data

+Azure Cosmos DB offers various options to enable large-scale analytics and BI reporting on your operational data.

+To get meaningful insights on your Azure Cosmos DB data, you may need to query across multiple partitions, collections, or databases. In some cases, you might combine this data with other data sources in your organization such as Azure SQL Database, Azure Data Lake Storage Gen2 etc. You might also query with aggregate functions such as sum, count etc. Such queries need heavy computational power, which likely consumes more request units (RUs) and as a result, these queries might potentially affect your mission critical workload performance.

+To isolate transactional workloads from the performance impact of complex analytical queries, database data is ingested nightly to a central location using complex Extract-Transform-Load (ETL) pipelines. Such ETL-based analytics are complex, costly with delayed insights on business data.

+Azure Cosmos DB addresses these challenges by providing no-ETL, cost-effective analytics offerings.

+## No-ETL, near real-time analytics on Azure Cosmos DB

+Azure Cosmos DB offers no-ETL, near real-time analytics on your data without affecting the performance of your transactional workloads or request units (RUs). These offerings remove the need for complex ETL pipelines, making your Azure Cosmos DB data seamlessly available to analytics engines. With reduced latency to insights, you can provide enhanced customer experience and react more quickly to changes in market conditions or business environment. Here are some sample [scenarios](synapse-link-use-cases.md) you can achieve with quick insights into your data.

+

+ You can enable no-ETL analytics and BI reporting on Azure Cosmos DB using the following options:

+* Mirroring your data into Microsoft Fabric

+* Enabling Azure Synapse Link to access data from Azure Synapse Analytics

+

+### Option 1: Mirroring your Azure Cosmos DB data into Microsoft Fabric

+Mirroring enables you to seamlessly bring your Azure Cosmos DB database data into Microsoft Fabric. With no-ETL, you can get rich business insights on your Azure Cosmos DB data using FabricΓÇÖs built-in analytics, BI, and AI capabilities.

+Your Cosmos DB operational data is incrementally replicated into Fabric OneLake in near real-time. Data in OneLake is stored in open-source Delta Parquet format and made available to all analytical engines in Fabric. With open access, you can use it with various Azure services such as Azure Databricks, Azure HDInsight, and more. OneLake also helps unify your data estate for your analytical needs. Mirrored data can be joined with any other data in OneLake, such as Lakehouses, Warehouses or shortcuts. You can also join Azure Cosmos DB data with other mirrored database sources such as Azure SQL Database, Snowflake.

+You can query across Azure Cosmos DB collections or databases mirrored into OneLake.

+With Mirroring in Fabric, you don't need to piece together different services from multiple vendors. Instead, you can enjoy a highly integrated, end-to-end, and easy-to-use product that is designed to simplify your analytics needs.

+You can use T-SQL to run complex aggregate queries and Spark for data exploration. You can seamlessly access the data in notebooks, use data science to build machine learning models, and build Power BI reports using Direct Lake powered by rich Copilot integration.

+If you're looking for analytics on your operational data in Azure Cosmos DB, mirroring provides:

+* No-ETL, cost-effective near real-time analytics on Azure Cosmos DB data without affecting your request unit (RU) consumption

+* Ease of bringing data across various sources into Fabric OneLake.

+* Improved query performance of SQL engine handling delta tables, with V-order optimizations

+* Improved cold start time for Spark engine with deep integration with ML/notebooks

+* One-click integration with Power BI with Direct Lake and Copilot

+* Richer app integration to access queries and views with GraphQL

+* Open access to and from other services such as Azure Databricks

+To get started with mirroring, visit ["Get started with mirroring tutorial"](/fabric/database/mirrored-database/azure-cosmos-db-tutorial?context=/azure/cosmos-db/context/context).

+### Option 2: Azure Synapse Link to access data from Azure Synapse Analytics

+Azure Synapse Link for Azure Cosmos DB creates a tight seamless integration between Azure Cosmos DB and Azure Synapse Analytics, enabling no-ETL, near real-time analytics on your operational data.

+Transactional data is seamlessly synced to Analytical store, which stores the data in columnar format optimized for analytics.

+Azure Synapse Analytics can access this data in Analytical store, without further movement, using Azure Synapse Link. Business analysts, data engineers, and data scientists can now use Synapse Spark or Synapse SQL interchangeably to run near real time business intelligence, analytics, and machine learning pipelines.

+The following image shows the Azure Synapse Link integration with Azure Cosmos DB and Azure Synapse Analytics:

+ > [!IMPORTANT]

+ > Mirroring in Microsoft Fabric is now available in preview for NoSql API. This feature provides all the capabilities of Azure Synapse Link with better analytical performance, ability to unify your data estate with Fabric OneLake and open access to your data in OneLake with Delta Parquet format. If you are considering Azure Synapse Link, we recommend that you try mirroring to assess overall fit for your organization. To get started with mirroring, click [here](/fabric/database/mirrored-database/azure-cosmos-db?context=/azure/cosmos-db/context/context).

+To get started with Azure Synapse Link, visit ["Getting started with Azure Synapse Link"](synapse-link.md).

+## Real-time analytics and BI on Azure Cosmos DB: Other options

+There are a few other options to enable real-time analytics on Azure Cosmos DB data:

+* Using [change feed](nosql/changefeed-ecommerce-solution.md)

+* Using [Spark connector directly on Azure Cosmos DB](nosql/tutorial-spark-connector.md)

+* Using Power BI connector directly on Azure Cosmos DB

+While these options are included for completeness and work well with single partition queries in real-time, these methods have the following challenges for analytical queries:

+* Performance impact on your workload:

+ Analytical queries tend to be complex and consume significant compute capacity. When these queries are run against your Azure Cosmos DB data directly, you might experience performance degradation on your transactional queries.

+* Cost impact:

+

+ When analytical queries are run directly against your database or collections, they increase the need for request units allocated, as analytical queries tend to be complex and need more computation power. Increased RU usage will likely lead to significant cost impact over time, if you run aggregate queries.

+Instead of these options, we recommend that you use Mirroring in Microsoft Fabric or Azure Synapse Link, which provide no-ETL analytics, without affecting transactional workload performance or request units.

+## Related content

+* [Mirroring Azure Cosmos DB overview](/fabric/database/mirrored-database/azure-cosmos-db?context=/azure/cosmos-db/context/context)

+* [Getting started with mirroring](/fabric/database/mirrored-database/azure-cosmos-db-tutorial?context=/azure/cosmos-db/context/context)

+* [Azure Synapse Link for Azure Cosmos DB](synapse-link.md)

+* [Working with Azure Synapse Link for Azure Cosmos DB](configure-synapse-link.md)

+ > [!IMPORTANT]

+ > Mirroring in Microsoft Fabric is now available in preview for NoSql API. This feature provides all the capabilities of Azure Synapse Link with better analytical performance, ability to unify your data estate with Fabric OneLake and open access to your data in OneLake with Delta Parquet format. If you are considering Azure Synapse Link, we recommend that you try mirroring to assess overall fit for your organization. To get started with mirroring, click [here](/fabric/database/mirrored-database/azure-cosmos-db?context=/azure/cosmos-db/context/context).

+ Title: Near real-time analytics use cases for Azure Cosmos DB

+description: Learn how real-time analytics is used in Supply chain analytics, forecasting, reporting, real-time personalization, and IOT predictive maintenance.

+# Azure Cosmos DB: No-ETL analytics use cases

+Azure Cosmos DB provides various analytics options for no-ETL, near real-time analytics over operational data. You can enable analytics on your Azure Cosmos DB data using following options:

+* Mirroring Azure Cosmos DB in Microsoft Fabric

+* Azure Synapse Link for Azure Cosmos DB

+To learn more about these options, see ["Analytics and BI on your Azure Cosmos DB data."](analytics-and-business-intelligence-overview.md)

+> [!IMPORTANT]

+> Mirroring Azure Cosmos DB in Microsoft Fabric is now available in preview for NoSql API. This feature provides all the capabilities of Azure Synapse Link with better analytical performance, ability to unify your data estate with Fabric OneLake and open access to your data in OneLake with Delta Parquet format. If you are considering Azure Synapse Link, we recommend that you try mirroring to assess overall fit for your organization. To get started with mirroring, click [here](/fabric/database/mirrored-database/azure-cosmos-db?context=/azure/cosmos-db/context/context).

+No-ETL, near real-time analytics can open up various possibilities for your businesses. Here are three sample scenarios:

+The following architecture shows the power of using Azure Cosmos DB as the cloud-native operational database in supply chain analytics:

+Based on previous architecture, you can achieve the following use cases:

+ Mirroring and Synapse Link allow you to analyze the changing operational data in Azure Cosmos DB without any manual ETL processes. These offerings save you from additional cost, latency, and operational complexity. They enable data engineers and data scientists to build robust predictive pipelines:

+ * Query operational data from Azure Cosmos DB by using native integration with Apache Spark pools in Microsoft Fabric or Azure Synapse Analytics. You can query the data in an interactive notebook or scheduled remote jobs without complex data engineering.

+ * Build Machine Learning (ML) models with Spark ML algorithms and Azure Machine Learning (AML) integration in Microsoft Fabric or Azure Synapse Analytics.

+ Mirroring and Synapse Link for Azure Cosmos DB enable rich business intelligence (BI)/reporting scenarios:

+ * Query operational data from Azure Cosmos DB by using native integration with full expressiveness of T-SQL language.

+ * Model and publish auto refreshing BI dashboards over Azure Cosmos DB through Power BI integrated in Microsoft Fabric or Azure Synapse Analytics.

+* **Batch data integration & orchestration:** With supply chains getting more complex, supply chain data platforms need to integrate with variety of data sources and formats. Microsoft Fabric and Azure Synapse come built-in with the same data integration engine and experiences as Azure Data Factory. This integration allows data engineers to create rich data pipelines without a separate orchestration engine:

+ * Move data from 85+ supported data sources to [Azure Cosmos DB with Azure Data Factory](../data-factory/connector-azure-cosmos-db.md).

+* **Prepare & train predictive pipeline:** You can generate insights over the operational data across your business units or customer segments using Fabric or Synapse Spark and machine learning models. This translates to personalized delivery to target customer segments, predictive end-user experiences, and targeted marketing to fit your end-user requirements.

+)

+The following is an architecture using the cloud native HTAP capabilities in IoT predictive maintenance:

+* **Operational reporting:** With the growth of digital twin initiatives, companies are collecting vast amounts of operational data from large number of sensors to build a digital copy of each machine. This data powers BI needs to understand trends over historical data in addition to recent hot data.

+## Related content

+* [Mirroring Azure Cosmos DB overview](/fabric/database/mirrored-database/azure-cosmos-db?context=/azure/cosmos-db/context/context)

+* [Getting started with mirroring](/fabric/database/mirrored-database/azure-cosmos-db-tutorial?context=/azure/cosmos-db/context/context)

+

+> [!IMPORTANT]

+> Mirroring Azure Cosmos DB in Microsoft Fabric is now available in preview for NoSql API. This feature provides all the capabilities of Azure Synapse Link with better analytical performance, ability to unify your data estate with Fabric OneLake and open access to your data in OneLake with Delta Parquet format. If you are considering Azure Synapse Link, we recommend that you try mirroring to assess overall fit for your organization. To get started with mirroring, click [here](/fabric/database/mirrored-database/azure-cosmos-db?context=/azure/cosmos-db/context/context).

+description: Learn how to create a trigger in Azure Data Factory that runs a pipeline in response to a custom event published to Azure Event Grid.

+Event-driven architecture is a common data integration pattern that involves production, detection, consumption, and reaction to events. Data integration scenarios often require Azure Data Factory customers to trigger pipelines when certain events occur. Data Factory native integration with [Azure Event Grid](https://azure.microsoft.com/services/event-grid/) now covers [custom topics](../event-grid/custom-topics.md). You send events to an Event Grid topic. Data Factory subscribes to the topic, listens, and then triggers pipelines accordingly.

+The integration described in this article depends on [Azure Event Grid](https://azure.microsoft.com/services/event-grid/). Make sure that your subscription is registered with the Event Grid resource provider. For more information, see [Resource providers and types](../azure-resource-manager/management/resource-providers-and-types.md#azure-portal). You must be able to do the `Microsoft.EventGrid/eventSubscriptions/` action. This action is part of the [EventGrid EventSubscription Contributor](../role-based-access-control/built-in-roles.md#eventgrid-eventsubscription-contributor) built-in role.

+> If you're using this feature in Azure Synapse Analytics, ensure that your subscription is also registered with a Data Factory resource provider. Otherwise, you get a message stating that "the creation of an Event Subscription failed."

+If you combine pipeline parameters and a custom event trigger, you can parse and reference custom `data` payloads in pipeline runs. Because the `data` field in a custom event payload is a freeform, JSON key-value structure, you can control event-driven pipeline runs.

+> If a key referenced in parameterization is missing in the custom event payload, `trigger run` fails. You get a message that states the expression can't be evaluated because the `keyName` property doesn't exist. In this case, **no** `pipeline run` is triggered by the event.

+Go to Event Grid and create the topic yourself. For more information on how to create the custom topic, see Event Grid [portal tutorials](../event-grid/custom-topics.md#azure-portal-tutorials) and [Azure CLI tutorials](../event-grid/custom-topics.md#azure-cli-tutorials).

+> The workflow is different from a storage event trigger. Here, Data Factory doesn't set up the topic for you.

+Data Factory expects events to follow the [Event Grid event schema](../event-grid/event-schema.md). Make sure that event payloads have the following fields:

+1. Go to Data Factory and sign in.

+1. On the **Add Triggers** page, select **Choose trigger**, and then select **+ New**.

+1. Under **Type**, select **Custom events**.

+ :::image type="content" source="media/how-to-create-custom-event-trigger/custom-event-1-creation.png" alt-text="Screenshot that shows creating a new custom event trigger in the Data Factory UI." lightbox="media/how-to-create-custom-event-trigger/custom-event-1-creation-expanded.png":::

+1. Select your custom topic from the Azure subscription dropdown list or manually enter the event topic scope.

+ > To create or modify a custom event trigger in Data Factory, you need to use an Azure account with appropriate Azure role-based access control (Azure RBAC). No other permission is required. The Data Factory service principal does *not* require special permission to your Event Grid. For more information about access control, see the [Role-based access control](#role-based-access-control) section.

+1. The `Subject begins with` and `Subject ends with` properties allow you to filter for trigger events. Both properties are optional.

+1. Use **+ New** to add **Event types** to filter on. The list of custom event triggers uses an OR relationship. When a custom event with an `eventType` property matches one on the list, a pipeline run is triggered. The event type is case insensitive. For example, in the following screenshot, the trigger matches all `copycompleted` or `copysucceeded` events that have a subject that begins with *factories*.

+ :::image type="content" source="media/how-to-create-custom-event-trigger/custom-event-2-properties.png" alt-text="Screenshot that shows the Edit trigger page to explain Event types and Subject filtering in the Data Factory UI.":::

+1. A custom event trigger can parse and send a custom `data` payload to your pipeline. You create the pipeline parameters and then fill in the values on the **Parameters** page. Use the format `@triggerBody().event.data._keyName_` to parse the data payload and pass values to the pipeline parameters.

+ For a detailed explanation, see:

+ :::image type="content" source="media/how-to-create-custom-event-trigger/custom-event-4-trigger-values.png" alt-text="Screenshot that shows pipeline parameters settings.":::

+ :::image type="content" source="media/how-to-create-custom-event-trigger/custom-event-3-parameters.png" alt-text="Screenshot that shows the parameters page to reference data payload in a custom event.":::

+1. After you enter the parameters, select **OK**.

+Custom event triggers support advanced filtering capabilities, similar to [Event Grid advanced filtering](../event-grid/event-filtering.md#advanced-filtering). These conditional filters allow pipelines to trigger based on the _values_ of the event payload. For instance, you might have a field in the event payload named _Department_, and the pipeline should only trigger if _Department_ equals _Finance_. You might also specify complex logic, such as the _date_ field in list [1, 2, 3, 4, 5], the _month_ field *not* in the list [11, 12], and if the _tag_ field contains [Fiscal Year 2021, FiscalYear2021, or FY2021].

+ :::image type="content" source="media/how-to-create-custom-event-trigger/custom-event-5-advanced-filters.png" alt-text="Screenshot that shows setting advanced filters for a customer event trigger.":::

+As of today, custom event triggers support a *subset* of [advanced filtering operators](../event-grid/event-filtering.md#advanced-filtering) in Event Grid. The following filter conditions are supported:

+* `NumberIn`

+* `NumberNotIn`

+* `NumberLessThan`

+* `NumberGreaterThan`

+* `NumberLessThanOrEquals`

+* `NumberGreaterThanOrEquals`

+* `BoolEquals`

+* `StringContains`

+* `StringBeginsWith`

+* `StringEndsWith`

+* `StringIn`

+* `StringNotIn`

+Select **+ New** to add new filter conditions.

+Custom event triggers also obey the [same limitations as Event Grid](../event-grid/event-filtering.md#limitations), such as:

+* 5 advanced filters and 25 filter values across all the filters per custom event trigger.

+* 512 characters per string value.

+* 5 values for `in` and `not in` operators.

+* Keys can't have the `.` (dot) character in them, for example, `john.doe@contoso.com`. Currently, there's no support for escape characters in keys.

+Data Factory relies on the latest general availability (GA) version of the [Event Grid API](../event-grid/whats-new.md). As new API versions get to the GA stage, Data Factory expands its support for more advanced filtering operators.

+The following table provides an overview of the schema elements that are related to custom event triggers.

+| `scope` | The Azure Resource Manager resource ID of the Event Grid topic. | String | Azure Resource Manager ID | Yes. |

+| `subjectBeginsWith` | The `subject` field must begin with the provided pattern for the trigger to fire. For example, *factories* only fire the trigger for event subjects that start with *factories*. | String | | No. |

+| `subjectEndsWith` | The `subject` field must end with the provided pattern for the trigger to fire. | String | | No. |

+| `advancedFilters` | List of JSON blobs, each specifying a filter condition. Each blob specifies `key`, `operatorType`, and `values`. | List of JSON blobs | | No. |

+Data Factory uses Azure RBAC to prohibit unauthorized access. To function properly, Data Factory requires access to:

+To successfully create or update a custom event trigger, you need to sign in to Data Factory with an Azure account that has appropriate access. Otherwise, the operation fails with the message "Access Denied."

+Data Factory doesn't require special permission to your instance of Event Grid. You also do *not* need to assign special Azure RBAC role permission to the Data Factory service principal for the operation.

+- When you author in the data factory (in the development environment, for instance), the Azure account signed in needs to have the preceding permission.

+- When you publish through [continuous integration and continuous delivery](continuous-integration-delivery.md), the account used to publish the Azure Resource Manager template into the testing or production factory needs to have the preceding permission.

+description: Learn how to create a trigger in Azure Data Factory or Azure Synapse Analytics that runs a pipeline in response to an event.

+This article describes the storage event triggers that you can create in your Azure Data Factory or Azure Synapse Analytics pipelines.

+Event-driven architecture is a common data integration pattern that involves production, detection, consumption, and reaction to events. Data integration scenarios often require customers to trigger pipelines that are triggered from events on an Azure Storage account, such as the arrival or deletion of a file in Azure Blob Storage account. Data Factory and Azure Synapse Analytics pipelines natively integrate with [Azure Event Grid](https://azure.microsoft.com/services/event-grid/), which lets you trigger pipelines on such events.

+Consider the following points when you use storage event triggers:

+- The integration described in this article depends on [Azure Event Grid](https://azure.microsoft.com/services/event-grid/). Make sure that your subscription is registered with the Event Grid resource provider. For more information, see [Resource providers and types](../azure-resource-manager/management/resource-providers-and-types.md#azure-portal). You must be able to do the `Microsoft.EventGrid/eventSubscriptions/` action. This action is part of the `EventGrid EventSubscription Contributor` built-in role.

+- If you're using this feature in Azure Synapse Analytics, ensure that you also register your subscription with the Data Factory resource provider. Otherwise, you get a message stating that "the creation of an Event Subscription failed."

+- If the Blob Storage account resides behind a [private endpoint](../storage/common/storage-private-endpoints.md) and blocks public network access, you need to configure network rules to allow communications from Blob Storage to Event Grid. You can either grant storage access to trusted Azure services, such as Event Grid, following [Storage documentation](../storage/common/storage-network-security.md#grant-access-to-trusted-azure-services), or configure private endpoints for Event Grid that map to a virtual network address space, following [Event Grid documentation](../event-grid/configure-private-endpoints.md).

+- The storage event trigger currently supports only Azure Data Lake Storage Gen2 and General-purpose version 2 storage accounts. If you're working with Secure File Transfer Protocol (SFTP) storage events, you need to specify the SFTP Data API under the filtering section too. Because of an Event Grid limitation, Data Factory only supports a maximum of 500 storage event triggers per storage account.

+- To create a new storage event trigger or modify an existing one, the Azure account you use to sign in to the service and publish the storage event trigger must have appropriate role-based access control (Azure RBAC) permission on the storage account. No other permissions are required. Service principal for the Azure Data Factory and Azure Synapse Analytics does _not_ need special permission to either the storage account or Event Grid. For more information about access control, see the [Role-based access control](#role-based-access-control) section.

+- If you applied an Azure Resource Manager lock to your storage account, it might affect the blob trigger's ability to create or delete blobs. A `ReadOnly` lock prevents both creation and deletion, while a `DoNotDelete` lock prevents deletion. Ensure that you account for these restrictions to avoid any issues with your triggers.

+- We don't recommend file arrival triggers as a triggering mechanism from data flow sinks. Data flows perform a number of file renaming and partition file shuffling tasks in the target folder that can inadvertently trigger a file arrival event before the complete processing of your data.

+## Create a trigger with the UI

+This section shows you how to create a storage event trigger within the Azure Data Factory and Azure Synapse Analytics pipeline user interface (UI).

+1. Switch to the **Edit** tab in Data Factory or the **Integrate** tab in Azure Synapse Analytics.

+1. On the menu, select **Trigger**, and then select **New/Edit**.

+1. On the **Add Triggers** page, select **Choose trigger**, and then select **+ New**.

+1. Select the trigger type **Storage events**.

+ :::image type="content" source="media/how-to-create-event-trigger/event-based-trigger-image-1.png" lightbox="media/how-to-create-event-trigger/event-based-trigger-image-1.png" alt-text="Screenshot that shows creating a new storage event trigger in the Data Factory UI." :::

+ # [Azure Synapse Analytics](#tab/synapse-analytics)

+ :::image type="content" source="media/how-to-create-event-trigger/event-based-trigger-image-1-synapse.png" lightbox="media/how-to-create-event-trigger/event-based-trigger-image-1-synapse.png" alt-text="Screenshot that shows creating a new storage event trigger in the Azure Synapse Analytics UI.":::

+1. Select your storage account from the Azure subscription dropdown list or manually by using its storage account resource ID. Choose the container on which you want the events to occur. Container selection is required, but selecting all containers can lead to a large number of events.

+1. The `Blob path begins with` and `Blob path begins with` properties allow you to specify the containers, folders, and blob names for which you want to receive events. Your storage event trigger requires at least one of these properties to be defined. You can use various patterns for both `Blob path begins with` and `Blob path begins with` properties, as shown in the examples later in this article.

+ * `Blob path begins with`: The blob path must start with a folder path. Valid values include `2018/` and `2018/april/shoes.csv`. This field can't be selected if a container isn't selected.

+ * `Blob path begins with`: The blob path must end with a file name or extension. Valid values include `shoes.csv` and `.csv`. Container and folder names, when specified, must be separated by a `/blobs/` segment. For example, a container named `orders` can have a value of `/orders/blobs/2018/april/shoes.csv`. To specify a folder in any container, omit the leading `/` character. For example, `april/shoes.csv` triggers an event on any file named `shoes.csv` in a folder called `april` in any container.

+

+ Note that `Blob path begins with` and `Blob path ends with` are the only pattern matching allowed in a storage event trigger. Other types of wildcard matching aren't supported for the trigger type.

+1. Select whether your trigger responds to a **Blob created** event, a **Blob deleted** event, or both. In your specified storage location, each event triggers the Data Factory and Azure Synapse Analytics pipelines associated with the trigger.

+ :::image type="content" source="media/how-to-create-event-trigger/event-based-trigger-image-2.png" alt-text="Screenshot that shows a storage event trigger creation page.":::

+1. After you configure your trigger, select **Next: Data preview**. This screen shows the existing blobs matched by your storage event trigger configuration. Make sure you have specific filters. Configuring filters that are too broad can match a large number of files that were created or deleted and might significantly affect your cost. After your filter conditions are verified, select **Finish**.

+ :::image type="content" source="media/how-to-create-event-trigger/event-based-trigger-image-3.png" alt-text="Screenshot that shows the storage event trigger preview page.":::

+1. To attach a pipeline to this trigger, go to the pipeline canvas and select **Trigger** > **New/Edit**. When the side pane appears, select the **Choose trigger** dropdown list and select the trigger you created. Select **Next: Data preview** to confirm that the configuration is correct. Then select **Next** to validate that the data preview is correct.

+1. If your pipeline has parameters, you can specify them on the **Trigger Run Parameters** side pane. The storage event trigger captures the folder path and file name of the blob into the properties `@triggerBody().folderPath` and `@triggerBody().fileName`. To use the values of these properties in a pipeline, you must map the properties to pipeline parameters. After you map the properties to parameters, you can access the values captured by the trigger through the `@pipeline().parameters.parameterName` expression throughout the pipeline. For a detailed explanation, see [Reference trigger metadata in pipelines](how-to-use-trigger-parameterization.md).

+ :::image type="content" source="media/how-to-create-event-trigger/event-based-trigger-image-4.png" alt-text="Screenshot that shows storage event trigger mapping properties to pipeline parameters.":::

+ In the preceding example, the trigger is configured to fire when a blob path ending in .csv is created in the folder _event-testing_ in the container _sample-data_. The `folderPath` and `fileName` properties capture the location of the new blob. For example, when MoviesDB.csv is added to the path _sample-data/event-testing_, `@triggerBody().folderPath` has a value of `sample-data/event-testing` and `@triggerBody().fileName` has a value of `moviesDB.csv`. These values are mapped, in the example, to the pipeline parameters `sourceFolder` and `sourceFile`, which can be used throughout the pipeline as `@pipeline().parameters.sourceFolder` and `@pipeline().parameters.sourceFile`, respectively.

+1. After you're finished, select **Finish**.

+The following table provides an overview of the schema elements that are related to storage event triggers.

+| JSON element | Description | Type | Allowed values | Required |

+| scope | The Azure Resource Manager resource ID of the storage account. | String | Azure Resource Manager ID | Yes. |

+| events | The type of events that cause this trigger to fire. | Array | `Microsoft.Storage.BlobCreated`, `Microsoft.Storage.BlobDeleted` | Yes, any combination of these values. |

+| `blobPathBeginsWith` | The blob path must begin with the pattern provided for the trigger to fire. For example, `/records/blobs/december/` only fires the trigger for blobs in the `december` folder under the `records` container. | String | | Provide a value for at least one of these properties: `blobPathBeginsWith` or `blobPathEndsWith`. |

+| `blobPathEndsWith` | The blob path must end with the pattern provided for the trigger to fire. For example, `december/boxes.csv` only fires the trigger for blobs named `boxes` in a `december` folder. | String | | Provide a value for at least one of these properties: `blobPathBeginsWith` or `blobPathEndsWith`. |

+| `ignoreEmptyBlobs` | Whether or not zero-byte blobs triggers a pipeline run. By default, this is set to `true`. | Boolean | true or false | No. |

+> You have to include the `/blobs/` segment of the path, as shown in the following examples, whenever you specify container and folder, container and file, or container, folder, and file. For `blobPathBeginsWith`, the UI automatically adds `/blobs/` between the folder and container name in the trigger JSON.

+| `Blob path begins with` | `/containername/` | Receives events for any blob in the container. |

+| `Blob path begins with` | `/containername/blobs/foldername/` | Receives events for any blobs in the `containername` container and `foldername` folder. |

+| `Blob path begins with` | `/containername/blobs/foldername/subfoldername/` | You can also reference a subfolder. |

+| `Blob path begins with` | `/containername/blobs/foldername/file.txt` | Receives events for a blob named `file.txt` in the `foldername` folder under the `containername` container. |

+| `Blob path ends with` | `file.txt` | Receives events for a blob named `file.txt` in any path. |

+| `Blob path ends with` | `/containername/blobs/file.txt` | Receives events for a blob named `file.txt` under the container `containername`. |

+| `Blob path ends with` | `foldername/file.txt` | Receives events for a blob named `file.txt` in the `foldername` folder under any container. |

+Data Factory and Azure Synapse Analytics pipelines use Azure role-based access control (Azure RBAC) to ensure that unauthorized access to listen to, subscribe to updates from, and trigger pipelines linked to blob events are strictly prohibited.

+* To successfully create a new storage event trigger or update an existing one, the Azure account signed in to the service needs to have appropriate access to the relevant storage account. Otherwise, the operation fails with the message "Access Denied."

+* Data Factory and Azure Synapse Analytics need no special permission to your Event Grid instance, and you do *not* need to assign special RBAC permission to the Data Factory or Azure Synapse Analytics service principal for the operation.

+Any of the following RBAC settings work for storage event triggers:

+* `Microsoft.EventGrid/EventSubscriptions/Write` permission to the storage account `/subscriptions/####/resourceGroups/####/providers/Microsoft.Storage/storageAccounts/storageAccountName`

+Specifically:

+- When you author in the data factory (in the development environment for instance), the Azure account signed in needs to have the preceding permission.

+- When you publish through [continuous integration and continuous delivery](continuous-integration-delivery.md), the account used to publish the Azure Resource Manager template into the testing or production factory needs to have the preceding permission.

+To understand how the service delivers the two promises, let's take a step back and peek behind the scenes. Here are the high-level workflows for integration between Data Factory/Azure Synapse Analytics, Storage, and Event Grid.

+### Create a new storage event trigger

+This high-level workflow describes how Data Factory interacts with Event Grid to create a storage event trigger. The data flow is the same in Azure Synapse Analytics, with Azure Synapse Analytics pipelines taking the role of the data factory in the following diagram.

+Two noticeable callouts from the workflows:

+* Data Factory and Azure Synapse Analytics make _no_ direct contact with the storage account. The request to create a subscription is instead relayed and processed by Event Grid. The service needs no permission to access the storage account for this step.

+* Access control and permission checking happen within the service. Before the service sends a request to subscribe to a storage event, it checks the permission for the user. More specifically, it checks whether the Azure account that's signed in and attempting to create the storage event trigger has appropriate access to the relevant storage account. If the permission check fails, trigger creation also fails.

+This high-level workflow describes how storage event trigger pipelines run through Event Grid. For Azure Synapse Analytics, the data flow is the same, with Azure Synapse Analytics pipelines taking the role of Data Factory in the following diagram.

+Three noticeable callouts in the workflow are related to event triggering pipelines within the service:

+* Event Grid uses a Push model that relays the message as soon as possible when storage drops the message into the system. This approach is different from a messaging system, such as Kafka, where a Pull system is used.

+* The event trigger serves as an active listener to the incoming message and it properly triggers the associated pipeline.

+* The storage event trigger itself makes no direct contact with the storage account.

+ * If you have a Copy activity or another activity inside the pipeline to process the data in the storage account, the service makes direct contact with the storage account by using the credentials stored in the linked service. Ensure that the linked service is set up appropriately.

+ * If you make no reference to the storage account in the pipeline, you don't need to grant permission to the service to access the storage account.

+* For more information about triggers, see [Pipeline execution and triggers](concepts-pipeline-execution-triggers.md#trigger-execution-with-json).

+* To reference trigger metadata in a pipeline, see [Reference trigger metadata in pipeline runs](how-to-use-trigger-parameterization.md).

+Tumbling window triggers are a type of trigger that fires at a periodic time interval from a specified start time, while retaining state. Tumbling windows are a series of fixed-sized, nonoverlapping, and contiguous time intervals. A tumbling window trigger has a one-to-one relationship with a pipeline and can only reference a singular pipeline.

+A tumbling window trigger is a more heavyweight alternative for a schedule trigger. It offers a suite of features for complex scenarios like ([dependency on other tumbling window triggers](#tumbling-window-trigger-dependency), [rerunning a failed job](tumbling-window-trigger-dependency.md#monitor-dependencies), and [setting user retry for pipelines](#user-assigned-retries-of-pipelines)). To further understand the difference between a schedule trigger and a tumbling window trigger, see [Trigger type comparison](concepts-pipeline-execution-triggers.md#trigger-type-comparison).

+## Azure Data Factory and Azure Synapse portal experience

+1. To create a tumbling window trigger in the Azure portal, select the **Triggers** tab, and then select **New**.

+1. After the trigger configuration pane opens, select **Tumbling window**. Then define your tumbling window trigger properties.

+1. When you're finished, select **Save**.

+The following table provides a high-level overview of the major JSON elements that are related to recurrence and scheduling of a tumbling window trigger.

+| `type` | The type of the trigger. The `type` is the fixed value `TumblingWindowTrigger`. | `String` | `TumblingWindowTrigger` | Yes |

+| `runtimeState` | The current state of the trigger run time.<br/>This element is \<readOnly>. | `String` | `Started`, `Stopped`, `Disabled` | Yes |

+| `frequency` | A string that represents the frequency unit (minutes, hours, or months) at which the trigger recurs. If the `startTime` date values are more granular than the `frequency` value, the `startTime` dates are considered when the window boundaries are computed. For example, if the `frequency` value is `hourly` and the `startTime` value is 2017-09-01T10:10:10Z, the first window is (2017-09-01T10:10:10Z, 2017-09-01T11:10:10Z). | `String` | `Minute`, `Hour`, `Month` | Yes |

+| `interval` | A positive integer that denotes the interval for the `frequency` value, which determines how often the trigger runs. For example, if the `interval` is `3` and the `frequency` is `hour`, the trigger recurs every 3 hours. <br/>The minimum window interval is 5 minutes. | `Integer` | A positive integer. | Yes |

+| `startTime`| The first occurrence, which can be in the past. The first trigger interval is (`startTime`, `startTime + interval`). | `DateTime` | A `DateTime` value. | Yes |

+| `endTime`| The last occurrence, which can be in the past. | `DateTime` | A `DateTime` value. | Yes |

+| `delay` | The amount of time to delay the start of data processing for the window. The pipeline run is started after the expected execution time plus the amount of delay. The delay defines how long the trigger waits past the due time before triggering a new run. The delay doesn't alter the window `startTime`. For example, a `delay` value of 00:10:00 implies a delay of 10 minutes. | `Timespan`<br/>(hh:mm:ss) | A `timespan` value where the default is `00:00:00`. | No |

+| `maxConcurrency` | The number of simultaneous trigger runs that are fired for windows that are ready. For example, to backfill hourly runs for yesterday results in 24 windows. If `maxConcurrency` = 10, trigger events are fired only for the first 10 windows (00:00-01:00 - 09:00-10:00). After the first 10 triggered pipeline runs are complete, trigger runs are fired for the next 10 windows (10:00-11:00 - 19:00-20:00). Continuing with this example of `maxConcurrency` = 10, if there are 10 windows ready, there are 10 total pipeline runs. If only one window is ready, only one pipeline runs. | `Integer` | An integer between 1 and 50. | Yes |

+| `retryPolicy: Count` | The number of retries before the pipeline run is marked as `Failed`. | `Integer` | An integer, where the default is 0 (no retries). | No |

+| `retryPolicy: intervalInSeconds` | The delay between retry attempts specified in seconds. | `Integer` | The number of seconds, where the default is 30. The minimum value is `30`. | No |

+| `dependsOn: type` | The type of `TumblingWindowTriggerReference`. Required if a dependency is set. | `String` | `TumblingWindowTriggerDependencyReference`, `SelfDependencyTumblingWindowTriggerReference` | No |

+| `dependsOn: size` | The size of the dependency tumbling window. | `Timespan`<br/>(hh:mm:ss) | A positive `timespan` value where the default is the window size of the child trigger. | No |

+| `dependsOn: offset` | The offset of the dependency trigger. | `Timespan`<br/>(hh:mm:ss) | A `timespan` value that must be negative in a self-dependency. If no value is specified, the window is the same as the trigger itself. | Self-Dependency: Yes<br/>Other: No |

+> After a tumbling window trigger is published, the `interval` and `frequency` values can't be edited.

+You can use the `WindowStart` and `WindowEnd` system variables of the tumbling window trigger in your **pipeline** definition (that is, for part of a query). Pass the system variables as parameters to your pipeline in the **trigger** definition. The following example shows you how to pass these variables as parameters.

+To use the `WindowStart` and `WindowEnd` system variable values in the pipeline definition, use your `MyWindowStart` and `MyWindowEnd` parameters, accordingly.

+If the trigger `startTime` is in the past, then based on the formula M=(CurrentTime- TriggerStartTime)/TumblingWindowSize, the trigger generates {M} backfill(past) runs in parallel, honoring trigger concurrency, before executing the future runs. The order of execution for windows is deterministic, from oldest to newest intervals. Currently, this behavior can't be modified.

+> In this scenario, all runs from the selected `startTime` are run before executing future runs. If you need to backfill a long period of time, we recommend doing an initial historical load.

+The following points apply to updating existing `TriggerResource` elements:

+* The value for the `frequency` element (or window size) of the trigger along with the `interval` element can't be changed after the trigger is created. This restriction is required for proper functioning of `triggerRun` reruns and dependency evaluations.

+* If the value for the `endTime` element of the trigger changes (by adding or updating), the state of the windows that are already processed is *not* reset. The trigger honors the new `endTime` value. If the new `endTime` value is before the windows that are already executed, the trigger stops. Otherwise, the trigger stops when the new `endTime` value is encountered.

+### User-assigned retries of pipelines

+In the case of pipeline failures, a tumbling window trigger can retry the execution of the referenced pipeline automatically by using the same input parameters, without user intervention. Use the `retryPolicy` property in the trigger definition to specify this action.

+### Cancel a tumbling window run

+You can cancel runs for a tumbling window trigger if the specific window is in a **Waiting**, **Waiting on dependency**, or **Running** state:

+* If the window is in a **Running** state, cancel the associated **Pipeline Run**, and the trigger run is marked as **Canceled** afterwards.

+* If the window is in a **Waiting** or **Waiting on dependency** state, you can cancel the window from **Monitoring**.

+You can also rerun a canceled window. The rerun takes the _latest_ published definitions of the trigger. Dependencies for the specified window are _reevaluated_ upon rerun.

+## Sample for Azure PowerShell and the Azure CLI

+- **Azure subscription**: If you don't have an Azure subscription, [create a free account](https://azure.microsoft.com/free/) before you begin.

+- **Azure PowerShell**: Follow the instructions in [Install Azure PowerShell on Windows with PowerShellGet](/powershell/azure/install-azure-powershell).

+- **Azure Data Factory**: Follow the instructions in [Create an Azure Data Factory by using PowerShell](./quickstart-create-data-factory-powershell.md) to create a data factory and a pipeline.

+### Sample code

+ > Before you save the JSON file, set the value of the `startTime` element to the current Coordinated Universal Time (UTC) time. Set the value of the `endTime` element to one hour past the current UTC time.

+1. Create a trigger by using the [Set-AzDataFactoryV2Trigger](/powershell/module/az.datafactory/set-azdatafactoryv2trigger) cmdlet:

+1. Confirm that the status of the trigger is **Stopped** by using the [Get-AzDataFactoryV2Trigger](/powershell/module/az.datafactory/get-azdatafactoryv2trigger) cmdlet:

+1. Start the trigger by using the [Start-AzDataFactoryV2Trigger](/powershell/module/az.datafactory/start-azdatafactoryv2trigger) cmdlet:

+1. Confirm that the status of the trigger is **Started** by using the [Get-AzDataFactoryV2Trigger](/powershell/module/az.datafactory/get-azdatafactoryv2trigger) cmdlet:

+1. Get the trigger runs in Azure PowerShell by using the [Get-AzDataFactoryV2TriggerRun](/powershell/module/az.datafactory/get-azdatafactoryv2triggerrun) cmdlet. To get information about the trigger runs, execute the following command periodically. Update the `TriggerRunStartedAfter` and `TriggerRunStartedBefore` values to match the values in your trigger definition:

+This section shows you how to use the Azure CLI to create, start, and monitor a trigger.

+- Follow the instructions in [Create an Azure Data Factory by using the Azure CLI](./quickstart-create-data-factory-azure-cli.md) to create a data factory and a pipeline.

+### Sample code

+ > Before you save the JSON file, set the value of `referenceName` to your pipeline name. Set the value of the `startTime` element to the current UTC time. Set the value of the `endTime` element to one hour past the current UTC time.

+1. Create a trigger by using the [az datafactory trigger create](/cli/azure/datafactory/trigger#az-datafactory-trigger-create) command:

+ > For this step and all subsequent steps, replace `ResourceGroupName` with your resource group name. Replace `DataFactoryName` with your data factory's name.

+1. Confirm that the status of the trigger is **Stopped** by using the [az datafactory trigger show](/cli/azure/datafactory/trigger#az-datafactory-trigger-show) command:

+1. Start the trigger by using the [az datafactory trigger start](/cli/azure/datafactory/trigger#az-datafactory-trigger-start) command:

+1. Confirm that the status of the trigger is **Started** by using the [az datafactory trigger show](/cli/azure/datafactory/trigger#az-datafactory-trigger-show) command:

+1. Get the trigger runs in the Azure CLI by using the [az datafactory trigger-run query-by-factory](/cli/azure/datafactory/trigger-run#az-datafactory-trigger-run-query-by-factory) command. To get information about the trigger runs, execute the following command periodically. Update the `last-updated-after` and `last-updated-before` values to match the values in your trigger definition:

+* [Pipeline execution and triggers](concepts-pipeline-execution-triggers.md#trigger-execution-with-json)

+* [Create a tumbling window trigger dependency](tumbling-window-trigger-dependency.md)

+* [Reference trigger metadata in pipeline runs](how-to-use-trigger-parameterization.md)

+description: Learn how to reference trigger metadata in pipelines.

+This article describes how trigger metadata, such as the trigger start time, can be used in a pipeline run.

+A pipeline sometimes needs to understand and read metadata from the trigger that invokes it. For instance, with a tumbling window trigger run, based on the window start and end time, the pipeline processes different data slices or folders. In Azure Data Factory, we use parameterization and [system variables](control-flow-system-variables.md) to pass metadata from triggers to pipelines.

+This pattern is especially useful for [tumbling window triggers](how-to-create-tumbling-window-trigger.md), where the trigger provides the window start and end time, and [custom event triggers](how-to-create-custom-event-trigger.md), where the trigger parses and processes values in a [custom-defined *data* field](../event-grid/event-schema.md).

+> Different trigger types provide different metadata information. For more information, see [System variables](control-flow-system-variables.md).

+This section shows you how to pass metadata information from triggers to pipelines, within the Data Factory user interface (UI).

+1. Go to the **Authoring Canvas** and edit a pipeline.

+1. Select the blank canvas to bring up pipeline settings. Don't select any activity. You might need to pull up the setting pane from the bottom of the canvas because it might be collapsed.

+1. Select the **Parameters** tab and select **+ New** to add parameters.

+ :::image type="content" source="media/how-to-use-trigger-parameterization/01-create-parameter.png" alt-text="Screenshot that shows a pipeline setting showing how to define parameters in a pipeline.":::

+1. Add triggers to the pipeline by selecting **+ Trigger**.

+1. Create or attach a trigger to the pipeline and select **OK**.

+1. After you select **OK**, another **New trigger** page appears with a list of the parameters specified for the pipeline, as shown in the following screenshot. On that page, fill in the trigger metadata for each parameter. Use the format defined in [System variables](control-flow-system-variables.md) to retrieve trigger information. You don't need to fill in the information for all parameters. Just fill in the ones that will assume trigger metadata values. For instance, here we assign the trigger run start time to `parameter_1`.

+ :::image type="content" source="media/how-to-use-trigger-parameterization/02-pass-in-system-variable.png" alt-text="Screenshot that shows the Trigger Run Parameters page showing how to pass trigger information to pipeline parameters.":::

+1. To use the values in the pipeline, utilize parameters, like `@pipeline().parameters.parameterName`, *not* system variables, in pipeline definitions. For instance, in this case, to read the trigger start time, we reference `@pipeline().parameters.parameter_1`.

+To pass in trigger information to pipeline runs, both the trigger and the pipeline JSON need to be updated with the `parameters` section.

+Under the `properties` section, add parameter definitions to the `parameters` section.

+Under the `pipelines` section, assign parameter values in the `parameters` section. You don't need to fill in the information for all parameters. Just fill in the ones that will assume trigger metadata values.

+### Use trigger information in a pipeline

+To use the values in a pipeline, utilize parameters, like `@pipeline().parameters.parameterName`, *not* system variables, in pipeline definitions.

+For more information about triggers, see [Pipeline execution and triggers](concepts-pipeline-execution-triggers.md#trigger-execution-with-json).

+This article provides steps to create a dependency on a tumbling window trigger. For general information about tumbling window triggers, see [Create a tumbling window trigger](how-to-create-tumbling-window-trigger.md).

+To build a dependency chain and make sure that a trigger is executed only after the successful execution of another trigger within the service, use this advanced feature to create a tumbling window dependency.

+For a demonstration on how to create dependent pipelines by using a tumbling window trigger, watch the following video:

+To create dependency on a trigger, select **Trigger** > **Advanced** > **New**. Then choose the trigger to depend on with the appropriate offset and size. Select **Finish** and publish the changes for the dependencies to take effect.

+The following table provides the list of attributes needed to define a tumbling window dependency.

+| Property name | Description | Type | Required |

+| `type` | All the existing tumbling window triggers are displayed in this dropdown list. Choose the trigger to take dependency on. | `TumblingWindowTriggerDependencyReference` or `SelfDependencyTumblingWindowTriggerReference` | Yes |

+| `offset` | Offset of the dependency trigger. Provide a value in the timespan format. Both negative and positive offsets are allowed. This property is mandatory if the trigger is depending on itself. In all other cases, it's optional. Self-dependency should always be a negative offset. If no value is specified, the window is the same as the trigger itself. | Timespan<br/>(hh:mm:ss) | Self-Dependency: Yes<br/>Other: No |

+| `size` | Size of the dependency tumbling window. Provide a positive timespan value. This property is optional. | Timespan<br/>(hh:mm:ss) | No |

+In scenarios where the trigger shouldn't proceed to the next window until the preceding window is successfully completed, build a self-dependency. A self-dependency trigger that's dependent on the success of earlier runs of itself within the preceding hour has the properties indicated in the following code.

+> If your triggered pipeline relies on the output of pipelines in previously triggered windows, we recommend using only tumbling window trigger self-dependency. To limit parallel trigger runs, set the maximum trigger concurrency.

+The following scenarios show the use of tumbling window dependency properties.

+The following example shows a daily telemetry processing job that depends on another daily job aggregating the last seven days of output and generates seven-day rolling window streams.

+The following example shows a daily job with no gaps in the output streams of the job.

+You can monitor the dependency chain and the corresponding windows from the trigger run monitoring page. Go to **Monitoring** > **Trigger Runs**. If a tumbling window trigger has dependencies, the trigger name bears a hyperlink to a dependency monitoring view.

+Click through the trigger name to view trigger dependencies. The pane on the right shows trigger run information such as the run ID, window time, and status.

+You can see the status of the dependencies and windows for each dependent trigger. If one of the dependencies triggers fails, you must successfully rerun it for the dependent trigger to run.

+A tumbling window trigger waits on dependencies for _seven days_ before timing out. After seven days, the trigger run fails.

+> A tumbling window trigger can't be canceled while it's in the **Waiting on dependency** state. The dependent activity must finish before the tumbling window trigger can be canceled. This restriction is by design to ensure that dependent activities can complete once they're started. It also helps to reduce the likelihood of unexpected results.

+For a more visual way to view the trigger dependency schedule, select the Gantt view.

+Transparent boxes show the dependency windows for each downstream-dependent trigger. Solid-colored boxes shown in the preceding image show individual window runs. Here are some tips for interpreting the Gantt chart view:

+* Transparent boxes render blue when dependent windows are in a **Pending** or **Running** state.

+* After all windows succeed for a dependent trigger, the transparent box turns green.

+* Transparent boxes render red when a dependent window fails. Look for a solid red box to identify the failure window run.

+To rerun a window in the Gantt chart view, select the solid color box for the window. An action pane pops up with information and rerun options.

+- [Create a tumbling window trigger](how-to-create-tumbling-window-trigger.md)

+This update resolves six CVEs, which are listed in [software version 24.1.3 feature documentation](release-notes.md#version-2413).

+>

+1. After deployment finishes, enable virtual network integration between your logic app and the private endpoints on the virtual network connected to your storage account.

+ 1. In the **Outbound traffic configuration** section, next to **Virtual network integration**, select **Not configured** > **Add virtual network integration** .

+ 1. On the **Add virtual network integration** pane that opens, select your Azure subscription and your virtual network.

+ 1. From the **Subnet** list, select the subnet where you want to add your logic app. When you're done, select **Connect**.

+1. To access your logic app workflow data over the virtual network, follow these steps:

+ 1. On the logic app resource menu, under **Settings**, select **Environment variables**.

+ 1. On the **App settings** tab, add the **WEBSITE_CONTENTOVERVNET** app setting, if none exist, and set the value to **1**.

+ 1. If you use your own domain name server (DNS) with your virtual network, add the **WEBSITE_DNS_SERVER** app setting, if none exist, and set the value to the IP address for your DNS. If you have a secondary DNS, add another app setting named **WEBSITE_DNS_ALT_SERVER**, and set the value to the IP for your secondary DNS.

+ >

+To specify the allowed IP ranges, follow these steps for your Consumption or Standard logic app in the Azure portal or your Azure Resource Manager template:

+1. In the [Azure portal](https://portal.azure.com), open your Consumption logic app workflow in the designer.

+1. In the **Access control configuration** section, under **Allowed inbound IP addresses**, from the **Trigger access option** list, select **Specific IP ranges**.

+1. In the **IP ranges for contents** box, specify the IP address ranges that can access content from inputs and outputs.

+1. In the [Azure portal](https://portal.azure.com), open your Standard logic app resource.

+1. In the **Inbound traffic configuration** section, next to **Public network access**, select **Enabled with no access restriction**.

+1. On the **Access restrictions** page, under **App access**, select **Enabled from select virtual networks and IP addresses**.

+1. Under **Site access and rules**, on the **Main site** tab, add one or more rules to either **Allow** or **Deny** requests from specific IP ranges. You can also use the HTTP header filter settings and forwarding settings. A valid IP range uses these formats: *x.x.x.x/x* or *x.x.x.x-x.x.x.x*

+1. In the [Azure portal](https://portal.azure.com), open your Standard logic app resource.

+1. In the **Inbound traffic configuration** section, next to **Public network access**, select **Enabled with no access restriction**.

+1. On the **Access restrictions** page, under **App access**, select **Enabled from select virtual networks and IP addresses**.

+1. Under **Site access and rules**, on the **Main site** tab, add one or more rules to either **Allow** or **Deny** requests from specific IP ranges. A valid IP range uses these formats: *x.x.x.x/x* or *x.x.x.x-x.x.x.x*

+1. On the logic app resource menu, under **Settings**, select **Networking**.

+1. On the **Networking** page, in the **Inbound traffic configuration** section, select the link next to **Private endpoints**.

+1. On the **Private Endpoint connections** page, select **Add** > **Express** or **Advanced**.

+ For more information about the **Advanced** option, see [Create a private endpoint](../private-link/create-private-endpoint-portal.md#create-a-private-endpoint).

+1. On the **Add Private Endpoint** pane, provide the requested information about the endpoint.

+1. In the [Azure portal](https://portal.azure.com), on the logic app resource menu, under **Settings**, select **Networking**.

+1. On the **Networking** page, in the **Outbound traffic configuration** section, select the link next to **Virtual network integration**.

+1. On the **Virtual network integration** page, select **Add virtual network integration**.

+1. On the **Add virtual network integration** pane, select the subscription, the virtual network that connects to your internal service, and the subnet where to add the logic app. When you finish, select **Connect**.

+ On the **Virtual Network Integration** page, by default, the **Outbound internet traffic** setting is selected, which routes all outbound traffic through the virtual network. In this scenario, the app setting named **WEBSITE_VNET_ROUTE_ALL** is ignored.

+ To find this app setting, on the logic app resource menu, under **Settings**, select **Environment variables**.

+1. If you use your own domain name server (DNS) with your virtual network, add the **WEBSITE_DNS_SERVER** app setting, if none exist, and set the value to the IP address for your DNS. If you have a secondary DNS, add another app setting named **WEBSITE_DNS_ALT_SERVER**, and set the value to the IP for your secondary DNS.

+Azure Machine Learning checks and validates any machine learning packages that might require an upgrade. Updates incorporate the latest OS-related patches from Canonical as the original Linux OS publisher. In addition to patches applied by the original publisher, Azure Machine Learning updates system packages when updates are available. For details on the patching process, see [Vulnerability Management](./concept-vulnerability-management.md).

+CUDnn==`9.1.1`

+autokeras==`1.0.16`

+keras=`2.15.0`

+ray==`2.2.0`

+docker version==`24.0.9-1`

+## February 16, 2024

+- `.NET` updated from `3.1` (end-of-life) to `6.0`

+In this article, you learn how to use Azure Machine Learning studio to deploy the Mistral family of models as serverless APIs with pay-as-you-go token-based billing.

+Mistral AI offers two categories of models in Azure Machine Learning studio. These models are available in the [model catalog](concept-model-catalog.md).

+- __Premium models__: Mistral Large and Mistral Small. These models can be deployed as serverless APIs with pay-as-you-go token-based billing.

+- __Open models__: Mixtral-8x7B-Instruct-v01, Mixtral-8x7B-v01, Mistral-7B-Instruct-v01, and Mistral-7B-v01. These models can be deployed to managed computes in your own Azure subscription.

+You can browse the Mistral family of models in the model catalog by filtering on the Mistral collection.

+## Deploy Mistral family of models as a serverless API

+Certain models in the model catalog can be deployed as a serverless API with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription, while keeping the enterprise security and compliance that organizations need. This deployment option doesn't require quota from your subscription.

+**Mistral Large** and **Mistral Small** can be deployed as a serverless API with pay-as-you-go billing and are offered by Mistral AI through the Microsoft Azure Marketplace. Mistral AI can change or update the terms of use and pricing of these models.

+- An Azure Machine Learning workspace. If you don't have a workspace, use the steps in the [Quickstart: Create workspace resources](quickstart-create-resources.md) article to create one. The serverless API model deployment offering for eligible models in the Mistral family is only available in workspaces created in these regions:

+ - East US

+ - East US 2

+ - North Central US

+ - South Central US

+ - West US

+ - West US 3

+ - Sweden Central

+ For a list of regions that are available for each of the models supporting serverless API endpoint deployments, see [Region availability for models in serverless API endpoints](concept-endpoint-serverless-availability.md)

+1. Select the workspace in which you want to deploy your model. To use the serverless API model deployment offering, your workspace must belong to one of the regions listed in the [prerequisites](#prerequisites).

+1. Choose the model you want to deploy, for example Mistral-large, from the [model catalog](https://ml.azure.com/model/catalog).

+1. On the model's overview page in the model catalog, select **Deploy** to open a serverless API deployment window for the model.

+1. Select the checkbox to acknowledge the Microsoft purchase policy.

+ :::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api.png" alt-text="A screenshot showing how to deploy a model as a serverless API." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api.png":::

+1. You can also select the **Pricing and terms** tab to learn about pricing for the selected model.

+ :::image type="content" source="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api-project.png" alt-text="A screenshot showing a workspace that is already subscribed to the offering." lightbox="media/how-to-deploy-models-mistral/mistral-deploy-serverless-api-project.png":::

+To learn about billing for Mistral models deployed as a serverless API with pay-as-you-go token-based billing, see [Cost and quota considerations for Mistral family of models deployed as a service](#cost-and-quota-considerations-for-mistral-family-of-models-deployed-as-a-service).

+## Reference for Mistral family of models deployed as a service

+### Mistral Chat API

+#### Request example

+#### Response example

+- [Region availability for models in serverless API endpoints](concept-endpoint-serverless-availability.md)

+- [Plan and manage costs for Azure AI Studio](concept-plan-manage-cost.md)

+- [Review](best-practices-assessment.md) the best practices for creating assessments.

+- Learn more on how to [build](how-to-build-a-business-case.md) and [view](how-to-view-a-business-case.md) a business case.

+4. **Install VDDK**: _(Needed only for VMware appliance.)_ The appliance checks that the VMware vSphere Virtual Disk Development Kit (VDDK) is installed. If it isn't installed, download VDDK 6.7, 7, or 8(depending on the compatibility of VDDK and ESXi versions) from VMware. Extract the downloaded zipped contents to the specified location on the appliance, as provided in the installation instructions.

+1. **Install the VDDK**: The appliance checks that VMware vSphere Virtual Disk Development Kit (VDDK) is installed. If the VDDK isn't installed, download VDDK 6.7, 7, or 8(depending on the compatibility of VDDK and ESXi versions) from VMware. Extract the downloaded zip file contents to the specified location on the appliance, as indicated in the *Installation instructions*.

+1. **Install the VDDK**: The appliance checks that VMware vSphere Virtual Disk Development Kit (VDDK) is installed. If the VDDK isn't installed, download VDDK 6.7, 7.0, or 8(depending on the compatibility of VDDK and ESXi versions) from VMware. Extract the downloaded zip file contents to the specified location on the appliance, the default path is *C:\Program Files\VMware\VMware Virtual Disk Development Kit* as indicated in the *Installation instructions*.

+Required agents | On each server that you want to analyze, install the following agents:<br />- Azure Monitor agent (AMA)<br />- [Dependency agent](../../azure-monitor/vm/vminsights-dependency-agent-maintenance.md)<br /><br /> If on-premises servers aren't connected to the internet, download and install the Log Analytics gateway on them.<br /><br /> Learn more about installing the [Dependency agent](../how-to-create-group-machine-dependencies.md#install-the-dependency-agent) and the Azure Monitor agent.

+- CentOS 9.x (Release and Stream)

+ Review the list of [required packages](../../virtual-machines/extensions/agent-linux.md#requirements) to install Linux VM agent. Azure Migrate installs the Linux VM agent automatically for RHEL 8.x/7.x/6.x, Ubuntu 14.04/16.04/18.04/19.04/19.10/20.04, SUSE 15 SP0/15 SP1/12, Debian 9/8/7, and Oracle 7/6 when using the agentless method of VMware migration. Follow these instructions to [install the Linux Agent manually](../../virtual-machines/extensions/agent-linux.md#installation) for other OS versions.

+**SQL Server access** | To discover SQL Server instances and databases, the Windows account, or SQL Server account [requires these permissions](migrate-support-matrix-vmware.md#configure-the-custom-login-for-sql-server-discovery) for each SQL Server instance. You can use the [account provisioning utility](../least-privilege-credentials.md) to create custom accounts or use any existing account that is a member of the sysadmin server role for simplicity.

+1. **Install the VDDK**: The appliance checks that VMware vSphere Virtual Disk Development Kit (VDDK) is installed. Download VDDK 6.7, 7, or 8(depending on the compatibility of VDDK and ESXi versions) from VMware. Extract the downloaded zip file contents to the specified location on the appliance, the default path is *C:\Program Files\VMware\VMware Virtual Disk Development Kit* as indicated in the *Installation instructions*.

+The **Operating system license support status** column displays the support status of the Operating system, whether it is in mainstream support, extended support, or out of support. Selecting the support status opens a pane on the right, which provides clear guidance regarding actionable steps that can be taken to secure servers and databases in extended support or out of support.

+The **Database instances** displays the number of instances discovered by Azure Migrate. Select the number of instances to view the database instance details. The **Database instance license support status** displays the support status of the database instance. Selecting the support status opens a pane on the right, which provides clear guidance regarding actionable steps that can be taken to secure servers and databases in extended support or out of support.

+For MySQL version 5.7, default value is 1 in Azure Database for MySQL flexible server. It's important to note that while it's possible to change the supported value to 2, reverting from 2 back to 1 isn't allowed. Contact our [support team](https://azure.microsoft.com/support/create-ticket/) for assistance in changing the default value.

+The [Physical Memory Size](./concepts-service-tiers-storage.md#physical-memory-size-gb) (GB) in the table below represents the available random-access memory (RAM) in gigabytes (GB) on your Azure Database for MySQL flexible server.

+|**Pricing Tier**|**vCore(s)**|**Physical Memory Size (GiB)**|**Default value (bytes)**|**Min value (bytes)**|**Max value (bytes)**|

+|Burstable (B1s)|1|1|134217728|33554432|268435456|

+|Burstable (B1ms)|1|2|536870912|134217728|1073741824|

+|General Purpose|2|8|4294967296|134217728|5368709120|

+|Business Critical|20|160|128849018880|134217728|128849018880|

+[innodb_log_file_size](https://dev.mysql.com/doc/refman/8.0/en/innodb-parameters.html#sysvar_innodb_log_file_size) is the size in bytes of each [log file](https://dev.mysql.com/doc/refman/8.0/en/glossary.html#glos_log_file) in a [log group](https://dev.mysql.com/doc/refman/8.0/en/glossary.html#glos_log_group). The combined size of log files [(innodb_log_file_size](https://dev.mysql.com/doc/refman/8.0/en/innodb-parameters.html#sysvar_innodb_log_file_size) * [innodb_log_files_in_group](https://dev.mysql.com/doc/refman/8.0/en/innodb-parameters.html#sysvar_innodb_log_files_in_group)) can't exceed a maximum value that is slightly less than 512 GB). A bigger log file size is better for performance, but it has a drawback that the recovery time after a crash is high. You need to balance recovery time in the rare event of a crash recovery versus maximizing throughput during peak operations. These can also result in longer restart times. You can configure innodb_log_size to any of these values - 256 MB, 512 MB, 1 GB, or 2 GB for Azure Database for MySQL flexible server. The parameter is static and requires a restart.

+The binary log contains "events" that describe database changes such as table creation operations or changes to table data. It also contains events for statements that potentially could have made changes. The binary log is used mainly for two purposes, replication and data recovery operations. Usually, the binary logs are purged as soon as the handle is free from service, backup, or the replica set. If there are multiple replicas, the binary logs wait for the slowest replica to read the changes before it's been purged. If you want to persist binary logs for a more duration of time, you can configure the parameter binlog_expire_logs_seconds. If the binlog_expire_logs_seconds is set to 0, which is the default value, it purges as soon as the handle to the binary log is freed. If binlog_expire_logs_seconds > 0, then it would wait until the seconds configured before it purges. For Azure Database for MySQL flexible server, managed features like backup and read replica purging of binary files are handled internally. When you replicate the data-out from Azure Database for MySQL flexible server, this parameter needs to be set in primary to avoid purging of binary logs before the replica reads from the changes from the primary. If you set the binlog_expire_logs_seconds to a higher value, then the binary logs won't be purged soon enough and can lead to increase in the storage billing.