| Service | Microsoft Docs article | Related commit history on GitHub | Change details |
|---|---|---|---|
| advisor | Advisor Resiliency Reviews | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/advisor/advisor-resiliency-reviews.md | You can manage access to Advisor personalized recommendations using the followin | **Name** | **Description** | ||::| |Subscription Reader|View reviews for a workload and recommendations linked to them.|-|Subscription Owner<br>Subscription Contributor|View reviews for a workload, triage recommendations linked to those reviews, manage review recommendation lifecycle.| -|Advisor Recommendations Contributor (Assessments and Reviews)|View review recommendations, accept review recommendations, manage review recommendations' lifecycle.| +|Subscription Owner<br>Subscription Contributor|View reviews for a workload, triage recommendations linked to those reviews, manage the recommendation lifecycle.| +|Advisor Recommendations Contributor (Assessments and Reviews)|View accepted recommendations, and manage the recommendation lifecycle.| You can find detailed instructions on how to assign a role using the Azure portal - [Assign Azure roles using the Azure portal - Azure RBAC](/azure/role-based-access-control/role-assignments-portal?tabs=delegate-condition). Additional information is available in [Steps to assign an Azure role - Azure RBAC](/azure/role-based-access-control/role-assignments-steps). |
| ai-services | Customizing Llms | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/openai/concepts/customizing-llms.md | + + Title: Azure OpenAI Service getting started with customizing a large language model (LLM) ++description: Learn more about the concepts behind customizing an LLM with Azure OpenAI. + Last updated : 03/26/2024+++++recommendations: false +++# Getting started with customizing a large language model (LLM) ++There are several techniques for adapting a pre-trained language model to suit a specific task or domain. These include prompt engineering, RAG (Retrieval Augmented Generation), and fine-tuning. These three techniques are not mutually exclusive but are complementary methods that in combination can be applicable to a specific use case. In this article, we'll explore these techniques, illustrative use cases, things to consider, and provide links to resources to learn more and get started with each. ++## Prompt engineering ++### Definition ++[Prompt engineering](./prompt-engineering.md) is a technique that is both art and science, which involves designing prompts for generative AI models. This process utilizes in-context learning ([zero shot and few shot](./prompt-engineering.md#examples)) and, with iteration, improves accuracy and relevancy in responses, optimizing the performance of the model. ++### Illustrative use cases ++A Marketing Manager at an environmentally conscious company can use prompt engineering to help guide the model to generate descriptions that are more aligned with their brandΓÇÖs tone and style. For instance, they can add a prompt like "Write a product description for a new line of eco-friendly cleaning products that emphasizes quality, effectiveness, and highlights the use of environmentally friendly ingredients" to the input. This will help the model generate descriptions that are aligned with their brandΓÇÖs values and messaging. ++### Things to consider ++- **Prompt engineering** is the starting point for generating desired output from generative AI models. ++- **Craft clear instructions**: Instructions are commonly used in prompts and guide the model's behavior. Be specific and leave as little room for interpretation as possible. Use analogies and descriptive language to help the model understand your desired outcome. ++- **Experiment and iterate**: Prompt engineering is an art that requires experimentation and iteration. Practice and gain experience in crafting prompts for different tasks. Every model might behave differently, so it's important to adapt prompt engineering techniques accordingly. ++### Getting started ++- [Introduction to prompt engineering](./prompt-engineering.md) +- [Prompt engineering techniques](./advanced-prompt-engineering.md) +- [15 tips to become a better prompt engineer for generative AI](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/15-tips-to-become-a-better-prompt-engineer-for-generative-ai/ba-p/3882935) +- [The basics of prompt engineering (video)](https://www.youtube.com/watch?v=e7w6QV1NX1c) ++## RAG (Retrieval Augmented Generation) ++### Definition ++[RAG (Retrieval Augmented Generation)](../../../ai-studio/concepts/retrieval-augmented-generation.md) is a method that integrates external data into a Large Language Model prompt to generate relevant responses. This approach is particularly beneficial when using a large corpus of unstructured text based on different topics. It allows for answers to be grounded in the organizationΓÇÖs knowledge base (KB), providing a more tailored and accurate response. ++RAG is also advantageous when answering questions based on an organizationΓÇÖs private data or when the public data that the model was trained on might have become outdated. This helps ensure that the responses are always up-to-date and relevant, regardless of the changes in the data landscape. ++### Illustrative use case ++A corporate HR department is looking to provide an intelligent assistant that answers specific employee health insurance related questions such as "are eyeglasses covered?" RAG is used to ingest the extensive and numerous documents associated with insurance plan policies to enable the answering of these specific types of questions. ++### Things to consider ++- RAG helps ground AI output in real-world data and reduces the likelihood of fabrication. ++- RAG is helpful when there is a need to answer questions based on private proprietary data. ++- RAG is helpful when you might want questions answered that are recent (for example, before the cutoff date of when the [model version](./models.md) was last trained). ++### Getting started ++- [Retrieval Augmented Generation in Azure AI Studio - Azure AI Studio | Microsoft Learn](../../../ai-studio/concepts/retrieval-augmented-generation.md) +- [Retrieval Augmented Generation (RAG) in Azure AI Search](../../../search/retrieval-augmented-generation-overview.md) +- [Retrieval Augmented Generation using Azure Machine Learning prompt flow (preview)](../../../machine-learning/concept-retrieval-augmented-generation.md) ++## Fine-tuning ++### Definition ++[Fine-tuning](../how-to/fine-tuning.md), specifically [supervised fine-tuning](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/fine-tuning-now-available-with-azure-openai-service/ba-p/3954693?lightbox-message-images-3954693=516596iC5D02C785903595A) in this context, is an iterative process that adapts an existing large language model to a provided training set in order to improve performance, teach the model new skills, or reduce latency. This approach is used when the model needs to learn and generalize over specific topics, particularly when these topics are generally small in scope. ++Fine-tuning requires the use of high-quality training data, in a [special example based format](../how-to/fine-tuning.md#example-file-format), to create the new fine-tuned Large Language Model. By focusing on specific topics, fine-tuning allows the model to provide more accurate and relevant responses within those areas of focus. ++### Illustrative use case ++An IT department has been using GPT-4 to convert natural language queries to SQL, but they have found that the responses are not always reliably grounded in their schema, and the cost is prohibitively high. ++They fine-tune GPT-3.5-Turbo with hundreds of requests and correct responses and produce a model that performs better than the base model with lower costs and latency. ++### Things to consider ++- Fine-tuning is an advanced capability; it enhances LLM with after-cutoff-date knowledge and/or domain specific knowledge. Start by evaluating the baseline performance of a standard model against their requirements before considering this option. ++- Having a baseline for performance without fine-tuning is essential for knowing whether fine-tuning has improved model performance. Fine-tuning with bad data makes the base model worse, but without a baseline, it's hard to detect regressions. ++- Good cases for fine-tuning include steering the model to output content in a specific and customized style, tone, or format, or tasks where the information needed to steer the model is too long or complex to fit into the prompt window. ++- Fine-tuning costs: ++ - Fine-tuning can reduce costs across two dimensions: (1) by using fewer tokens depending on the task (2) by using a smaller model (for example GPT 3.5 Turbo can potentially be fine-tuned to achieve the same quality of GPT-4 on a particular task). ++ - Fine-tuning has upfront costs for training the model. And additional hourly costs for hosting the custom model once it's deployed. ++### Getting started ++- [When to use Azure OpenAI fine-tuning](./fine-tuning-considerations.md) +- [Customize a model with fine-tuning](../how-to/fine-tuning.md) +- [Azure OpenAI GPT 3.5 Turbo fine-tuning tutorial](../tutorials/fine-tune.md) +- [To fine-tune or not to fine-tune? (Video)](https://www.youtube.com/watch?v=0Jo-z-MFxJs) |
| ai-services | Models | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/openai/concepts/models.md | See [model versions](../concepts/model-versions.md) to learn about how Azure Ope **<sup>1</sup>** This model will accept requests > 4,096 tokens. It is not recommended to exceed the 4,096 input token limit as the newer version of the model are capped at 4,096 tokens. If you encounter issues when exceeding 4,096 input tokens with this model this configuration is not officially supported. +#### Azure Government regions ++The following GPT-3.5 turbo models are available with [Azure Government](/azure/azure-government/documentation-government-welcome): ++|Model ID | Model Availability | +|--|--| +| `gpt-35-turbo` (1106-Preview) | US Gov Virginia | + ### Embeddings models These models can only be used with Embedding API requests. |
| ai-services | Use Your Data | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/openai/concepts/use-your-data.md | You can modify the following additional settings in the **Data parameters** sect |**Retrieved documents** | This parameter is an integer that can be set to 3, 5, 10, or 20, and controls the number of document chunks provided to the large language model for formulating the final response. By default, this is set to 5. The search process can be noisy and sometimes, due to chunking, relevant information might be spread across multiple chunks in the search index. Selecting a top-K number, like 5, ensures that the model can extract relevant information, despite the inherent limitations of search and chunking. However, increasing the number too high can potentially distract the model. Additionally, the maximum number of documents that can be effectively used depends on the version of the model, as each has a different context size and capacity for handling documents. If you find that responses are missing important context, try increasing this parameter. This is the `topNDocuments` parameter in the API, and is 5 by default. | | **Strictness** | Determines the system's aggressiveness in filtering search documents based on their similarity scores. The system queries Azure Search or other document stores, then decides which documents to provide to large language models like ChatGPT. Filtering out irrelevant documents can significantly enhance the performance of the end-to-end chatbot. Some documents are excluded from the top-K results if they have low similarity scores before forwarding them to the model. This is controlled by an integer value ranging from 1 to 5. Setting this value to 1 means that the system will minimally filter documents based on search similarity to the user query. Conversely, a setting of 5 indicates that the system will aggressively filter out documents, applying a very high similarity threshold. If you find that the chatbot omits relevant information, lower the filter's strictness (set the value closer to 1) to include more documents. Conversely, if irrelevant documents distract the responses, increase the threshold (set the value closer to 5). This is the `strictness` parameter in the API, and set to 3 by default. | +### Uncited references ++It's possible for the model to return `"TYPE":"UNCITED_REFERENCE"` instead of `"TYPE":CONTENT` in the API for documents that are retrieved from the data source, but not included in the citation. This can be useful for debugging, and you can control this behavior by modifying the **strictness** and **retrieved documents** runtime parameters described above. + ### System message You can define a system message to steer the model's reply when using Azure OpenAI On Your Data. This message allows you to customize your replies on top of the retrieval augmented generation (RAG) pattern that Azure OpenAI On Your Data uses. The system message is used in addition to an internal base prompt to provide the experience. To support this, we truncate the system message after a specific [number of tokens](#token-usage-estimation-for-azure-openai-on-your-data) to ensure the model can answer questions using your data. If you are defining extra behavior on top of the default experience, ensure that your system prompt is detailed and explains the exact expected customization. token_output = TokenEstimator.estimate_tokens(input_text) ## Troubleshooting -### Failed ingestion jobs --To troubleshoot a failed job, always look out for errors or warnings specified either in the API response or Azure OpenAI studio. Here are some of the common errors and warnings: +To troubleshoot failed operations, always look out for errors or warnings specified either in the API response or Azure OpenAI studio. Here are some of the common errors and warnings: +### Failed ingestion jobs **Quota Limitations Issues** Resolution: This means the storage account isn't accessible with the given credentials. In this case, please review the storage account credentials passed to the API and ensure the storage account isn't hidden behind a private endpoint (if a private endpoint isn't configured for this resource). +### 503 errors when sending queries with Azure AI Search ++Each user message can translate to multiple search queries, all of which get sent to the search resource in parallel. This can produce throttling behavior when the amount of search replicas and partitions is low. The maximum number of queries per second that a single partition and single replica can support may not be sufficient. In this case, consider increasing your replicas and partitions, or adding sleep/retry logic in your application. See the [Azure AI Search documentation](../../../search/performance-benchmarks.md) for more information. + ## Regional availability and model support You can use Azure OpenAI On Your Data with an Azure OpenAI resource in the following regions: |
| ai-services | Chat Markup Language | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/openai/how-to/chat-markup-language.md | + + Title: How to work with the Chat Markup Language (preview) ++description: Learn how to work with Chat Markup Language (preview) ++++ Last updated : 04/05/2024++keywords: ChatGPT +++# Chat Markup Language ChatML (Preview) ++> [!IMPORTANT] +> Using GPT-3.5-Turbo models with the completion endpoint as described in this article remains in preview and is only possible with `gpt-35-turbo` version (0301) which is [slated for retirement as early as June 13th, 2024](../concepts/model-retirements.md#current-models). We strongly recommend using the [GA Chat Completion API/endpoint](./chatgpt.md). The Chat Completion API is the recommended method of interacting with the GPT-3.5-Turbo models. The Chat Completion API is also the only way to access the GPT-4 models. ++The following code snippet shows the most basic way to use the GPT-3.5-Turbo models with ChatML. If this is your first time using these models programmatically we recommend starting with our [GPT-35-Turbo & GPT-4 Quickstart](../chatgpt-quickstart.md). ++> [!NOTE] +> In the Azure OpenAI documentation we refer to GPT-3.5-Turbo, and GPT-35-Turbo interchangeably. The official name of the model on OpenAI is `gpt-3.5-turbo`, but for Azure OpenAI due to Azure specific character constraints the underlying model name is `gpt-35-turbo`. ++```python +import os +import openai +openai.api_type = "azure" +openai.api_base = "https://{your-resource-name}.openai.azure.com/" +openai.api_version = "2024-02-01" +openai.api_key = os.getenv("OPENAI_API_KEY") ++response = openai.Completion.create( + engine="gpt-35-turbo", # The deployment name you chose when you deployed the GPT-35-Turbo model + prompt="<|im_start|>system\nAssistant is a large language model trained by OpenAI.\n<|im_end|>\n<|im_start|>user\nWho were the founders of Microsoft?\n<|im_end|>\n<|im_start|>assistant\n", + temperature=0, + max_tokens=500, + top_p=0.5, + stop=["<|im_end|>"]) ++print(response['choices'][0]['text']) +``` ++> [!NOTE] +> The following parameters aren't available with the gpt-35-turbo model: `logprobs`, `best_of`, and `echo`. If you set any of these parameters, you'll get an error. ++The `<|im_end|>` token indicates the end of a message. When using ChatML it is recommended to include `<|im_end|>` token as a stop sequence to ensure that the model stops generating text when it reaches the end of the message. ++Consider setting `max_tokens` to a slightly higher value than normal such as 300 or 500. This ensures that the model doesn't stop generating text before it reaches the end of the message. ++## Model versioning ++> [!NOTE] +> `gpt-35-turbo` is equivalent to the `gpt-3.5-turbo` model from OpenAI. ++Unlike previous GPT-3 and GPT-3.5 models, the `gpt-35-turbo` model as well as the `gpt-4` and `gpt-4-32k` models will continue to be updated. When creating a [deployment](../how-to/create-resource.md#deploy-a-model) of these models, you'll also need to specify a model version. ++You can find the model retirement dates for these models on our [models](../concepts/models.md) page. ++## Working with Chat Markup Language (ChatML) ++> [!NOTE] +> OpenAI continues to improve the GPT-35-Turbo and the Chat Markup Language used with the models will continue to evolve in the future. We'll keep this document updated with the latest information. ++OpenAI trained GPT-35-Turbo on special tokens that delineate the different parts of the prompt. The prompt starts with a system message that is used to prime the model followed by a series of messages between the user and the assistant. ++The format of a basic ChatML prompt is as follows: ++``` +<|im_start|>system +Provide some context and/or instructions to the model. +<|im_end|> +<|im_start|>user +The userΓÇÖs message goes here +<|im_end|> +<|im_start|>assistant +``` ++### System message ++The system message is included at the beginning of the prompt between the `<|im_start|>system` and `<|im_end|>` tokens. This message provides the initial instructions to the model. You can provide various information in the system message including: ++* A brief description of the assistant +* Personality traits of the assistant +* Instructions or rules you would like the assistant to follow +* Data or information needed for the model, such as relevant questions from an FAQ ++You can customize the system message for your use case or just include a basic system message. The system message is optional, but it's recommended to at least include a basic one to get the best results. ++### Messages ++After the system message, you can include a series of messages between the **user** and the **assistant**. Each message should begin with the `<|im_start|>` token followed by the role (`user` or `assistant`) and end with the `<|im_end|>` token. ++``` +<|im_start|>user +What is thermodynamics? +<|im_end|> +``` ++To trigger a response from the model, the prompt should end with `<|im_start|>assistant` token indicating that it's the assistant's turn to respond. You can also include messages between the user and the assistant in the prompt as a way to do few shot learning. ++### Prompt examples ++The following section shows examples of different styles of prompts that you could use with the GPT-35-Turbo and GPT-4 models. These examples are just a starting point, and you can experiment with different prompts to customize the behavior for your own use cases. ++#### Basic example ++If you want the GPT-35-Turbo and GPT-4 models to behave similarly to [chat.openai.com](https://chat.openai.com/), you can use a basic system message like "Assistant is a large language model trained by OpenAI." ++``` +<|im_start|>system +Assistant is a large language model trained by OpenAI. +<|im_end|> +<|im_start|>user +Who were the founders of Microsoft? +<|im_end|> +<|im_start|>assistant +``` ++#### Example with instructions ++For some scenarios, you might want to give additional instructions to the model to define guardrails for what the model is able to do. ++``` +<|im_start|>system +Assistant is an intelligent chatbot designed to help users answer their tax related questions. ++Instructions: +- Only answer questions related to taxes. +- If you're unsure of an answer, you can say "I don't know" or "I'm not sure" and recommend users go to the IRS website for more information. +<|im_end|> +<|im_start|>user +When are my taxes due? +<|im_end|> +<|im_start|>assistant +``` ++#### Using data for grounding ++You can also include relevant data or information in the system message to give the model extra context for the conversation. If you only need to include a small amount of information, you can hard code it in the system message. If you have a large amount of data that the model should be aware of, you can use [embeddings](../tutorials/embeddings.md?tabs=command-line) or a product like [Azure AI Search](https://techcommunity.microsoft.com/t5/ai-applied-ai-blog/revolutionize-your-enterprise-data-with-chatgpt-next-gen-apps-w/ba-p/3762087) to retrieve the most relevant information at query time. ++``` +<|im_start|>system +Assistant is an intelligent chatbot designed to help users answer technical questions about Azure OpenAI Serivce. Only answer questions using the context below and if you're not sure of an answer, you can say "I don't know". ++Context: +- Azure OpenAI Service provides REST API access to OpenAI's powerful language models including the GPT-3, Codex and Embeddings model series. +- Azure OpenAI Service gives customers advanced language AI with OpenAI GPT-3, Codex, and DALL-E models with the security and enterprise promise of Azure. Azure OpenAI co-develops the APIs with OpenAI, ensuring compatibility and a smooth transition from one to the other. +- At Microsoft, we're committed to the advancement of AI driven by principles that put people first. Microsoft has made significant investments to help guard against abuse and unintended harm, which includes requiring applicants to show well-defined use cases, incorporating MicrosoftΓÇÖs principles for responsible AI use +<|im_end|> +<|im_start|>user +What is Azure OpenAI Service? +<|im_end|> +<|im_start|>assistant +``` ++#### Few shot learning with ChatML ++You can also give few shot examples to the model. The approach for few shot learning has changed slightly because of the new prompt format. You can now include a series of messages between the user and the assistant in the prompt as few shot examples. These examples can be used to seed answers to common questions to prime the model or teach particular behaviors to the model. ++This is only one example of how you can use few shot learning with GPT-35-Turbo. You can experiment with different approaches to see what works best for your use case. ++``` +<|im_start|>system +Assistant is an intelligent chatbot designed to help users answer their tax related questions. +<|im_end|> +<|im_start|>user +When do I need to file my taxes by? +<|im_end|> +<|im_start|>assistant +In 2023, you will need to file your taxes by April 18th. The date falls after the usual April 15th deadline because April 15th falls on a Saturday in 2023. For more details, see https://www.irs.gov/filing/individuals/when-to-file +<|im_end|> +<|im_start|>user +How can I check the status of my tax refund? +<|im_end|> +<|im_start|>assistant +You can check the status of your tax refund by visiting https://www.irs.gov/refunds +<|im_end|> +``` ++#### Using Chat Markup Language for non-chat scenarios ++ChatML is designed to make multi-turn conversations easier to manage, but it also works well for non-chat scenarios. ++For example, for an entity extraction scenario, you might use the following prompt: ++``` +<|im_start|>system +You are an assistant designed to extract entities from text. Users will paste in a string of text and you will respond with entities you've extracted from the text as a JSON object. Here's an example of your output format: +{ + "name": "", + "company": "", + "phone_number": "" +} +<|im_end|> +<|im_start|>user +Hello. My name is Robert Smith. IΓÇÖm calling from Contoso Insurance, Delaware. My colleague mentioned that you are interested in learning about our comprehensive benefits policy. Could you give me a call back at (555) 346-9322 when you get a chance so we can go over the benefits? +<|im_end|> +<|im_start|>assistant +``` +++## Preventing unsafe user inputs ++It's important to add mitigations into your application to ensure safe use of the Chat Markup Language. ++We recommend that you prevent end-users from being able to include special tokens in their input such as `<|im_start|>` and `<|im_end|>`. We also recommend that you include additional validation to ensure the prompts you're sending to the model are well formed and follow the Chat Markup Language format as described in this document. ++You can also provide instructions in the system message to guide the model on how to respond to certain types of user inputs. For example, you can instruct the model to only reply to messages about a certain subject. You can also reinforce this behavior with few shot examples. +++## Managing conversations ++The token limit for `gpt-35-turbo` is 4096 tokens. This limit includes the token count from both the prompt and completion. The number of tokens in the prompt combined with the value of the `max_tokens` parameter must stay under 4096 or you'll receive an error. ++ItΓÇÖs your responsibility to ensure the prompt and completion falls within the token limit. This means that for longer conversations, you need to keep track of the token count and only send the model a prompt that falls within the token limit. ++The following code sample shows a simple example of how you could keep track of the separate messages in the conversation. ++```python +import os +import openai +openai.api_type = "azure" +openai.api_base = "https://{your-resource-name}.openai.azure.com/" #This corresponds to your Azure OpenAI resource's endpoint value +openai.api_version = "2024-02-01" +openai.api_key = os.getenv("OPENAI_API_KEY") ++# defining a function to create the prompt from the system message and the conversation messages +def create_prompt(system_message, messages): + prompt = system_message + for message in messages: + prompt += f"\n<|im_start|>{message['sender']}\n{message['text']}\n<|im_end|>" + prompt += "\n<|im_start|>assistant\n" + return prompt ++# defining the user input and the system message +user_input = "<your user input>" +system_message = f"<|im_start|>system\n{'<your system message>'}\n<|im_end|>" ++# creating a list of messages to track the conversation +messages = [{"sender": "user", "text": user_input}] ++response = openai.Completion.create( + engine="gpt-35-turbo", # The deployment name you chose when you deployed the GPT-35-Turbo model. + prompt=create_prompt(system_message, messages), + temperature=0.5, + max_tokens=250, + top_p=0.9, + frequency_penalty=0, + presence_penalty=0, + stop=['<|im_end|>'] +) ++messages.append({"sender": "assistant", "text": response['choices'][0]['text']}) +print(response['choices'][0]['text']) +``` ++## Staying under the token limit ++The simplest approach to staying under the token limit is to remove the oldest messages in the conversation when you reach the token limit. ++You can choose to always include as many tokens as possible while staying under the limit or you could always include a set number of previous messages assuming those messages stay within the limit. It's important to keep in mind that longer prompts take longer to generate a response and incur a higher cost than shorter prompts. ++You can estimate the number of tokens in a string by using the [tiktoken](https://github.com/openai/tiktoken) Python library as shown below. ++```python +import tiktoken ++cl100k_base = tiktoken.get_encoding("cl100k_base") ++enc = tiktoken.Encoding( + name="gpt-35-turbo", + pat_str=cl100k_base._pat_str, + mergeable_ranks=cl100k_base._mergeable_ranks, + special_tokens={ + **cl100k_base._special_tokens, + "<|im_start|>": 100264, + "<|im_end|>": 100265 + } +) ++tokens = enc.encode( + "<|im_start|>user\nHello<|im_end|><|im_start|>assistant", + allowed_special={"<|im_start|>", "<|im_end|>"} +) ++assert len(tokens) == 7 +assert tokens == [100264, 882, 198, 9906, 100265, 100264, 78191] +``` ++## Next steps ++* [Learn more about Azure OpenAI](../overview.md). +* Get started with the GPT-35-Turbo model with [the GPT-35-Turbo & GPT-4 quickstart](../chatgpt-quickstart.md). +* For more examples, check out the [Azure OpenAI Samples GitHub repository](https://aka.ms/AOAICodeSamples) |
| ai-services | Chatgpt | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/openai/how-to/chatgpt.md | -# Learn how to work with the GPT-35-Turbo and GPT-4 models +# Learn how to work with the GPT-3.5-Turbo and GPT-4 models -The GPT-35-Turbo and GPT-4 models are language models that are optimized for conversational interfaces. The models behave differently than the older GPT-3 models. Previous models were text-in and text-out, meaning they accepted a prompt string and returned a completion to append to the prompt. However, the GPT-35-Turbo and GPT-4 models are conversation-in and message-out. The models expect input formatted in a specific chat-like transcript format, and return a completion that represents a model-written message in the chat. While this format was designed specifically for multi-turn conversations, you'll find it can also work well for non-chat scenarios too. +The GPT-3.5-Turbo and GPT-4 models are language models that are optimized for conversational interfaces. The models behave differently than the older GPT-3 models. Previous models were text-in and text-out, meaning they accepted a prompt string and returned a completion to append to the prompt. However, the GPT-3.5-Turbo and GPT-4 models are conversation-in and message-out. The models expect input formatted in a specific chat-like transcript format, and return a completion that represents a model-written message in the chat. While this format was designed specifically for multi-turn conversations, you'll find it can also work well for non-chat scenarios too. -In Azure OpenAI there are two different options for interacting with these type of models: +This article walks you through getting started with the GPT-3.5-Turbo and GPT-4 models. It's important to use the techniques described here to get the best results. If you try to interact with the models the same way you did with the older model series, the models will often be verbose and provide less useful responses. -- Chat Completion API.-- Completion API with Chat Markup Language (ChatML). -The Chat Completion API is a new dedicated API for interacting with the GPT-35-Turbo and GPT-4 models. This API is the preferred method for accessing these models. **It is also the only way to access the new GPT-4 models**. --ChatML uses the same [completion API](../reference.md#completions) that you use for other models like text-davinci-002, it requires a unique token based prompt format known as Chat Markup Language (ChatML). This provides lower level access than the dedicated Chat Completion API, but also requires additional input validation, only supports gpt-35-turbo models, and **the underlying format is more likely to change over time**. --This article walks you through getting started with the GPT-35-Turbo and GPT-4 models. It's important to use the techniques described here to get the best results. If you try to interact with the models the same way you did with the older model series, the models will often be verbose and provide less useful responses. ------ |
| ai-services | Use Web App | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/openai/how-to/use-web-app.md | Sample source code for the web app is available on [GitHub](https://github.com/m We recommend pulling changes from the `main` branch for the web app's source code frequently to ensure you have the latest bug fixes, API version, and improvements. Additionally, the web app must be synchronized every time the API version being used is [retired](../api-version-deprecation.md#retiring-soon). +Consider either clicking the **watch** or **star** buttons on the web app's [GitHub](https://github.com/microsoft/sample-app-aoai-chatGPT) repo to be notified about changes and updates to the source code. + **If you haven't customized the app:** * You can follow the synchronization steps below |
| ai-services | Use Your Data Securely | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/openai/how-to/use-your-data-securely.md | Make sure your sign-in credential has `Cognitive Services OpenAI Contributor` ro ### Ingestion API -See the [ingestion API reference article](/azure/ai-services/openai/reference#start-an-ingestion-job) for details on the request and response objects used by the ingestion API. +See the [ingestion API reference article](/rest/api/azureopenai/ingestion-jobs?context=/azure/ai-services/openai/context/context) for details on the request and response objects used by the ingestion API. More notes: |
| ai-services | Reference | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/openai/reference.md | The operation returns a `204` status code if successful. This API only succeeds ## Speech to text +You can use a Whisper model in Azure OpenAI Service for speech to text transcription or speech translation. For more information about using a Whisper model, see the [quickstart](./whisper-quickstart.md) and [the Whisper model overview](../speech-service/whisper-overview.md). + ### Request a speech to text transcription Transcribes an audio file. POST https://{your-resource-name}.openai.azure.com/openai/deployments/{deploymen | Parameter | Type | Required? | Default | Description | |--|--|--|--|--|-| ```file```| file | Yes | N/A | The audio file object (not file name) to transcribe, in one of these formats: `flac`, `mp3`, `mp4`, `mpeg`, `mpga`, `m4a`, `ogg`, `wav`, or `webm`.<br/><br/>The file size limit for the Azure OpenAI Whisper model is 25 MB. If you need to transcribe a file larger than 25 MB, break it into chunks. Alternatively you can use the Azure AI Speech [batch transcription](../speech-service/batch-transcription-create.md#use-a-whisper-model) API.<br/><br/>You can get sample audio files from the [Azure AI Speech SDK repository at GitHub](https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/sampledata/audiofiles). | +| ```file```| file | Yes | N/A | The audio file object (not file name) to transcribe, in one of these formats: `flac`, `mp3`, `mp4`, `mpeg`, `mpga`, `m4a`, `ogg`, `wav`, or `webm`.<br/><br/>The file size limit for the Whisper model in Azure OpenAI Service is 25 MB. If you need to transcribe a file larger than 25 MB, break it into chunks. Alternatively you can use the Azure AI Speech [batch transcription](../speech-service/batch-transcription-create.md#use-a-whisper-model) API.<br/><br/>You can get sample audio files from the [Azure AI Speech SDK repository at GitHub](https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/sampledata/audiofiles). | | ```language``` | string | No | Null | The language of the input audio such as `fr`. Supplying the input language in [ISO-639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) format improves accuracy and latency.<br/><br/>For the list of supported languages, see the [OpenAI documentation](https://platform.openai.com/docs/guides/speech-to-text/supported-languages). | | ```prompt``` | string | No | Null | An optional text to guide the model's style or continue a previous audio segment. The prompt should match the audio language.<br/><br/>For more information about prompts including example use cases, see the [OpenAI documentation](https://platform.openai.com/docs/guides/speech-to-text/supported-languages). | | ```response_format``` | string | No | json | The format of the transcript output, in one of these options: json, text, srt, verbose_json, or vtt.<br/><br/>The default value is *json*. | |
| ai-services | Rest Api Resources | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/reference/rest-api-resources.md | Select a service from the table to learn how it can help you meet your developme | Service documentation | Description | Reference documentation | | : | : | : |-|  [Azure AI Search](../../search/index.yml) | Bring AI-powered cloud search to your mobile and web apps | [Azure AI Search API](/rest/api/searchservice) | -| </br>• [fine-tuning](/rest/api/azureopenai/fine-tuning) | -|  [Bot Service](/composer/) | Create bots and connect them across channels | [Bot Service API](/azure/bot-service/rest-api/bot-framework-rest-connector-api-reference?view=azure-bot-service-4.0&preserve-view=true) | -|  [Content Safety](../../ai-services/content-safety/index.yml) | An AI service that detects unwanted contents | [Content Safety API](https://westus.dev.cognitive.microsoft.com/docs/services/content-safety-service-2023-10-15-preview/operations/TextBlocklists_AddOrUpdateBlocklistItems) | -|  [Custom Vision](../../ai-services/custom-vision-service/index.yml) | Customize image recognition for your business applications. |**Custom Vision APIs**<br>• [prediction](https://westus2.dev.cognitive.microsoft.com/docs/services/Custom_Vision_Prediction_3.1/operations/5eb37d24548b571998fde5f3)<br>• [training](https://westus2.dev.cognitive.microsoft.com/docs/services/Custom_Vision_Training_3.3/operations/5eb0bcc6548b571998fddebd)| -|  [Document Intelligence](../../ai-services/document-intelligence/index.yml) | Turn documents into intelligent data-driven solutions | [Document Intelligence API](/rest/api/aiservices/document-models?view=rest-aiservices-2023-07-31&preserve-view=true) | -|  | -|  [Language](../../ai-services/language-service/index.yml) | Build apps with industry-leading natural language understanding capabilities | [REST API](/rest/api/language/) | -|  | -| | -|  [Video Indexer](/azure/azure-video-indexer) | Extract actionable insights from your videos | [Video Indexer API](/rest/api/videoindexer/accounts?view=rest-videoindexer-2024-01-01&preserve-view=true) | -|  [Vision](../../ai-services/computer-vision/index.yml) | Analyze content in images and videos | [Vision API](https://eastus.dev.cognitive.microsoft.com/docs/services/Cognitive_Services_Unified_Vision_API_2024-02-01/operations/61d65934cd35050c20f73ab6) | +|  [Azure AI Search](../../search/index.yml) | Bring AI-powered cloud search to your mobile and web apps | [Azure AI Search API](/rest/api/searchservice) | +| </br>• [fine-tuning](/rest/api/azureopenai/fine-tuning) | +|  [Bot Service](/composer/) | Create bots and connect them across channels | [Bot Service API](/azure/bot-service/rest-api/bot-framework-rest-connector-api-reference?view=azure-bot-service-4.0&preserve-view=true) | +|  [Content Safety](../content-safety/index.yml) | An AI service that detects unwanted contents | [Content Safety API](https://westus.dev.cognitive.microsoft.com/docs/services/content-safety-service-2023-10-15-preview/operations/TextBlocklists_AddOrUpdateBlocklistItems) | +|  [Custom Vision](../custom-vision-service/index.yml) | Customize image recognition for your business applications. |**Custom Vision APIs**<br>• [prediction](https://westus2.dev.cognitive.microsoft.com/docs/services/Custom_Vision_Prediction_3.1/operations/5eb37d24548b571998fde5f3)<br>• [training](https://westus2.dev.cognitive.microsoft.com/docs/services/Custom_Vision_Training_3.3/operations/5eb0bcc6548b571998fddebd)| +|  [Document Intelligence](../document-intelligence/index.yml) | Turn documents into intelligent data-driven solutions | [Document Intelligence API](/rest/api/aiservices/document-models?view=rest-aiservices-2023-07-31&preserve-view=true) | +|  | +|  [Language](../language-service/index.yml) | Build apps with industry-leading natural language understanding capabilities | [REST API](/rest/api/language/) | +|  | +| | +|  [Video Indexer](/azure/azure-video-indexer) | Extract actionable insights from your videos | [Video Indexer API](/rest/api/videoindexer/accounts?view=rest-videoindexer-2024-01-01&preserve-view=true) | +|  [Vision](../computer-vision/index.yml) | Analyze content in images and videos | [Vision API](https://eastus.dev.cognitive.microsoft.com/docs/services/Cognitive_Services_Unified_Vision_API_2024-02-01/operations/61d65934cd35050c20f73ab6) | ## Deprecated services | Service documentation | Description | Reference documentation | | | | |-|  [Anomaly Detector](../../ai-services/Anomaly-Detector/index.yml) <br>(deprecated 2023) | Identify potential problems early on | [Anomaly Detector API](https://westus2.dev.cognitive.microsoft.com/docs/services/AnomalyDetector-v1-1/operations/CreateMultivariateModel) | -|  | -|  [Language understanding (LUIS)](../../ai-services/luis/index.yml) <br>(deprecated 2023) | Understand natural language in your apps | [LUIS API](https://westus.dev.cognitive.microsoft.com/docs/services/luis-endpoint-api-v3-0/operations/5cb0a9459a1fe8fa44c28dd8) | -|  [Metrics Advisor](../../ai-services/metrics-advisor/index.yml) <br>(deprecated 2023) | An AI service that detects unwanted contents | [Metrics Advisor API](https://westus.dev.cognitive.microsoft.com/docs/services/MetricsAdvisor/operations/createDataFeed) | -|  [Personalizer](../../ai-services/personalizer/index.yml) <br>(deprecated 2023) | Create rich, personalized experiences for each user | [Personalizer API](https://westus2.dev.cognitive.microsoft.com/docs/services/personalizer-api/operations/Rank) | -|  [QnA maker](../../ai-services/qnamaker/index.yml) <br>(deprecated 2022) | Distill information into easy-to-navigate questions and answers | [QnA Maker API](https://westus.dev.cognitive.microsoft.com/docs/services/5a93fcf85b4ccd136866eb37/operations/5ac266295b4ccd1554da75ff) | +|  [Anomaly Detector](../Anomaly-Detector/index.yml) <br>(deprecated 2023) | Identify potential problems early on | [Anomaly Detector API](https://westus2.dev.cognitive.microsoft.com/docs/services/AnomalyDetector-v1-1/operations/CreateMultivariateModel) | +|  | +|  [Language understanding (LUIS)](../luis/index.yml) <br>(deprecated 2023) | Understand natural language in your apps | [LUIS API](https://westus.dev.cognitive.microsoft.com/docs/services/luis-endpoint-api-v3-0/operations/5cb0a9459a1fe8fa44c28dd8) | +|  [Metrics Advisor](../metrics-advisor/index.yml) <br>(deprecated 2023) | An AI service that detects unwanted contents | [Metrics Advisor API](https://westus.dev.cognitive.microsoft.com/docs/services/MetricsAdvisor/operations/createDataFeed) | +|  [Personalizer](../personalizer/index.yml) <br>(deprecated 2023) | Create rich, personalized experiences for each user | [Personalizer API](https://westus2.dev.cognitive.microsoft.com/docs/services/personalizer-api/operations/Rank) | +|  [QnA maker](../qnamaker/index.yml) <br>(deprecated 2022) | Distill information into easy-to-navigate questions and answers | [QnA Maker API](https://westus.dev.cognitive.microsoft.com/docs/services/5a93fcf85b4ccd136866eb37/operations/5ac266295b4ccd1554da75ff) | ## Next steps |
| ai-services | Batch Transcription Create | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/speech-service/batch-transcription-create.md | To use a Whisper model for batch transcription, you need to set the `model` prop > [!IMPORTANT] > For Whisper models, you should always use [version 3.2](./migrate-v3-1-to-v3-2.md) of the speech to text API. -Whisper models by batch transcription are supported in the East US, Southeast Asia, and West Europe regions. +Whisper models by batch transcription are supported in the Australia East, Central US, East US, North Central US, South Central US, Southeast Asia, and West Europe regions. ::: zone pivot="rest-api" You can make a [Models_ListBaseModels](https://westus.dev.cognitive.microsoft.com/docs/services/speech-to-text-api-v3-2-preview2/operations/Models_ListBaseModels) request to get available base models for all locales. |
| ai-services | Whisper Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-services/speech-service/whisper-overview.md | Whisper Model via Azure AI Speech might be best for: - Customization of the Whisper base model to improve accuracy for your scenario (coming soon) Regional support is another consideration. -- The Whisper model via Azure OpenAI Service is available in the following regions: North Central US and West Europe. -- The Whisper model via Azure AI Speech is available in the following regions: East US, Southeast Asia, and West Europe.+- The Whisper model via Azure OpenAI Service is available in the following regions: EastUS 2, India South, North Central, Norway East, Sweden Central, and West Europe. +- The Whisper model via Azure AI Speech is available in the following regions: Australia East, Central US, East US, North Central US, South Central US, Southeast Asia, and West Europe. ## Next steps |
| ai-studio | Develop In Vscode | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-studio/how-to/develop-in-vscode.md | |
| ai-studio | Generate Data Qa | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-studio/how-to/generate-data-qa.md | In this article, you learn how to get question and answer pairs from your source ## Install the Synthetics Package ```shell-python --version # ensure you've >=3.8 +python --version # use version 3.8 or later pip3 install azure-identity azure-ai-generative pip3 install wikipedia langchain nltk unstructured ``` |
| ai-studio | Index Add | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-studio/how-to/index-add.md | You must have: - An Azure AI project - An Azure AI Search resource -## Create an index +## Create an index from the Indexes tab 1. Sign in to [Azure AI Studio](https://ai.azure.com). 1. Go to your project or [create a new project](../how-to/create-projects.md) in Azure AI Studio. You must have: :::image type="content" source="../media/index-retrieve/project-left-menu.png" alt-text="Screenshot of Project Left Menu." lightbox="../media/index-retrieve/project-left-menu.png"::: 1. Select **+ New index**-1. Choose your **Source data**. You can choose source data from a list of your recent data sources, a storage URL on the cloud or even upload files and folders from the local machine. You can also add a connection to another data source such as Azure Blob Storage. +1. Choose your **Source data**. You can choose source data from a list of your recent data sources, a storage URL on the cloud, or upload files and folders from the local machine. You can also add a connection to another data source such as Azure Blob Storage. :::image type="content" source="../media/index-retrieve/select-source-data.png" alt-text="Screenshot of select source data." lightbox="../media/index-retrieve/select-source-data.png"::: You must have: 1. Select **Next** after choosing index storage 1. Configure your **Search Settings**- 1. The search type defaults to **Hybrid + Semantic**, which is a combination of keyword search, vector search and semantic search to give the best possible search results. - 1. For the hybrid option to work, you need an embedding model. Choose the Azure OpenAI resource, which has the embedding model + 1. The ***Vector settings*** defaults to true for Add vector search to this search resource. As noted, this enables Hybrid and Hybrid + Semantic search options. Disabling this limits vector search options to Keyword and Semantic. + 1. For the hybrid option to work, you need an embedding model. Choose an embedding model from the dropdown. 1. Select the acknowledgment to deploy an embedding model if it doesn't already exist in your resource- + :::image type="content" source="../media/index-retrieve/search-settings.png" alt-text="Screenshot of configure search settings." lightbox="../media/index-retrieve/search-settings.png":::+ + If a non-Azure OpenAI model isn't appearing in the dropdown follow these steps: + 1. Navigate to the Project settings in [Azure AI Studio](https://ai.azure.com). + 1. Navigate to connections section in the settings tab and select New connection. + 1. Select **Serverless Model**. + 1. Type in the name of your embedding model deployment and select Add connection. If the model doesn't appear in the dropdown, select the **Enter manually** option. + 1. Enter the deployment API endpoint, model name, and API key in the corresponding fields. Then add connection. + 1. The embedding model should now appear in the dropdown. + + :::image type="content" source="../media/index-retrieve/serverless-connection.png" alt-text="Screenshot of connect a serverless model." lightbox="../media/index-retrieve/serverless-connection.png"::: -1. Use the prefilled name or type your own name for New Vector index name 1. Select **Next** after configuring search settings 1. In the **Index settings** 1. Enter a name for your index or use the autopopulated name+ 1. Schedule updates. You can choose to update the index hourly or daily. 1. Choose the compute where you want to run the jobs to create the index. You can - Auto select to allow Azure AI to choose an appropriate VM size that is available - Choose a VM size from a list of recommended options You must have: 1. Select **Next** after configuring index settings 1. Review the details you entered and select **Create**- - > [!NOTE] - > If you see a **DeploymentNotFound** error, you need to assign more permissions. See [mitigate DeploymentNotFound error](#mitigate-deploymentnotfound-error) for more details. - 1. You're taken to the index details page where you can see the status of your index creation. +## Create an index from the Playground +1. Open your AI Studio project. +1. Navigate to the Playground tab. +1. The Select available project index is displayed for existing indexes in the project. If an existing index isn't being used, continue to the next steps. +1. Select the Add your data dropdown. + + :::image type="content" source="../media/index-retrieve/add-data-dropdown.png" alt-text="Screenshot of the playground add your data dropdown." lightbox="../media/index-retrieve/add-data-dropdown.png"::: -### Mitigate DeploymentNotFound error --When you try to create a vector index, you might see the following error at the **Review + Finish** step: --**Failed to create vector index. DeploymentNotFound: A valid deployment for the model=text-embedding-ada-002 was not found in the workspace connection=Default_AzureOpenAI provided.** --This can happen if you are trying to create an index using an **Owner**, **Contributor**, or **Azure AI Developer** role at the project level. To mitigate this error, you might need to assign more permissions using either of the following methods. --> [!NOTE] -> You need to be assigned the **Owner** role of the resource group or higher scope (like Subscription) to perform the operation in the next steps. This is because only the Owner role can assign roles to others. See details [here](/azure/role-based-access-control/built-in-roles). --#### Method 1: Assign more permissions to the user on the Azure AI hub resource --If the Azure AI hub resource the project uses was created through Azure AI Studio: -1. Sign in to [Azure AI Studio](https://aka.ms/azureaistudio) and select your project via **Build** > **Projects**. -1. Select **AI project settings** from the collapsible left menu. -1. From the **Resource Configuration** section, select the link for your resource group name that takes you to the Azure portal. -1. In the Azure portal under **Overview** > **Resources** select the Azure AI service type. It's named similar to "YourAzureAIResourceName-aiservices." -- :::image type="content" source="../media/roles-access/resource-group-azure-ai-service.png" alt-text="Screenshot of Azure AI service in a resource group." lightbox="../media/roles-access/resource-group-azure-ai-service.png"::: --1. Select **Access control (IAM)** > **+ Add** to add a role assignment. -1. Add the **Cognitive Services OpenAI User** role to the user who wants to make an index. `Cognitive Services OpenAI Contributor` and `Cognitive Services Contributor` also work, but they assign more permissions than needed for creating an index in Azure AI Studio. --> [!NOTE] -> You can also opt to assign more permissions [on the resource group](#method-2-assign-more-permissions-on-the-resource-group). However, that method assigns more permissions than needed to mitigate the **DeploymentNotFound** error. --#### Method 2: Assign more permissions on the resource group +1. If a new index is being created, select the ***Add your data*** option. Then follow the steps from ***Create an index from the Indexes tab*** to navigate through the wizard to create an index. + 1. If there's an external index that is being used, select the ***Connect external index*** option. + 1. In the **Index Source** + 1. Select your data source + 1. Select your AI Search Service + 1. Select the index to be used. -If the Azure AI hub resource the project uses was created through Azure portal: -1. Sign in to [Azure AI Studio](https://aka.ms/azureaistudio) and select your project via **Build** > **Projects**. -1. Select **AI project settings** from the collapsible left menu. -1. From the **Resource Configuration** section, select the link for your resource group name that takes you to the Azure portal. -1. Select **Access control (IAM)** > **+ Add** to add a role assignment. -1. Add the **Cognitive Services OpenAI User** role to the user who wants to make an index. `Cognitive Services OpenAI Contributor` and `Cognitive Services Contributor` also work, but they assign more permissions than needed for creating an index in Azure AI Studio. + :::image type="content" source="../media/index-retrieve/connect-external-index.png" alt-text="Screenshot of the page where you select an index." lightbox="../media/index-retrieve/connect-external-index.png"::: + + 1. Select **Next** after configuring search settings. + 1. In the **Index settings** + 1. Enter a name for your index or use the autopopulated name + 1. Schedule updates. You can choose to update the index hourly or daily. + 1. Choose the compute where you want to run the jobs to create the index. You can + - Auto select to allow Azure AI to choose an appropriate VM size that is available + - Choose a VM size from a list of recommended options + - Choose a VM size from a list of all possible options + 1. Review the details you entered and select **Create.** + 1. The index is now ready to be used in the Playground. ## Use an index in prompt flow If the Azure AI hub resource the project uses was created through Azure portal: 1. Provide a name for your Index Lookup Tool and select **Add**. 1. Select the **mlindex_content** value box, and select your index. After completing this step, enter the queries and **query_types** to be performed against the index. - :::image type="content" source="../media/index-retrieve/configure-index-lookup-tool.png" alt-text="Screenshot of Configure Index Lookup." lightbox="../media/index-retrieve/configure-index-lookup-tool.png"::: + :::image type="content" source="../media/index-retrieve/configure-index-lookup-tool.png" alt-text="Screenshot of the prompt flow node to configure index lookup." lightbox="../media/index-retrieve/configure-index-lookup-tool.png"::: + ## Next steps -- [Learn more about RAG](../concepts/retrieval-augmented-generation.md)+- [Learn more about RAG](../concepts/retrieval-augmented-generation.md) |
| ai-studio | Llm Tool | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-studio/how-to/prompt-flow-tools/llm-tool.md | Title: LLM tool for flows in Azure AI Studio -description: This article introduces the LLM tool for flows in Azure AI Studio. +description: This article introduces you to the large language model (LLM) tool for flows in Azure AI Studio. -The prompt flow *LLM* tool enables you to use large language models (LLM) for natural language processing. +To use large language models (LLMs) for natural language processing, you use the prompt flow LLM tool. > [!NOTE] > For embeddings to convert text into dense vector representations for various natural language processing tasks, see [Embedding tool](embedding-tool.md). ## Prerequisites -Prepare a prompt as described in the [prompt tool](prompt-tool.md#prerequisites) documentation. The LLM tool and Prompt tool both support [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) templates. For more information and best practices, see [prompt engineering techniques](../../../ai-services/openai/concepts/advanced-prompt-engineering.md). +Prepare a prompt as described in the [Prompt tool](prompt-tool.md#prerequisites) documentation. The LLM tool and Prompt tool both support [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) templates. For more information and best practices, see [Prompt engineering techniques](../../../ai-services/openai/concepts/advanced-prompt-engineering.md). ## Build with the LLM tool 1. Create or open a flow in [Azure AI Studio](https://ai.azure.com). For more information, see [Create a flow](../flow-develop.md). 1. Select **+ LLM** to add the LLM tool to your flow. - :::image type="content" source="../../media/prompt-flow/llm-tool.png" alt-text="Screenshot of the LLM tool added to a flow in Azure AI Studio." lightbox="../../media/prompt-flow/llm-tool.png"::: + :::image type="content" source="../../media/prompt-flow/llm-tool.png" alt-text="Screenshot that shows the LLM tool added to a flow in Azure AI Studio." lightbox="../../media/prompt-flow/llm-tool.png"::: 1. Select the connection to one of your provisioned resources. For example, select **Default_AzureOpenAI**.-1. From the **Api** drop-down list, select *chat* or *completion*. -1. Enter values for the LLM tool input parameters described [here](#inputs). If you selected the *chat* API, see [chat inputs](#chat-inputs). If you selected the *completion* API, see [text completion inputs](#text-completion-inputs). For information about how to prepare the prompt input, see [prerequisites](#prerequisites). -1. Add more tools to your flow as needed, or select **Run** to run the flow. -1. The outputs are described [here](#outputs). -+1. From the **Api** dropdown list, select **chat** or **completion**. +1. Enter values for the LLM tool input parameters described in the [Text completion inputs table](#inputs). If you selected the **chat** API, see the [Chat inputs table](#chat-inputs). If you selected the **completion** API, see the [Text completion inputs table](#text-completion-inputs). For information about how to prepare the prompt input, see [Prerequisites](#prerequisites). +1. Add more tools to your flow, as needed. Or select **Run** to run the flow. +1. The outputs are described in the [Outputs table](#outputs). ## Inputs -The following are available input parameters: +The following input parameters are available. ### Text completion inputs | Name | Type | Description | Required | ||-|--|-|-| prompt | string | text prompt for the language model | Yes | -| model, deployment_name | string | the language model to use | Yes | -| max\_tokens | integer | the maximum number of tokens to generate in the completion. Default is 16. | No | -| temperature | float | the randomness of the generated text. Default is 1. | No | -| stop | list | the stopping sequence for the generated text. Default is null. | No | -| suffix | string | text appended to the end of the completion | No | -| top_p | float | the probability of using the top choice from the generated tokens. Default is 1. | No | -| logprobs | integer | the number of log probabilities to generate. Default is null. | No | -| echo | boolean | value that indicates whether to echo back the prompt in the response. Default is false. | No | -| presence\_penalty | float | value that controls the model's behavior regarding repeating phrases. Default is 0. | No | -| frequency\_penalty | float | value that controls the model's behavior regarding generating rare phrases. Default is 0. | No | -| best\_of | integer | the number of best completions to generate. Default is 1. | No | -| logit\_bias | dictionary | the logit bias for the language model. Default is empty dictionary. | No | -+| prompt | string | Text prompt for the language model. | Yes | +| model, deployment_name | string | The language model to use. | Yes | +| max\_tokens | integer | The maximum number of tokens to generate in the completion. Default is 16. | No | +| temperature | float | The randomness of the generated text. Default is 1. | No | +| stop | list | The stopping sequence for the generated text. Default is null. | No | +| suffix | string | The text appended to the end of the completion. | No | +| top_p | float | The probability of using the top choice from the generated tokens. Default is 1. | No | +| logprobs | integer | The number of log probabilities to generate. Default is null. | No | +| echo | boolean | The value that indicates whether to echo back the prompt in the response. Default is false. | No | +| presence\_penalty | float | The value that controls the model's behavior regarding repeating phrases. Default is 0. | No | +| frequency\_penalty | float | The value that controls the model's behavior regarding generating rare phrases. Default is 0. | No | +| best\_of | integer | The number of best completions to generate. Default is 1. | No | +| logit\_bias | dictionary | The logit bias for the language model. Default is empty dictionary. | No | ### Chat inputs | Name | Type | Description | Required | ||-||-|-| prompt | string | text prompt that the language model should reply to | Yes | -| model, deployment_name | string | the language model to use | Yes | -| max\_tokens | integer | the maximum number of tokens to generate in the response. Default is inf. | No | -| temperature | float | the randomness of the generated text. Default is 1. | No | -| stop | list | the stopping sequence for the generated text. Default is null. | No | -| top_p | float | the probability of using the top choice from the generated tokens. Default is 1. | No | -| presence\_penalty | float | value that controls the model's behavior regarding repeating phrases. Default is 0. | No | -| frequency\_penalty | float | value that controls the model's behavior regarding generating rare phrases. Default is 0. | No | -| logit\_bias | dictionary | the logit bias for the language model. Default is empty dictionary. | No | +| prompt | string | The text prompt that the language model should reply to. | Yes | +| model, deployment_name | string | The language model to use. | Yes | +| max\_tokens | integer | The maximum number of tokens to generate in the response. Default is inf. | No | +| temperature | float | The randomness of the generated text. Default is 1. | No | +| stop | list | The stopping sequence for the generated text. Default is null. | No | +| top_p | float | The probability of using the top choice from the generated tokens. Default is 1. | No | +| presence\_penalty | float | The value that controls the model's behavior regarding repeating phrases. Default is 0. | No | +| frequency\_penalty | float | The value that controls the model's behavior regarding generating rare phrases. Default is 0. | No | +| logit\_bias | dictionary | The logit bias for the language model. Default is empty dictionary. | No | ## Outputs The output varies depending on the API you selected for inputs. -| API | Return Type | Description | +| API | Return type | Description | ||-||-| Completion | string | The text of one predicted completion | -| Chat | string | The text of one response of conversation | +| Completion | string | The text of one predicted completion. | +| Chat | string | The text of one response of conversation. | ## Next steps |
| ai-studio | Prompt Flow Tools Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-studio/how-to/prompt-flow-tools/prompt-flow-tools-overview.md | -The following table provides an index of tools in prompt flow. +The following table provides an index of tools in prompt flow. -| Tool (set) name | Description | Environment | Package name | +| Tool or tool set name | Description | Environment | Package name | ||--|-|--|-| [LLM](./llm-tool.md) | Use Azure OpenAI large language models (LLM) for tasks such as text completion or chat. | Default | [promptflow-tools](https://pypi.org/project/promptflow-tools/) | +| [Large language models](./llm-tool.md) | Use Azure OpenAI large language models for tasks such as text completion or chat. | Default | [promptflow-tools](https://pypi.org/project/promptflow-tools/) | | [Prompt](./prompt-tool.md) | Craft a prompt by using Jinja as the templating language. | Default | [promptflow-tools](https://pypi.org/project/promptflow-tools/) | | [Python](./python-tool.md) | Run Python code. | Default | [promptflow-tools](https://pypi.org/project/promptflow-tools/) | | [Azure OpenAI GPT-4 Turbo with Vision](./azure-open-ai-gpt-4v-tool.md) | Use AzureOpenAI GPT-4 Turbo with Vision model deployment to analyze images and provide textual responses to questions about them. | Default | [promptflow-tools](https://pypi.org/project/promptflow-tools/) | | [Content Safety (Text)](./content-safety-tool.md) | Use Azure AI Content Safety to detect harmful content. | Default | [promptflow-tools](https://pypi.org/project/promptflow-tools/) |-| [Index Lookup*](./index-lookup-tool.md) | Search an Azure Machine Learning Vector Index for relevant results using one or more text queries. | Default | [promptflow-vectordb](https://pypi.org/project/promptflow-vectordb/) | +| [Index Lookup*](./index-lookup-tool.md) | Search an Azure Machine Learning Vector Index for relevant results by using one or more text queries. | Default | [promptflow-vectordb](https://pypi.org/project/promptflow-vectordb/) | | [Vector Index Lookup*](./vector-index-lookup-tool.md) | Search text or a vector-based query from a vector index. | Default | [promptflow-vectordb](https://pypi.org/project/promptflow-vectordb/) | | [Faiss Index Lookup*](./faiss-index-lookup-tool.md) | Search a vector-based query from the Faiss index file. | Default | [promptflow-vectordb](https://pypi.org/project/promptflow-vectordb/) | | [Vector DB Lookup*](./vector-db-lookup-tool.md) | Search a vector-based query from an existing vector database. | Default | [promptflow-vectordb](https://pypi.org/project/promptflow-vectordb/) | | [Embedding](./embedding-tool.md) | Use Azure OpenAI embedding models to create an embedding vector that represents the input text. | Default | [promptflow-tools](https://pypi.org/project/promptflow-tools/) | | [Serp API](./serp-api-tool.md) | Use Serp API to obtain search results from a specific search engine. | Default | [promptflow-tools](https://pypi.org/project/promptflow-tools/) |-| [Azure AI Language tools*](https://microsoft.github.io/promptflow/integrations/tools/azure-ai-language-tool.html) | This collection of tools is a wrapper for various Azure AI Language APIs, which can help effectively understand and analyze documents and conversations. The capabilities currently supported include: Abstractive Summarization, Extractive Summarization, Conversation Summarization, Entity Recognition, Key Phrase Extraction, Language Detection, PII Entity Recognition, Conversational PII, Sentiment Analysis, Conversational Language Understanding, Translator. You can learn how to use them by the [Sample flows](https://github.com/microsoft/promptflow/tree/e4542f6ff5d223d9800a3687a7cfd62531a9607c/examples/flows/integrations/azure-ai-language). Support contact: taincidents@microsoft.com | Custom | [promptflow-azure-ai-language](https://pypi.org/project/promptflow-azure-ai-language/) | +| [Azure AI Language tools*](https://microsoft.github.io/promptflow/integrations/tools/azure-ai-language-tool.html) | This collection of tools is a wrapper for various Azure AI Language APIs, which can help effectively understand and analyze documents and conversations. The capabilities currently supported include Abstractive Summarization, Extractive Summarization, Conversation Summarization, Entity Recognition, Key Phrase Extraction, Language Detection, PII Entity Recognition, Conversational PII, Sentiment Analysis, Conversational Language Understanding, and Translator. You can learn how to use them by the [Sample flows](https://github.com/microsoft/promptflow/tree/e4542f6ff5d223d9800a3687a7cfd62531a9607c/examples/flows/integrations/azure-ai-language). Support contact: taincidents@microsoft.com | Custom | [promptflow-azure-ai-language](https://pypi.org/project/promptflow-azure-ai-language/) | -_*The asterisk marks indicate custom tools, which are created by the community that extend prompt flow's capabilities for specific use cases. They aren't officially maintained or endorsed by prompt flow team. When you encounter questions or issues for these tools, please prioritize using the support contact if it is provided in the description._ +*The asterisks indicate custom tools that the community created. Custom tools extend prompt flow's capabilities for specific use cases. The prompt flow team doesn't officially maintain or endorse these tools. When you encounter questions or issues for these tools, prioritize by using the support contact if the description provides it. To discover more custom tools developed by the open-source community, see [More custom tools](https://microsoft.github.io/promptflow/integrations/tools/https://docsupdatetracker.net/index.html). ## Remarks+ - If existing tools don't meet your requirements, you can [develop your own custom tool and make a tool package](https://microsoft.github.io/promptflow/how-to-guides/develop-a-tool/create-and-use-tool-package.html).-- To install the custom tools, if you're using the automatic runtime, you can readily install the publicly released package by adding the custom tool package name into the `requirements.txt` file in the flow folder. Then select the **Save and install** button to start installation. After completion, you can see the custom tools displayed in the tool list. In addition, if you want to use local or private feed package, please build an image first, then set up the runtime based on your image. To learn more, see [How to create and manage a runtime](../create-manage-runtime.md).+- To install the custom tools, if you're using the automatic runtime, you can readily install the publicly released package by adding the custom tool package name in the `requirements.txt` file in the flow folder. Then select **Save and install** to start installation. After completion, the custom tools appear in the tool list. If you want to use a local or private feed package, build an image first, and then set up the runtime based on your image. To learn more, see [How to create and manage a runtime](../create-manage-runtime.md). ++ :::image type="content" source="../../media/prompt-flow/install-package-on-automatic-runtime.png" alt-text="Screenshot that shows how to install packages on automatic runtime."lightbox = "../../media/prompt-flow/install-package-on-automatic-runtime.png"::: ## Next steps - [Create a flow](../flow-develop.md)-- [Build your own copilot using prompt flow](../../tutorials/deploy-copilot-ai-studio.md)+- [Build your own copilot by using prompt flow](../../tutorials/deploy-copilot-ai-studio.md) |
| ai-studio | Prompt Tool | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-studio/how-to/prompt-flow-tools/prompt-tool.md | Title: Prompt tool for flows in Azure AI Studio -description: This article introduces the Prompt tool for flows in Azure AI Studio. +description: This article introduces you to the Prompt tool for flows in Azure AI Studio. -The prompt flow *Prompt* tool offers a collection of textual templates that serve as a starting point for creating prompts. These templates, based on the [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) template engine, facilitate the definition of prompts. The tool proves useful when prompt tuning is required prior to feeding the prompts into the large language model (LLM) in prompt flow. +The prompt flow Prompt tool offers a collection of textual templates that serve as a starting point for creating prompts. These templates, based on the [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) template engine, facilitate the definition of prompts. The tool proves useful when prompt tuning is required before the prompts are fed into the large language model (LLM) in the prompt flow. ## Prerequisites -Prepare a prompt. The [LLM tool](llm-tool.md) and Prompt tool both support [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) templates. +Prepare a prompt. The [LLM tool](llm-tool.md) and Prompt tool both support [Jinja](https://jinja.palletsprojects.com/en/3.1.x/) templates. -In this example, the prompt incorporates Jinja templating syntax to dynamically generate the welcome message and personalize it based on the user's name. It also presents a menu of options for the user to choose from. Depending on whether the user_name variable is provided, it either addresses the user by name or uses a generic greeting. +In this example, the prompt incorporates Jinja templating syntax to dynamically generate the welcome message and personalize it based on the user's name. It also presents a menu of options for the user to choose from. Depending on whether the `user_name` variable is provided, it either addresses the user by name or uses a generic greeting. ```jinja Welcome to {{ website_name }}! Please select an option from the menu below: 4. Contact customer support ``` -For more information and best practices, see [prompt engineering techniques](../../../ai-services/openai/concepts/advanced-prompt-engineering.md). +For more information and best practices, see [Prompt engineering techniques](../../../ai-services/openai/concepts/advanced-prompt-engineering.md). ## Build with the Prompt tool 1. Create or open a flow in [Azure AI Studio](https://ai.azure.com). For more information, see [Create a flow](../flow-develop.md). 1. Select **+ Prompt** to add the Prompt tool to your flow. - :::image type="content" source="../../media/prompt-flow/prompt-tool.png" alt-text="Screenshot of the Prompt tool added to a flow in Azure AI Studio." lightbox="../../media/prompt-flow/prompt-tool.png"::: --1. Enter values for the Prompt tool input parameters described [here](#inputs). For information about how to prepare the prompt input, see [prerequisites](#prerequisites). -1. Add more tools (such as the [LLM tool](llm-tool.md)) to your flow as needed, or select **Run** to run the flow. -1. The outputs are described [here](#outputs). + :::image type="content" source="../../media/prompt-flow/prompt-tool.png" alt-text="Screenshot that shows the Prompt tool added to a flow in Azure AI Studio." lightbox="../../media/prompt-flow/prompt-tool.png"::: +1. Enter values for the Prompt tool input parameters described in the [Inputs table](#inputs). For information about how to prepare the prompt input, see [Prerequisites](#prerequisites). +1. Add more tools (such as the [LLM tool](llm-tool.md)) to your flow, as needed. Or select **Run** to run the flow. +1. The outputs are described in the [Outputs table](#outputs). ## Inputs -The following are available input parameters: +The following input parameters are available. | Name | Type | Description | Required | |--|--|-|-|-| prompt | string | The prompt template in Jinja | Yes | -| Inputs | - | List of variables of prompt template and its assignments | - | +| prompt | string | The prompt template in Jinja. | Yes | +| Inputs | - | The list of variables of a prompt template and its assignments. | - | ## Outputs ### Example 1 -Inputs +Inputs: -| Variable | Type | Sample Value | +| Variable | Type | Sample value | ||--|--| | website_name | string | "Microsoft" | | user_name | string | "Jane" | -Outputs +Outputs: ``` Welcome to Microsoft! Hello, Jane! Please select an option from the menu below: 1. View your account 2. Update personal information 3. Browse available products 4. Contact customer support Welcome to Microsoft! Hello, Jane! Please select an option from the menu below: ### Example 2 -Inputs +Inputs: -| Variable | Type | Sample Value | +| Variable | Type | Sample value | |--|--|-| | website_name | string | "Bing" | | user_name | string | " | -Outputs +Outputs: ``` Welcome to Bing! Hello there! Please select an option from the menu below: 1. View your account 2. Update personal information 3. Browse available products 4. Contact customer support |
| ai-studio | Python Tool | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-studio/how-to/prompt-flow-tools/python-tool.md | Title: Python tool for flows in Azure AI Studio -description: This article introduces the Python tool for flows in Azure AI Studio. +description: This article introduces you to the Python tool for flows in Azure AI Studio. -The prompt flow *Python* tool offers customized code snippets as self-contained executable nodes. You can quickly create Python tools, edit code, and verify results. +The prompt flow Python tool offers customized code snippets as self-contained executable nodes. You can quickly create Python tools, edit code, and verify results. ## Build with the Python tool 1. Create or open a flow in [Azure AI Studio](https://ai.azure.com). For more information, see [Create a flow](../flow-develop.md). 1. Select **+ Python** to add the Python tool to your flow. - :::image type="content" source="../../media/prompt-flow/python-tool.png" alt-text="Screenshot of the Python tool added to a flow in Azure AI Studio." lightbox="../../media/prompt-flow/python-tool.png"::: + :::image type="content" source="../../media/prompt-flow/python-tool.png" alt-text="Screenshot that shows the Python tool added to a flow in Azure AI Studio." lightbox="../../media/prompt-flow/python-tool.png"::: -1. Enter values for the Python tool input parameters described [here](#inputs). For example, in the **Code** input text box you can enter the following Python code: +1. Enter values for the Python tool input parameters that are described in the [Inputs table](#inputs). For example, in the **Code** input text box, you can enter the following Python code: ```python from promptflow import tool The prompt flow *Python* tool offers customized code snippets as self-contained For more information, see [Python code input requirements](#python-code-input-requirements). -1. Add more tools to your flow as needed, or select **Run** to run the flow. -1. The outputs are described [here](#outputs). Given the previous example Python code input, if the input message is "world", the output is `hello world`. -+1. Add more tools to your flow, as needed. Or select **Run** to run the flow. +1. The outputs are described in the [Outputs table](#outputs). Based on the previous example Python code input, if the input message is "world," the output is `hello world`. ## Inputs -The list of inputs will change based on the arguments of the tool function, after you save the code. Adding type to arguments and return values help the tool show the types properly. +The list of inputs change based on the arguments of the tool function, after you save the code. Adding type to arguments and `return` values helps the tool show the types properly. | Name | Type | Description | Required | |--|--|||-| Code | string | Python code snippet | Yes | -| Inputs | - | List of tool function parameters and its assignments | - | -+| Code | string | The Python code snippet. | Yes | +| Inputs | - | The list of the tool function parameters and its assignments. | - | ## Outputs -The output is the `return` value of the python tool function. For example, consider the following python tool function: +The output is the `return` value of the Python tool function. For example, consider the following Python tool function: ```python from promptflow import tool def my_python_tool(message: str) -> str: return 'hello ' + message ``` -If the input message is "world", the output is `hello world`. +If the input message is "world," the output is `hello world`. ### Types If the input message is "world", the output is `hello world`. | double | param: float | Double type | | list | param: list or param: List[T] | List type | | object | param: dict or param: Dict[K, V] | Object type |-| Connection | param: CustomConnection | Connection type will be handled specially | +| Connection | param: CustomConnection | Connection type is handled specially. | ++Parameters with `Connection` type annotation are treated as connection inputs, which means: -Parameters with `Connection` type annotation will be treated as connection inputs, which means: -- Prompt flow extension will show a selector to select the connection.-- During execution time, prompt flow will try to find the connection with the name same from parameter value passed in.+- The prompt flow extension shows a selector to select the connection. +- During execution time, the prompt flow tries to find the connection with the same name from the parameter value that was passed in. -> [!Note] -> `Union[...]` type annotation is only supported for connection type, for example, `param: Union[CustomConnection, OpenAIConnection]`. +> [!NOTE] +> The `Union[...]` type annotation is only supported for connection type. An example is `param: Union[CustomConnection, OpenAIConnection]`. ## Python code input requirements This section describes requirements of the Python code input for the Python tool. -- Python Tool Code should consist of a complete Python code, including any necessary module imports.-- Python Tool Code must contain a function decorated with `@tool` (tool function), serving as the entry point for execution. The `@tool` decorator should be applied only once within the snippet.-- Python tool function parameters must be assigned in 'Inputs' section+- Python tool code should consist of a complete Python code, including any necessary module imports. +- Python tool code must contain a function decorated with `@tool` (tool function), serving as the entry point for execution. The `@tool` decorator should be applied only once within the snippet. +- Python tool function parameters must be assigned in the `Inputs` section. - Python tool function shall have a return statement and value, which is the output of the tool. The following Python code is an example of best practices: def my_python_tool(message: str) -> str: return 'hello ' + message ``` -## Consume custom connection in the Python tool +## Consume a custom connection in the Python tool -If you're developing a python tool that requires calling external services with authentication, you can use the custom connection in prompt flow. It allows you to securely store the access key and then retrieve it in your python code. +If you're developing a Python tool that requires calling external services with authentication, you can use the custom connection in a prompt flow. It allows you to securely store the access key and then retrieve it in your Python code. ### Create a custom connection -Create a custom connection that stores all your LLM API KEY or other required credentials. +Create a custom connection that stores all your large language model API key or other required credentials. -1. Go to **AI project settings**, then select **New Connection**. -1. Select **Custom** service. You can define your connection name, and you can add multiple *Key-value pairs* to store your credentials and keys by selecting **Add key-value pairs**. +1. Go to **AI project settings**. Then select **New Connection**. +1. Select **Custom** service. You can define your connection name. You can add multiple key-value pairs to store your credentials and keys by selecting **Add key-value pairs**. > [!NOTE]- > Make sure at least one key-value pair is set as secret, otherwise the connection will not be created successfully. You can set one Key-Value pair as secret by **is secret** checked, which will be encrypted and stored in your key value. -- :::image type="content" source="../../media/prompt-flow/create-connection.png" alt-text="Screenshot that shows create connection in AI Studio." lightbox = "../../media/prompt-flow/create-connection.png"::: + > Make sure at least one key-value pair is set as secret. Otherwise, the connection won't be created successfully. To set one key-value pair as secret, select **is secret** to encrypt and store your key value. + :::image type="content" source="../../media/prompt-flow/create-connection.png" alt-text="Screenshot that shows creating a connection in AI Studio." lightbox = "../../media/prompt-flow/create-connection.png"::: 1. Add the following custom keys to the connection: - `azureml.flow.connection_type`: `Custom` - `azureml.flow.module`: `promptflow.connections` - :::image type="content" source="../../media/prompt-flow/custom-connection-keys.png" alt-text="Screenshot that shows add extra meta to custom connection in AI Studio." lightbox = "../../media/prompt-flow/custom-connection-keys.png"::: -- + :::image type="content" source="../../media/prompt-flow/custom-connection-keys.png" alt-text="Screenshot that shows adding extra information to a custom connection in AI Studio." lightbox = "../../media/prompt-flow/custom-connection-keys.png"::: -### Consume custom connection in Python +### Consume a custom connection in Python -To consume a custom connection in your python code, follow these steps: +To consume a custom connection in your Python code: -1. In the code section in your python node, import custom connection library `from promptflow.connections import CustomConnection`, and define an input parameter of type `CustomConnection` in the tool function. -1. Parse the input to the input section, then select your target custom connection in the value dropdown. +1. In the code section in your Python node, import the custom connection library `from promptflow.connections import CustomConnection`. Define an input parameter of the type `CustomConnection` in the tool function. +1. Parse the input to the input section. Then select your target custom connection in the value dropdown list. For example: def my_python_tool(message: str, myconn: CustomConnection) -> str: connection_key2_value = myconn.key2 ``` - ## Next steps - [Learn more about how to create a flow](../flow-develop.md) |
| ai-studio | Deploy Copilot Ai Studio | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/ai-studio/tutorials/deploy-copilot-ai-studio.md | Now that you have your evaluation dataset, you can evaluate your flow by followi 1. Select a model to use for evaluation. In this example, select **gpt-35-turbo-16k**. Then select **Next**. > [!NOTE]- > Evaluation with AI-assisted metrics needs to call another GPT model to do the calculation. For best performance, use a GPT-4 or gpt-35-turbo-16k model. If you didn't previously deploy a GPT-4 or gpt-35-turbo-16k model, you can deploy another model by following the steps in [Deploy a chat model](#deploy-a-chat-model). Then return to this step and select the model you deployed. - > The evaluation process may take up lots of tokens, so it's recommended to use a model which can support >=16k tokens. + > Evaluation with AI-assisted metrics needs to call another GPT model to do the calculation. For best performance, use a model that supports at least 16k tokens such as gpt-4-32k or gpt-35-turbo-16k model. If you didn't previously deploy such a model, you can deploy another model by following the steps in [Deploy a chat model](#deploy-a-chat-model). Then return to this step and select the model you deployed. 1. Select **Add new dataset**. Then select **Next**. |
| aks | App Routing | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/aks/app-routing.md | With the retirement of [Open Service Mesh][open-service-mesh-docs] (OSM) by the - All global Azure DNS zones integrated with the add-on have to be in the same resource group. - All private Azure DNS zones integrated with the add-on have to be in the same resource group. - Editing the ingress-nginx `ConfigMap` in the `app-routing-system` namespace isn't supported.+- The following snippet annotations are blocked and will prevent an Ingress from being configured: `load_module`, `lua_package`, `_by_lua`, `location`, `root`, `proxy_pass`, `serviceaccount`, `{`, `}`, `'`. ## Enable application routing using Azure CLI |
| aks | Azure Csi Disk Storage Provision | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/aks/azure-csi-disk-storage-provision.md | The following table includes parameters you can use to define a custom storage c |fsType | File System Type | `ext4`, `ext3`, `ext2`, `xfs`, `btrfs` for Linux, `ntfs` for Windows | No | `ext4` for Linux, `ntfs` for Windows| |cachingMode | [Azure Data Disk Host Cache Setting][disk-host-cache-setting] | `None`, `ReadOnly`, `ReadWrite` | No | `ReadOnly`| |resourceGroup | Specify the resource group for the Azure Disks | Existing resource group name | No | If empty, driver uses the same resource group name as current AKS cluster|-|DiskIOPSReadWrite | [UltraSSD disk][ultra-ssd-disks] IOPS Capability (minimum: 2 IOPS/GiB) | 100~160000 | No | `500`| -|DiskMBpsReadWrite | [UltraSSD disk][ultra-ssd-disks] Throughput Capability(minimum: 0.032/GiB) | 1~2000 | No | `100`| +|DiskIOPSReadWrite | [UltraSSD disk][ultra-ssd-disks] or [Premium SSD v2][premiumv2_lrs_disks] IOPS Capability (minimum: 2 IOPS/GiB) | 100~160000 | No | `500`| +|DiskMBpsReadWrite | [UltraSSD disk][ultra-ssd-disks] or [Premium SSD v2][premiumv2_lrs_disks] Throughput Capability(minimum: 0.032/GiB) | 1~2000 | No | `100`| |LogicalSectorSize | Logical sector size in bytes for ultra disk. Supported values are 512 ad 4096. 4096 is the default. | `512`, `4096` | No | `4096`| |tags | Azure Disk [tags][azure-tags] | Tag format: `key1=val1,key2=val2` | No | ""| |diskEncryptionSetID | ResourceId of the disk encryption set to use for [enabling encryption at rest][disk-encryption] | format: `/subscriptions/{subs-id}/resourceGroups/{rg-name}/providers/Microsoft.Compute/diskEncryptionSets/{diskEncryptionSet-name}` | No | ""| kubectl delete -f azure-pvc.yaml [disk-host-cache-setting]: ../virtual-machines/windows/premium-storage-performance.md#disk-caching [use-ultra-disks]: use-ultra-disks.md [ultra-ssd-disks]: ../virtual-machines/linux/disks-ultra-ssd.md+[premiumv2_lrs_disks]: ../virtual-machines/disks-types.md#premium-ssd-v2 [azure-tags]: ../azure-resource-manager/management/tag-resources.md [disk-encryption]: ../virtual-machines/windows/disk-encryption.md [azure-disk-write-accelerator]: ../virtual-machines/windows/how-to-enable-write-accelerator.md |
| aks | Deploy Marketplace | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/aks/deploy-marketplace.md | Kubernetes application-based container offers can't be deployed on AKS for Azure 1. You can search for an offer or publisher directly by name, or you can browse all offers. To find Kubernetes application offers, on the left side under **Categories** select **Containers**. :::image type="content" source="./media/deploy-marketplace/containers-inline.png" alt-text="Screenshot of Azure Marketplace offers in the Azure portal, with the container category on the left side highlighted." lightbox="./media/deploy-marketplace/containers.png":::-+ > [!IMPORTANT]- > The **Containers** category includes both Kubernetes applications and standalone container images. This walkthrough is specific to Kubernetes applications. If you find that the steps to deploy an offer differ in some way, you're most likely trying to deploy a container image-based offer instead of a Kubernetes application-based offer. -+ > The **Containers** category includes Kubernetes applications. This walkthrough is specific to Kubernetes applications. 1. You'll see several Kubernetes application offers displayed on the page. To view all of the Kubernetes application offers, select **See more**. :::image type="content" source="./media/deploy-marketplace/see-more-inline.png" alt-text="Screenshot of Azure Marketplace K8s offers in the Azure portal. 'See More' is highlighted." lightbox="./media/deploy-marketplace/see-more.png"::: If you experience issues, see the [troubleshooting checklist for failed deployme - Learn more about [exploring and analyzing costs][billing]. - Learn more about [deploying a Kubernetes application programmatically using Azure CLI](/azure/aks/deploy-application-az-cli)+ - Learn more about [deploying a Kubernetes application through an ARM template](/azure/aks/deploy-application-template) <!-- LINKS --> |
| aks | Howto Deploy Java Liberty App | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/aks/howto-deploy-java-liberty-app.md | Title: Deploy a Java application with Open Liberty/WebSphere Liberty on an Azure Kubernetes Service (AKS) cluster recommendations: false description: Deploy a Java application with Open Liberty/WebSphere Liberty on an Azure Kubernetes Service (AKS) cluster-+ Previously updated : 01/16/2024 Last updated : 04/02/2024 keywords: java, jakartaee, javaee, microprofile, open-liberty, websphere-liberty, aks, kubernetes The Open Liberty Operator simplifies the deployment and management of applicatio For more information on Open Liberty, see [the Open Liberty project page](https://openliberty.io/). For more information on IBM WebSphere Liberty, see [the WebSphere Liberty product page](https://www.ibm.com/cloud/websphere-liberty). -This article uses the Azure Marketplace offer for Open/WebSphere Liberty to accelerate your journey to AKS. The offer automatically provisions a number of Azure resources including an Azure Container Registry (ACR) instance, an AKS cluster, an Azure App Gateway Ingress Controller (AGIC) instance, the Liberty Operator, and optionally a container image including Liberty and your application. To see the offer, visit the [Azure portal](https://aka.ms/liberty-aks). If you prefer manual step-by-step guidance for running Liberty on AKS that doesn't utilize the automation enabled by the offer, see [Manually deploy a Java application with Open Liberty or WebSphere Liberty on an Azure Kubernetes Service (AKS) cluster](/azure/developer/java/ee/howto-deploy-java-liberty-app-manual). +This article uses the Azure Marketplace offer for Open/WebSphere Liberty to accelerate your journey to AKS. The offer automatically provisions a number of Azure resources including an Azure Container Registry (ACR) instance, an AKS cluster, an Azure App Gateway Ingress Controller (AGIC) instance, the Liberty Operators, and optionally a container image including Liberty and your application. To see the offer, visit the [Azure portal](https://aka.ms/liberty-aks). If you prefer manual step-by-step guidance for running Liberty on AKS that doesn't utilize the automation enabled by the offer, see [Manually deploy a Java application with Open Liberty or WebSphere Liberty on an Azure Kubernetes Service (AKS) cluster](/azure/developer/java/ee/howto-deploy-java-liberty-app-manual). This article is intended to help you quickly get to deployment. Before going to production, you should explore [Tuning Liberty](https://www.ibm.com/docs/was-liberty/base?topic=tuning-liberty). [!INCLUDE [quickstarts-free-trial-note](../../includes/quickstarts-free-trial-note.md)] -* You can use Azure Cloud Shell or a local terminal. +## Prerequisites --* This article requires at least version 2.31.0 of Azure CLI. If using Azure Cloud Shell, the latest version is already installed. +* Install the [Azure CLI](/cli/azure/install-azure-cli). If you're running on Windows or macOS, consider running Azure CLI in a Docker container. For more information, see [How to run the Azure CLI in a Docker container](/cli/azure/run-azure-cli-docker). +* Sign in to the Azure CLI by using the [az login](/cli/azure/reference-index#az-login) command. To finish the authentication process, follow the steps displayed in your terminal. For other sign-in options, see [Sign in with the Azure CLI](/cli/azure/authenticate-azure-cli). +* When you're prompted, install the Azure CLI extension on first use. For more information about extensions, see [Use extensions with the Azure CLI](/cli/azure/azure-cli-extensions-overview). +* Run [az version](/cli/azure/reference-index?#az-version) to find the version and dependent libraries that are installed. To upgrade to the latest version, run [az upgrade](/cli/azure/reference-index?#az-upgrade). This article requires at least version 2.31.0 of Azure CLI. +* Install a Java SE implementation, version 17 or later. (for example, [Eclipse Open J9](https://www.eclipse.org/openj9/)). +* Install [Maven](https://maven.apache.org/download.cgi) 3.5.0 or higher. +* Install [Docker](https://docs.docker.com/get-docker/) for your OS. +* Ensure [Git](https://git-scm.com) is installed. +* Make sure you're assigned either the `Owner` role or the `Contributor` and `User Access Administrator` roles in the subscription. You can verify it by following steps in [List role assignments for a user or group](../role-based-access-control/role-assignments-list-portal.md#list-role-assignments-for-a-user-or-group). > [!NOTE] > You can also execute this guidance from the [Azure Cloud Shell](/azure/cloud-shell/quickstart). This approach has all the prerequisite tools pre-installed, with the exception of Docker. > > :::image type="icon" source="~/reusable-content/ce-skilling/azure/media/cloud-shell/launch-cloud-shell-button.png" alt-text="Button to launch the Azure Cloud Shell." border="false" link="https://shell.azure.com"::: -* If running the commands in this guide locally (instead of Azure Cloud Shell): - * Prepare a local machine with Unix-like operating system installed (for example, Ubuntu, Azure Linux, macOS, Windows Subsystem for Linux). - * Install a Java SE implementation, version 17 or later. (for example, [Eclipse Open J9](https://www.eclipse.org/openj9/)). - * Install [Maven](https://maven.apache.org/download.cgi) 3.5.0 or higher. - * Install [Docker](https://docs.docker.com/get-docker/) for your OS. -* Make sure you're assigned either the `Owner` role or the `Contributor` and `User Access Administrator` roles in the subscription. You can verify it by following steps in [List role assignments for a user or group](../role-based-access-control/role-assignments-list-portal.md#list-role-assignments-for-a-user-or-group). - ## Create a Liberty on AKS deployment using the portal The following steps guide you to create a Liberty runtime on AKS. After completing these steps, you have an Azure Container Registry and an Azure Kubernetes Service cluster for deploying your containerized application. -1. Visit the [Azure portal](https://portal.azure.com/). In the search box at the top of the page, type *IBM WebSphere Liberty and Open Liberty on Azure Kubernetes Service*. When the suggestions start appearing, select the one and only match that appears in the **Marketplace** section. If you prefer, you can go directly to the offer with this shortcut link: [https://aka.ms/liberty-aks](https://aka.ms/liberty-aks). +1. Visit the [Azure portal](https://portal.azure.com/). In the search box at the top of the page, type *IBM Liberty on AKS*. When the suggestions start appearing, select the one and only match that appears in the **Marketplace** section. If you prefer, you can go directly to the offer with this shortcut link: [https://aka.ms/liberty-aks](https://aka.ms/liberty-aks). 1. Select **Create**. The following steps guide you to create a Liberty runtime on AKS. After completi 1. Create a new resource group. Because resource groups must be unique within a subscription, pick a unique name. An easy way to have unique names is to use a combination of your initials, today's date, and some identifier. For example, `ejb0913-java-liberty-project-rg`. 1. Select *East US* as **Region**. - Create environment variables in your shell for the resource group names for the cluster and the database. - - ### [Bash](#tab/in-bash) - - ```bash - export RESOURCE_GROUP_NAME=<your-resource-group-name> - export DB_RESOURCE_GROUP_NAME=<your-resource-group-name> - ``` - - ### [PowerShell](#tab/in-powershell) - - ```powershell - $Env:RESOURCE_GROUP_NAME="<your-resource-group-name>" - $Env:DB_RESOURCE_GROUP_NAME="<your-resource-group-name>" - ``` - - + 1. Create environment variables in your shell for the resource group name for the cluster. ++ ### [Bash](#tab/in-bash) ++ ```bash + export RESOURCE_GROUP_NAME=<your-resource-group-name> + ``` ++ ### [PowerShell](#tab/in-powershell) ++ ```powershell + $Env:RESOURCE_GROUP_NAME="<your-resource-group-name>" + ``` -1. Select **Next**, enter the **AKS** pane. This pane allows you to select an existing AKS cluster and Azure Container Registry (ACR), instead of causing the deployment to create a new one, if desired. This capability enables you to use the sidecar pattern, as shown in the [Azure architecture center](/azure/architecture/patterns/sidecar). You can also adjust the settings for the size and number of the virtual machines in the AKS node pool. The remaining values do not need to be changed from their default values. +1. Select **Next**, enter the **AKS** pane. This pane allows you to select an existing AKS cluster and Azure Container Registry (ACR), instead of causing the deployment to create a new one, if desired. This capability enables you to use the sidecar pattern, as shown in the [Azure architecture center](/azure/architecture/patterns/sidecar). You can also adjust the settings for the size and number of the virtual machines in the AKS node pool. For our purposes, just keep all the defaults on this pane. 1. Select **Next**, enter the **Load Balancing** pane. Next to **Connect to Azure Application Gateway?** select **Yes**. This section lets you customize the following deployment options. - 1. You can customize the **virtual network** and **subnet** into which the deployment will place the resources. The remaining values do not need to be changed from their default values. + 1. You can optionally customize the **virtual network** and **subnet** into which the deployment places the resources. The remaining values don't need to be changed from their default values. 1. You can provide the **TLS/SSL certificate** presented by the Azure Application Gateway. Leave the values at the default to cause the offer to generate a self-signed certificate. Don't go to production using a self-signed certificate. For more information about self-signed certificates, see [Create a self-signed public certificate to authenticate your application](../active-directory/develop/howto-create-self-signed-certificate.md). 1. You can select **Enable cookie based affinity**, also known as sticky sessions. We want sticky sessions enabled for this article, so ensure this option is selected. The following steps guide you to create a Liberty runtime on AKS. After completi ## Capture selected information from the deployment -If you navigated away from the **Deployment is in progress** page, the following steps will show you how to get back to that page. If you're still on the page that shows **Your deployment is complete**, you can skip to the third step. +If you navigated away from the **Deployment is in progress** page, the following steps show you how to get back to that page. If you're still on the page that shows **Your deployment is complete**, go to the newly created resource group and skip to the third step. 1. In the upper left of any portal page, select the hamburger menu and select **Resource groups**. 1. In the box with the text **Filter for any field**, enter the first few characters of the resource group you created previously. If you followed the recommended convention, enter your initials, then select the appropriate resource group. If you navigated away from the **Deployment is in progress** page, the following 1. Save aside the values for **Login server**, **Registry name**, **Username**, and **password**. You may use the copy icon at the right of each field to copy the value of that field to the system clipboard. 1. Navigate again to the resource group into which you deployed the resources. 1. In the **Settings** section, select **Deployments**.-1. Select the bottom-most deployment in the list. The **Deployment name** will match the publisher ID of the offer. It will contain the string `ibm`. +1. Select the bottom-most deployment in the list. The **Deployment name** matches the publisher ID of the offer. It contains the string `ibm`. 1. In the left pane, select **Outputs**. 1. Using the same copy technique as with the preceding values, save aside the values for the following outputs: If you navigated away from the **Deployment is in progress** page, the following ### [Bash](#tab/in-bash) - Paste the value of `appDeploymentTemplateYaml` or `appDeploymentYaml` into a Bash shell, append `| grep secretName`, and execute. This command will output the Ingress TLS secret name, such as `- secretName: secret785e2c`. Save aside the value for `secretName` from the output. + Paste the value of `appDeploymentTemplateYaml` or `appDeploymentYaml` into a Bash shell, append `| grep secretName`, and execute. This command outputs the Ingress TLS secret name, such as `- secretName: secret785e2c`. Save aside the value for `secretName` from the output. ### [PowerShell](#tab/in-powershell) - Paste the quoted string in `appDeploymentTemplateYaml` or `appDeploymentYaml` into a PowerShell, append `| ForEach-Object { [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($_)) } | Select-String "secretName"`, and execute. This command will output the Ingress TLS secret name, such as `- secretName: secret785e2c`. Save aside the value for `secretName` from the output. + Paste the quoted string in `appDeploymentTemplateYaml` or `appDeploymentYaml` into a PowerShell (excluding the `| base64` portion), append `| ForEach-Object { [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($_)) } | Select-String "secretName"`, and execute. This command outputs the Ingress TLS secret name, such as `- secretName: secret785e2c`. Save aside the value for `secretName` from the output. -These values will be used later in this article. Note that several other useful commands are listed in the outputs. + These values are used later in this article. Note that several other useful commands are listed in the outputs. -> [!NOTE] -> You may notice a similar output named **appDeploymentYaml**. The difference between output *appDeploymentTemplateYaml* and *appDeploymentYaml* is: -> * *appDeploymentTemplateYaml* is populated if and only if the deployment **does not include** an application. -> * *appDeploymentYaml* is populated if and only if the deployment **does include** an application. + > [!NOTE] + > You may notice a similar output named **appDeploymentYaml**. The difference between output *appDeploymentTemplateYaml* and *appDeploymentYaml* is: + > * *appDeploymentTemplateYaml* is populated if and only if the deployment **does not include** an application. + > * *appDeploymentYaml* is populated if and only if the deployment **does include** an application. ## Create an Azure SQL Database [!INCLUDE [create-azure-sql-database](includes/jakartaee/create-azure-sql-database.md)] -Now that the database and AKS cluster have been created, we can proceed to preparing AKS to host your Open Liberty application. +Create environment variables in your shell for the resource group name for the database. + +### [Bash](#tab/in-bash) + +```bash +export DB_RESOURCE_GROUP_NAME=<your-database-resource-group-name> +``` + +### [PowerShell](#tab/in-powershell) + +```powershell +$Env:DB_RESOURCE_GROUP_NAME="<your-database-resource-group-name>" +``` ++++Now that the database and AKS cluster are created, we can proceed to preparing AKS to host your Open Liberty application. ## Configure and deploy the sample application Follow the steps in this section to deploy the sample application on the Liberty Clone the sample code for this guide. The sample is on [GitHub](https://github.com/Azure-Samples/open-liberty-on-aks). -There are a few samples in the repository. We'll use *java-app/*. Here's the file structure of the application. +There are a few samples in the repository. We use *java-app/*. Here's the file structure of the application. #### [Bash](#tab/in-bash) git checkout 20240109 -If you see a message about being in "detached HEAD" state, this message is safe to ignore. It just means you have checked out a tag. +If you see a message about being in "detached HEAD" state, this message is safe to ignore. It just means you checked out a tag. ``` java-app In directory *liberty/config*, the *server.xml* file is used to configure the DB ### Build the project -Now that you've gathered the necessary properties, you can build the application. The POM file for the project reads many variables from the environment. As part of the Maven build, these variables are used to populate values in the YAML files located in *src/main/aks*. You can do something similar for your application outside Maven if you prefer. +Now that you gathered the necessary properties, you can build the application. The POM file for the project reads many variables from the environment. As part of the Maven build, these variables are used to populate values in the YAML files located in *src/main/aks*. You can do something similar for your application outside Maven if you prefer. #### [Bash](#tab/in-bash) - ```bash cd $BASE_DIR/java-app-# The following variables will be used for deployment file generation into target. +# The following variables are used for deployment file generation into target. export LOGIN_SERVER=<Azure-Container-Registry-Login-Server-URL> export REGISTRY_NAME=<Azure-Container-Registry-name> export USER_NAME=<Azure-Container-Registry-username> mvn clean install ```powershell cd $env:BASE_DIR\java-app -# The following variables will be used for deployment file generation into target. -$Env:LOGIN_SERVER=<Azure-Container-Registry-Login-Server-URL> -$Env:REGISTRY_NAME=<Azure-Container-Registry-name> -$Env:USER_NAME=<Azure-Container-Registry-username> -$Env:PASSWORD=<Azure-Container-Registry-password> -$Env:DB_SERVER_NAME=<server-name>.database.windows.net -$Env:DB_NAME=<database-name> -$Env:DB_USER=<server-admin-login>@<server-name> -$Env:DB_PASSWORD=<server-admin-password> -$Env:INGRESS_TLS_SECRET=<ingress-TLS-secret-name> +# The following variables are used for deployment file generation into target. +$Env:LOGIN_SERVER="<Azure-Container-Registry-Login-Server-URL>" +$Env:REGISTRY_NAME="<Azure-Container-Registry-name>" +$Env:USER_NAME="<Azure-Container-Registry-username>" +$Env:PASSWORD="<Azure-Container-Registry-password>" +$Env:DB_SERVER_NAME="<server-name>.database.windows.net" +$Env:DB_NAME="<database-name>" +$Env:DB_USER="<server-admin-login>@<server-name>" +$Env:DB_PASSWORD="<server-admin-password>" +$Env:INGRESS_TLS_SECRET="<ingress-TLS-secret-name>" mvn clean install ``` mvn clean install You can now run and test the project locally before deploying to Azure. For convenience, we use the `liberty-maven-plugin`. To learn more about the `liberty-maven-plugin`, see [Building a web application with Maven](https://openliberty.io/guides/maven-intro.html). For your application, you can do something similar using any other mechanism, such as your local IDE. You can also consider using the `liberty:devc` option intended for development with containers. You can read more about `liberty:devc` in the [Liberty docs](https://openliberty.io/docs/latest/development-mode.html#_container_support_for_dev_mode). -1. Start the application using `liberty:run`. `liberty:run` will also use the environment variables defined in the previous step. +1. Start the application using `liberty:run`. `liberty:run` also uses the environment variables defined in the previous step. #### [Bash](#tab/in-bash) Use the following steps to deploy and test the application: 1. Connect to the AKS cluster. - Paste the value of **cmdToConnectToCluster** into a Bash shell and execute. + Paste the value of **cmdToConnectToCluster** into a shell and execute. 1. Apply the DB secret. Use the following steps to deploy and test the application: - You'll see the output `secret/db-secret-sql created`. + You see the output `secret/db-secret-sql created`. 1. Apply the deployment file. Use the following steps to deploy and test the application: Copy the value of **ADDRESS** from the output, this is the frontend public IP address of the deployed Azure Application Gateway. - 1. Go to `https://<ADDRESS>` to test the application. For your convenience, this shell command will create an environment variable whose value you can paste straight into the browser. + 1. Go to `https://<ADDRESS>` to test the application. For your convenience, this shell command creates an environment variable whose value you can paste straight into the browser. #### [Bash](#tab/in-bash) You can learn more from the following references: * [Open Liberty](https://openliberty.io/) * [Open Liberty Operator](https://github.com/OpenLiberty/open-liberty-operator) * [Open Liberty Server Configuration](https://openliberty.io/docs/ref/config/)- |
| aks | Howto Deploy Java Quarkus App | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/aks/howto-deploy-java-quarkus-app.md | Title: "Deploy Quarkus on Azure Kubernetes Service" description: Shows how to quickly stand up Quarkus on Azure Kubernetes Service.-+ Instead of `quarkus dev`, you can accomplish the same thing with Maven by using You may be asked if you want to send telemetry of your usage of Quarkus dev mode. If so, answer as you like. -Quarkus dev mode enables live reload with background compilation. If you modify any aspect of your app source code and refresh your browser, you can see the changes. If there are any issues with compilation or deployment, an error page lets you know. Quarkus dev mode listens for a debugger on port 5005. If you want to wait for the debugger to attach before running, pass `-Dsuspend` on the command line. If you donΓÇÖt want the debugger at all, you can use `-Ddebug=false`. +Quarkus dev mode enables live reload with background compilation. If you modify any aspect of your app source code and refresh your browser, you can see the changes. If there are any issues with compilation or deployment, an error page lets you know. Quarkus dev mode listens for a debugger on port 5005. If you want to wait for the debugger to attach before running, pass `-Dsuspend` on the command line. If you don't want the debugger at all, you can use `-Ddebug=false`. The output should look like the following example: |
| aks | Howto Deploy Java Wls App | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/aks/howto-deploy-java-wls-app.md | Title: "Deploy WebLogic Server on Azure Kubernetes Service using the Azure portal" description: Shows how to quickly stand up WebLogic Server on Azure Kubernetes Service.-+ Last updated 02/09/2024 Use the following steps to build the image: => => naming to docker.io/library/model-in-image:WLS-v1 0.2s ``` -1. If you have successfully created the image, then it should now be in your local machineΓÇÖs Docker repository. You can verify the image creation by using the following command: +1. If you have successfully created the image, then it should now be in your local machine's Docker repository. You can verify the image creation by using the following command: ```text docker images model-in-image:WLS-v1 |
| aks | Supported Kubernetes Versions | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/aks/supported-kubernetes-versions.md | For the past release history, see [Kubernetes history](https://github.com/kubern | K8s version | Upstream release | AKS preview | AKS GA | End of life | Platform support | |--|-|--||-|--|-| 1.25 | Aug 2022 | Oct 2022 | Dec 2022 | Jan 14, 2024 | Until 1.29 GA | | 1.26 | Dec 2022 | Feb 2023 | Apr 2023 | Mar 2024 | Until 1.30 GA | | 1.27* | Apr 2023 | Jun 2023 | Jul 2023 | Jul 2024, LTS until Jul 2025 | Until 1.31 GA | | 1.28 | Aug 2023 | Sep 2023 | Nov 2023 | Nov 2024 | Until 1.32 GA| | 1.29 | Dec 2023 | Feb 2024 | Mar 2024 | | Until 1.33 GA |+| 1.30 | Apr 2024 | May 2024 | Jun 2024 | | Until 1.34 GA | *\* Indicates the version is designated for Long Term Support* Note the following important changes before you upgrade to any of the available |Kubernetes Version | AKS Managed Addons | AKS Components | OS components | Breaking Changes | Notes |--||-||-||-| 1.25 | Azure policy 1.0.1<br>Metrics-Server 0.6.3<br>KEDA 2.9.3<br>Open Service Mesh 1.2.3<br>Core DNS V1.9.4<br>Overlay VPA 0.11.0<br>Azure-Keyvault-SecretsProvider 1.4.1<br>Application Gateway Ingress Controller (AGIC) 1.5.3<br>Image Cleaner v1.1.1<br>Azure Workload identity v1.0.0<br>MDC Defender 1.0.56<br>Azure Active Directory Pod Identity 1.8.13.6<br>GitOps 1.7.0<br>KMS 0.5.0| Cilium 1.12.8<br>CNI 1.4.44<br> Cluster Autoscaler 1.8.5.3<br> | OS Image Ubuntu 18.04 Cgroups V1 <br>ContainerD 1.7<br>Azure Linux 2.0<br>Cgroups V1<br>ContainerD 1.6<br>| Ubuntu 22.04 by default with cgroupv2 and Overlay VPA 0.13.0 |CgroupsV2 - If you deploy Java applications with the JDK, prefer to use JDK 11.0.16 and later or JDK 15 and later, which fully support cgroup v2 | 1.26 | Azure policy 1.3.0<br>Metrics-Server 0.6.3<br>KEDA 2.10.1<br>Open Service Mesh 1.2.3<br>Core DNS V1.9.4<br>Overlay VPA 0.11.0<br>Azure-Keyvault-SecretsProvider 1.4.1<br>Application Gateway Ingress Controller (AGIC) 1.5.3<br>Image Cleaner v1.2.3<br>Azure Workload identity v1.0.0<br>MDC Defender 1.0.56<br>Azure Active Directory Pod Identity 1.8.13.6<br>GitOps 1.7.0<br>KMS 0.5.0<br>azurefile-csi-driver 1.26.10<br>| Cilium 1.12.8<br>CNI 1.4.44<br> Cluster Autoscaler 1.8.5.3<br> | OS Image Ubuntu 22.04 Cgroups V2 <br>ContainerD 1.7<br>Azure Linux 2.0<br>Cgroups V1<br>ContainerD 1.6<br>|azurefile-csi-driver 1.26.10 |None | 1.27 | Azure policy 1.3.0<br>azuredisk-csi driver v1.28.5<br>azurefile-csi driver v1.28.7<br>blob-csi v1.22.4<br>csi-attacher v4.3.0<br>csi-resizer v1.8.0<br>csi-snapshotter v6.2.2<br>snapshot-controller v6.2.2<br>Metrics-Server 0.6.3<br>Keda 2.11.2<br>Open Service Mesh 1.2.3<br>Core DNS V1.9.4<br>Overlay VPA 0.11.0<br>Azure-Keyvault-SecretsProvider 1.4.1<br>Application Gateway Ingress Controller (AGIC) 1.7.2<br>Image Cleaner v1.2.3<br>Azure Workload identity v1.0.0<br>MDC Defender 1.0.56<br>Azure Active Directory Pod Identity 1.8.13.6<br>GitOps 1.7.0<br>azurefile-csi-driver 1.28.7<br>KMS 0.5.0<br>CSI Secret store driver 1.3.4-1<br>|Cilium 1.13.10-1<br>CNI 1.4.44<br> Cluster Autoscaler 1.8.5.3<br> | OS Image Ubuntu 22.04 Cgroups V2 <br>ContainerD 1.7 for Linux and 1.6 for Windows<br>Azure Linux 2.0<br>Cgroups V1<br>ContainerD 1.6<br>|Keda 2.11.2<br>Cilium 1.13.10-1<br>azurefile-csi-driver 1.28.7<br>azuredisk-csi driver v1.28.5<br>blob-csi v1.22.4<br>csi-attacher v4.3.0<br>csi-resizer v1.8.0<br>csi-snapshotter v6.2.2<br>snapshot-controller v6.2.2|Because of Ubuntu 22.04 FIPS certification status, we'll switch AKS FIPS nodes from 18.04 to 20.04 from 1.27 onwards. | 1.28 | Azure policy 1.3.0<br>azurefile-csi-driver 1.29.2<br>csi-node-driver-registrar v2.9.0<br>csi-livenessprobe 2.11.0<br>azuredisk-csi-linux v1.29.2<br>azuredisk-csi-windows v1.29.2<br>csi-provisioner v3.6.2<br>csi-attacher v4.5.0<br>csi-resizer v1.9.3<br>csi-snapshotter v6.2.2<br>snapshot-controller v6.2.2<br>Metrics-Server 0.6.3<br>KEDA 2.11.2<br>Open Service Mesh 1.2.7<br>Core DNS V1.9.4<br>Overlay VPA 0.13.0<br>Azure-Keyvault-SecretsProvider 1.4.1<br>Application Gateway Ingress Controller (AGIC) 1.7.2<br>Image Cleaner v1.2.3<br>Azure Workload identity v1.2.0<br>MDC Defender Security Publisher 1.0.68<br>CSI Secret store driver 1.3.4-1<br>MDC Defender Old File Cleaner 1.3.68<br>MDC Defender Pod Collector 1.0.78<br>MDC Defender Low Level Collector 1.3.81<br>Azure Active Directory Pod Identity 1.8.13.6<br>GitOps 1.8.1|Cilium 1.13.10-1<br>CNI v1.4.43.1 (Default)/v1.5.11 (Azure CNI Overlay)<br> Cluster Autoscaler 1.27.3<br>Tigera-Operator 1.28.13| OS Image Ubuntu 22.04 Cgroups V2 <br>ContainerD 1.7.5 for Linux and 1.7.1 for Windows<br>Azure Linux 2.0<br>Cgroups V1<br>ContainerD 1.6<br>|azurefile-csi-driver 1.29.2<br>csi-resizer v1.9.3<br>csi-attacher v4.4.2<br>csi-provisioner v4.4.2<br>blob-csi v1.23.2<br>azurefile-csi driver v1.29.2<br>azuredisk-csi driver v1.29.2<br>csi-livenessprobe v2.11.0<br>csi-node-driver-registrar v2.9.0|None New Supported Version List Platform support policy is a reduced support plan for certain unsupported Kubernetes versions. During platform support, customers only receive support from Microsoft for AKS/Azure platform related issues. Any issues related to Kubernetes functionality and components aren't supported. -Platform support policy applies to clusters in an n-3 version (where n is the latest supported AKS GA minor version), before the cluster drops to n-4. For example, Kubernetes v1.25 is considered platform support when v1.28 is the latest GA version. However, during the v1.29 GA release, v1.25 will then auto-upgrade to v1.26. If you are a running an n-2 version, the moment it becomes n-3 it also becomes deprecated, and you enter into the platform support policy. +Platform support policy applies to clusters in an n-3 version (where n is the latest supported AKS GA minor version), before the cluster drops to n-4. For example, Kubernetes v1.26 is considered platform support when v1.29 is the latest GA version. However, during the v1.30 GA release, v1.26 will then auto-upgrade to v1.27. If you are a running an n-2 version, the moment it becomes n-3 it also becomes deprecated, and you enter into the platform support policy. AKS relies on the releases and patches from [Kubernetes](https://kubernetes.io/releases/), which is an Open Source project that only supports a sliding window of three minor versions. AKS can only guarantee [full support](#kubernetes-version-support-policy) while those versions are being serviced upstream. Since there's no more patches being produced upstream, AKS can either leave those versions unpatched or fork. Due to this limitation, platform support doesn't support anything from relying on Kubernetes upstream. For information on how to upgrade your cluster, see: [get-azaksversion]: /powershell/module/az.aks/get-azaksversion [aks-tracker]: release-tracker.md [fleet-multi-cluster-upgrade]: /azure/kubernetes-fleet/update-orchestration- |
| api-management | Validate Azure Ad Token Policy | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/api-management/validate-azure-ad-token-policy.md | The `validate-azure-ad-token` policy enforces the existence and validity of a JS | - | -- | -- | | audiences | Contains a list of acceptable audience claims that can be present on the token. If multiple `audience` values are present, then each value is tried until either all are exhausted (in which case validation fails) or until one succeeds. Policy expressions are allowed. | No | | backend-application-ids | Contains a list of acceptable backend application IDs. This is only required in advanced cases for the configuration of options and can generally be removed. Policy expressions aren't allowed. | No |-| client-application-ids | Contains a list of acceptable client application IDs. If multiple `application-id` elements are present, then each value is tried until either all are exhausted (in which case validation fails) or until one succeeds. If a client application ID isn't provided, one or more `audience` claims should be specified. Policy expressions aren't allowed. | No | +| client-application-ids | Contains a list of acceptable client application IDs. If multiple `application-id` elements are present, then each value is tried until either all are exhausted (in which case validation fails) or until one succeeds. If a client application ID isn't provided, one or more `audience` claims should be specified. Policy expressions aren't allowed. | Yes | | required-claims | Contains a list of `claim` elements for claim values expected to be present on the token for it to be considered valid. When the `match` attribute is set to `all`, every claim value in the policy must be present in the token for validation to succeed. When the `match` attribute is set to `any`, at least one claim must be present in the token for validation to succeed. Policy expressions are allowed. | No | ### claim attributes |
| app-service | Configure Custom Container | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/app-service/configure-custom-container.md | This article shows you how to configure a custom container to run on Azure App S ::: zone pivot="container-windows" -This guide provides key concepts and instructions for containerization of Windows apps in App Service. If you've never used Azure App Service, follow the [custom container quickstart](quickstart-custom-container.md) and [tutorial](tutorial-custom-container.md) first. +This guide provides key concepts and instructions for containerization of Windows apps in App Service. New Azure App Service users should follow the [custom container quickstart](quickstart-custom-container.md) and [tutorial](tutorial-custom-container.md) first. ::: zone-end ::: zone pivot="container-linux" -This guide provides key concepts and instructions for containerization of Linux apps in App Service. If you've never used Azure App Service, follow the [custom container quickstart](quickstart-custom-container.md) and [tutorial](tutorial-custom-container.md) first. There's also a [multi-container app quickstart](quickstart-multi-container.md) and [tutorial](tutorial-multi-container-app.md). For sidecar containers (preview), see [Tutorial: Configure a sidecar container for custom container in Azure App Service (preview)](tutorial-custom-container-sidecar.md). +This guide provides key concepts and instructions for containerization of Linux apps in App Service. If are new to Azure App Service, follow the [custom container quickstart](quickstart-custom-container.md) and [tutorial](tutorial-custom-container.md) first. There's also a [multi-container app quickstart](quickstart-multi-container.md) and [tutorial](tutorial-multi-container-app.md). For sidecar containers (preview), see [Tutorial: Configure a sidecar container for custom container in Azure App Service (preview)](tutorial-custom-container-sidecar.md). ::: zone-end For *\<username>* and *\<password>*, supply the sign-in credentials for your pri ## Use managed identity to pull image from Azure Container Registry -Use the following steps to configure your web app to pull from ACR using managed identity. The steps use system-assigned managed identity, but you can use user-assigned managed identity as well. +Use the following steps to configure your web app to pull from Azure Container Registry (ACR) using managed identity. The steps use system-assigned managed identity, but you can use user-assigned managed identity as well. 1. Enable [the system-assigned managed identity](./overview-managed-identity.md) for the web app by using the [`az webapp identity assign`](/cli/azure/webapp/identity#az-webapp-identity-assign) command: ```azurecli-interactive az webapp identity assign --resource-group <group-name> --name <app-name> --query principalId --output tsv ```- Replace `<app-name>` with the name you used in the previous step. The output of the command (filtered by the `--query` and `--output` arguments) is the service principal ID of the assigned identity, which you use shortly. + Replace `<app-name>` with the name you used in the previous step. The output of the command (filtered by the `--query` and `--output` arguments) is the service principal ID of the assigned identity. 1. Get the resource ID of your Azure Container Registry: ```azurecli-interactive az acr show --resource-group <group-name> --name <registry-name> --query id --output tsv Use the following steps to configure your web app to pull from ACR using managed - `<app-name>` with the name of your web app. >[!Tip] > If you are using PowerShell console to run the commands, you need to escape the strings in the `--generic-configurations` argument in this and the next step. For example: `--generic-configurations '{\"acrUseManagedIdentityCreds\": true'`-1. (Optional) If your app uses a [user-assigned managed identity](overview-managed-identity.md#add-a-user-assigned-identity), make sure this is configured on the web app and then set the `acrUserManagedIdentityID` property to specify its client ID: +1. (Optional) If your app uses a [user-assigned managed identity](overview-managed-identity.md#add-a-user-assigned-identity), make sure the identity is configured on the web app and then set the `acrUserManagedIdentityID` property to specify its client ID: ```azurecli-interactive az identity show --resource-group <group-name> --name <identity-name> --query clientId --output tsv You're all set, and the web app now uses managed identity to pull from Azure Con ## Use an image from a network protected registry -To connect and pull from a registry inside a virtual network or on-premises, your app must integrate with a virtual network. This is also needed for Azure Container Registry with private endpoint. When your network and DNS resolution is configured, you enable the routing of the image pull through the virtual network by configuring the `vnetImagePullEnabled` site setting: +To connect and pull from a registry inside a virtual network or on-premises, your app must integrate with a virtual network (VNET). VNET integration is also needed for Azure Container Registry with private endpoint. When your network and DNS resolution is configured, you enable the routing of the image pull through the virtual network by configuring the `vnetImagePullEnabled` site setting: ```azurecli-interactive az resource update --resource-group <group-name> --name <app-name> --resource-type "Microsoft.Web/sites" --set properties.vnetImagePullEnabled [true|false] You can connect to your Windows container directly for diagnostic tasks by navig - It functions separately from the graphical browser above it, which only shows the files in your [shared storage](#use-persistent-shared-storage). - In a scaled-out app, the SSH session is connected to one of the container instances. You can select a different instance from the **Instance** dropdown in the top Kudu menu.-- Any change you make to the container from within the SSH session does *not* persist when your app is restarted (except for changes in the shared storage), because it's not part of the Docker image. To persist your changes, such as registry settings and software installation, make them part of the Dockerfile.+- Any change you make to the container from within the SSH session **doesn't** persist when your app is restarted (except for changes in the shared storage), because it's not part of the Docker image. To persist your changes, such as registry settings and software installation, make them part of the Dockerfile. ## Access diagnostic logs App Service logs actions by the Docker host and activities from within the cont There are several ways to access Docker logs: -- [In the Azure portal](#in-azure-portal)-- [From Kudu](#from-kudu)-- [With the Kudu API](#with-the-kudu-api)-- [Send logs to Azure monitor](troubleshoot-diagnostic-logs.md#send-logs-to-azure-monitor)+- [Azure portal](#in-azure-portal) +- [Kudu](#from-kudu) +- [Kudu API](#with-the-kudu-api) +- [Azure monitor](troubleshoot-diagnostic-logs.md#send-logs-to-azure-monitor) ### In Azure portal Docker logs are displayed in the portal, in the **Container Settings** page of y ### From Kudu -Navigate to `https://<app-name>.scm.azurewebsites.net/DebugConsole` and select the **LogFiles** folder to see the individual log files. To download the entire **LogFiles** directory, select the **Download** icon to the left of the directory name. You can also access this folder using an FTP client. +Navigate to `https://<app-name>.scm.azurewebsites.net/DebugConsole` and select the **LogFiles** folder to see the individual log files. To download the entire **LogFiles** directory, select the **"Download"** icon to the left of the directory name. You can also access this folder using an FTP client. In the SSH terminal, you can't access the `C:\home\LogFiles` folder by default because persistent shared storage isn't enabled. To enable this behavior in the console terminal, [enable persistent shared storage](#use-persistent-shared-storage). To download all the logs together in one ZIP file, access `https://<app-name>.sc ## Customize container memory -By default all Windows Containers deployed in Azure App Service have a memory limit configured. The following table lists the default settings per App Service Plan SKU. +By default all Windows Containers deployed in Azure App Service have a memory limit configured. The following table lists the default settings per App Service Plan SKU. | App Service Plan SKU | Default memory limit per app in MB | |-|-| In PowerShell: Set-AzWebApp -ResourceGroupName <group-name> -Name <app-name> -AppSettings @{"WEBSITE_MEMORY_LIMIT_MB"=2000} ``` -The value is defined in MB and must be less and equal to the total physical memory of the host. For example, in an App Service plan with 8GB RAM, the cumulative total of `WEBSITE_MEMORY_LIMIT_MB` for all the apps must not exceed 8 GB. Information on how much memory is available for each pricing tier can be found in [App Service pricing](https://azure.microsoft.com/pricing/details/app-service/windows/), in the **Premium v3 service plan** section. +The value is defined in MB and must be less and equal to the total physical memory of the host. For example, in an App Service plan with 8 GB RAM, the cumulative total of `WEBSITE_MEMORY_LIMIT_MB` for all the apps must not exceed 8 GB. Information on how much memory is available for each pricing tier can be found in [App Service pricing](https://azure.microsoft.com/pricing/details/app-service/windows/), in the **Premium v3 service plan** section. ## Customize the number of compute cores The processors might be multicore or hyperthreading processors. Information on h ## Customize health ping behavior -App Service considers a container to be successfully started when the container starts and responds to an HTTP ping. The health ping request contains the header `User-Agent= "App Service Hyper-V Container Availability Check"`. If the container starts but doesn't respond to a ping after a certain amount of time, App Service logs an event in the Docker log, saying that the container didn't start. +App Service considers a container to be successfully started when the container starts and responds to an HTTP ping. The health ping request contains the header `User-Agent= "App Service Hyper-V Container Availability Check"`. If the container starts but doesn't respond pings after a certain amount of time, App Service logs an event in the Docker log, saying that the container didn't start. If your application is resource-intensive, the container might not respond to the HTTP ping in time. To control the actions when HTTP pings fail, set the `CONTAINER_AVAILABILITY_CHECK_MODE` app setting. You can set it via the [Cloud Shell](https://shell.azure.com). In Bash: Secure Shell (SSH) is commonly used to execute administrative commands remotely 4. Rebuild and push the Docker image to the registry, and then test the Web App SSH feature on Azure portal. -Further troubleshooting information is available at the Azure App Service OSS blog: [Enabling SSH on Linux Web App for Containers](https://azureossd.github.io/2022/04/27/2022-Enabling-SSH-on-Linux-Web-App-for-Containers/https://docsupdatetracker.net/index.html#troubleshooting) +Further troubleshooting information is available at the Azure App Service blog: [Enabling SSH on Linux Web App for Containers](https://azureossd.github.io/2022/04/27/2022-Enabling-SSH-on-Linux-Web-App-for-Containers/https://docsupdatetracker.net/index.html#troubleshooting) ## Access diagnostic logs In your *docker-compose.yml* file, map the `volumes` option to `${WEBAPP_STORAGE wordpress: image: <image name:tag> volumes:- - ${WEBAPP_STORAGE_HOME}/site/wwwroot:/var/www/html - - ${WEBAPP_STORAGE_HOME}/phpmyadmin:/var/www/phpmyadmin - - ${WEBAPP_STORAGE_HOME}/LogFiles:/var/log + - "${WEBAPP_STORAGE_HOME}/site/wwwroot:/var/www/html" + - "${WEBAPP_STORAGE_HOME}/phpmyadmin:/var/www/phpmyadmin" + - "${WEBAPP_STORAGE_HOME}/LogFiles:/var/log" ``` ### Preview limitations The following lists show supported and unsupported Docker Compose configuration - "version x.x" always needs to be the first YAML statement in the file - ports section must use quoted numbers-- image > volume section must be quoted and cannot have permissions definitions+- image > volume section must be quoted and can't have permissions definitions - volumes section must not have an empty curly brace after the volume name > [!NOTE] |
| app-service | How To Side By Side Migrate | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/app-service/environment/how-to-side-by-side-migrate.md | description: Learn how to migrate your App Service Environment v2 to App Service Previously updated : 4/1/2024 Last updated : 4/4/2024 # Use the side-by-side migration feature to migrate App Service Environment v2 to App Service Environment v3 (Preview) az appservice ase show --name $ASE_NAME --resource-group $ASE_RG ## 10. Get the inbound IP addresses for your new App Service Environment v3 and update dependent resources -You have two App Service Environments at this stage in the migration process. Your apps are running in both environments. You need to update any dependent resources to use the new IP inbound address for your new App Service Environment v3. For internal facing (ILB) App Service Environments, you need to update your private DNS zones to point to the new inbound IP address. You should account for both the old and new inbound IP at this point. You can remove the dependencies on the previous IP address after you complete the next step. +You have two App Service Environments at this stage in the migration process. Your apps are running in both environments. You need to update any dependent resources to use the new IP inbound address for your new App Service Environment v3. For internal facing (ILB) App Service Environments, you need to update your private DNS zones to point to the new inbound IP address. This step is where you can validate your new environment and make any remaining necessary updates to your dependent resources. > [!IMPORTANT] > During the preview, the new inbound IP might be returned incorrectly due to a known bug. Open a support ticket to receive the correct IP addresses for your App Service Environment v3. For ELB App Service Environments, get the public inbound IP address by running t az rest --method get --uri "${ASE_ID}?api-version=2022-03-01" --query properties.networkingConfiguration.externalInboundIpAddresses ``` -## 11. Redirect customer traffic and complete migration +## 11. Redirect customer traffic, validate your App Service Environment v3, and complete migration -This step is your opportunity to test and validate your new App Service Environment v3. Your App Service Environment v2 front ends are still running, but the backing compute is an App Service Environment v3. If you're able to access your apps without issues, that means you're ready to complete the migration. +This step is your opportunity to test and validate your new App Service Environment v3. Your App Service Environment v2 front ends are still running, but the backing compute is an App Service Environment v3. If you're able to access your apps without issues that means you're ready to complete the migration. If you want to test your App Service Environment v3 front ends, you can do so by using the inbound IP address you got in the previous step. -Once you confirm your apps are working as expected, you can redirect customer traffic to your new App Service Environment v3 front ends by running the following command. This command also deletes your old environment. +Once you confirm your apps are working as expected, you can redirect customer traffic to your new App Service Environment v3 by running the following command. This command also deletes your old environment. ++If you find any issues or decide at this point that you no longer want to proceed with the migration, contact support to revert the migration. Don't run the DNS change command if you need to revert the migration. For more information, see [Revert migration](./side-by-side-migrate.md#redirect-customer-traffic-validate-your-app-service-environment-v3-and-complete-migration). ```azurecli az rest --method post --uri "${ASE_ID}/NoDowntimeMigrate?phase=DnsChange&api-version=2022-03-01" az rest --method get --uri "${ASE_ID}?api-version=2022-03-01" --query properties During this step, you get a status of `CompletingMigration`. When you get a status of `MigrationCompleted`, the traffic redirection step is done and your migration is complete. -If you find any issues or decide at this point that you no longer want to proceed with the migration, contact support to revert the migration. Don't run the above command if you need to revert the migration. For more information, see [Revert migration](side-by-side-migrate.md#redirect-customer-traffic-and-complete-migration). - ## Next steps > [!div class="nextstepaction"] |
| app-service | Side By Side Migrate | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/app-service/environment/side-by-side-migrate.md | Title: Migrate to App Service Environment v3 by using the side-by-side migration description: Overview of the side-by-side migration feature for migration to App Service Environment v3. Previously updated : 3/28/2024 Last updated : 4/4/2024 The platform creates the [the new outbound IP addresses](networking.md#addresses When completed, the new outbound IPs that your future App Service Environment v3 uses are created. These new IPs have no effect on your existing environment. -You receive the new inbound IP address once migration is complete but before you make the [DNS change to redirect customer traffic to your new App Service Environment v3](#redirect-customer-traffic-and-complete-migration). You don't get the inbound IP at this point in the process because there are dependencies on App Service Environment v3 resources that get created during the migration step. You have a chance to update any resources that are dependent on the new inbound IP before you redirect traffic to your new App Service Environment v3. +You receive the new inbound IP address once migration is complete but before you make the [DNS change to redirect customer traffic to your new App Service Environment v3](#redirect-customer-traffic-validate-your-app-service-environment-v3-and-complete-migration). You don't get the inbound IP at this point in the process because there are dependencies on App Service Environment v3 resources that get created during the migration step. You have a chance to update any resources that are dependent on the new inbound IP before you redirect traffic to your new App Service Environment v3. This step is also where you decide if you want to enable zone redundancy for your new App Service Environment v3. Zone redundancy can be enabled as long as your App Service Environment v3 is [in a region that supports zone redundancy](./overview.md#regions). Side-by-side migration requires a three to six hour service window for App Servi - The new App Service Environment v3 is created in the subnet you selected. - Your new App Service plans are created in the new App Service Environment v3 with the corresponding Isolated v2 tier. - Your apps are created in the new App Service Environment v3.-- The underlying compute for your apps is moved to the new App Service Environment v3. Your App Service Environment v2 front ends are still serving traffic. The migration process doesn't redirect to the App Service Environment v3 front ends until you complete the final step of the migration.+- The underlying compute for your apps is moved to the new App Service Environment v3. Your App Service Environment v2 front ends are still serving traffic. Your old inbound IP address remains in use. + - For ILB App Service Environments, your App Service Environment v3 front ends aren't used until you update your private DNS zones with the new inbound IP address. + - For ELB App Service Environments, the migration process doesn't redirect to the App Service Environment v3 front ends until you complete the final step of the migration. When this step completes, your application traffic is still going to your old App Service Environment front ends and the inbound IP that was assigned to it. However, you also now have an App Service Environment v3 with all of your apps. ### Get the inbound IP address for your new App Service Environment v3 and update dependent resources -The new inbound IP address is given so that you can set up new endpoints with services like [Traffic Manager](../../traffic-manager/traffic-manager-overview.md) or [Azure Front Door](../../frontdoor/front-door-overview.md) and update any of your private DNS zones. Don't move on to the next step until you account for these changes. There's downtime if you don't update dependent resources with the new inbound IP. **It's your responsibility to update any and all resources that are impacted by the IP address change associated with the new App Service Environment v3. Don't move on to the next step until you've made all required updates.** +The new inbound IP address is given so that you can set up new endpoints with services like [Traffic Manager](../../traffic-manager/traffic-manager-overview.md) or [Azure Front Door](../../frontdoor/front-door-overview.md) and update any of your private DNS zones. Don't move on to the next step until you make these changes. There's downtime if you don't update dependent resources with the new inbound IP. **It's your responsibility to update any and all resources that are impacted by the IP address change associated with the new App Service Environment v3. Don't move on to the next step until you've made all required updates.** -### Redirect customer traffic and complete migration +### Redirect customer traffic, validate your App Service Environment v3, and complete migration -The final step is to redirect traffic to your new App Service Environment v3 and complete the migration. The platform does this change for you, but only when you initiate it. Before you do this step, you should review your new App Service Environment v3 and perform any needed testing to validate that it's functioning as intended. Your App Service Environment v2 front ends are still running, but the backing compute is an App Service Environment v3. If you're able to access your apps without issues, that means you're ready to complete the migration. +The final step is to redirect traffic to your new App Service Environment v3 and complete the migration. The platform does this change for you, but only when you initiate it. Before you do this step, you should review your new App Service Environment v3 and perform any needed testing to validate that it's functioning as intended. Your App Service Environment v2 front ends are still running, but the backing compute is an App Service Environment v3. If you're using an ILB App Service Environment v3, you can test your App Service Environment v3 front ends by updating your private DNS zones with the new inbound IP address. Testing this change allows you to fully validate your App Service Environment v3 before initiating the final step of the migration where your old App Service Environment is deleted. Once you're ready to redirect traffic, you can complete the final step of the migration. This step updates internal DNS records to point to the load balancer IP address of your new App Service Environment v3 and the front ends that were created during the migration. Changes are effective within a couple minutes. If you run into issues, check your cache and TTL settings. This step also shuts down your old App Service Environment and deletes it. Your new App Service Environment v3 is now your production environment. |
| app-service | Upgrade To Asev3 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/app-service/environment/upgrade-to-asev3.md | This page is your one-stop shop for guidance and resources to help you upgrade s |-||| |**1**|**Pre-flight check**|Determine if your environment meets the prerequisites to automate your upgrade using one of the automated migration features. Decide whether an in-place or side-by-side migration is right for your use case.<br><br>- [Migration path decision tree](#migration-path-decision-tree)<br>- [Automated upgrade using the in-place migration feature](migrate.md)<br>- [Automated upgrade using the side-by-side migration feature](side-by-side-migrate.md)<br><br>If not, you can upgrade manually.<br><br>- [Manual migration](migration-alternatives.md)| |**2**|**Migrate**|Based on results of your review, either upgrade using one of the automated migration features or follow the manual steps.<br><br>- [Use the in-place automated migration feature](how-to-migrate.md)<br>- [Use the side-by-side automated migration feature](how-to-side-by-side-migrate.md)<br>- [Migrate manually](migration-alternatives.md)|-|**3**|**Testing and troubleshooting**|Upgrading using one of the automated migration features requires a 3-6 hour service window. If you use the side-by-side migration feature, you have the opportunity to [test and validate your App Service Environment v3](side-by-side-migrate.md#redirect-customer-traffic-and-complete-migration) before completing the upgrade. Support teams are monitoring upgrades to ensure success. If you have a support plan and you need technical help, create a [support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest).| +|**3**|**Testing and troubleshooting**|Upgrading using one of the automated migration features requires a 3-6 hour service window. If you use the side-by-side migration feature, you have the opportunity to [test and validate your App Service Environment v3](./side-by-side-migrate.md#redirect-customer-traffic-validate-your-app-service-environment-v3-and-complete-migration) before completing the upgrade. Support teams are monitoring upgrades to ensure success. If you have a support plan and you need technical help, create a [support request](https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest).| |**4**|**Optimize your App Service plans**|Once your upgrade is complete, you can optimize the App Service plans for additional benefits.<br><br>Review the autoselected Isolated v2 SKU sizes and scale up or scale down your App Service plans as needed.<br><br>- [Scale down your App Service plans](../manage-scale-up.md)<br>- [App Service Environment post-migration scaling guidance](migrate.md#pricing)<br><br>Explore reserved instance pricing, savings plans, and check out the pricing estimates if needed.<br><br>- [App Service pricing page](https://azure.microsoft.com/pricing/details/app-service/windows/)<br>- [How reservation discounts apply to Isolated v2 instances](../../cost-management-billing/reservations/reservation-discount-app-service.md#how-reservation-discounts-apply-to-isolated-v2-instances)<br>- [Azure pricing calculator](https://azure.microsoft.com/pricing/calculator)| |**5**|**Learn more**|On-demand: [Learn Live webinar with Azure FastTrack Architects](https://www.youtube.com/watch?v=lI9TK_v-dkg&ab_channel=MicrosoftDeveloper).<br><br>Need more help? [Submit a request](https://cxp.azure.com/nominationportal/nominationform/fasttrack) to contact FastTrack.<br><br>[Frequently asked questions](migrate.md#frequently-asked-questions)<br><br>[Community support](https://aka.ms/asev1v2retirement)| |
| app-service | Provision Resource Bicep | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/app-service/provision-resource-bicep.md | To deploy a different language stack, update `linuxFxVersion` with appropriate v | **PHP** | linuxFxVersion="PHP|7.4" | | **Node.js** | linuxFxVersion="NODE|10.15" | | **Java** | linuxFxVersion="JAVA|1.8 |TOMCAT|9.0" |-| **Python** | linuxFxVersion="PYTHON|3.7" | +| **Python** | linuxFxVersion="PYTHON|3.8" | |
| application-gateway | Ipv6 Application Gateway Portal | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/application-gateway/ipv6-application-gateway-portal.md | description: Learn how to configure Application Gateway with a frontend public I Previously updated : 03/17/2024 Last updated : 04/04/2024 |
| azure-app-configuration | Concept Enable Rbac | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-app-configuration/concept-enable-rbac.md | Title: Authorize access to Azure App Configuration using Microsoft Entra ID -description: Enable Azure RBAC to authorize access to your Azure App Configuration instance +description: Enable Azure RBAC to authorize access to your Azure App Configuration instance. Last updated 05/26/2020-Besides using Hash-based Message Authentication Code (HMAC), Azure App Configuration supports using Microsoft Entra ID to authorize requests to App Configuration instances. Microsoft Entra ID allows you to use Azure role-based access control (Azure RBAC) to grant permissions to a security principal. A security principal may be a user, a [managed identity](../active-directory/managed-identities-azure-resources/overview.md) or an [application service principal](../active-directory/develop/app-objects-and-service-principals.md). To learn more about roles and role assignments, see [Understanding different roles](../role-based-access-control/overview.md). +Besides using Hash-based Message Authentication Code (HMAC), Azure App Configuration supports using Microsoft Entra ID to authorize requests to App Configuration instances. Microsoft Entra ID allows you to use Azure role-based access control (Azure RBAC) to grant permissions to a security principal. A security principal may be a user, a [managed identity](../active-directory/managed-identities-azure-resources/overview.md), or an [application service principal](../active-directory/develop/app-objects-and-service-principals.md). To learn more about roles and role assignments, see [Understanding different roles](../role-based-access-control/overview.md). ## Overview Requests made by a security principal to access an App Configuration resource must be authorized. With Microsoft Entra ID, access to a resource is a two-step process:-1. The security principal's identity is authenticated and an OAuth 2.0 token is returned. The resource name to request a token is `https://login.microsoftonline.com/{tenantID}` where `{tenantID}` matches the Microsoft Entra tenant ID to which the service principal belongs. +1. The security principal's identity is authenticated and an OAuth 2.0 token is returned. The resource name to request a token is `https://login.microsoftonline.com/{tenantID}` where `{tenantID}` matches the Microsoft Entra tenant ID to which the service principal belongs. 2. The token is passed as part of a request to the App Configuration service to authorize access to the specified resource. -The authentication step requires that an application request contains an OAuth 2.0 access token at runtime. If an application is running within an Azure entity, such as an Azure Functions app, an Azure Web App, or an Azure VM, it can use a managed identity to access the resources. To learn how to authenticate requests made by a managed identity to Azure App Configuration, see [Authenticate access to Azure App Configuration resources with Microsoft Entra ID and managed identities for Azure Resources](howto-integrate-azure-managed-service-identity.md). +The authentication step requires that an application request contains an OAuth 2.0 access token at runtime. If an application is running within an Azure entity, such as an Azure Functions app, an Azure Web App, or an Azure VM, it can use a managed identity to access the resources. To learn how to authenticate requests made by a managed identity to Azure App Configuration, see [Authenticate access to Azure App Configuration resources with Microsoft Entra ID and managed identities for Azure Resources](howto-integrate-azure-managed-service-identity.md). The authorization step requires that one or more Azure roles be assigned to the security principal. Azure App Configuration provides Azure roles that encompass sets of permissions for App Configuration resources. The roles that are assigned to a security principal determine the permissions provided to the principal. For more information about Azure roles, see [Azure built-in roles for Azure App Configuration](#azure-built-in-roles-for-azure-app-configuration). When an Azure role is assigned to a Microsoft Entra security principal, Azure gr ## Azure built-in roles for Azure App Configuration Azure provides the following Azure built-in roles for authorizing access to App Configuration data using Microsoft Entra ID: -- **App Configuration Data Owner**: Use this role to give read/write/delete access to App Configuration data. This does not grant access to the App Configuration resource.-- **App Configuration Data Reader**: Use this role to give read access to App Configuration data. This does not grant access to the App Configuration resource.-- **Contributor** or **Owner**: Use this role to manage the App Configuration resource. It grants access to the resource's access keys. While the App Configuration data can be accessed using access keys, this role does not grant direct access to the data using Microsoft Entra ID. This role is required if you access the App Configuration data via ARM template, Bicep, or Terraform during deployment. For more information, see [authorization](quickstart-resource-manager.md#authorization).-- **Reader**: Use this role to give read access to the App Configuration resource. This does not grant access to the resource's access keys, nor to the data stored in App Configuration.+- **App Configuration Data Owner**: Use this role to give read/write/delete access to App Configuration data. This role doesn't grant access to the App Configuration resource. +- **App Configuration Data Reader**: Use this role to give read access to App Configuration data. This role doesn't grant access to the App Configuration resource. +- **Contributor** or **Owner**: Use this role to manage the App Configuration resource. It grants access to the resource's access keys. While the App Configuration data can be accessed using access keys, this role doesn't grant direct access to the data using Microsoft Entra ID. This role is required if you access the App Configuration data via ARM template, Bicep, or Terraform during deployment. For more information, see [deployment](quickstart-deployment-overview.md). +- **Reader**: Use this role to give read access to the App Configuration resource. This role doesn't grant access to the resource's access keys, nor to the data stored in App Configuration. > [!NOTE] > After a role assignment is made for an identity, allow up to 15 minutes for the permission to propagate before accessing data stored in App Configuration using this identity. |
| azure-app-configuration | Howto Disable Access Key Authentication | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-app-configuration/howto-disable-access-key-authentication.md | Title: Disable access key authentication for an Azure App Configuration instance -description: Learn how to disable access key authentication for an Azure App Configuration instance +description: Learn how to disable access key authentication for an Azure App Configuration instance. When you disable access key authentication for an Azure App Configuration resour ## Disable access key authentication -Disabling access key authentication will delete all access keys. If any running applications are using access keys for authentication they will begin to fail once access key authentication is disabled. Enabling access key authentication again will generate a new set of access keys and any applications attempting to use the old access keys will still fail. +Disabling access key authentication will delete all access keys. If any running applications are using access keys for authentication, they will begin to fail once access key authentication is disabled. Enabling access key authentication again will generate a new set of access keys and any applications attempting to use the old access keys will still fail. > [!WARNING] > If any clients are currently accessing data in your Azure App Configuration resource with access keys, then Microsoft recommends that you migrate those clients to [Microsoft Entra ID](./concept-enable-rbac.md) before disabling access key authentication.-> Additionally, it is recommended to read the [limitations](#limitations) section below to verify the limitations won't affect the intended usage of the resource. # [Azure portal](#tab/portal) To disallow access key authentication for an Azure App Configuration resource in the Azure portal, follow these steps: 1. Navigate to your Azure App Configuration resource in the Azure portal.-2. Locate the **Access keys** setting under **Settings**. +2. Locate the **Access settings** setting under **Settings**. - :::image type="content" border="true" source="./media/access-keys-blade.png" alt-text="Screenshot showing how to access an Azure App Configuration resources access key blade"::: + :::image type="content" border="true" source="./media/access-settings-blade.png" alt-text="Screenshot showing how to access an Azure App Configuration resources access key blade."::: 3. Set the **Enable access keys** toggle to **Disabled**. The capability to disable access key authentication using the Azure CLI is in de ### Verify that access key authentication is disabled -To verify that access key authentication is no longer permitted, a request can be made to list the access keys for the Azure App Configuration resource. If access key authentication is disabled there will be no access keys and the list operation will return an empty list. +To verify that access key authentication is no longer permitted, a request can be made to list the access keys for the Azure App Configuration resource. If access key authentication is disabled, there will be no access keys, and the list operation will return an empty list. # [Azure portal](#tab/portal) To verify access key authentication is disabled for an Azure App Configuration resource in the Azure portal, follow these steps: 1. Navigate to your Azure App Configuration resource in the Azure portal.-2. Locate the **Access keys** setting under **Settings**. +2. Locate the **Access settings** setting under **Settings**. - :::image type="content" border="true" source="./media/access-keys-blade.png" alt-text="Screenshot showing how to access an Azure App Configuration resources access key blade"::: + :::image type="content" border="true" source="./media/access-settings-blade.png" alt-text="Screenshot showing how to access an Azure App Configuration resources access key blade."::: 3. Verify there are no access keys displayed and **Enable access keys** is toggled to **Disabled**. az appconfig credential list \ --resource-group <resource-group> ``` -If access key authentication is disabled then an empty list will be returned. +If access key authentication is disabled, then an empty list will be returned. ``` C:\Users\User>az appconfig credential list -g <resource-group> -n <app-configuration-name> These roles do not provide access to data in an Azure App Configuration resource Role assignments must be scoped to the level of the Azure App Configuration resource or higher to permit a user to allow or disallow access key authentication for the resource. For more information about role scope, see [Understand scope for Azure RBAC](../role-based-access-control/scope-overview.md). -Be careful to restrict assignment of these roles only to those who require the ability to create an App Configuration resource or update its properties. Use the principle of least privilege to ensure that users have the fewest permissions that they need to accomplish their tasks. For more information about managing access with Azure RBAC, see [Best practices for Azure RBAC](../role-based-access-control/best-practices.md). +Be careful to restrict assignment of these roles only to those users who require the ability to create an App Configuration resource or update its properties. Use the principle of least privilege to ensure that users have the fewest permissions that they need to accomplish their tasks. For more information about managing access with Azure RBAC, see [Best practices for Azure RBAC](../role-based-access-control/best-practices.md). > [!NOTE] > The classic subscription administrator roles Service Administrator and Co-Administrator include the equivalent of the Azure Resource Manager [Owner](../role-based-access-control/built-in-roles.md#owner) role. The **Owner** role includes all actions, so a user with one of these administrative roles can also create and manage App Configuration resources. For more information, see [Azure roles, Microsoft Entra roles, and classic subscription administrator roles](../role-based-access-control/rbac-and-directory-admin-roles.md#classic-subscription-administrator-roles). -## Limitations --The capability to disable access key authentication has the following limitation: --### ARM template access --When access key authentication is disabled, the capability to read/write key-values in an [ARM template](./quickstart-resource-manager.md) will be disabled as well. This is because access to the Microsoft.AppConfiguration/configurationStores/keyValues resource used in ARM templates requires an Azure Resource Manager role, such as contributor or owner. When access key authentication is disabled, access to the resource requires one of the Azure App Configuration [data plane roles](concept-enable-rbac.md), therefore ARM template access is rejected. +> [!NOTE] +> When access key authentication is disabled and [ARM authentication mode](./quickstart-deployment-overview.md#azure-resource-manager-authentication-mode) of App Configuration store is local, the capability to read/write key-values in an [ARM template](./quickstart-resource-manager.md) will be disabled as well. This is because access to the Microsoft.AppConfiguration/configurationStores/keyValues resource used in ARM templates requires access key authentication with local ARM authentication mode. It's recommended to use pass-through ARM authentication mode. For more information, see [Deployment overview](./quickstart-deployment-overview.md). ## Next steps |
| azure-app-configuration | Quickstart Bicep | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-app-configuration/quickstart-bicep.md | This quickstart describes how you can use Bicep to: If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin. +## Authorization ++Managing an Azure App Configuration resource with Bicep file requires an Azure Resource Manager role, such as contributor or owner. Accessing Azure App Configuration data (key-values, snapshots) requires an Azure Resource Manager role and an additional Azure App Configuration [data plane role](concept-enable-rbac.md) when the configuration store's ARM authentication mode is set to [pass-through](./quickstart-deployment-overview.md#azure-resource-manager-authentication-mode) ARM authentication mode. + ## Review the Bicep file The Bicep file used in this quickstart is from [Azure Quickstart Templates](https://azure.microsoft.com/resources/templates/app-configuration-store-kv/). |
| azure-app-configuration | Quickstart Deployment Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-app-configuration/quickstart-deployment-overview.md | + + Title: Deployment overview ++description: Learn how to use Azure App Configuration in deployment. ++ Last updated : 03/15/2024++++++# Deployment ++Azure App Configuration supports the following methods to read and manage your configuration during deployment: ++- [ARM template](./quickstart-resource-manager.md) +- [Bicep](./quickstart-bicep.md) +- Terraform ++## Manage Azure App Configuration resources in deployment ++### Azure Resource Manager Authorization ++You must have Azure Resource Manager permissions to manage Azure App Configuration resources. Azure role-based access control (Azure RBAC) roles that provide these permissions include the Microsoft.AppConfiguration/configurationStores/write or Microsoft.AppConfiguration/configurationStores/* action. Built-in roles with this action include: ++- Owner +- Contributor ++To learn more about Azure RBAC and Microsoft Entra ID, see [Authorize access to Azure App Configuration using Microsoft Entra ID](./concept-enable-rbac.md). ++## Manage Azure App Configuration data in deployment ++Azure App Configuration data, such as key-values and snapshots, can be managed in deployment. When managing App Configuration data using this method, it's recommended to set your configuration store's Azure Resource Manager authentication mode to **Pass-through**. This authentication mode ensures that data access requires a combination of data plane and Azure Resource Manager management roles and ensuring that data access can be properly attributed to the deployment caller for auditing purpose. ++### Azure Resource Manager authentication mode ++# [Azure portal](#tab/portal) ++To configure the Azure Resource Manager authentication mode of an Azure App Configuration resource in the Azure portal, follow these steps: ++1. Navigate to your Azure App Configuration resource in the Azure portal +2. Locate the **Access settings** setting under **Settings** ++ :::image type="content" border="true" source="./media/access-settings-blade.png" alt-text="Screenshot showing how to access an Azure App Configuration resources access settings blade."::: ++3. Select the recommended **Pass-through** authentication mode under **Azure Resource Manager Authentication Mode** ++ :::image type="content" border="true" source="./media/quickstarts/deployment/select-passthrough-authentication-mode.png" alt-text="Screenshot showing pass-through authentication mode being selected under Azure Resource Manager Authentication Mode."::: ++++> [!NOTE] +> Local authentication mode is for backward compatibility and has several limitations. It does not support proper auditing for accessing data in deployment. Under local authentication mode, key-value data access inside an ARM template/Bicep/Terraform is disabled if [access key authentication is disabled](./howto-disable-access-key-authentication.md). Azure App Configuration data plane permissions are not required for accessing data under local authentication mode. ++### Azure App Configuration Authorization ++When your App Configuration resource has its Azure Resource Manager authentication mode set to **Pass-through**, you must have Azure App Configuration data plane permissions to read and manage Azure App Configuration data in deployment. This requirement is in addition to baseline management permission requirements of the resource. Azure App Configuration data plane permissions include Microsoft.AppConfiguration/configurationStores/\*/read and Microsoft.AppConfiguration/configurationStores/\*/write. Built-in roles with this action include: ++- App Configuration Data Owner +- App Configuration Data Reader ++To learn more about Azure RBAC and Microsoft Entra ID, see [Authorize access to Azure App Configuration using Microsoft Entra ID](./concept-enable-rbac.md). ++### Private network access ++When an App Configuration resource is restricted to private network access, deployments accessing App Configuration data through public networks will be blocked. To enable successful deployments when access to an App Configuration resource is restricted to private networks the following actions must be taken: ++- [Azure Resource Management Private Link](../azure-resource-manager/management/create-private-link-access-portal.md) must be set up +- The App Configuration resource must have Azure Resource Manager authentication mode set to **Pass-through** +- The App Configuration resource must have Azure Resource Manager private network access enabled +- Deployments accessing App Configuration data must run through the configured Azure Resource Manager private link ++If all of these criteria are met, then deployments accessing App Configuration data will be successful. ++# [Azure portal](#tab/portal) ++To enable Azure Resource Manager private network access for an Azure App Configuration resource in the Azure portal, follow these steps: ++1. Navigate to your Azure App Configuration resource in the Azure portal +2. Locate the **Networking** setting under **Settings** ++ :::image type="content" border="true" source="./media/networking-blade.png" alt-text="Screenshot showing how to access an Azure App Configuration resources networking blade."::: ++3. Check **Enable Azure Resource Manager Private Access** under **Private Access** ++ :::image type="content" border="true" source="./media/quickstarts/deployment/enable-azure-resource-manager-private-access.png" alt-text="Screenshot showing Enable Azure Resource Manager Private Access is checked."::: ++> [!NOTE] +> Azure Resource Manager private network access can only be enabled under **Pass-through** authentication mode. ++++## Next steps ++To learn about deployment using ARM template and Bicep, check the documentations linked below. ++- [Quickstart: Create an Azure App Configuration store by using an ARM template](./quickstart-resource-manager.md) +- [Quickstart: Create an Azure App Configuration store using Bicep](./quickstart-bicep.md) |
| azure-app-configuration | Quickstart Resource Manager | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-app-configuration/quickstart-resource-manager.md | -This quickstart describes how to : +This quickstart describes how to: - Deploy an App Configuration store using an Azure Resource Manager template (ARM template). - Create key-values in an App Configuration store using ARM template. If you don't have an Azure subscription, create a [free account](https://azure.m ## Authorization -Accessing key-value data inside an ARM template requires an Azure Resource Manager role, such as contributor or owner. Access via one of the Azure App Configuration [data plane roles](concept-enable-rbac.md) currently is not supported. --> [!NOTE] -> Key-value data access inside an ARM template is disabled if access key authentication is disabled. For more information, see [disable access key authentication](./howto-disable-access-key-authentication.md#limitations). +Managing Azure App Configuration resource inside an ARM template requires Azure Resource Manager role, such as contributor or owner. Accessing Azure App Configuration data (key-values, snapshots) requires Azure Resource Manager role and Azure App Configuration [data plane role](concept-enable-rbac.md) under [pass-through](./quickstart-deployment-overview.md#azure-resource-manager-authentication-mode) ARM authentication mode. ## Review the template |
| azure-arc | Troubleshoot Resource Bridge | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-arc/resource-bridge/troubleshoot-resource-bridge.md | This article provides information on troubleshooting and resolving issues that c ### Logs collection -For issues encountered with Arc resource bridge, collect logs for further investigation using the Azure CLI [`az arcappliance logs`](/cli/azure/arcappliance/logs) command. This command needs to be run from the same management machine that was used to run commands to deploy the Arc resource bridge. If you are using a different machine to collect logs, you need to run the `az arcappliance get-credentials` command first before collecting logs. +For issues encountered with Arc resource bridge, collect logs for further investigation using the Azure CLI [`az arcappliance logs`](/cli/azure/arcappliance/logs) command. This command needs to be run from the same management machine that was used to run commands to deploy the Arc resource bridge. If you're using a different machine to collect logs, you need to run the `az arcappliance get-credentials` command first before collecting logs. If there's a problem collecting logs, most likely the management machine is unable to reach the Appliance VM. Contact your network administrator to allow SSH communication from the management machine to the Appliance VM on TCP port 22. To collect Arc resource bridge logs for Azure Stack HCI using the appliance VM I az arcappliance logs hci --ip <appliance VM IP> --cloudagent <cloud agent service IP/FQDN> --loginconfigfile <file path of kvatoken.tok> ``` -If you are unsure of your appliance VM IP, there is also the option to use the kubeconfig. You can retrieve the kubeconfig by running the [get-credentials command](/cli/azure/arcappliance) then run the logs command. +If you're unsure of your appliance VM IP, there's also the option to use the kubeconfig. You can retrieve the kubeconfig by running the [get-credentials command](/cli/azure/arcappliance) then run the logs command. To retrieve the kubeconfig and log key then collect logs for Arc-enabled VMware from a different machine than the one used to deploy Arc resource bridge for Arc-enabled VMware: az arcappliance logs vmware --kubeconfig kubeconfig --out-dir <path to specified ### Arc resource bridge is offline -If the resource bridge is offline, this is typically due to a networking change in the infrastructure, environment or cluster that stops the appliance VM from being able to communicate with its counterpart Azure resource. If you are unable to determine what changed, you can reboot the appliance VM, collect logs and submit a support ticket for further investigation. +If the resource bridge is offline, this is typically due to a networking change in the infrastructure, environment or cluster that stops the appliance VM from being able to communicate with its counterpart Azure resource. If you're unable to determine what changed, you can reboot the appliance VM, collect logs and submit a support ticket for further investigation. ### Remote PowerShell isn't supported To resolve this problem, delete the resource bridge, register the providers, the Arc resource bridge consists of an appliance VM that is deployed to the on-premises infrastructure. The appliance VM maintains a connection to the management endpoint of the on-premises infrastructure using locally stored credentials. If these credentials aren't updated, the resource bridge is no longer able to communicate with the management endpoint. This can cause problems when trying to upgrade the resource bridge or manage VMs through Azure. To fix this, the credentials in the appliance VM need to be updated. For more information, see [Update credentials in the appliance VM](maintenance.md#update-credentials-in-the-appliance-vm). -### Private Link is unsupported +### Private link is unsupported Arc resource bridge doesn't support private link. All calls coming from the appliance VM shouldn't be going through your private link setup. The Private Link IPs may conflict with the appliance IP pool range, which isn't configurable on the resource bridge. Arc resource bridge reaches out to [required URLs](network-requirements.md#firewallproxy-url-allowlist) that shouldn't go through a private link connection. You must deploy Arc resource bridge on a separate network segment unrelated to the private link setup. To resolve this issue, reboot the resource bridge VM, and it should recover its Be sure that the proxy server on your management machine trusts both the SSL certificate for your SSL proxy and the SSL certificate of the Microsoft download servers. For more information, see [SSL proxy configuration](network-requirements.md#ssl-proxy-configuration). +### No such host - dp.kubernetesconfiguration.azure.com ++An error that contains `dial tcp: lookup westeurope.dp.kubernetesconfiguration.azure.com: no such host` while deploying Arc resource bridge means that the configuration dataplane is currently unavailable in the specified region. The service may be temporarily unavailable. Please wait for the service to be available and then retry the deployment. ++### No such host for Arc resource bridge required URL ++An error that contains an Arc resource bridge required URL with the message `no such host` indicates that DNS is not able to resolve the URL. The error may look similar to the example below, where the required URL is `https://msk8s.api.cdp.microsoft.com`: ++`Error: { _errorCode_: _InvalidEntityError_, _errorResponse_: _{\n\_message\_: \_Post \\\_https://msk8s.api.cdp.microsoft.com/api/v1.1/contents/default/namespaces/default/names/arc-appliance-stable-catalogs-ext/versions/latest?action=select\\\_: POST https://msk8s.api.cdp.microsoft.com/api/v1.1/contents/default/namespaces/default/names/arc-appliance-stable-catalogs-ext/versions/latest?action=select giving up after 6 attempt(s): Post \\\_https://msk8s.api.cdp.microsoft.com/api/v1.1/contents/default/namespaces/default/names/arc-appliance-stable-catalogs-ext/versions/latest?action=select\\\_: proxyconnect tcp: dial tcp: lookup http: no such host\_\n}_ }` ++This error can occur if the DNS settings provided during deployment are not correct or there is a problem with the DNS server(s). You can check if your DNS server is able to resolve the url by running the following command from the management machine or a machine that has access to the DNS server(s): ++``` +nslookup +> set debug +> <hostname> <DNS server IP> +``` ++In order to resolve the error, your DNS server(s) must be configured to resolve all Arc resource bridge required URLs and the DNS server(s) should be correctly provided during deployment of Arc resource bridge. + ### KVA timeout error -While trying to deploy Arc Resource Bridge, a "KVA timeout error" might appear. The "KVA timeout error" is a generic error that can be the result of a variety of network misconfigurations that involve the management machine, Appliance VM, or Control Plane IP not having communication with each other, to the internet, or required URLs. This communication failure is often due to issues with DNS resolution, proxy settings, network configuration, or internet access. +The KVA timeout error is a generic error that can be the result of a variety of network misconfigurations that involve the management machine, Appliance VM, or Control Plane IP not having communication with each other, to the internet, or required URLs. This communication failure is often due to issues with DNS resolution, proxy settings, network configuration, or internet access. -For clarity, "management machine" refers to the machine where deployment CLI commands are being run. "Appliance VM" is the VM that hosts Arc resource bridge. "Control Plane IP" is the IP of the control plane for the Kubernetes management cluster in the Appliance VM. +For clarity, management machine refers to the machine where deployment CLI commands are being run. Appliance VM is the VM that hosts Arc resource bridge. Control Plane IP is the IP of the control plane for the Kubernetes management cluster in the Appliance VM. #### Top causes of the KVA timeout errorΓÇ» To resolve the error, one or more network misconfigurations might need to be add Once logs are collected, extract the folder and open kva.log. Review the kva.log for more information on the failure to help pinpoint the cause of the KVA timeout error. -1. The management machine must be able to communicate with the Appliance VM IP and Control Plane IP. Ping the Control Plane IP and Appliance VM IP from the management machine and verify there is a response from both IPs. +1. The management machine must be able to communicate with the Appliance VM IP and Control Plane IP. Ping the Control Plane IP and Appliance VM IP from the management machine and verify there's a response from both IPs. If a request times out, the management machine can't communicate with the IP(s). This could be caused by a closed port, network misconfiguration or a firewall block. Work with your network administrator to allow communication between the management machine to the Control Plane IP and Appliance VM IP. To resolve the error, one or more network misconfigurations might need to be add 1. Appliance VM needs to be able to reach a DNS server that can resolve internal names such as vCenter endpoint for vSphere or cloud agent endpoint for Azure Stack HCI. The DNS server also needs to be able to resolve external/internal addresses, such as Azure service addresses and container registry names for download of the Arc resource bridge container images from the cloud. Verify that the DNS server IP used to create the configuration files has internal and external address resolution. If not, [delete the appliance](/cli/azure/arcappliance/delete), recreate the Arc resource bridge configuration files with the correct DNS server settings, and then deploy Arc resource bridge using the new configuration files.-+ ## Move Arc resource bridge location Resource move of Arc resource bridge isn't currently supported. You'll need to delete the Arc resource bridge, then re-deploy it to the desired location. To install Azure Arc resource bridge on an Azure Stack HCI cluster, `az arcappli ## Azure Arc-enabled VMware VCenter issues -### `az arcappliance prepare` failure +### vSphere SDK client 403 Forbidden or 404 not found -The `arcappliance` extension for Azure CLI enables a [prepare](/cli/azure/arcappliance/prepare) command, which enables you to download an OVA template to your vSphere environment. This OVA file is used to deploy the Azure Arc resource bridge. The `az arcappliance prepare` command uses the vSphere SDK and can result in the following error: +If you receive an error that contains `errorCode_: _CreateConfigKvaCustomerError_, _errorResponse_: _error getting the vsphere sdk client: POST \_/sdk\_: 403 Forbidden` or `404 not found` while deploying Arc resource bridge, this is most likely due to an incorrect vCenter URL being provided during configuration file creation where you're prompted to enter the vCenter address as either FQDN or IP address. There are different ways to find your vCenter address. One option is to access the vSphere client via its web interface. The vCenter FQDN or IP address is typically what you use in the browser to access the vSphere client. If you're already logged in, you can look at the browser's address bar; the URL you use to access vSphere is your vCenter server's FQDN or IP address. Alternatively, after logging in, go to the Menu > Administration section. Under System Configuration, choose Nodes. Your vCenter server instance(s) will be listed there along with its FQDN. Verify your vCenter address and then re-try the deployment. -```azurecli -$ az arcappliance prepare vmware --config-file <path to config> +### Pre-deployment validation errors -Error: Error in reading OVA file: failed to parse ovf: strconv.ParseInt: parsing "3670409216": -value out of range. -``` +If you're receiving a variety of `pre-deployment validation of your download\upload connectivity wasn't successful` errors, such as: ++`Pre-deployment validation of your download/upload connectivity wasn't successful. {\\n \\\_code\\\_: \\\_ImageProvisionError\\\_,\\n \\\_message\\\_: \\\_Post \\\\\\\_https://vcenter-server.com/nfc/unique-identifier/disk-0.vmdk\\\\\\\_: Service Unavailable` ++`Pre-deployment validation of your download/upload connectivity wasn't successful. {\\n \\\_code\\\_: \\\_ImageProvisionError\\\_,\\n \\\_message\\\_: \\\_Post \\\\\\\_https://vcenter-server.com/nfc/unique-identifier/disk-0.vmdk\\\\\\\_: dial tcp 172.16.60.10:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.` ++`Pre-deployment validation of your download/upload connectivity wasn't successful. {\\n \\\_code\\\_: \\\_ImageProvisionError\\\_,\\n \\\_message\\\_: \\\_Post \\\\\\\_https://vcenter-server.com/nfc/unique-identifier/disk-0.vmdk\\\\\\\_: use of closed network connection.` ++`Pre-deployment validation of your download/upload connectivity wasn't successful. {\\n \\\_code\\\_: \\\_ImageProvisionError\\\_,\\n \\\_message\\\_: \\\_Post \\\\\\\_https://vcenter-server.com/nfc/unique-identifier/disk-0.vmdk\\\\\\\_: dial tcp: lookup hostname.domain: no such host` ++A combination of these errors usually indicates that the management machine has lost connection to the datastore or there's a networking issue causing the datastore to be unreachable. This connection is needed in order to upload the OVA from the management machine used to build the appliance VM in vCenter. The connection between the management machine and datastore needs to be reestablished, then retry deployment of Arc resource bridge. ++### x509 certificate has expired or isn't yet valid ++When you deploy Arc resource bridge, you may encounter the error: ++`Error: { _errorCode_: _PostOperationsError_, _errorResponse_: _{\n\_message\_: \_{\\n \\\_code\\\_: \\\_GuestInternetConnectivityError\\\_,\\n \\\_message\\\_: \\\_Not able to connect to https://msk8s.api.cdp.microsoft.com. Error returned: action failed after 3 attempts: Get \\\\\\\_https://msk8s.api.cdp.microsoft.com\\\\\\\_: x509: certificate has expired or isn't yet valid: current time 2022-01-18T11:35:56Z is before 2023-09-07T19:13:21Z. Arc Resource Bridge network and internet connectivity validation failed: http-connectivity-test-arc. 1. Please check your networking setup and ensure the URLs mentioned in : https://aka.ms/AAla73m are reachable from the Appliance VM. 2. Check firewall/proxy settings` -This error occurs when you run the Azure CLI commands in a 32-bit context, which is the default behavior. The vSphere SDK only supports running in a 64-bit context. The specific error returned from the vSphere SDK is `Unable to import ova of size 6GB using govc`. To resolve the error, install and use Azure CLI 64-bit. +This error is caused when there's a clock/time difference between ESXi host(s) and the management machine where the deployment commands for Arc resource bridge are being executed. To resolve this issue, turn on NTP time sync on the ESXi host(s) and confirm that the management machine is also synced to NTP, then try the deployment again. ### Error during host configuration -When you deploy the resource bridge on VMware vCenter, if you have been using the same template to deploy and delete the appliance multiple times, you might encounter the following error: +If you have been using the same template to deploy and delete the Arc resource bridge multiple times, you might encounter the following error: -`Appliance cluster deployment failed with error: -Error: An error occurred during host configuration` +`Appliance cluster deployment failed with error: Error: An error occurred during host configuration` -To resolve this issue, delete the existing template manually. Then run [`az arcappliance prepare`](/cli/azure/arcappliance/prepare) to download a new template for deployment. +To resolve this issue, manually delete the existing template. Then run [`az arcappliance prepare`](/cli/azure/arcappliance/prepare) to download a new template for deployment. ### Unable to find folders -When deploying the resource bridge on VMware vCenter, you specify the folder in which the template and VM will be created. The folder must be VM and template folder type. Other types of folder, such as storage folders, network folders, or host and cluster folders, can't be used by the resource bridge deployment. +When deploying the resource bridge on VMware vCenter, you specify the folder in which the template and VM will be created. The folder must be VM and template folder type. Other types of folder, such as storage folders, network folders, or host and cluster folders, can't be used for the resource bridge deployment. ### Insufficient permissions |
| azure-cache-for-redis | Cache Azure Active Directory For Authentication | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-cache-for-redis/cache-azure-active-directory-for-authentication.md | To use the ACL integration, your client application must assume the identity of > [!IMPORTANT] > Once the enable operation is complete, the nodes in your cache instance reboots to load the new configuration. We recommend performing this operation during your maintenance window or outside your peak business hours. The operation can take up to 30 minutes. +For information on using Microsoft Entra ID with Azure CLI, see the [references pages for identity](/cli/azure/redis/identity). + ## Using data access configuration with your cache If you would like to use a custom access policy instead of Redis Data Owner, go to the **Data Access Configuration** on the Resource menu. For more information, see [Configure a custom data access policy for your application](cache-configure-role-based-access-control.md#configure-a-custom-data-access-policy-for-your-application). The following table includes links to code samples, which demonstrate how to con - When calling the Redis server `AUTH` command periodically, consider adding a jitter so that the `AUTH` commands are staggered, and your Redis server doesn't receive lot of `AUTH` commands at the same time. -## Next steps +## Related content - [Configure role-based access control with Data Access Policy](cache-configure-role-based-access-control.md)+- [Reference pages for identity](/cli/azure/redis/identity) + |
| azure-functions | Functions Add Output Binding Cosmos Db Vs Code | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-functions/functions-add-output-binding-cosmos-db-vs-code.md | Now, you create an Azure Cosmos DB account as a [serverless account type](../cos |Prompt| Selection| |--|--|- |**Select an Azure Database Server**| Choose **Core (SQL)** to create a document database that you can query by using a SQL syntax. [Learn more about the Azure Cosmos DB](../cosmos-db/introduction.md). | + |**Select an Azure Database Server**| Choose **Core (NoSQL)** to create a document database that you can query by using a SQL syntax or a Query Copilot ([Preview](../cosmos-db/nosql/query/how-to-enable-use-copilot.md)) converting natural language prompts to queries. [Learn more about the Azure Cosmos DB](../cosmos-db/introduction.md). | |**Account name**| Enter a unique name to identify your Azure Cosmos DB account. The account name can use only lowercase letters, numbers, and hyphens (-), and must be between 3 and 31 characters long.| |**Select a capacity model**| Select **Serverless** to create an account in [serverless](../cosmos-db/serverless.md) mode. |**Select a resource group for new resources**| Choose the resource group where you created your function app in the [previous article](./create-first-function-vs-code-csharp.md). | |
| azure-maps | Tutorial Iot Hub Maps | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-maps/tutorial-iot-hub-maps.md | To learn more about how to send device-to-cloud telemetry, and the other way aro [C# script]: https://github.com/Azure-Samples/iothub-to-azure-maps-geofencing/blob/master/src/Azure%20Function/run.csx [create a storage account]: ../storage/common/storage-account-create.md?tabs=azure-portal [Create an Azure storage account]: #create-an-azure-storage-account-[create an IoT hub]: ../iot-develop/quickstart-send-telemetry-iot-hub.md?pivots=programming-language-csharp#create-an-iot-hub +[create an IoT hub]: ../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-csharp#create-an-iot-hub [Create a function and add an Event Grid subscription]: #create-a-function-and-add-an-event-grid-subscription [free account]: https://azure.microsoft.com/free/ [general-purpose v2 storage account]: ../storage/common/storage-account-overview.md To learn more about how to send device-to-cloud telemetry, and the other way aro [resource group]: ../azure-resource-manager/management/manage-resource-groups-portal.md#create-resource-groups [the root of the sample]: https://github.com/Azure-Samples/iothub-to-azure-maps-geofencing [Search Address Reverse]: /rest/api/maps/search/getsearchaddressreverse?view=rest-maps-1.0&preserve-view=true-[Send telemetry from a device]: ../iot-develop/quickstart-send-telemetry-iot-hub.md?pivots=programming-language-csharp +[Send telemetry from a device]: ../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-csharp [Spatial Geofence Get API]: /rest/api/maps/spatial/getgeofence [subscription key]: quick-demo-map-app.md#get-the-subscription-key-for-your-account [Upload a geofence into your Azure storage account]: #upload-a-geofence-into-your-azure-storage-account |
| azure-maps | Understanding Azure Maps Transactions | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-maps/understanding-azure-maps-transactions.md | The following table summarizes the Azure Maps services that generate transaction | Data service (Deprecated<sup>1</sup>) | Yes, except for `MapDataStorageService.GetDataStatus` and `MapDataStorageService.GetUserData`, which are nonbillable| One request = 1 transaction| <ul><li>Location Insights Data (Gen2 pricing)</li></ul>| | [Data registry] | Yes | One request = 1 transaction| <ul><li>Location Insights Data (Gen2 pricing)</li></ul>| | [Geolocation]| Yes| One request = 1 transaction| <ul><li>Location Insights Geolocation (Gen2 pricing)</li><li>Standard S1 Geolocation Transactions (Gen1 S1 pricing)</li><li>Standard Geolocation Transactions (Gen1 S0 pricing)</li></ul>|-| [Render] | Yes, except for Terra maps (`MapTile.GetTerraTile` and `layer=terra`) which are nonbillable.|<ul><li>15 tiles = 1 transaction</li><li>One request for Get Copyright = 1 transaction</li><li>One request for Get Map Attribution = 1 transaction</li><li>One request for Get Static Map = 1 transaction</li><li>One request for Get Map Tileset = 1 transaction</li></ul> <br> For Creator related usage, see the [Creator table]. |<ul><li>Maps Base Map Tiles (Gen2 pricing)</li><li>Maps Imagery Tiles (Gen2 pricing)</li><li>Maps Static Map Images (Gen2 pricing)</li><li>Maps Weather Tiles (Gen2 pricing)</li><li>Standard Hybrid Aerial Imagery Transactions (Gen1 S0 pricing)</li><li>Standard Aerial Imagery Transactions (Gen1 S0 pricing)</li><li>Standard S1 Aerial Imagery Transactions (Gen1 S1 pricing)</li><li>Standard S1 Hybrid Aerial Imagery Transactions (Gen1 S1 pricing)</li><li>Standard S1 Rendering Transactions (Gen1 S1 pricing)</li><li>Standard S1 Tile Transactions (Gen1 S1 pricing)</li><li>Standard S1 Weather Tile Transactions (Gen1 S1 pricing)</li><li>Standard Tile Transactions (Gen1 S0 pricing)</li><li>Standard Weather Tile Transactions (Gen1 S0 pricing)</li><li>Maps Copyright (Gen2 pricing, Gen1 S0 pricing and Gen1 S1 pricing)</li></ul>| +| [Render] | Yes, except Get Copyright API, Get Attribution API and Terra maps (`MapTile.GetTerraTile` and `layer=terra`) which are nonbillable.|<ul><li>15 tiles = 1 transaction</li><li>One request for Get Copyright = 1 transaction</li><li>One request for Get Map Attribution = 1 transaction</li><li>One request for Get Static Map = 1 transaction</li><li>One request for Get Map Tileset = 1 transaction</li></ul> <br> For Creator related usage, see the [Creator table]. |<ul><li>Maps Base Map Tiles (Gen2 pricing)</li><li>Maps Imagery Tiles (Gen2 pricing)</li><li>Maps Static Map Images (Gen2 pricing)</li><li>Maps Weather Tiles (Gen2 pricing)</li><li>Standard Hybrid Aerial Imagery Transactions (Gen1 S0 pricing)</li><li>Standard Aerial Imagery Transactions (Gen1 S0 pricing)</li><li>Standard S1 Aerial Imagery Transactions (Gen1 S1 pricing)</li><li>Standard S1 Hybrid Aerial Imagery Transactions (Gen1 S1 pricing)</li><li>Standard S1 Rendering Transactions (Gen1 S1 pricing)</li><li>Standard S1 Tile Transactions (Gen1 S1 pricing)</li><li>Standard S1 Weather Tile Transactions (Gen1 S1 pricing)</li><li>Standard Tile Transactions (Gen1 S0 pricing)</li><li>Standard Weather Tile Transactions (Gen1 S0 pricing)</li><li>Maps Copyright (Gen2 pricing, Gen1 S0 pricing and Gen1 S1 pricing)</li></ul>| | [Route] | Yes | One request = 1 transaction<br><ul><li>If using the Route Matrix, each cell in the Route Matrix request generates a billable Route transaction.</li><li>If using Batch Directions, each origin/destination coordinate pair in the Batch request call generates a billable Route transaction. Note, the billable Route transaction usage results generated by the batch request has **-Batch** appended to the API name of your Azure portal metrics report.</li></ul> | <ul><li>Location Insights Routing (Gen2 pricing)</li><li>Standard S1 Routing Transactions (Gen1 S1 pricing)</li><li>Standard Services API Transactions (Gen1 S0 pricing)</li></ul> | | [Search v1]<br>[Search v2] | Yes | One request = 1 transaction.<br><ul><li>If using Batch Search, each location in the Batch request generates a billable Search transaction. Note, the billable Search transaction usage results generated by the batch request has **-Batch** appended to the API name of your Azure portal metrics report.</li></ul> | <ul><li>Location Insights Search</li><li>Standard S1 Search Transactions (Gen1 S1 pricing)</li><li>Standard Services API Transactions (Gen1 S0 pricing)</li></ul> | | [Spatial] | Yes, except for `Spatial.GetBoundingBox`, `Spatial.PostBoundingBox` and `Spatial.PostPointInPolygonBatch`, which are nonbillable.| One request = 1 transaction.<br><ul><li>If using Geofence, five requests = 1 transaction</li></ul> | <ul><li>Location Insights Spatial Calculations (Gen2 pricing)</li><li>Standard S1 Spatial Transactions (Gen1 S1 pricing)</li></ul> | |
| azure-monitor | Alerts Automatic Migration | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/alerts/alerts-automatic-migration.md | - Title: Understand how the automatic migration process for your Azure Monitor classic alerts works -description: Learn how the automatic migration process works. --- Previously updated : 06/20/2023---# Understand the automatic migration process for your classic alert rules --As [previously announced](monitoring-classic-retirement.md), classic alerts in Azure Monitor are retired for public cloud users, though still in limited use until **31 May 2021**. Classic alerts for Azure Government cloud and Microsoft Azure operated by 21Vianet will retire on **29 February 2024**. --A migration tool is available in the Azure portal for customers to trigger migration themselves. This article explains the automatic migration process in public cloud, that will start after 31 May 2021. It also details issues and solutions you might run into. --## Important things to note --The migration process converts classic alert rules to new, equivalent alert rules, and creates action groups. In preparation, be aware of the following points: --- The notification payload formats for new alert rules are different from payloads of the classic alert rules because they support more features. If you have a classic alert rule with logic apps, runbooks, or webhooks, they might stop functioning as expected after migration, because of differences in payload. [Learn how to prepare for the migration](alerts-prepare-migration.md).--- Some classic alert rules can't be migrated by using the tool. [Learn which rules can't be migrated and what to do with them](alerts-understand-migration.md#manually-migrating-classic-alerts-to-newer-alerts).--## What will happen during the automatic migration process in public cloud? --- Starting 31 May 2021, you won't be able to create any new classic alert rules and migration of classic alerts will be triggered in batches.-- Any classic alert rules that are monitoring deleted target resources or on [metrics that are no longer supported](alerts-understand-migration.md#classic-alert-rules-on-deprecated-metrics) are considered invalid.-- Classic alert rules that are invalid will be removed sometime after 31 May 2021.-- Once migration for your subscription starts, it should be complete within an hour. Customers can monitor the status of migration on [the migration tool in Azure Monitor](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/MigrationBladeViewModel).-- Subscription owners will receive an email on success or failure of the migration.-- > [!NOTE] - > If you don't want to wait for the automatic migration process to start, you can still trigger the migration voluntarily using the migration tool. --## What if the automatic migration fails? --When the automatic migration process fails, subscription owners will receive an email notifying them of the issue. You can use the migration tool in Azure Monitor to see the full details of the issue. See the [troubleshooting guide](alerts-understand-migration.md#common-problems-and-remedies) for help with problems you might face during migration. -- > [!NOTE] - > In case an action is needed from customers, like temporarily disabling a resource lock or changing a policy assignment, customers will need to resolve any such issues. If the issues are not resolved by then, successful migration of your classic alerts cannot be guaranteed. --## Next steps --- [Prepare for the migration](alerts-prepare-migration.md)-- [Understand how the migration tool works](alerts-understand-migration.md) |
| azure-monitor | Alerts Classic Portal | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/alerts/alerts-classic-portal.md | - Title: Create and manage classic metric alerts using Azure Monitor -description: Learn how to use Azure portal or PowerShell to create, view and manage classic metric alert rules. --- Previously updated : 06/20/2023----# Create, view, and manage classic metric alerts using Azure Monitor --> [!WARNING] -> This article describes how to create older classic metric alerts. Azure Monitor now supports [newer near-real time metric alerts and a new alerts experience](./alerts-overview.md). Classic alerts are [retired](./monitoring-classic-retirement.md) for public cloud users. Classic alerts for Azure Government cloud and Microsoft Azure operated by 21Vianet will retire on **29 February 2024**. -> --Classic metric alerts in Azure Monitor provide a way to get notified when one of your metrics crosses a threshold. Classic metric alerts is an older functionality that allows for alerting only on non-dimensional metrics. There's an existing newer functionality called Metric alerts, which has improved functionality over classic metric alerts. You can learn more about the new metric alerts functionality in [metric alerts overview](./alerts-metric-overview.md). In this article, we'll describe how to create, view and manage classic metric alert rules through Azure portal and PowerShell. --## With Azure portal --1. In the [portal](https://portal.azure.com/), locate the resource that you want to monitor, and then select it. --2. In the **MONITORING** section, select **Alerts (Classic)**. The text and icon might vary slightly for different resources. If you don't find **Alerts (Classic)** here, you might find it in **Alerts** or **Alert Rules**. -- :::image type="content" source="media/alerts-classic-portal/AlertRulesButton.png" lightbox="media/alerts-classic-portal/AlertRulesButton.png" alt-text="Monitoring"::: --3. Select the **Add metric alert (classic)** command, and then fill in the fields. -- :::image type="content" source="media/alerts-classic-portal/AddAlertOnlyParamsPage.png" lightbox="media/alerts-classic-portal/AddAlertOnlyParamsPage.png" alt-text="Add Alert"::: --4. **Name** your alert rule. Then choose a **Description**, which also appears in notification emails. --5. Select the **Metric** that you want to monitor. Then choose a **Condition** and **Threshold** value for the metric. Also choose the **Period** of time that the metric rule must be satisfied before the alert triggers. For example, if you use the period "Over the last 5 minutes" and your alert looks for a CPU above 80%, the alert triggers when the CPU has been consistently above 80% for 5 minutes. After the first trigger occurs, it triggers again when the CPU stays below 80% for 5 minutes. The CPU metric measurement happens every minute. --6. Select **Email owners...** if you want administrators and co-administrators to receive email notifications when the alert fires. --7. If you want to send notifications to additional email addresses when the alert fires, add them in the **Additional Administrator email(s)** field. Separate multiple emails with semicolons, in the following format: *email\@contoso.com;email2\@contoso.com* --8. Put in a valid URI in the **Webhook** field if you want it to be called when the alert fires. --9. If you use Azure Automation, you can select a runbook to be run when the alert fires. --10. Select **OK** to create the alert. --Within a few minutes, the alert is active and triggers as previously described. --After you create an alert, you can select it and do one of the following tasks: --* View a graph that shows the metric threshold and the actual values from the previous day. -* Edit or delete it. -* **Disable** or **Enable** it if you want to temporarily stop or resume receiving notifications for that alert. --## With PowerShell ---This section shows how to use PowerShell commands create, view and manage classic metric alerts.The examples in the article illustrate how you can use Azure Monitor cmdlets for classic metric alerts. --1. If you haven't already, set up PowerShell to run on your computer. For more information, see [How to Install and Configure PowerShell](/powershell/azure/). You can also review the entire list of Azure Monitor PowerShell cmdlets at [Azure Monitor (Insights) Cmdlets](/powershell/module/az.applicationinsights). --2. First, log in to your Azure subscription. -- ```powershell - Connect-AzAccount - ``` --3. You'll see a sign in screen. Once you sign in your Account, TenantID, and default Subscription ID are displayed. All the Azure cmdlets work in the context of your default subscription. To view the list of subscriptions you have access to, use the following command: -- ```powershell - Get-AzSubscription - ``` --4. To change your working context to a different subscription, use the following command: -- ```powershell - Set-AzContext -SubscriptionId <subscriptionid> - ``` --5. You can retrieve all classic metric alert rules on a resource group: -- ```powershell - Get-AzAlertRule -ResourceGroup montest - ``` --6. You can view details of a classic metric alert rule -- ```powershell - Get-AzAlertRule -Name simpletestCPU -ResourceGroup montest -DetailedOutput - ``` --7. You can retrieve all alert rules set for a target resource. For example, all alert rules set on a VM. -- ```powershell - Get-AzAlertRule -ResourceGroup montest -TargetResourceId /subscriptions/s1/resourceGroups/montest/providers/Microsoft.Compute/virtualMachines/testconfig - ``` --8. Classic alert rules can no longer be created via PowerShell. Use the new ['Add-AzMetricAlertRuleV2'](/powershell/module/az.monitor/add-azmetricalertrulev2) command to create a metric alert rule instead. --## Next steps --- [Create a classic metric alert with a Resource Manager template](./alerts-enable-template.md).-- [Have a classic metric alert notify a non-Azure system using a webhook](./alerts-webhooks.md). |
| azure-monitor | Alerts Classic.Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/alerts/alerts-classic.overview.md | - Title: Overview of classic alerts in Azure Monitor -description: Classic alerts will be deprecated. Alerts enable you to monitor Azure resource metrics, events, or logs, and they notify you when a condition you specify is met. --- Previously updated : 06/20/2023---# What are classic alerts in Azure? --> [!NOTE] -> This article describes how to create older classic metric alerts. Azure Monitor now supports [near real time metric alerts and a new alerts experience](./alerts-overview.md). Classic alerts are [retired](./monitoring-classic-retirement.md) for public cloud users. Classic alerts for Azure Government cloud and Microsoft Azure operated by 21Vianet will retire on **February 29, 2024**. -> --Alerts allow you to configure conditions over data, and they notify you when the conditions match the latest monitoring data. --## Old and new alerting capabilities --In the past, Azure Monitor, Application Insights, Log Analytics, and Service Health had separate alerting capabilities. Over time, Azure improved and combined both the user interface and different methods of alerting. The consolidation is still in process. --You can view classic alerts only on the classic alerts user screen in the Azure portal. To see this screen, select **View classic alerts** on the **Alerts** screen. -- :::image type="content" source="media/alerts-classic.overview/monitor-alert-screen2.png" lightbox="media/alerts-classic.overview/monitor-alert-screen2.png" alt-text="Screenshot that shows alert choices in the Azure portal."::: --The new alerts user experience has the following benefits over the classic alerts experience: -- **Better notification system:** All newer alerts use action groups. You can reuse these named groups of notifications and actions in multiple alerts. Classic metric alerts and older Log Analytics alerts don't use action groups.-- **A unified authoring experience:** All alert creation for metrics, logs, and activity logs across Azure Monitor, Log Analytics, and Application Insights is in one place.-- **View fired Log Analytics alerts in the Azure portal:** You can now also see fired Log Analytics alerts in your subscription. Previously, these alerts were in a separate portal.-- **Separation of fired alerts and alert rules:** Alert rules (the definition of condition that triggers an alert) and fired alerts (an instance of the alert rule firing) are differentiated. Now the operational and configuration views are separated.-- **Better workflow:** The new alerts authoring experience guides the user along the process of configuring an alert rule. This change makes it simpler to discover the right things to get alerted on.-- **Smart alerts consolidation and setting alert state:** Newer alerts include auto grouping functionality that shows similar alerts together to reduce overload in the user interface.--The newer metric alerts have the following benefits over the classic metric alerts: -- **Improved latency:** Newer metric alerts can run as frequently as every minute. Older metric alerts always run at a frequency of 5 minutes. Newer alerts have increasing smaller delay from issue occurrence to notification or action (3 to 5 minutes). Older alerts are 5 to 15 minutes depending on the type. Log alerts typically have a delay of 10 minutes to 15 minutes because of the time it takes to ingest the logs. Newer processing methods are reducing that time.-- **Support for multidimensional metrics:** You can alert on dimensional metrics. Now you can monitor an interesting segment of the metric.-- **More control over metric conditions:** You can define richer alert rules. The newer alerts support monitoring the maximum, minimum, average, and total values of metrics.-- **Combined monitoring of multiple metrics:** You can monitor multiple metrics (currently, up to two metrics) with a single rule. An alert triggers if both metrics breach their respective thresholds for the specified time period.-- **Better notification system:** All newer alerts use [action groups](./action-groups.md). You can reuse these named groups of notifications and actions in multiple alerts. Classic metric alerts and older Log Analytics alerts don't use action groups.-- **Metrics from logs (preview):** You can now extract and convert log data that goes into Log Analytics into Azure Monitor metrics and then alert on it like other metrics. For the terminology specific to classic alerts, see [Alerts (classic)]().--## Classic alerts on Azure Monitor data -Two types of classic alerts are available: --* **Classic metric alerts**: This alert triggers when the value of a specified metric crosses a threshold that you assign. The alert generates a notification when that threshold is crossed and the alert condition is met. At that point, the alert is considered "Activated." It generates another notification when it's "Resolved," that is, when the threshold is crossed again and the condition is no longer met. -* **Classic activity log alerts**: A streaming log alert that triggers on an activity log event entry that matches your filter criteria. These alerts have only one state: "Activated." The alert engine applies the filter criteria to any new event. It doesn't search to find older entries. These alerts can notify you when a new Service Health incident occurs or when a user or application performs an operation in your subscription. An example of an operation might be "Delete virtual machine." --For resource log data available through Azure Monitor, route the data into Log Analytics and use a log query alert. Log Analytics now uses the [new alerting method](./alerts-overview.md). --The following diagram summarizes sources of data in Azure Monitor and, conceptually, how you can alert off of that data. ---## Taxonomy of alerts (classic) -Azure uses the following terms to describe classic alerts and their functions: -* **Alert**: A definition of criteria (one or more rules or conditions) that becomes activated when met. -* **Active**: The state when the criteria defined by a classic alert are met. -* **Resolved**: The state when the criteria defined by a classic alert are no longer met after they were previously met. -* **Notification**: The action taken based off of a classic alert becoming active. -* **Action**: A specific call sent to a receiver of a notification (for example, emailing an address or posting to a webhook URL). Notifications can usually trigger multiple actions. --## How do I receive a notification from an Azure Monitor classic alert? -Historically, Azure alerts from different services used their own built-in notification methods. --Azure Monitor created a reusable notification grouping called *action groups*. Action groups specify a set of receivers for a notification. Any time an alert is activated that references the action group, all receivers receive that notification. With action groups, you can reuse a grouping of receivers (for example, your on-call engineer list) across many alert objects. --Action groups support notification by posting to a webhook URL and to email addresses, SMS numbers, and several other actions. For more information, see [Action groups](./action-groups.md). --Older classic activity log alerts use action groups. But the older metric alerts don't use action groups. Instead, you can configure the following actions: --- Send email notifications to the service administrator, co-administrators, or other email addresses that you specify.-- Call a webhook, which enables you to launch other automation actions.--Webhooks enable automation and remediation, for example, by using: -- Azure Automation runbooks-- Azure Functions-- Azure Logic Apps-- A third-party service--## Next steps -Get information about alert rules and how to configure them: --* Learn more about [metrics](../data-platform.md). -* Configure [classic metric alerts via the Azure portal](alerts-classic-portal.md). -* Configure [classic metric alerts via PowerShell](alerts-classic-portal.md). -* Configure [classic metric alerts via the command-line interface (CLI)](alerts-classic-portal.md). -* Configure [classic metric alerts via the Azure Monitor REST API](/rest/api/monitor/alertrules). -* Learn more about [activity logs](../essentials/platform-logs-overview.md). -* Configure [activity log alerts via the Azure portal](./activity-log-alerts.md). -* Configure [activity log alerts via Azure Resource Manager](./alerts-activity-log.md). -* Review the [activity log alert webhook schema](activity-log-alerts-webhook.md). -* Learn more about [action groups](./action-groups.md). -* Configure [newer alerts](alerts-metric.md). |
| azure-monitor | Alerts Enable Template | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/alerts/alerts-enable-template.md | - Title: Resource Manager template - create metric alert -description: Learn how to use a Resource Manager template to create a classic metric alert to receive notifications by email or webhook. -- Previously updated : 05/28/2023----# Create a classic metric alert rule with a Resource Manager template --> [!WARNING] -> This article describes how to create older classic metric alert rules. Azure Monitor now supports [newer near-real time metric alerts and a new alerts experience](./alerts-overview.md). Classic alerts are [retired](./monitoring-classic-retirement.md) for public cloud users. Classic alerts for Azure Government cloud and Microsoft Azure operated by 21Vianet will retire on **29 February 2024**. -> --This article shows how you can use an [Azure Resource Manager template](../../azure-resource-manager/templates/syntax.md) to configure Azure classic metric alert rules. This enables you to automatically set up alert rules on your resources when they are created to ensure that all resources are monitored correctly. --The basic steps are as follows: --1. Create a template as a JSON file that describes how to create the alert rule. -2. [Deploy the template using any deployment method](../../azure-resource-manager/templates/deploy-powershell.md). --Below we describe how to create a Resource Manager template first for an alert rule alone, then for an alert rule during the creation of another resource. --## Resource Manager template for a classic metric alert rule -To create an alert rule using a Resource Manager template, you create a resource of type `Microsoft.Insights/alertRules` and fill in all related properties. Below is a template that creates an alert rule. --```json -{ - "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "parameters": { - "alertName": { - "type": "string", - "metadata": { - "description": "Name of alert" - } - }, - "alertDescription": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Description of alert" - } - }, - "isEnabled": { - "type": "bool", - "defaultValue": true, - "metadata": { - "description": "Specifies whether alerts are enabled" - } - }, - "resourceId": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Resource ID of the resource emitting the metric that will be used for the comparison." - } - }, - "metricName": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Name of the metric used in the comparison to activate the alert." - } - }, - "operator": { - "type": "string", - "defaultValue": "GreaterThan", - "allowedValues": [ - "GreaterThan", - "GreaterThanOrEqual", - "LessThan", - "LessThanOrEqual" - ], - "metadata": { - "description": "Operator comparing the current value with the threshold value." - } - }, - "threshold": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "The threshold value at which the alert is activated." - } - }, - "aggregation": { - "type": "string", - "defaultValue": "Average", - "allowedValues": [ - "Average", - "Last", - "Maximum", - "Minimum", - "Total" - ], - "metadata": { - "description": "How the data that is collected should be combined over time." - } - }, - "windowSize": { - "type": "string", - "defaultValue": "PT5M", - "metadata": { - "description": "Period of time used to monitor alert activity based on the threshold. Must be between five minutes and one day. ISO 8601 duration format." - } - }, - "sendToServiceOwners": { - "type": "bool", - "defaultValue": true, - "metadata": { - "description": "Specifies whether alerts are sent to service owners" - } - }, - "customEmailAddresses": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "Comma-delimited email addresses where the alerts are also sent" - } - }, - "webhookUrl": { - "type": "string", - "defaultValue": "", - "metadata": { - "description": "URL of a webhook that will receive an HTTP POST when the alert activates." - } - } - }, - "variables": { - "customEmails": "[split(parameters('customEmailAddresses'), ',')]" - }, - "resources": [ - { - "type": "Microsoft.Insights/alertRules", - "name": "[parameters('alertName')]", - "location": "[resourceGroup().location]", - "apiVersion": "2016-03-01", - "properties": { - "name": "[parameters('alertName')]", - "description": "[parameters('alertDescription')]", - "isEnabled": "[parameters('isEnabled')]", - "condition": { - "odata.type": "Microsoft.Azure.Management.Insights.Models.ThresholdRuleCondition", - "dataSource": { - "odata.type": "Microsoft.Azure.Management.Insights.Models.RuleMetricDataSource", - "resourceUri": "[parameters('resourceId')]", - "metricName": "[parameters('metricName')]" - }, - "operator": "[parameters('operator')]", - "threshold": "[parameters('threshold')]", - "windowSize": "[parameters('windowSize')]", - "timeAggregation": "[parameters('aggregation')]" - }, - "actions": [ - { - "odata.type": "Microsoft.Azure.Management.Insights.Models.RuleEmailAction", - "sendToServiceOwners": "[parameters('sendToServiceOwners')]", - "customEmails": "[variables('customEmails')]" - }, - { - "odata.type": "Microsoft.Azure.Management.Insights.Models.RuleWebhookAction", - "serviceUri": "[parameters('webhookUrl')]", - "properties": {} - } - ] - } - } - ] -} -``` --An explanation of the schema and properties for an alert rule [is available here](/rest/api/monitor/alertrules). --## Resource Manager template for a resource with a classic metric alert rule -An alert rule on a Resource Manager template is most often useful when creating an alert rule while creating a resource. For example, you may want to ensure that a ΓÇ£CPU % > 80ΓÇ¥ rule is set up every time you deploy a Virtual Machine. To do this, you add the alert rule as a resource in the resource array for your VM template and add a dependency using the `dependsOn` property to the VM resource ID. HereΓÇÖs a full example that creates a Windows VM and adds an alert rule that notifies subscription admins when the CPU utilization goes above 80%. --```json -{ - "$schema": "https://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#", - "contentVersion": "1.0.0.0", - "parameters": { - "newStorageAccountName": { - "type": "string", - "metadata": { - "Description": "The name of the storage account where the VM disk is stored." - } - }, - "adminUsername": { - "type": "string", - "metadata": { - "Description": "The name of the administrator account on the VM." - } - }, - "adminPassword": { - "type": "securestring", - "metadata": { - "Description": "The administrator account password on the VM." - } - }, - "dnsNameForPublicIP": { - "type": "string", - "metadata": { - "Description": "The name of the public IP address used to access the VM." - } - } - }, - "variables": { - "location": "Central US", - "imagePublisher": "MicrosoftWindowsServer", - "imageOffer": "WindowsServer", - "windowsOSVersion": "2012-R2-Datacenter", - "OSDiskName": "osdisk1", - "nicName": "nc1", - "addressPrefix": "10.0.0.0/16", - "subnetName": "sn1", - "subnetPrefix": "10.0.0.0/24", - "storageAccountType": "Standard_LRS", - "publicIPAddressName": "ip1", - "publicIPAddressType": "Dynamic", - "vmStorageAccountContainerName": "vhds", - "vmName": "vm1", - "vmSize": "Standard_A0", - "virtualNetworkName": "vn1", - "vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]", - "subnetRef": "[concat(variables('vnetID'),'/subnets/',variables('subnetName'))]", - "vmID":"[resourceId('Microsoft.Compute/virtualMachines',variables('vmName'))]", - "alertName": "highCPUOnVM", - "alertDescription":"CPU is over 80%", - "alertIsEnabled": true, - "resourceId": "", - "metricName": "Percentage CPU", - "operator": "GreaterThan", - "threshold": "80", - "windowSize": "PT5M", - "aggregation": "Average", - "customEmails": "", - "sendToServiceOwners": true, - "webhookUrl": "http://testwebhook.test" - }, - "resources": [ - { - "type": "Microsoft.Storage/storageAccounts", - "name": "[parameters('newStorageAccountName')]", - "apiVersion": "2015-06-15", - "location": "[variables('location')]", - "properties": { - "accountType": "[variables('storageAccountType')]" - } - }, - { - "apiVersion": "2016-03-30", - "type": "Microsoft.Network/publicIPAddresses", - "name": "[variables('publicIPAddressName')]", - "location": "[variables('location')]", - "properties": { - "publicIPAllocationMethod": "[variables('publicIPAddressType')]", - "dnsSettings": { - "domainNameLabel": "[parameters('dnsNameForPublicIP')]" - } - } - }, - { - "apiVersion": "2016-03-30", - "type": "Microsoft.Network/virtualNetworks", - "name": "[variables('virtualNetworkName')]", - "location": "[variables('location')]", - "properties": { - "addressSpace": { - "addressPrefixes": [ - "[variables('addressPrefix')]" - ] - }, - "subnets": [ - { - "name": "[variables('subnetName')]", - "properties": { - "addressPrefix": "[variables('subnetPrefix')]" - } - } - ] - } - }, - { - "apiVersion": "2016-03-30", - "type": "Microsoft.Network/networkInterfaces", - "name": "[variables('nicName')]", - "location": "[variables('location')]", - "dependsOn": [ - "[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]", - "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]" - ], - "properties": { - "ipConfigurations": [ - { - "name": "ipconfig1", - "properties": { - "privateIPAllocationMethod": "Dynamic", - "publicIPAddress": { - "id": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]" - }, - "subnet": { - "id": "[variables('subnetRef')]" - } - } - } - ] - } - }, - { - "apiVersion": "2016-03-30", - "type": "Microsoft.Compute/virtualMachines", - "name": "[variables('vmName')]", - "location": "[variables('location')]", - "dependsOn": [ - "[concat('Microsoft.Storage/storageAccounts/', parameters('newStorageAccountName'))]", - "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'))]" - ], - "properties": { - "hardwareProfile": { - "vmSize": "[variables('vmSize')]" - }, - "osProfile": { - "computername": "[variables('vmName')]", - "adminUsername": "[parameters('adminUsername')]", - "adminPassword": "[parameters('adminPassword')]" - }, - "storageProfile": { - "imageReference": { - "publisher": "[variables('imagePublisher')]", - "offer": "[variables('imageOffer')]", - "sku": "[variables('windowsOSVersion')]", - "version": "latest" - }, - "osDisk": { - "name": "osdisk", - "vhd": { - "uri": "[concat('http://',parameters('newStorageAccountName'),'.blob.core.windows.net/',variables('vmStorageAccountContainerName'),'/',variables('OSDiskName'),'.vhd')]" - }, - "caching": "ReadWrite", - "createOption": "FromImage" - } - }, - "networkProfile": { - "networkInterfaces": [ - { - "id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]" - } - ] - } - } - }, - { - "type": "Microsoft.Insights/alertRules", - "name": "[variables('alertName')]", - "dependsOn": [ - "[variables('vmID')]" - ], - "location": "[variables('location')]", - "apiVersion": "2016-03-01", - "properties": { - "name": "[variables('alertName')]", - "description": "variables('alertDescription')", - "isEnabled": "[variables('alertIsEnabled')]", - "condition": { - "odata.type": "Microsoft.Azure.Management.Insights.Models.ThresholdRuleCondition", - "dataSource": { - "odata.type": "Microsoft.Azure.Management.Insights.Models.RuleMetricDataSource", - "resourceUri": "[variables('vmID')]", - "metricName": "[variables('metricName')]" - }, - "operator": "[variables('operator')]", - "threshold": "[variables('threshold')]", - "windowSize": "[variables('windowSize')]", - "timeAggregation": "[variables('aggregation')]" - }, - "actions": [ - { - "odata.type": "Microsoft.Azure.Management.Insights.Models.RuleEmailAction", - "sendToServiceOwners": "[variables('sendToServiceOwners')]", - "customEmails": "[variables('customEmails')]" - }, - { - "odata.type": "Microsoft.Azure.Management.Insights.Models.RuleWebhookAction", - "serviceUri": "[variables('webhookUrl')]", - "properties": {} - } - ] - } - } - ] -} -``` --## Next Steps -* [Read more about Alerts](./alerts-overview.md) -* [Add Diagnostic Settings](../essentials/resource-manager-diagnostic-settings.md) to your Resource Manager template -* For the JSON syntax and properties, see [Microsoft.Insights/alertrules](/azure/templates/microsoft.insights/alertrules) template reference. |
| azure-monitor | Alerts Prepare Migration | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/alerts/alerts-prepare-migration.md | - Title: Update logic apps & runbooks for alerts migration -description: Learn how to modify your webhooks, logic apps, and runbooks to prepare for voluntary migration. -- Previously updated : 06/20/2023----# Prepare your logic apps and runbooks for migration of classic alert rules --> [!NOTE] -> As [previously announced](monitoring-classic-retirement.md), classic alerts in Azure Monitor are retired for public cloud users, though still in limited use until **31 May 2021**. Classic alerts for Azure Government cloud and Microsoft Azure operated by 21Vianet will retire on **29 February 2024**. -> --If you choose to voluntarily migrate your classic alert rules to new alert rules, there are some differences between the two systems. This article explains those differences and how you can prepare for the change. --## API changes --The APIs that create and manage classic alert rules (`microsoft.insights/alertrules`) are different from the APIs that create and manage new metric alerts (`microsoft.insights/metricalerts`). If you programmatically create and manage classic alert rules today, update your deployment scripts to work with the new APIs. --The following table is a reference to the programmatic interfaces for both classic and new alerts: --| Deployment script type | Classic alerts | New metric alerts | -| - | -- | -- | -|REST API | [microsoft.insights/alertrules](/rest/api/monitor/alertrules) | [microsoft.insights/metricalerts](/rest/api/monitor/metricalerts) | -|Azure CLI | `az monitor alert` | [az monitor metrics alert](/cli/azure/monitor/metrics/alert) | -|PowerShell | [Reference](/powershell/module/az.monitor/add-azmetricalertrule) | [Reference](/powershell/module/az.monitor/add-azmetricalertrulev2) | -| Azure Resource Manager template | [For classic alerts](./alerts-enable-template.md)|[For new metric alerts](./alerts-metric-create-templates.md)| --## Notification payload changes --The notification payload format is slightly different between [classic alert rules](alerts-webhooks.md) and [new metric alerts](alerts-metric-near-real-time.md#payload-schema). If you have classic alert rules with webhook, logic app, or runbook actions, you must update the targets to accept the new payload format. --Use the following table to map the webhook payload fields from the classic format to the new format: --| Notification endpoint type | Classic alerts | New metric alerts | -| -- | -- | -- | -|Was the alert activated or resolved? | **status** | **data.status** | -|Contextual information about the alert | **context** | **data.context** | -|Time stamp at which the alert was activated or resolved | **context.timestamp** | **data.context.timestamp** | -| Alert rule ID | **context.id** | **data.context.id** | -| Alert rule name | **context.name** | **data.context.name** | -| Description of the alert rule | **context.description** | **data.context.description** | -| Alert rule condition | **context.condition** | **data.context.condition** | -| Metric name | **context.condition.metricName** | **data.context.condition.allOf[0].metricName** | -| Time aggregation (how the metric is aggregated over the evaluation window)| **context.condition.timeAggregation** | **context.condition.timeAggregation** | -| Evaluation period | **context.condition.windowSize** | **data.context.condition.windowSize** | -| Operator (how the aggregated metric value is compared against the threshold) | **context.condition.operator** | **data.context.condition.operator** | -| Threshold | **context.condition.threshold** | **data.context.condition.allOf[0].threshold** | -| Metric value | **context.condition.metricValue** | **data.context.condition.allOf[0].metricValue** | -| Subscription ID | **context.subscriptionId** | **data.context.subscriptionId** | -| Resource group of the affected resource | **context.resourceGroup** | **data.context.resourceGroup** | -| Name of the affected resource | **context.resourceName** | **data.context.resourceName** | -| Type of the affected resource | **context.resourceType** | **data.context.resourceType** | -| Resource ID of the affected resource | **context.resourceId** | **data.context.resourceId** | -| Direct link to the portal resource summary page | **context.portalLink** | **data.context.portalLink** | -| Custom payload fields to be passed to the webhook or logic app | **properties** | **data.properties** | --The payloads are similar, as you can see. The following section offers: --- Details about modifying logic apps to work with the new format.-- A runbook example that parses the notification payload for new alerts.--## Modify a logic app to receive a metric alert notification --If you're using logic apps with classic alerts, you must modify your logic-app code to parse the new metric alerts payload. Follow these steps: --1. Create a new logic app. --1. Use the template "Azure Monitor - Metrics Alert Handler". This template has an **HTTP request** trigger with the appropriate schema defined. -- :::image type="content" source="media/alerts-prepare-migration/logic-app-template.png" lightbox="media/alerts-prepare-migration/logic-app-template.png" alt-text="Screenshot shows two buttons, Blank Logic App and Azure Monitor ΓÇô Metrics Alert Handler."::: --1. Add an action to host your processing logic. --## Use an automation runbook that receives a metric alert notification --The following example provides PowerShell code to use in your runbook. This code can parse the payloads for both classic metric alert rules and new metric alert rules. --```PowerShell -## Example PowerShell code to use in a runbook to handle parsing of both classic and new metric alerts. --[OutputType("PSAzureOperationResponse")] --param -( - [Parameter (Mandatory=$false)] - [object] $WebhookData -) --$ErrorActionPreference = "stop" --if ($WebhookData) -{ - # Get the data object from WebhookData. - $WebhookBody = (ConvertFrom-Json -InputObject $WebhookData.RequestBody) -- # Determine whether the alert triggering the runbook is a classic metric alert or a new metric alert (depends on the payload schema). - $schemaId = $WebhookBody.schemaId - Write-Verbose "schemaId: $schemaId" -Verbose - if ($schemaId -eq "AzureMonitorMetricAlert") { -- # This is the new metric alert schema. - $AlertContext = [object] ($WebhookBody.data).context - $status = ($WebhookBody.data).status -- # Parse fields related to alert rule condition. - $metricName = $AlertContext.condition.allOf[0].metricName - $metricValue = $AlertContext.condition.allOf[0].metricValue - $threshold = $AlertContext.condition.allOf[0].threshold - $timeAggregation = $AlertContext.condition.allOf[0].timeAggregation - } - elseif ($schemaId -eq $null) { - # This is the classic metric alert schema. - $AlertContext = [object] $WebhookBody.context - $status = $WebhookBody.status -- # Parse fields related to alert rule condition. - $metricName = $AlertContext.condition.metricName - $metricValue = $AlertContext.condition.metricValue - $threshold = $AlertContext.condition.threshold - $timeAggregation = $AlertContext.condition.timeAggregation - } - else { - # The schema is neither a classic metric alert nor a new metric alert. - Write-Error "The alert data schema - $schemaId - is not supported." - } -- # Parse fields related to resource affected. - $ResourceName = $AlertContext.resourceName - $ResourceType = $AlertContext.resourceType - $ResourceGroupName = $AlertContext.resourceGroupName - $ResourceId = $AlertContext.resourceId - $SubId = $AlertContext.subscriptionId -- ## Your logic to handle the alert here. -} -else { - # Error - Write-Error "This runbook is meant to be started from an Azure alert webhook only." -} --``` --For a full example of a runbook that stops a virtual machine when an alert is triggered, see the [Azure Automation documentation](../../automation/automation-create-alert-triggered-runbook.md). --## Partner integration via webhooks --Most of our partners that integrate with classic alerts already support newer metric alerts through their integrations. Known integrations that already work with new metric alerts include: --- [PagerDuty](https://www.pagerduty.com/docs/guides/azure-integration-guide/)-- [OpsGenie](https://docs.opsgenie.com/docs/microsoft-azure-integration)-- [Signl4](https://www.signl4.com/blog/mobile-alert-notifications-azure-monitor/)--If you're using a partner integration that's not listed here, confirm with the provider that they work with new metric alerts. --## Next steps --- [Understand how the migration tool works](alerts-understand-migration.md) |
| azure-monitor | Alerts Understand Migration | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/alerts/alerts-understand-migration.md | - Title: Understand migration for Azure Monitor alerts -description: Understand how the alerts migration works and troubleshoot problems. -- Previously updated : 06/20/2023----# Understand migration options to newer alerts --Classic alerts are [retired](./monitoring-classic-retirement.md) for public cloud users. Classic alerts for Azure Government cloud and Microsoft Azure operated by 21Vianet will retire on **29 February 2024**. --This article explains how the manual migration and voluntary migration tool work, which will be used to migrate remaining alert rules. It also describes solutions for some common problems. --> [!IMPORTANT] -> Activity log alerts (including Service health alerts) and log search alerts are not impacted by the migration. The migration only applies to classic alert rules described [here](./monitoring-classic-retirement.md#retirement-of-classic-monitoring-and-alerting-platform). --> [!NOTE] -> If your classic alert rules are invalid i.e. they are on [deprecated metrics](#classic-alert-rules-on-deprecated-metrics) or resources that have been deleted, they will not be migrated and will not be available after service is retired. --## Manually migrating classic alerts to newer alerts --Customers that are interested in manually migrating their remaining alerts can already do so using the following sections. It also includes metrics that are retired and so cannot be migrated directly. --### Guest metrics on virtual machines --Before you can create new metric alerts on guest metrics, the guest metrics must be sent to the Azure Monitor logs store. Follow these instructions to create alerts: --- [Enabling guest metrics collection to log analytics](../agents/agent-data-sources.md)-- [Creating log search alerts in Azure Monitor](./alerts-log.md)--There are more options to collect guest metrics and alert on them, [learn more](../agents/agents-overview.md). --### Storage and Classic Storage account metrics --All classic alerts on storage accounts can be migrated except alerts on these metrics: --- PercentAuthorizationError-- PercentClientOtherError-- PercentNetworkError-- PercentServerOtherError-- PercentSuccess-- PercentThrottlingError-- PercentTimeoutError-- AnonymousThrottlingError-- SASThrottlingError-- ThrottlingError--Classic alert rules on Percent metrics must be migrated based on [the mapping between old and new storage metrics](../../storage/common/storage-metrics-migration.md#metrics-mapping-between-old-metrics-and-new-metrics). Thresholds will need to be modified appropriately because the new metric available is an absolute one. --Classic alert rules on AnonymousThrottlingError, SASThrottlingError, and ThrottlingError must be split into two new alerts because there's no combined metric that provides the same functionality. Thresholds will need to be adapted appropriately. --### Azure Cosmos DB metrics --All classic alerts on Azure Cosmos DB metrics can be migrated except alerts on these metrics: --- Average Requests per Second-- Consistency Level-- Http 2xx-- Http 3xx-- Max RUPM Consumed Per Minute-- Max RUs Per Second-- Mongo Other Request Charge-- Mongo Other Request Rate-- Observed Read Latency-- Observed Write Latency-- Service Availability-- Storage Capacity--Average Requests per Second, Consistency Level, Max RUPM Consumed Per Minute, Max RUs Per Second, Observed Read Latency, Observed Write Latency, and Storage Capacity aren't currently available in the [new system](../essentials/metrics-supported.md#microsoftdocumentdbdatabaseaccounts). --Alerts on request metrics like Http 2xx, Http 3xx, and Service Availability aren't migrated because the way requests are counted is different between classic metrics and new metrics. Alerts on these metrics will need to be manually recreated with thresholds adjusted. --### Classic alert rules on deprecated metrics --The following are classic alert rules on metrics that were previously supported but were eventually deprecated. A small percentage of customer might have invalid classic alert rules on such metrics. Since these alert rules are invalid, they won't be migrated. --| Resource type| Deprecated metric(s) | -|-|-- | -| Microsoft.DBforMySQL/servers | compute_consumption_percent, compute_limit | -| Microsoft.DBforPostgreSQL/servers | compute_consumption_percent, compute_limit | -| Microsoft.Network/publicIPAddresses | defaultddostriggerrate | -| Microsoft.SQL/servers/databases | service_level_objective, storage_limit, storage_used, throttling, dtu_consumption_percent, storage_used | -| Microsoft.Web/hostingEnvironments/multirolepools | averagememoryworkingset | -| Microsoft.Web/hostingEnvironments/workerpools | bytesreceived, httpqueuelength | --## How equivalent new alert rules and action groups are created --The migration tool converts your classic alert rules to equivalent new alert rules and action groups. For most classic alert rules, equivalent new alert rules are on the same metric with the same properties such as `windowSize` and `aggregationType`. However, there are some classic alert rules are on metrics that have a different, equivalent metric in the new system. The following principles apply to the migration of classic alerts unless specified in the section below: --- **Frequency**: Defines how often a classic or new alert rule checks for the condition. The `frequency` in classic alert rules wasn't configurable by the user and was always 5 mins for all resource types. Frequency of equivalent rules is also set to 5 min.-- **Aggregation Type**: Defines how the metric is aggregated over the window of interest. The `aggregationType` is also the same between classic alerts and new alerts for most metrics. In some cases, since the metric is different between classic alerts and new alerts, equivalent `aggregationType` or the `primary Aggregation Type` defined for the metric is used.-- **Units**: Property of the metric on which alert is created. Some equivalent metrics have different units. The threshold is adjusted appropriately as needed. For example, if the original metric has seconds as units but equivalent new metric has milliseconds as units, the original threshold is multiplied by 1000 to ensure same behavior.-- **Window Size**: Defines the window over which metric data is aggregated to compare against the threshold. For standard `windowSize` values like 5 mins, 15 mins, 30 mins, 1 hour, 3 hours, 6 hours, 12 hours, 1 day, there is no change made for equivalent new alert rule. For other values, the closest `windowSize` is used. For most customers, there's no effect with this change. For a small percentage of customers, there might be a need to tweak the threshold to get exact same behavior.--In the following sections, we detail the metrics that have a different, equivalent metric in the new system. Any metric that remains the same for classic and new alert rules isn't listed. You can find a list of metrics supported in the new system [here](../essentials/metrics-supported.md). --### Microsoft.Storage/storageAccounts and Microsoft.ClassicStorage/storageAccounts --For Storage account services like blob, table, file, and queue, the following metrics are mapped to equivalent metrics as shown below: --| Metric in classic alerts | Equivalent metric in new alerts | Comments| -|--||| -| AnonymousAuthorizationError| Transactions metric with dimensions "ResponseType"="AuthorizationError" and "Authentication" = "Anonymous"| | -| AnonymousClientOtherError | Transactions metric with dimensions "ResponseType"="ClientOtherError" and "Authentication" = "Anonymous" | | -| AnonymousClientTimeOutError| Transactions metric with dimensions "ResponseType"="ClientTimeOutError" and "Authentication" = "Anonymous" | | -| AnonymousNetworkError | Transactions metric with dimensions "ResponseType"="NetworkError" and "Authentication" = "Anonymous" | | -| AnonymousServerOtherError | Transactions metric with dimensions "ResponseType"="ServerOtherError" and "Authentication" = "Anonymous" | | -| AnonymousServerTimeOutError | Transactions metric with dimensions "ResponseType"="ServerTimeOutError" and "Authentication" = "Anonymous" | | -| AnonymousSuccess | Transactions metric with dimensions "ResponseType"="Success" and "Authentication" = "Anonymous" | | -| AuthorizationError | Transactions metric with dimensions "ResponseType"="AuthorizationError" | | -| AverageE2ELatency | SuccessE2ELatency | | -| AverageServerLatency | SuccessServerLatency | | -| Capacity | BlobCapacity | Use `aggregationType` 'average' instead of 'last'. Metric only applies to Blob services | -| ClientOtherError | Transactions metric with dimensions "ResponseType"="ClientOtherError" | | -| ClientTimeoutError | Transactions metric with dimensions "ResponseType"="ClientTimeOutError" | | -| ContainerCount | ContainerCount | Use `aggregationType` 'average' instead of 'last'. Metric only applies to Blob services | -| NetworkError | Transactions metric with dimensions "ResponseType"="NetworkError" | | -| ObjectCount | BlobCount| Use `aggregationType` 'average' instead of 'last'. Metric only applies to Blob services | -| SASAuthorizationError | Transactions metric with dimensions "ResponseType"="AuthorizationError" and "Authentication" = "SAS" | | -| SASClientOtherError | Transactions metric with dimensions "ResponseType"="ClientOtherError" and "Authentication" = "SAS" | | -| SASClientTimeOutError | Transactions metric with dimensions "ResponseType"="ClientTimeOutError" and "Authentication" = "SAS" | | -| SASNetworkError | Transactions metric with dimensions "ResponseType"="NetworkError" and "Authentication" = "SAS" | | -| SASServerOtherError | Transactions metric with dimensions "ResponseType"="ServerOtherError" and "Authentication" = "SAS" | | -| SASServerTimeOutError | Transactions metric with dimensions "ResponseType"="ServerTimeOutError" and "Authentication" = "SAS" | | -| SASSuccess | Transactions metric with dimensions "ResponseType"="Success" and "Authentication" = "SAS" | | -| ServerOtherError | Transactions metric with dimensions "ResponseType"="ServerOtherError" | | -| ServerTimeOutError | Transactions metric with dimensions "ResponseType"="ServerTimeOutError" | | -| Success | Transactions metric with dimensions "ResponseType"="Success" | | -| TotalBillableRequests| Transactions | | -| TotalEgress | Egress | | -| TotalIngress | Ingress | | -| TotalRequests | Transactions | | --### Microsoft.DocumentDB/databaseAccounts --For Azure Cosmos DB, equivalent metrics are as shown below: --| Metric in classic alerts | Equivalent metric in new alerts | Comments| -|--||| -| AvailableStorage | AvailableStorage|| -| Data Size | DataUsage| | -| Document Count | DocumentCount|| -| Index Size | IndexUsage|| -| Service Unavailable | ServiceAvailability|| -| TotalRequestUnits | TotalRequestUnits|| -| Throttled Requests | TotalRequests with dimension "StatusCode" = "429"| 'Average' aggregation type is corrected to 'Count'| -| Internal Server Errors | TotalRequests with dimension "StatusCode" = "500"}| 'Average' aggregation type is corrected to 'Count'| -| Http 401 | TotalRequests with dimension "StatusCode" = "401"| 'Average' aggregation type is corrected to 'Count'| -| Http 400 | TotalRequests with dimension "StatusCode" = "400"| 'Average' aggregation type is corrected to 'Count'| -| Total Requests | TotalRequests| 'Max' aggregation type is corrected to 'Count'| -| Mongo Count Request Charge| MongoRequestCharge with dimension "CommandName" = "count"|| -| Mongo Count Request Rate | MongoRequestsCount with dimension "CommandName" = "count"|| -| Mongo Delete Request Charge | MongoRequestCharge with dimension "CommandName" = "delete"|| -| Mongo Delete Request Rate | MongoRequestsCount with dimension "CommandName" = "delete"|| -| Mongo Insert Request Charge | MongoRequestCharge with dimension "CommandName" = "insert"|| -| Mongo Insert Request Rate | MongoRequestsCount with dimension "CommandName" = "insert"|| -| Mongo Query Request Charge | MongoRequestCharge with dimension "CommandName" = "find"|| -| Mongo Query Request Rate | MongoRequestsCount with dimension "CommandName" = "find"|| -| Mongo Update Request Charge | MongoRequestCharge with dimension "CommandName" = "update"|| -| Mongo Insert Failed Requests | MongoRequestCount with dimensions "CommandName" = "insert" and "Status" = "failed"| 'Average' aggregation type is corrected to 'Count'| -| Mongo Query Failed Requests | MongoRequestCount with dimensions "CommandName" = "query" and "Status" = "failed"| 'Average' aggregation type is corrected to 'Count'| -| Mongo Count Failed Requests | MongoRequestCount with dimensions "CommandName" = "count" and "Status" = "failed"| 'Average' aggregation type is corrected to 'Count'| -| Mongo Update Failed Requests | MongoRequestCount with dimensions "CommandName" = "update" and "Status" = "failed"| 'Average' aggregation type is corrected to 'Count'| -| Mongo Other Failed Requests | MongoRequestCount with dimensions "CommandName" = "other" and "Status" = "failed"| 'Average' aggregation type is corrected to 'Count'| -| Mongo Delete Failed Requests | MongoRequestCount with dimensions "CommandName" = "delete" and "Status" = "failed"| 'Average' aggregation type is corrected to 'Count'| --### How equivalent action groups are created --Classic alert rules had email, webhook, logic app, and runbook actions tied to the alert rule itself. New alert rules use action groups that can be reused across multiple alert rules. The migration tool creates single action group for same actions no matter of how many alert rules are using the action. Action groups created by the migration tool use the naming format 'Migrated_AG*'. --> [!NOTE] -> Classic alerts sent localized emails based on the locale of classic administrator when used to notify classic administrator roles. New alert emails are sent via Action Groups and are only in English. --## Rollout phases --The migration tool is rolling out in phases to customers that use classic alert rules. Subscription owners will receive an email when the subscription is ready to be migrated by using the tool. --> [!NOTE] -> Because the tool is being rolled out in phases, you might see that some of your subscriptions are not yet ready to be migrated during the early phases. --Most of the subscriptions are currently marked as ready for migration. Only subscriptions that have classic alerts on following resource types are still not ready for migration. --- Microsoft.classicCompute/domainNames/slots/roles-- Microsoft.insights/components--## Who can trigger the migration? --Any user who has the built-in role of Monitoring Contributor at the subscription level can trigger the migration. Users who have a custom role with the following permissions can also trigger the migration: --- */read-- Microsoft.Insights/actiongroups/*-- Microsoft.Insights/AlertRules/*-- Microsoft.Insights/metricAlerts/*-- Microsoft.AlertsManagement/smartDetectorAlertRules/*--> [!NOTE] -> In addition to having above permissions, your subscription should additionally be registered with Microsoft.AlertsManagement resource provider. This is required to successfully migrate Failure Anomaly alerts on Application Insights. --## Common problems and remedies --After you trigger the migration, you'll receive email at the addresses you provided to notify you that migration is complete or if any action is needed from you. This section describes some common problems and how to deal with them. --### Validation failed --Because of some recent changes to classic alert rules in your subscription, the subscription cannot be migrated. This problem is temporary. You can restart the migration after the migration status moves back **Ready for migration** in a few days. --### Scope lock preventing us from migrating your rules --As part of the migration, new metric alerts and new action groups will be created, and then classic alert rules will be deleted. However, a scope lock can prevent us from creating or deleting resources. Depending on the scope lock, some or all rules couldn't be migrated. You can resolve this problem by removing the scope lock for the subscription, resource group, or resource, which is listed in the [migration tool](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/MigrationBladeViewModel), and triggering the migration again. Scope lock can't be disabled and must be removed during the migration process. [Learn more about managing scope locks](../../azure-resource-manager/management/lock-resources.md#portal). --### Policy with 'Deny' effect preventing us from migrating your rules --As part of the migration, new metric alerts and new action groups will be created, and then classic alert rules will be deleted. However, an [Azure Policy](../../governance/policy/index.yml) assignment can prevent us from creating resources. Depending on the policy assignment, some or all rules couldn't be migrated. The policy assignments that are blocking the process are listed in the [migration tool](https://portal.azure.com/#blade/Microsoft_Azure_Monitoring/MigrationBladeViewModel). Resolve this problem by either: --- Excluding the subscriptions, resource groups, or individual resources during the migration process from the policy assignment. [Learn more about managing policy exclusion scopes](../../governance/policy/tutorials/create-and-manage.md#remove-a-non-compliant-or-denied-resource-from-the-scope-with-an-exclusion).-- Set the 'Enforcement Mode' to **Disabled** on the policy assignment. [Learn more about policy assignment's enforcementMode property](../../governance/policy/concepts/assignment-structure.md#enforcement-mode).-- Set an Azure Policy exemption (preview) on the subscriptions, resource groups, or individual resources to the policy assignment. [Learn more about the Azure Policy exemption structure](../../governance/policy/concepts/exemption-structure.md).-- Removing or changing effect to 'disabled', 'audit', 'append', or 'modify' (which, for example, can solve issues relating to missing tags). [Learn more about managing policy effects](../../governance/policy/concepts/definition-structure.md#policy-rule).--## Next steps --- [Prepare for the migration](alerts-prepare-migration.md) |
| azure-monitor | Alerts Webhooks | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/alerts/alerts-webhooks.md | - Title: Call a webhook with a classic metric alert in Azure Monitor -description: Learn how to reroute Azure metric alerts to other, non-Azure systems. -- Previously updated : 05/28/2023----# Call a webhook with a classic metric alert in Azure Monitor --> [!WARNING] -> This article describes how to use older classic metric alerts. Azure Monitor now supports [newer near-real time metric alerts and a new alerts experience](./alerts-overview.md). Classic alerts are [retired](./monitoring-classic-retirement.md) for public cloud users. Classic alerts for Azure Government cloud and Microsoft Azure operated by 21Vianet will retire on **29 February 2024**. -> --You can use webhooks to route an Azure alert notification to other systems for post-processing or custom actions. You can use a webhook on an alert to route it to services that send SMS messages, to log bugs, to notify a team via chat or messaging services, or for various other actions. --This article describes how to set a webhook on an Azure metric alert. It also shows you what the payload for the HTTP POST to a webhook looks like. For information about the setup and schema for an Azure activity log alert (alert on events), see [Call a webhook on an Azure activity log alert](../alerts/alerts-log-webhook.md). --Azure alerts use HTTP POST to send the alert contents in JSON format to a webhook URI that you provide when you create the alert. The schema is defined later in this article. The URI must be a valid HTTP or HTTPS endpoint. Azure posts one entry per request when an alert is activated. --## Configure webhooks via the Azure portal -To add or update the webhook URI, in the [Azure portal](https://portal.azure.com/), go to **Create/Update Alerts**. ---You can also configure an alert to post to a webhook URI by using [Azure PowerShell cmdlets](../powershell-samples.md#create-metric-alerts), a [cross-platform CLI](../cli-samples.md#work-with-alerts), or [Azure Monitor REST APIs](/rest/api/monitor/alertrules). --## Authenticate the webhook -The webhook can authenticate by using token-based authorization. The webhook URI is saved with a token ID. For example: `https://mysamplealert/webcallback?tokenid=sometokenid&someparameter=somevalue` --## Payload schema -The POST operation contains the following JSON payload and schema for all metric-based alerts: --```JSON -{ - "status": "Activated", - "context": { - "timestamp": "2015-08-14T22:26:41.9975398Z", - "id": "/subscriptions/s1/resourceGroups/useast/providers/microsoft.insights/alertrules/ruleName1", - "name": "ruleName1", - "description": "some description", - "conditionType": "Metric", - "condition": { - "metricName": "Requests", - "metricUnit": "Count", - "metricValue": "10", - "threshold": "10", - "windowSize": "15", - "timeAggregation": "Average", - "operator": "GreaterThanOrEqual" - }, - "subscriptionId": "s1", - "resourceGroupName": "useast", - "resourceName": "mysite1", - "resourceType": "microsoft.foo/sites", - "resourceId": "/subscriptions/s1/resourceGroups/useast/providers/microsoft.foo/sites/mysite1", - "resourceRegion": "centralus", - "portalLink": "https://portal.azure.com/#resource/subscriptions/s1/resourceGroups/useast/providers/microsoft.foo/sites/mysite1" - }, - "properties": { - "key1": "value1", - "key2": "value2" - } -} -``` ---| Field | Mandatory | Fixed set of values | Notes | -|: |: |: |: | -| status |Y |Activated, Resolved |The status for the alert based on the conditions you set. | -| context |Y | |The alert context. | -| timestamp |Y | |The time at which the alert was triggered. | -| id |Y | |Every alert rule has a unique ID. | -| name |Y | |The alert name. | -| description |Y | |A description of the alert. | -| conditionType |Y |Metric, Event |Two types of alerts are supported: metric and event. Metric alerts are based on a metric condition. Event alerts are based on an event in the activity log. Use this value to check whether the alert is based on a metric or on an event. | -| condition |Y | |The specific fields to check based on the **conditionType** value. | -| metricName |For metric alerts | |The name of the metric that defines what the rule monitors. | -| metricUnit |For metric alerts |Bytes, BytesPerSecond, Count, CountPerSecond, Percent, Seconds |The unit allowed in the metric. See [allowed values](/previous-versions/azure/reference/dn802430(v=azure.100)). | -| metricValue |For metric alerts | |The actual value of the metric that caused the alert. | -| threshold |For metric alerts | |The threshold value at which the alert is activated. | -| windowSize |For metric alerts | |The period of time that's used to monitor alert activity based on the threshold. The value must be between 5 minutes and 1 day. The value must be in ISO 8601 duration format. | -| timeAggregation |For metric alerts |Average, Last, Maximum, Minimum, None, Total |How the data that's collected should be combined over time. The default value is Average. See [allowed values](/previous-versions/azure/reference/dn802410(v=azure.100)). | -| operator |For metric alerts | |The operator that's used to compare the current metric data to the set threshold. | -| subscriptionId |Y | |The Azure subscription ID. | -| resourceGroupName |Y | |The name of the resource group for the affected resource. | -| resourceName |Y | |The resource name of the affected resource. | -| resourceType |Y | |The resource type of the affected resource. | -| resourceId |Y | |The resource ID of the affected resource. | -| resourceRegion |Y | |The region or location of the affected resource. | -| portalLink |Y | |A direct link to the portal resource summary page. | -| properties |N |Optional |A set of key/value pairs that has details about the event. For example, `Dictionary<String, String>`. The properties field is optional. In a custom UI or logic app-based workflow, users can enter key/value pairs that can be passed via the payload. An alternate way to pass custom properties back to the webhook is via the webhook URI itself (as query parameters). | --> [!NOTE] -> You can set the **properties** field only by using [Azure Monitor REST APIs](/rest/api/monitor/alertrules). -> -> --## Next steps -* Learn more about Azure alerts and webhooks in the video [Integrate Azure alerts with PagerDuty](https://go.microsoft.com/fwlink/?LinkId=627080). -* Learn how to [execute Azure Automation scripts (runbooks) on Azure alerts](https://go.microsoft.com/fwlink/?LinkId=627081). -* Learn how to [use a logic app to send an SMS message via Twilio from an Azure alert](https://github.com/Azure/azure-quickstart-templates/tree/master/demos/alert-to-text-message-with-logic-app). -* Learn how to [use a logic app to send a Slack message from an Azure alert](https://github.com/Azure/azure-quickstart-templates/tree/master/demos/alert-to-slack-with-logic-app). -* Learn how to [use a logic app to send a message to an Azure Queue from an Azure alert](https://github.com/Azure/azure-quickstart-templates/tree/master/demos/alert-to-queue-with-logic-app). |
| azure-monitor | Api Alerts | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/alerts/api-alerts.md | - Title: Legacy Log Analytics Alert REST API -description: The Log Analytics Alert REST API allows you to create and manage alerts in Log Analytics. This article provides details about the API and examples for performing different operations. -- Previously updated : 06/20/2023----# Legacy Log Analytics Alert REST API --This article describes how to manage alert rules using the legacy API. --> [!IMPORTANT] -> As [announced](https://azure.microsoft.com/updates/switch-api-preference-log-alerts/), the Log Analytics Alert API will be retired on October 1, 2025. You must transition to using the Scheduled Query Rules API for log search alerts by that date. -> Log Analytics workspaces created after June 1, 2019 use the [scheduledQueryRules API](/rest/api/monitor/scheduledqueryrule-2021-08-01/scheduled-query-rules) to manage alert rules. [Switch to the current API](./alerts-log-api-switch.md) in older workspaces to take advantage of Azure Monitor scheduledQueryRules [benefits](./alerts-log-api-switch.md#benefits). --The Log Analytics Alert REST API allows you to create and manage alerts in Log Analytics. This article provides details about the API and several examples for performing different operations. --The Log Analytics Search REST API is RESTful and can be accessed via the Azure Resource Manager REST API. In this article, you'll find examples where the API is accessed from a PowerShell command line by using [ARMClient](https://github.com/projectkudu/ARMClient). This open-source command-line tool simplifies invoking the Azure Resource Manager API. --The use of ARMClient and PowerShell is one of many options you can use to access the Log Analytics Search API. With these tools, you can utilize the RESTful Azure Resource Manager API to make calls to Log Analytics workspaces and perform search commands within them. The API outputs search results in JSON format so that you can use the search results in many different ways programmatically. --## Prerequisites --Currently, alerts can only be created with a saved search in Log Analytics. For more information, see the [Log Search REST API](../logs/log-query-overview.md). --## Schedules --A saved search can have one or more schedules. The schedule defines how often the search is run and the time interval over which the criteria are identified. Schedules have the properties described in the following table: --| Property | Description | -|: |: | -| `Interval` |How often the search is run. Measured in minutes. | -| `QueryTimeSpan` |The time interval over which the criteria are evaluated. Must be equal to or greater than `Interval`. Measured in minutes. | -| `Version` |The API version being used. Currently, this setting should always be `1`. | --For example, consider an event query with an `Interval` of 15 minutes and a `Timespan` of 30 minutes. In this case, the query would be run every 15 minutes. An alert would be triggered if the criteria continued to resolve to `true` over a 30-minute span. --### Retrieve schedules --Use the Get method to retrieve all schedules for a saved search. --```powershell -armclient get /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules?api-version=2015-03-20 -``` --Use the Get method with a schedule ID to retrieve a particular schedule for a saved search. --```powershell -armclient get /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Subscription ID}/schedules/{Schedule ID}?api-version=2015-03-20 -``` --The following sample response is for a schedule: --```json -{ - "value": [{ - "id": "subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/sampleRG/providers/Microsoft.OperationalInsights/workspaces/MyWorkspace/savedSearches/0f0f4853-17f8-4ed1-9a03-8e888b0d16ec/schedules/a17b53ef-bd70-4ca4-9ead-83b00f2024a8", - "etag": "W/\"datetime'2016-02-25T20%3A54%3A49.8074679Z'\"", - "properties": { - "Interval": 15, - "QueryTimeSpan": 15, - "Enabled": true, - } - }] -} -``` --### Create a schedule --Use the Put method with a unique schedule ID to create a new schedule. Two schedules can't have the same ID even if they're associated with different saved searches. When you create a schedule in the Log Analytics console, a GUID is created for the schedule ID. --> [!NOTE] -> The name for all saved searches, schedules, and actions created with the Log Analytics API must be in lowercase. --```powershell -$scheduleJson = "{'properties': { 'Interval': 15, 'QueryTimeSpan':15, 'Enabled':'true' } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/mynewschedule?api-version=2015-03-20 $scheduleJson -``` --### Edit a schedule --Use the Put method with an existing schedule ID for the same saved search to modify that schedule. In the following example, the schedule is disabled. The body of the request must include the *etag* of the schedule. --```powershell -$scheduleJson = "{'etag': 'W/\"datetime'2016-02-25T20%3A54%3A49.8074679Z'\""','properties': { 'Interval': 15, 'QueryTimeSpan':15, 'Enabled':'false' } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/mynewschedule?api-version=2015-03-20 $scheduleJson -``` --### Delete schedules --Use the Delete method with a schedule ID to delete a schedule. --```powershell -armclient delete /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Subscription ID}/schedules/{Schedule ID}?api-version=2015-03-20 -``` --## Actions --A schedule can have multiple actions. An action might define one or more processes to perform, such as sending an email or starting a runbook. An action also might define a threshold that determines when the results of a search match some criteria. Some actions will define both so that the processes are performed when the threshold is met. --All actions have the properties described in the following table. Different types of alerts have other different properties, which are described in the following table: --| Property | Description | -|: |: | -| `Type` |Type of the action. Currently, the possible values are `Alert` and `Webhook`. | -| `Name` |Display name for the alert. | -| `Version` |The API version being used. Currently, this setting should always be `1`. | --### Retrieve actions --Use the Get method to retrieve all actions for a schedule. --```powershell -armclient get /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions?api-version=2015-03-20 -``` --Use the Get method with the action ID to retrieve a particular action for a schedule. --```powershell -armclient get /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Subscription ID}/schedules/{Schedule ID}/actions/{Action ID}?api-version=2015-03-20 -``` --### Create or edit actions --Use the Put method with an action ID that's unique to the schedule to create a new action. When you create an action in the Log Analytics console, a GUID is for the action ID. --> [!NOTE] -> The name for all saved searches, schedules, and actions created with the Log Analytics API must be in lowercase. --Use the Put method with an existing action ID for the same saved search to modify that schedule. The body of the request must include the etag of the schedule. --The request format for creating a new action varies by action type, so these examples are provided in the following sections. --### Delete actions --Use the Delete method with the action ID to delete an action. --```powershell -armclient delete /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Subscription ID}/schedules/{Schedule ID}/Actions/{Action ID}?api-version=2015-03-20 -``` --### Alert actions --A schedule should have one and only one Alert action. Alert actions have one or more of the sections described in the following table: --| Section | Description | Usage | -|: |: |: | -| Threshold |Criteria for when the action is run.| Required for every alert, before or after they're extended to Azure. | -| Severity |Label used to classify the alert when triggered.| Required for every alert, before or after they're extended to Azure. | -| Suppress |Option to stop notifications from alerts. | Optional for every alert, before or after they're extended to Azure. | -| Action groups |IDs of Azure `ActionGroup` where actions required are specified, like emails, SMSs, voice calls, webhooks, automation runbooks, and ITSM Connectors.| Required after alerts are extended to Azure.| -| Customize actions|Modify the standard output for select actions from `ActionGroup`.| Optional for every alert and can be used after alerts are extended to Azure. | --### Thresholds --An Alert action should have one and only one threshold. When the results of a saved search match the threshold in an action associated with that search, any other processes in that action are run. An action can also contain only a threshold so that it can be used with actions of other types that don't contain thresholds. --Thresholds have the properties described in the following table: --| Property | Description | -|: |: | -| `Operator` |Operator for the threshold comparison. <br> gt = Greater than <br> lt = Less than | -| `Value` |Value for the threshold. | --For example, consider an event query with an `Interval` of 15 minutes, a `Timespan` of 30 minutes, and a `Threshold` of greater than 10. In this case, the query would be run every 15 minutes. An alert would be triggered if it returned 10 events that were created over a 30-minute span. --The following sample response is for an action with only a `Threshold`: --```json -"etag": "W/\"datetime'2016-02-25T20%3A54%3A20.1302566Z'\"", -"properties": { - "Type": "Alert", - "Name": "My threshold action", - "Threshold": { - "Operator": "gt", - "Value": 10 - }, - "Version": 1 -} -``` --Use the Put method with a unique action ID to create a new threshold action for a schedule. --```powershell -$thresholdJson = "{'properties': { 'Name': 'My Threshold', 'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 10 } } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/mythreshold?api-version=2015-03-20 $thresholdJson -``` --Use the Put method with an existing action ID to modify a threshold action for a schedule. The body of the request must include the etag of the action. --```powershell -$thresholdJson = "{'etag': 'W/\"datetime'2016-02-25T20%3A54%3A20.1302566Z'\"','properties': { 'Name': 'My Threshold', 'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 10 } } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/mythreshold?api-version=2015-03-20 $thresholdJson -``` --#### Severity --Log Analytics allows you to classify your alerts into categories for easier management and triage. The Alerts severity levels are `informational`, `warning`, and `critical`. These categories are mapped to the normalized severity scale of Azure Alerts as shown in the following table: --|Log Analytics severity level |Azure Alerts severity level | -||| -|`critical` |Sev 0| -|`warning` |Sev 1| -|`informational` | Sev 2| --The following sample response is for an action with only `Threshold` and `Severity`: --```json -"etag": "W/\"datetime'2016-02-25T20%3A54%3A20.1302566Z'\"", -"properties": { - "Type": "Alert", - "Name": "My threshold action", - "Threshold": { - "Operator": "gt", - "Value": 10 - }, - "Severity": "critical", - "Version": 1 -} -``` --Use the Put method with a unique action ID to create a new action for a schedule with `Severity`. --```powershell -$thresholdWithSevJson = "{'properties': { 'Name': 'My Threshold', 'Version':'1','Severity': 'critical', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 10 } } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/mythreshold?api-version=2015-03-20 $thresholdWithSevJson -``` --Use the Put method with an existing action ID to modify a severity action for a schedule. The body of the request must include the etag of the action. --```powershell -$thresholdWithSevJson = "{'etag': 'W/\"datetime'2016-02-25T20%3A54%3A20.1302566Z'\"','properties': { 'Name': 'My Threshold', 'Version':'1','Severity': 'critical', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 10 } } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/mythreshold?api-version=2015-03-20 $thresholdWithSevJson -``` --#### Suppress --Log Analytics-based query alerts fire every time the threshold is met or exceeded. Based on the logic implied in the query, an alert might get fired for a series of intervals. The result is that notifications are sent constantly. To prevent such a scenario, you can set the `Suppress` option that instructs Log Analytics to wait for a stipulated amount of time before notification is fired the second time for the alert rule. --For example, if `Suppress` is set for 30 minutes, the alert will fire the first time and send notifications configured. It will then wait for 30 minutes before notification for the alert rule is again used. In the interim period, the alert rule will continue to run. Only notification is suppressed by Log Analytics for a specified time regardless of how many times the alert rule fired in this period. --The `Suppress` property of a log search alert rule is specified by using the `Throttling` value. The suppression period is specified by using the `DurationInMinutes` value. --The following sample response is for an action with only `Threshold`, `Severity`, and `Suppress` properties. --```json -"etag": "W/\"datetime'2016-02-25T20%3A54%3A20.1302566Z'\"", -"properties": { - "Type": "Alert", - "Name": "My threshold action", - "Threshold": { - "Operator": "gt", - "Value": 10 - }, - "Throttling": { - "DurationInMinutes": 30 - }, - "Severity": "critical", - "Version": 1 -} -``` --Use the Put method with a unique action ID to create a new action for a schedule with `Severity`. --```powershell -$AlertSuppressJson = "{'properties': { 'Name': 'My Threshold', 'Version':'1','Severity': 'critical', 'Type':'Alert', 'Throttling': { 'DurationInMinutes': 30 },'Threshold': { 'Operator': 'gt', 'Value': 10 } } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/myalert?api-version=2015-03-20 $AlertSuppressJson -``` --Use the Put method with an existing action ID to modify a severity action for a schedule. The body of the request must include the etag of the action. --```powershell -$AlertSuppressJson = "{'etag': 'W/\"datetime'2016-02-25T20%3A54%3A20.1302566Z'\"','properties': { 'Name': 'My Threshold', 'Version':'1','Severity': 'critical', 'Type':'Alert', 'Throttling': { 'DurationInMinutes': 30 },'Threshold': { 'Operator': 'gt', 'Value': 10 } } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/myalert?api-version=2015-03-20 $AlertSuppressJson -``` --#### Action groups --All alerts in Azure use action group as the default mechanism for handling actions. With an action group, you can specify your actions once and then associate the action group to multiple alerts across Azure without the need to declare the same actions repeatedly. Action groups support multiple actions like email, SMS, voice call, ITSM connection, automation runbook, and webhook URI. --For users who have extended their alerts into Azure, a schedule should now have action group details passed along with `Threshold` to be able to create an alert. E-mail details, webhook URLs, runbook automation details, and other actions need to be defined inside an action group first before you create an alert. You can create an [action group from Azure Monitor](./action-groups.md) in the Azure portal or use the [Action Group API](/rest/api/monitor/actiongroups). --To associate an action group to an alert, specify the unique Azure Resource Manager ID of the action group in the alert definition. The following sample illustrates the use: --```json -"etag": "W/\"datetime'2017-12-13T10%3A52%3A21.1697364Z'\"", -"properties": { - "Type": "Alert", - "Name": "test-alert", - "Description": "I need to put a description here", - "Threshold": { - "Operator": "gt", - "Value": 12 - }, - "AzNsNotification": { - "GroupIds": [ - "/subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-group/providers/microsoft.insights/actiongroups/test-actiongroup" - ] - }, - "Severity": "critical", - "Version": 1 -} -``` --Use the Put method with a unique action ID to associate an already existing action group for a schedule. The following sample illustrates the use: --```powershell -$AzNsJson = "{'properties': { 'Name': 'test-alert', 'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 12 },'Severity': 'critical', 'AzNsNotification': {'GroupIds': ['subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-group/providers/microsoft.insights/actiongroups/test-actiongroup']} } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{Resource Group Name}/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/myAzNsaction?api-version=2015-03-20 $AzNsJson -``` --Use the Put method with an existing action ID to modify an action group associated for a schedule. The body of the request must include the etag of the action. --```powershell -$AzNsJson = "{'etag': 'datetime'2017-12-13T10%3A52%3A21.1697364Z'\"', 'properties': { 'Name': 'test-alert', 'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 12 },'Severity': 'critical', 'AzNsNotification': { 'GroupIds': ['subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-group/providers/microsoft.insights/actiongroups/test-actiongroup'] } } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{Resource Group Name}/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/myAzNsaction?api-version=2015-03-20 $AzNsJson -``` --#### Customize actions --By default, actions follow standard templates and format for notifications. But you can customize some actions, even if they're controlled by action groups. Currently, customization is possible for `EmailSubject` and `WebhookPayload`. --##### Customize EmailSubject for an action group --By default, the email subject for alerts is Alert Notification `<AlertName>` for `<WorkspaceName>`. But the subject can be customized so that you can specify words or tags to allow you to easily employ filter rules in your Inbox. The customized email header details need to be sent along with `ActionGroup` details, as in the following sample: --```json -"etag": "W/\"datetime'2017-12-13T10%3A52%3A21.1697364Z'\"", -"properties": { - "Type": "Alert", - "Name": "test-alert", - "Description": "I need to put a description here", - "Threshold": { - "Operator": "gt", - "Value": 12 - }, - "AzNsNotification": { - "GroupIds": [ - "/subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-group/providers/microsoft.insights/actiongroups/test-actiongroup" - ], - "CustomEmailSubject": "Azure Alert fired" - }, - "Severity": "critical", - "Version": 1 -} -``` --Use the Put method with a unique action ID to associate an existing action group with customization for a schedule. The following sample illustrates the use: --```powershell -$AzNsJson = "{'properties': { 'Name': 'test-alert', 'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 12 },'Severity': 'critical', 'AzNsNotification': {'GroupIds': ['subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-group/providers/microsoft.insights/actiongroups/test-actiongroup'], 'CustomEmailSubject': 'Azure Alert fired'} } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{Resource Group Name}/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/myAzNsaction?api-version=2015-03-20 $AzNsJson -``` --Use the Put method with an existing action ID to modify an action group associated for a schedule. The body of the request must include the etag of the action. --```powershell -$AzNsJson = "{'etag': 'datetime'2017-12-13T10%3A52%3A21.1697364Z'\"', 'properties': { 'Name': 'test-alert', 'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 12 },'Severity': 'critical', 'AzNsNotification': {'GroupIds': ['subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-group/providers/microsoft.insights/actiongroups/test-actiongroup']}, 'CustomEmailSubject': 'Azure Alert fired' } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{Resource Group Name}/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/myAzNsaction?api-version=2015-03-20 $AzNsJson -``` --##### Customize WebhookPayload for an action group --By default, the webhook sent via an action group for Log Analytics has a fixed structure. But you can customize the JSON payload by using specific variables supported to meet requirements of the webhook endpoint. For more information, see [Webhook action for log search alert rules](./alerts-log-webhook.md). --The customized webhook details must be sent along with `ActionGroup` details. They'll be applied to all webhook URIs specified inside the action group. The following sample illustrates the use: --```json -"etag": "W/\"datetime'2017-12-13T10%3A52%3A21.1697364Z'\"", -"properties": { - "Type": "Alert", - "Name": "test-alert", - "Description": "I need to put a description here", - "Threshold": { - "Operator": "gt", - "Value": 12 - }, - "AzNsNotification": { - "GroupIds": [ - "/subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-group/providers/microsoft.insights/actiongroups/test-actiongroup" - ], - "CustomWebhookPayload": "{\"field1\":\"value1\",\"field2\":\"value2\"}", - "CustomEmailSubject": "Azure Alert fired" - }, - "Severity": "critical", - "Version": 1 -}, -``` --Use the Put method with a unique action ID to associate an existing action group with customization for a schedule. The following sample illustrates the use: --```powershell -$AzNsJson = "{'properties': { 'Name': 'test-alert', 'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 12 },'Severity': 'critical', 'AzNsNotification': {'GroupIds': ['subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-group/providers/microsoft.insights/actiongroups/test-actiongroup'], 'CustomEmailSubject': 'Azure Alert fired','CustomWebhookPayload': '{\"field1\":\"value1\",\"field2\":\"value2\"}'} } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{Resource Group Name}/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/myAzNsaction?api-version=2015-03-20 $AzNsJson -``` --Use the Put method with an existing action ID to modify an action group associated for a schedule. The body of the request must include the etag of the action. --```powershell -$AzNsJson = "{'etag': 'datetime'2017-12-13T10%3A52%3A21.1697364Z'\"', 'properties': { 'Name': 'test-alert', 'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 12 },'Severity': 'critical', 'AzNsNotification': {'GroupIds': ['subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-group/providers/microsoft.insights/actiongroups/test-actiongroup']}, 'CustomEmailSubject': 'Azure Alert fired','CustomWebhookPayload': '{\"field1\":\"value1\",\"field2\":\"value2\"}' } }" -armclient put /subscriptions/{Subscription ID}/resourceGroups/{Resource Group Name}/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/myAzNsaction?api-version=2015-03-20 $AzNsJson -``` --## Next steps --* Use the [REST API to perform log searches](../logs/log-query-overview.md) in Log Analytics. -* Learn about [log search alerts in Azure Monitor](./alerts-types.md#log-alerts). -* Learn how to [create, edit, or manage log search alert rules in Azure Monitor](./alerts-log.md). |
| azure-monitor | Azure Vm Vmss Apps | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/app/azure-vm-vmss-apps.md | Title: Monitor performance on Azure VMs - Azure Application Insights description: Application performance monitoring for Azure virtual machines and virtual machine scale sets. Previously updated : 03/22/2023 Last updated : 04/05/2024 ms.devlang: csharp # ms.devlang: csharp, java, javascript, python We recommend the [Application Insights Java 3.0 agent](./opentelemetry-enable.md ### [Node.js](#tab/nodejs) -To instrument your Node.js application, use the [SDK](./nodejs.md). +To instrument your Node.js application, use the [OpenTelemetry Distro](./opentelemetry-enable.md). ### [Python](#tab/python) -To monitor Python apps, use the [SDK](/previous-versions/azure/azure-monitor/app/opencensus-python). +To monitor Python apps, use the [OpenTelemetry Distro](./opentelemetry-enable.md). |
| azure-monitor | Azure Web Apps Net | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/app/azure-web-apps-net.md | Title: Monitor Azure app services performance ASP.NET | Microsoft Docs description: Learn about application performance monitoring for Azure app services by using ASP.NET. Chart load and response time and dependency information, and set alerts on performance. Previously updated : 03/22/2023 Last updated : 04/05/2024 ms.devlang: javascript |
| azure-monitor | Azure Monitor Monitoring Reference | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/azure-monitor-monitoring-reference.md | - Title: Monitoring Azure monitor data reference -description: Important reference material needed when you monitor parts of Azure Monitor ----- Previously updated : 04/03/2022----# Monitoring Azure Monitor data reference --> [!NOTE] -> This article may seem confusing because it lists the parts of the Azure Monitor service that are monitored by itself. --See [Monitoring Azure Monitor](monitor-azure-monitor.md) for an explanation of how Azure Monitor monitors itself. --## Metrics --This section lists all the platform metrics collected automatically for Azure Monitor into Azure Monitor. --|Metric Type | Resource Provider / Type Namespace<br/> and link to individual metrics | -|-|--| -| [Autoscale behaviors for VMs and AppService](./autoscale/autoscale-overview.md) | [microsoft.insights/autoscalesettings](/azure/azure-monitor/platform/metrics-supported#microsoftinsightsautoscalesettings) | --While technically not about Azure Monitor operations, the following metrics are collected into Azure Monitor namespaces. --|Metric Type | Resource Provider / Type Namespace<br/> and link to individual metrics | -|-|--| -| Log Analytics agent gathered data for the [Metric alerts on logs](./alerts/alerts-metric-logs.md#metrics-and-dimensions-supported-for-logs) feature | [Microsoft.OperationalInsights/workspaces](/azure/azure-monitor/platform/metrics-supported##microsoftoperationalinsightsworkspaces) -| [Application Insights availability tests](./app/availability-overview.md) | [Microsoft.Insights/Components](./essentials/metrics-supported.md#microsoftinsightscomponents) --See a complete list of [platform metrics for other resources types](/azure/azure-monitor/platform/metrics-supported). --## Metric Dimensions --For more information on what metric dimensions are, see [Multi-dimensional metrics](/azure/azure-monitor/platform/data-platform-metrics#multi-dimensional-metrics). --The following dimensions are relevant for the following areas of Azure Monitor. --### Autoscale --| Dimension Name | Description | -| - | -- | -|MetricTriggerRule | The autoscale rule that triggered the scale action | -|MetricTriggerSource | The metric value that triggered the scale action | -|ScaleDirection | The direction of the scale action (up or down) --## Resource logs --This section lists all the Azure Monitor resource log category types collected. --|Resource Log Type | Resource Provider / Type Namespace<br/> and link | -|-|--| -| [Autoscale for VMs and AppService](./autoscale/autoscale-overview.md) | [Microsoft.insights/autoscalesettings](./essentials/resource-logs-categories.md#microsoftinsightsautoscalesettings)| -| [Application Insights availability tests](./app/availability-overview.md) | [Microsoft.insights/Components](./essentials/resource-logs-categories.md#microsoftinsightscomponents) | --For additional reference, see a list of [all resource logs category types supported in Azure Monitor](/azure/azure-monitor/platform/resource-logs-schema). ---## Azure Monitor Logs tables --This section refers to all of the Azure Monitor Logs Kusto tables relevant to Azure Monitor resource types and available for query by Log Analytics. --|Resource Type | Notes | -|--|-| -| [Autoscale for VMs and AppService](./autoscale/autoscale-overview.md) | [Autoscale Tables](/azure/azure-monitor/reference/tables/tables-resourcetype#azure-monitor-autoscale-settings) | ---## Activity log --For a partial list of entires that the Azure Monitor services writes to the activity log, see [Azure resource provider operations](../role-based-access-control/resource-provider-operations.md#monitor). There may be other entires not listed here. --For more information on the schema of Activity Log entries, see [Activity Log schema](./essentials/activity-log-schema.md). --## Schemas --The following schemas are in use by Azure Monitor. --### Action Groups --The following schemas are relevant to action groups, which are part of the notification infrastructure for Azure Monitor. Following are example calls and responses for action groups. --#### Create Action Group -```json -{ - "authorization": { - "action": "microsoft.insights/actionGroups/write", - "scope": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc" - }, - "caller": "test.cam@ieee.org", - "channels": "Operation", - "claims": { - "aud": "https://management.core.windows.net/", - "iss": "https://sts.windows.net/04ebb17f-c9d2-bbbb-881f-8fd503332aac/", - "iat": "1627074914", - "nbf": "1627074914", - "exp": "1627078814", - "http://schemas.microsoft.com/claims/authnclassreference": "1", - "aio": "AUQAu/8TbbbbyZJhgackCVdLETN5UafFt95J8/bC1SP+tBFMusYZ3Z4PBQRZUZ4SmEkWlDevT4p7Wtr4e/R+uksbfixGGQumxw==", - "altsecid": "1:live.com:00037FFE809E290F", - "http://schemas.microsoft.com/claims/authnmethodsreferences": "pwd", - "appid": "c44b4083-3bb0-49c1-bbbb-974e53cbdf3c", - "appidacr": "2", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress": "test.cam@ieee.org", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname": "cam", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname": "test", - "groups": "d734c6d5-bbbb-4b39-8992-88fd979076eb", - "http://schemas.microsoft.com/identity/claims/identityprovider": "live.com", - "ipaddr": "73.254.xxx.xx", - "name": "test cam", - "http://schemas.microsoft.com/identity/claims/objectidentifier": "f19e58c4-5bfa-4ac6-8e75-9823bbb1ea0a", - "puid": "1003000086500F96", - "rh": "0.AVgAf7HrBNLJbkKIH4_VAzMqrINAS8SwO8FJtH2XTlPL3zxYAFQ.", - "http://schemas.microsoft.com/identity/claims/scope": "user_impersonation", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier": "SzEgbtESOKM8YsOx9t49Ds-L2yCyUR-hpIDinBsS-hk", - "http://schemas.microsoft.com/identity/claims/tenantid": "04ebb17f-c9d2-bbbb-881f-8fd503332aac", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name": "live.com#test.cam@ieee.org", - "uti": "KuRF5PX4qkyvxJQOXwZ2AA", - "ver": "1.0", - "wids": "62e90394-bbbb-4237-9190-012177145e10", - "xms_tcdt": "1373393473" - }, - "correlationId": "74d253d8-bd5a-4e8d-a38e-5a52b173b7bd", - "description": "", - "eventDataId": "0e9bc114-dcdb-4d2d-b1ea-d3f45a4d32ea", - "eventName": { - "value": "EndRequest", - "localizedValue": "End request" - }, - "category": { - "value": "Administrative", - "localizedValue": "Administrative" - }, - "eventTimestamp": "2021-07-23T21:21:22.9871449Z", - "id": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc/events/0e9bc114-dcdb-4d2d-b1ea-d3f45a4d32ea/ticks/637626720829871449", - "level": "Informational", - "operationId": "74d253d8-bd5a-4e8d-a38e-5a52b173b7bd", - "operationName": { - "value": "microsoft.insights/actionGroups/write", - "localizedValue": "Create or update action group" - }, - "resourceGroupName": "testK-TEST", - "resourceProviderName": { - "value": "microsoft.insights", - "localizedValue": "Microsoft Insights" - }, - "resourceType": { - "value": "microsoft.insights/actionGroups", - "localizedValue": "microsoft.insights/actionGroups" - }, - "resourceId": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc", - "status": { - "value": "Succeeded", - "localizedValue": "Succeeded" - }, - "subStatus": { - "value": "Created", - "localizedValue": "Created (HTTP Status Code: 201)" - }, - "submissionTimestamp": "2021-07-23T21:22:22.1634251Z", - "subscriptionId": "52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a", - "tenantId": "04ebb17f-c9d2-bbbb-881f-8fd503332aac", - "properties": { - "statusCode": "Created", - "serviceRequestId": "33658bb5-fc62-4e40-92e8-8b1f16f649bb", - "eventCategory": "Administrative", - "entity": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc", - "message": "microsoft.insights/actionGroups/write", - "hierarchy": "52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a" - }, - "relatedEvents": [] -} -``` --#### Delete Action Group -```json -{ - "authorization": { - "action": "microsoft.insights/actionGroups/delete", - "scope": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testk-test/providers/microsoft.insights/actionGroups/TestingLogginc" - }, - "caller": "test.cam@ieee.org", - "channels": "Operation", - "claims": { - "aud": "https://management.core.windows.net/", - "iss": "https://sts.windows.net/04ebb17f-c9d2-bbbb-881f-8fd503332aac/", - "iat": "1627076795", - "nbf": "1627076795", - "exp": "1627080695", - "http://schemas.microsoft.com/claims/authnclassreference": "1", - "aio": "AUQAu/8TbbbbTkWb9O23RavxIzqfHvA2fJUU/OjdhtHPNAjv0W4pyNnoZ3ShUOEzDut700WhNXth6ZYpd7al4XyJPACEfmtr9g==", - "altsecid": "1:live.com:00037FFE809E290F", - "http://schemas.microsoft.com/claims/authnmethodsreferences": "pwd", - "appid": "c44b4083-3bb0-49c1-bbbb-974e53cbdf3c", - "appidacr": "2", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress": "test.cam@ieee.org", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname": "cam", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname": "test", - "groups": "d734c6d5-bbbb-4b39-8992-88fd979076eb", - "http://schemas.microsoft.com/identity/claims/identityprovider": "live.com", - "ipaddr": "73.254.xxx.xx", - "name": "test cam", - "http://schemas.microsoft.com/identity/claims/objectidentifier": "f19e58c4-5bfa-4ac6-8e75-9823bbb1ea0a", - "puid": "1003000086500F96", - "rh": "0.AVgAf7HrBNLJbkKIH4_VAzMqrINAS8SwO8FJtH2XTlPL3zxYAFQ.", - "http://schemas.microsoft.com/identity/claims/scope": "user_impersonation", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier": "SzEgbtESOKM8YsOx9t49Ds-L2yCyUR-hpIDinBsS-hk", - "http://schemas.microsoft.com/identity/claims/tenantid": "04ebb17f-c9d2-bbbb-881f-8fd503332aac", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name": "live.com#test.cam@ieee.org", - "uti": "E1BRdcfDzk64rg0eFx8vAA", - "ver": "1.0", - "wids": "62e90394-bbbb-4237-9190-012177145e10", - "xms_tcdt": "1373393473" - }, - "correlationId": "a0bd5f9f-d87f-4073-8650-83f03cf11733", - "description": "", - "eventDataId": "8c7c920e-6a50-47fe-b264-d762e60cc788", - "eventName": { - "value": "EndRequest", - "localizedValue": "End request" - }, - "category": { - "value": "Administrative", - "localizedValue": "Administrative" - }, - "eventTimestamp": "2021-07-23T21:52:07.2708782Z", - "id": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testk-test/providers/microsoft.insights/actionGroups/TestingLogginc/events/8c7c920e-6a50-47fe-b264-d762e60cc788/ticks/637626739272708782", - "level": "Informational", - "operationId": "f7cb83ba-36fa-47dd-8ec4-bcac40879241", - "operationName": { - "value": "microsoft.insights/actionGroups/delete", - "localizedValue": "Delete action group" - }, - "resourceGroupName": "testk-test", - "resourceProviderName": { - "value": "microsoft.insights", - "localizedValue": "Microsoft Insights" - }, - "resourceType": { - "value": "microsoft.insights/actionGroups", - "localizedValue": "microsoft.insights/actionGroups" - }, - "resourceId": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testk-test/providers/microsoft.insights/actionGroups/TestingLogginc", - "status": { - "value": "Succeeded", - "localizedValue": "Succeeded" - }, - "subStatus": { - "value": "OK", - "localizedValue": "OK (HTTP Status Code: 200)" - }, - "submissionTimestamp": "2021-07-23T21:54:00.1811815Z", - "subscriptionId": "52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a", - "tenantId": "04ebb17f-c9d2-bbbb-881f-8fd503332aac", - "properties": { - "statusCode": "OK", - "serviceRequestId": "88fe5ac8-ee1a-4b97-9d5b-8a3754e256ad", - "eventCategory": "Administrative", - "entity": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testk-test/providers/microsoft.insights/actionGroups/TestingLogginc", - "message": "microsoft.insights/actionGroups/delete", - "hierarchy": "52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a" - }, - "relatedEvents": [] -} -``` --#### Unsubscribe using Email --```json -{ - "caller": "test.cam@ieee.org", - "channels": "Operation", - "claims": { - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name": "", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress": "person@contoso.com", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn": "", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/spn": "", - "http://schemas.microsoft.com/identity/claims/objectidentifier": "" - }, - "correlationId": "8f936022-18d0-475f-9704-5151c75e81e4", - "description": "User with email address:person@contoso.com has unsubscribed from action group:TestingLogginc, Action:testEmail_-EmailAction-", - "eventDataId": "9b4b7b3f-79a2-4a6a-b1ed-30a1b8907765", - "eventName": { - "value": "", - "localizedValue": "" - }, - "category": { - "value": "Administrative", - "localizedValue": "Administrative" - }, - "eventTimestamp": "2021-07-23T21:38:35.1687458Z", - "id": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc/events/9b4b7b3f-79a2-4a6a-b1ed-30a1b8907765/ticks/637626731151687458", - "level": "Informational", - "operationId": "", - "operationName": { - "value": "microsoft.insights/actiongroups/write", - "localizedValue": "Create or update action group" - }, - "resourceGroupName": "testK-TEST", - "resourceProviderName": { - "value": "microsoft.insights", - "localizedValue": "Microsoft Insights" - }, - "resourceType": { - "value": "microsoft.insights/actiongroups", - "localizedValue": "microsoft.insights/actiongroups" - }, - "resourceId": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc", - "status": { - "value": "Succeeded", - "localizedValue": "Succeeded" - }, - "subStatus": { - "value": "Updated", - "localizedValue": "Updated" - }, - "submissionTimestamp": "2021-07-23T21:38:35.1687458Z", - "subscriptionId": "52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a", - "tenantId": "", - "properties": {}, - "relatedEvents": [] -} -``` --#### Unsubscribe using SMS -```json -{ - "caller": "", - "channels": "Operation", - "claims": { - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name": "4252137109", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress": "", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn": "", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/spn": "", - "http://schemas.microsoft.com/identity/claims/objectidentifier": "" - }, - "correlationId": "e039f06d-c0d1-47ac-b594-89239101c4d0", - "description": "User with phone number:4255557109 has unsubscribed from action group:TestingLogginc, Action:testPhone_-SMSAction-", - "eventDataId": "789d0b03-2a2f-40cf-b223-d228abb5d2ed", - "eventName": { - "value": "", - "localizedValue": "" - }, - "category": { - "value": "Administrative", - "localizedValue": "Administrative" - }, - "eventTimestamp": "2021-07-23T21:31:47.1537759Z", - "id": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc/events/789d0b03-2a2f-40cf-b223-d228abb5d2ed/ticks/637626727071537759", - "level": "Informational", - "operationId": "", - "operationName": { - "value": "microsoft.insights/actiongroups/write", - "localizedValue": "Create or update action group" - }, - "resourceGroupName": "testK-TEST", - "resourceProviderName": { - "value": "microsoft.insights", - "localizedValue": "Microsoft Insights" - }, - "resourceType": { - "value": "microsoft.insights/actiongroups", - "localizedValue": "microsoft.insights/actiongroups" - }, - "resourceId": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc", - "status": { - "value": "Succeeded", - "localizedValue": "Succeeded" - }, - "subStatus": { - "value": "Updated", - "localizedValue": "Updated" - }, - "submissionTimestamp": "2021-07-23T21:31:47.1537759Z", - "subscriptionId": "52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a", - "tenantId": "", - "properties": {}, - "relatedEvents": [] -} -``` --#### Update Action Group -```json -{ - "authorization": { - "action": "microsoft.insights/actionGroups/write", - "scope": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc" - }, - "caller": "test.cam@ieee.org", - "channels": "Operation", - "claims": { - "aud": "https://management.core.windows.net/", - "iss": "https://sts.windows.net/04ebb17f-c9d2-bbbb-881f-8fd503332aac/", - "iat": "1627074914", - "nbf": "1627074914", - "exp": "1627078814", - "http://schemas.microsoft.com/claims/authnclassreference": "1", - "aio": "AUQAu/8TbbbbyZJhgackCVdLETN5UafFt95J8/bC1SP+tBFMusYZ3Z4PBQRZUZ4SmEkWlDevT4p7Wtr4e/R+uksbfixGGQumxw==", - "altsecid": "1:live.com:00037FFE809E290F", - "http://schemas.microsoft.com/claims/authnmethodsreferences": "pwd", - "appid": "c44b4083-3bb0-49c1-bbbb-974e53cbdf3c", - "appidacr": "2", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress": "test.cam@ieee.org", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname": "cam", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname": "test", - "groups": "d734c6d5-bbbb-4b39-8992-88fd979076eb", - "http://schemas.microsoft.com/identity/claims/identityprovider": "live.com", - "ipaddr": "73.254.xxx.xx", - "name": "test cam", - "http://schemas.microsoft.com/identity/claims/objectidentifier": "f19e58c4-5bfa-4ac6-8e75-9823bbb1ea0a", - "puid": "1003000086500F96", - "rh": "0.AVgAf7HrBNLJbkKIH4_VAzMqrINAS8SwO8FJtH2XTlPL3zxYAFQ.", - "http://schemas.microsoft.com/identity/claims/scope": "user_impersonation", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier": "SzEgbtESOKM8YsOx9t49Ds-L2yCyUR-hpIDinBsS-hk", - "http://schemas.microsoft.com/identity/claims/tenantid": "04ebb17f-c9d2-bbbb-881f-8fd503332aac", - "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name": "live.com#test.cam@ieee.org", - "uti": "KuRF5PX4qkyvxJQOXwZ2AA", - "ver": "1.0", - "wids": "62e90394-bbbb-4237-9190-012177145e10", - "xms_tcdt": "1373393473" - }, - "correlationId": "5a239734-3fbb-4ff7-b029-b0ebf22d3a19", - "description": "", - "eventDataId": "62c3ebd8-cfc9-435f-956f-86c45eecbeae", - "eventName": { - "value": "BeginRequest", - "localizedValue": "Begin request" - }, - "category": { - "value": "Administrative", - "localizedValue": "Administrative" - }, - "eventTimestamp": "2021-07-23T21:24:34.9424246Z", - "id": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc/events/62c3ebd8-cfc9-435f-956f-86c45eecbeae/ticks/637626722749424246", - "level": "Informational", - "operationId": "5a239734-3fbb-4ff7-b029-b0ebf22d3a19", - "operationName": { - "value": "microsoft.insights/actionGroups/write", - "localizedValue": "Create or update action group" - }, - "resourceGroupName": "testK-TEST", - "resourceProviderName": { - "value": "microsoft.insights", - "localizedValue": "Microsoft Insights" - }, - "resourceType": { - "value": "microsoft.insights/actionGroups", - "localizedValue": "microsoft.insights/actionGroups" - }, - "resourceId": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc", - "status": { - "value": "Started", - "localizedValue": "Started" - }, - "subStatus": { - "value": "", - "localizedValue": "" - }, - "submissionTimestamp": "2021-07-23T21:25:22.1522025Z", - "subscriptionId": "52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a", - "tenantId": "04ebb17f-c9d2-bbbb-881f-8fd503332aac", - "properties": { - "eventCategory": "Administrative", - "entity": "/subscriptions/52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a/resourceGroups/testK-TEST/providers/microsoft.insights/actionGroups/TestingLogginc", - "message": "microsoft.insights/actionGroups/write", - "hierarchy": "52c65f65-bbbb-bbbb-bbbb-7dbbfc68c57a" - }, - "relatedEvents": [] -} -``` --## See Also --- See [Monitoring Azure Monitor](monitor-azure-monitor.md) for a description of what Azure Monitor monitors in itself. -- See [Monitoring Azure resources with Azure Monitor](./essentials/monitor-azure-resource.md) for details on monitoring Azure resources. |
| azure-monitor | Prometheus Metrics Scrape Configuration | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/containers/prometheus-metrics-scrape-configuration.md | The secret should be created in kube-system namespace and then the configmap/CRD Below are the details about how to provide the TLS config settings through a configmap or CRD. -- To provide the TLS config setting in a configmap, please create the self-signed certificate and key inside /etc/prometheus/certs directory inside your mtls enabled app.+- To provide the TLS config setting in a configmap, please create the self-signed certificate and key inside your mtls enabled app. An example tlsConfig inside the config map should look like this: ```yaml tls_config: insecure_skip_verify: false ``` -- To provide the TLS config setting in a CRD, please create the self-signed certificate and key inside /etc/prometheus/certs directory inside your mtls enabled app.+- To provide the TLS config setting in a CRD, please create the self-signed certificate and key inside your mtls enabled app. An example tlsConfig inside a Podmonitor should look like this: ```yaml |
| azure-monitor | Cost Usage | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/cost-usage.md | This article describes the different ways that Azure Monitor charges for usage a [!INCLUDE [azure-monitor-cost-optimization](../../includes/azure-monitor-cost-optimization.md)] ## Pricing model-Azure Monitor uses a consumption-based pricing (pay-as-you-go) billing model where you only pay for what you use. Features of Azure Monitor that are enabled by default do not incur any charge, including collection and alerting on the [Activity log](essentials/activity-log.md) and collection and analysis of [platform metrics](essentials/metrics-supported.md). ++Azure Monitor uses a consumption-based pricing (pay-as-you-go) billing model where you only pay for what you use. Features of Azure Monitor that are enabled by default don't incur any charge. This includes collection and alerting on the [Activity log](essentials/activity-log.md) and collection and analysis of [platform metrics](essentials/metrics-supported.md). Several other features don't have a direct cost, but you instead pay for the ingestion and retention of data that they collect. The following table describes the different types of usage that are charged in Azure Monitor. Detailed current pricing for each is provided in [Azure Monitor pricing](https://azure.microsoft.com/pricing/details/monitor/). | Type | Description | |:|:|-| Logs | Ingestion, retention, and export of data in [Log Analytics workspaces](logs/log-analytics-workspace-overview.md) and [legacy Application insights resources](app/convert-classic-resource.md). This will typically be the bulk of Azure Monitor charges for most customers. There is no charge for querying this data except in the case of [Basic Logs](logs/basic-logs-configure.md) or [Archived Logs](logs/data-retention-archive.md).<br><br>Charges for Logs can vary significantly on the configuration that you choose. See [Azure Monitor Logs pricing details](logs/cost-logs.md) for details on how charges for Logs data are calculated and the different pricing tiers available. | -| Platform Logs | Processing of [diagnostic and auditing information](essentials/resource-logs.md) is charged for [certain services](essentials/resource-logs-categories.md#costs) when sent to destinations other than a Log Analytics workspace. There's no direct charge when this data is sent to a Log Analytics workspace, but there is a charge for the workspace data ingestion and collection. | -| Metrics | There is no charge for [standard metrics](essentials/metrics-supported.md) collected from Azure resources. There is a cost for collecting [custom metrics](essentials/metrics-custom-overview.md) and for retrieving metrics from the [REST API](essentials/rest-api-walkthrough.md#retrieve-metric-values). | +| Logs |Ingestion, retention, and export of data in [Log Analytics workspaces](logs/log-analytics-workspace-overview.md) and [legacy Application insights resources](app/convert-classic-resource.md). Log data ingestion will typically be the largest component of Azure Monitor charges for most customers. There's no charge for querying this data except in the case of [Basic Logs](logs/basic-logs-configure.md) or [Archived Logs](logs/data-retention-archive.md).<br><br>Charges for Logs can vary significantly on the configuration that you choose. See [Azure Monitor Logs pricing details](logs/cost-logs.md) for details on how charges for Logs data are calculated and the different pricing tiers available. | +| Platform Logs | Processing of [diagnostic and auditing information](essentials/resource-logs.md) is charged for [certain services](essentials/resource-logs-categories.md#costs) when sent to destinations other than a Log Analytics workspace. There's no direct charge when this data is sent to a Log Analytics workspace, but there's a charge for the workspace data ingestion and collection. | +| Metrics | There's no charge for [standard metrics](essentials/metrics-supported.md) collected from Azure resources. There's a cost for collecting [custom metrics](essentials/metrics-custom-overview.md) and for retrieving metrics from the [REST API](essentials/rest-api-walkthrough.md#retrieve-metric-values). | | Prometheus Metrics | Pricing for [Azure Monitor managed service for Prometheus](essentials/prometheus-metrics-overview.md) is based on [data samples ingested](containers/kubernetes-monitoring-enable.md#enable-prometheus-and-grafana) and [query samples processed](essentials/azure-monitor-workspace-manage.md#link-a-grafana-workspace). Data is retained for 18 months at no extra charge. |-| Alerts | Alerts are charged based on the type and number of [signals](alerts/alerts-overview.md) used by the alert rule, its frequency, and the type of [notification](alerts/action-groups.md) used in response. For [log search alerts](alerts/alerts-types.md#log-alerts) configured for [at scale monitoring](alerts/alerts-types.md#monitor-the-same-condition-on-multiple-resources-using-splitting-by-dimensions-1), the cost will also depend on the number of time series created by the dimensions resulting from your query. | -| Web tests | There is a cost for [standard web tests](app/availability-standard-tests.md) and [multi-step web tests](app/availability-multistep.md) in Application Insights. Multi-step web tests have been deprecated. +| Alerts | Alerts are charged based on the type and number of [signals](alerts/alerts-overview.md) used by the alert rule, its frequency, and the type of [notification](alerts/action-groups.md) used in response. For [log search alerts](alerts/alerts-types.md#log-alerts) configured for [at scale monitoring](alerts/alerts-types.md#monitor-the-same-condition-on-multiple-resources-using-splitting-by-dimensions-1), the cost also depends on the number of time series created by the dimensions resulting from your query. | +| Web tests | There's a cost for [standard web tests](app/availability-standard-tests.md) and [multi-step web tests](app/availability-multistep.md) in Application Insights. Multi-step web tests are deprecated.| A list of Azure Monitor billing meter names is available [here](cost-meters.md). Sending data to Azure Monitor can incur data bandwidth charges. As described in > Data sent to a different region using [Diagnostic Settings](essentials/diagnostic-settings.md) does not incur data transfer charges ## View Azure Monitor usage and charges-There are two primary tools to view, analyze and optimize your Azure Monitor costs. Each is described in detail in the following sections. +There are two primary tools to view, analyze, and optimize your Azure Monitor costs. Each is described in detail in the following sections. | Tool | Description | |:|:| There are two primary tools to view, analyze and optimize your Azure Monitor cos ## Azure Cost Management + Billing-To get started analyzing your Azure Monitor charges, open [Cost Management + Billing](../cost-management-billing/costs/quick-acm-cost-analysis.md?toc=/azure/billing/TOC.json) in the Azure portal. This tool includes several built-in dashboards for deep cost analysis like cost by resource and invoice details. Select **Cost Management** and then **Cost analysis**. Select your subscription or another [scope](../cost-management-billing/costs/understand-work-scopes.md). +To get started analyzing your Azure Monitor charges, open [Cost Management + Billing](../cost-management-billing/costs/quick-acm-cost-analysis.md?toc=/azure/billing/TOC.json) in the Azure portal. This tool includes several built-in dashboards for deep cost analysis like cost by resource and invoice details. Select **Cost Management** and then **Cost analysis**. Select your subscription or another [scope](../cost-management-billing/costs/understand-work-scopes.md). >[!NOTE] >You might need additional access to use Cost Management data. See [Assign access to Cost Management data](../cost-management-billing/costs/assign-access-acm-data.md). To limit the view to Azure Monitor charges, [create a filter](../cost-management - Insight and Analytics - Application Insights -Other services such as Microsoft Defender for Cloud and Microsoft Sentinel also bill their usage against Log Analytics workspace resources, so you might want to add them to your filter. See [Common cost analysis uses](../cost-management-billing/costs/cost-analysis-common-uses.md) for details on using this view. +Other services such as Microsoft Defender for Cloud and Microsoft Sentinel also bill their usage against Log Analytics workspace resources. See [Common cost analysis uses](../cost-management-billing/costs/cost-analysis-common-uses.md) for details on using this view. >[!NOTE] Other services such as Microsoft Defender for Cloud and Microsoft Sentinel also ### Automated mails and alerts Rather than manually analyzing your costs in the Azure portal, you can automate delivery of information using the following methods. -- **Daily cost analysis emails.** Once you've configured your Cost Analysis view, you should click **Subscribe** at the top of the screen to receive regular email updates from Cost Analysis.- - **Budget alerts.** To be notified if there are significant increases in your spending, create a [budget alerts](../cost-management-billing/costs/cost-mgt-alerts-monitor-usage-spending.md) for a single workspace or group of workspaces. +- **Daily cost analysis emails.** After you configure your Cost Analysis view, you should click **Subscribe** at the top of the screen to receive regular email updates from Cost Analysis. +- **Budget alerts.** To be notified if there are significant increases in your spending, create a [budget alerts](../cost-management-billing/costs/cost-mgt-alerts-monitor-usage-spending.md) for a single workspace or group of workspaces. ### Export usage details To gain deeper understanding of your usage and costs, create exports using **Cost Analysis**. See [Tutorial: Create and manage exported data](../cost-management-billing/costs/tutorial-export-acm-data.md) to learn how to automatically create a daily export you can use for regular analysis. -These exports are in CSV format and will contain a list of daily usage (billed quantity and cost) by resource, [billing meter](cost-meters.md), and several other fields such as [AdditionalInfo](../cost-management-billing/automate/understand-usage-details-fields.md#list-of-fields-and-descriptions). You can use Microsoft Excel to do rich analyses of your usage not possible in the **Cost Analytics** experiences in the portal. +These exports are in CSV format and contain a list of daily usage (billed quantity and cost) by resource, [billing meter](cost-meters.md), and several other fields such as [AdditionalInfo](../cost-management-billing/automate/understand-usage-details-fields.md#list-of-fields-and-descriptions). You can use Microsoft Excel to do rich analyses of your usage not possible in the **Cost Analytics** experiences in the portal. -For example, usage from Log Analytics can be found by first filtering on the **Meter Category** column to show +For example, usage from Log Analytics can be found by first filtering on the **Meter Category** column to show: 1. **Log Analytics** (for Pay-as-you-go data ingestion and interactive Data Retention), 2. **Insight and Analytics** (used by some of the legacy pricing tiers), and Add a filter on the **Instance ID** column for **contains workspace** or **conta ## View data allocation benefits -There are several approaches to view the benefits a workspace receives from various offers such as the [Defender for Servers data allowance](logs/cost-logs.md#workspaces-with-microsoft-defender-for-cloud) and the [Microsoft Sentinel benefit for Microsoft 365 E5, A5, F5, and G5 customers](https://azure.microsoft.com/offers/sentinel-microsoft-365-offer/). +There are several approaches to view the benefits a workspace receives from offers that are part of other products. These offers are: ++1. [Defender for Servers data allowance](logs/cost-logs.md#workspaces-with-microsoft-defender-for-cloud) and ++1. [Microsoft Sentinel benefit for Microsoft 365 E5, A5, F5, and G5 customers](https://azure.microsoft.com/offers/sentinel-microsoft-365-offer/). ### View benefits in a usage export -Since a usage export has both the number of units of usage and their cost, you can use this export to see the amount of benefits you are receiving. In the usage export, to see the benefits, filter the *Instance ID* column to your workspace. (To select all of your workspaces in the spreadsheet, filter the *Instance ID* column to "contains /workspaces/".) Then filter on the Meter to either of the following two meters: +Since a usage export has both the number of units of usage and their cost, you can use this export to see the benefits you're receiving. In the usage export, to see the benefits, filter the *Instance ID* column to your workspace. (To select all of your workspaces in the spreadsheet, filter the *Instance ID* column to "contains /workspaces/".) Then filter on the Meter to either of the following 2 meters: -- **Standard Data Included per Node**: this meter is under the service "Insight and Analytics" and tracks the benefits received when a workspace in either in Log Analytics [Per Node tier](logs/cost-logs.md#per-node-pricing-tier) data allowance and/or has [Defender for Servers](logs/cost-logs.md#workspaces-with-microsoft-defender-for-cloud) enabled. Each of these provide a 500 MB/server/day data allowance.-- **Free Benefit - M365 Defender Data Ingestion**: this meter, under the service "Azure Monitor", tracks the benefit from the [Microsoft Sentinel benefit for Microsoft 365 E5, A5, F5, and G5 customers](https://azure.microsoft.com/offers/sentinel-microsoft-365-offer/).+- **Standard Data Included per Node**: this meter is under the service "Insight and Analytics" and tracks the benefits received when a workspace in either in Log Analytics [Per Node tier](logs/cost-logs.md#per-node-pricing-tier) data allowance and/or has [Defender for Servers](logs/cost-logs.md#workspaces-with-microsoft-defender-for-cloud) enabled. Each of these allowances provide a 500 MB/server/day data allowance. ++- **Free Benefit - M365 Defender Data Ingestion**: this meter, under the service "Azure Monitor", tracks the benefit from the [Microsoft Sentinel benefit for Microsoft 365 E5, A5, F5, and G5 customers](https://azure.microsoft.com/offers/sentinel-microsoft-365-offer/). ### View benefits in Usage and estimated costs -You can also see these data benefits in the Log Analytics Usage and estimated costs page. If the workspace is receiving these benefits, there will be a sentence below the cost estimate table that gives the data volume of the benefits used over the last 31 days. +You can also see these data benefits in the Log Analytics Usage and estimated costs page. If the workspace is receiving these benefits, there's a sentence below the cost estimate table that gives the data volume of the benefits used over the last 31 days. :::image type="content" source="media/cost-usage/log-analytics-workspace-benefit.png" lightbox="media/cost-usage/log-analytics-workspace-benefit.png" alt-text="Screenshot of monthly usage with benefits from Defender and Sentinel offers."::: ### Query benefits from the Operation table -The [Operation](/azure/azure-monitor/reference/tables/operation) table contains daily events which given the amount of benefit used from the [Defender for Servers data allowance](logs/cost-logs.md#workspaces-with-microsoft-defender-for-cloud) and the [Microsoft Sentinel benefit for Microsoft 365 E5, A5, F5, and G5 customers](https://azure.microsoft.com/offers/sentinel-microsoft-365-offer/). The `Detail` column for these events are all of the format `Benefit amount used 1.234 GB`, and the type of benefit is in the `OperationKey` column. Here is a query that charts the benefits used in the last 31-days: +The [Operation](/azure/azure-monitor/reference/tables/operation) table contains daily events which given the amount of benefit used from the [Defender for Servers data allowance](logs/cost-logs.md#workspaces-with-microsoft-defender-for-cloud) and the [Microsoft Sentinel benefit for Microsoft 365 E5, A5, F5, and G5 customers](https://azure.microsoft.com/offers/sentinel-microsoft-365-offer/). The `Detail` column for these events is in the format `Benefit amount used 1.234 GB`, and the type of benefit is in the `OperationKey` column. Here's a query that charts the benefits used in the last 31-days: ```kusto Operation Operation > ## Usage and estimated costs-You can get additional usage details about Log Analytics workspaces and Application Insights resources from the **Usage and Estimated Costs** option for each. +You can get more usage details about Log Analytics workspaces and Application Insights resources from the **Usage and Estimated Costs** option for each. ### Log Analytics workspace To learn about your usage trends and optimize your costs using the most cost-effective [commitment tier](logs/cost-logs.md#commitment-tiers) for your Log Analytics workspace, select **Usage and Estimated Costs** from the **Log Analytics workspace** menu in the Azure portal. :::image type="content" source="media/cost-usage/usage-estimated-cost-dashboard-01.png" lightbox="media/cost-usage/usage-estimated-cost-dashboard-01.png" alt-text="Screenshot of usage and estimated costs screen in Azure portal."::: -This view includes the following: +This view includes the following sections: A. Estimated monthly charges based on usage from the past 31 days using the current pricing tier.<br> B. Estimated monthly charges using different commitment tiers.<br> Customers who purchased Microsoft Operations Management Suite E1 and E2 are elig To receive these entitlements for Log Analytics workspaces or Application Insights resources in a subscription, they must use the Per-Node (OMS) pricing tier. This entitlement isn't visible in the estimated costs shown in the Usage and estimated cost pane. -Depending on the number of nodes of the suite that your organization purchased, moving some subscriptions into a Per GB (pay-as-you-go) pricing tier might be advantageous, but this requires careful consideration. ---Also, if you move a subscription to the new Azure monitoring pricing model in April 2018, the Per GB tier is the only tier available. Moving a subscription to the new Azure monitoring pricing model isn't advisable if you have an Operations Management Suite subscription. +Depending on the number of nodes of the suite that your organization purchased, moving some subscriptions into a Per GB (pay-as-you-go) pricing tier might be advantageous, but this change in pricing tier requires careful consideration. > [!TIP] > If your organization has Microsoft Operations Management Suite E1 or E2, it's usually best to keep your Log Analytics workspaces in the Per-Node (OMS) pricing tier and your Application Insights resources in the Enterprise pricing tier. > +## Azure Migrate data benefits ++Workspaces linked to [classic Azure Migrate](/azure/migrate/migrate-services-overview#azure-migrate-versions) receive free data benefits for the data tables related to Azure Migrate (`ServiceMapProcess_CL`, `ServiceMapComputer_CL`, `VMBoundPort`, `VMConnection`, `VMComputer`, `VMProcess`, `InsightsMetrics`). This version of Azure Migrate was retired in February 2024. ++Starting from 1 July 2024, the data benefit for Azure Migrate in Log Analytics will no longer be available. We suggest moving to the [Azure Migrate agentless dependency analysis](/azure/migrate/how-to-create-group-machine-dependencies-agentless). If you continue with agent-based dependency analysis, standard [Azure Monitor charges](https://azure.microsoft.com/pricing/details/monitor/) will apply for the data ingestion that enables dependency visualization. + ## Next steps - See [Azure Monitor Logs pricing details](logs/cost-logs.md) for details on how charges are calculated for data in a Log Analytics workspace and different configuration options to reduce your charges. |
| azure-monitor | Cost Logs | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/logs/cost-logs.md | In some scenarios, combining this data can result in cost savings. Typically, th - [SysmonEvent](/azure/azure-monitor/reference/tables/sysmonevent) - [ProtectionStatus](/azure/azure-monitor/reference/tables/protectionstatus) - [Update](/azure/azure-monitor/reference/tables/update) and [UpdateSummary](/azure/azure-monitor/reference/tables/updatesummary) when the Update Management solution isn't running in the workspace or solution targeting is enabled.+- [MDCFileIntegrityMonitoringEvents](/azure/azure-monitor/reference/tables/mdcfileintegritymonitoringevents) If the workspace is in the legacy Per Node pricing tier, the Defender for Cloud and Log Analytics allocations are combined and applied jointly to all billable ingested data. To learn more on how Microsoft Sentinel customers can benefit, please see the [Microsoft Sentinel Pricing page](https://azure.microsoft.com/pricing/details/microsoft-sentinel/). |
| azure-monitor | Monitor Azure Monitor Reference | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/monitor-azure-monitor-reference.md | + + Title: Monitoring data reference for Azure Monitor +description: This article contains important reference material you need when you monitor Azure Monitor. Last updated : 03/31/2024++++++++# Azure Monitor monitoring data reference +++See [Monitor Azure Monitor](monitor-azure-monitor.md) for details on the data you can collect for Azure Monitor and how to use it. ++<!-- ## Metrics. Required section. --> ++<!-- Repeat the following section for each resource type/namespace in your service. For each ### section, replace the <ResourceType/namespace> placeholder, add the metrics-tableheader #include, and add the table #include. ++To add the table #include, find the table(s) for the resource type in the Metrics column at https://review.learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/metrics-index?branch=main#supported-metrics-and-log-categories-by-resource-type, which is autogenerated from underlying systems. --> ++### Supported metrics for Microsoft.Monitor/accounts +The following table lists the metrics available for the Microsoft.Monitor/accounts resource type. ++### Supported metrics for microsoft.insights/autoscalesettings +The following table lists the metrics available for the microsoft.insights/autoscalesettings resource type. ++### Supported metrics for microsoft.insights/components +The following table lists the metrics available for the microsoft.insights/components resource type. ++### Supported metrics for Microsoft.Insights/datacollectionrules +The following table lists the metrics available for the Microsoft.Insights/datacollectionrules resource type. ++### Supported metrics for Microsoft.operationalinsight/workspaces ++Azure Monitor Logs / Log Analytics workspaces +++<!-- ## Metric dimensions. Required section. --> +++Microsoft.Monitor/accounts: ++- `Stamp color` ++microsoft.insights/autoscalesettings: ++- `MetricTriggerRule` +- `MetricTriggerSource` +- `ScaleDirection` ++microsoft.insights/components: ++- `availabilityResult/name` +- `availabilityResult/location` +- `availabilityResult/success` +- `dependency/type` +- `dependency/performanceBucket` +- `dependency/success` +- `dependency/target` +- `dependency/resultCode` +- `operation/synthetic` +- `cloud/roleInstance` +- `cloud/roleName` +- `client/isServer` +- `client/type` ++Microsoft.Insights/datacollectionrules: ++- `InputStreamId` +- `ResponseCode` +- `ErrorType` +++### Supported resource logs for Microsoft.Monitor/accounts ++### Supported resource logs for microsoft.insights/autoscalesettings ++### Supported resource logs for microsoft.insights/components ++### Supported resource logs for Microsoft.Insights/datacollectionrules +++### Application Insights +microsoft.insights/components ++- [AzureActivity](/azure/azure-monitor/reference/tables/AzureActivity#columns) +- [AzureMetrics](/azure/azure-monitor/reference/tables/AzureMetrics#columns) +- [AppAvailabilityResults](/azure/azure-monitor/reference/tables/AppAvailabilityResults#columns) +- [AppBrowserTimings](/azure/azure-monitor/reference/tables/AutoscaleScaleActionsLog#columns) +- [AppDependencies](/azure/azure-monitor/reference/tables/AppDependencies#columns) +- [AppEvents](/azure/azure-monitor/reference/tables/AppEvents#columns) +- [AppPageViews](/azure/azure-monitor/reference/tables/AppPageViews#columns) +- [AppPerformanceCounters](/azure/azure-monitor/reference/tables/AppPerformanceCounters#columns) +- [AppRequests](/azure/azure-monitor/reference/tables/AppRequests#columns) +- [AppSystemEvents](/azure/azure-monitor/reference/tables/AppSystemEvents#columns) +- [AppTraces](/azure/azure-monitor/reference/tables/AppTraces#columns) +- [AppExceptions](/azure/azure-monitor/reference/tables/AppExceptions#columns) ++### Azure Monitor autoscale settings +Microsoft.Insights/AutoscaleSettings ++- [AzureActivity](/azure/azure-monitor/reference/tables/AzureActivity#columns) +- [AzureMetrics](/azure/azure-monitor/reference/tables/AzureMetrics#columns) +- [AutoscaleEvaluationsLog](/azure/azure-monitor/reference/tables/AutoscaleEvaluationsLog#columns) +- [AutoscaleScaleActionsLog](/azure/azure-monitor/reference/tables/AutoscaleScaleActionsLog#columns) ++### Azure Monitor Workspace +Microsoft.Monitor/accounts ++- [AMWMetricsUsageDetails](/azure/azure-monitor/reference/tables/AMWMetricsUsageDetails#columns) ++### Data Collection Rules +Microsoft.Insights/datacollectionrules ++- [DCRLogErrors](/azure/azure-monitor/reference/tables/DCRLogErrors#columns) ++### Workload Monitoring of Azure Monitor Insights +Microsoft.Insights/WorkloadMonitoring ++- [InsightsMetrics](/azure/azure-monitor/reference/tables/InsightsMetrics#columns) ++- [Monitor resource provider operations](/azure/role-based-access-control/resource-provider-operations#monitor) ++## Related content ++- See [Monitor Azure Monitor](monitor-azure-monitor.md) for a description of monitoring Azure Monitor. +- See [Monitor Azure resources with Azure Monitor](/azure/azure-monitor/essentials/monitor-azure-resource) for details on monitoring Azure resources. |
| azure-monitor | Monitor Azure Monitor | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/monitor-azure-monitor.md | Title: Monitoring Azure Monitor -description: Learn about how Azure Monitor monitors itself + Title: Monitor Azure Monitor +description: Start here to learn how to monitor Azure Monitor. Last updated : 03/31/2024++ - - Previously updated : 04/07/2022- -<!-- VERSION 2.2--> +# Monitor Azure Monitor -# Monitoring Azure Monitor -When you have critical applications and business processes relying on Azure resources, you want to monitor those resources for their availability, performance, and operation. +Azure Monitor has many separate larger components. Information on monitoring each of these components follows. -This article describes the monitoring data generated by Azure Monitor. Azure Monitor uses [itself](./overview.md) to monitor certain parts of its own functionality. You can monitor: +## Azure Monitor core -- Autoscale operations-- Monitoring operations in the audit log+**Autoscale** - Azure Monitor Autoscale has a diagnostics feature that provides insights into the performance of your autoscale settings. For more information, see [Azure Monitor Autoscale diagnostics](autoscale/autoscale-diagnostics.md) and [Troubleshooting using autoscale metrics](autoscale/autoscale-troubleshoot.md#autoscale-metrics). - If you're unfamiliar with the features of Azure Monitor common to all Azure services that use it, read [Monitoring Azure resources with Azure Monitor](./essentials/monitor-azure-resource.md). +**Agent Monitoring** - You can now monitor the health of your agents easily and seamlessly across Azure, on premises and other clouds using this interactive experience. For more information, see [Azure Monitor Agent Health](agents/azure-monitor-agent-health.md). -For an overview showing where autoscale and the audit log fit into Azure Monitor, see [Introduction to Azure Monitor](overview.md). +**Data Collection Rules(DCRs)** - Use [detailed metrics and log](essentials/data-collection-monitor.md) to monitor the performance of your DCRs. -## Monitoring overview page in Azure portal +## Azure Monitor Logs and Log Analytics -The **Overview** page in the Azure portal for Azure Monitor shows links and tutorials on how to use Azure Monitor in general. It doesn't mention any of the specific resources discussed later in this article. +**[Log Analytics Workspace Insights](logs/log-analytics-workspace-insights-overview.md)** provides a dashboard that shows you the volume of data going through your workspace(s). You can calculate the cost of your workspace based on the data volume. + +**[Log Analytics workspace health](logs/log-analytics-workspace-health.md)** provides a set of queries that you can use to monitor the health of your workspace. -## Monitoring data +**Optimizing and troubleshooting log queries** - Sometimes Azure Monitor KQL Log queries can take more time to run than needed or never return at all. By monitoring the various aspects of the query, you can troubleshoot and optimize them. For more information, see [Audit queries in Azure Monitor Logs](logs/query-audit.md) and [Optimize log queries](logs/query-optimization.md). -Azure Monitor collects the same kinds of monitoring data as other Azure resources that are described in [Monitoring data from Azure resources](./essentials/monitor-azure-resource.md#monitoring-data-from-azure-resources). +**Log Ingestion pipeline latency** - Azure Monitor provides a highly scalable log ingestion pipeline that can ingest logs from any source. You can monitor the latency of this pipeline using Kusto queries. For more information, see [Log data ingestion time in Azure Monitor](logs/data-ingestion-time.md#check-ingestion-time). -See [Monitoring *Azure Monitor* data reference](azure-monitor-monitoring-reference.md) for detailed information on the metrics and logs metrics created by Azure Monitor. +**Log Analytics usage** - You can monitor the data ingestion for your Log Analytics workspace. For more information, see [Analyze usage in Log Analytics](logs/analyze-usage.md). -## Collection and routing +## All resources -Platform metrics and the Activity log are collected and stored automatically, but can be routed to other locations by using a diagnostic setting. +**Health of any Azure resource** - Azure Monitor resources are tied into the resource health feature, which provides insights into the health of any Azure resource. For more information, see [Resource health](/azure/service-health/resource-health-overview/). -Resource Logs aren't collected and stored until you create a diagnostic setting and route them to one or more locations. -See [Create diagnostic setting to collect platform logs and metrics in Azure](/azure/azure-monitor/platform/diagnostic-settings) for the detailed process for creating a diagnostic setting using the Azure portal, CLI, or PowerShell. When you create a diagnostic setting, you specify which categories of logs to collect. The categories for *Azure Monitor* are listed in [Azure Monitor monitoring data reference](azure-monitor-monitoring-reference.md#resource-logs). -The metrics and logs you can collect are discussed in the following sections. +For more information about the resource types for Azure Monitor, see [Azure Monitor monitoring data reference](monitor-azure-monitor-reference.md). -## Analyzing metrics -You can analyze metrics for *Azure Monitor* with metrics from other Azure services using metrics explorer by opening **Metrics** from the **Azure Monitor** menu. See [Analyze metrics with Azure Monitor metrics explorer](./essentials/analyze-metrics.md) for details on using this tool. -For a list of the platform metrics collected for Azure Monitor into itself, see [Azure Monitor monitoring data reference](azure-monitor-monitoring-reference.md#metrics). +For a list of available metrics for Azure Monitor, see [Azure Monitor monitoring data reference](monitor-azure-monitor-reference.md#metrics). -For reference, you can see a list of [all resource metrics supported in Azure Monitor](./essentials/metrics-supported.md). -<!-- Optional: Call out additional information to help your customers. For example, you can include additional information here about how to use metrics explorer specifically for your service. Remember that the UI is subject to change quite often so you will need to maintain these screenshots yourself if you add them in. --> +For the available resource log categories, their associated Log Analytics tables, and the logs schemas for Azure Monitor, see [Azure Monitor monitoring data reference](monitor-azure-monitor-reference.md#resource-logs). -## Analyzing logs -Data in Azure Monitor Logs is stored in tables where each table has its own set of unique properties. -All resource logs in Azure Monitor have the same fields followed by service-specific fields. The common schema is outlined in [Azure Monitor resource log schema](./essentials/resource-logs-schema.md) The schemas for autoscale resource logs are found in the [Azure Monitor Data Reference](azure-monitor-monitoring-reference.md#resource-logs) -The [Activity log](./essentials/activity-log.md) is a type of platform log in Azure that provides insight into subscription-level events. You can view it independently or route it to Azure Monitor Logs, where you can do much more complex queries using Log Analytics. -For a list of the types of resource logs collected for Azure Monitor, see [Monitoring Azure Monitor data reference](azure-monitor-monitoring-reference.md#resource-logs). -For a list of the tables used by Azure Monitor Logs and queryable by Log Analytics, see [Monitoring Azure Monitor data reference](azure-monitor-monitoring-reference.md#azure-monitor-logs-tables) +Refer to the links in the beginning of this article for specific Kusto queries for each of the Azure Monitor components. -### Sample Kusto queries -These are now listed in the [Log Analytics user interface](./logs/queries.md). +## Related content -## Alerts --Azure Monitor alerts proactively notify you when important conditions are found in your monitoring data. They allow you to identify and address issues in your system before your customers notice them. You can set alerts on [metrics](./alerts/alerts-metric-overview.md), [logs](./alerts/alerts-types.md#log-alerts), and the [activity log](./alerts/activity-log-alerts.md). Different types of alerts have benefits and drawbacks. --For an in-depth discussion of using alerts with autoscale, see [Troubleshoot Azure autoscale](./autoscale/autoscale-troubleshoot.md). --## Next steps --- See [Monitoring Azure Monitor data reference](azure-monitor-monitoring-reference.md) for a reference of the metrics, logs, and other important values created by Azure Monitor to monitor itself.-- See [Monitoring Azure resources with Azure Monitor](./essentials/monitor-azure-resource.md) for details on monitoring Azure resources.+- See [Azure Monitor monitoring data reference](monitor-azure-monitor-reference.md) for a reference of the metrics, logs, and other important values created for Azure Monitor. +- See [Monitoring Azure resources with Azure Monitor](/azure/azure-monitor/essentials/monitor-azure-resource) for general details on monitoring Azure resources. |
| azure-monitor | Snapshot Debugger Vm | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/snapshot-debugger/snapshot-debugger-vm.md | using Microsoft.ApplicationInsights.SnapshotCollector; builder.Services.Configure<SnapshotCollectorConfiguration>(builder.Configuration.GetSection("SnapshotCollector")); ``` -Next, add a `SnapshotCollector` section to *appsettings.json* where you can override the defaults. The following example shows a configuration equivalent to the default configuration: +Next, add a `SnapshotCollector` section to _appsettings.json_ where you can override the defaults. The following example shows a configuration equivalent to the default configuration: ```json { Next, add a `SnapshotCollector` section to *appsettings.json* where you can over } ``` -If you need to customize the Snapshot Collector's behavior manually, without using *appsettings.json*, use the overload of `AddSnapshotCollector` that takes a delegate. For example: +If you need to customize the Snapshot Collector's behavior manually, without using _appsettings.json_, use the overload of `AddSnapshotCollector` that takes a delegate. For example: ```csharp builder.Services.AddSnapshotCollector(config => config.IsEnabledInDeveloperMode = true); ``` builder.Services.AddSnapshotCollector(config => config.IsEnabledInDeveloperMode Snapshots are collected only on exceptions that are reported to Application Insights. For ASP.NET and ASP.NET Core applications, the Application Insights SDK automatically reports unhandled exceptions that escape a controller method or endpoint route handler. For other applications, you might need to modify your code to report them. The exception handling code depends on the structure of your application. Here's an example: ```csharp-TelemetryClient _telemetryClient = new TelemetryClient(); -void ExampleRequest() +using Microsoft.ApplicationInsights; +using Microsoft.ApplicationInsights.DataContracts; +using Microsoft.ApplicationInsights.Extensibility; ++internal class ExampleService {+ private readonly TelemetryClient _telemetryClient; ++ public ExampleService(TelemetryClient telemetryClient) + { + // Obtain the TelemetryClient via dependency injection. + _telemetryClient = telemetryClient; + } ++ public void HandleExampleRequest() + { + using IOperationHolder<RequestTelemetry> operation = + _telemetryClient.StartOperation<RequestTelemetry>("Example"); try {- // TODO: Handle the request. + // TODO: Handle the request. + operation.Telemetry.Success = true; } catch (Exception ex) {- // Report the exception to Application Insights. - _telemetryClient.TrackException(ex); - // TODO: Rethrow the exception if desired. + // Report the exception to Application Insights. + operation.Telemetry.Success = false; + _telemetryClient.TrackException(ex); + // TODO: Rethrow the exception if desired. }+ } } ``` +The following example uses `ILogger` instead of `TelemetryClient`. This example assumes you're using the [Application Insights Logger Provider](../app/ilogger.md#console-application). As the example shows, when handling an exception, be sure to pass the exception as the first parameter to `LogError`. ++```csharp +using Microsoft.Extensions.Logging; ++internal class LoggerExample +{ + private readonly ILogger _logger; ++ public LoggerExample(ILogger<LoggerExample> logger) + { + _logger = logger; + } ++ public void HandleExampleRequest() + { + using IDisposable scope = _logger.BeginScope("Example"); + try + { + // TODO: Handle the request + } + catch (Exception ex) + { + // Use the LogError overload with an Exception as the first parameter. + _logger.LogError(ex, "An error occurred."); + } + } +} +``` ++> [!NOTE] +> By default, the Application Insights Logger (`ApplicationInsightsLoggerProvider`) forwards exceptions to the Snapshot Debugger via `TelemetryClient.TrackException`. This behavior is controlled via the `TrackExceptionsAsExceptionTelemetry` property on the `ApplicationInsightsLoggerOptions` class. If you set `TrackExceptionsAsExceptionTelemetry` to `false` when configuring the Application Insights Logger, then the preceding example will not trigger the Snapshot Debugger. In this case, modify your code to call `TrackException` manually. + [!INCLUDE [azure-monitor-log-analytics-rebrand](../../../includes/azure-monitor-instrumentation-key-deprecation.md)] ## Next steps - Generate traffic to your application that can trigger an exception. Then wait 10 to 15 minutes for snapshots to be sent to the Application Insights instance. - See [snapshots](snapshot-debugger-data.md?toc=/azure/azure-monitor/toc.json#view-snapshots-in-the-portal) in the Azure portal.-- For help with troubleshooting Snapshot Debugger issues, see [Snapshot Debugger troubleshooting](snapshot-debugger-troubleshoot.md).+- [Troubleshoot](snapshot-debugger-troubleshoot.md) Snapshot Debugger problems. |
| azure-monitor | Whats New | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-monitor/whats-new.md | Title: "What's new in Azure Monitor documentation" description: "What's new in Azure Monitor documentation" Previously updated : 02/08/2024 Last updated : 04/04/2024 This article lists significant changes to Azure Monitor documentation. ## [2024](#tab/2024) +## March 2024 ++|Subservice | Article | Description | +|||| +|Alerts|[Improve the reliability of your application by using Azure Advisor](../../articles/advisor/advisor-high-availability-recommendations.md)|WeΓÇÖve updated the alerts troubleshooting articles to remove out of date content and include common support issues.| +|Application-Insights|[Enable Azure Monitor OpenTelemetry for .NET, Node.js, Python, and Java applications](app/opentelemetry-enable.md)|OpenTelemetry sample applications are now provided in a centralized location.| +|Application-Insights|[Migrate to workspace-based Application Insights resources](app/convert-classic-resource.md)|Classic Application Insights resources have been retired. For more information, see this article for migration information and frequently asked questions.| +|Application-Insights|[Sampling overrides - Azure Monitor Application Insights for Java](app/java-standalone-sampling-overrides.md)|The sampling overrides feature has reached general availability (GA), starting from 3.5.0.| +|Containers|[Configure data collection and cost optimization in Container insights using data collection rule](containers/container-insights-data-collection-dcr.md)|Updated to include new Logs and Events cost preset.| +|Containers|[Enable private link with Container insights](containers/container-insights-private-link.md)|Updated with ARM templates.| +|Essentials|[Data collection rules in Azure Monitor](essentials/data-collection-rule-overview.md)|Rewritten to consolidate previous data collection article.| +|Essentials|[Workspace transformation data collection rule (DCR) in Azure Monitor](essentials/data-collection-transformations-workspace.md)|Content moved to a new article dedicated to workspace transformation DCR.| +|Essentials|[Data collection transformations in Azure Monitor](essentials/data-collection-transformations.md)|Rewritten to remove redundancy and make the article more consistent with related articles.| +|Essentials|[Create and edit data collection rules (DCRs) in Azure Monitor](essentials/data-collection-rule-create-edit.md)|Updated API version in REST API calls.| +|Essentials|[Tutorial: Edit a data collection rule (DCR)](essentials/data-collection-rule-edit.md)|Updated API version in REST API calls.| +|Essentials|[Monitor and troubleshoot DCR data collection in Azure Monitor](essentials/data-collection-monitor.md)|New article documenting new DCR monitoring feature.| +|Logs|[Monitor Log Analytics workspace health](logs/log-analytics-workspace-health.md)|Added new metrics for monitoring data export from a Log Analytics workspace.| +|Logs|[Set a table's log data plan to Basic or Analytics](logs/basic-logs-configure.md)|Azure Databricks logs tables now support the basic logs data plan.| + ## February 2024 |Subservice | Article | Description | |
| azure-netapp-files | Azure Netapp Files Performance Considerations | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-netapp-files/azure-netapp-files-performance-considerations.md | -The combination of the quota assigned to the volume and the selected service level determines the [throughput limit](azure-netapp-files-service-levels.md) for a volume with automatic QoS . For volumes with manual QoS, the throughput limit can be defined individually. When you make performance plans about Azure NetApp Files, you need to understand several considerations. +The combination of the quota assigned to the volume and the selected service level determines the [throughput limit](azure-netapp-files-service-levels.md) for a volume with automatic QoS. For volumes with manual QoS, the throughput limit can be defined individually. When you make performance plans about Azure NetApp Files, you need to understand several considerations. ## Quota and throughput Typical storage performance considerations contribute to the total performance d Metrics are reported as aggregates of multiple data points collected during a five-minute interval. For more information about metrics aggregation, see [Azure Monitor Metrics aggregation and display explained](../azure-monitor/essentials/metrics-aggregation-explained.md). -The maximum empirical throughput that has been observed in testing is 4,500 MiB/s. At the Premium storage tier, an automatic QoS volume quota of 70.31 TiB will provision a throughput limit that is high enough to achieve this level of performance. +The maximum empirical throughput that has been observed in testing is 4,500 MiB/s. At the Premium storage tier, an automatic QoS volume quota of 70.31 TiB provisions a throughput limit high enough to achieve this performance level. -For automatic QoS volumes, if you are considering assigning volume quota amounts beyond 70.31 TiB, additional quota may be assigned to a volume for storing more data. However, the added quota doesn't result in a further increase in actual throughput. +For automatic QoS volumes, if you're considering assigning volume quota amounts beyond 70.31 TiB, additional quota may be assigned to a volume for storing more data. However, the added quota doesn't result in a further increase in actual throughput. The same empirical throughput ceiling applies to volumes with manual QoS. The maximum throughput can assign to a volume is 4,500 MiB/s. ## Automatic QoS volume quota and throughput -This section describes quota management and throughput for volumes with the automatic QoS type. +Learn about quota management and throughput for volumes with the automatic QoS type. ### Overprovisioning the volume quota -If a workloadΓÇÖs performance is throughput-limit bound, it is possible to overprovision the automatic QoS volume quota to set a higher throughput level and achieve higher performance. +If a workloadΓÇÖs performance is throughput-limit bound, it's possible to overprovision the automatic QoS volume quota to set a higher throughput level and achieve higher performance. -For example, if an automatic QoS volume in the Premium storage tier has only 500 GiB of data but requires 128 MiB/s of throughput, you can set the quota to 2 TiB so that the throughput level is set accordingly (64 MiB/s per TB * 2 TiB = 128 MiB/s). +For example, if an automatic QoS volume in the Premium storage tier has only 500 GiB of data but requires 128 MiB/s of throughput, you can set the quota to 2 TiB so the throughput level is set accordingly (64 MiB/s per TB * 2 TiB = 128 MiB/s). -If you consistently overprovision a volume for achieving a higher throughput, consider using the manual QoS volumes or using a higher service level instead. In this example, you can achieve the same throughput limit with half the automatic QoS volume quota by using the Ultra storage tier instead (128 MiB/s per TiB * 1 TiB = 128 MiB/s). +If you consistently overprovision a volume for achieving a higher throughput, consider using the manual QoS volumes or using a higher service level instead. In this example, you can achieve the same throughput limit with half the automatic QoS volume quota by using the Ultra storage tier instead (128 MiB/s per TiB * 1 TiB = 128 MiB/s). ### Dynamically increasing or decreasing volume quota If your performance requirements are temporary in nature, or if you have increas If you use manual QoS volumes, you donΓÇÖt have to overprovision the volume quota to achieve a higher throughput because the throughput can be assigned to each volume independently. However, you still need to ensure that the capacity pool is pre-provisioned with sufficient throughput for your performance needs. The throughput of a capacity pool is provisioned according to its size and service level. See [Service levels for Azure NetApp Files](azure-netapp-files-service-levels.md) for more details. +## Monitoring volumes for performance ++Azure NetApp Files volumes can be monitored using available [Performance metrics](azure-netapp-files-metrics.md#performance-metrics-for-volumes). ++When volume throughput reaches its maximum (as determined by the QoS setting), the volume response times (latency) increase. This effect can be incorrectly perceived as a performance issue caused by the storage. Increasing the volume QoS setting (manual QoS) or increasing the volume size (auto QoS) increases the allowable volume throughput. ++To check if the maximum throughput limit has been reached, monitor the metric [Throughput limit reached](azure-netapp-files-metrics.md#volumes). For more recommendations, see [Performance FAQs for Azure NetApp Files](faq-performance.md#what-should-i-do-to-optimize-or-tune-azure-netapp-files-performance). ## Next steps |
| azure-netapp-files | Faq Performance | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-netapp-files/faq-performance.md | You can take the following actions per the performance requirements: There is no need to set accelerated networking for the NICs in the dedicated subnet of Azure NetApp Files. [Accelerated networking](../virtual-network/virtual-machine-network-throughput.md) is a capability that only applies to Azure virtual machines. Azure NetApp Files NICs are optimized by design. +## How do I monitor Azure NetApp Files volume performance ++Azure NetApp Files volumes performance can be monitored through [available metrics](azure-netapp-files-metrics.md). + ## How do I convert throughput-based service levels of Azure NetApp Files to IOPS? You can convert MB/s to IOPS by using the following formula: No, Azure NetApp Files does not support SMB Direct. ## Is NIC Teaming supported in Azure? -NIC Teaming is not supported in Azure. Although multiple network interfaces are supported on Azure virtual machines, they represent a logical rather than a physical construct. As such, they provide no fault tolerance. Also, the bandwidth available to an Azure virtual machine is calculated for the machine itself and not any individual network interface. +NIC Teaming isn't supported in Azure. Although multiple network interfaces are supported on Azure virtual machines, they represent a logical rather than a physical construct. As such, they provide no fault tolerance. Also, the bandwidth available to an Azure virtual machine is calculated for the machine itself and not any individual network interface. ## Are jumbo frames supported? -Jumbo frames are not supported with Azure virtual machines. +Jumbo frames aren't supported with Azure virtual machines. ## Next steps |
| azure-portal | Microsoft Entra Id | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/azure-portal/mobile-app/microsoft-entra-id.md | Title: Use Microsoft Entra ID with the Azure mobile app description: Use the Azure mobile app to manage users and groups with Microsoft Entra ID. Previously updated : 03/08/2024 Last updated : 04/04/2024 The Azure mobile app provides access to Microsoft Entra ID. You can perform task To access Microsoft Entra ID, open the Azure mobile app and sign in with your Azure account. From **Home**, scroll down to select the **Microsoft Entra ID** card. > [!NOTE]-> Your account must have the appropriate permissions in order to perform these tasks. For example, to invite a user to your tenant, you must have a role that includes this permission, such as [Guest Inviter](/entra/identity/role-based-access-control/permissions-reference) role or [User Administrator](/entra/identity/role-based-access-control/permissions-reference). +> Your account must have the appropriate permissions in order to perform these tasks. For example, to invite a user to your tenant, you must have a role that includes this permission, such as [Guest Inviter](/entra/identity/role-based-access-control/permissions-reference) or [User Administrator](/entra/identity/role-based-access-control/permissions-reference). ## Invite a user to the tenant To add one or more users to a group from the Azure mobile app: 1. Search or scroll to find the desired group, then tap to select it. 1. On the **Members** card, select **See All**. The current list of members is displayed. 1. Select the **+** icon in the top right corner.-1. Search or scroll to find users you want to add to the group, then select the user(s) by tapping the circle next to their name. -1. Select **Add** in the top right corner to add the selected users(s) to the group. +1. Search or scroll to find users you want to add to the group, then select one or more users by tapping the circle next to their name. +1. Select **Add** in the top right corner to add the selected users to the group. ## Add group memberships for a specified user You can also add a single user to one or more groups in the **Users** section of 1. In **Microsoft Entra ID**, select **Users**, then search or scroll to find and select the desired user. 1. On the **Groups** card, select **See All** to display all current group memberships for that user. 1. Select the **+** icon in the top right corner.-1. Search or scroll to find groups to which this user should be added, then select the group(s) by tapping the circle next to the group name. -1. Select **Add** in the top right corner to add the user to the selected group(s). +1. Search or scroll to find groups to which this user should be added, then select one or more groups by tapping the circle next to the group name. +1. Select **Add** in the top right corner to add the user to the selected groups. ## Manage authentication methods or reset password for a user -To [manage authentication methods](/entra/identity/authentication/concept-authentication-methods-manage) or [reset a user's password](/entra/fundamentals/users-reset-password-azure-portal), you need to do the following steps: +To [manage authentication methods](/entra/identity/authentication/concept-authentication-methods-manage) or [reset a user's password](/entra/fundamentals/users-reset-password-azure-portal): 1. In **Microsoft Entra ID**, select **Users**, then search or scroll to find and select the desired user. 1. On the **Authentication methods** card, select **Manage**.-1. Select **Reset password** to assign a temporary password to the user, or **Authentication methods** to manage to Tap on the desired user, then tap on ΓÇ£Reset passwordΓÇ¥ or ΓÇ£Authentication methodsΓÇ¥ based on your permissions. +1. Select **Reset password** to assign a temporary password to the user, or **Authentication methods** to manage authentication methods for self-service password reset. > [!NOTE] > You won't see the **Authentication methods** card if you don't have the appropriate permissions to manage authentication methods and/or password changes for a user. +## Investigate risky users and sign-ins ++[Microsoft Entra ID Protection](/entra/id-protection/overview-identity-protection) provides organizations with reporting they can use to [investigate identity risks in their environment](/entra/id-protection/howto-identity-protection-investigate-risk). ++If you have the [necessary permissions and license](/entra/id-protection/overview-identity-protection#required-roles), you'll see details in the **Risky users** and **Risky sign-ins** sections within **Microsoft Entra ID**. You can open these sections to view more information and perform some management tasks. ++### Manage risky users ++1. In **Microsoft Entra ID**, scroll down to the **Security** card and then select **Risky users**. +1. Search or scroll to find and select a specific risky user. +1. Review basic information for this user, a list of their risky sign-ins, and their risk history. +1. To [take action on the user](/entra/id-protection/howto-identity-protection-investigate-risk), select the three dots near the top of the screen. You can: ++ * Reset the user's password + * Confirm user compromise + * Dismiss user risk + * Block the user from signing in (or unblock, if previously blocked) ++### Monitor risky sign-ins ++1. In **Microsoft Entra ID**, scroll down to the **Security** card and then select **Risky sign-ins**. It may take a minute or two for the list of all risky sign-ins to load. ++1. Search or scroll to find and select a specific risky sign-in. ++1. Review details about the risky sign-in. + ## Activate Privileged Identity Management (PIM) roles If you have been made eligible for an administrative role through Microsoft Entra Privileged Identity Management (PIM), you must activate the role assignment when you need to perform privileged actions. This activation can be done from within the Azure mobile app. |
| backup | Azure File Share Support Matrix | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/backup/azure-file-share-support-matrix.md | Vaulted backup for Azure Files (preview) is available in West Central US, Southe | File share type | Support | | -- | |-| Standard | Supported | +| Standard (with large file shares enabled) | Supported | | Large | Supported | | Premium | Supported | | File shares connected with Azure File Sync service | Supported | |
| backup | Backup Azure Files | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/backup/backup-azure-files.md | Title: Back up Azure File shares in the Azure portal description: Learn how to use the Azure portal to back up Azure File shares in the Recovery Services vault Previously updated : 03/04/2024 Last updated : 04/05/2024 Azure File share backup is a native, cloud based backup solution that protects y ## Prerequisites -* Ensure that the file share is present in one of the [supported storage account types](azure-file-share-support-matrix.md). +* Ensure that the file share is present in one of the supported storage account types. Review the [support matrix](azure-file-share-support-matrix.md). * Identify or create a [Recovery Services vault](#create-a-recovery-services-vault) in the same region and subscription as the storage account that hosts the file share. * In case you have restricted access to your storage account, check the firewall settings of the account to ensure that the exception "Allow Azure services on the trusted services list to access this storage account" is granted. You can refer to [this](../storage/common/storage-network-security.md?tabs=azure-portal#manage-exceptions) link for the steps to grant an exception. |
| backup | Restore Afs | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/backup/restore-afs.md | Title: Restore Azure File shares description: Learn how to use the Azure portal to restore an entire file share or specific files from a restore point created by Azure Backup. Previously updated : 03/04/2024 Last updated : 04/05/2024 You can also monitor restore progress from the Recovery Services vault: >[!NOTE] >- Folders will be restored with original permissions if there is atleast one file present in them. >- Trailing dots in any directory path can lead to failures in the restore.+>- Restore of a file or folder with length *>2 KB* or with characters `xFFFF` or `xFFFE` isn't supported from snapshots. + ## Next steps |
| bastion | Bastion Faq | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/bastion/bastion-faq.md | Azure Bastion doesn't move or store customer data out of the region it's deploye Some regions support the ability to deploy Azure Bastion in an availability zone (or multiple, for zone redundancy). To deploy zonally, you can select the availability zones you want to deploy under instance details when you deploy Bastion using manually specified settings. You can't change zonal availability after Bastion is deployed. If you aren't able to select a zone, you might have selected an Azure region that doesn't yet support availability zones.-For more information about availability zones, see [Availability Zones](https://learn.microsoft.com/azure/reliability/availability-zones-overview?tabs=azure-cli). +For more information about availability zones, see [Availability Zones](../reliability/availability-zones-overview.md?tabs=azure-cli). ### <a name="vwan"></a>Does Azure Bastion support Virtual WAN? |
| bastion | Bastion Vm Copy Paste | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/bastion/bastion-vm-copy-paste.md | description: Learn how copy and paste to and from a Windows VM using Bastion. Previously updated : 10/31/2023 Last updated : 04/04/2024 # Customer intent: I want to copy and paste to and from VMs using Azure Bastion. By default, Azure Bastion is automatically enabled to allow copy and paste for a ## <a name="to"></a> Copy and paste -For browsers that support the advanced Clipboard API access, you can copy and paste text between your local device and the remote session in the same way you copy and paste between applications on your local device. For other browsers, you can use the Bastion clipboard access tool palette. +For browsers that support the advanced Clipboard API access, you can copy and paste text between your local device and the remote session in the same way you copy and paste between applications on your local device. For other browsers, you can use the Bastion clipboard access tool palette. Note that copy and paste isn't supported for passwords. > [!NOTE] > Only text copy/paste is currently supported. |
| bastion | Tutorial Create Host Portal | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/bastion/tutorial-create-host-portal.md | This section helps you deploy Bastion to your virtual network. After Bastion is * **Region**: The Azure public region in which the resource will be created. Choose the region where your virtual network resides. - * **Availability zone**: Select the zone(s) from the dropdown, if desired. Only certain regions are supported. For more information, see the [What are availability zones?](https://learn.microsoft.com/azure/reliability/availability-zones-overview?tabs=azure-cli) article. + * **Availability zone**: Select the zone(s) from the dropdown, if desired. Only certain regions are supported. For more information, see the [What are availability zones?](../reliability/availability-zones-overview.md?tabs=azure-cli) article. * **Tier**: The SKU. For this tutorial, select **Standard**. For information about the features available for each SKU, see [Configuration settings - SKU](configuration-settings.md#skus). |
| batch | Accounts | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/batch/accounts.md | Title: Batch accounts and Azure Storage accounts description: Learn about Azure Batch accounts and how they're used from a development standpoint. Previously updated : 06/01/2023 Last updated : 04/04/2024 # Batch accounts and Azure Storage accounts An Azure Batch account is a uniquely identified entity within the Batch service. ## Batch accounts -All processing and resources are associated with a Batch account. When your application makes a request against the Batch service, it authenticates the request using the Azure Batch account name, the URL of the account, and either an access key or a Microsoft Entra token. +All processing and resources are associated with a Batch account. When your application makes a request against the Batch service, it authenticates the request using the Azure Batch account name and the account URL. Additionally, it can use either an access key or a Microsoft Entra token. You can run multiple Batch workloads in a single Batch account. You can also distribute your workloads among Batch accounts that are in the same subscription but located in different Azure regions. For more information about storage accounts, see [Azure storage account overview You can associate a storage account with your Batch account when you create the Batch account, or later. Consider your cost and performance requirements when choosing a storage account. For example, the GPv2 and blob storage account options support greater [capacity and scalability limits](https://azure.microsoft.com/blog/announcing-larger-higher-scale-storage-accounts/) compared with GPv1. (Contact Azure Support to request an increase in a storage limit.) These account options can improve the performance of Batch solutions that contain a large number of parallel tasks that read from or write to the storage account. -When a storage account is linked to a Batch account, it's considered to be the *autostorage account*. An autostorage account is required if you plan to use the [application packages](batch-application-packages.md) capability, as it's used to store the application package .zip files. It can also be used for [task resource files](resource-files.md#storage-container-name-autostorage). Linking Batch accounts to autostorage can avoid the need for shared access signature (SAS) URLs to access the resource files. +When a storage account is linked to a Batch account, it becomes the *autostorage account*. An autostorage account is necessary if you intend to use the [application packages](batch-application-packages.md) capability, as it stores the application package .zip files. It can also be used for [task resource files](resource-files.md#storage-container-name-autostorage). Linking Batch accounts to autostorage can avoid the need for shared access signature (SAS) URLs to access the resource files. ++> [!NOTE] +> Batch nodes automatically unzip application package .zip files when they are pulled down from a linked storage account. This can cause the compute node local storage to fill up. For more information, see [Manage Batch application package](/cli/azure/batch/application/package). ## Next steps |
| batch | Batch Account Create Portal | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/batch/batch-account-create-portal.md | Title: Create a Batch account in the Azure portal description: Learn how to use the Azure portal to create and manage an Azure Batch account for running large-scale parallel workloads in the cloud. Previously updated : 07/18/2023- Last updated : 04/04/2024+ # Create a Batch account in the Azure portal Select **Add**, then ensure that the **Azure Virtual Machines for deployment** a :::image type="content" source="media/batch-account-create-portal/key-vault-access-policy.png" alt-text="Screenshot of the Access policy screen."::: -->+> [!NOTE] +> Currently, the Batch account name supports only access policies. When creating a Batch account, ensure that the key vault uses the associated access policy instead of the EntraID RBAC permissions. For more information on how to add an access policy to your Azure key vault instance, see [Configure your Azure Key Vault instance](batch-customer-managed-key.md). ### Configure subscription quotas |
| communication-services | Email Optout Management | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/concepts/email/email-optout-management.md | -This article provides the Email delivery best practices and how to use the Azure Communication Services Email suppression list feature that allows customers to manage opt-out capabilities for email communications. It also provides information on the features that are important for emails opt out management that helps you improve email complaint management, promote better email practices, and increase your email delivery success, boosting the likelihood of getting to recipients' inboxes efficiently. +This article provides Email delivery best practices and describes how to use the Azure Communication Services Email suppression list. This feature enables customers to manage opt-out capabilities for email communications. It also provides information about the features that are important for emails opt out management. Use these features to improve email compliance management, promote better email practices, increase your email delivery success, and boost the likelihood of reaching recipient inboxes. ## Opt out or unsubscribe management: Ensuring transparent sender reputation-It's important to know how interested your customers are in your email communication and to respect their opt-out or unsubscribe requests when they decide not to get emails from you. This helps you keep a good sender reputation. Whether you have a manual or automated process in place for handling unsubscribes, it's important to provide an "unsubscribe" link in the email payload you send. When recipients decide not to receive further emails, they can click on the 'unsubscribe' link and remove their email address from your mailing list. +It's important to know how interested your customers are in your email communication and to respect their opt-out or unsubscribe requests when they decide not to get emails from you. This helps you keep a good sender reputation. Whether you have a manual or automated process in place for handling unsubscribes, it's important to provide an "unsubscribe" link in the email payload you send. When recipients decide not to receive further emails, they can click on the 'unsubscribe' link to remove their email address from your mailing list. -The functionality of the links and instructions in the email is vital; they must be working correctly and promptly notify the application mailing list to remove the contact from the appropriate list or lists. A proper unsubscribe mechanism should be explicit and transparent from the subscriber's perspective, ensuring they know precisely which messages they're unsubscribing from. Ideally, they should be offered a preferences center that gives them the option to unsubscribe in cases where they're subscribed to multiple lists within your organization. This process prevents accidental unsubscribes and allows users to manage their opt-in and opt-out preferences effectively through the unsubscribe management process. +The function of the links and instructions in the email is vital. They must be working correctly and promptly notify the application mailing list to remove the contact from the appropriate list or lists. A proper unsubscribe mechanism should be explicit and transparent from the subscriber's perspective. This helps ensure that they know precisely which messages they're unsubscribing from. Ideally, they should be offered a preferences center that gives them the option to unsubscribe in cases where they're subscribed to multiple lists within your organization. This process prevents accidental unsubscribes and enables users to manage their opt-in and opt-out preferences effectively through the unsubscribe management process. ## Managing emails opt out preferences with suppression list in Azure Communication Service Email-Azure Communication Service Email offers a powerful platform with a centralized managed unsubscribe list with opt out preferences saved to our data store. This feature helps the developers to meet guidelines of email providers, requiring one-click list-unsubscribe implementation in the emails sent from our platform. To proactively identify and avoid significant delivery problems, suppression list features, including but not limited to: ++Azure Communication Service Email offers a powerful platform with a centralized managed unsubscribe list with opt-out preferences saved to our data store. This feature helps developers meet the guidelines of email providers, requiring one-click list-unsubscribe implementation in the emails sent from our platform. To proactively identify and avoid significant delivery problems, suppression list features include but aren't limited to: * Offers domain-level, customer managed lists that provide opt-out capabilities. * Provides Azure resources that allow for Create, Read, Update, and Delete (CRUD) operations via Azure portal, Management SDKs, or REST APIs. Azure Communication Service Email offers a powerful platform with a centralized * Adds Email addresses programmatically for an easy opt-out process for unsubscribing. ### Benefits of opt out or unsubscribe management+ Using a suppression list in Azure Communication Services offers several benefits:-* Compliance and Legal Considerations: This feature is crucial for adhering to legal responsibilities defined in local government legislation like the CAN-SPAM Act in the United States. It ensures that customers can easily manage opt-outs and maintain compliance with these regulations. -* Better Sender Reputation: When emails aren't sent to users who have chosen to opt out, it helps protect the senderΓÇÖs reputation and lowers the chance of being blocked by email providers. -* Improved User Experience: It respects the preferences of users who don't wish to receive communications, leading to a better user experience and potentially higher engagement rates with recipients who choose to receive emails. -* Operational Efficiency: Suppression lists can be managed programmatically, allowing for efficient handling of large numbers of opt-out requests without manual intervention. -* Cost-Effectiveness: By not sending emails to recipients who opted out, it reduces the volume of sent emails, which can lower operational costs associated with email delivery. -* Data-Driven Decisions: The suppression list feature provides insights into the number of opt-outs, which can be valuable data for making informed decisions about email campaign strategies. +* Compliance and Legal Considerations: Use opt-out links to meet legal responsibilities defined in local government legislation like the CAN-SPAM Act in the United States. It ensures that customers can easily manage opt-outs and maintain compliance with these regulations. +* Better Sender Reputation: When emails aren't sent to users who opted out, it helps protect the senderΓÇÖs reputation and lower the chance of being blocked by email providers. +* Improved User Experience: It respects the preferences of users who don't wish to receive communications. Collecting and storing email preferences lead to a better user experience and potentially higher engagement rates with recipients who choose to receive emails. +* Operational Efficiency: Suppression lists can be managed programmatically. Use this feature to efficiently handle large numbers of opt-out requests without manual intervention. +* Cost-Effectiveness: By not sending emails to recipients who opted out, it reduces the volume of sent emails. This can lower operational costs associated with email delivery. +* Data-Driven Decisions: The suppression list feature provides insights into the number of opt-outs. Use this valuable data to make informed decisions about email campaign strategies. -These benefits contribute to a more efficient, compliant, and user-friendly email communication system when using Azure Communication Services. To enable email logs and monitor your email delivery, follow the steps outlined in [Azure Communication Services email logs Communication Service in Azure Communication Service](../../concepts/analytics/logs/email-logs.md). +These benefits contribute to a more efficient, compliant, and user-friendly email communication system using Azure Communication Services. To enable email logs and monitor your email delivery, follow the steps outlined in [Azure Communication Services email logs Communication Service in Azure Communication Service](../../concepts/analytics/logs/email-logs.md). ## Next steps +* [Get started with creating and managing Domain level Suppression List in Email Azure Communication Services](../../quickstarts/email/manage-suppression-list-management-sdks.md) + The following documents may be interesting to you: - Familiarize yourself with the [Email client library](../email/sdk-features.md) |
| communication-services | Migrate To Azure Communication Services | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/concepts/migrate-to-azure-communication-services.md | + + Title: Migrate to Azure Communication Services Calling SDK ++description: Migrate a calling product from Twilio Video to Azure Communication Services Calling SDK. ++++ Last updated : 04/04/2024++++++# Migrate to Azure Communication Services Calling SDK ++Migrate now to a market leading CPaaS platform with regular updates and long-term support. The [Azure Communication Services Calling SDK](../concepts/voice-video-calling/calling-sdk-features.md) provides features and functions that improve upon the sunsetting Twilio Programmable Video. ++Both products are cloud-based platforms that enable developers to add voice and video calling features to their web applications. When you migrate to Azure Communication Services, the calling SDK has key advantages that may affect your choice of platform and require minimal changes to your existing code. ++In this article, we describe the main features and functions of the Azure Communication Services, and link to a document comparing both platforms. We also provide links to instructions for migrating an existing Twilio Programmable Video implementation to Azure Communication Services Calling SDK. ++## What is Azure Communication Services? ++Azure Communication Services are cloud-based APIs and SDKs that you can use to seamlessly integrate communication tools into your applications. Improve your customersΓÇÖ communication experience using our multichannel communication APIs to add voice, video, chat, text messaging/SMS, email, and more. ++## Why migrate from Twilio Video to Azure Communication Services? ++Expect more from your communication services platform: ++- **Ease of migration** ΓÇô Use existing APIs and SDKs including a UI library to quickly migrate from Twilio Programmable Video to Microsoft's Calling SDK. ++- **Feature parity** ΓÇô The Calling SDK provides features and performance that meet or exceed Twilio Video. ++- **Multichannel communication** ΓÇô Choose from enterprise-level communication tools including voice, video, chat, SMS, and email. ++- **Maintenance and support** ΓÇô Microsoft delivers stability and long-term commitment with active support and regular software updates. ++## Azure Communication Services and Microsoft are your video platform of the future ++Azure Communication Services Calling SDK is just one part of the Azure ecosystem. You can bundle the Calling SDK with many other Azure services to speed enterprise adoption of your Communications Platform as a Service (CPaaS) solution. Key points of why Microsoft is optimal solution: ++- **Teams integration** ΓÇô Seamlessly integrate with Microsoft Teams to extend cloud-based meeting and messaging. ++- **Long-term guidance and support** ΓÇô Microsoft continues to provide application support, updates, and innovation. ++- **Artificial Intelligence (AI)** ΓÇô Microsoft invests heavily in AI research and its practical applications. We're actively applying AI to speed up technology adoption and ultimately improve the end user experience. ++- **Leverage the Microsoft ecosystem** ΓÇô Azure Communication Services, the Calling SDK, the Teams platform, AI research and development, the list goes on. Microsoft invests heavily in data centers, cloud computing, AI, and dozens of business applications. ++- **Developer-centric approach** ΓÇô Microsoft has a long history of investing in developer tools and technologies including GitHub, Visual Studio, Visual Studio Code, Copilot, support for an active developer community, and more. ++## Video conference feature comparison ++The Azure Communication Services Calling SDK has feature parity with TwilioΓÇÖs Video platform, with several additional features to further improve your communications platform. For a detailed feature map, see [Calling SDK overview > Detailed capabilities](./voice-video-calling/calling-sdk-features.md#detailed-capabilities). ++## Understand call types in Azure Communication Services ++Azure Communication Services offers various call types. The type of call you choose impacts your signaling schema, the flow of media traffic, and your pricing model. For more information, see [Voice and video concepts](../concepts/voice-video-calling/about-call-types.md). ++- **Voice Over IP (VoIP)** - When a user of your application calls another over an internet or data connection. Both signaling and media traffic are routed over the internet. +- **Public Switched Telephone Network (PSTN)** - When your users call a traditional telephone number, calls are facilitated via PSTN voice calling. To make and receive PSTN calls, you need to introduce telephony capabilities to your Azure Communication Services resource. Here, signaling and media employ a mix of IP-based and PSTN-based technologies to connect your users. +- **One-to-One Calls** - When one of your users connects with another through our SDKs. You can establish the call via either VoIP or PSTN. +- **Group Calls** - When three or more participants connect in a single call. Any combination of VoIP and PSTN-connected users can be on a group call. A one-to-one call can evolve into a group call by adding more participants to the call, and one of these participants can be a bot. +- **Rooms Call** - A Room acts as a container that manages activity between end-users of Azure Communication Services. It provides application developers with enhanced control over who can join a call, when they can meet, and how they collaborate. For a more comprehensive understanding of Rooms, see the [Rooms overview](../concepts/rooms/room-concept.md). +++## Key features available in Azure Communication Services Calling SDK ++- **Addressing** - Azure Communication Services provides [identities](../concepts/identity-model.md) for authenticating and addressing communication endpoints. These identities are used within Calling APIs, providing your customers with a clear view of who is connected to a call (the roster). +- **Encryption** - The Calling SDK safeguards traffic by encrypting it and preventing tampering along the way. +- **Device Management and Media enablement** - The SDK manages audio and video devices, efficiently encodes content for transmission, and supports both screen and application sharing. +- **PSTN calling** - You can use the SDK to initiate voice calling using the traditional Public Switched Telephone Network (PSTN), [using phone numbers acquired either in the Azure portal](../quickstarts/telephony/get-phone-number.md) or programmatically. +- **Teams Meetings** ΓÇô Your customers can use Azure Communication Services to [join Teams meetings](../quickstarts/voice-video-calling/get-started-teams-interop.md) and interact with Teams voice and video calls. +- **Notifications** - Azure Communication Services provides APIs to notify clients of incoming calls. Notifications enable your application to listen for events (such as incoming calls) even when your application isn't running in the foreground. +- **User Facing Diagnostics** - Azure Communication Services uses [events](../concepts/voice-video-calling/user-facing-diagnostics.md) to provide insights into underlying issues that might affect call quality. You can subscribe your application to triggers such as weak network signals or muted microphones for proactive issue awareness. +- **Media Quality Statistics** - Provides comprehensive insights into VoIP and video call [metrics](../concepts/voice-video-calling/media-quality-sdk.md). Metrics include call quality information, empowering developers to enhance communication experiences. +- **Video Constraints** - Azure Communication Services offers APIs that control [video quality among other parameters](../quickstarts/voice-video-calling/get-started-video-constraints.md) during video calls. The SDK supports different call situations for varied levels of video quality, so developers can adjust parameters like resolution and frame rate. ++## Next steps ++[Migrate from Twilio Video to Azure Communication Services.](../tutorials/migrating-to-azure-communication-services-calling.md) ++For a feature map, see [Calling SDK overview > Detailed capabilities](./voice-video-calling/calling-sdk-features.md#detailed-capabilities) |
| communication-services | Teams Interop Call Automation | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/how-tos/call-automation/teams-interop-call-automation.md | In this quickstart, we use the Azure Communication Services Call Automation APIs ## Prerequisites - An Azure account with an active subscription.-- A Microsoft Teams phone license and a Teams tenant with administrative privileges. Teams phone license is a must in order to use this feature, learn more about Teams licenses [here](https://www.microsoft.com/en-us/microsoft-teams/compare-microsoft-teams-bundle-options). Administrative privileges are required to authorize Communication Services resource to call Teams users, explained later in Step 1. +- A Microsoft Teams phone license and a Teams tenant with administrative privileges. Teams phone license is a must in order to use this feature, learn more about Teams licenses [here](https://www.microsoft.com/microsoft-teams/compare-microsoft-teams-bundle-options). Administrative privileges are required to authorize Communication Services resource to call Teams users, explained later in Step 1. - A deployed [Communication Service resource](../../quickstarts/create-communication-resource.md) and valid connection string found by selecting Keys in left side menu on Azure portal. - [Acquire a PSTN phone number from the Communication Service resource](../../quickstarts/telephony/get-phone-number.md). Note the phone number you acquired to use in this quickstart. - An Azure Event Grid subscription to receive the `IncomingCall` event. If you want to clean up and remove a Communication Services subscription, you ca - Learn more about [Call Automation](../../concepts/call-automation/call-automation.md) and its features. - Learn more about capabilities of [Teams Interoperability support with Azure Communication Services Call Automation](../../concepts/call-automation/call-automation-teams-interop.md) - Learn about [Play action](../../concepts/call-automation/play-Action.md) to play audio in a call.-- Learn how to build a [call workflow](../../quickstarts/call-automation/callflows-for-customer-interactions.md) for a customer support scenario. +- Learn how to build a [call workflow](../../quickstarts/call-automation/callflows-for-customer-interactions.md) for a customer support scenario. |
| communication-services | Audio Conferencing | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/how-tos/calling-sdk/audio-conferencing.md | -In this article, you learn how to use Azure Communication Services Calling SDK to retrieve Microsoft Teams Meeting audio conferencing details. This functionality allows users who are already connected to a Microsoft Teams Meeting to be able to get the conference ID and dial in phone number associated with the meeting. At present, Teams audio conferencing feature returns a conference ID and only one dial-in toll or toll-free phone number depending on the priority assigned. In the future, Teams audio conferencing feature will return a collection of all toll and toll-free numbers, giving users control on what Teams meeting dial-in details to use +In this article, you learn how to use Azure Communication Services Calling SDK to retrieve Microsoft Teams Meeting audio conferencing details. This functionality allows users who are already connected to a Microsoft Teams Meeting to be able to get the conference ID and dial in phone number associated with the meeting. Teams audio conferencing feature returns a collection of all toll and toll-free numbers, with concomitant country names and city names, giving users control on what Teams meeting dial-in details to use. ## Prerequisites - An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F). |
| communication-services | Manage Suppression List Management Sdks | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/quickstarts/email/manage-suppression-list-management-sdks.md | + + Title: Manage domain suppression lists in Azure Communication Services using the management client libraries ++description: Learn about managing domain suppression ists in Azure Communication Services using the management client libraries ++++ Last updated : 11/21/2023++++zone_pivot_groups: acs-js-csharp-java-python +++# Quickstart: Manage domain suppression lists in Azure Communication Services using the management client libraries ++This quick start covers the process for managing domain suppression lists in Azure Communication Services using the Azure Communication Services management client libraries. ++++ |
| communication-services | How To Collect Browser Verbose Log | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/how-to-collect-browser-verbose-log.md | + + Title: References - How to collect verbose log from browsers ++description: Learn how to collect verbose log from browsers. ++++ Last updated : 02/24/2024++++++# How to collect verbose log from browsers +When an issue originates within the underlying layer, collecting verbose logs in addition to web logs can provide valuable information. ++To collect the verbose log from the browser, initiate a web browser session with specific command line arguments. You open your video application within the browser and execute the scenario you're debugging. +Once the scenario is executed, you can close the browser. +During log collection, ensure to keep only the necessary tabs open in the browser. ++To collect the verbose log of the Edge browser, open a command line window and execute: ++`"C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe" --user-data-dir=C:\edge-debug --enable-logging --v=0 --vmodule=*/webrtc/*=2,*/libjingle/*=2,*media*=4 --no-sandbox` ++For Chrome, replace the executable path in the command with `C:\Program Files\Google\Chrome\Application\chrome.exe`. ++DonΓÇÖt omit the `--user-data-dir` argument. This argument is used to specify where the logs are saved. ++This command enables verbose logging and saves the log to chrome\_debug.log. +It's important to have only the necessary pages open in the Edge browser, such as `edge://webrtc-internals` and the application web page. +Keeping only necessary pages open ensure that logs from different web applications don't mix in the same log file. ++Log file is located at: `C:\edge-debug\chrome_debug.log` ++The verbose log is flushed each time the browser is opened with the specified command line. +Therefore, after closing the browser, you should copy the log and check its file size and modification time to confirm that it contains the verbose log. |
| communication-services | How To Collect Client Logs | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/how-to-collect-client-logs.md | + + Title: References - How to collect client logs ++description: Learn how to collect client logs. ++++ Last updated : 02/24/2024++++++# How to collect client logs +The client logs can help when we want to get more details while debugging an issue. +To collect client logs, you can use [@azure/logger](https://www.npmjs.com/package/@azure/logger), which is used by WebJS calling SDK internally. ++```typescript +import { setLogLevel, createClientLogger, AzureLogger } from '@azure/logger'; +setLogLevel('info'); +let logger = createClientLogger('ACS'); +const callClient = new CallClient({ logger }); +// app logging +logger.info('....'); ++``` ++[@azure/logger](https://www.npmjs.com/package/@azure/logger) supports four different log levels: ++* verbose +* info +* warning +* error ++For debugging purposes, `info` level logging is sufficient in most cases. ++In the browser environment, [@azure/logger](https://www.npmjs.com/package/@azure/logger) outputs logs to the console by default. +You can redirect logs by overriding `AzureLogger.log` method. For more information, see [@azure/logger](/javascript/api/overview/azure/logger-readme). ++Your app might keep logs in memory if it has a \'download log file\' feature. +If that is the case, you have to set a limit on the log size. +Not setting a limit might cause memory issues on long running calls. ++Additionally, if you send logs to a remote service, consider mechanisms such as compression and scheduling. +If the client has insufficient bandwidth, sending a large amount of log data in a short period of time can affect call quality. |
| communication-services | How To Collect Diagnostic Audio Recordings | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/how-to-collect-diagnostic-audio-recordings.md | + + Title: References - How to collect diagnostic audio recordings ++description: Learn how to collect diagnostic audio recordings. ++++ Last updated : 02/24/2024++++++# How to collect diagnostic audio recordings +To debug some issue, you may need audio recordings, especially when investigating audio quality problems, such as distorted audio and echo issues. ++To collect diagnostic audio recordings, open the chrome://webrtc-internals(Chrome) or edge://webrtc-internals(Edge) page. ++When you click *Enable diagnostic audio recordings*, the browser prompts a dialog asking for the download file location. +++After you finish an ACS call, you should be able to see files saved in the folder you choose. +++`*.output.N.wav` is the audio output sent to the speaker. ++`*.input.M.wav` is the audio input captured from the microphone. ++`*.aecdump` contains the necessary wav files for debugging audio after processed by the audio processing module in browsers. |
| communication-services | How To Collect Windows Audio Event Log | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/how-to-collect-windows-audio-event-log.md | + + Title: References - How to collect Windows audio event log ++description: Learn how to collect Windows audio event log. ++++ Last updated : 02/24/2024++++++# How to collect Windows audio event logs +The Windows audio event log provides information on the audio device state around the time when the issue we're investigating occurred. ++To collect the audio event log: +* open Windows Event Viewer +* browse the logs in *Application and Services Logs > Microsoft > Windows > Audio > Operational* +* you can either + * select logs within time range, right click and choose *Save Selected Events*. + * right click on Operational, and choose *Save All Events As*. + |
| communication-services | Camera Freeze | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/camera-freeze.md | + + Title: Understanding cameraFreeze UFD - User Facing Diagnostics ++description: Overview and details reference for understanding cameraFreeze UFD. ++++ Last updated : 03/27/2024++++++# cameraFreeze UFD +A `cameraFreeze` UFD event with a `true` value occurs when the SDK detects that the input framerate goes down to zero, causing the video output to appear frozen or not changing. ++The underlying issue may suggest problems with the user's video camera, or in certain instances, the device may cease sending video frames. +For example, on certain Android device models, you may see a `cameraFreeze` UFD event when the user locks the screen or puts the browser in the background. +In this situation, the Android operating system stops sending video frames, and thus on the other end of the call a user may see a `cameraFreeze` UFD event. ++| cameraFreeze | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example code to catch a cameraFreeze UFD event +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'cameraFreeze') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The cameraFreeze UFD recovered, notify the user + } + } +}); +``` ++## How to mitigate or resolve +Your calling application should subscribe to events from the User Facing Diagnostics. +You should also consider displaying a message on your user interface to alert users of potential camera issues. +The user can try to stop and start the video again, switch to other cameras or switch calling devices to resolve the issue. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Camera Permission Denied | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/camera-permission-denied.md | + + Title: Understanding cameraPermissionDenied UFD - User Facing Diagnostics ++description: Overview and details reference for understanding cameraPermissionDenied UFD. ++++ Last updated : 03/27/2024++++++# cameraPermissionDenied UFD +The `cameraPermissionDenied` UFD event with a `true` value occurs when the SDK detects that the camera permission was denied either at browser layer or at Operating System level. ++| cameraPermissionDenied | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example code to catch a cameraPermissionDenided UFD event +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'cameraPermissionDenied') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The cameraPermissionDenied UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +Your application should invoke `DeviceManager.askDevicePermission` before the call starts to check whether the permission was granted or not. +If the permission to use the camera is denied, the application should display a message on your user interface. +Additionally, your application should acquire camera browser permission before listing the available camera devices. +If there's no permission granted, the application is unable to get the detailed information of the camera devices on the user's system. ++The camera permission can also be revoked during a call, so your application should also subscribe to events from the User Facing Diagnostics events to display a message on the user interface. +Users can then take steps to resolve the issue on their own, such as enabling the browser permission or checking whether they disabled the camera access at OS level. ++> [!NOTE] +> Some browser platforms cache the permission results. ++If a user denied the permission at browser layer previously, invoking `askDevicePermission` API doesn't trigger the permission UI prompt, but it can know the permission was denied. +Your application should show instructions and ask the user to reset or grant the browser camera permission manually. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Camera Start Failed | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/camera-start-failed.md | + + Title: Understanding cameraStartFailed UFD - User Facing Diagnostics ++description: Overview and detailed reference of cameraStartFailed UFD ++++ Last updated : 03/27/2024++++++# cameraStartFailed UFD +The `cameraStartFailed` UFD event with a `true` value occurs when the SDK is unable to acquire the camera stream because the source is unavailable. +This error typically happens when the specified video device is being used by another process. +For example, the user may see this `cameraStartFailed` UFD event when they attempt to join a call with video on a browser such as Chrome while another Edge browser has been using the same camera. ++| cameraStartFailed | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'cameraStartFailed') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // cameraStartFailed UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +The `cameraStartFailed` UFD event is due to external reasons, so your application should subscribe to events from the User Facing Diagnostics and display a message on the UI to alert users of camera start failures. To resolve this issue, users can check if there are other processes using the same camera and close them if necessary. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Camera Start Timed Out | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/camera-start-timed-out.md | + + Title: Understanding cameraStartTimedOut UFD - User Facing Diagnostics ++description: Overview and detailed reference of cameraStartTimedOut UFD ++++ Last updated : 03/27/2024++++++# cameraStartTimedOut UFD +The `cameraStartTimedOut` UFD event with a `true` value occurs when the SDK is unable to acquire the camera stream because the promise returned by `getUserMedia` browser method doesn't resolve within a certain period of time. +This issue can happen when the user starts a call with video enabled, but the browser displays a UI permission prompt and the user doesn't respond to it. ++| cameraStartTimedOut | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'cameraStartTimedOut') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The cameraStartTimedOut UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +The application should invoke `DeviceManager.askDevicePermission` before the call starts to check whether the permission was granted or not. +Invoking `DeviceManager.askDevicePermission` also reduces the possibility that the user doesn't respond to the UI permission prompt after the call starts. ++If the timeout issue is caused by hardware problems, users can try selecting a different camera device when starting the video stream. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Camera Stopped Unexpectedly | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/camera-stopped-unexpectedly.md | + + Title: Understanding cameraStoppedUnexpectedly UFD - User Facing Diagnostics ++description: Overview and detailed reference of cameraStoppedUnexpectedly UFD. ++++ Last updated : 03/27/2024++++++# cameraStoppedUnexpectedly UFD +The `cameraStoppedUnexpectedly` UFD event with a `true` value occurs when the SDK detects that the camera track was muted. ++Keep in mind that this event relates to the camera track's `mute` event triggered by an external source. +The event can be triggered on mobile browsers when the browser goes to background. +Additionally, in some browser implementations, the browser sends black frames when the video input track is muted. ++| cameraStoppedUnexpectedly | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'cameraStoppedUnexpectedly') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The cameraStoppedUnexpectedly UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +Your application should subscribe to events from the User Facing Diagnostics and display a message on the user interface to alert users of any camera state changes. +This way ensures that users are aware of camera stopped issues and aren't surprised if other participants can't see the video. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Capturer Start Failed | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/capturer-start-failed.md | + + Title: Understanding capturerStartFailed UFD - User Facing Diagnostics ++description: Overview and detailed reference of capturerStartFailed UFD. ++++ Last updated : 03/27/2024++++++# capturerStartFailed UFD +The `capturerStartFailed` UFD event with a `true` value occurs when the SDK is unable to acquire the screen sharing stream because the source is unavailable. +This issue can happen when the underlying layer prevents the sharing of the selected source. ++| capturerStartFailed | Details | +| -|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'capturerStartFailed') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The capturerStartFailed UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +The `capturerStartFailed` is due to external reasons, so your application should subscribe to events from the User Facing Diagnostics and display a message on your user interface to alert users of screen sharing failures. +Users can then take steps to resolve the issue on their own, such as checking if there are other processes causing this issue. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Capturer Stopped Unexpectedly | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/capturer-stopped-unexpectedly.md | + + Title: Understanding capturerStoppedUnexpectedly UFD - User Facing Diagnostics ++description: Overview and detailed reference of capturerStoppedUnexpectedly UFD. ++++ Last updated : 03/26/2024++++++# capturerStoppedUnexpectedly UFD +The `capturerStoppedUnexpectedly` UFD event with a `true` value occurs when the SDK detects that the screen sharing track was muted. +This issue can happen due to external reasons and depends on the browser implementation. +For example, if the user shares a window and minimize that window, the `capturerStoppedUnexpectedly` UFD event may fire. ++| capturerStoppedUnexpectedly | Details | +| -|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'capturerStoppedUnexpectedly') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The capturerStoppedUnexpectedly UFD recovered, notify the user + } + } +}); +``` ++## How to mitigate or resolve +Your application should subscribe to events from the User Facing Diagnostics and display a message on your user interface to alert users of screen sharing issues. +Users can then take steps to resolve the issue on their own, such as checking whether they accidentally minimize the window being shared. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Microphone Mute Unexpectedly | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/microphone-mute-unexpectedly.md | + + Title: Understanding microphoneMuteUnexpectedly UFD - User Facing Diagnostics ++description: Overview and detailed reference of microphoneMuteUnexpectedly UFD ++++ Last updated : 03/27/2024++++++# microphoneMuteUnexpectedly UFD +The `microphoneMuteUnexpectedly` UFD event with a `true` value occurs when the SDK detects that the microphone track was muted. Keep in mind, that the event is related to the `mute` event of the microphone track, when it's triggered by an external source rather than by the SDK mute API. The underlying layer triggers the event, such as the audio stack muting the audio input session. The hardware mute button of some headset models can also trigger the `microphoneMuteUnexpectedly` UFD. Additionally, some browser platforms, such as iOS Safari browser, may mute the microphone when certain interruptions occur, such as an incoming phone call. ++| microphoneMuteUnexpectedly | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'microphoneMuteUnexpectedly') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The microphoneMuteUnexpectedly UFD recovered, notify the user + } + } +}); +``` ++## How to mitigate or resolve +Your application should subscribe to events from the User Facing Diagnostics and display an alert message to users of any microphone state changes. By doing so, users are aware of muted issues and aren't surprised if they found other participants can't hear their audio during a call. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Microphone Not Functioning | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/microphone-not-functioning.md | + + Title: Understanding microphoneNotFunctioning UFD - User Facing Diagnostics ++description: Overview and detailed reference of microphoneNotFunctioning UFD ++++ Last updated : 03/27/2024++++++# microphoneNotFunctioning UFD +The `microphoneNotFunctioning` UFD event with a `true` value occurs when the SDK detects that the microphone track was ended. The microphone track ending happens in many situations. +For example, unplugging a microphone in use triggers the browser to end the microphone track. The SDK would then fire `microphoneNotFunctioning` UFD event. +It can also occur when the user removes the microphone permission at browser or at OS level. The underlying layers, such as audio driver or media stack at OS level, may also end the session, causing the browser to end the microphone track. ++| microphoneNotFunctioning | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'microphoneNotFunctioning') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The microphoneNotFunctioning UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +The application should subscribe to events from the User Facing Diagnostics and display a message on the UI to alert users of any microphone issues. +Users can then take steps to resolve the issue on their own. +For example, they can unplug and plug in the headset device, or sometimes muting and unmuting the microphone can help as well. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Microphone Permission Denied | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/microphone-permission-denied.md | + + Title: Understanding microphonePermissionDenied UFD - User Facing Diagnostics ++description: Overview and detailed reference of microphonePermissionDenied UFD. ++++ Last updated : 03/27/2024++++++# microphonePermissionDenied UFD +The `microphonePermissionDenied` UFD event with a `true` value occurs when the SDK detects that the microphone permission was denied either at browser or OS level. ++| microphonePermissionDenied | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'microphonePermissionDenied') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The microphonePermissionDenied UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +Your application should invoke `DeviceManager.askDevicePermission` before a call starts to check whether the proper permissions were granted or not. +If the permission is denied, your application should display a message in the user interface to alert about this situation. +Additionally, your application should acquire browser permission before listing the available microphone devices. +If there's no permission granted, your application is unable to get the detailed information of the microphone devices on the user's system. ++The permission can also be revoked during the call. +Your application should also subscribe to events from the User Facing Diagnostics and display a message on the user interface to alert users of any permission issues. +Users can resolve the issue on their own, by enabling the browser permission or checking whether they disabled the microphone access at OS level. ++> [!NOTE] +> Some browser platforms cache the permission results. ++If a user denied the permission at browser layer previously, invoking `askDevicePermission` API doesn't trigger the permission UI prompt, but the method can know the permission was denied. +Your application should show instructions and ask the user to reset or grant the browser microphone permission manually. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Network Receive Quality | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/network-receive-quality.md | + + Title: Understanding networkReceiveQuality UFD - User Facing Diagnostics ++description: Overview and detiled reference of networkReceiveQuality UFD ++++ Last updated : 03/27/2024++++++# networkReceiveQuality UFD +The `networkReceiveQuality` UFD event with a `Bad` value indicates the presence of network quality issues for incoming streams, as detected by the ACS Calling SDK. +This event suggests that there may be problems with the network connection between the local endpoint and remote endpoint. +When this UFD event fires with a`Bad` value, the user may experience degraded audio quality. ++| networkReceiveQualityUFD | Details | +| -|| +| UFD type | NetworkDiagnostics | +| value type | DiagnosticQuality | +| possible values | Good, Poor, Bad | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).network.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'networkReceiveQuality') { + if (diagnosticInfo.value === DiagnosticQuality.Bad) { + // network receive quality bad, show a warning message on UI + } else if (diagnosticInfo.value === DiagnosticQuality.Poor) { + // network receive quality poor, notify the user + } else if (diagnosticInfo.value === DiagnosticQuality.Good) { + // network receive quality recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +From the perspective of the ACS Calling SDK, network issues are considered external problems. +To solve network issues, you need to understand the network topology and identify the nodes that are causing the problem. +These parts involve network infrastructure, which is outside the scope of the ACS Calling SDK. ++Your application should subscribe to events from the User Facing Diagnostics. +Display a message on your user interface that informs users of network quality issues and potential audio quality degradation. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Network Reconnect | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/network-reconnect.md | + + Title: Understanding networkReconnect UFD - User Facing Diagnostics ++description: Overview and detailed reference of networkReconnect UFD ++++ Last updated : 03/27/2024++++++# networkReconnect UFD +The `networkReconnect` UFD event with a `Bad` value occurs when the Interactive Connectivity Establishment (ICE) transport state on the connection is `failed`. +This event indicates that there may be network issues between the two endpoints, such as packet loss or firewall issues. +The connection failure is detected by the ICE consent freshness mechanism implemented in the browser. ++When an endpoint doesn't receive a reply after a certain period, the ICE transport state will transition to `disconnected`. +If there's still no response received, the state then becomes `failed`. ++Since the endpoint didn't receive a reply for a period of time, it's possible that incoming packets weren't received or outgoing packets didn't reach to the other users. +This situation may result in the user not hearing or seeing the other party. ++| networkReconnect UFD | Details | +| || +| UFD type | NetworkDiagnostics | +| value type | DiagnosticQuality | +| possible values | Good, Bad | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).network.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'networkReconnect') { + if (diagnosticInfo.value === DiagnosticQuality.Bad) { + // media transport disconnected, show a warning message on UI + } else if (diagnosticInfo.value === DiagnosticQuality.Good) { + // media transport recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +From the perspective of the ACS Calling SDK, network issues are considered external problems. +To solve network issues, you need to understand the network topology and identify the nodes that are causing the problem. +These parts involve network infrastructure, which is outside the scope of the ACS Calling SDK. ++Internally, the ACS Calling SDK will trigger reconnection after a `networkReconnect` UFD event with a `Bad` value is fired. If the connection recovers, `networkReconnect` UFD event with a `Good` value is fired. ++Your application should subscribe to events from the User Facing Diagnostics. +Display a message on your user interface that informs users of network connection issues and potential audio loss. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Network Relays Not Reachable | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/network-relays-not-reachable.md | + + Title: Understanding networkRelaysNotReachable UFD - User Facing Diagnostics ++description: Overview and detailed reference of networkRelaysNotReachable UFD ++++ Last updated : 03/27/2024++++++# networkRelaysNotReachable UFD +The `networkRelaysNotReachable` UFD event with a `true` value occurs when the media connection fails to establish and no relay candidates are available. This issue usually happens when the firewall policy blocks connections between the local client and relay servers. ++When users see the `networkRelaysNotReachable` UFD event, it also indicates that the local client isn't able to make a direct connection to the remote endpoint. ++| networkRelaysNotReachable UFD | Details | +| || +| UFD type | NetworkDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).network.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'networkRelaysNotReachable') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The networkRelaysNotReachable UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +Your application should subscribe to events from the User Facing Diagnostics. +Display a message on your user interface and inform users of network setup issues. ++Users should follow the *Firewall Configuration* guideline mentioned in the [Network recommendations](../../../../../concepts/voice-video-calling/network-requirements.md) document. It's also recommended that the user also checks their Network address translation (NAT) settings or whether their firewall policy blocks User Datagram Protocol (UDP) packets. ++If the organization policy doesn't allow users to connect to Microsoft TURN relay servers, custom TURN servers can be configured to avoid connection failures. For more information, see [Force calling traffic to be proxied across your own server](../../../../../tutorials/proxy-calling-support-tutorial.md) tutorial. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Network Send Quality | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/network-send-quality.md | + + Title: Understanding networkSendQuality UFD - User Facing Diagnostics ++description: Overview and detailed reference of networkSendQuality UFD ++++ Last updated : 03/27/2024++++++# networkSendQuality UFD +The `networkSendQuality` UFD event with a `Bad` value indicates that there are network quality issues for outgoing streams, such as packet loss, as detected by the ACS Calling SDK. +This event suggests that there may be problems with the network quality issues between the local endpoint and remote endpoint. +++| networkSendQualityUFD | Details | +| -|| +| UFD type | NetworkDiagnostics | +| value type | DiagnosticQuality | +| possible values | Good, Bad | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).network.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'networkSendQuality') { + if (diagnosticInfo.value === DiagnosticQuality.Bad) { + // network send quality bad, show a warning message on UI + } else if (diagnosticInfo.value === DiagnosticQuality.Good) { + // network send quality recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +From the perspective of the ACS Calling SDK, network issues are considered external problems. +To solve network issues, it's typically necessary to have an understanding of the network topology and the nodes that are causing the problem. +These parts involve network infrastructure, which is outside the scope of the ACS Calling SDK. ++Your application should subscribe to events from the User Facing Diagnostics and display a message on the user interface, so that users are aware of network quality issues. While these issues are often temporary and recover soon, frequent occurrences of the `networkSendQuality` UFD event for a particular user may require further investigation. +For example, users should check their network equipment or check with their internet service provider (ISP). ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | No Microphone Devices Enumerated | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/no-microphone-devices-enumerated.md | + + Title: Understanding noMicrophoneDevicesEnumerated UFD - User Facing Diagnostics ++description: Overview and detailed reference of noMicrophoneDevicesEnumerated UFD ++++ Last updated : 03/27/2024++++++# noMicrophoneDevicesEnumerated UFD +The `noMicrophoneDevicesEnumerated` UFD event with a `true` value occurs when the browser API `navigator.mediaDevices.enumerateDevices` doesn't include any audio input devices. +This means that there are no microphones available on the user's machine. This issue is caused by the user unplugging or disabling the microphone. ++> [!NOTE] +> This UFD event is unrelated to the a user allowing microphone permission. ++Even if a user doesn't grant the microphone permission at the browser level, the `DeviceManager.getMicrophones` API still returns a microphone device info with an empty name, which indicates the presence of a microphone device on the user's machine. ++| noMicrophoneDevicesEnumeratedUFD | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'noMicrophoneDevicesEnumerated') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The noSpeakerDevicesEnumerated UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +Your application should subscribe to events from the User Facing Diagnostics and display a message on the user interface to alert users of any device setup issues. Users can then take steps to resolve the issue on their own, such as plugging in a headset or checking whether they disabled the microphone devices. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | No Network | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/no-network.md | + + Title: Understanding noNetwork UFD - User Facing Diagnostics ++description: Overview and detailed reference of noNetwork UFD ++++ Last updated : 03/27/2024++++++# noNetwork UFD +The `noNetwork` UFD event with a `true` value occurs when there's no network available for ICE candidates being gathered, which means there are network setup issues in the local environment, such as a disconnected Wi-Fi or Ethernet cable. +Additionally, if the adapter fails to acquire an IP address and there are no other networks available, this situation can also result in `noNetwork` UFD event. ++| noNetwork UFD | Details | +| || +| UFD type | NetworkDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).network.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'noNetwork') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // noNetwork UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +Your application should subscribe to events from the User Facing Diagnostics and display a message in your user interface to alert users of any network setup issues. +Users can then take steps to resolve the issue on their own. ++Users should also check if they disabled the network adapters or whether they have an available network. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | No Speaker Devices Enumerated | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/no-speaker-devices-enumerated.md | + + Title: Understanding noSpeakerDevicesEnumerated UFD - User Facing Diagnostics ++description: Overview and detailed reference of noSpeakerDevicesEnumerated UFD ++++ Last updated : 03/27/2024++++++# noSpeakerDevicesEnumerated UFD +The `noSpeakerDevicesEnumerated` UFD event with a `true` value occurs when there's no speaker device presented in the device list returned by the browser API. This issue occurs when the `navigator.mediaDevices.enumerateDevices` browser API doesn't include any audio output devices. This event indicates that there are no speakers available on the user's machine, which could be because the user unplugged or disabled the speaker. ++On some platforms such as iOS, the browser doesn't provide the audio output devices in the device list. In this case, the SDK considers it as expected behavior and doesn't fire `noSpeakerDevicesEnumerated` UFD event. ++| noSpeakerDevicesEnumerated UFD | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'noSpeakerDevicesEnumerated') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The noSpeakerDevicesEnumerated UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +Your application should subscribe to events from the User Facing Diagnostics and display a message on your user interface to alert users of any device setup issues. +Users can then take steps to resolve the issue on their own, such as plugging in a headset or checking whether they disabled the speaker devices. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Screenshare Recording Disabled | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/screenshare-recording-disabled.md | + + Title: Understanding screenshareRecordingDisabled UFD - User Facing Diagnostics ++description: Overview and detailed reference of screenshareRecordingDisabled UFD. ++++ Last updated : 03/27/2024++++++# screenshareRecordingDisabled UFD +The `screenshareRecordingDisabled` UFD event with a `true` value occurs when the SDK detects that the screen sharing permission was denied in the browser or OS settings on macOS. ++| screenshareRecordingDisabled | Details | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'screenshareRecordingDisabled') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The screenshareRecordingDisabled UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +Your application should subscribe to events from the User Facing Diagnostics and display a message on the user interface to alert users of any screen sharing permission issues. +Users can then take steps to resolve the issue on their own. ++Users should also check if they disabled the screen sharing permission from OS settings. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Speaking While Microphone Is Muted | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/resources/troubleshooting/voice-video-calling/references/ufd/speaking-while-microphone-is-muted.md | + + Title: Understanding speakingWhileMicrophoneIsMuted UFD - User Facing Diagnostics ++description: Overview and detailed reference of speakingWhileMicrophoneIsMuted UFD ++++ Last updated : 03/27/2024++++++# speakingWhileMicrophoneIsMuted UFD +The `speakingWhileMicrophoneIsMuted` UFD event with a `true` value occurs when the SDK detects that the audio input volume isn't muted although the user did mute the microphone. +This event can remind the user who may want to speak something but forgot to unmute their microphone. +In this case, since the microphone state in the SDK is muted, no audio is sent. ++| speakingWhileMicrophoneIsMuted | Detail | +| --|| +| UFD type | MediaDiagnostics | +| value type | DiagnosticFlag | +| possible values | true, false | ++## Example +```typescript +call.feature(Features.UserFacingDiagnostics).media.on('diagnosticChanged', (diagnosticInfo) => { + if (diagnosticInfo.diagnostic === 'speakingWhileMicrophoneIsMuted') { + if (diagnosticInfo.value === true) { + // show a warning message on UI + } else { + // The speakingWhileMicrophoneIsMuted UFD recovered, notify the user + } + } +}); +``` +## How to mitigate or resolve +The `speakingWhileMicrophoneIsMuted` UFD event isn't an error, but rather an indication of an inconsistency between the audio input volume and the microphone's muted state in the SDK. +The purpose of this event is for the application to show a message on your user interface as a hint, so the user can know that the microphone is muted while they're speaking. ++## Next steps +* Learn more about [User Facing Diagnostics feature](../../../../../concepts/voice-video-calling/user-facing-diagnostics.md?pivots=platform-web). |
| communication-services | Chat Hero Sample | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/samples/chat-hero-sample.md | Complete the following prerequisites and steps to set up the sample. `git clone https://github.com/Azure-Samples/communication-services-web-chat-hero.git` - Or clone the repo using any method described in [Clone an existing Git repo](https://learn.microsoft.com/azure/devops/repos/git/clone). + Or clone the repo using any method described in [Clone an existing Git repo](/azure/devops/repos/git/clone). 3. Get the `Connection String` and `Endpoint URL` from the Azure portal or by using the Azure CLI. |
| communication-services | Migrating To Azure Communication Services Calling | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/communication-services/tutorials/migrating-to-azure-communication-services-calling.md | Title: Tutorial - Migrating from Twilio video to ACS + Title: Tutorial - Migrate from Twilio Video to Azure Communication Services -description: Learn how to migrate a calling product from Twilio to Azure Communication Services. +description: Learn how to migrate a calling product from Twilio Video to Azure Communication Services. -# Migrating from Twilio Video to Azure Communication Services +# Migrate from Twilio Video to Azure Communication Services This article describes how to migrate an existing Twilio Video implementation to the [Azure Communication Services Calling SDK](../concepts/voice-video-calling/calling-sdk-features.md). Both Twilio Video and Azure Communication Services Calling SDK are cloud-based platforms that enable developers to add voice and video calling features to their web applications. Azure Communication Services offers various call types. The type of call you cho ::: zone pivot="platform-android" [!INCLUDE [Migrating to ACS on Android SDK](./includes/twilio-to-acs-video-android-tutorial.md)] |
| container-apps | Jobs Get Started Cli | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/container-apps/jobs-get-started-cli.md | To use manual jobs, you first create a job with trigger type `Manual` and then s az containerapp job create \ --name "$JOB_NAME" --resource-group "$RESOURCE_GROUP" --environment "$ENVIRONMENT" \ --trigger-type "Manual" \- --replica-timeout 1800 --replica-retry-limit 1 --replica-completion-count 1 --parallelism 1 \ + --replica-timeout 1800 \ --image "mcr.microsoft.com/k8se/quickstart-jobs:latest" \ --cpu "0.25" --memory "0.5Gi" ``` Create a job in the Container Apps environment that starts every minute using th az containerapp job create \ --name "$JOB_NAME" --resource-group "$RESOURCE_GROUP" --environment "$ENVIRONMENT" \ --trigger-type "Schedule" \- --replica-timeout 1800 --replica-retry-limit 1 --replica-completion-count 1 --parallelism 1 \ + --replica-timeout 1800 \ --image "mcr.microsoft.com/k8se/quickstart-jobs:latest" \ --cpu "0.25" --memory "0.5Gi" \ --cron-expression "*/1 * * * *" |
| container-apps | Jobs | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/container-apps/jobs.md | The following table compares common scenarios for apps and jobs: | An HTTP server that serves web content and API requests | App | Configure an [HTTP scale rule](scale-app.md#http). | | A process that generates financial reports nightly | Job | Use the [*Schedule* job type](#scheduled-jobs) and configure a cron expression. | | A continuously running service that processes messages from an Azure Service Bus queue | App | Configure a [custom scale rule](scale-app.md#custom). |-| A job that processes a single message or a small batch of messages from an Azure queue and exits | Job | Use the *Event* job type and [configure a custom scale rule](tutorial-event-driven-jobs.md) to trigger job executions. | +| A job that processes a single message or a small batch of messages from an Azure queue and exits | Job | Use the *Event* job type and [configure a custom scale rule](tutorial-event-driven-jobs.md) to trigger job executions when there are messages in the queue. | | A background task that's triggered on-demand and exits when finished | Job | Use the *Manual* job type and [start executions](#start-a-job-execution-on-demand) manually or programmatically using an API. | | A self-hosted GitHub Actions runner or Azure Pipelines agent | Job | Use the *Event* job type and configure a [GitHub Actions](tutorial-ci-cd-runners-jobs.md?pivots=container-apps-jobs-self-hosted-ci-cd-github-actions) or [Azure Pipelines](tutorial-ci-cd-runners-jobs.md?pivots=container-apps-jobs-self-hosted-ci-cd-azure-pipelines) scale rule. | | An Azure Functions app | App | [Deploy Azure Functions to Container Apps](../azure-functions/functions-container-apps-hosting.md). | | An event-driven app using the Azure WebJobs SDK | App | [Configure a scale rule](scale-app.md#custom) for each event source. | +## Concepts ++A Container Apps environment is a secure boundary around one or more container apps and jobs. Jobs involve a few key concepts: ++* **Job:** A job defines the default configuration that is used for each job execution. The configuration includes the container image to use, the resources to allocate, and the command to run. +* **Job execution:** A job execution is a single run of a job that is triggered manually, on a schedule, or in response to an event. +* **Job replica:** A typical job execution runs one replica defined by the job's configuration. In advanced scenarios, a job execution can run multiple replicas. ++ ## Job trigger types A job's trigger type determines how the job is started. The following trigger types are available: A job's trigger type determines how the job is started. The following trigger ty ### Manual jobs -Manual jobs are triggered on-demand using the Azure CLI or a request to the Azure Resource Manager API. +Manual jobs are triggered on-demand using the Azure CLI, Azure portal, or a request to the Azure Resource Manager API. Examples of manual jobs include: To create a manual job using the Azure CLI, use the `az containerapp job create` az containerapp job create \ --name "my-job" --resource-group "my-resource-group" --environment "my-environment" \ --trigger-type "Manual" \- --replica-timeout 1800 --replica-retry-limit 0 --replica-completion-count 1 --parallelism 1 \ + --replica-timeout 1800 \ --image "mcr.microsoft.com/k8se/quickstart-jobs:latest" \ --cpu "0.25" --memory "0.5Gi" ``` Container Apps jobs use cron expressions to define schedules. It supports the st | `0 0 * * 0` | Runs every Sunday at midnight. | | `0 0 1 * *` | Runs on the first day of every month at midnight. | -Cron expressions in scheduled jobs are evaluated in Universal Time Coordinated (UTC). +Cron expressions in scheduled jobs are evaluated in Coordinated Universal Time (UTC). # [Azure CLI](#tab/azure-cli) To create a scheduled job using the Azure CLI, use the `az containerapp job crea az containerapp job create \ --name "my-job" --resource-group "my-resource-group" --environment "my-environment" \ --trigger-type "Schedule" \- --replica-timeout 1800 --replica-retry-limit 0 --replica-completion-count 1 --parallelism 1 \ + --replica-timeout 1800 \ --image "mcr.microsoft.com/k8se/quickstart-jobs:latest" \ --cpu "0.25" --memory "0.5Gi" \ --cron-expression "*/1 * * * *" Event-driven jobs are triggered by events from supported [custom scalers](scale- Container apps and event-driven jobs use [KEDA](https://keda.sh/) scalers. They both evaluate scaling rules on a polling interval to measure the volume of events for an event source, but the way they use the results is different. -In an app, each replica continuously processes events and a scaling rule determines the number of replicas to run to meet demand. In event-driven jobs, each job typically processes a single event, and a scaling rule determines the number of jobs to run. +In an app, each replica continuously processes events and a scaling rule determines the number of replicas to run to meet demand. In event-driven jobs, each job execution typically processes a single event, and a scaling rule determines the number of job executions to run. Use jobs when each event requires a new instance of the container with dedicated resources or needs to run for a long time. Event-driven jobs are conceptually similar to [KEDA scaling jobs](https://keda.sh/docs/latest/concepts/scaling-jobs/). To create an event-driven job using the Azure CLI, use the `az containerapp job az containerapp job create \ --name "my-job" --resource-group "my-resource-group" --environment "my-environment" \ --trigger-type "Event" \- --replica-timeout 1800 --replica-retry-limit 0 --replica-completion-count 1 --parallelism 1 \ + --replica-timeout 1800 \ --image "docker.io/myuser/my-event-driven-job:latest" \ --cpu "0.25" --memory "0.5Gi" \ --min-executions "0" \ To start a job execution using the Azure Resource Manager REST API, make a `POST The following example starts an execution of a job named `my-job` in a resource group named `my-resource-group`: ```http-POST https://management.azure.com/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/my-resource-group/providers/Microsoft.App/jobs/my-job/start?api-version=2022-11-01-preview +POST https://management.azure.com/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/my-resource-group/providers/Microsoft.App/jobs/my-job/start?api-version=2023-05-01 Authorization: Bearer <TOKEN> ``` Replace `<SUBSCRIPTION_ID>` with your subscription ID. -To authenticate the request, replace `<TOKEN>` in the `Authorization` header with a valid bearer token. For more information, see [Azure REST API reference](/rest/api/azure). +To authenticate the request, replace `<TOKEN>` in the `Authorization` header with a valid bearer token. The identity used to generate the token must have `Contributor` permission to the Container Apps job resource. For more information, see [Azure REST API reference](/rest/api/azure). # [Azure portal](#tab/azure-portal) To start a job execution in the Azure portal, select **Run now** in the job's ov When you start a job execution, you can choose to override the job's configuration. For example, you can override an environment variable or the startup command to run the same job with different inputs. The overridden configuration is only used for the current execution and doesn't change the job's configuration. +> [!IMPORTANT] +> When overriding the configuration, the job's entire template configuration is replaced with the new configuration. Ensure that the new configuration includes all required settings. + # [Azure CLI](#tab/azure-cli) To override the job's configuration while starting an execution, use the `az containerapp job start` command and pass a YAML file containing the template to use for the execution. The following example starts an execution of a job named `my-job` in a resource group named `my-resource-group`. Retrieve the job's current configuration with the `az containerapp job show` com az containerapp job show --name "my-job" --resource-group "my-resource-group" --query "properties.template" --output yaml > my-job-template.yaml ``` +The `--query "properties.template"` option returns only the job's template configuration. + Edit the `my-job-template.yaml` file to override the job's configuration. For example, to override the environment variables, modify the `env` section: ```yaml az containerapp job start --name "my-job" --resource-group "my-resource-group" \ To override the job's configuration, include a template in the request body. The following example overrides the startup command to run a different command: ```http-POST https://management.azure.com/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/my-resource-group/providers/Microsoft.App/jobs/my-job/start?api-version=2022-11-01-preview +POST https://management.azure.com/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/my-resource-group/providers/Microsoft.App/jobs/my-job/start?api-version=2023-05-01 Content-Type: application/json Authorization: Bearer <TOKEN> Authorization: Bearer <TOKEN> } ``` -Replace `<SUBSCRIPTION_ID>` with your subscription ID and `<TOKEN>` in the `Authorization` header with a valid bearer token. For more information, see [Azure REST API reference](/rest/api/azure). +Replace `<SUBSCRIPTION_ID>` with your subscription ID and `<TOKEN>` in the `Authorization` header with a valid bearer token. The identity used to generate the token must have `Contributor` permission to the Container Apps job resource. For more information, see [Azure REST API reference](/rest/api/azure). # [Azure portal](#tab/azure-portal) az containerapp job execution list --name "my-job" --resource-group "my-resource To get the status of job executions using the Azure Resource Manager REST API, make a `GET` request to the job's `executions` operation. The following example returns the status of the most recent execution of a job named `my-job` in a resource group named `my-resource-group`: ```http-GET https://management.azure.com/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/my-resource-group/providers/Microsoft.App/jobs/my-job/executions?api-version=2022-11-01-preview +GET https://management.azure.com/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/my-resource-group/providers/Microsoft.App/jobs/my-job/executions?api-version=2023-05-01 ``` Replace `<SUBSCRIPTION_ID>` with your subscription ID. Container Apps jobs support advanced configuration options such as container set ### Container settings -Container settings define the containers to run in each replica of a job execution. They include environment variables, secrets, and resource limits. For more information, see [Containers](containers.md). +Container settings define the containers to run in each replica of a job execution. They include environment variables, secrets, and resource limits. For more information, see [Containers](containers.md). Running multiple containers in a single job is an advanced scenario. Most jobs run a single container. ### Job settings The following table includes the job settings that you can configure: | Setting | Azure Resource Manager property | CLI parameter| Description | ||||| | Job type | `triggerType` | `--trigger-type` | The type of job. (`Manual`, `Schedule`, or `Event`) |-| Parallelism | `parallelism` | `--parallelism` | The number of replicas to run per execution. For most jobs, set the value to `1`. | -| Replica completion count | `replicaCompletionCount` | `--replica-completion-count` | The number of replicas to complete successfully for the execution to succeed. For most jobs, set the value to `1`. | | Replica timeout | `replicaTimeout` | `--replica-timeout` | The maximum time in seconds to wait for a replica to complete. |+| Polling interval | `pollingInterval` | `--polling-interval` | The time in seconds to wait between polling for events. Default is 30 seconds. | | Replica retry limit | `replicaRetryLimit` | `--replica-retry-limit` | The maximum number of times to retry a failed replica. To fail a replica without retrying, set the value to `0`. |+| Parallelism | `parallelism` | `--parallelism` | The number of replicas to run per execution. For most jobs, set the value to `1`. | +| Replica completion count | `replicaCompletionCount` | `--replica-completion-count` | The number of replicas to complete successfully for the execution to succeed. Most be equal or less than the parallelism. For most jobs, set the value to `1`. | ### Example |
| container-apps | Quotas | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/container-apps/quotas.md | -To request an increase in quota amounts for your container app, learn [how to request a limit increase](faq.yml#how-can-i-request-a-quota-increase-) and [submit a support ticket](https://azure.microsoft.com/support/create-ticket/). +To request a [quota increase](faq.yml#how-can-i-request-a-quota-increase-), you can [submit a support ticket](https://azure.microsoft.com/support/create-ticket/). The *Is Configurable* column in the following tables denotes a feature maximum may be increased through a [support request](https://azure.microsoft.com/support/create-ticket/). For more information, see [how to request a limit increase](faq.yml#how-can-i-request-a-quota-increase-). -| Feature | Scope | Default | Is Configurable | Remarks | +| Feature | Scope | Default Quota | Is Configurable | Remarks | |--|--|--|--|--|-| Environments | Region | Up to 15 | Yes | Limit up to 15 environments per subscription, per region. | -| Environments | Global | Up to 20 | Yes | Limit up to 20 environments per subscription across all regions | +| Environments | Region | Up to 15 | Yes | Up to 15 environments per subscription, per region. | +| Environments | Global | Up to 20 | Yes | Up to 20 environments per subscription, across all regions. | | Container Apps | Environment | Unlimited | n/a | |-| Revisions | Container app | 100 | No | | -| Replicas | Revision | 300 | Yes | | +| Revisions | Container app | Up to 100 | No | | +| Replicas | Revision | Unlimited | No | Maximum replicas configurable are 300 in Azure portal and 1000 in Azure CLI. There must also be enough cores quota available. | ## Consumption plan |
| container-apps | Scale App | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/container-apps/scale-app.md | Scaling is defined by the combination of limits, rules, and behavior. | Scale limit | Default value | Min value | Max value | |||||- | Minimum number of replicas per revision | 0 | 0 | 300 | - | Maximum number of replicas per revision | 10 | 1 | 300 | + | Minimum number of replicas per revision | 0 | 0 | Maximum replicas configurable are 300 in Azure portal and 1,000 in Azure CLI. | + | Maximum number of replicas per revision | 10 | 1 | Maximum replicas configurable are 300 in Azure portal and 1,000 in Azure CLI. | - To request an increase in maximum replica amounts for your container app, [submit a support ticket](https://azure.microsoft.com/support/create-ticket/). + For more information see [Quotas for Azure Container Apps](quotas.md). - **Rules** are the criteria used by Container Apps to decide when to add or remove replicas. |
| container-apps | Spring Cloud Config Server Usage | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/container-apps/spring-cloud-config-server-usage.md | You can use client side decryption of properties by following the steps: ## Next steps > [!div class="nextstepaction"]-> [Set up a Spring Cloud Config Server](spring-cloud-config-server.md) +> [Tutorial: Connect to a managed Spring Cloud Config Server](spring-cloud-config-server.md) |
| container-apps | Spring Cloud Eureka Server Usage | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/container-apps/spring-cloud-eureka-server-usage.md | Now you have a caller and callee application that communicate with each other us ## Next steps > [!div class="nextstepaction"]-> [Use Spring Cloud Eureka Server](spring-cloud-eureka-server.md) +> [Tutorial: Connect to a managed Spring Cloud Eureka Server](spring-cloud-eureka-server.md) |
| container-apps | Tutorial Event Driven Jobs | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/container-apps/tutorial-event-driven-jobs.md | In this tutorial, you learn how to work with [event-driven jobs](jobs.md#event-d > * Deploy the job to the Container Apps environment > * Verify that the queue messages are processed by the container app -The job you create starts an execution for each message that is sent to an Azure Storage Queue. Each job execution runs a container that performs the following steps: +The job you create starts an execution for each message that is sent to an Azure Storage queue. Each job execution runs a container that performs the following steps: -1. Dequeues one message from the queue. +1. Gets one message from the queue. 1. Logs the message to the job execution logs. 1. Deletes the message from the queue. 1. Exits. +> [!IMPORTANT] +> The scaler monitors the queue's length to determine how many jobs to start. For accurate scaling, don't delete a message from the queue until the job execution has finished processing it. + The source code for the job you run in this tutorial is available in an Azure Samples [GitHub repository](https://github.com/Azure-Samples/container-apps-event-driven-jobs-tutorial/blob/main/index.js). [!INCLUDE [container-apps-create-cli-steps-jobs.md](../../includes/container-apps-create-cli-steps-jobs.md)] To deploy the job, you must first build a container image for the job and push i --environment "$ENVIRONMENT" \ --trigger-type "Event" \ --replica-timeout "1800" \- --replica-retry-limit "1" \ - --replica-completion-count "1" \ - --parallelism "1" \ --min-executions "0" \ --max-executions "10" \ --polling-interval "60" \ To deploy the job, you must first build a container image for the job and push i | Parameter | Description | | | | | `--replica-timeout` | The maximum duration a replica can execute. |- | `--replica-retry-limit` | The number of times to retry a replica. | - | `--replica-completion-count` | The number of replicas to complete successfully before a job execution is considered successful. | - | `--parallelism` | The number of replicas to start per job execution. | | `--min-executions` | The minimum number of job executions to run per polling interval. | | `--max-executions` | The maximum number of job executions to run per polling interval. | | `--polling-interval` | The polling interval at which to evaluate the scale rule. | |
| copilot | Capabilities | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/copilot/capabilities.md | While Microsoft Copilot for Azure (preview) can perform many types of tasks, it' Keep in mind these current limitations: -- The number of chats per day that a user can have, and the number of requests per chat, are limited. When you open Microsoft Copilot for Azure (preview), you'll see details about these limitations.+- Any action taken on more than 10 resources must be performed outside of Microsoft Copilot for Azure. ++- You can only make 15 requests during any given chat, and you only have 10 chats in a 24 hour period. + - Some responses that display lists will be limited to the top five items. - For some tasks and queries, using a resource's name will not work, and the Azure resource ID must be provided. - Microsoft Copilot for Azure (preview) is currently available in English only. |
| cosmos-db | Cmk Troubleshooting Guide | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cosmos-db/cmk-troubleshooting-guide.md | You see this error when the Azure Key Vault or specified Key are not found. Check if the Azure Key Vault or the specified key exist and restore them if accidentally got deleted, then wait for one hour. If the issue isn't resolved after more than 2 hours, contact customer service. +## Azure key Disabled or expired ++### Reason for error ++You see this error when the Azure Key Vault key has been expired or deleted. ++### Troubleshooting ++If your key has been disabled please enable it. If it has been expired please un-expire it, and once the account is not revoked anymore feel free to rotate the key as Azure Cosmos DB will update the key version once the account is online. + ## Invalid Azure Cosmos DB default identity ### Reason for error |
| cosmos-db | Free Tier | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cosmos-db/free-tier.md | Last updated 07/08/2022 Azure Cosmos DB free tier makes it easy to get started, develop, test your applications, or even run small production workloads for free. When free tier is enabled on an account, you'll get the first 1000 RU/s and 25 GB of storage in the account for free. The throughput and storage consumed beyond these limits are billed at regular price. Free tier is available for all API accounts with provisioned throughput, autoscale throughput, single, or multiple write regions. -Free tier lasts indefinitely for the lifetime of the account and it comes with all the [benefits and features](introduction.md#an-ai-database-with-unmatched-reliability-and-flexibility) of a regular Azure Cosmos DB account. These benefits include unlimited storage and throughput (RU/s), SLAs, high availability, turnkey global distribution in all Azure regions, and more. +Free tier lasts indefinitely for the lifetime of the account and it comes with all the [benefits and features](introduction.md#with-unmatched-reliability-and-flexibility) of a regular Azure Cosmos DB account. These benefits include unlimited storage and throughput (RU/s), SLAs, high availability, turnkey global distribution in all Azure regions, and more. You can have up to one free tier Azure Cosmos DB account per an Azure subscription and you must opt in when creating the account. If you don't see the option to apply the free tier discount, another account in the subscription has already been enabled with free tier. If you create an account with free tier and then delete it, you can apply free tier for a new account. When creating a new account, itΓÇÖs recommended to enable the free tier discount if itΓÇÖs available. |
| cosmos-db | How To Restore In Account Continuous Backup | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cosmos-db/how-to-restore-in-account-continuous-backup.md | Use the Azure CLI to restore a deleted container or database. Child containers a ΓÇ» ΓÇ» --resource-group <resource-group-name> \ΓÇ» ΓÇ» ΓÇ» --account-name <account-name> \ΓÇ» ΓÇ» ΓÇ» --name <database-name> \- ΓÇ» ΓÇ» --restore-timestamp <timestamp> + ΓÇ» ΓÇ» --restore-timestamp <timestamp> \ + --disable-ttl True ``` 1. Initiate a restore operation for a deleted container by using [az cosmosdb sql container restore](/cli/azure/cosmosdb/sql/container#az-cosmosdb-sql-container-restore): Use the Azure CLI to restore a deleted container or database. Child containers a ΓÇ» ΓÇ» --resource-group <resource-group-name> \ΓÇ» ΓÇ» ΓÇ» --account-name <account-name> \ΓÇ» ΓÇ» ΓÇ» --database-name <database-name> \- --name <container-name> \ - ΓÇ» ΓÇ» --restore-timestamp <timestamp> + --name <container-name> \ + --restore-timestamp <timestamp> \ + --disable-ttl True ``` :::zone-end Use the Azure CLI to restore a deleted container or database. Child containers a ΓÇ» ΓÇ» --account-name <account-name> \ΓÇ» ΓÇ» ΓÇ» --name <database-name> \ ΓÇ» ΓÇ» --restore-timestamp <timestamp>+ --disable-ttl True ``` 1. Initiate a restore operation for a deleted collection by using [az cosmosdb mongodb collection restore](/cli/azure/cosmosdb/mongodb/collection#az-cosmosdb-mongodb-collection-restore): Use the Azure CLI to restore a deleted container or database. Child containers a ΓÇ» ΓÇ» --account-name <account-name> \ΓÇ» ΓÇ» ΓÇ» --database-name <database-name> \ --name <container-name> \- ΓÇ» ΓÇ» --restore-timestamp <timestamp> + ΓÇ» ΓÇ» --restore-timestamp <timestamp> \ + --disable-ttl True ``` :::zone-end Use the Azure CLI to restore a deleted container or database. Child containers a --resource-group <resource-group-name> \ΓÇ» --account-name <account-name> \ΓÇ» --name <database-name> \ - --restore-timestamp <timestamp> + --restore-timestamp <timestamp> \ + --disable-ttl True ``` 1. Initiate a restore operation for a deleted graph by using [az cosmosdb gremlin graph restore](/cli/azure/cosmosdb/gremlin/graph#az-cosmosdb-gremlin-graph-restore): Use the Azure CLI to restore a deleted container or database. Child containers a --account-name <account-name> \ΓÇ» --database-name <database-name> \ --name <graph-name> \ - --restore-timestamp <timestamp> + --restore-timestamp <timestamp> \ + --disable-ttl True ``` :::zone-end Use the Azure CLI to restore a deleted container or database. Child containers a ΓÇ» ΓÇ» --resource-group <resource-group-name> \ ΓÇ» ΓÇ» --account-name <account-name> \ ΓÇ» ΓÇ» --table-name <table-name> \- ΓÇ» ΓÇ» --restore-timestamp <timestamp> + ΓÇ» ΓÇ» --restore-timestamp <timestamp> \ + --disable-ttl True ``` :::zone-end Use Azure PowerShell to restore a deleted container or database. Child container DatabaseName = "<database-name>" Name = "<container-name>" RestoreTimestampInUtc = "<timestamp>"+ DisableTtl= $true } Restore-AzCosmosDBSqlContainer @parameters ``` Use Azure PowerShell to restore a deleted container or database. Child container AccountName = "<account-name>" Name = "<database-name>" RestoreTimestampInUtc = "<timestamp>"+ DisableTtl=$true } Restore-AzCosmosDBMongoDBDatabase @parameters ``` Use Azure PowerShell to restore a deleted container or database. Child container DatabaseName = "<database-name>" Name = "<collection-name>" RestoreTimestampInUtc = "<timestamp>"+ DisableTtl=$true } Restore-AzCosmosDBMongoDBCollection @parametersΓÇ» ``` Use Azure PowerShell to restore a deleted container or database. Child container AccountName = "<account-name>" Name = "<database-name>" RestoreTimestampInUtc = "<timestamp>"+ DisableTtl=$true } Restore-AzCosmosDBGremlinDatabase @parameters ``` Use Azure PowerShell to restore a deleted container or database. Child container DatabaseName = "<database-name>" Name = "<graph-name>" RestoreTimestampInUtc = "<timestamp>"+ DisableTtl=$true } Restore-AzCosmosDBGremlinGraph @parameters ``` Use Azure PowerShell to restore a deleted container or database. Child container AccountName = "<account-name>" Name = "<table-name>" RestoreTimestampInUtc = "<timestamp>"+ DisableTtl=$true } Restore-AzCosmosDBTable @parameters ``` You can restore deleted containers and databases by using an Azure Resource Mana "name": "<name-of-database-or-container>", "restoreParameters": { "restoreSource": "<source-account-instance-id>",- "restoreTimestampInUtc": "<timestamp>" + "restoreTimestampInUtc": "<timestamp>", + "restoreWithTtlDisabled": "true" }, "createMode": "Restore" } You can restore deleted containers and databases by using an Azure Resource Mana "name": "<name-of-database-or-collection>", "restoreParameters": { "restoreSource": "<source-account-instance-id>",- "restoreTimestampInUtc": "<timestamp>" + "restoreTimestampInUtc": "<timestamp>", + "restoreWithTtlDisabled": "true" }, "createMode": "Restore" } You can restore deleted containers and databases by using an Azure Resource Mana "name": "<name-of-database-or-graph>", "restoreParameters": { "restoreSource": "<source-account-instance-id>",- "restoreTimestampInUtc": "<timestamp>" + "restoreTimestampInUtc": "<timestamp>", + "restoreWithTtlDisabled": "true" }, "createMode": "Restore" } You can restore deleted containers and databases by using an Azure Resource Mana "name": "<name-of-table>", "restoreParameters": { "restoreSource": "<source-account-instance-id>",- "restoreTimestampInUtc": "<timestamp>" + "restoreTimestampInUtc": "<timestamp>", + "restoreWithTtlDisabled": "true" }, "createMode": "Restore" } |
| cosmos-db | How To Setup Customer Managed Keys Existing Accounts | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cosmos-db/how-to-setup-customer-managed-keys-existing-accounts.md | -# Configure customer-managed keys for your existing Azure Cosmos DB account with Azure Key Vault (Preview) +# Configure customer-managed keys for your existing Azure Cosmos DB account with Azure Key Vault [!INCLUDE[NoSQL, MongoDB, Gremlin, Table](includes/appliesto-nosql-mongodb-cassandra-gremlin-table.md)] Enabling a second layer of encryption for data at rest using [Customer Managed K This feature eliminates the need for data migration to a new account to enable CMK. It helps to improve customersΓÇÖ security and compliance posture. -> [!NOTE] -> Currently, enabling customer-managed keys on existing Azure Cosmos DB accounts is in preview. This preview is provided without a service-level agreement. Certain features of this preview may not be supported or may have constrained capabilities. For more information, see [supplemental terms of use for Microsoft Azure previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). - Enabling CMK kicks off a background, asynchronous process to encrypt all the existing data in the account, while new incoming data are encrypted before persisting. There's no need to wait for the asynchronous operation to succeed. The enablement process consumes unused/spare RUs so that it doesn't affect your read/write workloads. You can refer to this [link](./how-to-setup-customer-managed-keys.md?tabs=azure-powershell#how-do-customer-managed-keys-influence-capacity-planning) for capacity planning once your account is encrypted. ## Get started by enabling CMK on your existing accounts +> [!IMPORTANT] +> Go through the prerequisites section thoroughly. These are important considerations. + ### Prerequisites All the prerequisite steps needed while configuring Customer Managed Keys for new accounts is applicable to enable CMK on your existing account. Refer to the steps [here](./how-to-setup-customer-managed-keys.md?tabs=azure-portal#prerequisites) +It is important to note that enabling encryption on your Azure Cosmos DB account will add a small overhead to your document's ID, limiting the maximum size of the document ID to 990 bytes instead of 1024 bytes. If your account has any documents with IDs larger than 990 bytes, the encryption process will fail until those documents are deleted. + +To verify if your account is compliant, you can use the provided console application [hosted here](https://github.com/AzureCosmosDB/Cosmos-DB-Non-CMK-to-CMK-Migration-Scanner) to scan your account. Make sure that you are using the endpoint from your 'sqlEndpoint' account property, no matter the API selected. ++If you wish to disable server-side validation for this during migration, please contact support. + ### Steps to enable CMK on your existing account To enable CMK on an existing account, update the account with an ARM template setting a Key Vault key identifier in the keyVaultKeyUri property ΓÇô just like you would when enabling CMK on a new account. This step can be done by issuing a PATCH call with the following payload: The state of the key is checked when CMK encryption is triggered. If the key in **Can we enable CMK encryption on our existing production account?** -Yes. Since the capability is currently in preview, we recommend testing all scenarios first on nonproduction accounts and once you're comfortable you can consider production accounts. +Yes. Go through the prerequisite section thoroughly. We recommend testing all scenarios first on nonproduction accounts and once you're comfortable you can consider production accounts. ## Next steps |
| cosmos-db | How To Setup Customer Managed Keys | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cosmos-db/how-to-setup-customer-managed-keys.md | Data stored in your Azure Cosmos DB account is automatically and seamlessly encr You must store customer-managed keys in [Azure Key Vault](../key-vault/general/overview.md) and provide a key for each Azure Cosmos DB account that is enabled with customer-managed keys. This key is used to encrypt all the data stored in that account. > [!NOTE]-> Currently, customer-managed keys are available only for new Azure Cosmos DB accounts. You should configure them during account creation. Enabling customer-managed keys on your existing accounts is available for preview. You can refer to the link [here](how-to-setup-customer-managed-keys-existing-accounts.md) for more details +> If you wish to enable customer-managed keys on your existing Azure Cosmos DB accounts then you can refer to the link [here](how-to-setup-customer-managed-keys-existing-accounts.md) for more details > [!WARNING] > The following field names are reserved on Cassandra API tables in accounts using Customer-managed Keys: |
| cosmos-db | Introduction | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cosmos-db/introduction.md | The surge of AI-powered applications created another layer of complexity, becaus Azure Cosmos DB simplifies and expedites your application development by being the single database for your operational data needs, from caching to backup to vector search. It provides the data infrastructure for modern applications like AI, digital commerce, Internet of Things, and booking management. It can accommodate all your operational data models, including relational, document, vector, key-value, graph, and table. -## An AI database providing industry-leading capabilities... for free +## An AI database providing industry-leading capabilities... ++## ...for free Azure Cosmos DB is a fully managed NoSQL, relational, and vector database. It offers single-digit millisecond response times, automatic and instant scalability, along with guaranteed speed at any scale. Business continuity is assured with [SLA-backed](https://azure.microsoft.com/support/legal/sla/cosmos-db) availability and enterprise-grade security. App development is faster and more productive thanks to: - Open source APIs - SDKs for popular languages - AI database functionalities like integrated vector database or seamless integration with Azure AI Services to support Retrieval Augmented Generation-- Query Copilot for generating NoSQL queries based on your natural language prompts [(preview)](nosql/query/how-to-enable-use-copilot.md)+- Query Copilot for generating NoSQL queries based on your natural language prompts ([preview](nosql/query/how-to-enable-use-copilot.md)) As a fully managed service, Azure Cosmos DB takes database administration off your hands with automatic management, updates, and patching. It also handles capacity management with cost-effective serverless and automatic scaling options that respond to application needs to match capacity with demand. -If you're an existing Azure AI or GitHub Copilot customer, you may try Azure Cosmos DB for free with 40,000 [RU/s](request-units.md) of throughput for 90 days under the Azure AI Advantage offer. +If you're an existing Azure AI or GitHub Copilot customer, you may try Azure Cosmos DB for free with 40,000 [RU/s](request-units.md) (equivalent of up to $6,000) of throughput for 90 days under the [Azure AI Advantage offer](ai-advantage.md). -> [!div class="nextstepaction"] -> [90-day Free Trial with Azure AI Advantage](ai-advantage.md) +Alternatively, you may use the [Azure Cosmos DB lifetime free tier](free-tier.md) with the first 1000 [RU/s](request-units.md) of throughput and 25 GB of storage free. -If you aren't an Azure customer, you may use the [30-day Free Trial without an Azure subscription](https://azure.microsoft.com/try/cosmosdb/). No commitment follows the end of your trial period. +If you aren't already using Azure, you may Try Azure Cosmos DB free for 30 days without an Azure subscription ([learn more](https://azure.microsoft.com/try/cosmosdb/)). No commitment follows the end of your trial period. -Alternatively, you may use the [Azure Cosmos DB lifetime free tier](free-tier.md) with the first 1000 [RU/s](request-units.md) of throughput and 25 GB of storage free. +> [!div class="nextstepaction"] +> [Try Azure Cosmos DB free](https://azure.microsoft.com/try/cosmosdb/) > [!TIP] > To learn more about Azure Cosmos DB, join us every Thursday at 1PM Pacific on Azure Cosmos DB Live TV. See the [Upcoming session schedule and past episodes](https://gotcosmos.com/tv). -## An AI database for more than just AI apps +## ...for more than just AI apps Besides AI, Azure Cosmos DB should also be your goto database for web, mobile, gaming, and IoT applications. Azure Cosmos DB is well positioned for solutions that handle massive amounts of data, reads, and writes at a global scale with near-real response times. Azure Cosmos DB's guaranteed high availability, high throughput, low latency, and tunable consistency are huge advantages when building these types of applications. Learn about how Azure Cosmos DB can be used to build IoT and telematics, retail and marketing, gaming and web and mobile applications. -## An AI database with unmatched reliability and flexibility +## ...with unmatched reliability and flexibility ### Guaranteed speed at any scale |
| cosmos-db | Optimize Dev Test | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cosmos-db/optimize-dev-test.md | This article describes the different options to use Azure Cosmos DB for developm Azure Cosmos DB free tier makes it easy to get started, develop and test your applications, or even run small production workloads for free. When free tier is enabled on an account, you'll get the first 1000 RU/s and 25 GB of storage in the account free. -Free tier lasts indefinitely for the lifetime of the account and comes with all the [benefits and features](introduction.md#an-ai-database-with-unmatched-reliability-and-flexibility) of a regular Azure Cosmos DB account, including unlimited storage and throughput (RU/s), SLAs, high availability, turnkey global distribution in all Azure regions, and more. You can create a free tier account using Azure portal, CLI, PowerShell, and a Resource Manager template. To learn more, see how to [create a free tier account](free-tier.md) article and the [pricing page](https://azure.microsoft.com/pricing/details/cosmos-db/). +Free tier lasts indefinitely for the lifetime of the account and comes with all the [benefits and features](introduction.md#with-unmatched-reliability-and-flexibility) of a regular Azure Cosmos DB account, including unlimited storage and throughput (RU/s), SLAs, high availability, turnkey global distribution in all Azure regions, and more. You can create a free tier account using Azure portal, CLI, PowerShell, and a Resource Manager template. To learn more, see how to [create a free tier account](free-tier.md) article and the [pricing page](https://azure.microsoft.com/pricing/details/cosmos-db/). ## Azure free account |
| cosmos-db | Restore Account Continuous Backup | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cosmos-db/restore-account-continuous-backup.md | description: Learn how to identify the restore time and restore a live or delete Previously updated : 03/31/2023 Last updated : 03/21/2024 Before restoring the account, install the [latest version of Azure PowerShell](/ ### <a id="trigger-restore-ps"></a>Trigger a restore operation for API for NoSQL account -The following cmdlet is an example to trigger a restore operation with the restore command by using the target account, source account, location, resource group, PublicNetworkAccess and timestamp: +The following cmdlet is an example to trigger a restore operation with the restore command by using the target account, source account, location, resource group, PublicNetworkAccess, DisableTtl, and timestamp: Restore-AzCosmosDBAccount ` -SourceDatabaseAccountName "SourceDatabaseAccountName" ` -RestoreTimestampInUtc "UTCTime" ` -Location "AzureRegionName" ` - -PublicNetworkAccess Disabled + -PublicNetworkAccess Disabled ` + -DisableTtl $true ``` Restore-AzCosmosDBAccount ` -RestoreTimestampInUtc "2021-01-05T22:06:00" ` -Location "West US" ` -PublicNetworkAccess Disabled+ -DisableTtl $false + ```-If `PublicNetworkAccess` is not set, restored account is accessible from public network, please ensure to pass `Disabled` to the `PublicNetworkAccess` option to disable public network access for restored account. +If `PublicNetworkAccess` is not set, restored account is accessible from public network, please ensure to pass `Disabled` to the `PublicNetworkAccess` option to disable public network access for restored account. Setting DisableTtl to $true ensures TTL is disabled on restored account, not providing parameter restores the account with TTL enabled if it was set earlier. > [!NOTE] > For restoring with public network access disabled, the minimum stable version of Az.CosmosDB required is 1.12.0. az cosmosdb restore \ --restore-timestamp 2020-07-13T16:03:41+0000 \ --resource-group <MyResourceGroup> \ --location "West US" \- --public-network-access Disabled + --public-network-access Disabled \ + --disable-ttl True ``` -If `--public-network-access` is not set, restored account is accessible from public network. Please ensure to pass `Disabled` to the `--public-network-access` option to prevent public network access for restored account. +If `--public-network-access` is not set, restored account is accessible from public network. Please ensure to pass `Disabled` to the `--public-network-access` option to prevent public network access for restored account. Setting disable-ttl to to $true ensures TTL is disabled on restored account, and not providing this parameter restores the account with TTL enabled if it was set earlier. > [!NOTE] > For restoring with public network access disabled, the minimum stable version of azure-cli is 2.52.0. This command output now shows when a database was created and deleted. [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.DocumentDB/locations/West US/restorableDatabaseAccounts/abcd1234-d1c0-4645-a699-abcd1234/restorableSqlDatabases/40e93dbd-2abe-4356-a31a-35567b777220",- .. - "name": "40e93dbd-2abe-4356-a31a-35567b777220", + "name": "40e93dbd-2abe-4356-a31a-35567b777220", "resource": { "database": { "id": "db1" This command output now shows when a database was created and deleted. "ownerId": "db1", "ownerResourceId": "YuZAAA==" },- .. + }, { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.DocumentDB/locations/West US/restorableDatabaseAccounts/abcd1234-d1c0-4645-a699-abcd1234/restorableSqlDatabases/243c38cb-5c41-4931-8cfb-5948881a40ea",- .. "name": "243c38cb-5c41-4931-8cfb-5948881a40ea", "resource": { "database": { This command output now shows when a database was created and deleted. "ownerId": "spdb1", "ownerResourceId": "OIQ1AA==" },- .. + } ] ``` This command output shows includes list of operations performed on all the conta ```json [ {- ... - "eventTimestamp": "2021-01-08T23:25:29Z", "operationType": "Replace", "ownerId": "procol3", "ownerResourceId": "OIQ1APZ7U18="-... }, {- ... "eventTimestamp": "2021-01-08T23:25:26Z", "operationType": "Create", "ownerId": "procol3", az cosmosdb gremlin restorable-resource list \ --restore-location "West US" \ --restore-timestamp "2021-01-10T01:00:00+0000" ```+This command output shows the graphs which are restorable: + ```-[ { -``` +[ + { "databaseName": "db1", -"graphNames": [ - "graph1", - "graph3", - "graph2" -] -``` +"graphNames": [ "graph1", "graph3", "graph2" ] } ] ``` az cosmosdb table restorable-table list \ --instance-id "abcd1234-d1c0-4645-a699-abcd1234" --location "West US" ```+ ``` [ { -``` + "id": "/subscriptions/23587e98-b6ac-4328-a753-03bcd3c8e744/providers/Microsoft.DocumentDB/locations/WestUS/restorableDatabaseAccounts/7e4d666a-c6ba-4e1f-a4b9-e92017c5e8df/restorableTables/59781d91-682b-4cc2-93a3-c25d03fab159", "name": "59781d91-682b-4cc2-93a3-c25d03fab159", "resource": { az cosmosdb table restorable-table list \ "ownerId": "table1", "ownerResourceId": "tOdDAKYiBhQ=", "rid": "9pvDGwAAAA==" -}, -"type": "Microsoft.DocumentDB/locations/restorableDatabaseAccounts/restorableTables" -``` }, -``` +"type": "Microsoft.DocumentDB/locations/restorableDatabaseAccounts/restorableTables" + }, + {"id": "/subscriptions/23587e98-b6ac-4328-a753-03bcd3c8e744/providers/Microsoft.DocumentDB/locations/eastus2euap/restorableDatabaseAccounts/7e4d666a-c6ba-4e1f-a4b9-e92017c5e8df/restorableTables/2c9f35eb-a14c-4ab5-a7e0-6326c4f6b785", "name": "2c9f35eb-a14c-4ab5-a7e0-6326c4f6b785", "resource": { az cosmosdb table restorable-table list \ "rid": "01DtkgAAAA==" }, "type": "Microsoft.DocumentDB/locations/restorableDatabaseAccounts/restorableTables" -``` + }, ] ``` az cosmosdb table restorable-resource list \ --restore-location "West US" \ --restore-timestamp "2020-07-20T16:09:53+0000" ```++Following is the result of the command. + ``` { "tableNames": [ -``` "table1", "table3", "table2" -``` + ] } ``` Use the following ARM template to restore an account for the Azure Cosmos DB API "restoreParameters": { "restoreSource": "/subscriptions/2296c272-5d55-40d9-bc05-4d56dc2d7588/providers/Microsoft.DocumentDB/locations/West US/restorableDatabaseAccounts/6a18ecb8-88c2-4005-8dce-07b44b9741df", "restoreMode": "PointInTime",- "restoreTimestampInUtc": "6/24/2020 4:01:48 AM" + "restoreTimestampInUtc": "6/24/2020 4:01:48 AM", + "restoreWithTtlDisabled": "true" } } } |
| cosmos-db | Restore In Account Continuous Backup Resource Model | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cosmos-db/restore-in-account-continuous-backup-resource-model.md | Title: Resource model for same account restore (preview) + Title: Resource model for same account restore description: Review the required parameters and resource model for the same account(in-account) point-in-time restore feature of Azure Cosmos DB. -# Resource model for restore in same account for Azure Cosmos DB (preview) +# Resource model for restore in same account for Azure Cosmos DB [!INCLUDE[NoSQL, MongoDB, Gremlin, Table](includes/appliesto-nosql-mongodb-gremlin-table.md)] |
| cosmos-db | Vector Database | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cosmos-db/vector-database.md | Last updated 03/30/2024 Vector databases are used in numerous domains and situations across analytical and generative AI, including natural language processing, video and image recognition, recommendation system, search, etc. -In 2023, a notable trend in software was the integration of AI enhancements, often achieved by incorporating specialized standalone vector databases into existing tech stacks. This article explains what vector databases are, as well as presents an alternative architecture that you might want to consider: using an integrated vector database in the NoSQL or relational database you already use, especially when working with multi-modal data. This approach not only allows you to reduce cost but also achieve greater data consistency, scale, and performance. +In 2023, a notable trend in software was the integration of AI enhancements, often achieved by incorporating specialized standalone vector databases into existing tech stacks. This article explains what vector databases are, as well as presents an alternative architecture that you might want to consider: using an integrated vector database in the NoSQL or relational database you already use, especially when working with multi-modal data. This approach not only allows you to reduce cost but also achieve greater data consistency, scalability, and performance. > [!TIP]-> Data consistency, scale, and performance guarantees are why OpenAI built its ChatGPT service on top of Azure Cosmos DB. You, too, can take advantage of its integrated vector database, as well as its single-digit millisecond response times, automatic and instant scalability, and guaranteed speed at any scale. Please consult the [implementation samples](#how-to-implement-integrated-vector-database-functionalities) section of this article and [try](#next-step) the lifetime free tier or one of the free trial options. +> Data consistency, scalability, and performance are critical for data-intensive applications, which is why OpenAI chose to build the ChatGPT service on top of Azure Cosmos DB. You, too, can take advantage of its integrated vector database, as well as its single-digit millisecond response times, automatic and instant scalability, and guaranteed speed at any scale. See [implementation samples](#how-to-implement-integrated-vector-database-functionalities) and [try](#next-step) it for free. ## What is a vector database? The natively integrated vector database in our NoSQL API will become available i [30-day Free Trial without Azure subscription](https://azure.microsoft.com/try/cosmosdb/) -[90-day Free Trial with Azure AI Advantage](ai-advantage.md) +[90-day Free Trial and up to $6,000 in throughput credits with Azure AI Advantage](ai-advantage.md) > [!div class="nextstepaction"] > [Use the Azure Cosmos DB lifetime free tier](free-tier.md) +> [!div class="nextstepaction"] +> [Use the Azure Cosmos DB for MongoDB lifetime free vCore cluster](mongodb/vcore/free-tier.md) + ## More Vector Databases - [Azure PostgreSQL Server pgvector Extension](../postgresql/flexible-server/how-to-use-pgvector.md) |
| cost-management-billing | Onboard Microsoft Customer Agreement | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cost-management-billing/microsoft-customer-agreement/onboard-microsoft-customer-agreement.md | -This playbook (guide) helps customers who buy Microsoft software and services through a Microsoft account manager set up an MCA. The guide was created to recommend best practices to onboard you to an MCA. +This playbook (guide) helps customers who buy Microsoft software and services through a Microsoft account manager to set up an MCA. The guide was created to recommend best practices to onboard you to an MCA. The onboarding processes and important considerations vary, depending on whether you are: -- New to MCA and have never signed an MCA contract but may have bought Azure and per-seat products using another method, such as licensing vehicle or contracting type.+- New to MCA and didn't already sign an MCA contract but might have bought Azure and per device or user products using another method, such as licensing vehicle or contracting type. -Or- This guide follows each path and provides information for each step of the proce - **[Enterprise Agreement (EA)](https://www.microsoft.com/en-us/licensing/licensing-programs/enterprise)** - A licensing agreement designed for large organizations with 500 or more users or devices. It's a volume licensing program that gives organizations the flexibility to buy Azure or seat-based cloud services and software licenses under one agreement. - **Microsoft Customer Agreement (MCA)** - A Microsoft licensing agreement designed for automated processing, dynamic updating of terms, transparent pricing, and enhanced billing management capabilities.-- **Pay-as-you-go (PAYG)** ΓÇô A utility computing billing method that's used in cloud computing and geared towards organizations and end users. PAYG is a pricing option where you pay for the resources you use on an hourly or monthly basis. You only pay for what you use and can scale up or down as needed.+- **Pay-as-you-go (PAYG)** ΓÇô A utility computing billing method used in cloud computing and geared towards organizations and end users. Pay-as-you-go is a pricing option where you pay for the resources you use on an hourly or monthly basis. You only pay for what you use and can scale up or down as needed. - **APIs** - A software intermediary that allows two applications to interact with each other. For example, it defines the kinds of calls or requests that can be made, how to make them, the data formats that should be used, and the conventions to follow. - **Power BI** - A suite of Microsoft data visualization tools used to deliver insights throughout organizations. The [MCA](https://www.microsoft.com/Licensing/how-to-buy/microsoft-customer-agre The MCA has several benefits that can improve your invoice process, billing operations, and overall cost management including: -Simplified purchasing with **fast and fully automated** access to Azure and per-seat licenses +Simplified purchasing with **fast and fully automated** access to Azure and per device or user licenses - A single, short agreement that doesn't expire and can be digitally signed - Allows you to complete a purchase and start using Azure right away - No upfront costs required with pay-as-you-go billing for most services - Buy only what you need when you need it and negotiate commitments when desired-- Per-seat subscriptions allow you to easily manage and track your organization's software usage+- You to easily manage and track your organization's software usage with per device or per user subscriptions Improved billing experience with **intuitive invoices** - Intuitive invoice layout displays charges in an easy-to-read format, making expenditures easier to understand Management, deployment, and optimization tools in a **single portal** - Manage all your Azure purchases through a single, unified portal at Azure.com - Centrally control user authorizations in a single place with a single set of roles - Integrated cost management capabilities provide enterprise-grade insights into usage with recommendations on how to save money-- Easily manage your per-seat subscriptions for Microsoft licenses through the same portal, streamlining your software management process.+- Easily manage your per device or user subscriptions for Microsoft licenses through the same portal, streamlining your software management process. ## New MCA Customer This section describes the steps you must take to enable and sign an MCA, which allows you to experience its benefits. >[!NOTE]-> The following steps apply only to **new MCA customers** that have never signed an MCA or EA but who may have bought Azure or per seat products through another method, such as a licensing vehicle or contracting type. If you're a **customer migrating to MCA from an existing Microsoft EA**, see [Migrate from an EA to transition to an MCA](#migrate-from-an-ea-to-an-mca). +> The following steps apply only to **new MCA customers** that have never signed an MCA or EA but who might have bought Azure or per device or user products through another method, such as a licensing vehicle or contracting type. If you're a **customer migrating to MCA from an existing Microsoft EA**, see [Migrate from an EA to transition to an MCA](#migrate-from-an-ea-to-an-mca). Start your journey to MCA by using the steps in the following diagram. More details and supporting links are in the sections that follow the diagram. You can accelerate proposal creation and contract signature by gathering the fol - **Company's VAT or Tax ID** - **The primary contact's name, phone number, and email address** -**The name and email address of the Billing Account Owner** who is the person in your organization that has authorization. They make the initial purchases and sign the MCA. They may or may not be the same person as the signer mentioned previously, depending on your organization's requirements. +**The name and email address of the Billing Account Owner** who is the person in your organization that has authorization. They make the initial purchases and sign the MCA. They might or might not be the same person as the signer mentioned previously, depending on your organization's requirements. If your organization has specific requirements for signing contracts such as who can sign, purchasing limits or how many people need to sign, advise your Microsoft account manager in advance. To become operational includes steps to manage billing accounts, fully understan Each billing account has at least one billing profile. Your first billing profile is set up when you sign up to use Azure. Users assigned to roles for a billing profile can view cost, set budgets, and can manage and pay invoices. Get an overview of how to [set up and manage your billing account](https://www.youtube.com/watch?v=gyvHl5VNWg4&ab_channel=MicrosoftAzure) and learn about the powerful [billing capabilities](../understand/mca-overview.md). +For more information, see the following how-to videos: ++- [How to organize your Microsoft Customer Agreement Billing Account in the Azure portal](https://www.youtube.com/watch?v=6lmaovgWiZw&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=7) +- [How to find a copy of your Microsoft Customer Agreement in the Azure portal](https://www.youtube.com/watch?v=SQbKGo8JV74&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=4) ++If you're looking for Microsoft 365 admin center video resources, see [Microsoft Customer Agreement Video Tutorials](https://www.microsoft.com/licensing/learn-more/microsoft-customer-agreement/video-tutorials). + ### Step 6 ΓÇô Understand your MCA invoice In the billing account for an MCA, an invoice is generated every month for each billing profile. The invoice includes all charges from the previous month organized by invoice sections that you can define. You can view your invoices in the Azure portal and compare the charges to the usage detail files. Learn how the [charges on your invoice](https://www.youtube.com/watch?v=e2LGZZ7GubA&feature) work and take a step-by-step [invoice tutorial](../understand/review-customer-agreement-bill.md). +For more information, see the [How to find and read your Microsoft Customer Agreement invoices in the Azure portal](https://www.youtube.com/watch?v=xkUkIunP4l8&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=5) video. + ### Step 7 ΓÇô Get to know MCA features Learn more about features that you can use to optimize your experience and accelerate the value of MCA for your organization. The following sections help you establish governance for your MCA. We recommend using billing account roles to manage your billing account on the MCA. These roles are in addition to the built-in Azure roles used to manage resource assignments. Billing account roles are used to manage your billing account, profiles, and invoice sections. Learn how to manage who has [access to your billing account](https://www.youtube.com/watch?v=9sqglBlKkho&ab_channel=AzureCostManagement) and get an overview of [how billing account roles work](../manage/understand-mca-roles.md) in Azure. +For more information, see the [How to manage access to your Microsoft Customer Agreement in the Azure portal](https://www.youtube.com/watch?v=jh7PUKeAb0M&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=6) video. + ### Step 9 ΓÇô Organize your costs and customize billing The MCA provides you with flexibility to organize your costs based on your needs, whether it's by department, project, or development environment. Understand how to [organize your costs](https://www.youtube.com/watch?v=7RxTfShGHwU) and to [customize your billing](../manage/mca-section-invoice.md) to meet your needs. +For more information, see the [How to optimize your workloads and reduce costs under your Microsoft Customer Agreement](https://www.youtube.com/watch?v=UxO2cFyWn0w&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=3) video. + ### Step 10 ΓÇô Evaluate your needs for more tenants -The MCA allows you to create multi-tenant billing relationships. They let you securely share your billing account with other tenants, while maintaining control over your billing data. If your organization needs multiple tenants, see [Manage billing across multiple tenants](../manage/manage-billing-across-tenants.md). +The MCA allows you to create multitenant billing relationships. They let you securely share your billing account with other tenants, while maintaining control over your billing data. If your organization needs multiple tenants, see [Manage billing across multiple tenants](../manage/manage-billing-across-tenants.md). ## Manage your new MCA An Azure subscription is a logical container used to create resources in Azure. To create a subscription, see Create a [Microsoft Customer Agreement subscription](../manage/create-subscription.md). +For more information about creating a subscription, see the [How to create an Azure Subscription under your Microsoft Customer Agreement](https://www.youtube.com/watch?v=u5wf8KMD_M8&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=8) video. ++If you're looking for Microsoft 365 admin center video resources, see [Microsoft Customer Agreement Video Tutorials](https://www.microsoft.com/licensing/learn-more/microsoft-customer-agreement/video-tutorials). + ## Migrate from an EA to an MCA This section of the onboarding guide describes the steps you follow to migrate from an EA to an MCA. Although the steps in this section are like those in the previous [New MCA customer](#new-mca-customer) section, there are important differences called out throughout this section. This section of the onboarding guide describes the steps you follow to migrate f The following points help you plan for your migration from EA to MCA: - Migrating from EA to MCA redirects your charges from your EA enrollment to your MCA billing account after you complete the subscription migration. The change goes into effect immediately. Any charges incurred up to the point of migration are invoiced to the EA and must be settled on that enrollment. There's no effect on your services and no downtime.-- You can continue to see your historic charges in the Azure portal under your EA enrollment billing scope.-- Depending on the timing of your migration, you may receive two invoices, one EA and one MCA, in the transition month. The MCA invoice covers usage for a calendar month and is generated from the fifth to the seventh day of the month following the usage.+- You can continue to see your historic charges in the Azure portal under your EA enrollment billing scope. Historical charges aren't visible in cost analysis when migration completes if you're an Account owner or a subscription owner without access to view the EA billing scope. We recommend that you [download your cost and usage data and invoices](../understand/download-azure-daily-usage.md) before you transfer subscriptions. +- Depending on the timing of your migration, you might receive two invoices, one EA and one MCA, in the transition month. The MCA invoice covers usage for a calendar month and is generated from the fifth to the seventh day of the month following the usage. - To ensure your MCA invoice gets received by the right person or group, you must add an accounts payable email address as an invoice recipient's contact to the MCA. For more information, see [share your billing profiles invoice](../understand/download-azure-invoice.md#share-your-billing-profiles-invoice). - If you use Cost Management APIs for reporting purposes, familiarize yourself with [Other actions to manage your MCA](#other-actions-to-manage-your-mca). - Be sure to alert your accounts payable team of the important change to your invoice. You get a final EA invoice and start receiving a new monthly MCA invoice. You can accelerate proposal creation and contract signature by gathering the fol - **Company's VAT or Tax ID.** - **The primary contact's name, phone number and email address.** -**The name and email address of the Billing Account Owner** who is the person in your organization that has authorization and signs the MCA and who makes the initial purchases. They may or may not be the same person as the signer mentioned previously, depending on your organization's requirements. +**The name and email address of the Billing Account Owner** who is the person in your organization that has authorization and signs the MCA and who makes the initial purchases. They might or might not be the same person as the signer mentioned previously, depending on your organization's requirements. If your organization has specific requirements for signing contracts such as who can sign, purchasing limits or how many people need to sign, advise your Microsoft account manager in advance. Becoming operational includes steps to manage billing accounts, fully understand Each billing account has at least one billing profile. Your first billing profile is set up when you sign up to use Azure. Users assigned to roles for a billing profile can view cost, set budgets, and manage and pay invoices. Get an overview of how to [set up and manage your billing account](https://www.youtube.com/watch?v=gyvHl5VNWg4&ab_channel=MicrosoftAzure) and learn about the powerful [billing capabilities](../understand/mca-overview.md). +For more information, see the following how-to videos: ++- [How to organize your Microsoft Customer Agreement Billing Account in the Azure portal](https://www.youtube.com/watch?v=6lmaovgWiZw&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=7) +- [How to find a copy of your Microsoft Customer Agreement in the Azure portal](https://www.youtube.com/watch?v=SQbKGo8JV74&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=4) ++If you're looking for Microsoft 365 admin center video resources, see [Microsoft Customer Agreement Video Tutorials](https://www.microsoft.com/licensing/learn-more/microsoft-customer-agreement/video-tutorials). + ### Step 6 - Understand your MCA invoice In the billing account for an MCA, an invoice is generated every month for each billing profile. The invoice includes all charges from the previous month organized by invoice sections that you can define. You can view your invoices in the Azure portal and compare the charges to the usage detail files. Learn how the [charges on your invoice](https://www.youtube.com/watch?v=e2LGZZ7GubA&feature) work and take a step-by-step [invoice tutorial](../understand/review-customer-agreement-bill.md). In the billing account for an MCA, an invoice is generated every month for each >[!IMPORTANT] > Bank remittance details for your new MCA will differ from those for your old EA. Use the remittance information at the bottom of your MCA invoice. For more information, see [Bank details used to send wire transfers](../understand/pay-bill.md#bank-details-used-to-send-wire-transfer-payments). +For more information, see the [How to find and read your Microsoft Customer Agreement invoices in the Azure portal](https://www.youtube.com/watch?v=xkUkIunP4l8&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=5) video. + ### Step 7 ΓÇô Get to know MCA features Learn more about features that can use to optimize your experience and accelerate the value of MCA for your organization. Use the following steps to establish governance for your MCA. We recommend using billing account roles to manage your billing account on the MCA. These roles are in addition to the built-in Azure roles used to manage resource assignments. Billing account roles are used to manage your billing account, profiles, and invoice sections. Learn how to manage who has [access to your billing account](https://www.youtube.com/watch?v=9sqglBlKkho&ab_channel=AzureCostManagement) and get an overview of [how billing account roles work](../manage/understand-mca-roles.md) in Azure. +For more information, see the [How to manage access to your Microsoft Customer Agreement in the Azure portal](https://www.youtube.com/watch?v=jh7PUKeAb0M&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=6) video. + ### Step 9 - Organize your costs and customize billing The MCA provides you with flexibility to organize your costs based on your needs whether it's by department, project, or development environment. Understand how to [organize your costs](https://www.youtube.com/watch?v=7RxTfShGHwU) and to [customize your billing](../manage/mca-section-invoice.md) to meet your needs. +For more information, see the [How to optimize your workloads and reduce costs under your Microsoft Customer Agreement](https://www.youtube.com/watch?v=UxO2cFyWn0w&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=3) video. + ### Step 10 - Evaluate your needs for more tenants -The MCA allows you to create multi-tenant billing relationships. They let you securely share your billing account with other tenants, while maintaining control over your billing data. If your organization needs multiple tenants, see [Manage billing across multiple tenants](../manage/manage-billing-across-tenants.md). +The MCA allows you to create multitenant billing relationships. They let you securely share your billing account with other tenants, while maintaining control over your billing data. If your organization needs multiple tenants, see [Manage billing across multiple tenants](../manage/manage-billing-across-tenants.md). ## Manage your MCA after migration Transition the billing ownership from your old agreement to your new one. For more information, see [Cost Management + Billing frequently asked questions](../cost-management-billing-faq.yml). +For more information about creating a subscription, see the [How to create an Azure Subscription under your Microsoft Customer Agreement](https://www.youtube.com/watch?v=u5wf8KMD_M8&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=8) video. ++If you're looking for Microsoft 365 admin center video resources, see [Microsoft Customer Agreement Video Tutorials](https://www.microsoft.com/licensing/learn-more/microsoft-customer-agreement/video-tutorials). + ## Other actions to manage your MCA -The MCA provides more features for automation, reporting, and billing optimization for multiple tenants. These features may not be applicable to all customers; however, for those customers who need more reporting and automation, these features offer significant benefits. Review the following steps if necessary: +The MCA provides more features for automation, reporting, and billing optimization for multiple tenants. These features might not be applicable to all customers; however, for those customers who need more reporting and automation, these features offer significant benefits. Review the following steps if necessary: ### Migrating APIs If you need more support, use your standard support contacts, such as: - Your Microsoft account manager. - Access [Microsoft support](https://portal.azure.com/#view/Microsoft_Azure_Support/NewSupportRequestV3Blade) in the Azure portal. +## MCA how-to videos ++The following videos provide more information about how to manage your MCA: ++- [Faster, Simpler Purchasing with the Microsoft Customer Agreement](https://www.youtube.com/watch?v=nhpIbhqojWE&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=2) +- [How to optimize your workloads and reduce costs under your Microsoft Customer Agreement](https://www.youtube.com/watch?v=UxO2cFyWn0w&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=3) +- [How to find a copy of your Microsoft Customer Agreement in the Azure portal](https://www.youtube.com/watch?v=SQbKGo8JV74&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=4) +- [How to find and read your Microsoft Customer Agreement invoices in the Azure portal](https://www.youtube.com/watch?v=xkUkIunP4l8&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=5) +- [How to manage access to your Microsoft Customer Agreement in the Azure portal](https://www.youtube.com/watch?v=jh7PUKeAb0M&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=6) +- [How to organize your Microsoft Customer Agreement Billing Account in the Azure portal](https://www.youtube.com/watch?v=6lmaovgWiZw&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=7) +- [How to create an Azure Subscription under your Microsoft Customer Agreement](https://www.youtube.com/watch?v=u5wf8KMD_M8&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=8) +- [How to manage your subscriptions and organize your account in the Microsoft 365 admin center](https://www.youtube.com/watch?v=NO25_5QXoy8&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=9) +- [How to find a copy of your Microsoft Customer Agreement in the Microsoft 365 admin center (MAC)](https://www.youtube.com/watch?v=pIe5yHljdcM&list=PLC6yPvO9Xb_fRexgBmBeILhzxdETFUZbv&index=10) + ## Next steps - [View and download your Azure invoice](../understand/download-azure-invoice.md) |
| cost-management-billing | Choose Commitment Amount | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cost-management-billing/savings-plan/choose-commitment-amount.md | Software costs aren't covered by savings plans. For more information, see [Softw ## Savings plan purchase recommendations -Savings plan purchase recommendations are calculated by analyzing your hourly usage data over the last 7, 30, and 60 days. Azure simulates what your costs would have been if you had a savings plan and compares it with your actual pay-as-you-go costs incurred over the time duration. The commitment amount that maximizes your savings is recommended. To learn more about how recommendations are generated, see [How hourly commitment recommendations are generated](purchase-recommendations.md#how-hourly-commitment-recommendations-are-generated). +Savings plan purchase recommendations are calculated by analyzing your hourly usage data over the last 7, 30, and 60 days. Azure simulates what your costs would have been if you had a savings plan and compares it with your actual pay-as-you-go costs incurred over the time duration. The commitment amount that maximizes your savings is recommended. To learn more about how recommendations are generated, see [How savings plan recommendations are generated](purchase-recommendations.md#how-savings-plan-recommendations-are-generated). For example, you might incur about $500 in hourly pay-as-you-go compute charges most of the time, but sometimes usage spikes to $700. Azure determines your total costs (hourly savings plan commitment plus pay-as-you-go charges) if you had either a $500/hour or a $700/hour savings plan. Since the $700 usage is sporadic, the recommendation calculation is likely to determine that a $500 hourly commitment provides greater total savings. As a result, the $500/hour plan would be the recommended commitment. |
| cost-management-billing | Purchase Recommendations | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cost-management-billing/savings-plan/purchase-recommendations.md | Last updated 11/17/2023 Azure savings plan purchase recommendations are provided through [Azure Advisor](https://portal.azure.com/#view/Microsoft_Azure_Expert/AdvisorMenuBlade/~/Cost), the savings plan purchase experience in [Azure portal](https://portal.azure.com/), and through the [Savings plan benefit recommendations API](/rest/api/cost-management/benefit-recommendations/list). -## How hourly commitment recommendations are generated +## How savings plan recommendations are generated -The goal of our savings plan recommendation is to help you make the most cost-effective commitment. Calculations are based on your actual on-demand costs, and don't include usage covered by existing reservations or savings plans. +The goal of our savings plan recommendation is to help you make the most cost-effective commitment. Saving plan recommendations are generated using your actual on-demand usage and costs (including any negotiated on-demand discounts). -We start by looking at your hourly and total on-demand usage costs incurred from savings plan-eligible resources in the last 7, 30, and 60 days. These costs are inclusive of any negotiated discounts that you have. We then run hundreds of simulations of what your total cost would have been if you had purchased either a one or three-year savings plan with an hourly commitment equivalent to your hourly costs. +We start by looking at your hourly and total on-demand usage costs incurred from savings plan-eligible resources in the last 7, 30, and 60 days. We determine what the optimal savings plan commitment would have been for each of these hours - this is done by applying the appropriate savings plan discounts to all your savings plan-eligible usage in each hour. We consider each one of these commitments a candidate for a savings plan recommendation. We then run hundreds of simulations using each of these candidates to determine what your total cost would have been if you had purchased a savings plan equal to the candiate. -As we simulate each candidate recommendation, some hours will result in savings. For example, when savings plan-discounted usage plus the hourly commitment less than that hourΓÇÖs historic on-demand charge. In other hours, no savings would be realized. For example, when discounted usage plus the hourly commitment is greater than or greater than on-demand charges. We sum up the simulated hourly charges for each candidate and compare it to your actual total on-demand charge. Only candidates that result in savings are eligible for consideration as recommendations. We also calculate the percentage of your compute usage costs that would be covered by the recommendation, plus any other previously purchased reservations or savings plan. +Here's a video that explains how savings plan recommendations are generated. -Finally, we present a differentiated set of one-year and three-year recommendations (currently up to 10 each). The recommendations provide the greatest savings across different compute coverage levels. The recommendations with the greatest savings for one year and three years are the highlighted options. +>[!VIDEO https://www.youtube.com/embed/4HV9GT9kX6A] -To account for scenarios where there were significant reductions in your usage, including recently decommissioned services, we run more simulations using only the last three days of usage. The lower of the three day and 30-day recommendations are highlighted, even in situations where the 30-day recommendation may appear to provide greater savings. The lower recommendation is to ensure that we don't encourage overcommitment based on stale data. +The goal of these simulations is to compare each candidate's total cost ((hourly commitment * 24 hours * # of days in simulation period) + total on-demand cost incurred during the simulation period) to the actual total on-demand costs. Only candidates that result in net savings are eligible for consideration as actual recommendations. We take up to 10 of the best recommendations and present them to you. For each recommendation, we also calculate the percentage of your compute usage costs would now be covered by this savings plan, and any other previously purchased reservations or savings plan. The recommendations with the greatest savings for one year and three years are the highlighted options. ++To account for scenarios where there were significant reductions in your usage, including recently decommissioned services, we run more simulations using only the last three days of usage. The lower recommendation (between the three day and 30-day recommendations) is shared, even in situations where the 30-day recommendation may appear to provide greater savings. This is done to ensure that we don't inadvertently recommend overcommitment based on stale data. Note the following points: - Recommendations are refreshed several times a day.-- The recommended quantity for a scope is reduced on the same day that you purchase a savings plan for the scope. However, an update for the savings plan recommendation across scopes can take up to 25 days.+- The savings plan recommendation for a specific scope is reduced on the same day that you purchase a savings plan for that scope. However, updates to recommendations for other scopes can take up to 25 days. - For example, if you purchase based on shared scope recommendations, the single subscription scope recommendations can take up to 25 days to adjust down. ## Recommendations in Azure Advisor When available, a savings plan purchase recommendation can also be found in Azure Advisor. While we may generate up to 10 recommendations, Azure Advisor only surfaces the single three-year recommendation with the greatest savings for each billing subscription. Keep the following points in mind: -- If you want to see recommendations for a one-year term or for other scopes, navigate to the savings plan purchase experience in Azure portal. For example, enrollment account, billing profile, resource groups, and so on. For more information, see [Who can buy a savings plan](buy-savings-plan.md#who-can-buy-a-savings-plan).-- Recommendations available in Advisor currently only consider your last 30 days of usage.-- Recommendations are for three-year savings plans.-- If you recently purchased a savings plan, Advisor reservation purchase and Azure saving plan recommendations can take up to five days to disappear.+- If you want to see recommendations for a one-year term or for other scopes, navigate to the savings plan purchase experience in Azure portal. For example, enrollment account, billing profile, resource groups, and so on. For more information, see [Who can buy a savings plan](buy-savings-plan.md#who-can-buy-a-savings-plan). +- Recommendations in Advisor currently only consider your last 30 days of usage. +- Recommendations in Advisor are only for three-year savings plans. +- If you recently purchased a savings plan or reserved instance, it can take up to 5 days for the purchase(s) to impact your recommendations in Advisor and Azure portal. ## Purchase recommendations in the Azure portal When available, up to 10 savings plan commitment recommendations can be found in the savings plan purchase experience in Azure portal. For more information, see [Who can buy a savings plan](buy-savings-plan.md#who-can-buy-a-savings-plan). Each recommendation includes the commitment amount, the estimated savings percentage (off your current pay-as-you-go costs) and the percentage of your compute usage costs that would be covered by this and any other previously purchased savings plans and reservations. -By default, the recommendations are for the entire billing scope (billing account or billing profile for MCA and billing account for EA). You can also view separate subscription and resource group-level recommendations by changing benefit application to one of those levels. +By default, the recommendations are for the entire billing scope (billing profile for MCA and enrollment account for EA). You can also view separate subscription and resource group-level recommendations by changing benefit application to one of those levels. We don't currently support management group-level recommendations. -Recommendations are term-specific, so you'll see the one-year or three-year recommendations at each level by toggling the term options. We don't currently support management group-level recommendations. +Recommendations are term-specific, so you'll see the one-year or three-year recommendations at each level by toggling the term options. -The highlighted recommendation is projected to result in the greatest savings. The other values allow you to see how increasing or decreasing your commitment could affect both your savings. They also show how much of your total compute usage cost would be covered by savings plans or reservation commitments. When the commitment amount is increased, your savings could be reduced because you may end up with lower utilization each hour. If you lower the commitment, your savings could also be reduced. In this case, although you'll likely have greater utilization each hour, there will likely be other hours where your savings plan won't fully cover your usage. Usage beyond your hourly commitment is charged at the more expensive pay-as-you-go rates. +The highlighted recommendation is projected to result in the greatest savings. The other values allow you to see how increasing or decreasing your commitment could affect both your savings. They also show how much of your total compute usage cost would be covered by savings plans or reservation commitments. When the commitment amount is increased, your savings may decline because you have lower utilization each hour. If you lower the commitment, your savings could also be reduced. In this case, although you'll have greater utilization, there will be more hours where your savings plan won't fully cover your usage. Usage beyond your hourly commitment is charged at the more expensive pay-as-you-go rates. ## Purchase recommendations with REST API For more information about retrieving savings plan commitment recommendations, s ## Reservation trade in recommendations -When you trade one or more reservations for a savings plan, you're shifting the balance of your previous commitments to a new savings plan commitment. For example, if you have a one-year reservation with a value of $500, and halfway through the term you look to trade it for a savings plan, you would still have an outstanding commitment of about $250. --The minimum hourly commitment must be at least equal to the outstanding amount divided by (24 times the term length in days). --As part of the trade in, the outstanding commitment is automatically included in your new savings plan. We do it by dividing the outstanding commitment by the number of hours in the term of the new savings plan. For example, 24 times the term length in days. And by making the value the minimum hourly commitment you can make during as part of the trade-in. Using the previous example, the $250 amount would be converted into an hourly commitment of about $0.029 for a new one-year savings plan. --If you're trading multiple reservations, the aggregate outstanding commitment is used. You may choose to increase the value, but you can't decrease it. The new savings plan is used to cover usage of eligible resources. +When you trade one or more reservations for a savings plan, you're shifting the balance of your previous commitments to a new savings plan commitment. For example, if you have a one-year reservation with a value of $500, and halfway through the term you look to trade it for a savings plan, you will still have an outstanding commitment of about $250. The minimum hourly commitment must be at least equal to the outstanding amount divided by (24 * the term length in days). -The minimum value doesn't necessarily represent the hourly commitment necessary to cover the resources that were covered by the exchanged reservation. If you want to cover those resources, you'll most likely have to increase the hourly commitment. To determine the appropriate hourly commitment: +As part of the trade in, the outstanding commitment is automatically included in your new savings plan. We do it by dividing the outstanding commitment by the number of hours in the term of the new savings plan. For example, 24 times the term length in days. And by making the value the minimum hourly commitment you can make during as part of the trade-in. Using the previous example, the $250 amount would be converted into an hourly commitment of about $0.029 for a new one-year savings plan. If you're trading multiple reservations, the total outstanding commitment is used. You may choose to increase the value, but you can't decrease it. -1. Download your price list. -2. For each reservation order you're returning, find the product in the price sheet and determine its unit price under either a one-year or three-year savings plan (filter by term and price type). -3. Multiply unit price by the number of instances that are being returned. The result gives you the total hourly commitment required to cover the product with your savings plan. -4. Repeat for each reservation order to be returned. -5. Sum the values and enter the total as the hourly commitment. +The minimum value doesn't necessarily represent the hourly commitment necessary to cover the resources that were covered by the exchanged reservation. If you want to cover those resources, you'll most likely have to increase the hourly commitment. To determine the appropriate hourly commitment, see [Determine savings plan commitment needed to replace your reservation](reservation-trade-in.md#determine-savings-plan-commitment-needed-to-replace-your-reservation). ## Next steps |
| cost-management-billing | Reservation Trade In | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/cost-management-billing/savings-plan/reservation-trade-in.md | Apart from [Azure Virtual Machines](https://azure.microsoft.com/pricing/details/ > > You may [trade-in](reservation-trade-in.md) your Azure compute reservations for a savings plan or may continue to use and purchase reservations for those predictable, stable workloads where the specific configuration need is known. For more information, see [Self-service exchanges and refunds for Azure Reservations](../reservations/exchange-and-refund-azure-reservations.md).ΓÇï -Although compute reservation exchanges become unavailable at the end of the grace period, noncompute reservation exchanges are unchanged. You're able to continue to trade-in reservations for saving plans.ΓÇï +Although compute reservation exchanges become unavailable at the end of the grace period, noncompute reservation exchanges are unchanged. You're able to continue to trade-in reservations for saving plans.ΓÇï To trade-in reservation(s) for a savings plan, you must meet the following criteria: - You must have owner access on the Reservation Order to trade in an existing reservation. You can [Add or change users who can manage a savings plan](manage-savings-plan.md#who-can-manage-a-savings-plan).-- To trade-in a reservation for a savings plan, you must have Azure RBAC Owner permission on the subscription you plan to use to purchase a savings plan.+- You must have the Savings plan purchaser role, or Owner permission on the subscription you plan to use to purchase the savings plan. - EA Admin write permission or Billing profile contributor and higher, which are Cost Management + Billing permissions, are supported only for direct Savings plan purchases. They can't be used for savings plans purchases as a part of a reservation trade-in.-- The new savings plan's lifetime commitment should equal or be greater than the returned reservation's remaining commitment. Example: for a three-year reservation that's $100 per month and exchanged after the 18th payment, the new savings plan's lifetime commitment should be $1,800 or more (paid monthly or upfront).-- Microsoft isn't currently charging early termination fees for reservation trade ins. We might charge the fees made in the future. We currently don't have a date for enabling the fee.++The new savings plan's lifetime commitment must equal or be greater than the returned reservation(s)'s remaining commitment. Example: for a three-year reservation that's $100 per month and exchanged after the 18th payment, the new savings plan's lifetime commitment should be $1,800 or more (paid monthly or upfront). ++Microsoft isn't currently charging early termination fees for reservation trade ins. We might charge the fees made in the future. We currently don't have a date for enabling the fee. ## How to trade in an existing reservation |
| data-factory | Control Flow Expression Language Functions | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/data-factory/control-flow-expression-language-functions.md | These functions are useful inside conditions, they can be used to evaluate any t | Math function | Task | | - | - | | [add](control-flow-expression-language-functions.md#add) | Return the result from adding two numbers. |-| [div](control-flow-expression-language-functions.md#div) | Return the result from dividing two numbers. | +| [div](control-flow-expression-language-functions.md#div) | Return the result from dividing one number by another number. | | [max](control-flow-expression-language-functions.md#max) | Return the highest value from a set of numbers or an array. | | [min](control-flow-expression-language-functions.md#min) | Return the lowest value from a set of numbers or an array. |-| [mod](control-flow-expression-language-functions.md#mod) | Return the remainder from dividing two numbers. | +| [mod](control-flow-expression-language-functions.md#mod) | Return the remainder from dividing one number by another number. | | [mul](control-flow-expression-language-functions.md#mul) | Return the product from multiplying two numbers. | | [rand](control-flow-expression-language-functions.md#rand) | Return a random integer from a specified range. | | [range](control-flow-expression-language-functions.md#range) | Return an integer array that starts from a specified integer. |-| [sub](control-flow-expression-language-functions.md#sub) | Return the result from subtracting the second number from the first number. | +| [sub](control-flow-expression-language-functions.md#sub) | Return the result from subtracting one number from another number. | ## Function reference And returns this result: `"https://contoso.com"` ### div -Return the integer result from dividing two numbers. -To get the remainder result, see [mod()](#mod). +Return the result of dividing one number by another number. ``` div(<dividend>, <divisor>) ``` +The precise return type of the function depends on the types of its parameters — see examples for detail. + | Parameter | Required | Type | Description | | | -- | - | -- | | <*dividend*> | Yes | Integer or Float | The number to divide by the *divisor* |-| <*divisor*> | Yes | Integer or Float | The number that divides the *dividend*, but cannot be 0 | +| <*divisor*> | Yes | Integer or Float | The number that divides the *dividend*. A *divisor* value of zero causes an error at runtime. | ||||| | Return value | Type | Description | | | - | -- |-| <*quotient-result*> | Integer | The integer result from dividing the first number by the second number | +| <*quotient-result*> | Integer or Float | The result of dividing the first number by the second number | |||| -*Example* +*Example 1* -Both examples divide the first number by the second number: +These examples divide the number 9 by 2: ```-div(10, 5) -div(11, 5) +div(9, 2.0) +div(9.0, 2) +div(9.0, 2.0) ``` -And return this result: `2` +And all return this result: `4.5` ++*Example 2* ++This example also divides the number 9 by 2, but because both parameters are integers the remainder is discarded (integer division): ++``` +div(9, 2) +``` ++The expression returns the result `4`. To obtain the value of the remainder, use the [mod()](#mod) function. <a name="encodeUriComponent"></a> And return this result: `1` ### mod -Return the remainder from dividing two numbers. -To get the integer result, see [div()](#div). +Return the remainder from dividing one number by another number. For integer division, see [div()](#div). ``` mod(<dividend>, <divisor>) mod(<dividend>, <divisor>) | Parameter | Required | Type | Description | | | -- | - | -- | | <*dividend*> | Yes | Integer or Float | The number to divide by the *divisor* |-| <*divisor*> | Yes | Integer or Float | The number that divides the *dividend*, but cannot be 0. | +| <*divisor*> | Yes | Integer or Float | The number that divides the *dividend*. A *divisor* value of zero causes an error at runtime. | ||||| | Return value | Type | Description | mod(<dividend>, <divisor>) *Example* -This example divides the first number by the second number: +This example calculates the remainder when the first number is divided by the second number: ``` mod(3, 2) ``` -And return this result: `1` +And returns this result: `1` <a name="mul"></a> mul(<multiplicand1>, <multiplicand2>) | Parameter | Required | Type | Description | | | -- | - | -- | | <*multiplicand1*> | Yes | Integer or Float | The number to multiply by *multiplicand2* |-| <*multiplicand2*> | Yes | Integer or Float | The number that multiples *multiplicand1* | +| <*multiplicand2*> | Yes | Integer or Float | The number that multiplies *multiplicand1* | ||||| | Return value | Type | Description | mul(<multiplicand1>, <multiplicand2>) *Example* -These examples multiple the first number by the second number: +These examples multiply the first number by the second number: ``` mul(1, 2) And returns this result: `"{ \\"name\\": \\"Sophie Owen\\" }"` ### sub -Return the result from subtracting the second number from the first number. +Return the result from subtracting one number from another number. ``` sub(<minuend>, <subtrahend>) |
| databox-online | Azure Stack Edge Gpu 2403 Release Notes | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/databox-online/azure-stack-edge-gpu-2403-release-notes.md | + + Title: Azure Stack Edge 2403 release notes +description: Describes critical open issues and resolutions for the Azure Stack Edge running 2403 release. +++ +++ Last updated : 04/03/2024++++# Azure Stack Edge 2403 release notes +++The following release notes identify critical open issues and resolved issues for the 2403 release for your Azure Stack Edge devices. Features and issues that correspond to a specific model of Azure Stack Edge are called out wherever applicable. ++The release notes are continuously updated, and as critical issues requiring a workaround are discovered, they're added. Before you deploy your device, carefully review the information contained in the release notes. ++This article applies to the **Azure Stack Edge 2403** release, which maps to software version **3.2.2642.2453**. ++> [!Warning] +> In this release, you must update the packet core version to AP5GC 2308 before you update to Azure Stack Edge 2403. For detailed steps, see [Azure Private 5G Core 2308 release notes](../private-5g-core/azure-private-5g-core-release-notes-2308.md). +> If you update to Azure Stack Edge 2403 before updating to Packet Core 2308.0.1, you will experience a total system outage. In this case, you must delete and re-create the Azure Kubernetes service cluster on your Azure Stack Edge device. +> Each time you change the Kubernetes workload profile, you are prompted for the Kubernetes update. Go ahead and apply the update. ++## Supported update paths ++To apply the 2403 update, your device must be running version 2303 or later. ++ - If you aren't running the minimum required version, you see this error: ++ *Update package can't be installed as its dependencies aren't met.* ++ - You can update to 2303 from 2207 or later, and then update to 2403. ++You can update to the latest version using the following update paths: ++| Current version of Azure Stack Edge software and Kubernetes | Update to Azure Stack Edge software and Kubernetes | Desired update to 2403 | +| --| --| --| +|2207 |2303 |2403 | +|2209 |2303 |2403 | +|2210 |2303 |2403 | +|2301 |2303 |2403 | +|2303 |Directly to |2403 | ++## What's new ++The 2403 release has the following new features and enhancements: ++- Deprecated support for Azure Kubernetes service telemetry on Azure Stack Edge. +- Zone-label support for two-node Kubernetes clusters. +- Hyper-V VM management, memory usage monitoring on Azure Stack Edge host. ++## Issues fixed in this release ++| No. | Feature | Issue | +| | | | +|**1.**| Clustering | Two-node cold boot of the server causes high availability VM cluster resources to come up as offline. Changed ColdStartSetting to AlwaysStart. | +|**2.**| Marketplace image support | Fixed bug allowing Windows Marketplace image on Azure Stack Edge A and TMA. | +|**3.**| Network connectivity | Fixed VM NIC link flapping after Azure Stack Edge host power off/on, which can cause VM losing its DHCP IP. | +|**4.**| Network connectivity |Due to proxy ARP configurations in some customer environments, **IP address in use** check returns false positive even though no endpoint in the network is using the IP. The fix skips the ARP-based VM **IP address in use** check if the IP address is allocated from an internal network managed by Azure Stack Edge. | +|**5.**| Network connectivity | VM NIC change operation times out after 3 hours, which blocks other VM update operations. On Microsoft Kubernetes clusters, Persistent Volume (PV) dependent pods get stuck. The issue occurs when multiple NICs within a VM are being transferred from a VLAN virtual network to a non-VLAN virtual network. After the fix, the VM NIC change operation times out quickly and the VM update won't be blocked. | +|**6.**| Kubernetes | Overall two-node Kubernetes resiliency improvements, like increasing memory for control plane for AKS workload cluster, increasing limits for etcd, multi-replica, and hard anti-affinity support for core DNS and Azure disk csi controller pods and improve VM failover times. | +|**7.**| Compute Diagnostic and Update | Resiliency fixes | +|**8.**| Security | STIG security fixes for Mariner Guest OS for Azure Kubernetes service on Azure Stack Edge. | +|**9.**| VM operations | On an Azure Stack Edge cluster that deploys an AP5GC workload, after a host power cycle test, when the host returns a transient error about CPU group configuration, AzSHostAgent would crash. This caused a VM operations failure. The fix made *AzSHostAgent* resilient to a transient CPU group error. | ++<!--!## Known issues in this release ++| No. | Feature | Issue | Workaround/comments | +| | | | | +|**1.**|AKS... |The AKS Kubernetes... | +|**2.**|Wi-Fi... |Starting this release... | |--> ++## Known issues in this release ++| No. | Feature | Issue | Workaround/comments | +| | | | | +|**1.**| Azure Storage Explorer | The Blob storage endpoint certificate that's autogenerated by the Azure Stack Edge device might not work properly with Azure Storage Explorer. | Replace the Blob storage endpoint certificate. For detailed steps, see [Bring your own certificates](azure-stack-edge-gpu-deploy-configure-certificates.md#bring-your-own-certificates). | +|**2.**| Network connectivity | On a two-node Azure Stack Edge Pro 2 cluster with a teamed virtual switch for Port 1 and Port 2, if a Port 1 or Port 2 link is down, it can take up to 5 seconds to resume network connectivity on the remaining active port. If a Kubernetes cluster uses this teamed virtual switch for management traffic, pod communication may be disrupted up to 5 seconds. | | +|**3.**| Virtual machine | After the host or Kubernetes node pool VM is shut down, there's a chance that kubelet in node pool VM fails to start due to a CPU static policy error. Node pool VM shows **Not ready** status, and pods won't be scheduled on this VM. | Enter a support session and ssh into the node pool VM, then follow steps in [Changing the CPU Manager Policy](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/#changing-the-cpu-manager-policy) to remediate the kubelet service. | + +## Known issues from previous releases ++The following table provides a summary of known issues carried over from the previous releases. ++| No. | Feature | Issue | Workaround/comments | +| | | | | +| **1.** |Azure Stack Edge Pro + Azure SQL | Creating SQL database requires Administrator access. |Do the following steps instead of Steps 1-2 in [Create-the-sql-database](../iot-edge/tutorial-store-data-sql-server.md#create-the-sql-database). <br> 1. In the local UI of your device, enable compute interface. Select **Compute > Port # > Enable for compute > Apply.**<br> 2. Download `sqlcmd` on your client machine from [SQL command utility](/sql/tools/sqlcmd-utility). <br> 3. Connect to your compute interface IP address (the port that was enabled), adding a ",1401" to the end of the address.<br> 4. Final command looks like this: sqlcmd -S {Interface IP},1401 -U SA -P "Strong!Passw0rd". After this, steps 3-4 from the current documentation should be identical. | +| **2.** |Refresh| Incremental changes to blobs restored via **Refresh** are NOT supported |For Blob endpoints, partial updates of blobs after a Refresh, might result in the updates not getting uploaded to the cloud. For example, sequence of actions such as:<br> 1. Create blob in cloud. Or delete a previously uploaded blob from the device.<br> 2. Refresh blob from the cloud into the appliance using the refresh functionality.<br> 3. Update only a portion of the blob using Azure SDK REST APIs. These actions can result in the updated sections of the blob to not get updated in the cloud. <br>**Workaround**: Use tools such as robocopy, or regular file copy through Explorer or command line, to replace entire blobs.| +|**3.**|Throttling|During throttling, if new writes to the device aren't allowed, writes by the NFS client fail with a "Permission Denied" error.| The error shows as below:<br>`hcsuser@ubuntu-vm:~/nfstest$ mkdir test`<br>mkdir: can't create directory 'test': Permission deniedΓÇï| +|**4.**|Blob Storage ingestion|When using AzCopy version 10 for Blob storage ingestion, run AzCopy with the following argument: `Azcopy <other arguments> --cap-mbps 2000`| If these limits aren't provided for AzCopy, it could potentially send a large number of requests to the device, resulting in issues with the service.| +|**5.**|Tiered storage accounts|The following apply when using tiered storage accounts:<br> - Only block blobs are supported. Page blobs aren't supported.<br> - There's no snapshot or copy API support.<br> - Hadoop workload ingestion through `distcp` isn't supported as it uses the copy operation heavily.|| +|**6.**|NFS share connection|If multiple processes are copying to the same share, and the `nolock` attribute isn't used, you might see errors during the copy.ΓÇï|The `nolock` attribute must be passed to the mount command to copy files to the NFS share. For example: `C:\Users\aseuser mount -o anon \\10.1.1.211\mnt\vms Z:`.| +|**7.**|Kubernetes cluster|When applying an update on your device that is running a Kubernetes cluster, the Kubernetes virtual machines will restart and reboot. In this instance, only pods that are deployed with replicas specified are automatically restored after an update. |If you have created individual pods outside a replication controller without specifying a replica set, these pods won't be restored automatically after the device update. You must restore these pods.<br>A replica set replaces pods that are deleted or terminated for any reason, such as node failure or disruptive node upgrade. For this reason, we recommend that you use a replica set even if your application requires only a single pod.| +|**8.**|Kubernetes cluster|Kubernetes on Azure Stack Edge Pro is supported only with Helm v3 or later. For more information, go to [Frequently asked questions: Removal of Tiller](https://v3.helm.sh/docs/faq/).| +|**9.**|Kubernetes |Port 31000 is reserved for Kubernetes Dashboard. Port 31001 is reserved for Edge container registry. Similarly, in the default configuration, the IP addresses 172.28.0.1 and 172.28.0.10, are reserved for Kubernetes service and Core DNS service respectively.|Don't use reserved IPs.| +|**10.**|Kubernetes |Kubernetes doesn't currently allow multi-protocol LoadBalancer services. For example, a DNS service that would have to listen on both TCP and UDP. |To work around this limitation of Kubernetes with MetalLB, two services (one for TCP, one for UDP) can be created on the same pod selector. These services use the same sharing key and spec.loadBalancerIP to share the same IP address. IPs can also be shared if you have more services than available IP addresses. <br> For more information, see [IP address sharing](https://metallb.universe.tf/usage/#ip-address-sharing).| +|**11.**|Kubernetes cluster|Existing Azure IoT Edge marketplace modules might require modifications to run on IoT Edge on Azure Stack Edge device.|For more information, see [Run existing IoT Edge modules from Azure Stack Edge Pro FPGA devices on Azure Stack Edge Pro GPU device](azure-stack-edge-gpu-modify-fpga-modules-gpu.md).| +|**12.**|Kubernetes |File-based bind mounts aren't supported with Azure IoT Edge on Kubernetes on Azure Stack Edge device.|IoT Edge uses a translation layer to translate `ContainerCreate` options to Kubernetes constructs. Creating `Binds` maps to `hostpath` directory and thus file-based bind mounts can't be bound to paths in IoT Edge containers. If possible, map the parent directory.| +|**13.**|Kubernetes |If you bring your own certificates for IoT Edge and add those certificates on your Azure Stack Edge device after the compute is configured on the device, the new certificates aren't picked up.|To work around this problem, you should upload the certificates before you configure compute on the device. If the compute is already configured, [Connect to the PowerShell interface of the device and run IoT Edge commands](azure-stack-edge-gpu-connect-powershell-interface.md#use-iotedge-commands). Restart `iotedged` and `edgehub` pods.| +|**14.**|Certificates |In certain instances, certificate state in the local UI might take several seconds to update. |The following scenarios in the local UI might be affected. <br> - **Status** column in **Certificates** page. <br> - **Security** tile in **Get started** page. <br> - **Configuration** tile in **Overview** page.<br> | +|**15.**|Certificates|Alerts related to signing chain certificates aren't removed from the portal even after uploading new signing chain certificates.| | +|**16.**|Web proxy |NTLM authentication-based web proxy isn't supported. || +|**17.**|Internet Explorer|If enhanced security features are enabled, you might not be able to access local web UI pages. | Disable enhanced security, and restart your browser.| +|**18.**|Kubernetes |Kubernetes doesn't support ":" in environment variable names that are used by .NET applications. This is also required for Event Grid IoT Edge module to function on Azure Stack Edge device and other applications. For more information, see [ASP.NET core documentation](/aspnet/core/fundamentals/configuration/?tabs=basicconfiguration#environment-variables).|Replace ":" by double underscore. For more information, see [Kubernetes issue](https://github.com/kubernetes/kubernetes/issues/53201)| +|**19.** |Azure Arc + Kubernetes cluster |By default, when resource `yamls` are deleted from the Git repository, the corresponding resources aren't deleted from the Kubernetes cluster. |To allow the deletion of resources when they're deleted from the git repository, set `--sync-garbage-collection` in Arc OperatorParams. For more information, see [Delete a configuration](../azure-arc/kubernetes/tutorial-use-gitops-connected-cluster.md#additional-parameters). | +|**20.**|NFS |Applications that use NFS share mounts on your device to write data should use Exclusive write. That ensures the writes are written to the disk.| | +|**21.**|Compute configuration |Compute configuration fails in network configurations where gateways or switches or routers respond to Address Resolution Protocol (ARP) requests for systems that don't exist on the network.| | +|**22.**|Compute and Kubernetes |If Kubernetes is set up first on your device, it claims all the available GPUs. Hence, it isn't possible to create Azure Resource Manager VMs using GPUs after setting up the Kubernetes. |If your device has 2 GPUs, then you can create one VM that uses the GPU and then configure Kubernetes. In this case, Kubernetes will use the remaining available one GPU. | +|**23.**|Custom script VM extension |There's a known issue in the Windows VMs that were created in an earlier release and the device was updated to 2103. <br> If you add a custom script extension on these VMs, the Windows VM Guest Agent (Version 2.7.41491.901 only) gets stuck in the update causing the extension deployment to time out. | To work around this issue: <br> 1. Connect to the Windows VM using remote desktop protocol (RDP). <br> 2. Make sure that the `waappagent.exe` is running on the machine: `Get-Process WaAppAgent`. <br> 3. If the `waappagent.exe` isn't running, restart the `rdagent` service: `Get-Service RdAgent` \| `Restart-Service`. Wait for 5 minutes.<br> 4. While the `waappagent.exe` is running, kill the `WindowsAzureGuest.exe` process. <br> 5. After you kill the process, the process starts running again with the newer version. <br> 6. Verify that the Windows VM Guest Agent version is 2.7.41491.971 using this command: `Get-Process WindowsAzureGuestAgent` \| `fl ProductVersion`.<br> 7. [Set up custom script extension on Windows VM](azure-stack-edge-gpu-deploy-virtual-machine-custom-script-extension.md). | +|**24.**|Multi-Process Service (MPS) |When the device software and the Kubernetes cluster are updated, the MPS setting isn't retained for the workloads. |[Re-enable MPS](azure-stack-edge-gpu-connect-powershell-interface.md#connect-to-the-powershell-interface) and redeploy the workloads that were using MPS. | +|**25.**|Wi-Fi |Wi-Fi doesn't work on Azure Stack Edge Pro 2 in this release. | +|**26.**|Azure IoT Edge |The managed Azure IoT Edge solution on Azure Stack Edge is running on an older, obsolete IoT Edge runtime that is at end of life. For more information, see [IoT Edge v1.1 EoL: What does that mean for me?](https://techcommunity.microsoft.com/t5/internet-of-things-blog/iot-edge-v1-1-eol-what-does-that-mean-for-me/ba-p/3662137). Although the solution doesn't stop working past end of life, there are no plans to update it. |To run the latest version of Azure IoT Edge [LTSs](../iot-edge/version-history.md#version-history) with the latest updates and features on their Azure Stack Edge, we **recommend** that you deploy a [customer self-managed IoT Edge solution](azure-stack-edge-gpu-deploy-iot-edge-linux-vm.md) that runs on a Linux VM. For more information, see [Move workloads from managed IoT Edge on Azure Stack Edge to an IoT Edge solution on a Linux VM](azure-stack-edge-move-to-self-service-iot-edge.md). | +|**27.**|AKS on Azure Stack Edge |In this release, you can't modify the virtual networks once the AKS cluster is deployed on your Azure Stack Edge cluster.| To modify the virtual network, you must delete the AKS cluster, then modify virtual networks, and then recreate AKS cluster on your Azure Stack Edge. | +|**28.**|AKS Update |The AKS Kubernetes update might fail if one of the AKS VMs isn't running. This issue might be seen in the two-node cluster. |If the AKS update has failed, [Connect to the PowerShell interface of the device](azure-stack-edge-gpu-connect-powershell-interface.md). Check the state of the Kubernetes VMs by running `Get-VM` cmdlet. If the VM is off, run the `Start-VM` cmdlet to restart the VM. Once the Kubernetes VM is running, reapply the update. | +|**29.**|Wi-Fi |Wi-Fi functionality for Azure Stack Edge Mini R is deprecated. | | ++## Next steps ++- [Update your device](azure-stack-edge-gpu-install-update.md). |
| databox-online | Azure Stack Edge Gpu Deploy Configure Compute | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/databox-online/azure-stack-edge-gpu-deploy-configure-compute.md | |
| databox-online | Azure Stack Edge Gpu Install Update | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/databox-online/azure-stack-edge-gpu-install-update.md | -This article describes the steps required to install update on your Azure Stack Edge Pro with GPU via the local web UI and via the Azure portal. You apply the software updates or hotfixes to keep your Azure Stack Edge Pro device and the associated Kubernetes cluster on the device up-to-date. +This article describes the steps required to install update on your Azure Stack Edge Pro device with GPU via the local web UI and via Azure portal. ++Apply the software updates or hotfixes to keep your Azure Stack Edge Pro device and the associated Kubernetes cluster on the device up-to-date. > [!NOTE] > The procedure described in this article was performed using a different version of software, but the process remains the same for the current software version. ## About latest updates -The current update is Update 2312. This update installs two updates, the device update followed by Kubernetes updates. +The current version is Update 2403. This update installs two updates, the device update followed by Kubernetes updates. The associated versions for this update are: -- Device software version: Azure Stack Edge 2312 (3.2.2510.2000)-- Device Kubernetes version: Azure Stack Kubernetes Edge 2312 (3.2.2510.2000)-- Device Kubernetes workload profile: Other workloads-- Kubernetes server version: v1.26.3-- IoT Edge version: 0.1.0-beta15-- Azure Arc version: 1.13.4-- GPU driver version: 535.104.05-- CUDA version: 12.2+- Device software version: Azure Stack Edge 2403 (3.2.2642.2453). +- Device Kubernetes version: Azure Stack Kubernetes Edge 2403 (3.2.2642.2453). +- Device Kubernetes workload profile: Azure Private MEC. +- Kubernetes server version: v1.27.8. +- IoT Edge version: 0.1.0-beta15. +- Azure Arc version: 1.14.5. +- GPU driver version: 525.85.12. +- CUDA version: 12.0. -For information on what's new in this update, go to [Release notes](azure-stack-edge-gpu-2312-release-notes.md). +For information on what's new in this update, go to [Release notes](azure-stack-edge-gpu-2403-release-notes.md). -**To apply the 2312 update, your device must be running version 2203 or later.** +**To apply the 2403 update, your device must be running version 2203 or later.** -- If you are not running the minimum required version, you'll see this error:+- If you aren't running the minimum required version, you see this error: - *Update package cannot be installed as its dependencies are not met.* + *Update package can't be installed as its dependencies aren't met.* -- You can update to 2303 from 2207 or later, and then install 2312.+- You can update to 2303 from 2207 or later, and then install 2403. Supported update paths: -| Current version of Azure Stack Edge software and Kubernetes | Upgrade to Azure Stack Edge software and Kubernetes | Desired update to 2312 | +| Current version of Azure Stack Edge software and Kubernetes | Upgrade to Azure Stack Edge software and Kubernetes | Desired update to 2403 | |-|-| |-| 2207 | 2303 | 2312 | -| 2209 | 2303 | 2312 | -| 2210 | 2303 | 2312 | -| 2301 | 2303 | 2312 | -| 2303 | Directly to | 2312 | +| 2207 | 2303 | 2403 | +| 2209 | 2303 | 2403 | +| 2210 | 2303 | 2403 | +| 2301 | 2303 | 2403 | +| 2303 | Directly to | 2403 | ### Update Azure Kubernetes service on Azure Stack Edge > [!IMPORTANT] > Use the following procedure only if you are an SAP or a PMEC customer. -If you have Azure Kubernetes service deployed and your Azure Stack Edge device and Kubernetes versions are either 2207 or 2209, you must update in multiple steps to apply 2312. +If you have Azure Kubernetes service deployed and your Azure Stack Edge device and Kubernetes versions are either 2207 or 2209, you must update in multiple steps to apply 2403. -Use the following steps to update your Azure Stack Edge version and Kubernetes version to 2312: +Use the following steps to update your Azure Stack Edge version and Kubernetes version to 2403: 1. Update your device version to 2303. 1. Update your Kubernetes version to 2210. 1. Update your Kubernetes version to 2303.-1. Update both device software and Kubernetes to 2312. +1. Update both device software and Kubernetes to 2403. -If you are running 2210 or 2301, you can update both your device version and Kubernetes version directly to 2303 and then to 2312. +If you're running 2210 or 2301, you can update both your device version and Kubernetes version directly to 2303 and then to 2403. -If you are running 2303, you can update both your device version and Kubernetes version directly to 2312. +If you're running 2303, you can update both your device version and Kubernetes version directly to 2403. -In Azure portal, the process will require two clicks, the first update gets your device version to 2303 and your Kubernetes version to 2210, and the second update gets your Kubernetes version upgraded to 2312. +In Azure portal, the process requires two clicks, the first update gets your device version to 2303 and your Kubernetes version to 2210, and the second update gets your Kubernetes version upgraded to 2403. -From the local UI, you will have to run each update separately: update the device version to 2303, update Kubernetes version to 2210, update Kubernetes version to 2303, and then the third update gets both the device version and Kubernetes version to 2312. +From the local UI, you'll have to run each update separately: update the device version to 2303, update Kubernetes version to 2210, update Kubernetes version to 2303, and then the third update gets both the device version and Kubernetes version to 2403. -Each time you change the Kubernetes profile, you are prompted for the Kubernetes update. Go ahead and apply the update. +Each time you change the Kubernetes profile, you're prompted for the Kubernetes update. Go ahead and apply the update. ### Updates for a single-node vs two-node -The procedure to update an Azure Stack Edge is the same whether it is a single-node device or a two-node cluster. This applies both to the Azure portal or the local UI procedure. +The procedure to update an Azure Stack Edge is the same whether it's a single-node device or a two-node cluster. This applies both to the Azure portal or the local UI procedure. -- **Single node** - For a single node device, installing an update or hotfix is disruptive and will restart your device. Your device will experience a downtime for the entire duration of the update.+- **Single node** - For a single node device, installing an update or hotfix is disruptive and restarts your device. Your device will experience a downtime for the entire duration of the update. -- **Two-node** - For a two-node cluster, this is an optimized update. The two-node cluster might experience short, intermittent disruptions while the update is in progress. We recommend that you shouldn't perform any operations on the device node when update is in progress. +- **Two-node** - For a two-node cluster, this is an optimized update. The two-node cluster might experience short, intermittent disruptions while the update is in progress. We recommend that you shouldn't perform any operations on the device node when an update is in progress. - The Kubernetes worker VMs will go down when a node goes down. The Kubernetes master VM will fail over to the other node. Workloads will continue to run. For more information, see [Kubernetes failover scenarios for Azure Stack Edge](azure-stack-edge-gpu-kubernetes-failover-scenarios.md). + The Kubernetes worker VMs goes down when a node goes down. The Kubernetes master VM fails over to the other node. Workloads continue to run. For more information, see [Kubernetes failover scenarios for Azure Stack Edge](azure-stack-edge-gpu-kubernetes-failover-scenarios.md). -Provisioning actions such as creating shares or virtual machines are not supported during update. The update takes about 60 to 75 minutes per node to complete. +Provisioning actions such as creating shares or virtual machines aren't supported during update. The update takes about 60 to 75 minutes per node to complete. To install updates on your device, follow these steps: Each of these steps is described in the following sections. 2. In **Select update server type**, from the dropdown list, choose from Microsoft Update server (default) or Windows Server Update Services. - If updating from the Windows Server Update Services, specify the server URI. The server at that URI will deploy the updates on all the devices connected to this server. + If updating from the Windows Server Update Services, specify the server URI. The server at that URI deploys the updates on all the devices connected to this server. <!----> Each of these steps is described in the following sections. ## Use the Azure portal -We recommend that you install updates through the Azure portal. The device automatically scans for updates once a day. Once the updates are available, you see a notification in the portal. You can then download and install the updates. +We recommend that you install updates through Azure portal. The device automatically scans for updates once a day. Once the updates are available, you see a notification in the portal. You can then download and install the updates. > [!NOTE] > - Make sure that the device is healthy and status shows as **Your device is running fine!** before you proceed to install the updates. +Depending on the software version that you're running, install process might differ slightly. -Depending on the software version that you are running, install process might differ slightly. --- If you are updating from 2106 to 2110 or later, you will have a one-click install. See the **version 2106 and later** tab for instructions.-- If you are updating to versions prior to 2110, you will have a two-click install. See **version 2105 and earlier** tab for instructions.+- If you're updating from 2106 to 2110 or later, you'll have a one-click install. See the **version 2106 and later** tab for instructions. +- If you're updating to versions prior to 2110, you'll have a two-click install. See **version 2105 and earlier** tab for instructions. ### [version 2106 and later](#tab/version-2106-and-later) Depending on the software version that you are running, install process might di ### [version 2105 and earlier](#tab/version-2105-and-earlier) -1. When the updates are available for your device, you see a notification in the **Overview** page of your Azure Stack Edge resource. Select the notification or from the top command bar, **Update device**. This will allow you to apply device software updates. +1. When the updates are available for your device, you see a notification in the **Overview** page of your Azure Stack Edge resource. Select the notification or from the top command bar, **Update device**. This allows you to apply device software updates.  Depending on the software version that you are running, install process might di  -4. After the download is complete, the notification banner updates to indicate the completion. If you chose to download and install the updates, the installation will begin automatically. +4. After the download is complete, the notification banner updates to indicate the completion. If you chose to download and install the updates, the installation begins automatically. If you chose to download updates only, then select the notification to open the **Device updates** blade. Select **Install**. Depending on the software version that you are running, install process might di  -7. After the restart, the device software will finish updating. After the update is complete, you can verify from the local web UI that the device software is updated. The Kubernetes software version has not been updated. +7. After the restart, the device software will finish updating. After the update is complete, you can verify from the local web UI that the device software is updated. The Kubernetes software version hasn't been updated.  -8. You will see a notification banner indicating that device updates are available. Select this banner to start updating the Kubernetes software on your device. +8. You'll see a notification banner indicating that device updates are available. Select this banner to start updating the Kubernetes software on your device.  Do the following steps to download the update from the Microsoft Update Catalog.  -1. In the search box of the Microsoft Update Catalog, enter the Knowledge Base (KB) number of the hotfix or terms for the update you want to download. For example, enter **Azure Stack Edge**, and then click **Search**. +1. In the search box of the Microsoft Update Catalog, enter the Knowledge Base (KB) number of the hotfix or terms for the update you want to download. For example, enter **Azure Stack Edge**, and then select **Search**. - The update listing appears as **Azure Stack Edge Update 2312**. + The update listing appears as **Azure Stack Edge Update 2403**. > [!NOTE] > Make sure to verify which workload you are running on your device [via the local UI](./azure-stack-edge-gpu-deploy-configure-network-compute-web-proxy.md#configure-compute-ips-1) or [via the PowerShell](./azure-stack-edge-connect-powershell-interface.md) interface of the device. Depending on the workload that you are running, the update package will differ. Do the following steps to download the update from the Microsoft Update Catalog. | Kubernetes | Local UI Kubernetes workload profile | Update package name | Example Update File | ||--||--|- | Azure Kubernetes Service | Azure Private MEC Solution in your environment<br><br>SAP Digital Manufacturing for Edge Computing or another Microsoft Partner Solution in your Environment | Azure Stack Edge Update 2312 Kubernetes Package for Private MEC/SAP Workloads | release~ase-2307d.3.2.2380.1632-42623-79365624-release_host_MsKubernetes_Package | - | Kubernetes for Azure Stack Edge |Other workloads in your environment | Azure Stack Edge Update 2312 Kubernetes Package for Non Private MEC/Non SAP Workloads | \release~ase-2307d.3.2.2380.1632-42623-79365624-release_host_AseKubernetes_Package | + | Azure Kubernetes Service | Azure Private MEC Solution in your environment<br><br>SAP Digital Manufacturing for Edge Computing or another Microsoft Partner Solution in your Environment | Azure Stack Edge Update 2403 Kubernetes Package for Private MEC/SAP Workloads | release~ase-2307d.3.2.2380.1632-42623-79365624-release_host_MsKubernetes_Package | + | Kubernetes for Azure Stack Edge |Other workloads in your environment | Azure Stack Edge Update 2403 Kubernetes Package for Non Private MEC/Non SAP Workloads | \release~ase-2307d.3.2.2380.1632-42623-79365624-release_host_AseKubernetes_Package | -1. Select **Download**. There are two packages to download for the update. The first package will have two files for the device software updates (*SoftwareUpdatePackage.0.exe*, *SoftwareUpdatePackage.1.exe*) and the second package has two files for the Kubernetes updates (*Kubernetes_Package.0.exe* and *Kubernetes_Package.1.exe*), respectively. Download the packages to a folder on the local system. You can also copy the folder to a network share that is reachable from the device. +1. Select **Download**. There are two packages to download for the update. The first package has two files for the device software updates (*SoftwareUpdatePackage.0.exe*, *SoftwareUpdatePackage.1.exe*) and the second package has two files for the Kubernetes updates (*Kubernetes_Package.0.exe* and *Kubernetes_Package.1.exe*), respectively. Download the packages to a folder on the local system. You can also copy the folder to a network share that is reachable from the device. ### Install the update or the hotfix Prior to the update or hotfix installation, make sure that: This procedure takes around 20 minutes to complete. Perform the following steps to install the update or hotfix. -1. In the local web UI, go to **Maintenance** > **Software update**. Make a note of the software version that you are running. +1. In the local web UI, go to **Maintenance** > **Software update**. Make a note of the software version that you're running. 2. Provide the path to the update file. You can also browse to the update installation file if placed on a network share. Select the two software files (with *SoftwareUpdatePackage.0.exe* and *SoftwareUpdatePackage.1.exe* suffix) together. This procedure takes around 20 minutes to complete. Perform the following steps <!----> -4. When prompted for confirmation, select **Yes** to proceed. Given the device is a single node device, after the update is applied, the device restarts and there is downtime. +4. When prompted for confirmation, select **Yes** to proceed. Given the device is a single node device, after the update is applied, the device restarts and there's downtime.  -5. The update starts. After the device is successfully updated, it restarts. The local UI is not accessible in this duration. +5. The update starts. After the device is successfully updated, it restarts. The local UI isn't accessible in this duration. -6. After the restart is complete, you are taken to the **Sign in** page. To verify that the device software has been updated, in the local web UI, go to **Maintenance** > **Software update**. For the current release, the displayed software version should be **Azure Stack Edge 2312**. +6. After the restart is complete, you're taken to the **Sign in** page. To verify that the device software has been updated, in the local web UI, go to **Maintenance** > **Software update**. For the current release, the displayed software version should be **Azure Stack Edge 2403**. -7. You will now update the Kubernetes software version. Select the remaining two Kubernetes files together (file with the *Kubernetes_Package.0.exe* and *Kubernetes_Package.1.exe* suffix) and repeat the above steps to apply update. +7. You'll now update the Kubernetes software version. Select the remaining two Kubernetes files together (file with the *Kubernetes_Package.0.exe* and *Kubernetes_Package.1.exe* suffix) and repeat the above steps to apply update. <!----> This procedure takes around 20 minutes to complete. Perform the following steps 9. When prompted for confirmation, select **Yes** to proceed. -10. After the Kubernetes update is successfully installed, there is no change to the displayed software in **Maintenance** > **Software update**. +10. After the Kubernetes update is successfully installed, there's no change to the displayed software in **Maintenance** > **Software update**.  ## Next steps -Learn more about [administering your Azure Stack Edge Pro](azure-stack-edge-manage-access-power-connectivity-mode.md). +- Learn more about [administering your Azure Stack Edge Pro](azure-stack-edge-manage-access-power-connectivity-mode.md). |
| databox-online | Azure Stack Edge Gpu Kubernetes Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/databox-online/azure-stack-edge-gpu-kubernetes-overview.md | Once the Kubernetes cluster is deployed, then you can manage the applications de For more information on deploying Kubernetes cluster, go to [Deploy a Kubernetes cluster on your Azure Stack Edge device](azure-stack-edge-gpu-create-kubernetes-cluster.md). For information on management, go to [Use kubectl to manage Kubernetes cluster on your Azure Stack Edge device](azure-stack-edge-gpu-create-kubernetes-cluster.md). --### Kubernetes and IoT Edge --This feature has been deprecated. Support will end soon. --All new deployments of IoT Edge on Azure Stack Edge must be on a Linux VM. For detailed steps, see [Deploy IoT runtime on Ubuntu VM on Azure Stack Edge](azure-stack-edge-gpu-deploy-iot-edge-linux-vm.md). - ### Kubernetes and Azure Arc Azure Arc is a hybrid management tool that will allow you to deploy applications on your Kubernetes clusters. Azure Arc also allows you to use Azure Monitor for containers to view and monitor your clusters. For more information, go to [What is Azure Arc-enabled Kubernetes?](../azure-arc/kubernetes/overview.md). For information on Azure Arc pricing, go to [Azure Arc pricing](https://azure.microsoft.com/services/azure-arc/#pricing). |
| databox-online | Azure Stack Edge Pro 2 Deploy Configure Compute | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/databox-online/azure-stack-edge-pro-2-deploy-configure-compute.md | In this tutorial, you learn how to: > * Configure compute > * Get Kubernetes endpoints - ## Prerequisites Before you set up a compute role on your Azure Stack Edge Pro device, make sure that: |
| databox-online | Azure Stack Edge Pro R Deploy Configure Network Compute Web Proxy | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/databox-online/azure-stack-edge-pro-r-deploy-configure-network-compute-web-proxy.md | You can add or delete virtual networks associated with your virtual switches. To ## Configure compute IPs -Follow these steps to configure compute IPs for your Kubernetes workloads. +After the virtual switches are created, you can enable the switches for Kubernetes compute traffic. 1. In the local UI, go to the **Kubernetes** page.-1. From the dropdown select a virtual switch that you will use for Kubernetes compute traffic. <!--By default, all switches are configured for management. You can't configure storage intent as storage traffic was already configured based on the network topology that you selected earlier.--> +1. Specify a workload from the options provided. + - If you're working with an Azure Private MEC solution, select the option for **an Azure Private MEC solution in your environment**. + - If you're working with an SAP Digital Manufacturing solution or another Microsoft partner solution, select the option for **a SAP Digital Manufacturing for Edge Computing or another Microsoft partner solution in your environment**. + - For other workloads, select the option for **other workloads in your environment**. -1. Assign **Kubernetes node IPs**. These static IP addresses are for the Kubernetes VMs. + If prompted, confirm the option you specified and then select **Apply**. - - For an *n*-node device, a contiguous range of a minimum of *n+1* IPv4 addresses (or more) are provided for the compute VM using the start and end IP addresses. For a 1-node device, provide a minimum of two, free, contiguous IPv4 addresses. + To use PowerShell to specify the workload, see detailed steps in [Change Kubernetes workload profiles](azure-stack-edge-gpu-connect-powershell-interface.md#change-kubernetes-workload-profiles). +  - > [!IMPORTANT] - > - Kubernetes on Azure Stack Edge uses 172.27.0.0/16 subnet for pod and 172.28.0.0/16 subnet for service. Make sure that these are not in use in your network. If these subnets are already in use in your network, you can change these subnets by running the ```Set-HcsKubeClusterNetworkInfo``` cmdlet from the PowerShell interface of the device. For more information, see Change Kubernetes pod and service subnets. <!--Target URL not available.--> - > - DHCP mode is not supported for Kubernetes node IPs. If you plan to deploy IoT Edge/Kubernetes, you must assign static Kubernetes IPs and then enable IoT role. This will ensure that static IPs are assigned to Kubernetes node VMs. - > - If your datacenter firewall is restricting or filtering traffic based on source IPs or MAC addresses, make sure that the compute IPs (Kubernetes node IPs) and MAC addresses are on the allowed list. The MAC addresses can be specified by running the ```Set-HcsMacAddressPool``` cmdlet on the PowerShell interface of the device. +1. From the dropdown list, select the virtual switch you want to enable for Kubernetes compute traffic. +1. Assign **Kubernetes node IPs**. These static IP addresses are for the Kubernetes VMs. -1. Assign **Kubernetes external service IPs**. These are also the load-balancing IP addresses. These contiguous IP addresses are for services that you want to expose outside of the Kubernetes cluster and you specify the static IP range depending on the number of services exposed. + If you select the **Azure Private MEC solution** or **SAP Digital Manufacturing for Edge Computing or another Microsoft partner** workload option for your environment, you must provide a contiguous range of a minimum of 6 IPv4 addresses (or more) for a 1-node configuration. - > [!IMPORTANT] - > We strongly recommend that you specify a minimum of one IP address for Azure Stack Edge Hub service to access compute modules. You can then optionally specify additional IP addresses for other services/IoT Edge modules (1 per service/module) that need to be accessed from outside the cluster. The service IP addresses can be updated later. + If you select the **other workloads** option for an *n*-node device, a contiguous range of a minimum of *n+1* IPv4 addresses (or more) are provided for the compute VM using the start and end IP addresses. For a 1-node device, provide a minimum of 2 free, contiguous IPv4 addresses. + > [!IMPORTANT] + > - If you're running **other workloads** in your environment, Kubernetes on Azure Stack Edge uses 172.27.0.0/16 subnet for pod and 172.28.0.0/16 subnet for service. Make sure that these are not in use in your network. For more information, see [Change Kubernetes pod and service subnets](azure-stack-edge-gpu-connect-powershell-interface.md#change-kubernetes-pod-and-service-subnets). + > - DHCP mode is not supported for Kubernetes node IPs. ++1. Assign **Kubernetes external service IPs**. These are also the load-balancing IP addresses. These contiguous IP addresses are for services that you want to expose outside of the Kubernetes cluster and you specify the static IP range depending on the number of services exposed. + + > [!IMPORTANT] + > We strongly recommend that you specify a minimum of 1 IP address for Azure Stack Edge Hub service to access compute modules. The service IP addresses can be updated later. + 1. Select **Apply**. -  +  -1. The configuration takes a couple minutes to apply and you may need to refresh the browser. +1. The configuration takes a couple minutes to apply and you may need to refresh the browser. 1. Select **Next: Web proxy** to configure web proxy. |
| defender-for-cloud | Azure Devops Extension | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/defender-for-cloud/azure-devops-extension.md | If you don't have access to install the extension, you must request access from ``` > [!NOTE]- > The artifactName 'CodeAnalysisLogs' is required for integration with Defender for Cloud. For additional tool configuration options, see [the Microsoft Security DevOps wiki](https://github.com/microsoft/security-devops-action/wiki) + > The artifactName 'CodeAnalysisLogs' is required for integration with Defender for Cloud. For additional tool configuration options and environment variables, see [the Microsoft Security DevOps wiki](https://github.com/microsoft/security-devops-action/wiki) 1. To commit the pipeline, select **Save and run**. The pipeline will run for a few minutes and save the results. > [!NOTE] > Install the SARIF SAST Scans Tab extension on the Azure DevOps organization in order to ensure that the generated analysis results will be displayed automatically under the Scans tab. +## Uploading findings from third-party security tooling into Defender for Cloud ++While Defender for Cloud provides the MSDO CLI for standardized functionality and poliy controls across a set of open source security analyzers, you have the flexibility to upload results from other third-party security tooling that you may have configured in CI/CD pipelines to Defender for Cloud for comprehensive code-to-cloud contextualization. All results uploaded to Defender for Cloud must be in standard SARIF format. ++First, ensure your Azure DevOps repositories are [onboarded to Defender for Cloud](quickstart-onboard-devops.md). After successfully onboarding, Defender for Cloud continuously monitors the 'CodeAnalysisLogs' artifact for SARIF output. ++You can use the 'PublishBuildArtifacts@1' task to ensure SARIF output is published to the correct artifact. For example, if a security analyzer outputs 'results.sarif', you can configure the following task in your job to ensure results are uploaded to Defender for Cloud: ++ ```yml + - task: PublishBuildArtifacts@1 + inputs: + PathtoPublish: 'results.sarif' + ArtifactName: 'CodeAnalysisLogs' + ``` +Findings from third-party security tools will appear as 'Azure DevOps repositories should have code scanning findings resolved' assessments associated with the repository the secuirty finding was identified in. + ## Learn more - Learn how to [create your first pipeline](/azure/devops/pipelines/create-first-pipeline). |
| defender-for-cloud | Concept Agentless Data Collection | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/defender-for-cloud/concept-agentless-data-collection.md | description: Learn how Defender for Cloud can gather information about your mult - Previously updated : 12/27/2023+ Last updated : 04/07/2024 +#customer intent: As a user, I want to understand how agentless machine scanning works in Defender for Cloud so that I can effectively collect data from my machines. # Agentless machine scanning Agentless scanning assists you in the identification process of actionable postu ||| |Release state:| GA | |Pricing:|Requires either [Defender Cloud Security Posture Management (CSPM)](concept-cloud-security-posture-management.md) or [Microsoft Defender for Servers Plan 2](plan-defender-for-servers-select-plan.md#plan-features)|-| Supported use cases:| :::image type="icon" source="./medi) **Only available with Defender for Servers plan 2**| -| Clouds: | :::image type="icon" source="./media/icons/yes-icon.png"::: Azure Commercial clouds<br> :::image type="icon" source="./media/icons/yes-icon.png"::: Azure Commercial clouds<br> :::image type="icon" source="./media/icons/no-icon.png"::: Azure Government<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Azure Commercial clouds<br> :::image type="icon" source="./media/icons/no-icon.png"::: Azure Government<br>:::image type="icon" source="./media/icons/no-icon.png"::: Microsoft Azure operated by 21Vianet<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Azure Commercial clouds<br> :::image type="icon" source="./media/icons/no-icon.png"::: Azure Government<br>:::image type="icon" source="./media/icons/no-icon.png"::: Microsoft Azure operated by 21Vianet<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Connected AWS accounts<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Azure Commercial clouds<br> :::image type="icon" source="./media/icons/no-icon.png"::: Azure Government<br>:::image type="icon" source="./media/icons/no-icon.png"::: Microsoft Azure operated by 21Vianet<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Connected AWS accounts<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Connected GCP projects | -| Operating systems: | :::image type="icon" source="./media/icons/yes-icon.png"::: Windows<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Windows<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Linux | +| Supported use cases:| :::image type="icon" source="./medi) **Only available with Defender for Servers plan 2**| +| Clouds: | :::image type="icon" source="./media/icons/yes-icon.png"::: Azure Commercial clouds<br> :::image type="icon" source="./media/icons/no-icon.png"::: Azure Government<br>:::image type="icon" source="./media/icons/no-icon.png"::: Microsoft Azure operated by 21Vianet<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Connected AWS accounts<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Connected GCP projects | +| Operating systems: | :::image type="icon" source="./media/icons/yes-icon.png"::: Windows<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Linux | | Instance and disk types: | **Azure**<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Standard VMs<br>:::image type="icon" source="./media/icons/no-icon.png"::: Unmanaged disks<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Virtual machine scale set - Flex<br>:::image type="icon" source="./media/icons/no-icon.png"::: Virtual machine scale set - Uniform<br><br>**AWS**<br>:::image type="icon" source="./media/icons/yes-icon.png"::: EC2<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Auto Scale instances<br>:::image type="icon" source="./media/icons/no-icon.png"::: Instances with a ProductCode (Paid AMIs)<br><br>**GCP**<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Compute instances<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Instance groups (managed and unmanaged) | | Encryption: | **Azure**<br>:::image type="icon" source="./medi) with platform-managed keys (PMK)<br>:::image type="icon" source="./media/icons/no-icon.png"::: Encrypted ΓÇô other scenarios using platform-managed keys (PMK)<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Encrypted ΓÇô customer-managed keys (CMK) (preview)<br><br>**AWS**<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Unencrypted<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Encrypted - PMK<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Encrypted - CMK<br><br>**GCP**<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Google-managed encryption key<br>:::image type="icon" source="./media/icons/yes-icon.png"::: Customer-managed encryption key (CMEK)<br>:::image type="icon" source="./media/icons/no-icon.png"::: Customer-supplied encryption key (CSEK) | |
| defender-for-cloud | Upcoming Changes | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/defender-for-cloud/upcoming-changes.md | If you're looking for the latest release notes, you can find them in the [What's **Estimated date for change: May 2024** -the recommendation ### [Virtual machines should encrypt temp disks, caches, and data flows between Compute and Storage resources](https://portal.azure.com/#blade/Microsoft_Azure_Security/RecommendationsBlade/assessmentKey/d57a4221-a804-52ca-3dea-768284f06bb7) is set to be deprecated. +The recommendation [Virtual machines should encrypt temp disks, caches, and data flows between Compute and Storage resources](https://portal.azure.com/#blade/Microsoft_Azure_Security/RecommendationsBlade/assessmentKey/d57a4221-a804-52ca-3dea-768284f06bb7) is set to be deprecated. ## Deprecating of virtual machine recommendation |
| dms | Tutorial Postgresql Azure Postgresql Online | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/dms/tutorial-postgresql-azure-postgresql-online.md | If you need to cancel or delete any DMS task, project, or service, perform the c az dms project task delete --service-name PostgresCLI --project-name PGMigration --resource-group PostgresDemo --name runnowtask ``` -3. To cancel a running project, use the following command: - ```azurecli - az dms project task cancel -n runnowtask --project-name PGMigration -g PostgresDemo --service-name PostgresCLI - ``` --4. To delete a running project, use the following command: +3. To delete a project, use the following command: ```azurecli- az dms project task delete -n runnowtask --project-name PGMigration -g PostgresDemo --service-name PostgresCLI + az dms project delete -n PGMigration -g PostgresDemo --service-name PostgresCLI ``` -5. To delete DMS service, use the following command: +4. To delete DMS service, use the following command: ```azurecli az dms delete -g ProgresDemo -n PostgresCLI |
| dns | Dns Private Resolver Get Started Portal | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/dns/dns-private-resolver-get-started-portal.md | description: In this quickstart, you create and test a private DNS resolver in A Previously updated : 02/28/2024 Last updated : 04/05/2024 Add or remove specific rules your DNS forwarding ruleset as desired, such as: - A rule to resolve an on-premises zone: internal.contoso.com. - A wildcard rule to forward unmatched DNS queries to a protective DNS service. +> [!IMPORTANT] +> The rules shown in this quickstart are examples of rules that can be used for specific scenarios. None of the fowarding rules described in this article are required. Be careful to test your forwarding rules and ensure that the rules don't cause DNS resolution issues. + ### Delete a rule from the forwarding ruleset Individual rules can be deleted or disabled. In this example, a rule is deleted. |
| dns | Dns Private Resolver Get Started Powershell | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/dns/dns-private-resolver-get-started-powershell.md | description: In this quickstart, you learn how to create and manage your first p Previously updated : 02/28/2024 Last updated : 04/05/2024 $virtualNetworkLink2.ToJsonString() ## Create forwarding rules ++ Create a forwarding rule for a ruleset to one or more target DNS servers. You must specify the fully qualified domain name (FQDN) with a trailing dot. The **New-AzDnsResolverTargetDnsServerObject** cmdlet sets the default port as 53, but you can also specify a unique port. ```Azure PowerShell In this example: - 192.168.1.2 and 192.168.1.3 are on-premises DNS servers. - 10.5.5.5 is a protective DNS service. +> [!IMPORTANT] +> The rules shown in this quickstart are examples of rules that can be used for specific scenarios. None of the fowarding rules described in this article are required. Be careful to test your forwarding rules and ensure that the rules don't cause DNS resolution issues. + ## Test the private resolver You should now be able to send DNS traffic to your DNS resolver and resolve records based on your forwarding rulesets, including: |
| dns | Private Resolver Endpoints Rulesets | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/dns/private-resolver-endpoints-rulesets.md | For example, if you have the following rules: A query for `secure.store.azure.contoso.com` matches the **AzurePrivate** rule for `azure.contoso.com` and also the **Contoso** rule for `contoso.com`, but the **AzurePrivate** rule takes precedence because the prefix `azure.contoso` is longer than `contoso`. > [!IMPORTANT]-> If a rule is present in the ruleset that has as its destination a private resolver inbound endpoint, do not link the ruleset to the VNet where the inbound endpoint is provisioned. This configuration can cause DNS resolution loops. For example: In the previous scenario, no ruleset link should be added to `myeastvnet` because the inbound endpoint at `10.10.0.4` is provisioned in `myeastvnet` and a rule is present that resolves `azure.contoso.com` using the inbound endpoint. +> If a rule is present in the ruleset that has as its destination a private resolver inbound endpoint, do not link the ruleset to the VNet where the inbound endpoint is provisioned. This configuration can cause DNS resolution loops. For example: In the previous scenario, no ruleset link should be added to `myeastvnet` because the inbound endpoint at `10.10.0.4` is provisioned in `myeastvnet` and a rule is present that resolves `azure.contoso.com` using the inbound endpoint.<br><br> +> The rules shown in this article are examples of rules that can be used for specific scenarios. None of the fowarding rules described here are required. Be careful to test your forwarding rules and ensure that the rules don't cause DNS resolution issues. #### Rule processing |
| dns | Private Resolver Hybrid Dns | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/dns/private-resolver-hybrid-dns.md | Title: Resolve Azure and on-premises domains -description: Configure Azure and on-premises DNS to resolve private DNS zones and on-premises domains + Title: Resolve Azure and on-premises domains. +description: Configure Azure and on-premises DNS to resolve private DNS zones and on-premises domains. Previously updated : 10/05/2023 Last updated : 04/05/2024 #Customer intent: As an administrator, I want to resolve on-premises domains in Azure and resolve Azure private zones on-premises. -This article provides guidance on how to configure hybrid DNS resolution by using an [Azure DNS Private Resolver](#azure-dns-private-resolver) with a [DNS forwarding ruleset](#dns-forwarding-ruleset). +This article provides guidance on how to configure hybrid DNS resolution by using an [Azure DNS Private Resolver](#azure-dns-private-resolver) with a [DNS forwarding ruleset](#dns-forwarding-ruleset). In this scenario, your Azure DNS resources are connected to an on-premises network using a VPN or ExpressRoute connection. *Hybrid DNS resolution* is defined here as enabling Azure resources to resolve your on-premises domains, and on-premises DNS to resolve your Azure private DNS zones. Create a private zone with at least one resource record to use for testing. The - [Create a private zone - PowerShell](private-dns-getstarted-powershell.md) - [Create a private zone - CLI](private-dns-getstarted-cli.md) -In this article, the private zone **azure.contoso.com** and the resource record **test** are used. Autoregistration isn't required for the current demonstration. +In this article, the private zone **azure.contoso.com** and the resource record **test** are used. Autoregistration isn't required for the current demonstration. > [!IMPORTANT] > A recursive server is used to forward queries from on-premises to Azure in this example. If the server is authoritative for the parent zone (contoso.com), forwarding is not possible unless you first create a delegation for azure.contoso.com. [  ](./media/private-resolver-hybrid-dns/private-zone-records.png#lightbox) -**Requirement**: You must create a virtual network link in the zone to the virtual network where you deploy your Azure DNS Private Resolver. In the following example, the private zone is linked to two VNets: **myeastvnet** and **mywestvnet**. At least one link is required. +**Requirement**: You must create a virtual network link in the zone to the virtual network where you deploy your Azure DNS Private Resolver. In the following example, the private zone is linked to two VNets: **myeastvnet** and **mywestvnet**. At least one link is required. [  ](./media/private-resolver-hybrid-dns/private-zone-links.png#lightbox) ## Create an Azure DNS Private Resolver -The following quickstarts are available to help you create a private resolver. These quickstarts walk you through creating a resource group, a virtual network, and Azure DNS Private Resolver. The steps to configure an inbound endpoint, outbound endpoint, and DNS forwarding ruleset are provided: +The following quickstarts are available to help you create a private resolver. These quickstarts walk you through creating a resource group, a virtual network, and Azure DNS Private Resolver. The steps to configure an inbound endpoint, outbound endpoint, and DNS forwarding ruleset are provided: - [Create a private resolver - portal](dns-private-resolver-get-started-portal.md) - [Create a private resolver - PowerShell](dns-private-resolver-get-started-powershell.md) - When you're finished, write down the IP address of the inbound endpoint for the Azure DNS Private Resolver. In this example, the IP address is **10.10.0.4**. This IP address is used later to configure on-premises DNS conditional forwarders. + When you're finished, write down the IP address of the inbound endpoint for the Azure DNS Private Resolver. In this example, the IP address is **10.10.0.4**. This IP address is used later to configure on-premises DNS conditional forwarders. [  ](./media/private-resolver-hybrid-dns/inbound-endpoint-ip.png#lightbox) Create a forwarding ruleset in the same region as your private resolver. The fol [  ](./media/private-resolver-hybrid-dns/forwarding-ruleset-region.png#lightbox) -**Requirement**: You must create a virtual network link to the vnet where your private resolver is deployed. In the following example, two virtual network links are present. The link **myeastvnet-link** is created to a hub vnet where the private resolver is provisioned. There's also a virtual network link **myeastspoke-link** that provides hybrid DNS resolution in a spoke vnet that doesn't have its own private resolver. The spoke network is able to use the private resolver because it peers with the hub network. The spoke vnet link isn't required for the current demonstration. +**Requirement**: You must create a virtual network link to the vnet where your private resolver is deployed. In the following example, two virtual network links are present. The link **myeastvnet-link** is created to a hub vnet where the private resolver is provisioned. There's also a virtual network link **myeastspoke-link** that provides hybrid DNS resolution in a spoke vnet that doesn't have its own private resolver. The spoke network is able to use the private resolver because it peers with the hub network. The spoke vnet link isn't required for the current demonstration. [  ](./media/private-resolver-hybrid-dns/ruleset-links.png#lightbox) -Next, create a rule in your ruleset for your on-premises domain. In this example, we use **contoso.com**. Set the destination IP address for your rule to be the IP address of your on-premises DNS server. In this example, the on-premises DNS server is at **10.100.0.2**. Verify that the rule is **Enabled**. +Next, create a rule in your ruleset for your on-premises domain. In this example, we use **contoso.com**. Set the destination IP address for your rule to be the IP address of your on-premises DNS server. In this example, the on-premises DNS server is at **10.100.0.2**. Verify that the rule is **Enabled**. [  ](./media/private-resolver-hybrid-dns/ruleset-rules.png#lightbox) The procedure to configure on-premises DNS depends on the type of DNS server you ## Demonstrate hybrid DNS -Using a VM located in the virtual network where the Azure DNS Private Resolver is provisioned, issue a DNS query for a resource record in your on-premises domain. In this example, a query is performed for the record **testdns.contoso.com**: +Using a VM located in the virtual network where the Azure DNS Private Resolver is provisioned, issue a DNS query for a resource record in your on-premises domain. In this example, a query is performed for the record **testdns.contoso.com**:  -The path for the query is: Azure DNS > inbound endpoint > outbound endpoint > ruleset rule for contoso.com > on-premises DNS (10.100.0.2). The DNS server at 10.100.0.2 is an on-premises DNS resolver, but it could also be an authoritative DNS server. +The path for the query is: Azure DNS > inbound endpoint > outbound endpoint > ruleset rule for contoso.com > on-premises DNS (10.100.0.2). The DNS server at 10.100.0.2 is an on-premises DNS resolver, but it could also be an authoritative DNS server. Using an on-premises VM or device, issue a DNS query for a resource record in your Azure private DNS zone. In this example, a query is performed for the record **test.azure.contoso.com**: The path for this query is: client's default DNS resolver (10.100.0.2) > on-prem * Learn how to create an Azure DNS Private Resolver by using [Azure PowerShell](./dns-private-resolver-get-started-powershell.md) or [Azure portal](./dns-private-resolver-get-started-portal.md). * Understand how to [Resolve Azure and on-premises domains](private-resolver-hybrid-dns.md) using the Azure DNS Private Resolver. * Learn about [Azure DNS Private Resolver endpoints and rulesets](private-resolver-endpoints-rulesets.md).-* Learn how to [Set up DNS failover using private resolvers](tutorial-dns-private-resolver-failover.md) +* Learn how to [Set up DNS failover using private resolvers](tutorial-dns-private-resolver-failover.md). * Learn about some of the other key [networking capabilities](../networking/fundamentals/networking-overview.md) of Azure. * [Learn module: Introduction to Azure DNS](/training/modules/intro-to-azure-dns). |
| event-hubs | Event Hubs Dotnet Standard Getstarted Send | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/event-hubs/event-hubs-dotnet-standard-getstarted-send.md | Title: 'Quickstart: Send or receive events using .NET' description: A quickstart that shows you how to create a .NET Core application that sends events to and receive events from Azure Event Hubs. Previously updated : 03/09/2023 Last updated : 04/05/2024 ms.devlang: csharp +#customer intent: As a .NET developer, I want to learn how to send events to an event hub and receive events from the event hub using C#. # Quickstart: Send events to and receive events from Azure Event Hubs using .NET In this quickstart, you learn how to send events to an event hub and then receive those events from the event hub using the **Azure.Messaging.EventHubs** .NET library. > [!NOTE]-> Quickstarts are for you to quickly ramp up on the service. If you are already familiar with the service, you may want to see .NET samples for Event Hubs in our .NET SDK repository on GitHub: [Event Hubs samples on GitHub](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/eventhub/Azure.Messaging.EventHubs/samples), [Event processor samples on GitHub](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/eventhub/Azure.Messaging.EventHubs.Processor/samples). +> Quickstarts are for you to quickly ramp up on the service. If you are already familiar with the service, you might want to see .NET samples for Event Hubs in our .NET SDK repository on GitHub: [Event Hubs samples on GitHub](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/eventhub/Azure.Messaging.EventHubs/samples), [Event processor samples on GitHub](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/eventhub/Azure.Messaging.EventHubs.Processor/samples). ## Prerequisites If you're new to Azure Event Hubs, see [Event Hubs overview](event-hubs-about.md) before you go through this quickstart. This section shows you how to create a .NET Core console application to send eve ```csharp A batch of 3 events has been published. ```-4. On the **Event Hubs Namespace** page in the Azure portal, you see three incoming messages in the **Messages** chart. Refresh the page to update the chart if needed. It may take a few seconds for it to show that the messages have been received. ++ > [!IMPORTANT] + > If you are using the Passwordless (Azure Active Directory's Role-based Access Control) authentication, select **Tools**, then select **Options**. In the **Options** window, expand **Azure Service Authentication**, and select **Account Selection**. Confirm that you are using the account that was added to the **Azure Event Hubs Data Owner** role on the Event Hubs namespace. +4. On the **Event Hubs Namespace** page in the Azure portal, you see three incoming messages in the **Messages** chart. Refresh the page to update the chart if needed. It might take a few seconds for it to show that the messages have been received. :::image type="content" source="./media/getstarted-dotnet-standard-send-v2/verify-messages-portal.png" alt-text="Image of the Azure portal page to verify that the event hub received the events" lightbox="./media/getstarted-dotnet-standard-send-v2/verify-messages-portal.png"::: In this quickstart, you use Azure Storage as the checkpoint store. Follow these [Get the connection string to the storage account](../storage/common/storage-account-get-info.md#get-a-connection-string-for-the-storage-account) -Note down the connection string and the container name. You use them in the receive code. +Note down the connection string and the container name. You use them in the code to receive events from the event hub. ### Create a project for the receiver Replace the contents of **Program.cs** with the following code: { // Write the body of the event to the console window Console.WriteLine("\tReceived event: {0}", Encoding.UTF8.GetString(eventArgs.Data.Body.ToArray()));- Console.ReadLine(); return Task.CompletedTask; } Replace the contents of **Program.cs** with the following code: // Write details about the error to the console window Console.WriteLine($"\tPartition '{eventArgs.PartitionId}': an unhandled exception was encountered. This was not expected to happen."); Console.WriteLine(eventArgs.Exception.Message);- Console.ReadLine(); return Task.CompletedTask; } ``` Replace the contents of **Program.cs** with the following code: > [!NOTE] > For the complete source code with more informational comments, see [this file on the GitHub](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs.Processor/samples/Sample01_HelloWorld.md). 3. Run the receiver application. -4. You should see a message that the events have been received. +4. You should see a message that the events have been received. Press ENTER after you see a received event message. ```bash Received event: Event 1 Replace the contents of **Program.cs** with the following code: Received event: Event 3 ``` These events are the three events you sent to the event hub earlier by running the sender program. -5. In the Azure portal, you can verify that there are three outgoing messages, which Event Hubs sent to the receiving application. Refresh the page to update the chart. It may take a few seconds for it to show that the messages have been received. +5. In the Azure portal, you can verify that there are three outgoing messages, which Event Hubs sent to the receiving application. Refresh the page to update the chart. It might take a few seconds for it to show that the messages have been received. :::image type="content" source="./media/getstarted-dotnet-standard-send-v2/verify-messages-portal-2.png" alt-text="Image of the Azure portal page to verify that the event hub sent events to the receiving app" lightbox="./media/getstarted-dotnet-standard-send-v2/verify-messages-portal-2.png"::: Azure Schema Registry of Event Hubs provides a centralized repository for managi To learn more, see [Validate schemas with Event Hubs SDK](schema-registry-dotnet-send-receive-quickstart.md). -## Clean up resources -Delete the resource group that has the Event Hubs namespace or delete only the namespace if you want to keep the resource group. ## Samples and reference This quick start provides step-by-step instructions to implement a scenario of sending a batch of events to an event hub and then receiving them. For more samples, select the following links. This quick start provides step-by-step instructions to implement a scenario of s For complete .NET library reference, see our [SDK documentation](/dotnet/api/overview/azure/event-hubs). -## Next steps +## Clean up resources +Delete the resource group that has the Event Hubs namespace or delete only the namespace if you want to keep the resource group. ++## Related content See the following tutorial: > [!div class="nextstepaction"] |
| event-hubs | Event Hubs Node Get Started Send | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/event-hubs/event-hubs-node-get-started-send.md | Title: Send or receive events from Azure Event Hubs using JavaScript + Title: Send or receive events using JavaScript description: This article provides a walkthrough for creating a JavaScript application that sends/receives events to/from Azure Event Hubs. Previously updated : 01/04/2023 Last updated : 04/05/2024 ms.devlang: javascript +#customer intent: As a JavaScript developer, I want to learn how to send events to an event hub and receive events from the event hub using C#. -# Send events to or receive events from event hubs by using JavaScript -This quickstart shows how to send events to and receive events from an event hub using the **@azure/event-hubs** npm package. +# Quickstart: Send events to or receive events from event hubs by using JavaScript +In this Quickstart, you learn how to send events to and receive events from an event hub using the **@azure/event-hubs** npm package. ## Prerequisites If you're new to Azure Event Hubs, see [Event Hubs overview](event-hubs-about.md To complete this quickstart, you need the following prerequisites: -- **Microsoft Azure subscription**. To use Azure services, including Azure Event Hubs, you need a subscription. If you don't have an existing Azure account, you can sign up for a [free trial](https://azure.microsoft.com/free/) or use your MSDN subscriber benefits when you [create an account](https://azure.microsoft.com).+- **Microsoft Azure subscription**. To use Azure services, including Azure Event Hubs, you need a subscription. If you don't have an existing Azure account, you can sign up for a [free trial](https://azure.microsoft.com/free/). - Node.js LTS. Download the latest [long-term support (LTS) version](https://nodejs.org). - Visual Studio Code (recommended) or any other integrated development environment (IDE). - **Create an Event Hubs namespace and an event hub**. The first step is to use the [Azure portal](https://portal.azure.com) to create a namespace of type Event Hubs, and obtain the management credentials your application needs to communicate with the event hub. To create a namespace and an event hub, follow the procedure in [this article](event-hubs-create.md). In this section, you create a JavaScript application that sends events to an eve -1. Run `node send.js` to execute this file. This command sends a batch of three events to your event hub. -1. In the Azure portal, verify that the event hub has received the messages. Refresh the page to update the chart. It might take a few seconds for it to show that the messages have been received. +1. Run `node send.js` to execute this file. This command sends a batch of three events to your event hub. If you're using the Passwordless (Azure Active Directory's Role-based Access Control) authentication, you might want to run `az login` and sign into Azure using the account that was added to the Azure Event Hubs Data Owner role. +1. In the Azure portal, verify that the event hub received the messages. Refresh the page to update the chart. It might take a few seconds for it to show that the messages are received. [](./media/node-get-started-send/verify-messages-portal.png#lightbox) > [!NOTE] > For the complete source code, including additional informational comments, go to the [GitHub sendEvents.js page](https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/eventhub/event-hubs/samples/v5/javascript/sendEvents.js). - You have now sent events to an event hub. --+ ## Receive events In this section, you receive events from an event hub by using an Azure Blob storage checkpoint store in a JavaScript application. It performs metadata checkpoints on received messages at regular intervals in an Azure Storage blob. This approach makes it easy to continue receiving messages later from where you left off. To create an Azure storage account and a blob container in it, do the following [Get the connection string to the storage account](../storage/common/storage-configure-connection-string.md). -Note the connection string and the container name. You'll use them in the receive code. +Note the connection string and the container name. You use them in the code to receive events. ### Install the npm packages to receive events -For the receiving side, you need to install two more packages. In this quickstart, you use Azure Blob storage to persist checkpoints so that the program doesn't read the events that it has already read. It performs metadata checkpoints on received messages at regular intervals in a blob. This approach makes it easy to continue receiving messages later from where you left off. +For the receiving side, you need to install two more packages. In this quickstart, you use Azure Blob storage to persist checkpoints so that the program doesn't read the events that it already read. It performs metadata checkpoints on received messages at regular intervals in a blob. This approach makes it easy to continue receiving messages later from where you left off. ### [Passwordless (Recommended)](#tab/passwordless) npm install @azure/eventhubs-checkpointstore-blob 1. Run `node receive.js` in a command prompt to execute this file. The window should display messages about received events. - ``` + ```bash C:\Self Study\Event Hubs\JavaScript>node receive.js Received event: 'First event' from partition: '0' and consumer group: '$Default' Received event: 'Second event' from partition: '0' and consumer group: '$Default' Received event: 'Third event' from partition: '0' and consumer group: '$Default' ```+ > [!NOTE] > For the complete source code, including additional informational comments, go to the [GitHub receiveEventsUsingCheckpointStore.js page](https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/eventhub/eventhubs-checkpointstore-blob/samples/v1/javascript/receiveEventsUsingCheckpointStore.js). -You have now received events from your event hub. The receiver program will receive events from all the partitions of the default consumer group in the event hub. + The receiver program receives events from all the partitions of the default consumer group in the event hub. ++## Clean up resources +Delete the resource group that has the Event Hubs namespace or delete only the namespace if you want to keep the resource group. -## Next steps +## Related content Check out these samples on GitHub: - [JavaScript samples](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/eventhub/event-hubs/samples/v5/javascript) |
| event-hubs | Monitor Event Hubs | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/event-hubs/monitor-event-hubs.md | Title: Monitoring Azure Event Hubs description: Learn how to use Azure Monitor to view, analyze, and create alerts on metrics from Azure Event Hubs. Previously updated : 03/01/2023 Last updated : 04/05/2024 # Monitor Azure Event Hubs See [Create diagnostic setting to collect platform logs and metrics in Azure](.. If you use **Azure Storage** to store the diagnostic logging information, the information is stored in containers named **insights-logs-operationlogs** and **insights-metrics-pt1m**. Sample URL for an operation log: `https://<Azure Storage account>.blob.core.windows.net/insights-logs-operationallogs/resourceId=/SUBSCRIPTIONS/<Azure subscription ID>/RESOURCEGROUPS/<Resource group name>/PROVIDERS/MICROSOFT.SERVICEBUS/NAMESPACES/<Namespace name>/y=<YEAR>/m=<MONTH-NUMBER>/d=<DAY-NUMBER>/h=<HOUR>/m=<MINUTE>/PT1H.json`. The URL for a metric log is similar. ### Azure Event Hubs-If you use **Azure Event Hubs** to store the diagnostic logging information, the information is stored in Event Hubs instances named **insights-logs-operationlogs** and **insights-metrics-pt1m**. You can also select an existing event hub except for the event hub for which you are configuring diagnostic settings. +If you use **Azure Event Hubs** to store the diagnostic logging information, the information is stored in Event Hubs instances named **insights-logs-operationlogs** and **insights-metrics-pt1m**. You can also select an existing event hub except for the event hub for which you're configuring diagnostic settings. ### Log Analytics If you use **Log Analytics** to store the diagnostic logging information, the information is stored in tables named **AzureDiagnostics** / **AzureMetrics** or **resource specific tables** The metrics and logs you can collect are discussed in the following sections. ## Analyze metrics You can analyze metrics for Azure Event Hubs, along with metrics from other Azure services, by selecting **Metrics** from the **Azure Monitor** section on the home page for your Event Hubs namespace. See [Analyze metrics with Azure Monitor metrics explorer](../azure-monitor/essentials/analyze-metrics.md) for details on using this tool. For a list of the platform metrics collected, see [Monitoring Azure Event Hubs data reference metrics](monitor-event-hubs-reference.md#metrics). - For reference, you can see a list of [all resource metrics supported in Azure Monitor](../azure-monitor/essentials/metrics-supported.md). For reference, you can see a list of [all resource metrics supported in Azure Mo ### Filter and split For metrics that support dimensions, you can apply filters using a dimension value. For example, add a filter with `EntityName` set to the name of an event hub. You can also split a metric by dimension to visualize how different segments of the metric compare with each other. For more information of filtering and splitting, see [Advanced features of Azure Monitor](../azure-monitor/essentials/metrics-charts.md). ## Analyze logs Using Azure Monitor Log Analytics requires you to create a diagnostic configuration and enable __Send information to Log Analytics__. For more information, see the [Collection and routing](#collection-and-routing) section. Data in Azure Monitor Logs is stored in tables, with each table having its own set of unique properties. Azure Event Hubs stores data in the following tables: **AzureDiagnostics** and **AzureMetrics**. Using *Runtime audit logs* you can capture aggregated diagnostic information for > Runtime audit logs are available only in **premium** and **dedicated** tiers. ### Enable runtime logs-You can enable either runtime audit logs or application metrics logs by selecting *Diagnostic settings* from the *Monitoring* section on the Event Hubs namespace page in Azure portal. Click on *Add diagnostic setting* as shown below. +You can enable either runtime audit or application metrics logging by selecting *Diagnostic settings* from the *Monitoring* section on the Event Hubs namespace page in Azure portal. Select **Add diagnostic setting** as shown in the following image. - Then you can enable log categories *RuntimeAuditLogs* or *ApplicationMetricsLogs* as needed. - -Once runtime logs are enabled, Event Hubs will start collecting and storing them according to the diagnostic setting configuration. ++Once runtime logs are enabled, Event Hubs start collecting and storing them according to the diagnostic setting configuration. ### Publish and consume sample data -To collect sample runtime audit logs in your Event Hubs namespace, you can publish and consume sample data using client applications which are based on [Event Hubs SDK](../event-hubs/event-hubs-dotnet-standard-getstarted-send.md) (AMQP) or using any [Apache Kafka client application](../event-hubs/event-hubs-quickstart-kafka-enabled-event-hubs.md). +To collect sample runtime audit logs in your Event Hubs namespace, you can publish and consume sample data using client applications, which are based on [Event Hubs SDK](../event-hubs/event-hubs-dotnet-standard-getstarted-send.md), which uses Advanced Message Queuing Protocol (AMQP) or using any [Apache Kafka client application](../event-hubs/event-hubs-quickstart-kafka-enabled-event-hubs.md). ### Analyze runtime audit logs AZMSRuntimeAuditLogs Up on the execution of the query you should be able to obtain corresponding audit logs in the following format. :::image type="content" source="./media/monitor-event-hubs/runtime-audit-logs.png" alt-text="Image showing the result of a sample query to analyze runtime audit logs." lightbox="./media/monitor-event-hubs/runtime-audit-logs.png"::: -By analyzing these logs you should be able to audit how each client application interacts with Event Hubs. Each field associated with runtime audit logs are defined in [runtime audit logs reference](../event-hubs/monitor-event-hubs-reference.md#runtime-audit-logs). +By analyzing these logs, you should be able to audit how each client application interacts with Event Hubs. Each field associated with runtime audit logs is defined in [runtime audit logs reference](../event-hubs/monitor-event-hubs-reference.md#runtime-audit-logs). ### Analyze application metrics AZMSApplicationMetricLogs | where Provider == "EVENTHUB" ``` -Application metrics includes the following runtime metrics. +Application metrics include the following runtime metrics. :::image type="content" source="./media/monitor-event-hubs/application-metrics-logs.png" alt-text="Image showing the result of a sample query to analyze application metrics." lightbox="./media/monitor-event-hubs/application-metrics-logs.png"::: -Therefore you can use application metrics to monitor runtime metrics such as consumer lag or active connection from a given client application. Each field associated with runtime audit logs are defined in [application metrics logs reference](../event-hubs/monitor-event-hubs-reference.md#runtime-audit-logs). +Therefore you can use application metrics to monitor runtime metrics such as consumer lag or active connection from a given client application. Fields associated with runtime audit logs are defined in [application metrics logs reference](../event-hubs/monitor-event-hubs-reference.md#runtime-audit-logs). ## Alerts |
| expressroute | Expressroute Locations Providers | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/expressroute/expressroute-locations-providers.md | The following table shows connectivity locations and the service providers for e | **Santiago** | [EdgeConnex SCL](https://www.edgeconnex.com/locations/south-america/santiago/) | 3 | n/a | Supported | PitChile | | **Sao Paulo** | [Equinix SP2](https://www.equinix.com/locations/americas-colocation/brazil-colocation/sao-paulo-data-centers/sp2/) | 3 | Brazil South | Supported | Aryaka Networks<br/>Ascenty Data Centers<br/>British Telecom<br/>Equinix<br/>InterCloud<br/>Level 3 Communications<br/>Neutrona Networks<br/>Orange<br/>RedCLARA<br/>Tata Communications<br/>Telefonica<br/>UOLDIVEO | | **Sao Paulo2** | [TIVIT TSM](https://www.tivit.com/en/tivit/) | 3 | Brazil South | Supported | Ascenty Data Centers<br/>Tivit |-| **Seattle** | [Equinix SE2](https://www.equinix.com/locations/americas-colocation/united-states-colocation/seattle-data-centers/se2/) | 1 | West US 2 | Supported | Aryaka Networks<br/>CenturyLink Cloud Connect<br/>DE-CIX<br/>Equinix<br/>Level 3 Communications<br/>Megaport<br/>PacketFabric<br/>Telus<br/>Zayo | +| **Seattle** | [Equinix SE2](https://www.equinix.com/locations/americas-colocation/united-states-colocation/seattle-data-centers/se2/) | 1 | West US 2 | Supported | Aryaka Networks<br/>CenturyLink Cloud Connect<br/>DE-CIX<br/>Equinix<br/>Level 3 Communications<br/>Megaport<br/>Pacific Northwest Gigapop<br/>PacketFabric<br/>Telus<br/>Zayo | | **Seoul** | [KINX Gasan IDC](https://www.kinx.net/?lang=en) | 2 | Korea Central | Supported | KINX<br/>KT<br/>LG CNS<br/>LGUplus<br/>Equinix<br/>Sejong Telecom<br/>SK Telecom | | **Seoul2** | [KT IDC](https://www.kt-idc.com/eng/introduce/sub1_4_10.jsp#tab) | 2 | Korea Central | n/a | KT | | **Silicon Valley** | [Equinix SV1](https://www.equinix.com/locations/americas-colocation/united-states-colocation/silicon-valley-data-centers/sv1/) | 1 | West US | Supported | Aryaka Networks<br/>AT&T Dynamic Exchange<br/>AT&T NetBond<br/>British Telecom<br/>CenturyLink Cloud Connect<br/>Colt<br/>Comcast<br/>Coresite<br/>Cox Business Cloud Port<br/>Equinix<br/>InterCloud<br/>Internet2<br/>IX Reach<br/>Packet<br/>PacketFabric<br/>Level 3 Communications<br/>Megaport<br/>Momentum Telecom<br/>Orange<br/>Sprint<br/>Tata Communications<br/>Telia Carrier<br/>Verizon<br/>Vodafone<br/>Zayo | |
| expressroute | Expressroute Locations | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/expressroute/expressroute-locations.md | The following table shows locations by service provider. If you want to view ava | **[Orange](https://www.orange-business.com/en/products/business-vpn-galerie)** |Supported |Supported | Amsterdam<br/>Amsterdam2<br/>Chicago<br/>Dallas<br/>Dubai2<br/>Dublin2<br/>Frankfurt<br/>Hong Kong<br/>Johannesburg<br/>London<br/>London2<br/>Mumbai2<br/>Melbourne<br/>Paris<br/>Paris2<br/>Sao Paulo<br/>Silicon Valley<br/>Singapore<br/>Sydney<br/>Tokyo<br/>Toronto<br/>Washington DC | | **[Orange Poland](https://www.orange.pl/duze-firmy/rozwiazania-chmurowe)** | Supported | Supported | Warsaw | | **[Orixcom](https://www.orixcom.com/solutions/azure-expressroute)** | Supported | Supported | Dubai2 |+| **Pacific Northwest Gigapop** | Supported | Supported | Seattle | | **[PacketFabric](https://www.packetfabric.com/cloud-connectivity/microsoft-azure)** | Supported | Supported | Amsterdam<br/>Chicago<br/>Dallas<br/>Denver<br/>Las Vegas<br/>London<br/>Los Angeles2<br/>Miami<br/>New York<br/>Seattle<br/>Silicon Valley<br/>Toronto<br/>Washington DC | | **[PCCW Global Limited](https://consoleconnect.com/clouds/#azureRegions)** | Supported | Supported | Chicago<br/>Hong Kong<br/>Hong Kong2<br/>London<br/>Singapore<br/>Singapore2<br/>Tokyo2 | | **PitChile** | Supported | Supported | Santiago<br/>Miami | |
| frontdoor | Domain | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/frontdoor/domain.md | After you've imported your certificate to a key vault, create an Azure Front Doo Then, configure your domain to use the Azure Front Door secret for its TLS certificate. -For a guided walkthrough of these steps, see [Configure HTTPS on an Azure Front Door custom domain using the Azure portal](standard-premium/how-to-configure-https-custom-domain.md#using-your-own-certificate). +For a guided walkthrough of these steps, see [Configure HTTPS on an Azure Front Door custom domain using the Azure portal](standard-premium/how-to-configure-https-custom-domain.md#use-your-own-certificate). ### Switch between certificate types |
| frontdoor | Front Door How To Onboard Apex Domain | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/frontdoor/front-door-how-to-onboard-apex-domain.md | Title: Onboard a root or apex domain to Azure Front Door -description: Learn how to onboard a root or apex domain to an existing Azure Front Door using the Azure portal. +description: Learn how to onboard a root or apex domain to an existing Azure Front Door by using the Azure portal. zone_pivot_groups: front-door-tiers [!INCLUDE [Azure Front Door (classic) retirement notice](../../includes/front-door-classic-retirement.md)] -Azure Front Door uses CNAME records to validate domain ownership for the onboarding of custom domains. Azure Front Door doesn't expose the frontend IP address associated with your Front Door profile. So you can't map your apex domain to an IP address if your intent is to onboard it to Azure Front Door. +Azure Front Door uses CNAME records to validate domain ownership for the onboarding of custom domains. Azure Front Door doesn't expose the front-end IP address associated with your Azure Front Door profile. So, you can't map your apex domain to an IP address if your intent is to onboard it to Azure Front Door. -The Domain Name System (DNS) protocol prevents the assignment of CNAME records at the zone apex. For example, if your domain is `contoso.com`; you can create CNAME records for `somelabel.contoso.com`; but you can't create CNAME for `contoso.com` itself. This restriction presents a problem for application owners who load balances applications behind Azure Front Door. Since using an Azure Front Door profile requires creation of a CNAME record, it isn't possible to point at the Azure Front Door profile from the zone apex. +The Domain Name System (DNS) protocol prevents the assignment of CNAME records at the zone apex. For example, if your domain is `contoso.com`, you can create CNAME records for `somelabel.contoso.com`, but you can't create a CNAME record for `contoso.com` itself. This restriction presents a problem for application owners who load balance applications behind Azure Front Door. Because using an Azure Front Door profile requires creation of a CNAME record, it isn't possible to point at the Azure Front Door profile from the zone apex. -This problem can be resolved by using alias records in Azure DNS. Unlike CNAME records, alias records are created at the zone apex. Application owners can use it to point their zone apex record to an Azure Front Door profile that has public endpoints. Application owners can point to the same Azure Front Door profile used for any other domain within their DNS zone. For example, `contoso.com` and `www.contoso.com` can point to the same Azure Front Door profile. +You can resolve this problem by using alias records in Azure DNS. Unlike CNAME records, alias records are created at the zone apex. Application owners can use it to point their zone apex record to an Azure Front Door profile that has public endpoints. Application owners can point to the same Azure Front Door profile used for any other domain within their DNS zone. For example, `contoso.com` and `www.contoso.com` can point to the same Azure Front Door profile. Mapping your apex or root domain to your Azure Front Door profile requires *CNAME flattening* or *DNS chasing*, which is when the DNS provider recursively resolves CNAME entries until it resolves an IP address. Azure DNS supports this functionality for Azure Front Door endpoints. > [!NOTE]-> There are other DNS providers as well that support CNAME flattening or DNS chasing. However, Azure Front Door recommends using Azure DNS for its customers for hosting their domains. +> Other DNS providers support CNAME flattening or DNS chasing. However, Azure Front Door recommends using Azure DNS for its customers for hosting their domains. -You can use the Azure portal to onboard an apex domain on your Azure Front Door and enable HTTPS on it by associating it with a Transport Layer Security (TLS) certificate. Apex domains are also referred as *root* or *naked* domains. +You can use the Azure portal to onboard an apex domain on your Azure Front Door and enable HTTPS on it by associating it with a Transport Layer Security (TLS) certificate. Apex domains are also referred to as *root* or *naked* domains. ::: zone-end You can use the Azure portal to onboard an apex domain on your Azure Front Door ## Onboard the custom domain to your Azure Front Door profile -1. Select **Domains** from under *Settings* on the left side pane for your Azure Front Door profile and then select **+ Add** to add a new custom domain. +1. Under **Settings**, select **Domains** for your Azure Front Door profile. Then select **+ Add** to add a new custom domain. - :::image type="content" source="./media/front-door-apex-domain/add-domain.png" alt-text="Screenshot of adding a new domain to an Azure Front Door profile."::: + :::image type="content" source="./media/front-door-apex-domain/add-domain.png" alt-text="Screenshot that shows adding a new domain to an Azure Front Door profile."::: -1. On **Add a domain** page, you enter information about the custom domain. You can choose Azure-managed DNS (recommended) or you can choose to use your DNS provider. +1. On the **Add a domain** pane, you enter information about the custom domain. You can choose Azure-managed DNS (recommended), or you can choose to use your DNS provider. - - **Azure-managed DNS** - select an existing DNS zone and for *Custom domain*, select **Add new**. Select **APEX domain** from the pop-up and then select **OK** to save. + - **Azure-managed DNS**: Select an existing DNS zone. For **Custom domain**, select **Add new**. Select **APEX domain** from the pop-up. Then select **OK** to save. - :::image type="content" source="./media/front-door-apex-domain/add-custom-domain.png" alt-text="Screenshot of adding a new custom domain to an Azure Front Door profile."::: + :::image type="content" source="./media/front-door-apex-domain/add-custom-domain.png" alt-text="Screenshot that shows adding a new custom domain to an Azure Front Door profile."::: - - **Another DNS provider** - make sure the DNS provider supports CNAME flattening and follow the steps for [adding a custom domain](standard-premium/how-to-add-custom-domain.md#add-a-new-custom-domain). + - **Another DNS provider**: Make sure the DNS provider supports CNAME flattening and follow the steps for [adding a custom domain](standard-premium/how-to-add-custom-domain.md#add-a-new-custom-domain). -1. Select the **Pending** validation state. A new page appears with DNS TXT record information needed to validate the custom domain. The TXT record is in the form of `_dnsauth.<your_subdomain>`. +1. Select the **Pending** validation state. A new pane appears with the DNS TXT record information needed to validate the custom domain. The TXT record is in the form of `_dnsauth.<your_subdomain>`. - :::image type="content" source="./media/front-door-apex-domain/pending-validation.png" alt-text="Screenshot of custom domain pending validation."::: + :::image type="content" source="./media/front-door-apex-domain/pending-validation.png" alt-text="Screenshot that shows the custom domain Pending validation."::: - - **Azure DNS-based zone** - select the **Add** button to create a new TXT record with the displayed value in the Azure DNS zone. + - **Azure DNS-based zone**: Select **Add** to create a new TXT record with the value that appears in the Azure DNS zone. - :::image type="content" source="./media/front-door-apex-domain/validate-custom-domain.png" alt-text="Screenshot of validate a new custom domain."::: + :::image type="content" source="./media/front-door-apex-domain/validate-custom-domain.png" alt-text="Screenshot that shows validating a new custom domain."::: - - If you're using another DNS provider, manually create a new TXT record of name `_dnsauth.<your_subdomain>` with the record value as shown on the page. + - If you're using another DNS provider, manually create a new TXT record with the name `_dnsauth.<your_subdomain>` with the record value as shown on the pane. -1. Close the *Validate the custom domain* page and return to the *Domains* page for the Azure Front Door profile. You should see the *Validation state* change from **Pending** to **Approved**. If not, wait up to 10 minutes for changes to reflect. If your validation doesn't get approved, make sure your TXT record is correct and name servers are configured correctly if you're using Azure DNS. +1. Close the **Validate the custom domain** pane and return to the **Domains** pane for the Azure Front Door profile. You should see **Validation state** change from **Pending** to **Approved**. If not, wait up to 10 minutes for changes to appear. If your validation doesn't get approved, make sure your TXT record is correct and that name servers are configured correctly if you're using Azure DNS. - :::image type="content" source="./media/front-door-apex-domain/validation-approved.png" alt-text="Screenshot of new custom domain passing validation."::: + :::image type="content" source="./media/front-door-apex-domain/validation-approved.png" alt-text="Screenshot that shows a new custom domain passing validation."::: -1. Select **Unassociated** from the *Endpoint association* column, to add the new custom domain to an endpoint. +1. Select **Unassociated** from the **Endpoint association** column to add the new custom domain to an endpoint. - :::image type="content" source="./media/front-door-apex-domain/unassociated-endpoint.png" alt-text="Screenshot of unassociated custom domain to an endpoint."::: + :::image type="content" source="./media/front-door-apex-domain/unassociated-endpoint.png" alt-text="Screenshot that shows an unassociated custom domain added to an endpoint."::: -1. On the *Associate endpoint and route* page, select the **Endpoint** and **Route** you would like to associate the domain to. Then select **Associate** to complete this step. +1. On the **Associate endpoint and route** pane, select the endpoint and route to which you want to associate the domain. Then select **Associate**. - :::image type="content" source="./media/front-door-apex-domain/associate-endpoint.png" alt-text="Screenshot of associated endpoint and route page for a domain."::: + :::image type="content" source="./media/front-door-apex-domain/associate-endpoint.png" alt-text="Screenshot that shows the associated endpoint and route pane for a domain."::: -1. Under the *DNS state* column, select the **CNAME record is currently not detected** to add the alias record to DNS provider. +1. Under the **DNS state** column, select **CNAME record is currently not detected** to add the alias record to the DNS provider. - - **Azure DNS** - select the **Add** button on the page. + - **Azure DNS**: Select **Add**. - :::image type="content" source="./media/front-door-apex-domain/cname-record.png" alt-text="Screenshot of add or update CNAME record page."::: + :::image type="content" source="./media/front-door-apex-domain/cname-record.png" alt-text="Screenshot that shows the Add or update the CNAME record pane."::: - - **A DNS provider that supports CNAME flattening** - you must manually enter the alias record name. + - **A DNS provider that supports CNAME flattening**: You must manually enter the alias record name. -1. Once the alias record gets created and the custom domain is associated to the Azure Front Door endpoint, traffic starts flowing. +1. After the alias record gets created and the custom domain is associated with the Azure Front Door endpoint, traffic starts flowing. - :::image type="content" source="./media/front-door-apex-domain/cname-record-added.png" alt-text="Screenshot of completed APEX domain configuration."::: + :::image type="content" source="./media/front-door-apex-domain/cname-record-added.png" alt-text="Screenshot that shows the completed APEX domain configuration."::: > [!NOTE]-> * The **DNS state** column is used for CNAME mapping check. Since an apex domain doesnΓÇÖt support a CNAME record, the DNS state will show 'CNAME record is currently not detected' even after you add the alias record to the DNS provider. -> * When placing service like an Azure Web App behind Azure Front Door, you need to configure with the web app with the same domain name as the root domain in Azure Front Door. You also need to configure the backend host header with that domain name to prevent a redirect loop. -> * Apex domains don't have CNAME records pointing to the Azure Front Door profile, therefore managed certificate autorotation will always fail unless domain validation is completed between rotations. +> * The **DNS state** column is used for CNAME mapping check. An apex domain doesn't support a CNAME record, so the DNS state shows **CNAME record is currently not detected** even after you add the alias record to the DNS provider. +> * When you place a service like an Azure Web App behind Azure Front Door, you need to configure the web app with the same domain name as the root domain in Azure Front Door. You also need to configure the back-end host header with that domain name to prevent a redirect loop. +> * Apex domains don't have CNAME records pointing to the Azure Front Door profile. Managed certificate autorotation always fails unless domain validation is finished between rotations. ## Enable HTTPS on your custom domain Follow the guidance for [configuring HTTPS for your custom domain](standard-prem 1. Create or edit the record for zone apex. -1. Select the record **type** as *A* record and then select *Yes* for **Alias record set**. **Alias type** should be set to *Azure resource*. +1. Select the record type as **A**. For **Alias record set**, select **Yes**. Set **Alias type** to **Azure resource**. -1. Select the Azure subscription that contains your Azure Front Door profile. Then select the Azure Front Door resource from the **Azure resource** dropdown. +1. Select the Azure subscription that contains your Azure Front Door profile. Then select the Azure Front Door resource from the **Azure resource** dropdown list. 1. Select **OK** to submit your changes. - :::image type="content" source="./media/front-door-apex-domain/front-door-apex-alias-record.png" alt-text="Alias record for zone apex"::: + :::image type="content" source="./media/front-door-apex-domain/front-door-apex-alias-record.png" alt-text="Screenshot that shows an alias record for zone apex."::: -1. The above step creates a zone apex record pointing to your Azure Front Door resource and also a CNAME record mapping *afdverify* (example - `afdverify.contosonews.com`) that is used for onboarding the domain on your Azure Front Door profile. +1. The preceding step creates a zone apex record that points to your Azure Front Door resource. It also creates a CNAME record mapping **afdverify** (for example, `afdverify.contosonews.com`) that's used for onboarding the domain on your Azure Front Door profile. ## Onboard the custom domain on your Azure Front Door -1. On the Azure Front Door designer tab, select on '+' icon on the Frontend hosts section to add a new custom domain. +1. On the Azure Front Door designer tab, select the **+** icon on the **Frontend hosts** section to add a new custom domain. -1. Enter the root or apex domain name in the custom host name field, example `contosonews.com`. +1. Enter the root or apex domain name in the **Custom host name** field. An example is `contosonews.com`. -1. Once the CNAME mapping from the domain to your Azure Front Door is validated, select on **Add** to add the custom domain. +1. After the CNAME mapping from the domain to your Azure Front Door is validated, select **Add** to add the custom domain. 1. Select **Save** to submit the changes. - :::image type="content" source="./media/front-door-apex-domain/front-door-onboard-apex-domain.png" alt-text="Custom domain menu"::: + :::image type="content" source="./media/front-door-apex-domain/front-door-onboard-apex-domain.png" alt-text="Screenshot that shows the Add a custom domain pane."::: ## Enable HTTPS on your custom domain -1. Select the custom domain that was added and under the section **Custom domain HTTPS**, change the status to **Enabled**. +1. Select the custom domain that was added. Under the section **Custom domain HTTPS**, change the status to **Enabled**. -1. Select the **Certificate management type** to *'Use my own certificate'*. +1. For **Certificate management type**, select **Use my own certificate**. - :::image type="content" source="./media/front-door-apex-domain/front-door-onboard-apex-custom-domain.png" alt-text="Custom domain HTTPS settings"::: + :::image type="content" source="./media/front-door-apex-domain/front-door-onboard-apex-custom-domain.png" alt-text="Screenshot that shows Custom domain HTTPS settings"::: > [!WARNING]- > Azure Front Door managed certificate management type is not currently supported for apex or root domains. The only option available for enabling HTTPS on an apex or root domain for Azure Front Door is using your own custom TLS/SSL certificate hosted on Azure Key Vault. + > An Azure Front Door-managed certificate management type isn't currently supported for apex or root domains. The only option available for enabling HTTPS on an apex or root domain for Azure Front Door is to use your own custom TLS/SSL certificate hosted on Azure Key Vault. -1. Ensure that you have setup the right permissions for Azure Front Door to access your key Vault as noted in the UI, before proceeding to the next step. +1. Ensure that you set up the right permissions for Azure Front Door to access your key vault, as noted in the UI, before you proceed to the next step. -1. Choose a **Key Vault account** from your current subscription and then select the appropriate **Secret** and **Secret version** to map to the right certificate. +1. Choose a **Key Vault account** from your current subscription. Then select the appropriate **Secret** and **Secret version** to map to the right certificate. -1. Select **Update** to save the selection and then Select **Save**. +1. Select **Update** to save the selection. Then select **Save**. -1. Select **Refresh** after a couple of minutes and then select the custom domain again to see the progress of certificate provisioning. +1. Select **Refresh** after a couple of minutes. Then select the custom domain again to see the progress of certificate provisioning. > [!WARNING]-> Ensure that you have created appropriate routing rules for your apex domain or added the domain to existing routing rules. +> Ensure that you created appropriate routing rules for your apex domain or added the domain to existing routing rules. ::: zone-end |
| frontdoor | How To Add Custom Domain | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/frontdoor/standard-premium/how-to-add-custom-domain.md | Title: 'How to add a custom domain - Azure Front Door' -description: In this article, you learn how to onboard a custom domain to Azure Front Door profile using the Azure portal. +description: In this article, you learn how to onboard a custom domain to an Azure Front Door profile by using the Azure portal. Last updated 09/07/2023 -#Customer intent: As a website owner, I want to add a custom domain to my Front Door configuration so that my users can use my custom domain to access my content. +#Customer intent: As a website owner, I want to add a custom domain to my Azure Front Door configuration so that my users can use my custom domain to access my content. -# Configure a custom domain on Azure Front Door using the Azure portal +# Configure a custom domain on Azure Front Door by using the Azure portal -When you use Azure Front Door for application delivery, a custom domain is necessary if you would like your own domain name to be visible in your end-user requests. Having a visible domain name can be convenient for your customers and useful for branding purposes. +When you use Azure Front Door for application delivery, a custom domain is necessary if you want your own domain name to be visible in your user requests. Having a visible domain name can be convenient for your customers and useful for branding purposes. -After you create an Azure Front Door Standard/Premium profile, the default frontend host will have a subdomain of `azurefd.net`. This subdomain gets included in the URL when Azure Front Door Standard/Premium delivers content from your backend by default. For example, `https://contoso-frontend.azurefd.net/activeusers.htm`. For your convenience, Azure Front Door provides the option of associating a custom domain with the default host. With this option, you deliver your content with a custom domain in your URL instead of an Azure Front Door owned domain name. For example, `https://www.contoso.com/photo.png`. +After you create an Azure Front Door Standard/Premium profile, the default front-end host has the subdomain `azurefd.net`. This subdomain gets included in the URL when Azure Front Door Standard/Premium delivers content from your back end by default. An example is `https://contoso-frontend.azurefd.net/activeusers.htm`. -## Prerequisites +For your convenience, Azure Front Door provides the option of associating a custom domain with the default host. With this option, you deliver your content with a custom domain in your URL instead of a domain name that Azure Front Door owns. An example is `https://www.contoso.com/photo.png`. -* Before you can complete the steps in this tutorial, you must first create an Azure Front Door profile. For more information, see [Quickstart: Create a Front Door Standard/Premium](create-front-door-portal.md). +## Prerequisites +* Before you can finish the steps in this tutorial, you must first create an Azure Front Door profile. For more information, see [Quickstart: Create an Azure Front Door Standard/Premium](create-front-door-portal.md). * If you don't already have a custom domain, you must first purchase one with a domain provider. For example, see [Buy a custom domain name](../../app-service/manage-custom-dns-buy-domain.md).- * If you're using Azure to host your [DNS domains](../../dns/dns-overview.md), you must delegate the domain provider's domain name system (DNS) to Azure DNS. For more information, see [Delegate a domain to Azure DNS](../../dns/dns-delegate-domain-azure-dns.md). Otherwise, if you're using a domain provider to handle your DNS domain, you must manually validate the domain by entering prompted DNS TXT records. ## Add a new custom domain After you create an Azure Front Door Standard/Premium profile, the default front > [!NOTE] > If a custom domain is validated in an Azure Front Door or a Microsoft CDN profile already, then it can't be added to another profile. -A custom domain is configured on the **Domains** page of the Azure Front Door profile. A custom domain can be set up and validated prior to endpoint association. A custom domain and its subdomains can only be associated with a single endpoint at a time. However, you can use different subdomains from the same custom domain for different Azure Front Door profiles. You may also map custom domains with different subdomains to the same Azure Front Door endpoint. +A custom domain is configured on the **Domains** pane of the Azure Front Door profile. A custom domain can be set up and validated before endpoint association. A custom domain and its subdomains can only be associated with a single endpoint at a time. However, you can use different subdomains from the same custom domain for different Azure Front Door profiles. You can also map custom domains with different subdomains to the same Azure Front Door endpoint. -1. Select **Domains** under settings for your Azure Front Door profile and then select **+ Add** button. +1. Under **Settings**, select **Domains** for your Azure Front Door profile. Then select **+ Add**. - :::image type="content" source="../media/how-to-add-custom-domain/add-domain-button.png" alt-text="Screenshot of add domain button on domain landing page."::: + :::image type="content" source="../media/how-to-add-custom-domain/add-domain-button.png" alt-text="Screenshot that shows the Add a domain button on the domain landing pane."::: -1. On the *Add a domain* page, select the **Domain type**. You can select between a **Non-Azure validated domain** or an **Azure pre-validated domain**. +1. On the **Add a domain** pane, select the domain type. You can choose **Non-Azure validated domain** or **Azure pre-validated domain**. - * **Non-Azure validated domain** is a domain that requires ownership validation. When you select Non-Azure validated domain, the recommended DNS management option is to use Azure-managed DNS. You may also use your own DNS provider. If you choose Azure-managed DNS, select an existing DNS zone. Then select an existing custom subdomain or create a new one. If you're using another DNS provider, manually enter the custom domain name. Then select **Add** to add your custom domain. + * **Non-Azure validated domain** is a domain that requires ownership validation. When you select **Non-Azure validated domain**, we recommend that you use the Azure-managed DNS option. You might also use your own DNS provider. If you choose an Azure-managed DNS, select an existing DNS zone. Then select an existing custom subdomain or create a new one. If you're using another DNS provider, manually enter the custom domain name. Then select **Add** to add your custom domain. - :::image type="content" source="../media/how-to-add-custom-domain/add-domain-page.png" alt-text="Screenshot of add a domain page."::: + :::image type="content" source="../media/how-to-add-custom-domain/add-domain-page.png" alt-text="Screenshot that shows the Add a domain pane."::: - * **Azure pre-validated domain** is a domain already validated by another Azure service. When you select this option, domain ownership validation isn't required from Azure Front Door. A dropdown list of validated domains by different Azure services appear. + * **Azure pre-validated domain** is a domain already validated by another Azure service. When you select this option, domain ownership validation isn't required from Azure Front Door. A dropdown list of validated domains by different Azure services appears. - :::image type="content" source="../media/how-to-add-custom-domain/pre-validated-custom-domain.png" alt-text="Screenshot of prevalidated custom domain in add a domain page."::: + :::image type="content" source="../media/how-to-add-custom-domain/pre-validated-custom-domain.png" alt-text="Screenshot that shows Pre-validated custom domains on the Add a domain pane."::: > [!NOTE]- > * Azure Front Door supports both Azure managed certificate and Bring Your Own Certificates. For Non-Azure validated domain, the Azure managed certificate is issued and managed by the Azure Front Door. For Azure pre-validated domain, the Azure managed certificate gets issued and is managed by the Azure service that validates the domain. To use own certificate, see [Configure HTTPS on a custom domain](how-to-configure-https-custom-domain.md). - > * Azure Front Door supports Azure pre-validated domains and Azure DNS zones in different subscriptions. - > * Currently Azure pre-validated domains only supports domains validated by Static Web App. + > * Azure Front Door supports both Azure-managed certificates and Bring Your Own Certificates (BYOCs). For a non-Azure validated domain, the Azure-managed certificate is issued and managed by Azure Front Door. For an Azure prevalidated domain, the Azure-managed certificate gets issued and is managed by the Azure service that validates the domain. To use your own certificate, see [Configure HTTPS on a custom domain](how-to-configure-https-custom-domain.md). + > * Azure Front Door supports Azure prevalidated domains and Azure DNS zones in different subscriptions. + > * Currently, Azure prevalidated domains only support domains validated by Azure Static Web Apps. A new custom domain has a validation state of **Submitting**. - :::image type="content" source="../media/how-to-add-custom-domain/validation-state-submitting.png" alt-text="Screenshot of domain validation state submitting."::: + :::image type="content" source="../media/how-to-add-custom-domain/validation-state-submitting.png" alt-text="Screenshot that shows the domain validation state as Submitting."::: > [!NOTE]- > * Starting September 2023, Azure Front Door supports Bring Your Own Certificates (BYOC) based domain ownership validation. Front Door will automatically approve the domain ownership so long as the Certificate Name (CN) or Subject Alternative Name (SAN) of provided certificate matches the custom domain. When you select Azure managed certificate, the domain ownership will continue to be valdiated via the DNS TXT record. - > * For custom domains created before BYOC based validation is supported and the domain validation status is anything but **Approved**, you need to trigger the auto approval of the domain ownership validation by selecting the **Validation State** and then click on the **Revalidate** button in the portal. If you're using the command line tool, you can trigger domain validation by sending an empty PATCH request to the domain API. - > * An Azure pre-validated domain will have a validation state of **Pending** and will automatically change to **Approved** after a few minutes. Once validation gets approved, skip to [**Associate the custom domain to your Front Door endpoint**](#associate-the-custom-domain-with-your-azure-front-door-endpoint) and complete the remaining steps. + > * As of September 2023, Azure Front Door now supports BYOC-based domain ownership validation. Azure Front Door automatically approves the domain ownership if the Certificate Name (CN) or Subject Alternative Name (SAN) of the provided certificate matches the custom domain. When you select **Azure managed certificate**, the domain ownership continues to be validated via the DNS TXT record. + > * For custom domains created before BYOC-based validation is supported and the domain validation status is anything but **Approved**, you need to trigger the auto-approval of the domain ownership validation by selecting **Validation State** > **Revalidate** in the portal. If you're using the command-line tool, you can trigger domain validation by sending an empty `PATCH` request to the domain API. + > * An Azure prevalidated domain has a validation state of **Pending**. It automatically changes to **Approved** after a few minutes. After validation gets approved, skip to [Associate the custom domain to your Front Door endpoint](#associate-the-custom-domain-with-your-azure-front-door-endpoint) and finish the remaining steps. - The validation state will change to **Pending** after a few minutes. + After a few minutes, the validation state changes to **Pending**. - :::image type="content" source="../media/how-to-add-custom-domain/validation-state-pending.png" alt-text="Screenshot of domain validation state pending."::: + :::image type="content" source="../media/how-to-add-custom-domain/validation-state-pending.png" alt-text="Screenshot that shows the domain validation state as Pending."::: -1. Select the **Pending** validation state. A new page appears with DNS TXT record information needed to validate the custom domain. The TXT record is in the form of `_dnsauth.<your_subdomain>`. If you're using Azure DNS-based zone, select the **Add** button, and a new TXT record with the displayed record value gets created in the Azure DNS zone. If you're using another DNS provider, manually create a new TXT record of name `_dnsauth.<your_subdomain>` with the record value as shown on the page. +1. Select the **Pending** validation state. A new pane appears with DNS TXT record information that's needed to validate the custom domain. The TXT record is in the form of `_dnsauth.<your_subdomain>`. If you're using an Azure DNS-based zone, select **Add**. A new TXT record with the record value that appears is created in the Azure DNS zone. If you're using another DNS provider, manually create a new TXT record named `_dnsauth.<your_subdomain>`, with the record value as shown on the pane. - :::image type="content" source="../media/how-to-add-custom-domain/validate-custom-domain.png" alt-text="Screenshot of validate custom domain page."::: + :::image type="content" source="../media/how-to-add-custom-domain/validate-custom-domain.png" alt-text="Screenshot that shows the Validate the custom domain pane."::: -1. Close the page to return to custom domains list landing page. The provisioning state of custom domain should change to **Provisioned** and validation state should change to **Approved**. +1. Close the pane to return to the custom domains list landing pane. The provisioning state of the custom domain should change to **Provisioned**. The validation state should change to **Approved**. - :::image type="content" source="../media/how-to-add-custom-domain/provisioned-approved-status.png" alt-text="Screenshot of provisioned and approved status."::: + :::image type="content" source="../media/how-to-add-custom-domain/provisioned-approved-status.png" alt-text="Screenshot that shows the Provisioning state and the Approved status."::: For more information about domain validation states, see [Domains in Azure Front Door](../domain.md#domain-validation). ## Associate the custom domain with your Azure Front Door endpoint -After you validate your custom domain, you can associate it to your Azure Front Door Standard/Premium endpoint. +After you validate your custom domain, you can associate it with your Azure Front Door Standard/Premium endpoint. -1. Select the **Unassociated** link to open the **Associate endpoint and routes** page. Select an endpoint and routes you want to associate the domain with. Then select **Associate** to update your configuration. +1. Select the **Unassociated** link to open the **Associate endpoint and routes** pane. Select an endpoint and the routes with which you want to associate the domain. Then select **Associate** to update your configuration. - :::image type="content" source="../media/how-to-add-custom-domain/associate-endpoint-routes.png" alt-text="Screenshot of associate endpoint and routes page."::: + :::image type="content" source="../media/how-to-add-custom-domain/associate-endpoint-routes.png" alt-text="Screenshot that shows the Associate endpoint and routes pane."::: - The Endpoint association status should change to reflect the endpoint to which the custom domain is currently associated. + The **Endpoint association** status should change to reflect the endpoint to which the custom domain is currently associated. - :::image type="content" source="../media/how-to-add-custom-domain/endpoint-association-status.png" alt-text="Screenshot of endpoint association link."::: + :::image type="content" source="../media/how-to-add-custom-domain/endpoint-association-status.png" alt-text="Screenshot that shows the Endpoint association link."::: -1. Select the DNS state link. +1. Select the **DNS state** link. - :::image type="content" source="../media/how-to-add-custom-domain/dns-state-link.png" alt-text="Screenshot of DNS state link."::: + :::image type="content" source="../media/how-to-add-custom-domain/dns-state-link.png" alt-text="Screenshot that shows the DNS state link."::: > [!NOTE]- > For an Azure pre-validated domain, go to the DNS hosting service and manually update the CNAME record for this domain from the other Azure service endpoint to Azure Front Door endpoint. This step is required, regardless of whether the domain is hosted with Azure DNS or with another DNS service. The link to update the CNAME from the DNS State column isn't available for this type of domain. + > For an Azure prevalidated domain, go to the DNS hosting service and manually update the CNAME record for this domain from the other Azure service endpoint to Azure Front Door endpoint. This step is required, regardless of whether the domain is hosted with Azure DNS or with another DNS service. The link to update the CNAME from the **DNS state** column isn't available for this type of domain. -1. The **Add or update the CNAME record** page appears and displays the CNAME record information that must be provided before traffic can start flowing. If you're using Azure DNS hosted zones, the CNAME records can be created by selecting the **Add** button on the page. If you're using another DNS provider, you must manually enter the CNAME record name and value as shown on the page. +1. The **Add or update the CNAME record** pane appears with the CNAME record information that must be provided before traffic can start flowing. If you're using Azure DNS hosted zones, the CNAME records can be created by selecting **Add** on the pane. If you're using another DNS provider, you must manually enter the CNAME record name and value as shown on the pane. - :::image type="content" source="../media/how-to-add-custom-domain/add-update-cname-record.png" alt-text="Screenshot of add or update CNAME record."::: + :::image type="content" source="../media/how-to-add-custom-domain/add-update-cname-record.png" alt-text="Screenshot that shows the Add or update the CNAME record pane."::: -1. Once the CNAME record gets created and the custom domain is associated to the Azure Front Door endpoint, traffic starts flowing. +1. After the CNAME record is created and the custom domain is associated with the Azure Front Door endpoint, traffic starts flowing. > [!NOTE]- > * If HTTPS is enabled, certificate provisioning and propagation may take a few minutes because propagation is being done to all edge locations. - > * If your domain CNAME is indirectly pointed to a Front Door endpoint, for example, using Azure Traffic Manager for multi-CDN failover, the **DNS state** column shows as **CNAME/Alias record currently not detected**. Azure Front Door can't guarantee 100% detection of the CNAME record in this case. If you've configured an Azure Front Door endpoint to Azure Traffic Manager and still see this message, it doesnΓÇÖt mean you didn't set up correctly, therefore further no action is necessary from your side. + > * If HTTPS is enabled, certificate provisioning and propagation might take a few minutes because propagation is being done to all edge locations. + > * If your domain CNAME is indirectly pointed to an Azure Front Door endpoint, for example, by using Azure Traffic Manager for multi-CDN failover, the **DNS state** column shows as **CNAME/Alias record currently not detected**. Azure Front Door can't guarantee 100% detection of the CNAME record in this case. If you configured an Azure Front Door endpoint to Traffic Manager and still see this message, it doesn't mean that you didn't set up correctly. No further action is necessary from your side. ## Verify the custom domain -After you've validated and associated the custom domain, verify that the custom domain is correctly referenced to your endpoint. +After you validate and associate the custom domain, verify that the custom domain is correctly referenced to your endpoint. -Lastly, validate that your application content is getting served using a browser. +Lastly, validate that your application content is getting served by using a browser. ## Next steps * Learn how to [enable HTTPS for your custom domain](how-to-configure-https-custom-domain.md). * Learn more about [custom domains in Azure Front Door](../domain.md).-* Learn about [End-to-end TLS with Azure Front Door](../end-to-end-tls.md). +* Learn about [end-to-end TLS with Azure Front Door](../end-to-end-tls.md). |
| frontdoor | How To Configure Https Custom Domain | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/frontdoor/standard-premium/how-to-configure-https-custom-domain.md | Title: 'Configure HTTPS for your custom domain - Azure Front Door' -description: In this article, you'll learn how to configure HTTPS on an Azure Front Door custom domain. +description: In this article, you learn how to configure HTTPS on an Azure Front Door custom domain by using the Azure portal. -#Customer intent: As a website owner, I want to add a custom domain to my Front Door configuration so that my users can use my custom domain to access my content. +#Customer intent: As a website owner, I want to add a custom domain to my Azure Front Door configuration so that my users can use my custom domain to access my content. -# Configure HTTPS on an Azure Front Door custom domain using the Azure portal +# Configure HTTPS on an Azure Front Door custom domain by using the Azure portal -Azure Front Door enables secure TLS delivery to your applications by default when you use your own custom domains. To learn more about custom domains, including how custom domains work with HTTPS, see [Domains in Azure Front Door](../domain.md). +Azure Front Door enables secure Transport Layer Security (TLS) delivery to your applications by default when you use your own custom domains. To learn more about custom domains, including how custom domains work with HTTPS, see [Domains in Azure Front Door](../domain.md). -Azure Front Door supports Azure-managed certificates and customer-managed certificates. In this article, you'll learn how to configure both types of certificates for your Azure Front Door custom domains. +Azure Front Door supports Azure-managed certificates and customer-managed certificates. In this article, you learn how to configure both types of certificates for your Azure Front Door custom domains. ## Prerequisites * Before you can configure HTTPS for your custom domain, you must first create an Azure Front Door profile. For more information, see [Create an Azure Front Door profile](../create-front-door-portal.md).- * If you don't already have a custom domain, you must first purchase one with a domain provider. For example, see [Buy a custom domain name](../../app-service/manage-custom-dns-buy-domain.md).- * If you're using Azure to host your [DNS domains](../../dns/dns-overview.md), you must delegate the domain provider's domain name system (DNS) to an Azure DNS. For more information, see [Delegate a domain to Azure DNS](../../dns/dns-delegate-domain-azure-dns.md). Otherwise, if you're using a domain provider to handle your DNS domain, you must manually validate the domain by entering prompted DNS TXT records. -## Azure Front Door-managed certificates for non-Azure pre-validated domains +## Azure Front Door-managed certificates for non-Azure prevalidated domains -Follow the steps below if you have your own domain, and the domain is not already associated with [another Azure service that pre-validates domains for Azure Front Door](../domain.md#domain-validation). +If you have your own domain, and the domain isn't already associated with [another Azure service that prevalidates domains for Azure Front Door](../domain.md#domain-validation), follow these steps: -1. Select **Domains** under settings for your Azure Front Door profile and then select **+ Add** to add a new domain. +1. Under **Settings**, select **Domains** for your Azure Front Door profile. Then select **+ Add** to add a new domain. - :::image type="content" source="../media/how-to-configure-https-custom-domain/add-new-custom-domain.png" alt-text="Screenshot of domain configuration landing page."::: + :::image type="content" source="../media/how-to-configure-https-custom-domain/add-new-custom-domain.png" alt-text="Screenshot that shows the domain configuration landing pane."::: -1. On the **Add a domain** page, enter or select the following information, then select **Add** to onboard the custom domain. +1. On the **Add a domain** pane, enter or select the following information. Then select **Add** to onboard the custom domain. - :::image type="content" source="../media/how-to-configure-https-custom-domain/add-domain-azure-managed.png" alt-text="Screenshot of add a domain page with Azure managed DNS selected."::: + :::image type="content" source="../media/how-to-configure-https-custom-domain/add-domain-azure-managed.png" alt-text="Screenshot that shows the Add a domain pane with Azure managed DNS selected."::: | Setting | Value | |--|--|- | Domain type | Select **Non-Azure pre-validated domain** | - | DNS management | Select **Azure managed DNS (Recommended)** | - | DNS zone | Select the **Azure DNS zone** that host the custom domain. | + | Domain type | Select **Non-Azure pre-validated domain**. | + | DNS management | Select **Azure managed DNS (Recommended)**. | + | DNS zone | Select the Azure DNS zone that hosts the custom domain. | | Custom domain | Select an existing domain or add a new domain. |- | HTTPS | Select **AFD Managed (Recommended)** | + | HTTPS | Select **AFD managed (Recommended)**. | -1. Validate and associate the custom domain to an endpoint by following the steps in enabling [custom domain](how-to-add-custom-domain.md). +1. Validate and associate the custom domain to an endpoint by following the steps to enable a [custom domain](how-to-add-custom-domain.md). -1. After the custom domain is associated with an endpoint successfully, Azure Front Door generates a certificate and deploys it. This process may take from several minutes to an hour to complete. +1. After the custom domain is successfully associated with an endpoint, Azure Front Door generates a certificate and deploys it. This process might take from several minutes to an hour to finish. -## Azure-managed certificates for Azure pre-validated domains +## Azure-managed certificates for Azure prevalidated domains -Follow the steps below if you have your own domain, and the domain is associated with [another Azure service that pre-validates domains for Azure Front Door](../domain.md#domain-validation). +If you have your own domain, and the domain is associated with [another Azure service that prevalidates domains for Azure Front Door](../domain.md#domain-validation), follow these steps: -1. Select **Domains** under settings for your Azure Front Door profile and then select **+ Add** to add a new domain. +1. Under **Settings**, select **Domains** for your Azure Front Door profile. Then select **+ Add** to add a new domain. - :::image type="content" source="../media/how-to-configure-https-custom-domain/add-new-custom-domain.png" alt-text="Screenshot of domain configuration landing page."::: + :::image type="content" source="../media/how-to-configure-https-custom-domain/add-new-custom-domain.png" alt-text="Screenshot that shows the Domains landing pane."::: -1. On the **Add a domain** page, enter or select the following information, then select **Add** to onboard the custom domain. +1. On the **Add a domain** pane, enter or select the following information. Then select **Add** to onboard the custom domain. - :::image type="content" source="../media/how-to-configure-https-custom-domain/add-pre-validated-domain.png" alt-text="Screenshot of add a domain page with pre-validated domain."::: + :::image type="content" source="../media/how-to-configure-https-custom-domain/add-pre-validated-domain.png" alt-text="Screenshot that shows the Add a domain pane with a prevalidated domain."::: | Setting | Value | |--|--|- | Domain type | Select **Azure pre-validated domain** | - | Pre-validated custom domain | Select a custom domain name from the drop-down list of Azure services. | - | HTTPS | Select **Azure managed (Recommended)** | + | Domain type | Select **Azure pre-validated domain**. | + | Pre-validated custom domains | Select a custom domain name from the dropdown list of Azure services. | + | HTTPS | Select **Azure managed**. | -1. Validate and associate the custom domain to an endpoint by following the steps in enabling [custom domain](how-to-add-custom-domain.md). +1. Validate and associate the custom domain to an endpoint by following the steps to enable a [custom domain](how-to-add-custom-domain.md). -1. Once the custom domain gets associated to endpoint successfully, an AFD managed certificate gets deployed to Front Door. This process may take from several minutes to an hour to complete. +1. After the custom domain is successfully associated with an endpoint, an Azure Front Door-managed certificate gets deployed to Azure Front Door. This process might take from several minutes to an hour to finish. -## Using your own certificate +## Use your own certificate You can also choose to use your own TLS certificate. Your TLS certificate must meet certain requirements. For more information, see [Certificate requirements](../domain.md?pivot=front-door-standard-premium#certificate-requirements). #### Prepare your key vault and certificate -We recommend you create a separate Azure Key Vault to store your Azure Front Door TLS certificates. For more information, see [create an Azure Key Vault](../../key-vault/general/quick-create-portal.md). If you already a certificate, you can upload it to your new Azure Key Vault. Otherwise, you can create a new certificate through Azure Key Vault from one of the certificate authorities (CAs) partners. +We recommend that you create a separate Azure Key Vault instance in which to store your Azure Front Door TLS certificates. For more information, see [Create a Key Vault instance](../../key-vault/general/quick-create-portal.md). If you already have a certificate, you can upload it to your new Key Vault instance. Otherwise, you can create a new certificate through Key Vault from one of the certificate authority (CA) partners. > [!WARNING]-> Azure Front Door currently only supports Azure Key Vault in the same subscription. Selecting an Azure Key Vault under a different subscription will result in a failure. +> Azure Front Door currently only supports Key Vault in the same subscription. Selecting Key Vault under a different subscription results in a failure. -> [!NOTE] -> * Azure Front Door doesn't support certificates with elliptic curve (EC) cryptography algorithms. Also, your certificate must have a complete certificate chain with leaf and intermediate certificates, and also the root certification authority (CA) must be part of the [Microsoft Trusted CA List](https://ccadb-public.secure.force.com/microsoft/IncludedCACertificateReportForMSFT). -> * We recommend using [**managed identity**](../managed-identity.md) to allow access to your Azure Key Vault certificates because App registration will be retired in the future. +Other points to note about certificates: ++* Azure Front Door doesn't support certificates with elliptic curve cryptography algorithms. Also, your certificate must have a complete certificate chain with leaf and intermediate certificates. The root CA also must be part of the [Microsoft Trusted CA List](https://ccadb-public.secure.force.com/microsoft/IncludedCACertificateReportForMSFT). +* We recommend that you use [managed identity](../managed-identity.md) to allow access to your Key Vault certificates because app registration will be retired in the future. #### Register Azure Front Door Register the service principal for Azure Front Door as an app in your Microsoft Entra ID by using Azure PowerShell or the Azure CLI. > [!NOTE]-> * This action requires you to have *Global Administrator* permissions in Microsoft Entra ID. The registration only needs to be performed **once per Microsoft Entra tenant**. -> * The application ID of **205478c0-bd83-4e1b-a9d6-db63a3e1e1c8** and **d4631ece-daab-479b-be77-ccb713491fc0** is predefined by Azure for Front Door Standard and Premium across all Azure tenants and subscriptions. Azure Front Door (Classic) has a different application ID. +> * This action requires you to have Global Administrator permissions in Microsoft Entra ID. The registration only needs to be performed *once per Microsoft Entra tenant*. +> * The application IDs of **205478c0-bd83-4e1b-a9d6-db63a3e1e1c8** and **d4631ece-daab-479b-be77-ccb713491fc0** are predefined by Azure for Azure Front Door Standard and Premium across all Azure tenants and subscriptions. Azure Front Door (classic) has a different application ID. # [Azure PowerShell](#tab/powershell) 1. If needed, install [Azure PowerShell](/powershell/azure/install-azure-powershell) in PowerShell on your local machine. -1. Use PowerShell, run the following command: +1. Use PowerShell to run the following command: - **Azure public cloud:** + Azure public cloud: ```azurepowershell-interactive New-AzADServicePrincipal -ApplicationId '205478c0-bd83-4e1b-a9d6-db63a3e1e1c8' ``` - **Azure government cloud:** + Azure government cloud: ```azurepowershell-interactive New-AzADServicePrincipal -ApplicationId 'd4631ece-daab-479b-be77-ccb713491fc0' Register the service principal for Azure Front Door as an app in your Microsoft # [Azure CLI](#tab/cli) -1. If needed, install [Azure CLI](/cli/azure/install-azure-cli) on your local machine. +1. If needed, install the [Azure CLI](/cli/azure/install-azure-cli) on your local machine. 1. Use the Azure CLI to run the following command: - **Azure public cloud:** + Azure public cloud: ```azurecli-interactive az ad sp create --id 205478c0-bd83-4e1b-a9d6-db63a3e1e1c8 ``` - **Azure government cloud:** + Azure government cloud: ```azurecli-interactive az ad sp create --id d4631ece-daab-479b-be77-ccb713491fc0 ``` -#### Grant Azure Front Door access to your Key Vault +#### Grant Azure Front Door access to your key vault -Grant Azure Front Door permission to access the certificates in your Azure Key Vault account. You only need to give **GET** permission to the certificate and secret in order for Azure Front Door to retrieve the certificate. +Grant Azure Front Door permission to access the certificates in your Key Vault account. You only need to give `GET` permission to the certificate and secret in order for Azure Front Door to retrieve the certificate. -1. In your key vault account, select **Access policies**. +1. In your Key Vault account, select **Access policies**. 1. Select **Add new** or **Create** to create a new access policy. -1. In **Secret permissions**, select **Get** to allow Front Door to retrieve the certificate. +1. In **Secret permissions**, select **Get** to allow Azure Front Door to retrieve the certificate. -1. In **Certificate permissions**, select **Get** to allow Front Door to retrieve the certificate. +1. In **Certificate permissions**, select **Get** to allow Azure Front Door to retrieve the certificate. -1. In **Select principal**, search for **205478c0-bd83-4e1b-a9d6-db63a3e1e1c8**, and select **Microsoft.AzureFrontDoor-Cdn**. Select **Next**. +1. In **Select principal**, search for **205478c0-bd83-4e1b-a9d6-db63a3e1e1c8** and select **Microsoft.AzureFrontDoor-Cdn**. Select **Next**. 1. In **Application**, select **Next**. Azure Front Door can now access this key vault and the certificates it contains. 1. Return to your Azure Front Door Standard/Premium in the portal. -1. Navigate to **Secrets** under *Settings* and select **+ Add certificate**. +1. Under **Settings**, go to **Secrets** and select **+ Add certificate**. - :::image type="content" source="../media/how-to-configure-https-custom-domain/add-certificate.png" alt-text="Screenshot of Azure Front Door secret landing page."::: + :::image type="content" source="../media/how-to-configure-https-custom-domain/add-certificate.png" alt-text="Screenshot that shows the Azure Front Door secret landing pane."::: -1. On the **Add certificate** page, select the checkbox for the certificate you want to add to Azure Front Door Standard/Premium. +1. On the **Add certificate** pane, select the checkbox for the certificate you want to add to Azure Front Door Standard/Premium. -1. When you select a certificate, you must [select the certificate version](../domain.md#rotate-own-certificate). If you select **Latest**, Azure Front Door will automatically update whenever the certificate is rotated (renewed). Alternatively, you can select a specific certificate version if you prefer to manage certificate rotation yourself. +1. When you select a certificate, you must [select the certificate version](../domain.md#rotate-own-certificate). If you select **Latest**, Azure Front Door automatically updates whenever the certificate is rotated (renewed). You can also select a specific certificate version if you prefer to manage certificate rotation yourself. - Leave the version selection as "Latest" and select **Add**. + Leave the version selection as **Latest** and select **Add**. - :::image type="content" source="../media/how-to-configure-https-custom-domain/add-certificate-page.png" alt-text="Screenshot of add certificate page."::: + :::image type="content" source="../media/how-to-configure-https-custom-domain/add-certificate-page.png" alt-text="Screenshot that shows the Add certificate pane."::: -1. Once the certificate gets provisioned successfully, you can use it when you add a new custom domain. +1. After the certificate gets provisioned successfully, you can use it when you add a new custom domain. - :::image type="content" source="../media/how-to-configure-https-custom-domain/successful-certificate-provisioned.png" alt-text="Screenshot of certificate successfully added to secrets."::: + :::image type="content" source="../media/how-to-configure-https-custom-domain/successful-certificate-provisioned.png" alt-text="Screenshot that shows the certificate successfully added to secrets."::: -1. Navigate to **Domains** under *Setting* and select **+ Add** to add a new custom domain. On the **Add a domain** page, choose -"Bring Your Own Certificate (BYOC)" for *HTTPS*. For *Secret*, select the certificate you want to use from the drop-down. +1. Under **Settings**, go to **Domains** and select **+ Add** to add a new custom domain. On the **Add a domain** pane, for **HTTPS**, select **Bring Your Own Certificate (BYOC)**. For **Secret**, select the certificate you want to use from the dropdown list. > [!NOTE]- > The common name (CN) of the selected certificate must match the custom domain being added. + > The common name of the selected certificate must match the custom domain being added. - :::image type="content" source="../media/how-to-configure-https-custom-domain/add-custom-domain-https.png" alt-text="Screenshot of add a custom domain page with HTTPS."::: + :::image type="content" source="../media/how-to-configure-https-custom-domain/add-custom-domain-https.png" alt-text="Screenshot that shows the Add a custom domain pane with HTTPS."::: -1. Follow the on-screen steps to validate the certificate. Then associate the newly created custom domain to an endpoint as outlined in [creating a custom domain](how-to-add-custom-domain.md) guide. +1. Follow the onscreen steps to validate the certificate. Then associate the newly created custom domain to an endpoint as outlined in [Configure a custom domain](how-to-add-custom-domain.md). ## Switch between certificate types You can change a domain between using an Azure Front Door-managed certificate and a customer-managed certificate. For more information, see [Domains in Azure Front Door](../domain.md#switch-between-certificate-types). -1. Select the certificate state to open the **Certificate details** page. +1. Select the certificate state to open the **Certificate details** pane. - :::image type="content" source="../media/how-to-configure-https-custom-domain/domain-certificate.png" alt-text="Screenshot of certificate state on domains landing page."::: + :::image type="content" source="../media/how-to-configure-https-custom-domain/domain-certificate.png" alt-text="Screenshot that shows the certificate state on the Domains landing pane."::: -1. On the **Certificate details** page, you can change between *Azure managed* and *Bring Your Own Certificate (BYOC)*. +1. On the **Certificate details** pane, you can change between **Azure Front Door managed** and **Bring Your Own Certificate (BYOC)**. - If you select *Bring Your Own Certificate (BYOC)*, follow the steps described above to select a certificate. + If you select **Bring Your Own Certificate (BYOC)**, follow the preceding steps to select a certificate. 1. Select **Update** to change the associated certificate with a domain. - :::image type="content" source="../media/how-to-configure-https-custom-domain/certificate-details-page.png" alt-text="Screenshot of certificate details page."::: + :::image type="content" source="../media/how-to-configure-https-custom-domain/certificate-details-page.png" alt-text="Screenshot that shows the Certificate details pane."::: ## Next steps * Learn about [caching with Azure Front Door Standard/Premium](../front-door-caching.md). * [Understand custom domains](../domain.md) on Azure Front Door.-* Learn about [End-to-end TLS with Azure Front Door](../end-to-end-tls.md). +* Learn about [end-to-end TLS with Azure Front Door](../end-to-end-tls.md). |
| governance | Definition Structure Policy Rule | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/governance/policy/concepts/definition-structure-policy-rule.md | In the `then` block, you define the effect that happens when the `if conditions For more information about _policyRule_, go to the [policy definition schema](https://schema.management.azure.com/schemas/2020-10-01/policyDefinition.json). -### Logical operators +## Logical operators Supported logical operators are: |
| hdinsight-aks | Azure Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/hdinsight-aks/flink/azure-iot-hub.md | Title: Process real-time IoT data on Apache Flink® with Azure HDInsight on AKS -description: How to integrate Azure IoT Hub and Apache Flink® +description: How to integrate Azure IoT Hub and Apache Flink®. Previously updated : 10/03/2023 Last updated : 04/04/2024 # Process real-time IoT data on Apache Flink® with Azure HDInsight on AKS Azure IoT Hub is a managed service hosted in the cloud that acts as a central message hub for communication between an IoT application and its attached devices. You can connect millions of devices and their backend solutions reliably and securely. Almost any device can be connected to an IoT hub. -## Prerequisites --1. [Create an Azure IoTHub](/azure/iot-hub/iot-hub-create-through-portal/) -2. [Create Flink cluster on HDInsight on AKS](./flink-create-cluster-portal.md) +In this example, the code processes real-time IoT data on Apache Flink® with Azure HDInsight on AKS and sinks to ADLS gen2 storage. -## Configure Flink cluster +## Prerequisites -Add ABFS storage account keys in your Flink cluster's configuration. +* [Create an Azure IoTHub](/azure/iot-hub/iot-hub-create-through-portal/) +* [Create Flink cluster 1.17.0 on HDInsight on AKS](./flink-create-cluster-portal.md) +* Use MSI to access ADLS Gen2 +* IntelliJ for development -Add the following configurations: +> [!NOTE] +> For this demonstration, we are using a Window VM as maven project develop env in the same VNET as HDInsight on AKS. -`fs.azure.account.key.<your storage account's dfs endpoint> = <your storage account's shared access key>` +## Flink cluster 1.17.0 on HDInsight on AKS :::image type="content" source="./media/azure-iot-hub/configuration-management.png" alt-text="Diagram showing search bar in Azure portal." lightbox="./media/azure-iot-hub/configuration-management.png"::: -## Writing the Flink job --### Set up configuration for ABFS --```java -Properties props = new Properties(); -props.put( - "fs.azure.account.key.<your storage account's dfs endpoint>", - "<your storage account's shared access key>" -); --Configuration conf = ConfigurationUtils.createConfiguration(props); +## Azure IOT Hub on Azure portal -StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(conf); +Within the connection string, you can find a service bus URL (URL of the underlying event hub namespace), which you need to add as a bootstrap server in your Kafka source. In this example, it's `iothub-ns-contosoiot-55642726-4642a54853.servicebus.windows.net:9093`. -``` +## Prepare message into Azure IOT device -This set up is required for Flink to authenticate with your ABFS storage account to write data to it. +Each IoT hub comes with built-in system endpoints to handle system and device messages. -### Defining the IoT Hub source +For more information, see [How to use VS Code as IoT Hub Device Simulator](https://devblogs.microsoft.com/iotdev/use-vs-code-as-iot-hub-device-simulator-say-hello-to-azure-iot-hub-in-5-minutes/). -IoTHub is build on top of event hub and hence supports a kafka-like API. So in our Flink job, we can define a `KafkaSource` with appropriate parameters to consume messages from IoTHub. -```java -String connectionString = "<your iot hub connection string>"; -KafkaSource<String> source = KafkaSource.<String>builder() - .setBootstrapServers("<your iot hub's service bus url>:9093") - .setTopics("<name of your iot hub>") - .setGroupId("$Default") - .setProperty("partition.discovery.interval.ms", "10000") - .setProperty("security.protocol", "SASL_SSL") - .setProperty("sasl.mechanism", "PLAIN") - .setProperty("sasl.jaas.config", String.format("org.apache.kafka.common.security.plain.PlainLoginModule required username=\"$ConnectionString\" password=\"%s\";", connectionString)) - .setStartingOffsets(OffsetsInitializer.committedOffsets(OffsetResetStrategy.EARLIEST)) - .setValueOnlyDeserializer(new SimpleStringSchema()) - .build(); +## Code in Flink -DataStream<String> kafka = env.fromSource(source, WatermarkStrategy.noWatermarks(), "Kafka Source"); -kafka.print(); -``` +`IOTdemo.java` -The connection string for IoT Hub can be found here - -+- KafkaSource: +IoTHub is build on top of event hub and hence supports a kafka-like API. So in our Flink job, we can define a KafkaSource with appropriate parameters to consume messages from IoTHub. -Within the connection string, you can find a service bus URL (URL of the underlying event hub namespace), which you need to add as a bootstrap server in your kafka source. In this case, it is: `iothub-ns-sagiri-iot-25146639-20dff4e426.servicebus.windows.net:9093` +- FileSink: +Define the ABFS sink. -### Defining the ABFS sink -```java -String outputPath = "abfs://<container name>@<your storage account's dfs endpoint>"; --final FileSink<String> sink = FileSink - .forRowFormat(new Path(outputPath), new SimpleStringEncoder<String>("UTF-8")) - .withRollingPolicy( - DefaultRollingPolicy.builder() - .withRolloverInterval(Duration.ofMinutes(2)) - .withInactivityInterval(Duration.ofMinutes(3)) - .withMaxPartSize(MemorySize.ofMebiBytes(5)) - .build()) - .build(); --kafka.sinkTo(sink); ```--### Flink job code --```java -package org.example; --import java.time.Duration; -import java.util.Properties; +package contoso.example +import org.apache.flink.api.common.eventtime.WatermarkStrategy; import org.apache.flink.api.common.serialization.SimpleStringEncoder;-import org.apache.flink.configuration.Configuration; -import org.apache.flink.configuration.ConfigurationUtils; +import org.apache.flink.api.common.serialization.SimpleStringSchema; +import org.apache.flink.client.program.StreamContextEnvironment; import org.apache.flink.configuration.MemorySize; import org.apache.flink.connector.file.sink.FileSink;-import org.apache.flink.core.fs.Path; -import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; -import org.apache.flink.streaming.api.datastream.DataStream; -import org.apache.flink.api.common.serialization.SimpleStringSchema; import org.apache.flink.connector.kafka.source.KafkaSource; import org.apache.flink.connector.kafka.source.enumerator.initializer.OffsetsInitializer;-import org.apache.flink.api.common.eventtime.WatermarkStrategy; +import org.apache.flink.core.fs.Path; +import org.apache.flink.streaming.api.datastream.DataStream; +import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; import org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy; import org.apache.kafka.clients.consumer.OffsetResetStrategy; -public class StreamingJob { - public static void main(String[] args) throws Throwable { -- Properties props = new Properties(); - props.put( - "fs.azure.account.key.<your storage account's dfs endpoint>", - "<your storage account's shared access key>" - ); -- Configuration conf = ConfigurationUtils.createConfiguration(props); +import java.time.Duration; +public class IOTdemo { - StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(conf); + public static void main(String[] args) throws Exception { - String connectionString = "<your iot hub connection string>"; + // create execution environment + StreamExecutionEnvironment env = StreamContextEnvironment.getExecutionEnvironment(); - - KafkaSource<String> source = KafkaSource.<String>builder() - .setBootstrapServers("<your iot hub's service bus url>:9093") - .setTopics("<name of your iot hub>") - .setGroupId("$Default") - .setProperty("partition.discovery.interval.ms", "10000") - .setProperty("security.protocol", "SASL_SSL") - .setProperty("sasl.mechanism", "PLAIN") - .setProperty("sasl.jaas.config", String.format("org.apache.kafka.common.security.plain.PlainLoginModule required username=\"$ConnectionString\" password=\"%s\";", connectionString)) - .setStartingOffsets(OffsetsInitializer.committedOffsets(OffsetResetStrategy.EARLIEST)) - .setValueOnlyDeserializer(new SimpleStringSchema()) - .build(); + String connectionString = "<your iot hub connection string>"; + KafkaSource<String> source = KafkaSource.<String>builder() + .setBootstrapServers("<your iot hub's service bus url>:9093") + .setTopics("<name of your iot hub>") + .setGroupId("$Default") + .setProperty("partition.discovery.interval.ms", "10000") + .setProperty("security.protocol", "SASL_SSL") + .setProperty("sasl.mechanism", "PLAIN") + .setProperty("sasl.jaas.config", String.format("org.apache.kafka.common.security.plain.PlainLoginModule required username=\"$ConnectionString\" password=\"%s\";", connectionString)) + .setStartingOffsets(OffsetsInitializer.committedOffsets(OffsetResetStrategy.EARLIEST)) + .setValueOnlyDeserializer(new SimpleStringSchema()) + .build(); - DataStream<String> kafka = env.fromSource(source, WatermarkStrategy.noWatermarks(), "Kafka Source"); - kafka.print(); + DataStream<String> kafka = env.fromSource(source, WatermarkStrategy.noWatermarks(), "Kafka Source"); - String outputPath = "abfs://<container name>@<your storage account's dfs endpoint>"; + String outputPath = "abfs://<container>@<account_name>.dfs.core.windows.net/flink/data/azureiothubmessage/"; - final FileSink<String> sink = FileSink - .forRowFormat(new Path(outputPath), new SimpleStringEncoder<String>("UTF-8")) - .withRollingPolicy( - DefaultRollingPolicy.builder() - .withRolloverInterval(Duration.ofMinutes(2)) - .withInactivityInterval(Duration.ofMinutes(3)) - .withMaxPartSize(MemorySize.ofMebiBytes(5)) - .build()) - .build(); + final FileSink<String> sink = FileSink + .forRowFormat(new Path(outputPath), new SimpleStringEncoder<String>("UTF-8")) + .withRollingPolicy( + DefaultRollingPolicy.builder() + .withRolloverInterval(Duration.ofMinutes(2)) + .withInactivityInterval(Duration.ofMinutes(3)) + .withMaxPartSize(MemorySize.ofMebiBytes(5)) + .build()) + .build(); - kafka.sinkTo(sink); + kafka.sinkTo(sink); - env.execute("Azure-IoTHub-Flink-ABFS"); - } + env.execute("Sink Azure IOT hub to ADLS gen2"); + } }- ``` -#### Maven dependencies +**Maven pom.xml** ```xml-<dependency> - <groupId>org.apache.flink</groupId> - <artifactId>flink-java</artifactId> - <version>${flink.version}</version> -</dependency> -<dependency> - <groupId>org.apache.flink</groupId> - <artifactId>flink-streaming-java</artifactId> - <version>${flink.version}</version> -</dependency> -<dependency> - <groupId>org.apache.flink</groupId> - <artifactId>flink-streaming-scala_2.12</artifactId> - <version>${flink.version}</version> -</dependency> -<dependency> - <groupId>org.apache.flink</groupId> - <artifactId>flink-clients</artifactId> - <version>${flink.version}</version> -</dependency> -<dependency> - <groupId>org.apache.flink</groupId> - <artifactId>flink-connector-kafka</artifactId> - <version>${flink.version}</version> -</dependency> -<dependency> - <groupId>org.apache.flink</groupId> - <artifactId>flink-connector-files</artifactId> - <version>${flink.version}</version> -</dependency> + <groupId>contoso.example</groupId> + <artifactId>FlinkIOTDemo</artifactId> + <version>1.0-SNAPSHOT</version> + <properties> + <maven.compiler.source>1.8</maven.compiler.source> + <maven.compiler.target>1.8</maven.compiler.target> + <flink.version>1.17.0</flink.version> + <java.version>1.8</java.version> + <scala.binary.version>2.12</scala.binary.version> + </properties> + <dependencies> + <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-java --> + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>${flink.version}</version> + </dependency> + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>${flink.version}</version> + </dependency> + <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-clients --> + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>${flink.version}</version> + </dependency> + <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-connector-files --> + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-connector-files</artifactId> + <version>${flink.version}</version> + </dependency> + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-connector-kafka</artifactId> + <version>${flink.version}</version> + </dependency> + </dependencies> + <build> + <plugins> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-assembly-plugin</artifactId> + <version>3.0.0</version> + <configuration> + <appendAssemblyId>false</appendAssemblyId> + <descriptorRefs> + <descriptorRef>jar-with-dependencies</descriptorRef> + </descriptorRefs> + </configuration> + <executions> + <execution> + <id>make-assembly</id> + <phase>package</phase> + <goals> + <goal>single</goal> + </goals> + </execution> + </executions> + </plugin> + </plugins> + </build> +</project> ``` +## Package the jar and submit the job in Flink cluster ++Upload the jar into webssh pod and submit the jar. ++``` +user@sshnode-0 [ ~ ]$ bin/flink run -c IOTdemo -j FlinkIOTDemo-1.0-SNAPSHOT.jar +SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". +SLF4J: Defaulting to no-operation (NOP) logger implementation +SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. +Job has been submitted with JobID de1931b1c1179e7530510b07b7ced858 +``` +## Check job on Flink Dashboard UI -### Submit job -Submit job using HDInsight on AKS's [Flink job submission API](./flink-job-management.md) +## Check Result on ADLS gen2 on Azure portal ### Reference - [Apache Flink Website](https://flink.apache.org/)-- Apache, Apache Kafka, Kafka, Apache Flink, Flink, and associated open source project names are [trademarks](../trademarks.md) of the [Apache Software Foundation](https://www.apache.org/) (ASF).+- Apache, Apache Kafka, Kafka, Apache Flink, Flink, and associated open source project names are [trademarks](../trademarks.md) of the [Apache Software Foundation](https://www.apache.org/) (ASF). |
| iot-central | Concepts Device Implementation | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-central/core/concepts-device-implementation.md | If the device gets any of the following errors when it connects, it should use a To learn more about device error codes, see [Troubleshooting device connections](troubleshooting.md). -To learn more about implementing automatic reconnections, see [Manage device reconnections to create resilient applications](../../iot-develop/concepts-manage-device-reconnections.md). +To learn more about implementing automatic reconnections, see [Manage device reconnections to create resilient applications](../../iot/concepts-manage-device-reconnections.md). ### Test failover capabilities |
| iot-develop | About Getting Started Device Development | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/about-getting-started-device-development.md | Each quickstart shows how to set up a code sample and tools, run a temperature c |Quickstart|Device SDK| |-|-|-|[Send telemetry from a device to Azure IoT Hub (C)](quickstart-send-telemetry-iot-hub.md?pivots=programming-language-ansi-c)|[Azure IoT C SDK](https://github.com/Azure/azure-iot-sdk-c)| -|[Send telemetry from a device to Azure IoT Hub (C#)](quickstart-send-telemetry-iot-hub.md?pivots=programming-language-csharp)|[Azure IoT SDK for .NET](https://github.com/Azure/azure-iot-sdk-csharp)| -|[Send telemetry from a device to Azure IoT Hub (Node.js)](quickstart-send-telemetry-iot-hub.md?pivots=programming-language-nodejs)|[Azure IoT Node.js SDK](https://github.com/Azure/azure-iot-sdk-node)| -|[Send telemetry from a device to Azure IoT Hub (Python)](quickstart-send-telemetry-iot-hub.md?pivots=programming-language-python)|[Azure IoT Python SDK](https://github.com/Azure/azure-iot-sdk-python)| -|[Send telemetry from a device to Azure IoT Hub (Java)](quickstart-send-telemetry-iot-hub.md?pivots=programming-language-java)|[Azure IoT SDK for Java](https://github.com/Azure/azure-iot-sdk-java)| +|[Send telemetry from a device to Azure IoT Hub (C)](../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-ansi-c)|[Azure IoT C SDK](https://github.com/Azure/azure-iot-sdk-c)| +|[Send telemetry from a device to Azure IoT Hub (C#)](../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-csharp)|[Azure IoT SDK for .NET](https://github.com/Azure/azure-iot-sdk-csharp)| +|[Send telemetry from a device to Azure IoT Hub (Node.js)](../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-nodejs)|[Azure IoT Node.js SDK](https://github.com/Azure/azure-iot-sdk-node)| +|[Send telemetry from a device to Azure IoT Hub (Python)](../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-python)|[Azure IoT Python SDK](https://github.com/Azure/azure-iot-sdk-python)| +|[Send telemetry from a device to Azure IoT Hub (Java)](../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-java)|[Azure IoT SDK for Java](https://github.com/Azure/azure-iot-sdk-java)| ## Quickstarts for embedded devices See the following articles to start using the Azure IoT embedded device SDKs to connect embedded, resource-constrained microcontroller unit (MCU) devices to Azure IoT. Examples of constrained MCU devices with compute and memory limitations, include sensors, and special purpose hardware modules or boards. The following quickstarts require you to have the listed MCU devices. Each quickstart shows how to set up a code sample and tools, flash the device, a |Quickstart|Device|Embedded device SDK| |-|-|-| |[Quickstart: Connect a Microchip ATSAME54-XPro Evaluation kit to IoT Hub](quickstart-devkit-microchip-atsame54-xpro-iot-hub.md)|Microchip ATSAME54-XPro|Azure RTOS middleware|-|[Quickstart: Connect an ESPRESSIF ESP32-Azure IoT Kit to IoT Hub](quickstart-devkit-espressif-esp32-freertos-iot-hub.md)|ESPRESSIF ESP32|FreeRTOS middleware| +|[Quickstart: Connect an ESPRESSIF ESP32-Azure IoT Kit to IoT Hub](../iot/tutorial-devkit-espressif-esp32-freertos-iot-hub.md)|ESPRESSIF ESP32|FreeRTOS middleware| |[Quickstart: Connect an STMicroelectronics B-L475E-IOT01A Discovery kit to IoT Hub](quickstart-devkit-stm-b-l475e-iot-hub.md)|STMicroelectronics L475E-IOT01A|Azure RTOS middleware| |[Quickstart: Connect an NXP MIMXRT1060-EVK Evaluation kit to IoT Hub](quickstart-devkit-nxp-mimxrt1060-evk-iot-hub.md)|NXP MIMXRT1060-EVK|Azure RTOS middleware| |[Connect an MXCHIP AZ3166 devkit to IoT Hub](quickstart-devkit-mxchip-az3166-iot-hub.md)|MXCHIP AZ3166|Azure RTOS middleware| To learn more about working with the IoT device SDKs and developing for general - [Build a device solution for IoT Hub](set-up-environment.md) To learn more about working with the IoT C SDK and embedded C SDK for embedded devices, see the following article.-- [C SDK and Embedded C SDK usage scenarios](concepts-using-c-sdk-and-embedded-c-sdk.md)+- [C SDK and Embedded C SDK usage scenarios](../iot/concepts-using-c-sdk-and-embedded-c-sdk.md) |
| iot-develop | About Iot Develop | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/about-iot-develop.md | The current embedded SDKs target the **C** language. The embedded SDKs provide e ## Choosing your hardware Azure IoT devices are the basic building blocks of an IoT solution and are responsible for observing and interacting with their environment. There are many different types of IoT devices, and it's helpful to understand the kinds of devices that exist and how they can affect your development process. -For more information on the difference between devices types covered in this article, see [About IoT Device Types](concepts-iot-device-types.md). +For more information on the difference between devices types covered in this article, see [About IoT Device Types](../iot/concepts-iot-device-types.md). ## Choosing an SDK As an Azure IoT device developer, you have a diverse set of SDKs, protocols and tools to help build device-enabled cloud applications. |
| iot-develop | About Iot Sdks | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/about-iot-sdks.md | The main consideration in choosing an SDK is the device's own hardware. General |[Embedded device SDKs](#embedded-device-sdks)|Embedded devices|Special-purpose MCU-based devices with compute and memory limitations|Sensors| > [!Note] -> For more information on different device categories so you can choose the best SDK for your device, see [Azure IoT Device Types](concepts-iot-device-types.md). +> For more information on different device categories so you can choose the best SDK for your device, see [Azure IoT Device Types](../iot/concepts-iot-device-types.md). ## Device SDKs |
| iot-develop | Quickstart Devkit Microchip Atsame54 Xpro Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-microchip-atsame54-xpro-iot-hub.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs to connect general devices, and embedded devices, to Azure IoT. > [!div class="nextstepaction"]-> [Connect a general simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a general simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!div class="nextstepaction"]-> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](concepts-using-c-sdk-and-embedded-c-sdk.md) +> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](../iot/concepts-using-c-sdk-and-embedded-c-sdk.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-develop | Quickstart Devkit Microchip Atsame54 Xpro | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-microchip-atsame54-xpro.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs to connect devices to Azure IoT. > [!div class="nextstepaction"]-> [Connect a simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-develop | Quickstart Devkit Mxchip Az3166 Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-mxchip-az3166-iot-hub.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs to connect general devices, and embedded devices, to Azure IoT. > [!div class="nextstepaction"]-> [Connect a general simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a general simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!div class="nextstepaction"]-> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](concepts-using-c-sdk-and-embedded-c-sdk.md) +> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](../iot/concepts-using-c-sdk-and-embedded-c-sdk.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-develop | Quickstart Devkit Mxchip Az3166 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-mxchip-az3166.md | As a next step, explore the following articles to learn more about using the IoT > [!div class="nextstepaction"] > [Connect an MXCHIP AZ3166 devkit to IoT Hub](quickstart-devkit-mxchip-az3166-iot-hub.md) > [!div class="nextstepaction"]-> [Connect a simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-develop | Quickstart Devkit Nxp Mimxrt1060 Evk Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-nxp-mimxrt1060-evk-iot-hub.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs, or Azure RTOS to connect devices to Azure IoT. > [!div class="nextstepaction"]-> [Connect a simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!div class="nextstepaction"]-> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](concepts-using-c-sdk-and-embedded-c-sdk.md) +> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](../iot/concepts-using-c-sdk-and-embedded-c-sdk.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-develop | Quickstart Devkit Nxp Mimxrt1060 Evk | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-nxp-mimxrt1060-evk.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs to connect devices to Azure IoT. > [!div class="nextstepaction"]-> [Connect a device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-develop | Quickstart Devkit Renesas Rx65n Cloud Kit Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-renesas-rx65n-cloud-kit-iot-hub.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs, or Azure RTOS to connect devices to Azure IoT. > [!div class="nextstepaction"]-> [Connect a general simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a general simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!div class="nextstepaction"]-> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](concepts-using-c-sdk-and-embedded-c-sdk.md) +> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](../iot/concepts-using-c-sdk-and-embedded-c-sdk.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-develop | Quickstart Devkit Renesas Rx65n Cloud Kit | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-renesas-rx65n-cloud-kit.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs to connect devices to Azure IoT. > [!div class="nextstepaction"]-> [Connect a simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-develop | Quickstart Devkit Stm B L475e Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-stm-b-l475e-iot-hub.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs, or Azure RTOS to connect devices to Azure IoT. > [!div class="nextstepaction"]-> [Connect a simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!div class="nextstepaction"] > [Connect an STMicroelectronics B-L475E-IOT01A to IoT Central](quickstart-devkit-stm-b-l475e.md) |
| iot-develop | Quickstart Devkit Stm B L475e | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-stm-b-l475e.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs to connect devices to Azure IoT. > [!div class="nextstepaction"]-> [Connect a simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!IMPORTANT] |
| iot-develop | Quickstart Devkit Stm B L4s5i Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-stm-b-l4s5i-iot-hub.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs, or Azure RTOS to connect devices to Azure IoT. > [!div class="nextstepaction"]-> [Connect a general device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a general device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!div class="nextstepaction"] > [Quickstart: Connect an STMicroelectronics B-L475E-IOT01A Discovery kit to IoT Hub](quickstart-devkit-stm-b-l475e-iot-hub.md) > [!div class="nextstepaction"]-> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](concepts-using-c-sdk-and-embedded-c-sdk.md) +> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](../iot/concepts-using-c-sdk-and-embedded-c-sdk.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-develop | Quickstart Devkit Stm B L4s5i | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-stm-b-l4s5i.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs to connect devices to Azure IoT. > [!div class="nextstepaction"]-> [Connect a simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-develop | Quickstart Devkit Stm B U585i Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-develop/quickstart-devkit-stm-b-u585i-iot-hub.md | In this quickstart, you built a custom image that contains Azure RTOS sample cod As a next step, explore the following articles to learn more about using the IoT device SDKs, or Azure RTOS to connect devices to Azure IoT. > [!div class="nextstepaction"]-> [Connect a general simulated device to IoT Hub](quickstart-send-telemetry-iot-hub.md) +> [Connect a general simulated device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md) > [!div class="nextstepaction"] > [Quickstart: Connect an STMicroelectronics B-L4S5I-IOT01A Discovery kit to IoT Hub](quickstart-devkit-stm-b-l4s5i-iot-hub.md) > [!div class="nextstepaction"]-> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](concepts-using-c-sdk-and-embedded-c-sdk.md) +> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](../iot/concepts-using-c-sdk-and-embedded-c-sdk.md) > [!IMPORTANT] > Azure RTOS provides OEMs with components to secure communication and to create code and data isolation using underlying MCU/MPU hardware protection mechanisms. However, each OEM is ultimately responsible for ensuring that their device meets evolving security requirements. |
| iot-hub | Device Twins Cli | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/device-twins-cli.md | In this article, you: To learn how to: -* Send telemetry from devices, see [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json). +* Send telemetry from devices, see [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json). * Configure devices using device twin's desired properties, see [Tutorial: Configure your devices from a back-end service](tutorial-device-twins.md). |
| iot-hub | Device Twins Dotnet | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/device-twins-dotnet.md | In this article, you: To learn how to: -* Send telemetry from devices, see [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-csharp). +* Send telemetry from devices, see [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-csharp). * Configure devices using device twin's desired properties, see [Tutorial: Configure your devices from a back-end service](tutorial-device-twins.md). |
| iot-hub | Device Twins Java | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/device-twins-java.md | In this article, you: To learn how to: -* Send telemetry from devices, see [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-java) +* Send telemetry from devices, see [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-java) * Configure devices using device twin's desired properties, see [Tutorial: Configure your devices from a back-end service](tutorial-device-twins.md) |
| iot-hub | Device Twins Node | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/device-twins-node.md | In this article, you: To learn how to: -* Send telemetry from devices, see [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-nodejs) +* Send telemetry from devices, see [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-nodejs) * Configure devices using device twin's desired properties, see [Tutorial: Configure your devices from a back-end service](tutorial-device-twins.md) |
| iot-hub | Device Twins Python | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/device-twins-python.md | In this article, you: To learn how to: -* Send telemetry from devices, see [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?pivots=programming-language-python) article. +* Send telemetry from devices, see [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-python) article. * Configure devices using device twin's desired properties, see [Tutorial: Configure your devices from a back-end service](tutorial-device-twins.md). |
| iot-hub | File Upload Dotnet | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/file-upload-dotnet.md | -The [Send telemetry from a device to an IoT hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-csharp) quickstart and [Send cloud-to-device messages with IoT Hub](c2d-messaging-dotnet.md) article show the basic device-to-cloud and cloud-to-device messaging functionality of IoT Hub. The [Configure Message Routing with IoT Hub](tutorial-routing.md) article shows a way to reliably store device-to-cloud messages in Microsoft Azure blob storage. However, in some scenarios, you can't easily map the data your devices send into the relatively small device-to-cloud messages that IoT Hub accepts. For example: +The [Send telemetry from a device to an IoT hub](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-csharp) quickstart and [Send cloud-to-device messages with IoT Hub](c2d-messaging-dotnet.md) article show the basic device-to-cloud and cloud-to-device messaging functionality of IoT Hub. The [Configure Message Routing with IoT Hub](tutorial-routing.md) article shows a way to reliably store device-to-cloud messages in Microsoft Azure blob storage. However, in some scenarios, you can't easily map the data your devices send into the relatively small device-to-cloud messages that IoT Hub accepts. For example: * Videos * Large files that contain images |
| iot-hub | File Upload Java | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/file-upload-java.md | -The [Send telemetry from a device to an IoT hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-java) quickstart and [Send cloud-to-device messages with IoT Hub](c2d-messaging-java.md) articles show the basic device-to-cloud and cloud-to-device messaging functionality of IoT Hub. The [Configure message routing with IoT Hub](tutorial-routing.md) tutorial shows a way to reliably store device-to-cloud messages in Azure blob storage. However, in some scenarios, you can't easily map the data your devices send into the relatively small device-to-cloud messages that IoT Hub accepts. For example: +The [Send telemetry from a device to an IoT hub](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-java) quickstart and [Send cloud-to-device messages with IoT Hub](c2d-messaging-java.md) articles show the basic device-to-cloud and cloud-to-device messaging functionality of IoT Hub. The [Configure message routing with IoT Hub](tutorial-routing.md) tutorial shows a way to reliably store device-to-cloud messages in Azure blob storage. However, in some scenarios, you can't easily map the data your devices send into the relatively small device-to-cloud messages that IoT Hub accepts. For example: * Videos * Large files that contain images |
| iot-hub | File Upload Node | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/file-upload-node.md | -The [Send telemetry from a device to an IoT hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?pivots=programming-language-nodejs) quickstart and [Send cloud-to-device messages with IoT Hub](c2d-messaging-node.md) articles show the basic device-to-cloud and cloud-to-device messaging functionality of IoT Hub. The [Configure Message Routing with IoT Hub](tutorial-routing.md) tutorial shows a way to reliably store device-to-cloud messages in Microsoft Azure blob storage. However, in some scenarios, you can't easily map the data your devices send into the relatively small device-to-cloud messages that IoT Hub accepts. For example: +The [Send telemetry from a device to an IoT hub](../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-nodejs) quickstart and [Send cloud-to-device messages with IoT Hub](c2d-messaging-node.md) articles show the basic device-to-cloud and cloud-to-device messaging functionality of IoT Hub. The [Configure Message Routing with IoT Hub](tutorial-routing.md) tutorial shows a way to reliably store device-to-cloud messages in Microsoft Azure blob storage. However, in some scenarios, you can't easily map the data your devices send into the relatively small device-to-cloud messages that IoT Hub accepts. For example: * Videos * Large files that contain images |
| iot-hub | File Upload Python | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/file-upload-python.md | -The [Send telemetry from a device to an IoT hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-python) quickstart and [Send cloud-to-device messages with IoT Hub](c2d-messaging-python.md) articles show the basic device-to-cloud and cloud-to-device messaging functionality of IoT Hub. The [Configure Message Routing with IoT Hub](tutorial-routing.md) tutorial shows a way to reliably store device-to-cloud messages in Microsoft Azure blob storage. However, in some scenarios, you can't easily map the data your devices send into the relatively small device-to-cloud messages that IoT Hub accepts. For example: +The [Send telemetry from a device to an IoT hub](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-python) quickstart and [Send cloud-to-device messages with IoT Hub](c2d-messaging-python.md) articles show the basic device-to-cloud and cloud-to-device messaging functionality of IoT Hub. The [Configure Message Routing with IoT Hub](tutorial-routing.md) tutorial shows a way to reliably store device-to-cloud messages in Microsoft Azure blob storage. However, in some scenarios, you can't easily map the data your devices send into the relatively small device-to-cloud messages that IoT Hub accepts. For example: * Videos * Large files that contain images |
| iot-hub | Iot Concepts And Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/iot-concepts-and-iot-hub.md | For more information, see [Compare message routing and Event Grid for IoT Hub](i To try out an end-to-end IoT solution, check out the IoT Hub quickstarts: - [Send telemetry from a device to IoT Hub](quickstart-send-telemetry-cli.md)-- [Send telemetry from an IoT Plug and Play device to IoT Hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json)+- [Send telemetry from an IoT Plug and Play device to IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json) - [Quickstart: Control a device connected to an IoT hub](quickstart-control-device.md) To learn more about the ways you can build and deploy IoT solutions with Azure IoT, visit: |
| iot-hub | Iot Hub Devguide Messages Construct | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/iot-hub-devguide-messages-construct.md | The **iothub-connection-auth-method** property contains a JSON serialized object ## Next steps * For information about message size limits in IoT Hub, see [IoT Hub quotas and throttling](iot-hub-devguide-quotas-throttling.md).-* To learn how to create and read IoT Hub messages in various programming languages, see the [Quickstarts](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json). +* To learn how to create and read IoT Hub messages in various programming languages, see the [Quickstarts](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json). * To learn about the structure of non-telemetry events generated by IoT Hub, see [IoT Hub non-telemetry event schemas](iot-hub-non-telemetry-event-schema.md). |
| iot-hub | Iot Hub Devguide Messages D2c | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/iot-hub-devguide-messages-d2c.md | For more information, see [IoT Hub message routing query syntax](./iot-hub-devgu Use the following articles to learn how to read messages from an endpoint. -* Read from a [built-in endpoint](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json) +* Read from a [built-in endpoint](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json) * Read from [Blob storage](../storage/blobs/storage-blob-event-quickstart.md) |
| iot-hub | Iot Hub Devguide Quotas Throttling | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/iot-hub-devguide-quotas-throttling.md | The tier also determines the throttling limits that IoT Hub enforces on all oper Operation throttles are rate limitations that are applied in minute ranges and are intended to prevent abuse. They're also subject to [traffic shaping](#traffic-shaping). -It's a good practice to throttle your calls so that you don't hit/exceed the throttling limits. If you do hit the limit, IoT Hub responds with error code 429 and the client should back-off and retry. These limits are per hub (or in some cases per hub/unit). For more information, see [Retry patterns](../iot-develop/concepts-manage-device-reconnections.md#retry-patterns). +It's a good practice to throttle your calls so that you don't hit/exceed the throttling limits. If you do hit the limit, IoT Hub responds with error code 429 and the client should back-off and retry. These limits are per hub (or in some cases per hub/unit). For more information, see [Retry patterns](../iot/concepts-manage-device-reconnections.md#retry-patterns). For pricing details about which operations are charged and under what circumstances, see [billing information](iot-hub-devguide-pricing.md). |
| iot-hub | Iot Hub Devguide Sdks | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/iot-hub-devguide-sdks.md | Azure IoT SDKs are also available for the following ## Next steps -Learn how to [manage connectivity and reliable messaging](../iot-develop/concepts-manage-device-reconnections.md) using the IoT Hub device SDKs. +Learn how to [manage connectivity and reliable messaging](../iot/concepts-manage-device-reconnections.md) using the IoT Hub device SDKs. |
| iot-hub | Iot Hub Distributed Tracing | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/iot-hub-distributed-tracing.md | In this section, you edit the [iothub_ll_telemetry_sample.c](https://github.com/ :::code language="c" source="~/samples-iot-distributed-tracing/iothub_ll_telemetry_sample-c/iothub_ll_telemetry_sample.c" range="56-60" highlight="2"::: - Replace the value of the `connectionString` constant with the device connection string that you saved in the [Register a device](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json#register-a-device) section of the quickstart for sending telemetry. + Replace the value of the `connectionString` constant with the device connection string that you saved in the [Register a device](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json#register-a-device) section of the quickstart for sending telemetry. 1. Find the line of code that calls `IoTHubDeviceClient_LL_SetConnectionStatusCallback` to register a connection status callback function before the send message loop. Add code under that line to call `IoTHubDeviceClient_LL_EnablePolicyConfiguration` and enable distributed tracing for the device: |
| iot-hub | Iot Hub Ha Dr | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/iot-hub-ha-dr.md | Depending on the uptime goals you define for your IoT solutions, you should dete ## Intra-region HA -The IoT Hub service provides intra-region HA by implementing redundancies in almost all layers of the service. The [SLA published by the IoT Hub service](https://azure.microsoft.com/support/legal/sl#retry-patterns) must be built in to the components interacting with a cloud application to deal with transient failures. +The IoT Hub service provides intra-region HA by implementing redundancies in almost all layers of the service. The [SLA published by the IoT Hub service](https://azure.microsoft.com/support/legal/sl#retry-patterns) must be built in to the components interacting with a cloud application to deal with transient failures. ## Availability zones Here's a summary of the HA/DR options presented in this article that can be used ## Next steps * [What is Azure IoT Hub?](about-iot-hub.md)-* [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json) +* [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json) * [Tutorial: Perform manual failover for an IoT hub](tutorial-manual-failover.md) |
| iot-hub | Iot Hub Live Data Visualization In Power Bi | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/iot-hub-live-data-visualization-in-power-bi.md | If you don't have an Azure subscription, [create a free account](https://azure.m Before you begin this tutorial, have the following prerequisites in place: -* Complete one of the [Send telemetry](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json) quickstarts in the development language of your choice. Alternatively, you can use any device app that sends temperature telemetry; for example, the [Raspberry Pi online simulator](raspberry-pi-get-started.md) or one of the [Embedded device](../iot-develop/quickstart-devkit-mxchip-az3166.md) quickstarts. These articles cover the following requirements: +* Complete one of the [Send telemetry](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json) quickstarts in the development language of your choice. Alternatively, you can use any device app that sends temperature telemetry; for example, the [Raspberry Pi online simulator](raspberry-pi-get-started.md) or one of the [Embedded device](../iot-develop/quickstart-devkit-mxchip-az3166.md) quickstarts. These articles cover the following requirements: * An active Azure subscription. * An Azure IoT hub in your subscription. |
| iot-hub | Iot Hub Troubleshoot Connectivity | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/iot-hub-troubleshoot-connectivity.md | If the previous steps didn't help, try: * To learn more about resolving transient issues, see [Transient fault handling](/azure/architecture/best-practices/transient-faults). -* To learn more about the Azure IoT device SDKs and managing retries, see [Retry patterns](../iot-develop/concepts-manage-device-reconnections.md#retry-patterns). +* To learn more about the Azure IoT device SDKs and managing retries, see [Retry patterns](../iot/concepts-manage-device-reconnections.md#retry-patterns). |
| iot-hub | Migrate Tls Certificate | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/migrate-tls-certificate.md | You can remove the Baltimore root certificate once all stages of the migration a If you're experiencing general connectivity issues with IoT Hub, check out these troubleshooting resources: -* [Connection and retry patterns with device SDKs](../iot-develop/concepts-manage-device-reconnections.md#connection-and-retry). +* [Connection and retry patterns with device SDKs](../iot/concepts-manage-device-reconnections.md#connection-and-retry). * [Understand and resolve Azure IoT Hub error codes](troubleshoot-error-codes.md). If you're watching Azure Monitor after migrating certificates, you should look for a DeviceDisconnect event followed by a DeviceConnect event, as demonstrated in the following screenshot: |
| iot-hub | Troubleshoot Error Codes | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/troubleshoot-error-codes.md | To resolve this error: * Use the latest versions of the [IoT SDKs](iot-hub-devguide-sdks.md). * See the guidance for [IoT Hub internal server errors](#500xxx-internal-errors). -We recommend using Azure IoT device SDKs to manage connections reliably. To learn more, see [Manage connectivity and reliable messaging by using Azure IoT Hub device SDKs](../iot-develop/concepts-manage-device-reconnections.md) +We recommend using Azure IoT device SDKs to manage connections reliably. To learn more, see [Manage connectivity and reliable messaging by using Azure IoT Hub device SDKs](../iot/concepts-manage-device-reconnections.md) ## 409001 Device already exists You may see that your request to IoT Hub fails with an error that begins with 50 There can be many causes for a 500xxx error response. In all cases, the issue is most likely transient. While the IoT Hub team works hard to maintain [the SLA](https://azure.microsoft.com/support/legal/sla/iot-hub/), small subsets of IoT Hub nodes can occasionally experience transient faults. When your device tries to connect to a node that's having issues, you receive this error. -To mitigate 500xxx errors, issue a retry from the device. To [automatically manage retries](../iot-develop/concepts-manage-device-reconnections.md#connection-and-retry), make sure you use the latest version of the [Azure IoT SDKs](iot-hub-devguide-sdks.md). For best practice on transient fault handling and retries, see [Transient fault handling](/azure/architecture/best-practices/transient-faults). +To mitigate 500xxx errors, issue a retry from the device. To [automatically manage retries](../iot/concepts-manage-device-reconnections.md#connection-and-retry), make sure you use the latest version of the [Azure IoT SDKs](iot-hub-devguide-sdks.md). For best practice on transient fault handling and retries, see [Transient fault handling](/azure/architecture/best-practices/transient-faults). If the problem persists, check [Resource Health](iot-hub-azure-service-health-integration.md#check-iot-hub-health-with-azure-resource-health) and [Azure Status](https://azure.status.microsoft/) to see if IoT Hub has a known problem. You can also use the [manual failover feature](tutorial-manual-failover.md). |
| iot-hub | Tutorial Use Metrics And Diags | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-hub/tutorial-use-metrics-and-diags.md | -This tutorial uses the Azure sample from the [.NET send telemetry quickstart](../iot-develop/quickstart-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-csharp) to send messages to the IoT hub. You can always use a device or another sample to send messages, but you may have to modify a few steps accordingly. +This tutorial uses the Azure sample from the [.NET send telemetry quickstart](../iot/tutorial-send-telemetry-iot-hub.md?toc=/azure/iot-hub/toc.json&bc=/azure/iot-hub/breadcrumb/toc.json&pivots=programming-language-csharp) to send messages to the IoT hub. You can always use a device or another sample to send messages, but you may have to modify a few steps accordingly. Some familiarity with Azure Monitor concepts might be helpful before you begin this tutorial. To learn more, see [Monitor IoT Hub](monitor-iot-hub.md). To learn more about the metrics and resource logs emitted by IoT Hub, see [Monitoring data reference](monitor-iot-hub-reference.md). |
| iot-operations | Howto Deploy Iot Operations | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-operations/deploy-iot-ops/howto-deploy-iot-operations.md | Title: Deploy extensions with Azure IoT Orchestrator -description: Use the Azure portal, Azure CLI, or GitHub Actions to deploy Azure IoT Operations extensions with the Azure IoT Orchestrator +description: Use the Azure CLI to deploy Azure IoT Operations extensions with the Azure IoT Orchestrator. Previously updated : 01/31/2024 Last updated : 04/05/2024 #CustomerIntent: As an OT professional, I want to deploy Azure IoT Operations to a Kubernetes cluster. Last updated 01/31/2024 [!INCLUDE [public-preview-note](../includes/public-preview-note.md)] -Deploy Azure IoT Operations Preview to a Kubernetes cluster using the Azure portal, Azure CLI, or GitHub actions. Once you have Azure IoT Operations deployed, then you can use the Azure IoT Orchestrator Preview service to manage and deploy additional workloads to your cluster. +Deploy Azure IoT Operations Preview to a Kubernetes cluster using the Azure CLI. Once you have Azure IoT Operations deployed, then you can use the Azure IoT Orchestrator Preview service to manage and deploy other workloads to your cluster. ## Prerequisites -Cloud resources: +Cloud resources: -* An Azure subscription. If you don't have an Azure subscription, [create one for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin. +* An Azure subscription. -* Azure access permissions. At a minimum, have **Contributor** permissions in your Azure subscription. Depending on the deployment method and feature flag status you select, you may also need **Microsoft/Authorization/roleAssignments/write** permissions. If you *don't* have role assignment write permissions, take the following additional steps when deploying: +* Azure access permissions. At a minimum, have **Contributor** permissions in your Azure subscription. Depending on the deployment feature flag status you select, you might also need **Microsoft/Authorization/roleAssignments/write** permissions for the resource group that contains your Arc-enabled Kubernetes cluster. You can make a custom role in Azure role-based access control or assign a built-in role that grants this permission. For more information, see [Azure built-in roles for General](../../role-based-access-control/built-in-roles/general.md). - * If deploying with an Azure Resource Manager template, set the `deployResourceSyncRules` parameter to `false`. - * If deploying with the Azure CLI, include the `--disable-rsync-rules`. + If you *don't* have role assignment write permissions, you can still deploy Azure IoT Operations by disabling some features. This approach is discussed in more detail in the [Deploy extensions](#deploy-extensions) section of this article. -* An [Azure Key Vault](../../key-vault/general/overview.md) that has the **Permission model** set to **Vault access policy**. You can check this setting in the **Access configuration** section of an existing key vault. + * In the Azure CLI, use the [az role assignment create](/cli/azure/role/assignment#az-role-assignment-create) command to give permissions. For example, `az role assignment create --assignee sp_name --role "Role Based Access Control Administrator" --scope subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyResourceGroup` ++ * In the Azure portal, you're prompted to restrict access using conditions when you assign privileged admin roles to a user or principal. For this scenario, select the **Allow user to assign all roles** condition in the **Add role assignment** page. ++ :::image type="content" source="./media/howto-deploy-iot-operations/add-role-assignment-conditions.png" alt-text="Screenshot that shows assigning users highly privileged role access in the Azure portal."::: ++* An Azure Key Vault that has the **Permission model** set to **Vault access policy**. You can check this setting in the **Access configuration** section of an existing key vault. If you need to create a new key vault, use the [az keyvault create](/cli/azure/keyvault#az-keyvault-create) command: ++ ```azurecli + az keyvault create --enable-rbac-authorization false --name "<KEYVAULT_NAME>" --resource-group "<RESOURCE_GROUP>" + ``` Development resources: Development resources: * The Azure IoT Operations extension for Azure CLI. Use the following command to add the extension or update it to the latest version: - ```bash + ```azurecli az extension add --upgrade --name azure-iot-ops ``` A cluster host: -* An Azure Arc-enabled Kubernetes cluster. If you don't have one, follow the steps in [Prepare your Azure Arc-enabled Kubernetes cluster](./howto-prepare-cluster.md?tabs=wsl-ubuntu). +* An Azure Arc-enabled Kubernetes cluster. If you don't have one, follow the steps in [Prepare your Azure Arc-enabled Kubernetes cluster](./howto-prepare-cluster.md?tabs=wsl-ubuntu). If you've already deployed Azure IoT Operations to your cluster, uninstall those resources before continuing. For more information, see [Update a deployment](#update-a-deployment). A cluster host: az iot ops verify-host ``` - ## Deploy extensions -### Azure CLI - Use the Azure CLI to deploy Azure IoT Operations components to your Arc-enabled Kubernetes cluster. -Sign in to Azure CLI. To prevent potential permission issues later, sign in interactively with a browser here even if you already logged in before. +1. Sign in to Azure CLI interactively with a browser even if you already signed in before. If you don't sign in interactively, you might get an error that says *Your device is required to be managed to access your resource* when you continue to the next step to deploy Azure IoT Operations. -```azurecli-interactive -az login -``` + ```azurecli-interactive + az login + ``` -> [!NOTE] -> If you're using GitHub Codespaces in a browser, `az login` returns a localhost error in the browser window after logging in. To fix, either: -> -> * Open the codespace in VS Code desktop, and then run `az login` in the terminal. This opens a browser window where you can log in to Azure. -> * After you get the localhost error on the browser, copy the URL from the browser and use `curl <URL>` in a new terminal tab. You should see a JSON response with the message "You have logged into Microsoft Azure!". + > [!NOTE] + > If you're using GitHub Codespaces in a browser, `az login` returns a localhost error in the browser window after logging in. To fix, either: + > + > * Open the codespace in VS Code desktop, and then run `az login` in the terminal. This opens a browser window where you can log in to Azure. + > * Or, after you get the localhost error on the browser, copy the URL from the browser and use `curl <URL>` in a new terminal tab. You should see a JSON response with the message "You have logged into Microsoft Azure!". -Deploy Azure IoT Operations to your cluster. The [az iot ops init](/cli/azure/iot/ops#az-iot-ops-init) command does the following steps: +1. Deploy Azure IoT Operations to your cluster. Use optional flags to customize the [az iot ops init](/cli/azure/iot/ops#az-iot-ops-init) command to fit your scenario. -* Creates a key vault in your resource group. -* Sets up a service principal to give your cluster access to the key vault. -* Configures TLS certificates. -* Configures a secrets store on your cluster that connects to the key vault. -* Deploys the Azure IoT Operations resources. + By default, the `az iot ops init` command takes the following actions, some of which require that the principal signed in to the CLI has elevated permissions: -```azurecli-interactive -az iot ops init --cluster <CLUSTER_NAME> -g <RESOURCE_GROUP> --kv-id $(az keyvault create -n <NEW_KEYVAULT_NAME> -g <RESOURCE_GROUP> -o tsv --query id) -``` + * Set up a service principal and app registration to give your cluster access to the key vault. + * Configure TLS certificates. + * Configure a secrets store on your cluster that connects to the key vault. + * Deploy the Azure IoT Operations resources. ->[!TIP] ->If you get an error that says *Your device is required to be managed to access your resource*, go back to the previous step and make sure that you signed in interactively. + ```azurecli-interactive + az iot ops init --cluster <CLUSTER_NAME> --resource-group <RESOURCE_GROUP> --kv-id <KEYVAULT_ID> + ``` -If you don't have **Microsoft.Authorization/roleAssignment/write** permissions in your Azure subscription, include the `--disable-rsync-rules` feature flag. + If you don't have **Microsoft.Authorization/roleAssignment/write** permissions in the resource group, add the `--disable-rsync-rules` feature flag. This flag disables the resource sync rules on the deployment. -If you encounter an issue with the KeyVault access policy and the Service Principal (SP) permissions, [pass service principal and KeyVault arguments](howto-manage-secrets.md#pass-service-principal-and-key-vault-arguments-to-azure-iot-operations-deployment). + If you want to use an existing service principal and app registration instead of allowing `init` to create new ones, include the `--sp-app-id,` `--sp-object-id`, and `--sp-secret` parameters. For more information, see [Configure service principal and Key Vault manually](howto-manage-secrets.md#configure-service-principal-and-key-vault-manually). -Use optional flags to customize the `az iot ops init` command. To learn more, see [az iot ops init](/cli/azure/iot/ops#az-iot-ops-init). +1. After the deployment is complete, you can use [az iot ops check](/cli/azure/iot/ops#az-iot-ops-check) to evaluate IoT Operations service deployment for health, configuration, and usability. The *check* command can help you find problems in your deployment and configuration. -> [!TIP] -> You can check the configurations of topic maps, QoS, message routes with the [CLI extension](/cli/azure/iot/ops#az-iot-ops-check-examples) `az iot ops check --detail-level 2`. + ```azurecli + az iot ops check + ``` ++ You can also check the configurations of topic maps, QoS, and message routes by adding the `--detail-level 2` parameter for a verbose view. ### Configure cluster network (AKS EE) To view the pods on your cluster, run the following command: kubectl get pods -n azure-iot-operations ``` -It can take several minutes for the deployment to complete. Continue running the `get pods` command to refresh your view. +It can take several minutes for the deployment to complete. Rerun the `get pods` command to refresh your view. To view your cluster on the Azure portal, use the following steps: To view your cluster on the Azure portal, use the following steps: ## Update a deployment -Currently, there is no support for updating an existing Azure IoT Operations deployment. Instead, start with a clean cluster for a new deployment. +Currently, there's no support for updating an existing Azure IoT Operations deployment. Instead, start with a clean cluster for a new deployment. If you want to delete the Azure IoT Operations deployment on your cluster so that you can redeploy to it, navigate to your cluster on the Azure portal. Select the extensions of the type **microsoft.iotoperations.x** and **microsoft.deviceregistry.assets**, then select **Uninstall**. Keep the secrets provider on your cluster, as that is a prerequisite for deployment and not included in a fresh deployment. |
| iot-operations | Quickstart Deploy | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot-operations/get-started/quickstart-deploy.md | In this section, you use the [az iot ops init](/cli/azure/iot/ops#az-iot-ops-ini | **KEYVAULT_NAME** | A name for a new key vault. | ```azurecli- az keyvault create --enable-rbac-authorization false --name "<KEYVAULT_NAME>" --resource-group "<RESOURCE_GROUP>" + az keyvault create --enable-rbac-authorization false --name $KEYVAULT_NAME --resource-group $RESOURCE_GROUP ``` >[!TIP] |
| iot | Concepts Iot Device Selection | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/concepts-iot-device-selection.md | + + Title: Azure IOT prototyping device selection list +description: This document provides guidance on choosing a hardware device for prototyping IoT Azure solutions. ++++ Last updated : 04/04/2024++# IoT device selection list ++This IoT device selection list aims to give partners a starting point with IoT hardware to build prototypes and proof-of-concepts quickly and easily.[^1] ++All boards listed support users of all experience levels. ++>[!NOTE] +>This table is not intended to be an exhaustive list or for bringing solutions to production. [^2] [^3] ++**Security advisory:** Except for the Azure Sphere, it's recommended to keep these devices behind a router and/or firewall. ++[^1]: *If you're new to hardware programming, for MCU dev work we recommend using VS Code Arduino Extension or VS Code Platform IO Extension. For SBC dev work, you program the device like you would a laptop, that is, on the device itself. The Raspberry Pi supports VS Code development.* ++[^2]: *Devices in the availability of support resources, common boards used for prototyping and PoCs, and boards that support beginner-friendly IDEs like Arduino IDE and VS Code extensions; for example, Arduino Extension and Platform IO extension. For simplicity, we aimed to keep the total device list <6. Other teams and individuals may have chosen to feature different boards based on their interpretation of the criteria.* ++[^3]: *For bringing devices to production, you likely want to test a PoC with a specific chipset, ST's STM32 or Microchip's Pic-IoT breakout board series, design a custom board that can be manufactured for lower cost than the MCUs and SBCs listed here, or even explore FPGA-based dev kits. You may also want to use a development environment for professional electrical engineering like STM32CubeMX or ARM mBed browser-based programmer.* ++## Contents ++| Section | Description | +|--|--| +| [Start here](#start-here) | A guide to using this selection list. Includes suggested selection criteria.| +| [Selection diagram](#application-selection-visual) | A visual that summarizes common selection criteria with possible hardware choices. | +| [Terminology and ML requirements](#terminology-and-ml-requirements) | Terminology and acronym definitions and device requirements for edge machine learning (ML). | +| [MCU device list](#mcu-device-list) | A list of recommended MCUs, for example, ESP32, with tech specs and alternatives. | +| [SBC device list](#sbc-device-list) | A list of recommended SBCs, for example, Raspberry Pi, with tech specs and alternatives. | ++## Start here ++### How to use this document ++Use this document to better understand IoT terminology, device selection considerations, and to choose an IoT device for prototyping or building a proof-of-concept. We recommend the following procedure: ++1. Read through the 'what to consider when choosing a board' section to identify needs and constraints. ++2. Use the Application Selection Visual to identify possible options for your IoT scenario. ++3. Using the MCU or SBC Device Lists, check device specifications and compare against your needs/constraints. ++### What to consider when choosing a board ++To choose a device for your IoT prototype, see the following criteria: ++- **Microcontroller unit (MCU) or single board computer (SBC)** + - An MCU is preferred for single tasks, like gathering and uploading sensor data or machine learning at the edge. MCUs also tend to be lower cost. + - An SBC is preferred when you need multiple different tasks, like gathering sensor data and controlling another device. It may also be preferred in the early stages when there are many options for possible solutions - an SBC enables you to try lots of different approaches. ++- **Processing power** ++ - **Memory**: Consider how much memory storage (in bytes), file storage, and memory to run programs your project needs. ++ - **Clock speed**: Consider how quickly your programs need to run or how quickly you need the device to communicate with the IoT server. ++ - **End-of-life**: Consider if you need a device with the most up-to-date features and documentation or if you can use a discontinued device as a prototype. ++- **Power consumption** ++ - **Power**: Consider how much voltage and current the board consumes. Determine if wall power is readily available or if you need a battery for your application. ++ - **Connection**: Consider the physical connection to the power source. If you need battery power, check if there's a battery connection port available on the board. If there's no battery connector, seek another comparable board, or consider other ways to add battery power to your device. ++- **Inputs and outputs** + - **Ports and pins**: Consider how many and of what types of ports and I/O pins your project may require. + * Other considerations include if your device will be communicating with other sensors or devices. If so, identify how many ports those signals require. ++ - **Protocols**: If you're working with other sensors or devices, consider what hardware communication protocols are required. + * For example, you may need CAN, UART, SPI, I2C, or other communication protocols. + - **Power**: Consider if your device will be powering other components like sensors. If your device is powering other components, identify the voltage, and current output of the device's available power pins and determine what voltage/current your other components need. ++ - **Types**: Determine if you need to communicate with analog components. If you are in need of analog components, identify how many analog I/O pins your project needs. ++ - **Peripherals**: Consider if you prefer a device with onboard sensors or other features like a screen, microphone, etc. ++- **Development** ++ - **Programming language**: Consider if your project requires higher-level languages beyond C/C++. If so, identify the common programming languages for the application you need (for example, Machine Learning is often done in Python). Think about what SDKs, APIs, and/or libraries are helpful or necessary for your project. Identify what programming language(s) these are supported in. ++ - **IDE**: Consider the development environments that the device supports and if this meets the needs, skill set, and/or preferences of your developers. ++ - **Community**: Consider how much assistance you want/need in building a solution. For example, consider if you prefer to start with sample code, if you want troubleshooting advice or assistance, or if you would benefit from an active community that generates new samples and updates documentation. + + - **Documentation**: Take a look at the device documentation. Identify if it's complete and easy to follow. Consider if you need schematics, samples, datasheets, or other types of documentation. If so, do some searching to see if those items are available for your project. Consider the software SDKs/APIs/libraries that are written for the board and if these items would make your prototyping process easier. Identify if this documentation is maintained and who the maintainers are. ++- **Security** ++ - **Networking**: Consider if your device is connected to an external network or if it can be kept behind a router and/or firewall. If your prototype needs to be connected to an externally facing network, we recommend using the Azure Sphere as it is the only reliably secure device. ++ - **Peripherals**: Consider if any of the peripherals your device connects to have wireless protocols (for example, WiFi, BLE). ++ - **Physical location**: Consider if your device or any of the peripherals it's connected to will be accessible to the public. If so, we recommend making the device physically inaccessible. For example, in a closed, locked box. ++## Application selection visual ++>[!NOTE] +>This list is for educational purposes only, it is not intended to endorse any products. +> ++## Terminology and ML requirements ++This section provides definitions for embedded terminology and acronyms and hardware specifications for visual, auditory, and sensor machine learning applications. ++### Terminology ++Terminology and acronyms are listed in alphabetical order. ++| Term | Definition | +| - | | +| ADC | Analog to digital converter; converts analog signals from connected components like sensors to digital signals that are readable by the device | +| Analog pins | Used for connecting analog components that have continuous signals like photoresistors (light sensors) and microphones | +| Clock speed | How quickly the CPU can retrieve and interpret instructions | +| Digital pins | Used for connecting digital components that have binary signals like LEDs and switches | +| Flash (or ROM) | Memory available for storing programs | +| IDE | Integrated development environment; a program for writing software code | +| IMU | Inertial measurement unit | +| IO (or I/O) pins | Input/Output pins used for communicating with other devices like sensors and other controllers | +| MCU | Microcontroller Unit; a small computer on a single chip that includes a CPU, RAM, and IO | +| MPU | Microprocessor unit; a computer processor that incorporates the functions of a computer's central processing unit (CPU) on a single integrated circuit (IC), or at most a few integrated circuits. | +| ML | Machine learning; special computer programs that do complex pattern recognition | +| PWM | Pulse width modulation; a way to modify digital signals to achieve analog-like effects like changing brightness, volume, and speed | +| RAM | Random access memory; how much memory is available to run programs | +| SBC | Single board computer | +| TF | TensorFlow; a machine learning software package designed for edge devices | +| TF Lite | TensorFlow Lite; a smaller version of TF for small edge devices | ++### Machine learning hardware requirements ++#### Vision ML ++- Speed: 200 MHz +- Flash: 300 kB +- RAM: 100 kB ++#### Speech ML ++- Speed: 60 MHz [^4] +- Flash: 50 kB +- RAM: 8 kB ++#### Sensor ML (for example, motion, distance) ++- Speed: 20 MHz +- Flash: 20 kB +- RAM: 2 kB ++[^4]: *Speed requirement is largely due to the need for processors to be able to sample a minimum of 6 kHz for microphones to be able to process human vocal frequencies.* ++## MCU device list ++Following is a comparison table of MCUs in alphabetical order. The list isn't not intended to be exhaustive. ++>[!NOTE] +>This list is for educational purposes only, it is not intended to endorse any products. Prices shown represent the average across multiple distributors and are for illustrative purposes only. ++| Board Name | Price Range (USD) | What is it used for? | Software| Speed | Processor | Memory | Onboard Sensors and Other Features | IO Pins | Video | Radio | Battery Connector? | Operating Voltage | Getting Stated Guides | **Alternatives** | +| - | - | - | -| - | - | - | - | - | - | - | - | - | - | - | +| [Azure Sphere MT3620 Dev Kit](https://aka.ms/IotDeviceList/Sphere) | ~$40 - $100 | Highly secure applications | C/C++, VS Code, VS | 500 MHz & 200 MHz | MT3620 (tri-core--1 x Cortex A7, 2 x Cortex M4) | 4-MB RAM + 2 x 64-KB RAM | Certifications: CE/FCC/MIC/RoHS | 4 x Digital IO, 1 x I2S, 4 x ADC, 1 x RTC | - | Dual-band 802.11 b/g/n with antenna diversity | - | 5 V | 1. [Azure Sphere Samples Gallery](https://github.com/Azure/azure-sphere-gallery#azure-sphere-gallery), 2. [Azure Sphere Weather Station](https://www.hackster.io/gatoninja236/azure-sphere-weather-station-d5a2bc)| N/A | +| [Adafruit HUZZAH32 – ESP32 Feather Board](https://aka.ms/IotDeviceList/AdafruitFeather) | ~$20 - $25 | Monitoring; Beginner IoT; Home automation | Arduino IDE, VS Code | 240 MHz | 32-Bit ESP32 (dual-core Tensilica LX6) | 4 MB SPI Flash, 520 KB SRAM | Hall sensor, 10x capacitive touch IO pins, 50+ add-on boards | 3 x UARTs, 3 x SPI, 2 x I2C, 12 x ADC inputs, 2 x I2S Audio, 2 x DAC | - | 802.11b/g/n HT40 Wi-Fi transceiver, baseband, stack and LWIP, Bluetooth and BLE | √ | 3.3 V | 1. [Scientific freezer monitor](https://www.hackster.io/adi-azulay/azure-edge-impulse-scientific-freezer-monitor-5448ee), 2. [Azure IoT SDK Arduino samples](https://github.com/Azure/azure-sdk-for-c-arduino) | [Arduino Uno WiFi Rev 2 (~$50 - $60)](https://aka.ms/IotDeviceList/ArduinoUnoWifi) | +| [Arduino Nano 33 BLE Sense](https://aka.ms/IotDeviceList/ArduinoNanoBLE) | ~$30 - $35 | Monitoring; ML; Game controller; Beginner IoT | Arduino IDE, VS Code | 64 MHz | 32-bit Nordic nRF52840 (Cortex M4F) | 1 MB Flash, 256 KB SRAM | 9-axis inertial sensor, Humidity and temp sensor, Barometric sensor, Microphone, Gesture, proximity, light color and light intensity sensor | 14 x Digital IO, 1 x UART, 1 x SPI, 1 x I2C, 8 x ADC input | - | Bluetooth and BLE | - | 3.3 V – 21 V | 1. [Connect Nano BLE to Azure IoT Hub](https://create.arduino.cc/projecthub/Arduino_Genuino/securely-connecting-an-arduino-nb-1500-to-azure-iot-hub-af6470), 2. [Monitor beehive with Azure Functions](https://www.hackster.io/clementchamayou/how-to-monitor-a-beehive-with-arduino-nano-33ble-bluetooth-eabc0d) | [Seeed XIAO BLE sense (~$15 - $20)](https://aka.ms/IotDeviceList/SeeedXiao) | +| [Arduino Nano RP2040 Connect](https://aka.ms/IotDeviceList/ArduinoRP2040Nano) | ~$20 - $25 | Remote control; Monitoring | Arduino IDE, VS Code, C/C++, MicroPython | 133 MHz | 32-bit RP2040 (dual-core Cortex M0+) | 16 MB Flash, 264-kB RAM | Microphone, Six-axis IMU with AI capabilities | 22 x Digital IO, 20 x PWM, 8 x ADC | - | WiFi, Bluetooth | - | 3.3 V | - |[Adafruit Feather RP2040 (NOTE: also need a FeatherWing for WiFi)](https://aka.ms/IotDeviceList/AdafruitRP2040) | +| [ESP32-S2 Saola-1](https://aka.ms/IotDeviceList/ESPSaola) | ~$10 - $15 | Home automation; Beginner IoT; ML; Monitoring; Mesh networking | Arduino IDE, Circuit Python, ESP IDF | 240 MHz | 32-bit ESP32-S2 (single-core Xtensa LX7) | 128 kB Flash, 320 kB SRAM, 16 kB SRAM (RTC) | 14 x capacitive touch IO pins, Temp sensor | 43 x Digital pins, 8 x PWM, 20 x ADC, 2 x DAC | Serial LCD, Parallel PCD | Wi-Fi 802.11 b/g/n (802.11n up to 150 Mbps) | - | 3.3 V | 1. [Secure face detection with Azure ML](https://www.hackster.io/achindra/microsoft-azure-machine-learning-and-face-detection-in-iot-2de40a), 2. [Azure Cost Monitor](https://www.hackster.io/jenfoxbot/azure-cost-monitor-31811a) | [ESP32-DevKitC (~$10 - $15)](https://aka.ms/IotDeviceList/ESPDevKit) | +| [Wio Terminal (Seeed Studio)](https://aka.ms/IotDeviceList/WioTerminal) | ~$40 - $50 | Monitoring; Home Automation; ML | Arduino IDE, VS Code, MicroPython, ArduPy | 120 MHz | 32-bit ATSAMD51 (single-core Cortex-M4F) | 4 MB SPI Flash, 192-kB RAM | On-board screen, Microphone, IMU, buzzer, microSD slot, light sensor, IR emitter, Raspberry Pi GPIO mount (as child device) | 26 x Digital Pins, 5 x PWM, 9 x ADC | 2.4" 320x420 Color LCD | dual-band 2.4Ghz/5Ghz (Realtek RTL8720DN) | - | 3.3 V | [Monitor plants with Azure IoT](https://github.com/microsoft/IoT-For-Beginners/tree/main/2-farm/lessons/4-migrate-your-plant-to-the-cloud) | [Adafruit FunHouse (~$30 - $40)](https://aka.ms/IotDeviceList/AdafruitFunhouse) | ++## SBC device list ++Following is a comparison table of SBCs in alphabetical order. This list isn't intended to be exhaustive. ++>[!NOTE] +>This list is for educational purposes only, it is not intended to endorse any products. Prices shown represent the average across multiple distributors and are for illustrative purposes only. ++| Board Name | Price Range (USD) | What is it used for? | Software| Speed | Processor | Memory | Onboard Sensors and Other Features | IO Pins | Video | Radio | Battery Connector? | Operating Voltage | Getting Started Guides | **Alternatives** | +| - | - | - | -| - | - | - | - | - | - | - | - | - | - | -| +| [Raspberry Pi 4, Model B](https://aka.ms/IotDeviceList/RpiModelB) | ~$30 - $80 | Home automation; Robotics; Autonomous vehicles; Control systems; Field science | Raspberry Pi OS, Raspbian, Ubuntu 20.04/21.04, RISC OS, Windows 10 IoT, more | 1.5 GHz CPU, 500 MHz GPU | 64-bit Broadcom BCM2711 (quad-core Cortex-A72), VideoCore VI GPU | 2GB/4GB/8GB LPDDR4 RAM, SD Card (not included) | 2 x USB 3 ports, 1 x MIPI DSI display port, 1 x MIPI CSI camera port, 4-pole stereo audio and composite video port, Power over Ethernet (requires HAT) | 26 x Digital, 4 x PWM | 2 micro-HDMI composite, MPI DSI | WiFi, Bluetooth | √ | 5 V | 1. [Send data to IoT Hub](https://www.hackster.io/jenfoxbot/how-to-send-see-data-from-a-raspberry-pi-to-azure-iot-hub-908924), 2. [Monitor plants with Azure IoT](https://github.com/microsoft/IoT-For-Beginners/tree/main/2-farm/lessons/4-migrate-your-plant-to-the-cloud)| [BeagleBone Black Wireless (~$50 - $60)](https://www.beagleboard.org/boards/beaglebone-black-wireless) | +| [NVIDIA Jetson 2 GB Nano Dev Kit](https://aka.ms/IotDeviceList/NVIDIAJetson) | ~$50 - $100 | AI/ML; Autonomous vehicles | Ubuntu-based JetPack | 1.43 GHz CPU, 921 MHz GPU | 64-bit Nvidia CPU (quad-core Cortex-A57), 128-CUDA-core Maxwell GPU coprocessor | 2GB/4GB LPDDR4 RAM | 472 GFLOPS for AI Perf, 1 x MIPI CSI-2 connector | 28 x Digital, 2 x PWM | HDMI, DP (4 GB only) | Gigabit Ethernet, 802.11ac WiFi | √ | 5 V | [Deepstream integration with Azure IoT Central](https://www.hackster.io/pjdecarlo/nvidia-deepstream-integration-with-azure-iot-central-d9f834) | [BeagleBone AI (~$110 - $120)](https://aka.ms/IotDeviceList/BeagleBoneAI) | +| [Raspberry Pi Zero W2](https://aka.ms/IotDeviceList/RpiZeroW) | ~$15 - $20 | Home automation; ML; Vehicle modifications; Field Science | Raspberry Pi OS, Raspbian, Ubuntu 20.04/21.04, RISC OS, Windows 10 IoT, more | 1 GHz CPU, 400 MHz GPU | 64-bit Broadcom BCM2837 (quad-core Cortez-A53), VideoCore IV GPU | 512 MB LPDDR2 RAM, SD Card (not included) | 1 x CSI-2 Camera connector | 26 x Digital, 4 x PWM | Mini-HDMI | WiFi, Bluetooth | - | 5 V | [Send and visualize data to Azure IoT Hub](https://www.hackster.io/jenfoxbot/how-to-send-see-data-from-a-raspberry-pi-to-azure-iot-hub-908924) | [Onion Omega2+ (~$10 - $15)](https://onion.io/Omega2/) | +| [DFRobot LattePanda](https://aka.ms/IotDeviceList/DFRobotLattePanda) | ~$100 - $160 | Home automation; Hyperscale cloud connectivity; AI/ML | Windows 10, Ubuntu 16.04, OpenSuSE 15 | 1.92 GHz | 64-bit Intel Z8350 (quad-core x86-64), Atmega32u4 coprocessor | 2 GB DDR3L RAM, 32 GB eMMC/4GB DDR3L RAM, 64-GB eMMC | - | 6 x Digital (20 x via Atmega32u4), 6 x PWM, 12 x ADC | HDMI, MIPI DSI | WiFi, Bluetooth | √ | 5 V | 1. [Getting started with Microsoft Azure](https://www.hackster.io/45361/dfrobot-lattepanda-with-microsoft-azure-getting-started-0ae8fb), 2. [Home Monitoring System with Azure](https://www.hackster.io/JiongShi/home-monitoring-system-based-on-lattepanda-zigbee-and-azure-ce4e03)| [Seeed Odyssey X86J4125800 (~$210 - $230)](https://aka.ms/IotDeviceList/SeeedOdyssey) | ++## Questions? Requests? ++Please submit an issue! ++## See Also ++Other helpful resources include: ++- [Overview of Azure IoT device types](./concepts-iot-device-types.md) +- [Overview of Azure IoT Device SDKs](./iot-sdks.md) +- [Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub](./tutorial-send-telemetry-iot-hub.md?pivots=programming-language-ansi-c) +- [AzureRTOS ThreadX Documentation](/azure/rtos/threadx/) |
| iot | Concepts Iot Device Types | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/concepts-iot-device-types.md | + + Title: Overview of Azure IoT device types +description: Learn the different device types supported by Azure IoT and the tools available. ++++ Last updated : 04/04/2024+++# Overview of Azure IoT device types +IoT devices exist across a broad selection of hardware platforms. There are small 8-bit MCUs all the way up to the latest x86 CPUs as found in a desktop computer. Many variables factor into the decision for which hardware you to choose for a IoT device and this article outlined some of the key differences. ++## Key hardware differentiators +Some important factors when choosing your hardware are cost, power consumption, networking, and available inputs and outputs. ++* **Cost:** Smaller cheaper devices are typically used when mass producing the final product. However the trade-off is that development of the device can be more expensive given the highly constrained device. The development cost can be spread across all produced devices so the per unit development cost will be low. ++* **Power:** How much power a device consumes is important if the device will be utilizing batteries and not connected to the power grid. MCUs are often designed for lower power scenarios and can be a better choice for extending battery life. ++* **Network Access:** There are many ways to connect a device to a cloud service. Ethernet, Wi-fi and cellular and some of the available options. The connection type you choose will depend on where the device is deployed and how it's used. For example, cellular can be an attractive option given the high coverage, however for high traffic devices it can an expensive. Hardwired ethernet provides cheaper data costs but with the downside of being less portable. ++* **Input and Outputs:** The inputs and outputs available on the device directly affect the devices operating capabilities. A microcontroller will typically have many I/O functions built directly into the chip and provides a wide choice of sensors to connect directly. ++## Microcontrollers vs Microprocessors +IoT devices can be separated into two broad categories, microcontrollers (MCUs) and microprocessors (MPUs). ++**MCUs** are less expensive and simpler to operate than MPUs. An MCU will contain many of the functions, such as memory, interfaces, and I/O within the chip itself. An MPU will draw this functionality from components in supporting chips. An MCU will often use a real-time OS (RTOS) or run bare-metal (No OS) and provide real-time response and highly deterministic reactions to external events. ++**MPUs** will generally run a general purpose OS, such as Windows, Linux, or MacOSX that provide a non-deterministic real-time response. There's typically no guarantee to when a task will be completed. +++Below is a table showing some of the defining differences between an MCU and an MPU based system: ++||Microcontroller (MCU)|Microprocessor (MPU)| +|-|-|-| +|**CPU**| Less | More | +|**RAM**| Less | More | +|**Flash**| Less | More | +|**OS**| Bare Metal / RTOS | General Purpose (Windows / Linux) | +|**Development Difficulty**| Harder | Easier | +|**Power Consumption**| Lower | Higher | +|**Cost**| Lower | Higher | +|**Deterministic**| Yes | No - with exceptions | +|**Device Size**| Smaller | Larger | ++## Next steps +The IoT device type that you choose directly impacts how the device is connected to Azure IoT. ++Browse the different [Azure IoT SDKs](./iot-sdks.md) to find the one that best suits your device needs. |
| iot | Concepts Manage Device Reconnections | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/concepts-manage-device-reconnections.md | + + Title: Manage device reconnections to create resilient applications ++description: Manage the device connection and reconnection process to ensure resilient applications by using the Azure IoT Hub device SDKs. +++ Last updated : 04/04/2024++++++# Manage device reconnections to create resilient applications ++This article provides high-level guidance to help you design resilient applications by adding a device reconnection strategy. It explains why devices disconnect and need to reconnect. And it describes specific strategies that developers can use to reconnect devices that have been disconnected. ++## What causes disconnections +The following are the most common reasons that devices disconnect from IoT Hub: ++- Expired SAS token or X.509 certificate. The device's SAS token or X.509 authentication certificate expired. +- Network interruption. The device's connection to the network is interrupted. +- Service disruption. The Azure IoT Hub service experiences errors or is temporarily unavailable. +- Service reconfiguration. After you reconfigure IoT Hub service settings, it can cause devices to require reprovisioning or reconnection. ++## Why you need a reconnection strategy ++It's important to have a strategy to reconnect devices as described in the following sections. Without a reconnection strategy, you could see a negative effect on your solution's performance, availability, and cost. ++### Mass reconnection attempts could cause a DDoS ++A high number of connection attempts per second can cause a condition similar to a distributed denial-of-service attack (DDoS). This scenario is relevant for large fleets of devices numbering in the millions. The issue can extend beyond the tenant that owns the fleet, and affect the entire scale-unit. A DDoS could drive a large cost increase for your Azure IoT Hub resources, due to a need to scale out. A DDoS could also hurt your solution's performance due to resource starvation. In the worse case, a DDoS can cause service interruption. ++### Hub failure or reconfiguration could disconnect many devices ++After an IoT hub experiences a failure, or after you reconfigure service settings on an IoT hub, devices might be disconnected. For proper failover, disconnected devices require reprovisioning. To learn more about failover options, see [IoT Hub high availability and disaster recovery](../iot-hub/iot-hub-ha-dr.md). ++### Reprovisioning many devices could increase costs ++After devices disconnect from IoT Hub, the optimal solution is to reconnect the device rather than reprovision it. If you use IoT Hub with DPS, DPS has a per provisioning cost. If you reprovision many devices on DPS, it increases the cost of your IoT solution. To learn more about DPS provisioning costs, see [IoT Hub DPS pricing](https://azure.microsoft.com/pricing/details/iot-hub). ++## Design for resiliency ++IoT devices often rely on noncontinuous or unstable network connections (for example, GSM or satellite). Errors can occur when devices interact with cloud-based services because of intermittent service availability and infrastructure-level or transient faults. An application that runs on a device has to manage the mechanisms for connection, reconnection, and the retry logic for sending and receiving messages. Also, the retry strategy requirements depend heavily on the device's IoT scenario, context, capabilities. ++The Azure IoT Hub device SDKs aim to simplify connecting and communicating from cloud-to-device and device-to-cloud. These SDKs provide a robust way to connect to Azure IoT Hub and a comprehensive set of options for sending and receiving messages. Developers can also modify existing implementation to customize a better retry strategy for a given scenario. ++The relevant SDK features that support connectivity and reliable messaging are available in the following IoT Hub device SDKs. For more information, see the API documentation or specific SDK: ++* [C SDK](https://github.com/Azure/azure-iot-sdk-c/blob/main/doc/connection_and_messaging_reliability.md) ++* [.NET SDK](https://github.com/Azure/azure-iot-sdk-csharp/blob/main/iothub/device/devdoc/retrypolicy.md) ++* [Java SDK](https://github.com/Azure/azure-iot-sdk-jav) ++* [Node SDK](https://github.com/Azure/azure-iot-sdk-node/wiki/Connectivity-and-Retries) ++* [Python SDK](https://github.com/Azure/azure-iot-sdk-python) ++The following sections describe SDK features that support connectivity. ++## Connection and retry ++This section gives an overview of the reconnection and retry patterns available when managing connections. It details implementation guidance for using a different retry policy in your device application and lists relevant APIs from the device SDKs. ++### Error patterns ++Connection failures can happen at many levels: ++* Network errors: disconnected socket and name resolution errors ++* Protocol-level errors for HTTP, AMQP, and MQTT transport: detached links or expired sessions ++* Application-level errors that result from either local mistakes: invalid credentials or service behavior (for example, exceeding the quota or throttling) ++The device SDKs detect errors at all three levels. However, device SDKs don't detect and handle OS-related errors and hardware errors. The SDK design is based on [The Transient Fault Handling Guidance](/azure/architecture/best-practices/transient-faults#general-guidelines) from the Azure Architecture Center. ++### Retry patterns ++The following steps describe the retry process when connection errors are detected: ++1. The SDK detects the error and the associated error in the network, protocol, or application. ++1. The SDK uses the error filter to determine the error type and decide if a retry is needed. ++1. If the SDK identifies an **unrecoverable error**, operations like connection, send, and receive are stopped. The SDK notifies the user. Examples of unrecoverable errors include an authentication error and a bad endpoint error. ++1. If the SDK identifies a **recoverable error**, it retries according to the specified retry policy until the defined timeout elapses. The SDK uses **Exponential back-off with jitter** retry policy by default. ++1. When the defined timeout expires, the SDK stops trying to connect or send. It notifies the user. ++1. The SDK allows the user to attach a callback to receive connection status changes. ++The SDKs typically provide three retry policies: ++* **Exponential back-off with jitter**: This default retry policy tends to be aggressive at the start and slow down over time until it reaches a maximum delay. The design is based on [Retry guidance from Azure Architecture Center](/azure/architecture/best-practices/retry-service-specific). ++* **Custom retry**: For some SDK languages, you can design a custom retry policy that is better suited for your scenario and then inject it into the RetryPolicy. Custom retry isn't available on the C SDK, and it isn't currently supported on the Python SDK. The Python SDK reconnects as-needed. ++* **No retry**: You can set retry policy to "no retry", which disables the retry logic. The SDK tries to connect once and send a message once, assuming the connection is established. This policy is typically used in scenarios with bandwidth or cost concerns. If you choose this option, messages that fail to send are lost and can't be recovered. ++### Retry policy APIs ++| SDK | SetRetryPolicy method | Policy implementations | Implementation guidance | +||||| +| C | [IOTHUB_CLIENT_RESULT IoTHubDeviceClient_SetRetryPolicy](https://azure.github.io/azure-iot-sdk-c/iothub__device__client_8h.html#a53604d8d75556ded769b7947268beec8) | See: [IOTHUB_CLIENT_RETRY_POLICY](https://azure.github.io/azure-iot-sdk-c/iothub__client__core__common_8h.html#a361221e523247855ff0a05c2e2870e4a) | [C implementation](https://github.com/Azure/azure-iot-sdk-c/blob/master/doc/connection_and_messaging_reliability.md) | +| Java | [SetRetryPolicy](/jav) | +| .NET | [DeviceClient.SetRetryPolicy](/dotnet/api/microsoft.azure.devices.client.deviceclient.setretrypolicy) | **Default**: [ExponentialBackoff class](/dotnet/api/microsoft.azure.devices.client.exponentialbackoff)<BR>**Custom:** implement [IRetryPolicy interface](/dotnet/api/microsoft.azure.devices.client.iretrypolicy)<BR>**No retry:** [NoRetry class](/dotnet/api/microsoft.azure.devices.client.noretry) | [C# implementation](https://github.com/Azure/azure-iot-sdk-csharp/blob/main/iothub/device/devdoc/retrypolicy.md) | +| Node | [setRetryPolicy](/javascript/api/azure-iot-device/client#azure-iot-device-client-setretrypolicy) | **Default**: [ExponentialBackoffWithJitter class](/javascript/api/azure-iot-common/exponentialbackoffwithjitter)<BR>**Custom:** implement [RetryPolicy interface](/javascript/api/azure-iot-common/retrypolicy)<BR>**No retry:** [NoRetry class](/javascript/api/azure-iot-common/noretry) | [Node implementation](https://github.com/Azure/azure-iot-sdk-node/wiki/Connectivity-and-Retries) | +| Python | Not currently supported | Not currently supported | Built-in connection retries: Dropped connections are retried with a fixed 10-second interval by default. This functionality can be disabled if desired, and the interval can be configured. | ++## Hub reconnection flow ++If you use IoT Hub only without DPS, use the following reconnection strategy. ++When a device fails to connect to IoT Hub, or is disconnected from IoT Hub: ++1. Use an exponential back-off with jitter delay function. +1. Reconnect to IoT Hub. ++The following diagram summarizes the reconnection flow: ++++## Hub with DPS reconnection flow ++If you use IoT Hub with DPS, use the following reconnection strategy. ++When a device fails to connect to IoT Hub, or is disconnected from IoT Hub, reconnect based on the following cases: ++|Reconnection scenario | Reconnection strategy | +||| +|For errors that allow connection retries (HTTP response code 500) | Use an exponential back-off with jitter delay function. <br> Reconnect to IoT Hub. | +|For errors that indicate a retry is possible, but reconnection has failed 10 consecutive times | Reprovision the device to DPS. | +|For errors that don't allow connection retries (HTTP responses 401, Unauthorized or 403, Forbidden or 404, Not Found) | Reprovision the device to DPS. | ++The following diagram summarizes the reconnection flow: +++## Next steps ++Suggested next steps include: ++- [Troubleshoot device disconnects](../iot-hub/iot-hub-troubleshoot-connectivity.md) ++- [Deploy devices at scale](../iot-dps/concepts-deploy-at-scale.md) |
| iot | Concepts Using C Sdk And Embedded C Sdk | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/concepts-using-c-sdk-and-embedded-c-sdk.md | + + Title: C SDK and Embedded C SDK usage scenarios +description: Helps developers decide which C-based Azure IoT device SDK to use for device development, based on their usage scenario. ++++ Last updated : 04/04/2024++#Customer intent: As a device developer, I want to understand when to use the Azure IoT C SDK or the Embedded C SDK to optimize device and application performance. +++# C SDK and Embedded C SDK usage scenarios ++Microsoft provides Azure IoT device SDKs and middleware for embedded and constrained device scenarios. This article helps device developers decide which one to use for your application. ++The following diagram shows four common scenarios in which customers connect devices to Azure IoT, using a C-based (C99) SDK. The rest of this article provides more details on each scenario. +++## Scenario 1 ΓÇô Azure IoT C SDK (for Linux and Windows) ++Starting in 2015, [Azure IoT C SDK](https://github.com/Azure/azure-iot-sdk-c) was the first Azure SDK created to connect devices to IoT services. It's a stable platform that was built to provide the following capabilities for connecting devices to Azure IoT: +- IoT Hub services +- Device Provisioning Service clients +- Three choices of communication transport (MQTT, AMQP and HTTP), which are created and maintained by Microsoft +- Multiple choices of common TLS stacks (OpenSSL, Schannel and Bed TLS according to the target platform) +- TCP sockets (Win32, Berkeley or Mbed) ++Providing communication transport, TLS and socket abstraction has a performance cost. Many paths require `malloc` and `memcpy` calls between the various abstraction layers. This performance cost is small compared to a desktop or a Raspberry Pi device. Yet on a truly constrained device, the cost becomes significant overhead with the possibility of memory fragmentation. The communication transport layer also requires a `doWork` function to be called at least every 100 milliseconds. These frequent calls make it harder to optimize the SDK for battery powered devices. The existence of multiple abstraction layers also makes it hard for customers to use or change to any given library. ++Scenario 1 is recommended for Windows or Linux devices, which normally are less sensitive to memory usage or power consumption. However, Windows and Linux-based devices can also use the Embedded C SDK as shown in Scenario 2. Other options for windows and Linux-based devices include the other Azure IoT device SDKs: [Java SDK](https://github.com/Azure/azure-iot-sdk-java), [.NET SDK](https://github.com/Azure/azure-iot-sdk-csharp), [Node SDK](https://github.com/Azure/azure-iot-sdk-node) and [Python SDK](https://github.com/Azure/azure-iot-sdk-python). ++## Scenario 2 ΓÇô Embedded C SDK (for Bare Metal scenarios and micro-controllers) ++In 2020, Microsoft released the [Azure SDK for Embedded C](https://github.com/Azure/azure-sdk-for-c/tree/main/sdk/docs/iot) (also known as the Embedded C SDK). This SDK was built based on customers feedback and a growing need to support constrained [micro-controller devices](./concepts-iot-device-types.md#microcontrollers-vs-microprocessors). Typically, constrained micro-controllers have reduced memory and processing power. ++The Embedded C SDK has the following key characteristics: +- No dynamic memory allocation. Customers must allocate data structures where they desire such as in global memory, a heap, or a stack. Then they must pass the address of the allocated structure into SDK functions to initialize and perform various operations. +- MQTT only. MQTT-only usage is ideal for constrained devices because it's an efficient, lightweight network protocol. Currently only MQTT v3.1.1 is supported. +- Bring your own network stack. The Embedded C SDK performs no I/O operations. This approach allows customers to select the MQTT, TLS and Socket clients that have the best fit to their target platform. +- Similar [feature set](./concepts-iot-device-types.md#microcontrollers-vs-microprocessors) as the C SDK. The Embedded C SDK provides similar features as the Azure IoT C SDK, with the following exceptions that the Embedded C SDK doesn't provide: + - Upload to blob + - The ability to run as an IoT Edge module + - AMQP-based features like content message batching and device multiplexing +- Smaller overall [footprint](https://github.com/Azure/azure-sdk-for-c/tree/main/sdk/docs/iot#size-chart). The Embedded C SDK, as see in a sample that shows how to connect to IoT Hub, can take as little as 74 KB of ROM and 8.26 KB of RAM. ++The Embedded C SDK supports micro-controllers with no operating system, micro-controllers with a real-time operating system (like Azure RTOS), Linux, and Windows. Customers can implement custom platform layers to use the SDK on custom devices. The SDK also provides some platform layers such as [Arduino](https://github.com/Azure/azure-sdk-for-c-arduino), and [Swift](https://github.com/Azure-Samples/azure-sdk-for-c-swift). Microsoft encourages the community to submit other platform layers to increase the out-of-the-box supported platforms. Wind River [VxWorks](https://github.com/Azure/azure-sdk-for-c/blob/main/sdk/samples/iot/docs/how_to_iot_hub_samples_vxworks.md) is an example of a platform layer submitted by the community. ++The Embedded C SDK adds some programming benefits because of its flexibility compared to the Azure IoT C SDK. In particular, applications that use constrained devices will benefit from enormous resource savings and greater programmatic control. In comparison, if you use Azure RTOS or FreeRTOS, you can have these same benefits along with other features per RTOS implementation. ++## Scenario 3 ΓÇô Azure RTOS with Azure RTOS middleware (for Azure RTOS-based projects) ++Scenario 3 involves using Azure RTOS and the [Azure RTOS middleware](https://github.com/azure-rtos/netxduo/tree/master/addons/azure_iot). Azure RTOS is built on top of the Embedded C SDK, and adds MQTT and TLS Support. The middleware for Azure RTOS exposes APIs for the application that are similar to the native Azure RTOS APIs. This approach makes it simpler for developers to use the APIs and connect their Azure RTOS-based devices to Azure IoT. Azure RTOS is a fully integrated, efficient, real time embedded platform, that provides all the networking and IoT features you need for your solution. ++Samples for several popular developer kits from ST, NXP, Renesas, and Microchip, are available. These samples work with Azure IoT Hub or Azure IoT Central, and are available as IAR Workbench or semiconductor IDE projects on [GitHub](https://github.com/azure-rtos/samples). ++Because it's based on the Embedded C SDK, the Azure IoT middleware for Azure RTOS is non-memory allocating. Customers must allocate SDK data structures in global memory, or a heap, or a stack. After customers allocate a data structure, they must pass the address of the structure into the SDK functions to initialize and perform various operations. ++## Scenario 4 ΓÇô FreeRTOS with FreeRTOS middleware (for use with FreeRTOS-based projects) ++Scenario 4 brings the embedded C middleware to FreeRTOS. The embedded C middleware is built on top of the Embedded C SDK and adds MQTT support via the open source coreMQTT library. This middleware for FreeRTOS operates at the MQTT level. It establishes the MQTT connection, subscribes and unsubscribes from topics, and sends and receives messages. Disconnections are handled by the customer via middleware APIs. ++Customers control the TLS/TCP configuration and connection to the endpoint. This approach allows for flexibility between software or hardware implementations of either stack. No background tasks are created by the Azure IoT middleware for FreeRTOS. Messages are sent and received synchronously. ++The core implementation is provided in this [GitHub repository](https://github.com/Azure/azure-iot-middleware-freertos). Samples for several popular developer kits are available, including the NXP1060, STM32, and ESP32. The samples work with Azure IoT Hub, Azure IoT Central, and Azure Device Provisioning Service, and are available in this [GitHub repository](https://github.com/Azure-Samples/iot-middleware-freertos-samples). ++Because it's based on the Azure Embedded C SDK, the Azure IoT middleware for FreeRTOS is also non-memory allocating. Customers must allocate SDK data structures in global memory, or a heap, or a stack. After customers allocate a data structure, they must pass the address of the allocated structures into the SDK functions to initialize and perform various operations. ++## C-based SDK technical usage scenarios ++The following diagram summarizes technical options for each SDK usage scenario described in this article. +++## C-based SDK comparison by memory and protocols ++The following table compares the four device SDK development scenarios based on memory and protocol usage. ++| | **Memory <br>allocation** | **Memory <br>usage** | **Protocols <br>supported** | **Recommended for** | +| :-- | :-- | :-- | :-- | :-- | +| **Azure IoT C SDK** | Mostly Dynamic | Unrestricted. Can span <br>to 1 MB or more in RAM. | AMQP<br>HTTP<br>MQTT v3.1.1 | Microprocessor-based systems<br>Microsoft Windows<br>Linux<br>Apple OS X | +| **Azure SDK for Embedded C** | Static only | Restricted by amount of <br>data application allocates. | MQTT v3.1.1 | Micro-controllers <br>Bare-metal Implementations <br>RTOS-based implementations | +| **Azure IoT Middleware for Azure RTOS** | Static only | Restricted | MQTT v3.1.1 | Micro-controllers <br>RTOS-based implementations | +| **Azure IoT Middleware for FreeRTOS** | Static only | Restricted | MQTT v3.1.1 | Micro-controllers <br>RTOS-based implementations | ++## Azure IoT Features Supported by each SDK ++The following table compares the four device SDK development scenarios based on support for Azure IoT features. ++| | **Azure IoT C SDK** | **Azure SDK for <br>Embedded C** | **Azure IoT <br>middleware for <br>Azure RTOS** | **Azure IoT <br>middleware for <br>FreeRTOS** | +| :-- | :-- | :-- | :-- | :-- | +| SAS Client Authentication | Yes | Yes | Yes | Yes | +| x509 Client Authentication | Yes | Yes | Yes | Yes | +| Device Provisioning | Yes | Yes | Yes | Yes | +| Telemetry | Yes | Yes | Yes | Yes | +| Cloud-to-Device Messages | Yes | Yes | Yes | Yes | +| Direct Methods | Yes | Yes | Yes | Yes | +| Device Twin | Yes | Yes | Yes | Yes | +| IoT Plug-And-Play | Yes | Yes | Yes | Yes | +| Telemetry batching <br>(AMQP, HTTP) | Yes | No | No | No | +| Uploads to Azure Blob | Yes | No | No | No | +| Automatic integration in <br>IoT Edge hosted containers | Yes | No | No | No | +++## Next steps ++To learn more about device development and the available SDKs for Azure IoT, see the following table. +- [Azure IoT Device Development](./iot-overview-device-development.md) +- [Which SDK should I use](./iot-sdks.md) |
| iot | Iot Introduction | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/iot-introduction.md | An IoT device is typically made up of a circuit board with sensors attached that * An accelerometer in an elevator. * Presence sensors in a room. -There's a wide variety of devices available from different manufacturers to build your solution. For prototyping a microprocessor device, you can use a device such as a [Raspberry Pi](https://www.raspberrypi.org/). The Raspberry Pi lets you attach many different types of sensor. For prototyping a microcontroller device, use devices such as the [ESPRESSIF ESP32](../iot-develop/quickstart-devkit-espressif-esp32-freertos-iot-hub.md), [STMicroelectronics B-U585I-IOT02A Discovery kit](../iot-develop/quickstart-devkit-stm-b-u585i-iot-hub.md), [STMicroelectronics B-L4S5I-IOT01A Discovery kit](../iot-develop/quickstart-devkit-stm-b-l4s5i-iot-hub.md), or [NXP MIMXRT1060-EVK Evaluation kit](../iot-develop/quickstart-devkit-nxp-mimxrt1060-evk-iot-hub.md). These boards typically have built-in sensors, such as temperature and accelerometer sensors. +There's a wide variety of devices available from different manufacturers to build your solution. For prototyping a microprocessor device, you can use a device such as a [Raspberry Pi](https://www.raspberrypi.org/). The Raspberry Pi lets you attach many different types of sensor. For prototyping a microcontroller device, use devices such as the [ESPRESSIF ESP32](./tutorial-devkit-espressif-esp32-freertos-iot-hub.md), [STMicroelectronics B-U585I-IOT02A Discovery kit](../iot-develop/quickstart-devkit-stm-b-u585i-iot-hub.md), [STMicroelectronics B-L4S5I-IOT01A Discovery kit](../iot-develop/quickstart-devkit-stm-b-l4s5i-iot-hub.md), or [NXP MIMXRT1060-EVK Evaluation kit](../iot-develop/quickstart-devkit-nxp-mimxrt1060-evk-iot-hub.md). These boards typically have built-in sensors, such as temperature and accelerometer sensors. Microsoft provides open-source [Device SDKs](../iot-hub/iot-hub-devguide-sdks.md) that you can use to build the apps that run on your devices. |
| iot | Iot Mqtt Connect To Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/iot-mqtt-connect-to-iot-hub.md | In the **CONNECT** packet, the device should use the following values: You can also use the cross-platform [Azure IoT Hub extension for Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=vsciot-vscode.azure-iot-toolkit) or the CLI extension command [az iot hub generate-sas-token](/cli/azure/iot/hub#az-iot-hub-generate-sas-token) to quickly generate a SAS token. You can then copy and paste the SAS token into your own code for testing purposes. -For a tutorial on using MQTT directly, see [Use MQTT to develop an IoT device client without using a device SDK](../iot-develop/tutorial-use-mqtt.md). +For a tutorial on using MQTT directly, see [Use MQTT to develop an IoT device client without using a device SDK](./tutorial-use-mqtt.md). ### Using the Azure IoT Hub extension for Visual Studio Code The [IoT MQTT Sample repository](https://github.com/Azure-Samples/IoTMQTTSample) The C/C++ samples use the [Eclipse Mosquitto](https://mosquitto.org) library, the Python sample uses [Eclipse Paho](https://www.eclipse.org/paho/), and the CLI samples use `mosquitto_pub`. -To learn more, see [Tutorial - Use MQTT to develop an IoT device client](../iot-develop/tutorial-use-mqtt.md). +To learn more, see [Tutorial - Use MQTT to develop an IoT device client](./tutorial-use-mqtt.md). ## TLS/SSL configuration For more information, see [Understand and invoke direct methods from IoT Hub](.. To learn more about using MQTT, see: * [MQTT documentation](https://mqtt.org/)-* [Use MQTT to develop an IoT device client without using a device SDK](../iot-develop/tutorial-use-mqtt.md) +* [Use MQTT to develop an IoT device client without using a device SDK](./tutorial-use-mqtt.md) * [MQTT application samples](https://github.com/Azure-Samples/MqttApplicationSamples) To learn more about using IoT device SDKS, see: |
| iot | Iot Overview Device Connectivity | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/iot-overview-device-connectivity.md | A device can establish a secure connection to an IoT hub: The advantage of using DPS is that you don't need to configure all of your devices with connection-strings that are specific to your IoT hub. Instead, you configure your devices to connect to a well-known, common DPS endpoint where they discover their connection details. To learn more, see [Device Provisioning Service](../iot-dps/about-iot-dps.md). -To learn more about implementing automatic reconnections to endpoints, see [Manage device reconnections to create resilient applications](../iot-develop/concepts-manage-device-reconnections.md). +To learn more about implementing automatic reconnections to endpoints, see [Manage device reconnections to create resilient applications](./concepts-manage-device-reconnections.md). ## Device connection strings |
| iot | Iot Overview Device Development | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/iot-overview-device-development.md | The [IoT device development](../iot-develop/about-iot-develop.md) site includes You can find more samples in the [code sample browser](/samples/browse/?expanded=azure&products=azure-iot%2Cazure-iot-edge%2Cazure-iot-pnp%2Cazure-rtos). -To learn more about implementing automatic reconnections to endpoints, see [Manage device reconnections to create resilient applications](../iot-develop/concepts-manage-device-reconnections.md). +To learn more about implementing automatic reconnections to endpoints, see [Manage device reconnections to create resilient applications](./concepts-manage-device-reconnections.md). ## Device development without a device SDK |
| iot | Iot Overview Scalability High Availability | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/iot-overview-scalability-high-availability.md | You can scale the IoT Hub service vertically and horizontally. For an automated For a guide to scalability in an IoT Central solution, see [What does it mean for IoT Central to have elastic scale](../iot-central/core/concepts-faq-scalability-availability.md#scalability). If you're using private endpoints with your IoT Central solution, you need to [plan the size of the subnet in your virtual network](../iot-central/core/concepts-private-endpoints.md#plan-the-size-of-the-subnet-in-your-virtual-network). -For devices that connect to an IoT hub directly or to an IoT hub in an IoT Central application, make sure that the devices continue to connect as your solution scales. To learn more, see [Manage device reconnections after autoscale](../iot-develop/concepts-manage-device-reconnections.md) and [Handle connection failures](../iot-central/core/concepts-device-implementation.md#best-practices). +For devices that connect to an IoT hub directly or to an IoT hub in an IoT Central application, make sure that the devices continue to connect as your solution scales. To learn more, see [Manage device reconnections after autoscale](./concepts-manage-device-reconnections.md) and [Handle connection failures](../iot-central/core/concepts-device-implementation.md#best-practices). IoT Edge can help to help scale your solution. IoT Edge lets you move cloud analytics and custom business logic from the cloud to your devices. This approach lets your cloud solution focus on business insights instead of data management. Scale out your IoT solution by packaging your business logic into standard containers, deploy those containers to your devices, and monitor them from the cloud. For more information, see [Azure IoT Edge](../iot-edge/about-iot-edge.md). The following tutorials and guides provide more detail and guidance: - [Tutorial: Perform manual failover for an IoT hub](../iot-hub/tutorial-manual-failover.md) - [How to manually migrate an Azure IoT hub to a new Azure region](../iot-hub/migrate-hub-arm.md)-- [Manage device reconnections to create resilient applications (IoT Hub and IoT Central)](../iot-develop/concepts-manage-device-reconnections.md)+- [Manage device reconnections to create resilient applications (IoT Hub and IoT Central)](./concepts-manage-device-reconnections.md) - [IoT Central device best practices](../iot-central/core/concepts-device-implementation.md#best-practices) ## Next steps |
| iot | Iot Sdks | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/iot-sdks.md | To learn more about how to use the device SDKs, see [What is Azure IoT device an Use the embedded device SDKs to develop code to run on IoT devices that connect to IoT Hub or IoT Central. -To learn more about when to use the embedded device SDKs, see [C SDK and Embedded C SDK usage scenarios](../iot-develop/concepts-using-c-sdk-and-embedded-c-sdk.md). +To learn more about when to use the embedded device SDKs, see [C SDK and Embedded C SDK usage scenarios](./concepts-using-c-sdk-and-embedded-c-sdk.md). ### Device SDK lifecycle and support |
| iot | Tutorial Devkit Espressif Esp32 Freertos Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/tutorial-devkit-espressif-esp32-freertos-iot-hub.md | + + Title: Connect an ESPRESSIF ESP-32 to Azure IoT Hub quickstart +description: Use Azure IoT middleware for FreeRTOS to connect an ESPRESSIF ESP32-Azure IoT Kit device to Azure IoT Hub and send telemetry. ++++ms.devlang: c + Last updated : 04/04/2024+#Customer intent: As a device builder, I want to see a working IoT device sample using FreeRTOS to connect to Azure IoT Hub. The device should be able to send telemetry and respond to commands. As a solution builder, I want to use a tool to view the properties, commands, and telemetry an IoT Plug and Play device reports to the IoT hub it connects to. +++# Quickstart: Connect an ESPRESSIF ESP32-Azure IoT Kit to IoT Hub ++In this quickstart, you use the Azure IoT middleware for FreeRTOS to connect the ESPRESSIF ESP32-Azure IoT Kit (from now on, the ESP32 DevKit) to Azure IoT. ++You complete the following tasks: ++* Install a set of embedded development tools for programming an ESP32 DevKit +* Build an image and flash it onto the ESP32 DevKit +* Use Azure CLI to create and manage an Azure IoT hub that the ESP32 DevKit connects to +* Use Azure IoT Explorer to register a device with your IoT hub, view device properties, view device telemetry, and call direct commands on the device ++## Prerequisites ++* A PC running Windows 10 or Windows 11 +* [Git](https://git-scm.com/downloads) for cloning the repository +* Hardware + * ESPRESSIF [ESP32-Azure IoT Kit](https://www.espressif.com/products/devkits/esp32-azure-kit/overview) + * USB 2.0 A male to Micro USB male cable + * Wi-Fi 2.4 GHz +* An active Azure subscription. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin. ++## Prepare the development environment ++### Install the tools +To set up your development environment, first you install the ESPRESSIF ESP-IDF build environment. The installer includes all the tools required to clone, build, flash, and monitor your device. ++To install the ESP-IDF tools: +1. Download and launch the [ESP-IDF v5.0 Offline-installer](https://dl.espressif.com/dl/esp-idf). +1. When the installer lists components to install, select all components and complete the installation. +++### Clone the repo ++Clone the following repo to download all sample device code, setup scripts, and SDK documentation. If you previously cloned this repo, you don't need to do it again. ++To clone the repo, run the following command: ++```shell +git clone --recursive https://github.com/Azure-Samples/iot-middleware-freertos-samples.git +``` ++For Windows 10 and 11, make sure long paths are enabled. ++1. To enable long paths, see [Enable long paths in Windows 10](/windows/win32/fileio/maximum-file-path-limitation?tabs=registry). +1. In git, run the following command in a terminal with administrator permissions: ++ ```shell + git config --system core.longpaths true + ``` +++## Prepare the device +To connect the ESP32 DevKit to Azure, you modify configuration settings, build the image, and flash the image to the device. ++### Set up the environment +To launch the ESP-IDF environment: +1. Select Windows **Start**, find **ESP-IDF 5.0 CMD** and run it. +1. In **ESP-IDF 5.0 CMD**, navigate to the *iot-middleware-freertos-samples* directory that you cloned previously. +1. Navigate to the ESP32-Azure IoT Kit project directory *demos\projects\ESPRESSIF\aziotkit*. +1. Run the following command to launch the configuration menu: ++ ```shell + idf.py menuconfig + ``` ++### Add configuration ++To add wireless network configuration: +1. In **ESP-IDF 5.0 CMD**, select **Azure IoT middleware for FreeRTOS Sample Configuration >**, and press <kbd>Enter</kbd>. +1. Set the following configuration settings using your local wireless network credentials. ++ |Setting|Value| + |-|--| + |**WiFi SSID** |{*Your Wi-Fi SSID*}| + |**WiFi Password** |{*Your Wi-Fi password*}| ++1. Select <kbd>Esc</kbd> to return to the previous menu. ++To add configuration to connect to Azure IoT Hub: +1. Select **Azure IoT middleware for FreeRTOS Main Task Configuration >**, and press <kbd>Enter</kbd>. +1. Set the following Azure IoT configuration settings to the values that you saved after you created Azure resources. ++ |Setting|Value| + |-|--| + |**Azure IoT Hub FQDN** |{*Your host name*}| + |**Azure IoT Device ID** |{*Your Device ID*}| + |**Azure IoT Device Symmetric Key** |{*Your primary key*}| ++ > [!NOTE] + > In the setting **Azure IoT Authentication Method**, confirm that the default value of *Symmetric Key* is selected. ++1. Select <kbd>Esc</kbd> to return to the previous menu. +++To save the configuration: +1. Select <kbd>Shift</kbd>+<kbd>S</kbd> to open the save options. This menu lets you save the configuration to a file named *skconfig* in the current *.\aziotkit* directory. +1. Select <kbd>Enter</kbd> to save the configuration. +1. Select <kbd>Enter</kbd> to dismiss the acknowledgment message. +1. Select <kbd>Q</kbd> to quit the configuration menu. +++### Build and flash the image +In this section, you use the ESP-IDF tools to build, flash, and monitor the ESP32 DevKit as it connects to Azure IoT. ++> [!NOTE] +> In the following commands in this section, use a short build output path near your root directory. Specify the build path after the `-B` parameter in each command that requires it. The short path helps to avoid a current issue in the ESPRESSIF ESP-IDF tools that can cause errors with long build path names. The following commands use a local path *C:\espbuild* as an example. ++To build the image: +1. In **ESP-IDF 5.0 CMD**, from the *iot-middleware-freertos-samples\demos\projects\ESPRESSIF\aziotkit* directory, run the following command to build the image. ++ ```shell + idf.py --no-ccache -B "C:\espbuild" build + ``` ++1. After the build completes, confirm that the binary image file was created in the build path that you specified previously. ++ *C:\espbuild\azure_iot_freertos_esp32.bin* ++To flash the image: +1. On the ESP32 DevKit, locate the Micro USB port, which is highlighted in the following image: ++ :::image type="content" source="media/tutorial-devkit-espressif-esp32-iot-hub/esp-azure-iot-kit.png" alt-text="Photo of the ESP32-Azure IoT Kit board."::: ++1. Connect the Micro USB cable to the Micro USB port on the ESP32 DevKit, and then connect it to your computer. +1. Open Windows **Device Manager**, and view **Ports** to find out which COM port the ESP32 DevKit is connected to. ++ :::image type="content" source="media/tutorial-devkit-espressif-esp32-iot-hub/esp-device-manager.png" alt-text="Screenshot of Windows Device Manager displaying COM port for a connected device."::: ++1. In **ESP-IDF 5.0 CMD**, run the following command, replacing the *\<Your-COM-port\>* placeholder and brackets with the correct COM port from the previous step. For example, replace the placeholder with `COM3`. ++ ```shell + idf.py --no-ccache -B "C:\espbuild" -p <Your-COM-port> flash + ``` ++1. Confirm that the output completes with the following text for a successful flash: ++ ```output + Hash of data verified + + Leaving... + Hard resetting via RTS pin... + Done + ``` ++To confirm that the device connects to Azure IoT Central: +1. In **ESP-IDF 5.0 CMD**, run the following command to start the monitoring tool. As you did in a previous command, replace the \<Your-COM-port\> placeholder, and brackets with the COM port that the device is connected to. ++ ```shell + idf.py -B "C:\espbuild" -p <Your-COM-port> monitor + ``` ++1. Check for repeating blocks of output similar to the following example. This output confirms that the device connects to Azure IoT and sends telemetry. ++ ```output + I (50807) AZ IOT: Successfully sent telemetry message + I (50807) AZ IOT: Attempt to receive publish message from IoT Hub. + + I (51057) MQTT: Packet received. ReceivedBytes=2. + I (51057) MQTT: Ack packet deserialized with result: MQTTSuccess. + I (51057) MQTT: State record updated. New state=MQTTPublishDone. + I (51067) AZ IOT: Puback received for packet id: 0x00000008 + I (53067) AZ IOT: Keeping Connection Idle... + ``` ++## View device properties ++You can use Azure IoT Explorer to view and manage the properties of your devices. In the following sections, you use the Plug and Play capabilities that are visible in IoT Explorer to manage and interact with the ESP32 DevKit. These capabilities rely on the device model published for the ESP32 DevKit in the public model repository. You configured IoT Explorer to search this repository for device models earlier in this quickstart. In many cases, you can perform the same action without using plug and play by selecting IoT Explorer menu options. However, using plug and play often provides an enhanced experience. IoT Explorer can read the device model specified by a plug and play device and present information specific to that device. ++To access IoT Plug and Play components for the device in IoT Explorer: ++1. From the home view in IoT Explorer, select **IoT hubs**, then select **View devices in this hub**. +1. Select your device. +1. Select **IoT Plug and Play components**. +1. Select **Default component**. IoT Explorer displays the IoT Plug and Play components that are implemented on your device. ++ :::image type="content" source="media/tutorial-devkit-espressif-esp32-iot-hub/iot-explorer-default-component-view.png" alt-text="Screenshot of the device's default component in IoT Explorer."::: ++1. On the **Interface** tab, view the JSON content in the device model **Description**. The JSON contains configuration details for each of the IoT Plug and Play components in the device model. ++ Each tab in IoT Explorer corresponds to one of the IoT Plug and Play components in the device model. ++ | Tab | Type | Name | Description | + ||||| + | **Interface** | Interface | `Espressif ESP32 Azure IoT Kit` | Example device model for the ESP32 DevKit | + | **Properties (writable)** | Property | `telemetryFrequencySecs` | The interval that the device sends telemetry | + | **Commands** | Command | `ToggleLed1` | Turn the LED on or off | + | **Commands** | Command | `ToggleLed2` | Turn the LED on or off | + | **Commands** | Command | `DisplayText` | Displays sent text on the device screen | ++To view and edit device properties using Azure IoT Explorer: ++1. Select the **Properties (writable)** tab. It displays the interval that telemetry is sent. +1. Change the `telemetryFrequencySecs` value to *5*, and then select **Update desired value**. Your device now uses this interval to send telemetry. ++ :::image type="content" source="media/tutorial-devkit-espressif-esp32-iot-hub/iot-explorer-set-telemetry-interval.png" alt-text="Screenshot of setting telemetry interval on the device in IoT Explorer."::: ++1. IoT Explorer responds with a notification. + +To use Azure CLI to view device properties: ++1. In your CLI console, run the [az iot hub device-twin show](/cli/azure/iot/hub/device-twin#az-iot-hub-device-twin-show) command. ++ ```azurecli + az iot hub device-twin show --device-id mydevice --hub-name {YourIoTHubName} + ``` ++1. Inspect the properties for your device in the console output. ++> [!TIP] +> You can also use Azure IoT Explorer to view device properties. In the left navigation select **Device twin**. ++## View telemetry ++With Azure IoT Explorer, you can view the flow of telemetry from your device to the cloud. Optionally, you can do the same task using Azure CLI. ++To view telemetry in Azure IoT Explorer: ++1. From the **IoT Plug and Play components** (Default Component) pane for your device in IoT Explorer, select the **Telemetry** tab. Confirm that **Use built-in event hub** is set to *Yes*. +1. Select **Start**. +1. View the telemetry as the device sends messages to the cloud. ++ :::image type="content" source="media/tutorial-devkit-espressif-esp32-iot-hub/iot-explorer-device-telemetry.png" alt-text="Screenshot of device telemetry in IoT Explorer."::: ++1. Select the **Show modeled events** checkbox to view the events in the data format specified by the device model. ++ :::image type="content" source="media/tutorial-devkit-espressif-esp32-iot-hub/iot-explorer-show-modeled-events.png" alt-text="Screenshot of modeled telemetry events in IoT Explorer."::: ++1. Select **Stop** to end receiving events. ++To use Azure CLI to view device telemetry: ++1. Run the [az iot hub monitor-events](/cli/azure/iot/hub#az-iot-hub-monitor-events) command. Use the names that you created previously in Azure IoT for your device and IoT hub. ++ ```azurecli + az iot hub monitor-events --device-id mydevice --hub-name {YourIoTHubName} + ``` ++1. View the JSON output in the console. ++ ```json + { + "event": { + "origin": "mydevice", + "module": "", + "interface": "dtmi:azureiot:devkit:freertos:Esp32AzureIotKit;1", + "component": "", + "payload": "{\"temperature\":28.6,\"humidity\":25.1,\"light\":116.66,\"pressure\":-33.69,\"altitude\":8764.9,\"magnetometerX\":1627,\"magnetometerY\":28373,\"magnetometerZ\":4232,\"pitch\":6,\"roll\":0,\"accelerometerX\":-1,\"accelerometerY\":0,\"accelerometerZ\":9}" + } + } + ``` ++1. Select CTRL+C to end monitoring. +++## Call a direct method on the device ++You can also use Azure IoT Explorer to call a direct method that you've implemented on your device. Direct methods have a name, and can optionally have a JSON payload, configurable connection, and method timeout. In this section, you call a method that turns an LED on or off. Optionally, you can do the same task using Azure CLI. ++To call a method in Azure IoT Explorer: ++1. From the **IoT Plug and Play components** (Default Component) pane for your device in IoT Explorer, select the **Commands** tab. +1. For the **ToggleLed1** command, select **Send command**. The LED on the ESP32 DevKit toggles on or off. You should also see a notification in IoT Explorer. ++ :::image type="content" source="media/tutorial-devkit-espressif-esp32-iot-hub/iot-explorer-invoke-method.png" alt-text="Screenshot of calling a method in IoT Explorer."::: ++1. For the **DisplayText** command, enter some text in the **content** field. +1. Select **Send command**. The text displays on the ESP32 DevKit screen. +++To use Azure CLI to call a method: ++1. Run the [az iot hub invoke-device-method](/cli/azure/iot/hub#az-iot-hub-invoke-device-method) command, and specify the method name and payload. For this method, setting `method-payload` to `true` means the LED toggles to the opposite of its current state. +++ ```azurecli + az iot hub invoke-device-method --device-id mydevice --method-name ToggleLed2 --method-payload true --hub-name {YourIoTHubName} + ``` ++ The CLI console shows the status of your method call on the device, where `200` indicates success. ++ ```json + { + "payload": {}, + "status": 200 + } + ``` ++1. Check your device to confirm the LED state. ++## Troubleshoot and debug ++If you experience issues building the device code, flashing the device, or connecting, see [Troubleshooting](../iot-develop/troubleshoot-embedded-device-quickstarts.md). ++For debugging the application, see [Debugging with Visual Studio Code](https://github.com/azure-rtos/getting-started/blob/master/docs/debugging.md). +++## Next steps ++In this quickstart, you built a custom image that contains the Azure IoT middleware for FreeRTOS sample code, and then you flashed the image to the ESP32 DevKit device. You connected the ESP32 DevKit to Azure IoT Hub, and carried out tasks such as viewing telemetry and calling methods on the device. ++As a next step, explore the following articles to learn more about using the IoT device SDKs to connect devices to Azure IoT. ++> [!div class="nextstepaction"] +> [Connect a simulated general device to IoT Hub](./tutorial-send-telemetry-iot-hub.md) +> [!div class="nextstepaction"] +> [Learn more about connecting embedded devices using C SDK and Embedded C SDK](./concepts-using-c-sdk-and-embedded-c-sdk.md) |
| iot | Tutorial Send Telemetry Iot Hub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/tutorial-send-telemetry-iot-hub.md | + + Title: Send device telemetry to Azure IoT Hub quickstart +description: "This quickstart shows device developers how to connect a device securely to Azure IoT Hub. You use an Azure IoT device SDK for C, C#, Python, Node.js, or Java, to build a device client for Windows, Linux, or Raspberry Pi (Raspbian). Then you connect and send telemetry." ++++ Last updated : 04/04/2024++zone_pivot_groups: iot-develop-set1 ++ms.devlang: azurecli +#Customer intent: As a device application developer, I want to learn the basic workflow of using an Azure IoT device SDK to build a client app on a device, connect the device securely to Azure IoT Hub, and send telemetry. +++# Quickstart: Send telemetry from an IoT Plug and Play device to Azure IoT Hub ++++++++++++++++ +## Clean up resources +If you no longer need the Azure resources created in this quickstart, you can use the Azure CLI to delete them. ++> [!IMPORTANT] +> Deleting a resource group is irreversible. The resource group and all the resources contained in it are permanently deleted. Make sure that you do not accidentally delete the wrong resource group or resources. ++To delete a resource group by name: +1. Run the [az group delete](/cli/azure/group#az-group-delete) command. This command removes the resource group, the IoT Hub, and the device registration you created. ++ ```azurecli-interactive + az group delete --name MyResourceGroup + ``` +1. Run the [az group list](/cli/azure/group#az-group-list) command to confirm the resource group is deleted. ++ ```azurecli-interactive + az group list + ``` ++## Next steps ++In this quickstart, you learned a basic Azure IoT application workflow for securely connecting a device to the cloud and sending device-to-cloud telemetry. You used Azure CLI to create an Azure IoT hub and a device instance. Then you used an Azure IoT device SDK to create a temperature controller, connect it to the hub, and send telemetry. You also used Azure CLI to monitor telemetry. ++As a next step, explore the following articles to learn more about building device solutions with Azure IoT. ++> [!div class="nextstepaction"] +> [Control a device connected to an IoT hub](../iot-hub/quickstart-control-device.md) +> [!div class="nextstepaction"] +> [Build a device solution with IoT Hub](set-up-environment.md) |
| iot | Tutorial Use Mqtt | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/iot/tutorial-use-mqtt.md | + + Title: "Tutorial: Use MQTT to create an IoT device client" +description: Tutorial - Use the MQTT protocol directly to create an IoT device client without using the Azure IoT Device SDKs +++ Last updated : 04/04/2024++++#Customer intent: As a device builder, I want to see how I can use the MQTT protocol to create an IoT device client without using the Azure IoT Device SDKs. +++# Tutorial - Use MQTT to develop an IoT device client without using a device SDK ++You should use one of the Azure IoT Device SDKs to build your IoT device clients if at all possible. However, in scenarios such as using a memory constrained device, you may need to use an MQTT library to communicate with your IoT hub. ++The samples in this tutorial use the [Eclipse Mosquitto](http://mosquitto.org/) MQTT library. ++In this tutorial, you learn how to: ++> [!div class="checklist"] +> * Build the C language device client sample applications. +> * Run a sample that uses the MQTT library to send telemetry. +> * Run a sample that uses the MQTT library to process a cloud-to-device message sent from your IoT hub. +> * Run a sample that uses the MQTT library to manage the device twin on the device. ++You can use either a Windows or Linux development machine to complete the steps in this tutorial. ++If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin. ++## Prerequisites +++### Development machine prerequisites ++If you're using Windows: ++1. Install [Visual Studio (Community, Professional, or Enterprise)](https://visualstudio.microsoft.com/downloads). Be sure to enable the **Desktop development with C++** workload. ++1. Install [CMake](https://cmake.org/download/). Enable the **Add CMake to the system PATH for all users** option. ++1. Install the **x64 version** of [Mosquitto](https://mosquitto.org/download/). ++If you're using Linux: ++1. Run the following command to install the build tools: ++ ```bash + sudo apt install cmake g++ + ``` ++1. Run the following command to install the Mosquitto client library: ++ ```bash + sudo apt install libmosquitto-dev + ``` ++## Set up your environment ++If you don't already have an IoT hub, run the following commands to create a free-tier IoT hub in a resource group called `mqtt-sample-rg`. The command uses the name `my-hub` as an example for the name of the IoT hub to create. Choose a unique name for your IoT hub to use in place of `my-hub`: ++```azurecli-interactive +az group create --name mqtt-sample-rg --location eastus +az iot hub create --name my-hub --resource-group mqtt-sample-rg --sku F1 +``` ++Make a note of the name of your IoT hub, you need it later. ++Register a device in your IoT hub. The following command registers a device called `mqtt-dev-01` in an IoT hub called `my-hub`. Be sure to use the name of your IoT hub: ++```azurecli-interactive +az iot hub device-identity create --hub-name my-hub --device-id mqtt-dev-01 +``` ++Use the following command to create a SAS token that grants the device access to your IoT hub. Be sure to use the name of your IoT hub: ++```dotnetcli +az iot hub generate-sas-token --device-id mqtt-dev-01 --hub-name my-hub --du 7200 +``` ++Make a note of the SAS token the command outputs as you need it later. The SAS token looks like `SharedAccessSignature sr=my-hub.azure-devices.net%2Fdevices%2Fmqtt-dev-01&sig=%2FnM...sNwtnnY%3D&se=1677855761` ++> [!TIP] +> By default, the SAS token is valid for 60 minutes. The `--du 7200` option in the previous command extends the token duration to two hours. If it expires before you're ready to use it, generate a new one. You can also create a token with a longer duration. To learn more, see [az iot hub generate-sas-token](/cli/azure/iot/hub#az-iot-hub-generate-sas-token). ++## Clone the sample repository ++Use the following command to clone the sample repository to a suitable location on your local machine: ++```cmd +git clone https://github.com/Azure-Samples/IoTMQTTSample.git +``` ++The repository also includes: ++* A Python sample that uses the `paho-mqtt` library. +* Instructions for using the `mosquitto_pub` CLI to interact with your IoT hub. ++## Build the C samples ++Before you build the sample, you need to add the IoT hub and device details. In the cloned IoTMQTTSample repository, open the _mosquitto/src/config.h_ file. Add your IoT hub name, device ID, and SAS token as follows. Be sure to use the name of your IoT hub: ++```c +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. ++#define IOTHUBNAME "my-hub" +#define DEVICEID "mqtt-dev-01" +#define SAS_TOKEN "SharedAccessSignature sr=my-hub.azure-devices.net%2Fdevices%2Fmqtt-dev-01&sig=%2FnM...sNwtnnY%3D&se=1677855761" ++#define CERTIFICATEFILE CERT_PATH "IoTHubRootCA.crt.pem" +``` ++> [!NOTE] +> The *IoTHubRootCA.crt.pem* file includes the CA root certificates for the TLS connection. ++Save the changes to the _mosquitto/src/config.h_ file. ++To build the samples, run the following commands in your shell: ++```bash +cd mosquitto +cmake -Bbuild +cmake --build build +``` ++In Linux, the binaries are in the _./build_ folder underneath the _mosquitto_ folder. ++In Windows, the binaries are in the _.\build\Debug_ folder underneath the _mosquitto_ folder. ++## Send telemetry ++The *mosquitto_telemetry* sample shows how to send a device-to-cloud telemetry message to your IoT hub by using the MQTT library. ++Before you run the sample application, run the following command to start the event monitor for your IoT hub. Be sure to use the name of your IoT hub: ++```azurecli-interactive +az iot hub monitor-events --hub-name my-hub +``` ++Run the _mosquitto_telemetry_ sample. For example, on Linux: ++```bash +./build/mosquitto_telemetry +``` ++The `az iot hub monitor-events` generates the following output that shows the payload sent by the device: ++```text +Starting event monitor, use ctrl-c to stop... +{ + "event": { + "origin": "mqtt-dev-01", + "module": "", + "interface": "", + "component": "", + "payload": "Bonjour MQTT from Mosquitto" + } +} +``` ++You can now stop the event monitor. ++### Review the code ++The following snippets are taken from the _mosquitto/src/mosquitto_telemetry.cpp_ file. ++The following statements define the connection information and the name of the MQTT topic you use to send the telemetry message: ++```c +#define HOST IOTHUBNAME ".azure-devices.net" +#define PORT 8883 +#define USERNAME HOST "/" DEVICEID "/?api-version=2020-09-30" ++#define TOPIC "devices/" DEVICEID "/messages/events/" +``` ++The `main` function sets the user name and password to authenticate with your IoT hub. The password is the SAS token you created for your device: ++```c +mosquitto_username_pw_set(mosq, USERNAME, SAS_TOKEN); +``` ++The sample uses the MQTT topic to send a telemetry message to your IoT hub: ++```c +int msgId = 42; +char msg[] = "Bonjour MQTT from Mosquitto"; ++// once connected, we can publish a Telemetry message +printf("Publishing....\r\n"); +rc = mosquitto_publish(mosq, &msgId, TOPIC, sizeof(msg) - 1, msg, 1, true); +if (rc != MOSQ_ERR_SUCCESS) +{ + return mosquitto_error(rc); +} +printf("Publish returned OK\r\n"); +``` ++To learn more, see [Sending device-to-cloud messages](./iot-mqtt-connect-to-iot-hub.md#sending-device-to-cloud-messages). ++## Receive a cloud-to-device message ++The *mosquitto_subscribe* sample shows how to subscribe to MQTT topics and receive a cloud-to-device message from your IoT hub by using the MQTT library. ++Run the _mosquitto_subscribe_ sample. For example, on Linux: ++```bash +./build/mosquitto_subscribe +``` ++Run the following command to send a cloud-to-device message from your IoT hub. Be sure to use the name of your IoT hub: ++```azurecli-interactive +az iot device c2d-message send --hub-name my-hub --device-id mqtt-dev-01 --data "hello world" +``` ++The output from _mosquitto_subscribe_ looks like the following example: ++```text +Waiting for C2D messages... +C2D message 'hello world' for topic 'devices/mqtt-dev-01/messages/devicebound/%24.mid=d411e727-...f98f&%24.to=%2Fdevices%2Fmqtt-dev-01%2Fmessages%2Fdevicebound&%24.ce=utf-8&iothub-ack=none' +Got message for devices/mqtt-dev-01/messages/# topic +``` ++### Review the code ++The following snippets are taken from the _mosquitto/src/mosquitto_subscribe.cpp_ file. ++The following statement defines the topic filter the device uses to receive cloud to device messages. The `#` is a multi-level wildcard: ++```c +#define DEVICEMESSAGE "devices/" DEVICEID "/messages/#" +``` ++The `main` function uses the `mosquitto_message_callback_set` function to set a callback to handle messages sent from your IoT hub and uses the `mosquitto_subscribe` function to subscribe to all messages. The following snippet shows the callback function: ++```c +void message_callback(struct mosquitto* mosq, void* obj, const struct mosquitto_message* message) +{ + printf("C2D message '%.*s' for topic '%s'\r\n", message->payloadlen, (char*)message->payload, message->topic); ++ bool match = 0; + mosquitto_topic_matches_sub(DEVICEMESSAGE, message->topic, &match); ++ if (match) + { + printf("Got message for " DEVICEMESSAGE " topic\r\n"); + } +} +``` ++To learn more, see [Use MQTT to receive cloud-to-device messages](./iot-mqtt-connect-to-iot-hub.md#receiving-cloud-to-device-messages). ++## Update a device twin ++The *mosquitto_device_twin* sample shows how to set a reported property in a device twin and then read the property back. ++Run the _mosquitto_device_twin_ sample. For example, on Linux: ++```bash +./build/mosquitto_device_twin +``` ++The output from _mosquitto_device_twin_ looks like the following example: ++```text +Setting device twin reported properties.... +Device twin message '' for topic '$iothub/twin/res/204/?$rid=0&$version=2' +Setting device twin properties SUCCEEDED. ++Getting device twin properties.... +Device twin message '{"desired":{"$version":1},"reported":{"temperature":32,"$version":2}}' for topic '$iothub/twin/res/200/?$rid=1' +Getting device twin properties SUCCEEDED. +``` ++### Review the code ++The following snippets are taken from the _mosquitto/src/mosquitto_device_twin.cpp_ file. ++The following statements define the topics the device uses to subscribe to device twin updates, read the device twin, and update the device twin: ++```c +#define DEVICETWIN_SUBSCRIPTION "$iothub/twin/res/#" +#define DEVICETWIN_MESSAGE_GET "$iothub/twin/GET/?$rid=%d" +#define DEVICETWIN_MESSAGE_PATCH "$iothub/twin/PATCH/properties/reported/?$rid=%d" +``` ++The `main` function uses the `mosquitto_connect_callback_set` function to set a callback to handle messages sent from your IoT hub and uses the `mosquitto_subscribe` function to subscribe to the `$iothub/twin/res/#` topic. ++The following snippet shows the `connect_callback` function that uses `mosquitto_publish` to set a reported property in the device twin. The device publishes the message to the `$iothub/twin/PATCH/properties/reported/?$rid=%d` topic. The `%d` value is incremented each time the device publishes a message to the topic: ++```c +void connect_callback(struct mosquitto* mosq, void* obj, int result) +{ + // ... other code ... ++ printf("\r\nSetting device twin reported properties....\r\n"); ++ char msg[] = "{\"temperature\": 32}"; + char mqtt_publish_topic[64]; + snprintf(mqtt_publish_topic, sizeof(mqtt_publish_topic), DEVICETWIN_MESSAGE_PATCH, device_twin_request_id++); ++ int rc = mosquitto_publish(mosq, NULL, mqtt_publish_topic, sizeof(msg) - 1, msg, 1, true); + if (rc != MOSQ_ERR_SUCCESS) ++ // ... other code ... +} +``` ++The device subscribes to the `$iothub/twin/res/#` topic and when it receives a message from your IoT hub, the `message_callback` function handles it. When you run the sample, the `message_callback` function gets called twice. The first time, the device receives a response from the IoT hub to the reported property update. The device then requests the device twin. The second time, the device receives the requested device twin. The following snippet shows the `message_callback` function: ++```c +void message_callback(struct mosquitto* mosq, void* obj, const struct mosquitto_message* message) +{ + printf("Device twin message '%.*s' for topic '%s'\r\n", message->payloadlen, (char*)message->payload, message->topic); ++ const char patchTwinTopic[] = "$iothub/twin/res/204/?$rid=0"; + const char getTwinTopic[] = "$iothub/twin/res/200/?$rid=1"; ++ if (strncmp(message->topic, patchTwinTopic, sizeof(patchTwinTopic) - 1) == 0) + { + // Process the reported property response and request the device twin + printf("Setting device twin properties SUCCEEDED.\r\n\r\n"); ++ printf("Getting device twin properties....\r\n"); ++ char msg[] = "{}"; + char mqtt_publish_topic[64]; + snprintf(mqtt_publish_topic, sizeof(mqtt_publish_topic), DEVICETWIN_MESSAGE_GET, device_twin_request_id++); ++ int rc = mosquitto_publish(mosq, NULL, mqtt_publish_topic, sizeof(msg) - 1, msg, 1, true); + if (rc != MOSQ_ERR_SUCCESS) + { + printf("Error: %s\r\n", mosquitto_strerror(rc)); + } + } + else if (strncmp(message->topic, getTwinTopic, sizeof(getTwinTopic) - 1) == 0) + { + // Process the device twin response and stop the client + printf("Getting device twin properties SUCCEEDED.\r\n\r\n"); ++ mosquitto_loop_stop(mosq, false); + mosquitto_disconnect(mosq); // finished, exit program + } +} +``` ++To learn more, see [Use MQTT to update a device twin reported property](./iot-mqtt-connect-to-iot-hub.md#update-device-twins-reported-properties) and [Use MQTT to retrieve a device twin property](./iot-mqtt-connect-to-iot-hub.md#retrieving-a-device-twins-properties). ++## Clean up resources +++## Next steps ++Now that you've learned how to use the Mosquitto MQTT library to communicate with IoT Hub, a suggested next step is to review: ++> [!div class="nextstepaction"] +> [Communicate with your IoT hub using the MQTT protocol](./iot-mqtt-connect-to-iot-hub.md) +> [!div class="nextstepaction"] +> [MQTT Application samples](https://github.com/Azure-Samples/MqttApplicationSamples) |
| key-vault | Quick Create Java | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/certificates/quick-create-java.md | Open the *pom.xml* file in your text editor. Add the following dependency elemen #### Grant access to your key vault -Create an access policy for your key vault that grants certificate permissions to your user account. --```azurecli -az keyvault set-policy --name <your-key-vault-name> --upn user@domain.com --certificate-permissions delete get list create purge -``` #### Set environment variables |
| key-vault | Quick Create Net | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/certificates/quick-create-net.md | This quickstart is using Azure Identity library with Azure CLI to authenticate u 2. Sign in with your account credentials in the browser. -#### Grant access to your key vault +### Grant access to your key vault -Create an access policy for your key vault that grants certificate permissions to your user account --```azurecli -az keyvault set-policy --name <your-key-vault-name> --upn user@domain.com --certificate-permissions delete get list create purge -``` ### Create new .NET console app |
| key-vault | Quick Create Node | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/certificates/quick-create-node.md | Create a Node.js application that uses your key vault. npm init -y ``` - ## Install Key Vault packages - 1. Using the terminal, install the Azure Key Vault secrets library, [@azure/keyvault-certificates](https://www.npmjs.com/package/@azure/keyvault-certificates) for Node.js. ```terminal Create a Node.js application that uses your key vault. ## Grant access to your key vault -Create a vault access policy for your key vault that grants key permissions to your user account. --```azurecli -az keyvault set-policy --name <YourKeyVaultName> --upn user@domain.com --certificate-permissions delete get list create purge update -``` ## Set environment variables |
| key-vault | Quick Create Powershell | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/certificates/quick-create-powershell.md | -In this quickstart, you create a key vault in Azure Key Vault with Azure PowerShell. Azure Key Vault is a cloud service that works as a secure secrets store. You can securely store keys, passwords, certificates, and other secrets. For more information on Key Vault you may review the [Overview](../general/overview.md). Azure PowerShell is used to create and manage Azure resources using commands or scripts. Once that you have completed that, you will store a certificate. +In this quickstart, you create a key vault in Azure Key Vault with Azure PowerShell. Azure Key Vault is a cloud service that works as a secure secrets store. You can securely store keys, passwords, certificates, and other secrets. For more information on Key Vault, review the [Overview](../general/overview.md). Azure PowerShell is used to create and manage Azure resources using commands or scripts. Afterwards, you store a certificate. If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin. Connect-AzAccount [!INCLUDE [Create a key vault](../../../includes/key-vault-powershell-kv-creation.md)] +### Grant access to your key vault ++ ## Add a certificate to Key Vault -To add a certificate to the vault, you just need to take a couple of additional steps. This certificate could be used by an application. +To can now add a certificate to the vault. This certificate could be used by an application. -Type the commands below to create a self-signed certificate with policy called **ExampleCertificate** : +Use these commands to create a self-signed certificate with policy called **ExampleCertificate** : ```azurepowershell-interactive $Policy = New-AzKeyVaultCertificatePolicy -SecretContentType "application/x-pkcs12" -SubjectName "CN=contoso.com" -IssuerName "Self" -ValidityInMonths 6 -ReuseKeyOnRenewal To view previously stored certificate: Get-AzKeyVaultCertificate -VaultName "<your-unique-keyvault-name>" -Name "ExampleCertificate" ``` -Now, you have created a Key Vault, stored a certificate, and retrieved it. - **Troubleshooting**: Operation returned an invalid status code 'Forbidden' Set-AzKeyVaultAccessPolicy -VaultName <KeyVaultName> -ObjectId <AzureObjectID> - ## Next steps -In this quickstart you created a Key Vault and stored a certificate in it. To learn more about Key Vault and how to integrate it with your applications, continue on to the articles below. +In this quickstart, you created a Key Vault and stored a certificate in it. To learn more about Key Vault and how to integrate it with your applications, continue on to the articles below. - Read an [Overview of Azure Key Vault](../general/overview.md) - See the reference for the [Azure PowerShell Key Vault cmdlets](/powershell/module/az.keyvault/) |
| key-vault | Quick Create Python | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/certificates/quick-create-python.md | This quickstart uses the Azure Identity library with Azure CLI or Azure PowerShe ### Grant access to your key vault -Create an access policy for your key vault that grants certificate permission to your user account --### [Azure CLI](#tab/azure-cli) --```azurecli -az keyvault set-policy --name <your-unique-keyvault-name> --upn user@domain.com --certificate-permissions delete get list create -``` --### [Azure PowerShell](#tab/azure-powershell) --```azurepowershell -Set-AzKeyVaultAccessPolicy -VaultName "<your-unique-keyvault-name>" -UserPrincipalName "user@domain.com" -PermissionsToCertificates delete,get,list,create -``` -- ## Create the sample code |
| key-vault | Rbac Guide | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/general/rbac-guide.md | To add role assignments, you must have `Microsoft.Authorization/roleAssignments/ > [!NOTE] > Changing permission model requires 'Microsoft.Authorization/roleAssignments/write' permission, which is part of [Owner](../../role-based-access-control/built-in-roles.md#owner) and [User Access Administrator](../../role-based-access-control/built-in-roles.md#user-access-administrator) roles. Classic subscription administrator roles like 'Service Administrator' and 'Co-Administrator' are not supported. -1. Enable Azure RBAC permissions on new key vault: +1. Enable Azure RBAC permissions on new key vault:  -2. Enable Azure RBAC permissions on existing key vault: +1. Enable Azure RBAC permissions on existing key vault:  To add role assignments, you must have `Microsoft.Authorization/roleAssignments/ > [!Note] > It's recommended to use the unique role ID instead of the role name in scripts. Therefore, if a role is renamed, your scripts would continue to work. In this document role name is used only for readability. -Run the following command to create a role assignment: - # [Azure CLI](#tab/azure-cli)++To create a role assignment using the Azure CLI, use the [az role assignment](/cli/azure/role/assignment) command: + ```azurecli az role assignment create --role <role_name_or_id> --assignee <assignee> --scope <scope> ``` For full details, see [Assign Azure roles using Azure CLI](../../role-based-acce # [Azure PowerShell](#tab/azurepowershell) +To create a role assignment using Azure PowerShell, use the [New-AzRoleAssignment](/powershell/module/az.resources/new-azroleassignment) cmdlet: + ```azurepowershell #Assign by User Principal Name New-AzRoleAssignment -RoleDefinitionName <role_name> -SignInName <assignee_upn> -Scope <scope> New-AzRoleAssignment -RoleDefinitionName Reader -ApplicationId <applicationId> - For full details, see [Assign Azure roles using Azure PowerShell](../../role-based-access-control/role-assignments-powershell.md). -+# [Azure portal](#tab/azure-portal) To assign roles using the Azure portal, see [Assign Azure roles using the Azure portal](../../role-based-access-control/role-assignments-portal.md). In the Azure portal, the Azure role assignments screen is available for all resources on the Access control (IAM) tab. ++ ### Resource group scope role assignment +# [Azure portal](#tab/azure-portal) + 1. Go to the Resource Group that contains your key vault.  To assign roles using the Azure portal, see [Assign Azure roles using the Azure  - # [Azure CLI](#tab/azure-cli) ```azurecli az role assignment create --role "Key Vault Reader" --assignee {i.e user@microsoft.com} --scope /subscriptions/{subscriptionid}/resourcegroups/{resource-group-name} Above role assignment provides ability to list key vault objects in key vault. ### Key Vault scope role assignment +# [Azure portal](#tab/azure-portal) + 1. Go to Key Vault \> Access control (IAM) tab 1. Select **Add** > **Add role assignment** to open the Add role assignment page. Above role assignment provides ability to list key vault objects in key vault.  - # [Azure CLI](#tab/azure-cli) ```azurecli az role assignment create --role "Key Vault Secrets Officer" --assignee {i.e jalichwa@microsoft.com} --scope /subscriptions/{subscriptionid}/resourcegroups/{resource-group-name}/providers/Microsoft.KeyVault/vaults/{key-vault-name} For full details, see [Assign Azure roles using Azure PowerShell](../../role-bas > [!NOTE] > Key vault secret, certificate, key scope role assignments should only be used for limited scenarios described [here](rbac-guide.md?i#best-practices-for-individual-keys-secrets-and-certificates-role-assignments) to comply with security best practices. +# [Azure portal](#tab/azure-portal) + 1. Open a previously created secret. 1. Click the Access control(IAM) tab For full details, see [Assign Azure roles using Azure PowerShell](../../role-bas  - # [Azure CLI](#tab/azure-cli)+ ```azurecli az role assignment create --role "Key Vault Secrets Officer" --assignee {i.e user@microsoft.com} --scope /subscriptions/{subscriptionid}/resourcegroups/{resource-group-name}/providers/Microsoft.KeyVault/vaults/{key-vault-name}/secrets/RBACSecret ``` For full details, see [Assign Azure roles using Azure PowerShell](../../role-bas  -### Creating custom roles +### Creating custom roles [az role definition create command](/cli/azure/role/definition#az-role-definition-create) # [Azure CLI](#tab/azure-cli)+ ```azurecli az role definition create --role-definition '{ \ "Name": "Backup Keys Operator", \ az role definition create --role-definition '{ \ "AssignableScopes": ["/subscriptions/{subscriptionId}"] \ }' ```+ # [Azure PowerShell](#tab/azurepowershell) ```azurepowershell $roleDefinition | Out-File role.json New-AzRoleDefinition -InputFile role.json ```++# [Azure portal](#tab/azure-portal) ++See [Create or update Azure custom roles using the Azure portal](../../role-based-access-control/custom-roles-portal.md). + For more Information about how to create custom roles, see: [Azure custom roles](../../role-based-access-control/custom-roles.md) -## Frequently Asked Questions: +## Frequently Asked Questions ### Can I use Key Vault role-based access control (RBAC) permission model object-scope assignments to provide isolation for application teams within Key Vault? No. RBAC permission model allows you to assign access to individual objects in Key Vault to user or application, but any administrative operations like network access control, monitoring, and objects management require vault level permissions, which will then expose secure information to operators across application teams. |
| key-vault | Quick Create Java | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/keys/quick-create-java.md | Open the *pom.xml* file in your text editor. Add the following dependency elemen #### Grant access to your key vault -Create an access policy for your key vault that grants key permissions to your user account. --```azurecli -az keyvault set-policy --name <your-key-vault-name> --upn user@domain.com --key-permissions delete get list create purge -``` #### Set environment variables |
| key-vault | Quick Create Net | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/keys/quick-create-net.md | This quickstart is using Azure Identity library with Azure CLI to authenticate u #### Grant access to your key vault -Create an access policy for your key vault that grants key permissions to your user account --```azurecli -az keyvault set-policy --name <your-key-vault-name> --upn user@domain.com --key-permissions delete get list create purge -``` ### Create new .NET console app |
| key-vault | Quick Create Node | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/keys/quick-create-node.md | Create a Node.js application that uses your key vault. ## Grant access to your key vault -Create an access policy for your key vault that grants key permissions to your user account --```azurecli -az keyvault set-policy --name <YourKeyVaultName> --upn user@domain.com --key-permissions delete get list create update purge -``` ## Set environment variables |
| key-vault | Quick Create Python | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/keys/quick-create-python.md | This quickstart is using the Azure Identity library with Azure CLI or Azure Powe ### Grant access to your key vault -Create an access policy for your key vault that grants key permission to your user account. --### [Azure CLI](#tab/azure-cli) --```azurecli -az keyvault set-policy --name <your-unique-keyvault-name> --upn user@domain.com --key-permissions get list create delete -``` --### [Azure PowerShell](#tab/azure-powershell) --```azurepowershell -Set-AzKeyVaultAccessPolicy -VaultName "<your-unique-keyvault-name>" -UserPrincipalName "user@domain.com" -PermissionsToKeys get,list,create,delete -``` -- ## Create the sample code Make sure the code in the previous section is in a file named *kv_keys.py*. Then python kv_keys.py ``` -- If you encounter permissions errors, make sure you ran the [`az keyvault set-policy` or `Set-AzKeyVaultAccessPolicy` command](#grant-access-to-your-key-vault).-- Rerunning the code with the same key name may produce the error, "(Conflict) Key \<name\> is currently in a deleted but recoverable state." Use a different key name.+Rerunning the code with the same key name may produce the error, "(Conflict) Key \<name\> is currently in a deleted but recoverable state." Use a different key name. ## Code details Remove-AzResourceGroup -Name myResourceGroup - [Overview of Azure Key Vault](../general/overview.md) - [Secure access to a key vault](../general/security-features.md)+- [RBAC Guide](../general/rbac-guide.md) - [Azure Key Vault developer's guide](../general/developers-guide.md)-- [Key Vault security overview](../general/security-features.md) - [Authenticate with Key Vault](../general/authentication.md) |
| key-vault | Quick Create Cli | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/secrets/quick-create-cli.md | This quickstart requires version 2.0.4 or later of the Azure CLI. If using Azure [!INCLUDE [Create a key vault](../../../includes/key-vault-cli-kv-creation.md)] +## Give your user account permissions to manage secrets in Key Vault ++ ## Add a secret to Key Vault To add a secret to the vault, you just need to take a couple of additional steps. This password could be used by an application. The password will be called **ExamplePassword** and will store the value of **hVFkk965BuUv** in it. |
| key-vault | Quick Create Java | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/secrets/quick-create-java.md | Open the *pom.xml* file in your text editor. Add the following dependency elemen #### Grant access to your key vault -Create an access policy for your key vault that grants secret permissions to your user account. --```azurecli -az keyvault set-policy --name <your-key-vault-name> --upn user@domain.com --secret-permissions delete get list set purge -``` #### Set environment variables |
| key-vault | Quick Create Net | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/secrets/quick-create-net.md | This quickstart is using Azure Identity library with Azure CLI to authenticate u ### Grant access to your key vault -Create an access policy for your key vault that grants secret permissions to your user account --```azurecli -az keyvault set-policy --name <YourKeyVaultName> --upn user@domain.com --secret-permissions delete get list set purge -``` ### [Azure PowerShell](#tab/azure-powershell) This quickstart is using Azure Identity library with Azure PowerShell to authent ### Grant access to your key vault -Create an access policy for your key vault that grants secret permissions to your user account --```azurepowershell -Set-AzKeyVaultAccessPolicy -VaultName "<YourKeyVaultName>" -UserPrincipalName "user@domain.com" -PermissionsToSecrets delete,get,list,set,purge -``` - ### Create new .NET console app |
| key-vault | Quick Create Node | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/secrets/quick-create-node.md | Create a Node.js application that uses your key vault. ## Grant access to your key vault -Create a vault access policy for your key vault that grants secret permissions to your user account with the [az keyvault set-policy](/cli/azure/keyvault#az-keyvault-set-policy) command. --```azurecli -az keyvault set-policy --name <your-key-vault-name> --upn user@domain.com --secret-permissions delete get list set purge update -``` ## Set environment variables |
| key-vault | Quick Create Portal | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/secrets/quick-create-portal.md | Sign in to the [Azure portal](https://portal.azure.com). To add a secret to the vault, follow the steps: -1. Navigate to your new key vault in the Azure portal -1. On the Key Vault settings pages, select **Secrets**. -1. Select on **Generate/Import**. +1. Navigate to your key vault in the Azure portal: +1. On the Key Vault left-hand sidebar, select **Objects** then select **Secrets**. +1. Select **+ Generate/Import**. 1. On the **Create a secret** screen choose the following values: - **Upload options**: Manual. - **Name**: Type a name for the secret. The secret name must be unique within a Key Vault. The name must be a 1-127 character string, starting with a letter and containing only 0-9, a-z, A-Z, and -. For more information on naming, see [Key Vault objects, identifiers, and versioning](../general/about-keys-secrets-certificates.md#objects-identifiers-and-versioning)- - **Value**: Type a value for the secret. Key Vault APIs accept and return secret values as strings. + - **Value**: Type a value for the secret. Key Vault APIs accept and return secret values as strings. - Leave the other values to their defaults. Select **Create**. -Once that you receive the message that the secret has been successfully created, you may select on it on the list. +Once you receive the message that the secret has been successfully created, you may select on it on the list. For more information on secrets attributes, see [About Azure Key Vault secrets](./about-secrets.md) If you select on the current version, you can see the value you specified in the :::image type="content" source="../media/quick-create-portal/current-version-hidden.png" alt-text="Secret properties"::: -By clicking "Show Secret Value" button in the right pane, you can see the hidden value. +By clicking "Show Secret Value" button in the right pane, you can see the hidden value. :::image type="content" source="../media/quick-create-portal/current-version-shown.png" alt-text="Secret value appeared"::: |
| key-vault | Quick Create Powershell | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/secrets/quick-create-powershell.md | Connect-AzAccount ## Give your user account permissions to manage secrets in Key Vault -Use the Azure PowerShell [Set-AzKeyVaultAccessPolicy](/powershell/module/az.keyvault/set-azkeyvaultaccesspolicy) cmdlet to update the Key Vault access policy and grant secret permissions to your user account. --```azurepowershell-interactive -Set-AzKeyVaultAccessPolicy -VaultName "<your-unique-keyvault-name>" -UserPrincipalName "user@domain.com" -PermissionsToSecrets get,set,delete -``` ## Adding a secret to Key Vault |
| key-vault | Quick Create Python | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/key-vault/secrets/quick-create-python.md | Get started with the Azure Key Vault secret client library for Python. Follow th This quickstart assumes you're running [Azure CLI](/cli/azure/install-azure-cli) or [Azure PowerShell](/powershell/azure/install-azure-powershell) in a Linux terminal window. - ## Set up your local environment This quickstart is using Azure Identity library with Azure CLI or Azure PowerShell to authenticate user to Azure Services. Developers can also use Visual Studio or Visual Studio Code to authenticate their calls, for more information, see [Authenticate the client with Azure Identity client library](/python/api/overview/azure/identity-readme). This quickstart is using Azure Identity library with Azure CLI or Azure PowerShe ### Grant access to your key vault -Create an access policy for your key vault that grants secret permission to your user account. --### [Azure CLI](#tab/azure-cli) --```azurecli -az keyvault set-policy --name <your-unique-keyvault-name> --upn user@domain.com --secret-permissions delete get list set -``` --### [Azure PowerShell](#tab/azure-powershell) --```azurepowershell -Set-AzKeyVaultAccessPolicy -VaultName "<your-unique-keyvault-name>" -UserPrincipalName "user@domain.com" -PermissionsToSecrets delete,get,list,set -``` -- ## Create the sample code |
| kubernetes-fleet | Concepts Resource Propagation | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/kubernetes-fleet/concepts-resource-propagation.md | Title: "Kubernetes resource propagation from hub cluster to member clusters (preview)" + Title: "Kubernetes resource propagation from hub cluster to member clusters (Preview)" description: This article describes the concept of Kubernetes resource propagation from hub cluster to member clusters. Last updated 03/04/2024 -# Kubernetes resource propagation from hub cluster to member clusters (preview) +# Kubernetes resource propagation from hub cluster to member clusters (Preview) +This article describes the concept of Kubernetes resource propagation from hub clusters to member clusters using Azure Kubernetes Fleet Manager (Fleet). ++Platform admins often need to deploy Kubernetes resources into multiple clusters for various reasons, for example: ++* Managing access control using roles and role bindings across multiple clusters. +* Running infrastructure applications, such as Prometheus or Flux, that need to be on all clusters. -Platform admins often need to deploy Kubernetes resources into multiple clusters, for example: -* Roles and role bindings to manage who can access what. -* An infrastructure application that needs to be on all clusters, for example, Prometheus, Flux. +Application developers often need to deploy Kubernetes resources into multiple clusters for various reasons, for example: -Application developers often need to deploy Kubernetes resources into multiple clusters, for example: -* Deploy a video serving application into multiple clusters, one per region, for low latency watching experience. -* Deploy a shopping cart application into two paired regions for customers to continue to shop during a single region outage. -* Deploy a batch compute application into clusters with inexpensive spot node pools available. +* Deploying a video serving application into multiple clusters in different regions for a low latency watching experience. +* Deploying a shopping cart application into two paired regions for customers to continue to shop during a single region outage. +* Deploying a batch compute application into clusters with inexpensive spot node pools available. ++It's tedious to create, update, and track these Kubernetes resources across multiple clusters manually. Fleet provides Kubernetes resource propagation to enable at-scale management of Kubernetes resources. With Fleet, you can create Kubernetes resources in the hub cluster and propagate them to selected member clusters via Kubernetes Custom Resources: `MemberCluster` and `ClusterResourcePlacement`. Fleet supports these custom resources based on an [open-source cloud-native multi-cluster solution][fleet-github]. For more information, see the [upstream Fleet documentation][fleet-github]. + -It's tedious to create and update these Kubernetes resources across tens or even hundreds of clusters, and track their current status in each cluster. -Azure Kubernetes Fleet Manager (Fleet) provides Kubernetes resource propagation to enable at-scale management of Kubernetes resources. +## Resource propagation workflow -You can create Kubernetes resources in the hub cluster and propagate them to selected member clusters via Kubernetes Customer Resources: `MemberCluster` and `ClusterResourcePlacement`. -Fleet supports these custom resources based on an [open-source cloud-native multi-cluster solution][fleet-github]. +[](./media/conceptual-resource-propagation.png#lightbox) -## What is `MemberCluster`? +## What is a `MemberCluster`? -Once a cluster joins a fleet, a corresponding `MemberCluster` custom resource is created on the hub cluster. -You can use it to select target clusters in resource propagation. +Once a cluster joins a fleet, a corresponding `MemberCluster` custom resource is created on the hub cluster. You can use this custom resource to select target clusters in resource propagation. -The following labels are added automatically to all member clusters, which can be used for target cluster selection in resource propagation. +The following labels can be used for target cluster selection in resource propagation and are automatically added to all member clusters: * `fleet.azure.com/location` * `fleet.azure.com/resource-group` * `fleet.azure.com/subscription-id` -You can find the API reference of `MemberCluster` [here][membercluster-api]. +For more information, see the [MemberCluster API reference][membercluster-api]. ++## What is a `ClusterResourcePlacement`? ++A `ClusterResourcePlacement` object is used to tell the Fleet scheduler how to place a given set of cluster-scoped objects from the hub cluster into member clusters. Namespace-scoped objects like Deployments, StatefulSets, DaemonSets, ConfigMaps, Secrets, and PersistentVolumeClaims are included when their containing namespace is selected. ++With `ClusterResourcePlacement`, you can: ++* Select which cluster-scoped Kubernetes resources to propagate to member clusters. +* Specify placement policies to manually or automatically select a subset or all of the member clusters as target clusters. +* Specify rollout strategies to safely roll out any updates of the selected Kubernetes resources to multiple target clusters. +* View the propagation progress towards each target cluster. ++The `ClusterResourcePlacement` object supports [using ConfigMap to envelope the object][envelope-object] to help propagate to member clusters without any unintended side effects. Selection methods include: ++* **Group, version, and kind**: Select and place all resources of the given type. +* **Group, version, kind, and name**: Select and place one particular resource of a given type. +* **Group, version, kind, and labels**: Select and place all resources of a given type that match the labels supplied. ++For more information, see the [`ClusterResourcePlacement` API reference][clusterresourceplacement-api]. ++Once you select the resources, multiple placement policies are available: ++* `PickAll` places the resources into all available member clusters. This policy is useful for placing infrastructure workloads, like cluster monitoring or reporting applications. +* `PickFixed` places the resources into a specific list of member clusters by name. +* `PickN` is the most flexible placement option and allows for selection of clusters based on affinity or topology spread constraints and is useful when spreading workloads across multiple appropriate clusters to ensure availability is desired. ++### `PickAll` placement policy ++You can use a `PickAll` placement policy to deploy a workload across all member clusters in the fleet (optionally matching a set of criteria). ++The following example shows how to deploy a `test-deployment` namespace and all of its objects across all clusters labeled with `environment: production`: ++```yaml +apiVersion: placement.kubernetes-fleet.io/v1beta1 +kind: ClusterResourcePlacement +metadata: + name: crp-1 +spec: + policy: + placementType: PickAll + affinity: + clusterAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + clusterSelectorTerms: + - labelSelector: + matchLabels: + environment: production + resourceSelectors: + - group: "" + kind: Namespace + name: prod-deployment + version: v1 +``` ++This simple policy takes the `test-deployment` namespace and all resources contained within it and deploys it to all member clusters in the fleet with the given `environment` label. If all clusters are desired, you can remove the `affinity` term entirely. ++### `PickFixed` placement policy ++If you want to deploy a workload into a known set of member clusters, you can use a `PickFixed` placement policy to select the clusters by name. ++The following example shows how to deploy the `test-deployment` namespace into member clusters `cluster1` and `cluster2`: ++```yaml +apiVersion: placement.kubernetes-fleet.io/v1beta1 +kind: ClusterResourcePlacement +metadata: + name: crp-2 +spec: + policy: + placementType: PickFixed + clusterNames: + - cluster1 + - cluster2 + resourceSelectors: + - group: "" + kind: Namespace + name: test-deployment + version: v1 +``` ++### `PickN` placement policy ++The `PickN` placement policy is the most flexible option and allows for placement of resources into a configurable number of clusters based on both affinities and topology spread constraints. ++#### `PickN` with affinities ++Using affinities with a `PickN` placement policy functions similarly to using affinities with pod scheduling. You can set both required and preferred affinities. Required affinities prevent placement to clusters that don't match them those specified affinities, and preferred affinities allow for ordering the set of valid clusters when a placement decision is being made. ++The following example shows how to deploy a workload into three clusters. Only clusters with the `critical-allowed: "true"` label are valid placement targets, and preference is given to clusters with the label `critical-level: 1`: ++```yaml +apiVersion: placement.kubernetes-fleet.io/v1beta1 +kind: ClusterResourcePlacement +metadata: + name: crp +spec: + resourceSelectors: + - ... + policy: + placementType: PickN + numberOfClusters: 3 + affinity: + clusterAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + weight: 20 + preference: + - labelSelector: + matchLabels: + critical-level: 1 + requiredDuringSchedulingIgnoredDuringExecution: + clusterSelectorTerms: + - labelSelector: + matchLabels: + critical-allowed: "true" +``` ++#### `PickN` with topology spread constraints ++You can use topology spread constraints to force the division of the cluster placements across topology boundaries to satisfy availability requirements, for example, splitting placements across regions or update rings. You can also configure topology spread constraints to prevent scheduling if the constraint can't be met (`whenUnsatisfiable: DoNotSchedule`) or schedule as best possible (`whenUnsatisfiable: ScheduleAnyway`). ++The following example shows how to spread a given set of resources out across multiple regions and attempts to schedule across member clusters with different update days: ++```yaml +apiVersion: placement.kubernetes-fleet.io/v1beta1 +kind: ClusterResourcePlacement +metadata: + name: crp +spec: + resourceSelectors: + - ... + policy: + placementType: PickN + topologySpreadConstraints: + - maxSkew: 2 + topologyKey: region + whenUnsatisfiable: DoNotSchedule + - maxSkew: 2 + topologyKey: updateDay + whenUnsatisfiable: ScheduleAnyway +``` ++For more information, see the [upstream topology spread constraints Fleet documentation][crp-topo]. ++## Update strategy ++Fleet uses a rolling update strategy to control how updates are rolled out across multiple cluster placements. ++The following example shows how to configure a rolling update strategy using the default settings: ++```yaml +apiVersion: placement.kubernetes-fleet.io/v1beta1 +kind: ClusterResourcePlacement +metadata: + name: crp +spec: + resourceSelectors: + - ... + policy: + ... + strategy: + type: RollingUpdate + rollingUpdate: + maxUnavailable: 25% + maxSurge: 25% + unavailablePeriodSeconds: 60 +``` ++The scheduler rolls out updates to each cluster sequentially, waiting at least `unavailablePeriodSeconds` between clusters. Rollout status is considered successful if all resources were correctly applied to the cluster. Rollout status checking doesn't cascade to child resources, for example, it doesn't confirm that pods created by a deployment become ready. ++For more information, see the [upstream rollout strategy Fleet documentation][fleet-rollout]. ++## Placement status ++The Fleet scheduler updates details and status on placement decisions onto the `ClusterResourcePlacement` object. You can view this information using the `kubectl describe crp <name>` command. The output includes the following information: ++* The conditions that currently apply to the placement, which include if the placement was successfully completed. +* A placement status section for each member cluster, which shows the status of deployment to that cluster. ++The following example shows a `ClusterResourcePlacement` that deployed the `test` namespace and the `test-1` ConfigMap into two member clusters using `PickN`. The placement was successfully completed and the resources were placed into the `aks-member-1` and `aks-member-2` clusters. ++``` +Name: crp-1 +Namespace: +Labels: <none> +Annotations: <none> +API Version: placement.kubernetes-fleet.io/v1beta1 +Kind: ClusterResourcePlacement +Metadata: + ... +Spec: + Policy: + Number Of Clusters: 2 + Placement Type: PickN + Resource Selectors: + Group: + Kind: Namespace + Name: test + Version: v1 + Revision History Limit: 10 +Status: + Conditions: + Last Transition Time: 2023-11-10T08:14:52Z + Message: found all the clusters needed as specified by the scheduling policy + Observed Generation: 5 + Reason: SchedulingPolicyFulfilled + Status: True + Type: ClusterResourcePlacementScheduled + Last Transition Time: 2023-11-10T08:23:43Z + Message: All 2 cluster(s) are synchronized to the latest resources on the hub cluster + Observed Generation: 5 + Reason: SynchronizeSucceeded + Status: True + Type: ClusterResourcePlacementSynchronized + Last Transition Time: 2023-11-10T08:23:43Z + Message: Successfully applied resources to 2 member clusters + Observed Generation: 5 + Reason: ApplySucceeded + Status: True + Type: ClusterResourcePlacementApplied + Placement Statuses: + Cluster Name: aks-member-1 + Conditions: + Last Transition Time: 2023-11-10T08:14:52Z + Message: Successfully scheduled resources for placement in aks-member-1 (affinity score: 0, topology spread score: 0): picked by scheduling policy + Observed Generation: 5 + Reason: ScheduleSucceeded + Status: True + Type: ResourceScheduled + Last Transition Time: 2023-11-10T08:23:43Z + Message: Successfully Synchronized work(s) for placement + Observed Generation: 5 + Reason: WorkSynchronizeSucceeded + Status: True + Type: WorkSynchronized + Last Transition Time: 2023-11-10T08:23:43Z + Message: Successfully applied resources + Observed Generation: 5 + Reason: ApplySucceeded + Status: True + Type: ResourceApplied + Cluster Name: aks-member-2 + Conditions: + Last Transition Time: 2023-11-10T08:14:52Z + Message: Successfully scheduled resources for placement in aks-member-2 (affinity score: 0, topology spread score: 0): picked by scheduling policy + Observed Generation: 5 + Reason: ScheduleSucceeded + Status: True + Type: ResourceScheduled + Last Transition Time: 2023-11-10T08:23:43Z + Message: Successfully Synchronized work(s) for placement + Observed Generation: 5 + Reason: WorkSynchronizeSucceeded + Status: True + Type: WorkSynchronized + Last Transition Time: 2023-11-10T08:23:43Z + Message: Successfully applied resources + Observed Generation: 5 + Reason: ApplySucceeded + Status: True + Type: ResourceApplied + Selected Resources: + Kind: Namespace + Name: test + Version: v1 + Kind: ConfigMap + Name: test-1 + Namespace: test + Version: v1 +Events: + Type Reason Age From Message + - - - - + Normal PlacementScheduleSuccess 12m (x5 over 3d22h) cluster-resource-placement-controller Successfully scheduled the placement + Normal PlacementSyncSuccess 3m28s (x7 over 3d22h) cluster-resource-placement-controller Successfully synchronized the placement + Normal PlacementRolloutCompleted 3m28s (x7 over 3d22h) cluster-resource-placement-controller Resources have been applied to the selected clusters +``` -## What is `ClusterResourcePlacement`? +## Placement changes -Fleet provides `ClusterResourcePlacement` as a mechanism to control how cluster-scoped Kubernetes resources are propagated to member clusters. +The Fleet scheduler prioritizes the stability of existing workload placements. This prioritization can limit the number of changes that cause a workload to be removed and rescheduled. The following scenarios can trigger placement changes: -Via `ClusterResourcePlacement`, you can: -- Select which cluster-scoped Kubernetes resources to propagate to member clusters-- Specify placement policies to manually or automatically select a subset or all of the member clusters as target clusters-- Specify rollout strategies to safely roll out any updates of the selected Kubernetes resources to multiple target clusters-- View the propagation progress towards each target cluster+* Placement policy changes in the `ClusterResourcePlacement` object can trigger removal and rescheduling of a workload. + * Scale out operations (increasing `numberOfClusters` with no other changes) place workloads only on new clusters and don't affect existing placements. +* Cluster changes, including: + * A new cluster becoming eligible might trigger placement if it meets the placement policy, for example, a `PickAll` policy. + * A cluster with a placement is removed from the fleet will attempt to replace all affected workloads without affecting their other placements. -In order to propagate namespace-scoped resources, you can select a namespace which by default selecting both the namespace and all the namespace-scoped resources under it. +Resource-only changes (updating the resources or updating the `ResourceSelector` in the `ClusterResourcePlacement` object) roll out gradually in existing placements but do **not** trigger rescheduling of the workload. -The following diagram shows a sample `ClusterResourcePlacement`. -[  ](./media/conceptual-resource-propagation.png#lightbox) +## Access the Kubernetes API of the Fleet resource cluster -You can find the API reference of `ClusterResourcePlacement` [here][clusterresourceplacement-api]. +If you created an Azure Kubernetes Fleet Manager resource with the hub cluster enabled, you can use it to centrally control scenarios like Kubernetes object propagation. To access the Kubernetes API of the Fleet resource cluster, follow the steps in [Access the Kubernetes API of the Fleet resource cluster with Azure Kubernetes Fleet Manager](./quickstart-access-fleet-kubernetes-api.md). -## Next Steps +## Next steps -* [Set up Kubernetes resource propagation from hub cluster to member clusters](./resource-propagation.md). +[Set up Kubernetes resource propagation from hub cluster to member clusters](./quickstart-resource-propagation.md). <!-- LINKS - external --> [fleet-github]: https://github.com/Azure/fleet [membercluster-api]: https://github.com/Azure/fleet/blob/main/docs/api-references.md#membercluster-[clusterresourceplacement-api]: https://github.com/Azure/fleet/blob/main/docs/api-references.md#clusterresourceplacement +[clusterresourceplacement-api]: https://github.com/Azure/fleet/blob/main/docs/api-references.md#clusterresourceplacement +[envelope-object]: https://github.com/Azure/fleet/blob/main/docs/concepts/ClusterResourcePlacement/README.md#envelope-object +[crp-topo]: https://github.com/Azure/fleet/blob/main/docs/howtos/topology-spread-constraints.md +[fleet-rollout]: https://github.com/Azure/fleet/blob/main/docs/howtos/crp.md#rollout-strategy |
| kubernetes-fleet | L4 Load Balancing | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/kubernetes-fleet/l4-load-balancing.md | You can follow this document to set up layer 4 load balancing for such multi-clu * These target clusters have to be [added as member clusters to the Fleet resource](./quickstart-create-fleet-and-members.md#join-member-clusters). * These target clusters should be using [Azure CNI (Container Networking Interface) networking](../aks/configure-azure-cni.md). -* You must gain access to the Kubernetes API of the hub cluster by following the steps in [Access the Kubernetes API of the Fleet resource](./access-fleet-kubernetes-api.md). +* You must gain access to the Kubernetes API of the hub cluster by following the steps in [Access the Kubernetes API of the Fleet resource](./quickstart-access-fleet-kubernetes-api.md). * Set the following environment variables and obtain the kubeconfigs for the fleet and all member clusters: |
| kubernetes-fleet | Quickstart Access Fleet Kubernetes Api | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/kubernetes-fleet/quickstart-access-fleet-kubernetes-api.md | + + Title: "Quickstart: Access the Kubernetes API of the Fleet resource" +description: Learn how to access the Kubernetes API of the Fleet resource with Azure Kubernetes Fleet Manager. + Last updated : 04/01/2024++++++# Quickstart: Access the Kubernetes API of the Fleet resource ++If your Azure Kubernetes Fleet Manager resource was created with the hub cluster enabled, then it can be used to centrally control scenarios like Kubernetes resource propagation. In this article, you learn how to access the Kubernetes API of the hub cluster managed by the Fleet resource. ++## Prerequisites +++* You need a Fleet resource with a hub cluster and member clusters. If you don't have one, see [Create an Azure Kubernetes Fleet Manager resource and join member clusters using Azure CLI](quickstart-create-fleet-and-members.md). +* The identity (user or service principal) you're using needs to have the Microsoft.ContainerService/fleets/listCredentials/action on the Fleet resource. ++## Access the Kubernetes API of the Fleet resource ++1. Set the following environment variables for your subscription ID, resource group, and Fleet resource: ++ ```azurecli-interactive + export SUBSCRIPTION_ID=<subscription-id> + export GROUP=<resource-group-name> + export FLEET=<fleet-name> + ``` ++2. Set the default Azure subscription to use using the [`az account set`][az-account-set] command. ++ ```azurecli-interactive + az account set --subscription ${SUBSCRIPTION_ID} + ``` ++3. Get the kubeconfig file of the hub cluster Fleet resource using the [`az fleet get-credentials`][az-fleet-get-credentials] command. ++ ```azurecli-interactive + az fleet get-credentials --resource-group ${GROUP} --name ${FLEET} + ``` ++ Your output should look similar to the following example output: ++ ```output + Merged "hub" as current context in /home/fleet/.kube/config + ``` ++4. Set the following environment variable for the `id` of the hub cluster Fleet resource: ++ ```azurecli-interactive + export FLEET_ID=/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${GROUP}/providers/Microsoft.ContainerService/fleets/${FLEET} + ``` ++5. Authorize your identity to the hub cluster Fleet resource's Kubernetes API server using the following commands: ++ For the `ROLE` environment variable, you can use one of the following four built-in role definitions as the value: ++ * Azure Kubernetes Fleet Manager RBAC Reader + * Azure Kubernetes Fleet Manager RBAC Writer + * Azure Kubernetes Fleet Manager RBAC Admin + * Azure Kubernetes Fleet Manager RBAC Cluster Admin ++ ```azurecli-interactive + export IDENTITY=$(az ad signed-in-user show --query "id" --output tsv) + export ROLE="Azure Kubernetes Fleet Manager RBAC Cluster Admin" + az role assignment create --role "${ROLE}" --assignee ${IDENTITY} --scope ${FLEET_ID} + ``` ++ Your output should look similar to the following example output: ++ ```output + { + "canDelegate": null, + "condition": null, + "conditionVersion": null, + "description": null, + "id": "/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<GROUP>/providers/Microsoft.ContainerService/fleets/<FLEET>/providers/Microsoft.Authorization/roleAssignments/<assignment>", + "name": "<name>", + "principalId": "<id>", + "principalType": "User", + "resourceGroup": "<GROUP>", + "roleDefinitionId": "/subscriptions/<SUBSCRIPTION_ID>/providers/Microsoft.Authorization/roleDefinitions/18ab4d3d-a1bf-4477-8ad9-8359bc988f69", + "scope": "/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<GROUP>/providers/Microsoft.ContainerService/fleets/<FLEET>", + "type": "Microsoft.Authorization/roleAssignments" + } + ``` ++6. Verify you can access the API server using the `kubectl get memberclusters` command. ++ ```bash + kubectl get memberclusters + ``` ++ If successful, your output should look similar to the following example output: ++ ```output + NAME JOINED AGE + aks-member-1 True 2m + aks-member-2 True 2m + aks-member-3 True 2m + ``` ++## Next steps ++* [Propagate resources from a Fleet hub cluster to member clusters](./quickstart-resource-propagation.md). ++<!-- LINKS > +[fleet-apispec]: https://github.com/Azure/fleet/blob/main/docs/api-references.md +[troubleshooting-guide]: https://github.com/Azure/fleet/blob/main/docs/troubleshooting/README.md +[az-fleet-get-credentials]: /cli/azure/fleet#az-fleet-get-credentials +[az-account-set]: /cli/azure/account#az-account-set |
| kubernetes-fleet | Quickstart Create Fleet And Members Portal | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/kubernetes-fleet/quickstart-create-fleet-and-members-portal.md | Get started with Azure Kubernetes Fleet Manager (Fleet) by using the Azure porta ## Prerequisites + * Read the [conceptual overview of this feature](./concepts-fleet.md), which provides an explanation of fleets and member clusters referenced in this document. * An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F). * An identity (user or service principal) with the following permissions on the Fleet and AKS resource types for completing the steps listed in this quickstart: Get started with Azure Kubernetes Fleet Manager (Fleet) by using the Azure porta ## Next steps -* [Orchestrate updates across multiple member clusters](./update-orchestration.md). -* [Set up Kubernetes resource propagation from hub cluster to member clusters](./resource-propagation.md). -* [Set up multi-cluster layer-4 load balancing](./l4-load-balancing.md). +* [Access the Kubernetes API of the Fleet resource](./quickstart-access-fleet-kubernetes-api.md). |
| kubernetes-fleet | Quickstart Create Fleet And Members | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/kubernetes-fleet/quickstart-create-fleet-and-members.md | Get started with Azure Kubernetes Fleet Manager (Fleet) by using the Azure CLI t ## Prerequisites + * Read the [conceptual overview of this feature](./concepts-fleet.md), which provides an explanation of fleets and member clusters referenced in this document. * An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F). * An identity (user or service principal) which can be used to [log in to Azure CLI](/cli/azure/authenticate-azure-cli). This identity needs to have the following permissions on the Fleet and AKS resource types for completing the steps listed in this quickstart: Fleet currently supports joining existing AKS clusters as member clusters. ```azurecli-interactive # Join the first member cluster- az fleet member create \ - --resource-group ${GROUP} \ - --fleet-name ${FLEET} \ - --name ${MEMBER_NAME_1} \ - --member-cluster-id ${MEMBER_CLUSTER_ID_1} + az fleet member create --resource-group ${GROUP} --fleet-name ${FLEET} --name ${MEMBER_NAME_1} --member-cluster-id ${MEMBER_CLUSTER_ID_1} ``` Your output should look similar to the following example output: Fleet currently supports joining existing AKS clusters as member clusters. ## Next steps -* [Orchestrate updates across multiple member clusters](./update-orchestration.md). -* [Set up Kubernetes resource propagation from hub cluster to member clusters](./resource-propagation.md). -* [Set up multi-cluster layer-4 load balancing](./l4-load-balancing.md). +* [Access the Kubernetes API of the Fleet resource](./quickstart-access-fleet-kubernetes-api.md). <!-- INTERNAL LINKS --> [az-extension-add]: /cli/azure/extension#az-extension-add |
| kubernetes-fleet | Quickstart Resource Propagation | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/kubernetes-fleet/quickstart-resource-propagation.md | + + Title: "Quickstart: Propagate resources from an Azure Kubernetes Fleet Manager (Fleet) hub cluster to member clusters (Preview)" +description: In this quickstart, you learn how to propagate resources from an Azure Kubernetes Fleet Manager (Fleet) hub cluster to member clusters. Last updated : 03/28/2024+++++++# Quickstart: Propagate resources from an Azure Kubernetes Fleet Manager (Fleet) hub cluster to member clusters ++In this quickstart, you learn how to propagate resources from an Azure Kubernetes Fleet Manager (Fleet) hub cluster to member clusters. ++## Prerequisites +++* Read the [resource propagation conceptual overview](./concepts-resource-propagation.md) to understand the concepts and terminology used in this quickstart. +* An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F). +* You need a Fleet resource with a hub cluster and member clusters. If you don't have one, see [Create an Azure Kubernetes Fleet Manager resource and join member clusters using Azure CLI](quickstart-create-fleet-and-members.md). +* Member clusters must be labeled appropriately in the hub cluster to match the desired selection criteria. Example labels include region, environment, team, availability zones, node availability, or anything else desired. +* You need access to the Kubernetes API of the hub cluster. If you don't have access, see [Access the Kubernetes API of the Fleet resource with Azure Kubernetes Fleet Manager](./quickstart-access-fleet-kubernetes-api.md). ++## Use the `ClusterResourcePlacement` API to propagate resources to member clusters ++The `ClusterResourcePlacement` API object is used to propagate resources from a hub cluster to member clusters. The `ClusterResourcePlacement` API object specifies the resources to propagate and the placement policy to use when selecting member clusters. The `ClusterResourcePlacement` API object is created in the hub cluster and is used to propagate resources to member clusters. This example demonstrates how to propagate a namespace to member clusters using the `ClusterResourcePlacement` API object with a `PickAll` placement policy. ++For more information, see [Kubernetes resource propagation from hub cluster to member clusters (Preview)](./concepts-resource-propagation.md) and the [upstream Fleet documentation](https://github.com/Azure/fleet/blob/main/docs/concepts/ClusterResourcePlacement/README.md). ++1. Create a namespace to place onto the member clusters using the `kubectl create namespace` command. The following example creates a namespace named `my-namespace`: ++ ```bash + kubectl create namespace my-namespace + ``` ++2. Create a `ClusterResourcePlacement` API object in the hub cluster to propagate the namespace to the member clusters and deploy it using the `kubectl apply -f` command. The following example `ClusterResourcePlacement` creates an object named `crp` and uses the `my-namespace` namespace with a `PickAll` placement policy to propagate the namespace to all member clusters: ++ ```bash + kubectl apply -f - <<EOF + apiVersion: placement.kubernetes-fleet.io/v1beta1 + kind: ClusterResourcePlacement + metadata: + name: crp + spec: + resourceSelectors: + - group: "" + kind: Namespace + version: v1 + name: my-namespace + policy: + placementType: PickAll + EOF + ``` ++3. Check the progress of the resource propagation using the `kubectl get clusterresourceplacement` command. The following example checks the status of the `ClusterResourcePlacement` object named `crp`: ++ ```bash + kubectl get clusterresourceplacement crp + ``` ++ Your output should look similar to the following example output: ++ ```output + NAME GEN SCHEDULED SCHEDULEDGEN APPLIED APPLIEDGEN AGE + crp 2 True 2 True 2 10s + ``` ++4. View the details of the `crp` object using the `kubectl describe crp` command. The following example describes the `ClusterResourcePlacement` object named `crp`: ++ ```bash + kubectl describe clusterresourceplacement crp + ``` ++ Your output should look similar to the following example output: ++ ```output + Name: crp + Namespace: + Labels: <none> + Annotations: <none> + API Version: placement.kubernetes-fleet.io/v1beta1 + Kind: ClusterResourcePlacement + Metadata: + Creation Timestamp: 2024-04-01T18:55:31Z + Finalizers: + kubernetes-fleet.io/crp-cleanup + kubernetes-fleet.io/scheduler-cleanup + Generation: 2 + Resource Version: 6949 + UID: 815b1d81-61ae-4fb1-a2b1-06794be3f986 + Spec: + Policy: + Placement Type: PickAll + Resource Selectors: + Group: + Kind: Namespace + Name: my-namespace + Version: v1 + Revision History Limit: 10 + Strategy: + Type: RollingUpdate + Status: + Conditions: + Last Transition Time: 2024-04-01T18:55:31Z + Message: found all the clusters needed as specified by the scheduling policy + Observed Generation: 2 + Reason: SchedulingPolicyFulfilled + Status: True + Type: ClusterResourcePlacementScheduled + Last Transition Time: 2024-04-01T18:55:36Z + Message: All 3 cluster(s) are synchronized to the latest resources on the hub cluster + Observed Generation: 2 + Reason: SynchronizeSucceeded + Status: True + Type: ClusterResourcePlacementSynchronized + Last Transition Time: 2024-04-01T18:55:36Z + Message: Successfully applied resources to 3 member clusters + Observed Generation: 2 + Reason: ApplySucceeded + Status: True + Type: ClusterResourcePlacementApplied + Observed Resource Index: 0 + Placement Statuses: + Cluster Name: membercluster1 + Conditions: + Last Transition Time: 2024-04-01T18:55:31Z + Message: Successfully scheduled resources for placement in membercluster1 (affinity score: 0, topology spread score: 0): picked by scheduling policy + Observed Generation: 2 + Reason: ScheduleSucceeded + Status: True + Type: ResourceScheduled + Last Transition Time: 2024-04-01T18:55:36Z + Message: Successfully Synchronized work(s) for placement + Observed Generation: 2 + Reason: WorkSynchronizeSucceeded + Status: True + Type: WorkSynchronized + Last Transition Time: 2024-04-01T18:55:36Z + Message: Successfully applied resources + Observed Generation: 2 + Reason: ApplySucceeded + Status: True + Type: ResourceApplied + Cluster Name: membercluster2 + Conditions: + Last Transition Time: 2024-04-01T18:55:31Z + Message: Successfully scheduled resources for placement in membercluster2 (affinity score: 0, topology spread score: 0): picked by scheduling policy + Observed Generation: 2 + Reason: ScheduleSucceeded + Status: True + Type: ResourceScheduled + Last Transition Time: 2024-04-01T18:55:36Z + Message: Successfully Synchronized work(s) for placement + Observed Generation: 2 + Reason: WorkSynchronizeSucceeded + Status: True + Type: WorkSynchronized + Last Transition Time: 2024-04-01T18:55:36Z + Message: Successfully applied resources + Observed Generation: 2 + Reason: ApplySucceeded + Status: True + Type: ResourceApplied + Cluster Name: membercluster3 + Conditions: + Last Transition Time: 2024-04-01T18:55:31Z + Message: Successfully scheduled resources for placement in membercluster3 (affinity score: 0, topology spread score: 0): picked by scheduling policy + Observed Generation: 2 + Reason: ScheduleSucceeded + Status: True + Type: ResourceScheduled + Last Transition Time: 2024-04-01T18:55:36Z + Message: Successfully Synchronized work(s) for placement + Observed Generation: 2 + Reason: WorkSynchronizeSucceeded + Status: True + Type: WorkSynchronized + Last Transition Time: 2024-04-01T18:55:36Z + Message: Successfully applied resources + Observed Generation: 2 + Reason: ApplySucceeded + Status: True + Type: ResourceApplied + Selected Resources: + Kind: Namespace + Name: my-namespace + Version: v1 + Events: + Type Reason Age From Message + - - - - + Normal PlacementScheduleSuccess 108s cluster-resource-placement-controller Successfully scheduled the placement + Normal PlacementSyncSuccess 103s cluster-resource-placement-controller Successfully synchronized the placement + Normal PlacementRolloutCompleted 103s cluster-resource-placement-controller Resources have been applied to the selected clusters + ```` ++## Clean up resources ++If you no longer wish to use the `ClusterResourcePlacement` object, you can delete it using the `kubectl delete` command. The following example deletes the `ClusterResourcePlacement` object named `crp`: ++```bash +kubectl delete clusterresourceplacement crp +``` ++## Next steps ++To learn more about resource propagation, see the following resources: ++* [Kubernetes resource propagation from hub cluster to member clusters (Preview)](./concepts-resource-propagation.md) +* [Upstream Fleet documentation](https://github.com/Azure/fleet/blob/main/docs/concepts/ClusterResourcePlacement/README.md) |
| kubernetes-fleet | Resource Propagation | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/kubernetes-fleet/resource-propagation.md | - Title: "Using cluster resource propagation (preview)" -description: Learn how to use Azure Kubernetes Fleet Manager to intelligently place workloads across multiple clusters. - Previously updated : 03/20/2024----- - ignite-2023 ---# Using cluster resource propagation (preview) --Azure Kubernetes Fleet Manager (Fleet) resource propagation, based on an [open-source cloud-native multi-cluster solution][fleet-github] allows for deployment of any Kubernetes objects to fleet member clusters according to specified criteria. Workload orchestration can handle many use cases where an application needs to be deployed across multiple clusters, including the following and more: --- An infrastructure application that needs to be on all clusters in the fleet-- A web application that should be deployed into multiple clusters in different regions for high availability, and should have updates rolled out in a nondisruptive manner-- A batch compute application that should be deployed into clusters with inexpensive spot node pools available--Fleet workload placement can deploy any Kubernetes objects to clusters In order to deploy resources to hub member clusters, the objects must be created in a Fleet hub cluster, and a `ClusterResourcePlacement` object must be created to indicate how the objects should be placed. --[  ](./media/conceptual-resource-propagation.png#lightbox) ---## Prerequisites --- Read the [conceptual overview of this feature](./concepts-resource-propagation.md), which provides an explanation of `MemberCluster` and `ClusterResourcePlacement` referenced in this document.-- You must have a Fleet resource with a hub cluster and member clusters. If you don't have this resource, follow [Quickstart: Create a Fleet resource and join member clusters](quickstart-create-fleet-and-members.md).-- Member clusters must be labeled appropriately in the hub cluster to match the desired selection criteria. Example labels could include region, environment, team, availability zones, node availability, or anything else desired.-- You must gain access to the Kubernetes API of the hub cluster by following the steps in [Access the Kubernetes API of the Fleet resource](./access-fleet-kubernetes-api.md).--## Resource placement with `ClusterResourcePlacement` resources --A `ClusterResourcePlacement` object is used to tell the Fleet scheduler how to place a given set of cluster-scoped objects from the hub cluster into member clusters. Namespace-scoped objects like Deployments, StatefulSets, DaemonSets, ConfigMaps, Secrets, and PersistentVolumeClaims are included when their containing namespace is selected. -(To propagate to the member clusters without any unintended side effects, the `ClusterResourcePlacement` object supports [using ConfigMap to envelope the object][envelope-object].) Multiple methods of selection can be used: --- Group, version, and kind - select and place all resources of the given type-- Group, version, kind, and name - select and place one particular resource of a given type-- Group, version, kind, and labels - select and place all resources of a given type that match the labels supplied--Once resources are selected, multiple types of placement are available: --- `PickAll` places the resources into all available member clusters. This policy is useful for placing infrastructure workloads, like cluster monitoring or reporting applications.-- `PickFixed` places the resources into a specific list of member clusters by name.-- `PickN` is the most flexible placement option and allows for selection of clusters based on affinity or topology spread constraints, and is useful when spreading workloads across multiple appropriate clusters to ensure availability is desired.--### Using a `PickAll` placement policy --To deploy a workload across all member clusters in the fleet (optionally matching a set of criteria), a `PickAll` placement policy can be used. To deploy the `test-deployment` Namespace and all of the objects in it across all of the clusters labeled with `environment: production`, create a `ClusterResourcePlacement` object as follows: --```yaml -apiVersion: placement.kubernetes-fleet.io/v1beta1 -kind: ClusterResourcePlacement -metadata: - name: crp-1 -spec: - policy: - placementType: PickAll - affinity: - clusterAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - clusterSelectorTerms: - - labelSelector: - matchLabels: - environment: production - resourceSelectors: - - group: "" - kind: Namespace - name: prod-deployment - version: v1 -``` --This simple policy takes the `test-deployment` namespace and all resources contained within it and deploys it to all member clusters in the fleet with the given `environment` label. If all clusters are desired, remove the `affinity` term entirely. --### Using a `PickFixed` placement policy --If a workload should be deployed into a known set of member clusters, a `PickFixed` policy can be used to select the clusters by name. This `ClusterResourcePlacement` deploys the `test-deployment` namespace into member clusters `cluster1` and `cluster2`: --```yaml -apiVersion: placement.kubernetes-fleet.io/v1beta1 -kind: ClusterResourcePlacement -metadata: - name: crp-2 -spec: - policy: - placementType: PickFixed - clusterNames: - - cluster1 - - cluster2 - resourceSelectors: - - group: "" - kind: Namespace - name: test-deployment - version: v1 -``` --### Using a `PickN` placement policy --The `PickN` placement policy is the most flexible option and allows for placement of resources into a configurable number of clusters based on both affinities and topology spread constraints. --#### `PickN` with affinities --Using affinities with `PickN` functions similarly to using affinities with pod scheduling. Both required and preferred affinities can be set. Required affinities prevent placement to clusters that don't match them; preferred affinities allow for ordering the set of valid clusters when a placement decision is being made. --As an example, the following `ClusterResourcePlacement` object places a workload into three clusters. Only clusters that have the label `critical-allowed: "true"` are valid placement targets, with preference given to clusters with the label `critical-level: 1`: --```yaml -apiVersion: placement.kubernetes-fleet.io/v1beta1 -kind: ClusterResourcePlacement -metadata: - name: crp -spec: - resourceSelectors: - - ... - policy: - placementType: PickN - numberOfClusters: 3 - affinity: - clusterAffinity: - preferredDuringSchedulingIgnoredDuringExecution: - weight: 20 - preference: - - labelSelector: - matchLabels: - critical-level: 1 - requiredDuringSchedulingIgnoredDuringExecution: - clusterSelectorTerms: - - labelSelector: - matchLabels: - critical-allowed: "true" -``` --#### `PickN` with topology spread constraints: --Topology spread constraints can be used to force the division of the cluster placements across topology boundaries to satisfy availability requirements (for example, splitting placements across regions or update rings). Topology spread constraints can also be configured to prevent scheduling if the constraint can't be met (`whenUnsatisfiable: DoNotSchedule`) or schedule as best possible (`whenUnsatisfiable: ScheduleAnyway`). --This `ClusterResourcePlacement` object spreads a given set of resources out across multiple regions and attempts to schedule across member clusters with different update days: --```yaml -apiVersion: placement.kubernetes-fleet.io/v1beta1 -kind: ClusterResourcePlacement -metadata: - name: crp -spec: - resourceSelectors: - - ... - policy: - placementType: PickN - topologySpreadConstraints: - - maxSkew: 2 - topologyKey: region - whenUnsatisfiable: DoNotSchedule - - maxSkew: 2 - topologyKey: updateDay - whenUnsatisfiable: ScheduleAnyway -``` --For more details on how placement works with topology spread constraints, review the documentation [in the open source fleet project on the topic.][crp-topo]. --## Update strategy --Azure Kubernetes Fleet uses a rolling update strategy to control how updates are rolled out across multiple cluster placements. The default settings are in this example: --```yaml -apiVersion: placement.kubernetes-fleet.io/v1beta1 -kind: ClusterResourcePlacement -metadata: - name: crp -spec: - resourceSelectors: - - ... - policy: - ... - strategy: - type: RollingUpdate - rollingUpdate: - maxUnavailable: 25% - maxSurge: 25% - unavailablePeriodSeconds: 60 -``` --The scheduler will roll updates to each cluster sequentially, waiting at least `unavailablePeriodSeconds` between clusters. Rollout status is considered successful if all resources were correctly applied to the cluster. Rollout status checking doesn't cascade to child resources - for example, it doesn't confirm that pods created by a deployment become ready. --For more details on cluster rollout strategy, see [the rollout strategy documentation in the open source project.][fleet-rollout] --## Placement status --The fleet scheduler updates details and status on placement decisions onto the `ClusterResourcePlacement` object. This information can be viewed via the `kubectl describe crp <name>` command. The output includes the following information: --- The conditions that currently apply to the placement, which include if the placement was successfully completed-- A placement status section for each member cluster, which shows the status of deployment to that cluster--This example shows a `ClusterResourcePlacement` that deployed the `test` namespace and the `test-1` ConfigMap it contained into two member clusters using `PickN`. The placement was successfully completed and the resources were placed into the `aks-member-1` and `aks-member-2` clusters. --``` -Name: crp-1 -Namespace: -Labels: <none> -Annotations: <none> -API Version: placement.kubernetes-fleet.io/v1beta1 -Kind: ClusterResourcePlacement -Metadata: - ... -Spec: - Policy: - Number Of Clusters: 2 - Placement Type: PickN - Resource Selectors: - Group: - Kind: Namespace - Name: test - Version: v1 - Revision History Limit: 10 -Status: - Conditions: - Last Transition Time: 2023-11-10T08:14:52Z - Message: found all the clusters needed as specified by the scheduling policy - Observed Generation: 5 - Reason: SchedulingPolicyFulfilled - Status: True - Type: ClusterResourcePlacementScheduled - Last Transition Time: 2023-11-10T08:23:43Z - Message: All 2 cluster(s) are synchronized to the latest resources on the hub cluster - Observed Generation: 5 - Reason: SynchronizeSucceeded - Status: True - Type: ClusterResourcePlacementSynchronized - Last Transition Time: 2023-11-10T08:23:43Z - Message: Successfully applied resources to 2 member clusters - Observed Generation: 5 - Reason: ApplySucceeded - Status: True - Type: ClusterResourcePlacementApplied - Placement Statuses: - Cluster Name: aks-member-1 - Conditions: - Last Transition Time: 2023-11-10T08:14:52Z - Message: Successfully scheduled resources for placement in aks-member-1 (affinity score: 0, topology spread score: 0): picked by scheduling policy - Observed Generation: 5 - Reason: ScheduleSucceeded - Status: True - Type: ResourceScheduled - Last Transition Time: 2023-11-10T08:23:43Z - Message: Successfully Synchronized work(s) for placement - Observed Generation: 5 - Reason: WorkSynchronizeSucceeded - Status: True - Type: WorkSynchronized - Last Transition Time: 2023-11-10T08:23:43Z - Message: Successfully applied resources - Observed Generation: 5 - Reason: ApplySucceeded - Status: True - Type: ResourceApplied - Cluster Name: aks-member-2 - Conditions: - Last Transition Time: 2023-11-10T08:14:52Z - Message: Successfully scheduled resources for placement in aks-member-2 (affinity score: 0, topology spread score: 0): picked by scheduling policy - Observed Generation: 5 - Reason: ScheduleSucceeded - Status: True - Type: ResourceScheduled - Last Transition Time: 2023-11-10T08:23:43Z - Message: Successfully Synchronized work(s) for placement - Observed Generation: 5 - Reason: WorkSynchronizeSucceeded - Status: True - Type: WorkSynchronized - Last Transition Time: 2023-11-10T08:23:43Z - Message: Successfully applied resources - Observed Generation: 5 - Reason: ApplySucceeded - Status: True - Type: ResourceApplied - Selected Resources: - Kind: Namespace - Name: test - Version: v1 - Kind: ConfigMap - Name: test-1 - Namespace: test - Version: v1 -Events: - Type Reason Age From Message - - - - - - Normal PlacementScheduleSuccess 12m (x5 over 3d22h) cluster-resource-placement-controller Successfully scheduled the placement - Normal PlacementSyncSuccess 3m28s (x7 over 3d22h) cluster-resource-placement-controller Successfully synchronized the placement - Normal PlacementRolloutCompleted 3m28s (x7 over 3d22h) cluster-resource-placement-controller Resources have been applied to the selected clusters -``` --## Placement changes --The Fleet scheduler prioritizes the stability of existing workload placements, and thus the number of changes that cause a workload to be removed and rescheduled is limited. --- Placement policy changes in the `ClusterResourcePlacement` object can trigger removal and rescheduling of a workload- - Scale out operations (increasing `numberOfClusters` with no other changes) will only place workloads on new clusters and won't affect existing placements. -- Cluster changes- - A new cluster becoming eligible may trigger placement if it meets the placement policy - for example, a `PickAll` policy. - - A cluster with a placement is removed from the fleet will attempt to re-place all affected workloads without affecting their other placements. --Resource-only changes (updating the resources or updating the `ResourceSelector` in the `ClusterResourcePlacement` object) will be rolled out gradually in existing placements but will **not** trigger rescheduling of the workload. --## Access the Kubernetes API of the Fleet resource cluster --If the Azure Kubernetes Fleet Manager resource was created with the hub cluster enabled, then it can be used to centrally control scenarios like Kubernetes object propagation. To access the Kubernetes API of the Fleet resource cluster, follow the steps in the [Access the Kubernetes API of the Fleet resource cluster with Azure Kubernetes Fleet Manager](access-fleet-kubernetes-api.md) article. --## Next steps --* Review the [`ClusterResourcePlacement` documentation and more in the open-source fleet repository][fleet-doc] for more examples -* Review the [API specifications][fleet-apispec] for all fleet custom resources. -* Review more information about [the fleet scheduler][fleet-scheduler] and how placement decisions are made. -* Review our [troubleshooting guide][troubleshooting-guide] to help resolve common issues related to the Fleet APIs. --<!-- LINKS - external --> -[fleet-github]: https://github.com/Azure/fleet -[fleet-doc]: https://github.com/Azure/fleet/blob/main/docs/README.md -[fleet-apispec]: https://github.com/Azure/fleet/blob/main/docs/api-references.md -[fleet-scheduler]: https://github.com/Azure/fleet/blob/main/docs/concepts/Scheduler/README.md -[fleet-rollout]: https://github.com/Azure/fleet/blob/main/docs/howtos/crp.md#rollout-strategy -[crp-topo]: https://github.com/Azure/fleet/blob/main/docs/howtos/topology-spread-constraints.md -[envelope-object]: https://github.com/Azure/fleet/blob/main/docs/concepts/ClusterResourcePlacement/README.md#envelope-object -[troubleshooting-guide]: https://github.com/Azure/fleet/blob/main/docs/troubleshooting/README.md |
| machine-learning | Apache Spark Azure Ml Concepts | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/machine-learning/apache-spark-azure-ml-concepts.md | To access data and other resources, a Spark job can use either a managed identit |Spark pool|Supported identities|Default identity| | - | -- | - |-|Serverless Spark compute|User identity and managed identity|User identity| -|Attached Synapse Spark pool|User identity and managed identity|Managed identity - compute identity of the attached Synapse Spark pool| +|Serverless Spark compute|User identity, user-assigned managed identity attached to the workspace|User identity| +|Attached Synapse Spark pool|User identity, user-assigned managed identity attached to the attached Synapse Spark pool, system-assigned managed identity of the attached Synapse Spark pool|System-assigned managed identity of the attached Synapse Spark pool| [This article](./apache-spark-environment-configuration.md#ensuring-resource-access-for-spark-jobs) describes resource access for Spark jobs. In a notebook session, both the serverless Spark compute and the attached Synapse Spark pool use user identity passthrough for data access during [interactive data wrangling](./interactive-data-wrangling-with-apache-spark-azure-ml.md). |
| machine-learning | Apache Spark Environment Configuration | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/machine-learning/apache-spark-environment-configuration.md | To access data and other resources, Spark jobs can use either a managed identity |Spark pool|Supported identities|Default identity| | - | -- | - |-|Serverless Spark compute|User identity and managed identity|User identity| -|Attached Synapse Spark pool|User identity and managed identity|Managed identity - compute identity of the attached Synapse Spark pool| +|Serverless Spark compute|User identity, user-assigned managed identity attached to the workspace|User identity| +|Attached Synapse Spark pool|User identity, user-assigned managed identity attached to the attached Synapse Spark pool, system-assigned managed identity of the attached Synapse Spark pool|System-assigned managed identity of the attached Synapse Spark pool| If the CLI or SDK code defines an option to use managed identity, Azure Machine Learning serverless Spark compute relies on a user-assigned managed identity attached to the workspace. You can attach a user-assigned managed identity to an existing Azure Machine Learning workspace using Azure Machine Learning CLI v2, or with `ARMClient`. If the CLI or SDK code defines an option to use managed identity, Azure Machine - [Interactive Data Wrangling with Apache Spark in Azure Machine Learning](./interactive-data-wrangling-with-apache-spark-azure-ml.md) - [Submit Spark jobs in Azure Machine Learning](./how-to-submit-spark-jobs.md) - [Code samples for Spark jobs using Azure Machine Learning CLI](https://github.com/Azure/azureml-examples/tree/main/cli/jobs/spark)-- [Code samples for Spark jobs using Azure Machine Learning Python SDK](https://github.com/Azure/azureml-examples/tree/main/sdk/python/jobs/spark)+- [Code samples for Spark jobs using Azure Machine Learning Python SDK](https://github.com/Azure/azureml-examples/tree/main/sdk/python/jobs/spark) |
| machine-learning | Concept Endpoints Batch | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/machine-learning/concept-endpoints-batch.md | description: Learn how Azure Machine Learning uses batch endpoints to simplify m -+ - devplatv2 - ignite-2023 Previously updated : 04/01/2023 Last updated : 04/04/2024 #Customer intent: As an MLOps administrator, I want to understand what a managed endpoint is and why I need it. # Batch endpoints -After you train a machine learning model, you need to deploy it so that others can consume its predictions. Such execution mode of a model is called *inference*. Azure Machine Learning uses the concept of [endpoints and deployments](concept-endpoints.md) for machine learning models inference. +Azure Machine Learning allows you to implement *batch endpoints and deployments* to perform long-running, asynchronous inferencing with machine learning models and pipelines. When you train a machine learning model or pipeline, you need to deploy it so that others can use it with new input data to generate predictions. This process of generating predictions with the model or pipeline is called _inferencing_. -**Batch endpoints** are endpoints that are used to do batch inferencing on large volumes of data over in asynchronous way. Batch endpoints receive pointers to data and run jobs asynchronously to process the data in parallel on compute clusters. Batch endpoints store outputs to a data store for further analysis. --We recommend using them when: +Batch endpoints receive pointers to data and run jobs asynchronously to process the data in parallel on compute clusters. Batch endpoints store outputs to a data store for further analysis. Use batch endpoints when: > [!div class="checklist"]-> * You have expensive models or pipelines that requires a longer time to run. +> * You have expensive models or pipelines that require a longer time to run. > * You want to operationalize machine learning pipelines and reuse components. > * You need to perform inference over large amounts of data, distributed in multiple files. > * You don't have low latency requirements. We recommend using them when: ## Batch deployments -A deployment is a set of resources and computes required to implement the functionality the endpoint provides. Each endpoint can host multiple deployments with different configurations, which helps *decouple the interface* indicated by the endpoint, from *the implementation details* indicated by the deployment. Batch endpoints automatically route the client to the default deployment which can be configured and changed at any time. +A deployment is a set of resources and computes required to implement the functionality that the endpoint provides. Each endpoint can host several deployments with different configurations, and this functionality helps to *decouple the endpoint's interface* from *the implementation details* that are defined by the deployment. When a batch endpoint is invoked, it automatically routes the client to its default deployment. This default deployment can be configured and changed at any time. -There are two types of deployments in batch endpoints: +Two types of deployments are possible in Azure Machine Learning batch endpoints: -* [Model deployments](#model-deployments) +* [Model deployment](#model-deployment) * [Pipeline component deployment](#pipeline-component-deployment) -### Model deployments +### Model deployment -Model deployment allows operationalizing model inference at scale, processing big amounts of data in a low latency and asynchronous way. Scalability is automatically instrumented by Azure Machine Learning by providing parallelization of the inferencing processes across multiple nodes in a compute cluster. +Model deployment enables the operationalization of model inferencing at scale, allowing you to process large amounts of data in a low latency and asynchronous way. Azure Machine Learning automatically instruments scalability by providing parallelization of the inferencing processes across multiple nodes in a compute cluster. -Use __Model deployments__ when: +Use __Model deployment__ when: > [!div class="checklist"]-> * You have expensive models that requires a longer time to run inference. +> * You have expensive models that require a longer time to run inference. > * You need to perform inference over large amounts of data, distributed in multiple files. > * You don't have low latency requirements. > * You can take advantage of parallelization. -The main benefit of this kind of deployments is that you can use the very same assets deployed in the online world (Online Endpoints) but now to run at scale in batch. If your model requires simple pre or pos processing, you can [author an scoring script](how-to-batch-scoring-script.md) that performs the data transformations required. +The main benefit of model deployments is that you can use the same assets that are deployed for real-time inferencing to online endpoints, but now, you get to run them at scale in batch. If your model requires simple preprocessing or post-processing, you can [author an scoring script](how-to-batch-scoring-script.md) that performs the data transformations required. To create a model deployment in a batch endpoint, you need to specify the following elements: To create a model deployment in a batch endpoint, you need to specify the follow ### Pipeline component deployment -Pipeline component deployment allows operationalizing entire processing graphs (pipelines) to perform batch inference in a low latency and asynchronous way. +Pipeline component deployment enables the operationalization of entire processing graphs (or pipelines) to perform batch inference in a low latency and asynchronous way. -Use __Pipeline component deployments__ when: +Use __Pipeline component deployment__ when: > [!div class="checklist"]-> * You need to operationalize complete compute graphs that can be decomposed in multiple steps. +> * You need to operationalize complete compute graphs that can be decomposed into multiple steps. > * You need to reuse components from training pipelines in your inference pipeline. > * You don't have low latency requirements. -The main benefit of this kind of deployments is reusability of components already existing in your platform and the capability to operationalize complex inference routines. +The main benefit of pipeline component deployments is the reusability of components that already exist in your platform and the capability to operationalize complex inference routines. To create a pipeline component deployment in a batch endpoint, you need to specify the following elements: To create a pipeline component deployment in a batch endpoint, you need to speci > [!div class="nextstepaction"] > [Create your first pipeline component deployment](how-to-use-batch-pipeline-deployments.md) -Batch endpoints also allow you to [create Pipeline component deployments from an existing pipeline job](how-to-use-batch-pipeline-from-job.md). When doing that, Azure Machine Learning automatically creates a Pipeline component out of the job. This simplifies the use of these kinds of deployments. However, it is a best practice to always [create pipeline components explicitly to streamline your MLOps practice](how-to-use-batch-pipeline-deployments.md). +Batch endpoints also allow you to [Create pipeline component deployments from an existing pipeline job](how-to-use-batch-pipeline-from-job.md). When doing that, Azure Machine Learning automatically creates a pipeline component out of the job. This simplifies the use of these kinds of deployments. However, it's a best practice to always [create pipeline components explicitly to streamline your MLOps practice](how-to-use-batch-pipeline-deployments.md). ## Cost management -Invoking a batch endpoint triggers an asynchronous batch inference job. Compute resources are automatically provisioned when the job starts, and automatically de-allocated as the job completes. So you only pay for compute when you use it. +Invoking a batch endpoint triggers an asynchronous batch inference job. Azure Machine Learning automatically provisions compute resources when the job starts, and automatically deallocates them as the job completes. This way, you only pay for compute when you use it. > [!TIP]-> When deploying models, you can [override compute resource settings](how-to-use-batch-endpoint.md#overwrite-deployment-configuration-per-each-job) (like instance count) and advanced settings (like mini batch size, error threshold, and so on) for each individual batch inference job to speed up execution and reduce cost if you know that you can take advantage of specific configurations. +> When deploying models, you can [override compute resource settings](how-to-use-batch-endpoint.md#overwrite-deployment-configuration-per-each-job) (like instance count) and advanced settings (like mini batch size, error threshold, and so on) for each individual batch inference job. By taking advantage of these specific configurations, you might be able to speed up execution and reduce cost. -Batch endpoints also can run on low-priority VMs. Batch endpoints can automatically recover from deallocated VMs and resume the work from where it was left when deploying models for inference. See [Use low-priority VMs in batch endpoints](how-to-use-low-priority-batch.md). +Batch endpoints can also run on low-priority VMs. Batch endpoints can automatically recover from deallocated VMs and resume the work from where it was left when deploying models for inference. For more information on how to use low priority VMs to reduce the cost of batch inference workloads, see [Use low-priority VMs in batch endpoints](how-to-use-low-priority-batch.md). -Finally, Azure Machine Learning doesn't charge for batch endpoints or batch deployments themselves, so you can organize your endpoints and deployments as best suits your scenario. Endpoints and deployment can use independent or shared clusters, so you can achieve fine grained control over which compute the produced jobs consume. Use __scale-to-zero__ in clusters to ensure no resources are consumed when they are idle. +Finally, Azure Machine Learning doesn't charge you for batch endpoints or batch deployments themselves, so you can organize your endpoints and deployments as best suits your scenario. Endpoints and deployments can use independent or shared clusters, so you can achieve fine-grained control over which compute the jobs consume. Use __scale-to-zero__ in clusters to ensure no resources are consumed when they're idle. ## Streamline the MLOps practice You can add, remove, and update deployments without affecting the endpoint itsel ## Flexible data sources and storage -Batch endpoints reads and write data directly from storage. You can indicate Azure Machine Learning datastores, Azure Machine Learning data asset, or Storage Accounts as inputs. For more information on supported input options and how to indicate them, see [Create jobs and input data to batch endpoints](how-to-access-data-batch-endpoints-jobs.md). +Batch endpoints read and write data directly from storage. You can specify Azure Machine Learning datastores, Azure Machine Learning data assets, or Storage Accounts as inputs. For more information on the supported input options and how to specify them, see [Create jobs and input data to batch endpoints](how-to-access-data-batch-endpoints-jobs.md). ## Security -Batch endpoints provide all the capabilities required to operate production level workloads in an enterprise setting. They support [private networking](how-to-secure-batch-endpoint.md) on secured workspaces and [Microsoft Entra authentication](how-to-authenticate-batch-endpoint.md), either using a user principal (like a user account) or a service principal (like a managed or unmanaged identity). Jobs generated by a batch endpoint run under the identity of the invoker which gives you flexibility to implement any scenario. See [How to authenticate to batch endpoints](how-to-authenticate-batch-endpoint.md) for details. +Batch endpoints provide all the capabilities required to operate production level workloads in an enterprise setting. They support [private networking](how-to-secure-batch-endpoint.md) on secured workspaces and [Microsoft Entra authentication](how-to-authenticate-batch-endpoint.md), either using a user principal (like a user account) or a service principal (like a managed or unmanaged identity). Jobs generated by a batch endpoint run under the identity of the invoker, which gives you the flexibility to implement any scenario. For more information on authorization while using batch endpoints, see [How to authenticate on batch endpoints](how-to-authenticate-batch-endpoint.md). > [!div class="nextstepaction"] > [Configure network isolation in Batch Endpoints](how-to-secure-batch-endpoint.md) -## Next steps +## Related content - [Deploy models with batch endpoints](how-to-use-batch-model-deployments.md) - [Deploy pipelines with batch endpoints](how-to-use-batch-pipeline-deployments.md) |
| machine-learning | How To Access Data Batch Endpoints Jobs | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/machine-learning/how-to-access-data-batch-endpoints-jobs.md | The following table summarizes the inputs and outputs for batch deployments: | Deployment type | Input's number | Supported input's types | Output's number | Supported output's types | |--|--|--|--|--|-| [Model deployment](concept-endpoints-batch.md#model-deployments) | 1 | [Data inputs](#data-inputs) | 1 | [Data outputs](#data-outputs) | +| [Model deployment](concept-endpoints-batch.md#model-deployment) | 1 | [Data inputs](#data-inputs) | 1 | [Data outputs](#data-outputs) | | [Pipeline component deployment](concept-endpoints-batch.md#pipeline-component-deployment) | [0..N] | [Data inputs](#data-inputs) and [literal inputs](#literal-inputs) | [0..N] | [Data outputs](#data-outputs) | > [!TIP] |
| machine-learning | How To Enable Studio Virtual Network | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/machine-learning/how-to-enable-studio-virtual-network.md | Use the following steps to enable access to data stored in Azure Blob and File s For more information, see the [Blob Data Reader](../role-based-access-control/built-in-roles.md#storage-blob-data-reader) built-in role. +1. Grant **your Azure user identity** the **Storage Blob Data reader** role for the Azure storage account. The studio uses your identity to access data to blob storage, even if the workspace managed identity has the Reader role. ++ For more information, see the [Blob Data Reader](../role-based-access-control/built-in-roles.md#storage-blob-data-reader) built-in role. + 1. **Grant the workspace managed identity the Reader role for storage private endpoints**. If your storage service uses a private endpoint, grant the workspace's managed identity *Reader* access to the private endpoint. The workspace's managed identity in Microsoft Entra ID has the same name as your Azure Machine Learning workspace. A private endpoint is necessary for both blob and file storage types. > [!TIP] |
| machine-learning | Reference Yaml Deployment Batch | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/machine-learning/reference-yaml-deployment-batch.md | The source JSON schema can be found at https://azuremlschemas.azureedge.net/late | `description` | string | Description of the deployment. | | | | `tags` | object | Dictionary of tags for the deployment. | | | | `endpoint_name` | string | **Required.** Name of the endpoint to create the deployment under. | | |-| `type` | string | **Required.** Type of the bath deployment. Use `model` for [model deployments](concept-endpoints-batch.md#model-deployments) and `pipeline` for [pipeline component deployments](concept-endpoints-batch.md#pipeline-component-deployment). <br><br>**New in version 1.7**. | `model`, `pipeline` | `model` | +| `type` | string | **Required.** Type of the bath deployment. Use `model` for [model deployments](concept-endpoints-batch.md#model-deployment) and `pipeline` for [pipeline component deployments](concept-endpoints-batch.md#pipeline-component-deployment). <br><br>**New in version 1.7**. | `model`, `pipeline` | `model` | | `settings` | object | Configuration of the deployment. See specific YAML reference for model and pipeline component for allowed values. <br><br>**New in version 1.7**. | | | > [!TIP] |
| machine-learning | Migrate Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/machine-learning/v1/migrate-overview.md | Title: Migrate to Azure Machine Learning from ML Studio (classic) -description: Learn how to migrate from ML Studio (classic) to Azure Machine Learning for a modernized data science platform. + Title: Migrate to Azure Machine Learning from Studio (classic) +description: Learn how to migrate from Machine Learning Studio (classic) to Azure Machine Learning for a modernized data science platform. -# Migrate to Azure Machine Learning from ML Studio (classic) +# Migrate to Azure Machine Learning from Studio (classic) > [!IMPORTANT]-> Support for Machine Learning Studio (classic) will end on 31 August 2024. We recommend that you transition to [Azure Machine Learning](../overview-what-is-azure-machine-learning.md) by that date. +> Support for Machine Learning Studio (classic) ends on 31 August 2024. We recommend that you transition to [Azure Machine Learning](../overview-what-is-azure-machine-learning.md) by that date. >-> After December 2021, you can no longer create new Machine Learning Studio (classic) resources. Through 31 August 2024, you can continue to use existing Machine Learning Studio (classic) resources. +> After December 2021, you can no longer create new Studio (classic) resources. Through 31 August 2024, you can continue to use existing Studio (classic) resources. >-> ML Studio (classic) documentation is being retired and might not be updated in the future. +> Studio (classic) documentation is being retired and might not be updated in the future. Learn how to migrate from Machine Learning Studio (classic) to Azure Machine Learning. Azure Machine Learning provides a modernized data science platform that combines no-code and code-first approaches. -This guide walks through a basic *lift and shift* migration. If you want to optimize an existing machine learning workflow, or modernize a machine learning platform, see the [Azure Machine Learning adoption framework](https://aka.ms/mlstudio-classic-migration-repo) for more resources, including digital survey tools, worksheets, and planning templates. +This guide walks through a basic *lift and shift* migration. If you want to optimize an existing machine learning workflow, or modernize a machine learning platform, see the [Azure Machine Learning Adoption Framework](https://aka.ms/mlstudio-classic-migration-repo) for more resources, including digital survey tools, worksheets, and planning templates. -Please work with your cloud solution architect on the migration. +Please work with your cloud solution architect on the migration. ## Recommended approach To migrate to Azure Machine Learning, we recommend the following approach: > * Step 5: Clean up Studio (classic) assets > * Step 6: Review and expand scenarios -## Step 1: Assess Azure Machine Learning +### Step 1: Assess Azure Machine Learning 1. Learn about [Azure Machine Learning](https://azure.microsoft.com/services/machine-learning/) and its benefits, costs, and architecture. -1. Compare the capabilities of Azure Machine Learning and ML Studio (classic). -- >[!NOTE] - > The **designer** feature in Azure Machine Learning provides a similar drag-and-drop experience to ML Studio (classic). However, Azure Machine Learning also provides robust [code-first workflows](../concept-model-management-and-deployment.md) as an alternative. This migration series focuses on the designer, since it's most similar to the Studio (classic) experience. +1. Compare the capabilities of Azure Machine Learning and Studio (classic). - The following table summarizes the key differences between ML Studio (classic) and Azure Machine Learning. + The following table summarizes the key differences. - | Feature | ML Studio (classic) | Azure Machine Learning | + | Feature | Studio (classic) | Azure Machine Learning | || | | | Drag-and-drop interface | Classic experience | Updated experience: [Azure Machine Learning designer](../concept-designer.md)|- | Code SDKs | Not supported | Fully integrated with [Azure Machine Learning Python](/python/api/overview/azure/ml/) and [R](https://github.com/Azure/azureml-sdk-for-r) SDKs | + | Code SDKs | Not supported | Fully integrated with Azure Machine Learning [Python](/python/api/overview/azure/ml/) and [R](https://github.com/Azure/azureml-sdk-for-r) SDKs | | Experiment | Scalable (10-GB training data limit) | Scale with compute target |- | Training compute targets | Proprietary compute target, CPU support only | Wide range of customizable [training compute targets](../concept-compute-target.md#training-compute-targets). Includes GPU and CPU support | - | Deployment compute targets | Proprietary web service format, not customizable | Wide range of customizable [deployment compute targets](../concept-compute-target.md#compute-targets-for-inference). Includes GPU and CPU support | - | ML pipeline | Not supported | Build flexible, modular [pipelines](../concept-ml-pipelines.md) to automate workflows | + | Training compute targets | Proprietary compute target, CPU support only | Wide range of customizable [training compute targets](../concept-compute-target.md#training-compute-targets); includes GPU and CPU support | + | Deployment compute targets | Proprietary web service format, not customizable | Wide range of customizable [deployment compute targets](../concept-compute-target.md#compute-targets-for-inference); includes GPU and CPU support | + | Machine learning pipeline | Not supported | Build flexible, modular [pipelines](../concept-ml-pipelines.md) to automate workflows | | MLOps | Basic model management and deployment; CPU-only deployments | Entity versioning (model, data, workflows), workflow automation, integration with CICD tooling, CPU and GPU deployments, [and more](../concept-model-management-and-deployment.md) | | Model format | Proprietary format, Studio (classic) only | Multiple supported formats depending on training job type |- | Automated model training and hyperparameter tuning | Not supported | [Supported](../concept-automated-ml.md). Code-first and no-code options. | + | Automated model training and hyperparameter tuning | Not supported | [Supported](../concept-automated-ml.md)<br><br> Code-first and no-code options | | Data drift detection | Not supported | [Supported](../v1/how-to-monitor-datasets.md) | | Data labeling projects | Not supported | [Supported](../how-to-create-image-labeling-projects.md) | | Role-based access control (RBAC) | Only contributor and owner role | [Flexible role definition and RBAC control](../how-to-assign-roles.md) |- | AI Gallery | [Supported](https://gallery.azure.ai) | Unsupported <br><br> Learn with [sample Python SDK notebooks](https://github.com/Azure/MachineLearningNotebooks) | + | AI Gallery | [Supported](https://gallery.azure.ai) | Not supported <br><br> Learn with [sample Python SDK notebooks](https://github.com/Azure/MachineLearningNotebooks) | ++ >[!NOTE] + > The **designer** feature in Azure Machine Learning provides a drag-and-drop experience that's similar to Studio (classic). However, Azure Machine Learning also provides robust [code-first workflows](../concept-model-management-and-deployment.md) as an alternative. This migration series focuses on the designer, since it's most similar to the Studio (classic) experience. -1. Verify that your critical Studio (classic) modules are supported in Azure Machine Learning designer. For more information, see the following [Studio (classic) and designer component-mapping](#studio-classic-and-designer-component-mapping) table. +1. Verify that your critical Studio (classic) modules are supported in Azure Machine Learning designer. For more information, see the [Studio (classic) and designer component-mapping](#studio-classic-and-designer-component-mapping) table. 1. Create an [Azure Machine Learning workspace](../quickstart-create-resources.md). -## Step 2: Define a strategy and plan +### Step 2: Define a strategy and plan 1. Define business justifications and expected outcomes. Please work with your cloud solution architect to define your strategy. For planning resources, including a planning doc template, see the [Azure Machine Learning Adoption Framework](https://aka.ms/mlstudio-classic-migration-repo). -## Step 3: Rebuild your first model +### Step 3: Rebuild your first model After you define a strategy, migrate your first model. -1. [Migrate datasets to Azure Machine Learning](migrate-register-dataset.md). +1. [Migrate a dataset to Azure Machine Learning](migrate-register-dataset.md). -1. Use the Azure Machine Learning designer to [rebuild experiments](migrate-rebuild-experiment.md). +1. Use the Azure Machine Learning designer to [rebuild an experiment](migrate-rebuild-experiment.md). -1. Use the Azure Machine Learning designer to [redeploy web services](migrate-rebuild-web-service.md). +1. Use the Azure Machine Learning designer to [redeploy a web service](migrate-rebuild-web-service.md). >[!NOTE]- > This guidance is built on top of Azure Machine Learning v1 concepts and features. Azure Machine Learning has CLI v2 and Python SDK v2. We suggest that you rebuild your ML Studio (classic) models using v2 instead of v1. Start with [Azure Machine Learning v2](../concept-v2.md). + > This guidance is built on top of Azure Machine Learning v1 concepts and features. Azure Machine Learning has CLI v2 and Python SDK v2. We suggest that you rebuild your Studio (classic) models using v2 instead of v1. Start with [Azure Machine Learning v2](../concept-v2.md). -## Step 4: Integrate client apps +### Step 4: Integrate client apps -Modify client applications that invoke ML Studio (classic) web services to use your new [Azure Machine Learning endpoints](migrate-rebuild-integrate-with-client-app.md). +Modify client applications that invoke Studio (classic) web services to use your new [Azure Machine Learning endpoints](migrate-rebuild-integrate-with-client-app.md). -## Step 5: Clean up Studio (classic) assets +### Step 5: Clean up Studio (classic) assets -To avoid extra charges, [clean up Studio (classic) assets](../classic/export-delete-personal-data-dsr.md). You might want to retain assets for fallback until you have validated Azure Machine Learning workloads. +To avoid extra charges, [clean up Studio (classic) assets](../classic/export-delete-personal-data-dsr.md). You might want to retain assets for fallback until you've validated Azure Machine Learning workloads. -## Step 6: Review and expand scenarios +### Step 6: Review and expand scenarios 1. Review the model migration for best practices and validate workloads. -1. Expand scenarios and migrate additional workloads to Azure Machine Learning. +1. Expand scenarios and migrate more workloads to Azure Machine Learning. ## Studio (classic) and designer component-mapping -Consult the following table to see which modules to use while rebuilding ML Studio (classic) experiments in the Azure Machine Learning designer. +Consult the following table to see which modules to use while rebuilding Studio (classic) experiments in the Azure Machine Learning designer. > [!IMPORTANT] > The designer implements modules through open-source Python packages rather than C# packages like Studio (classic). Because of this difference, the output of designer components might vary slightly from their Studio (classic) counterparts. Consult the following table to see which modules to use while rebuilding ML Stud |--|-|--| |Data input and output|- Enter data manually <br> - Export data <br> - Import data <br> - Load trained model <br> - Unpack zipped datasets|- Enter data manually <br> - Export data <br> - Import data| |Data format conversions|- Convert to CSV <br> - Convert to dataset <br> - Convert to ARFF <br> - Convert to SVMLight <br> - Convert to TSV|- Convert to CSV <br> - Convert to dataset|-|Data transformation - Manipulation|- Add columns<br> - Add rows <br> - Apply SQL transformation <br> - Clean missing data <br> - Convert to indicator values <br> - Edit metadata <br> - Join data <br> - Remove duplicate rows <br> - Select columns in dataset <br> - Select columns transform <br> - SMOTE <br> - Group categorical values|- Add columns<br> - Add rows <br> - Apply SQL transformation <br> - Clean missing data <br> - Convert to indicator values <br> - Edit metadata <br> - Join data <br> - Remove duplicate rows <br> - Select columns in dataset <br> - Select columns transform <br> - SMOTE| +|Data transformation ΓÇô Manipulation|- Add columns<br> - Add rows <br> - Apply SQL transformation <br> - Clean missing data <br> - Convert to indicator values <br> - Edit metadata <br> - Join data <br> - Remove duplicate rows <br> - Select columns in dataset <br> - Select columns transform <br> - SMOTE <br> - Group categorical values|- Add columns<br> - Add rows <br> - Apply SQL transformation <br> - Clean missing data <br> - Convert to indicator values <br> - Edit metadata <br> - Join data <br> - Remove duplicate rows <br> - Select columns in dataset <br> - Select columns transform <br> - SMOTE| |Data transformation ΓÇô Scale and reduce |- Clip values <br> - Group data into bins <br> - Normalize data <br>- Principal component analysis |- Clip values <br> - Group data into bins <br> - Normalize data| |Data transformation ΓÇô Sample and split|- Partition and sample <br> - Split data|- Partition and sample <br> - Split data| |Data transformation ΓÇô Filter |- Apply filter <br> - FIR filter <br> - IIR filter <br> - Median filter <br> - Moving average filter <br> - Threshold filter <br> - User-defined filter| | |Data transformation ΓÇô Learning with counts |- Build counting transform <br> - Export count table <br> - Import count table <br> - Merge count transform<br> - Modify count table parameters| | |Feature selection |- Filter-based feature selection <br> - Fisher linear discriminant analysis <br> - Permutation feature importance |- Filter-based feature selection <br> - Permutation feature importance|-| Model - Classification| - Multiclass decision forest <br> - Multiclass decision jungle <br> - Multiclass logistic regression <br>- Multiclass neural network <br>- One-vs-all multiclass <br>- Two-class averaged perceptron <br>- Two-class Bayes point machine <br>- Two-class boosted decision tree <br> - Two-class decision forest <br> - Two-class decision jungle <br> - Two-class locally-deep SVM <br> - Two-class logistic regression <br> - Two-class neural network <br> - Two-class support vector machine | - Multiclass decision forest <br> - Multiclass boost decision tree <br> - Multiclass logistic regression <br> - Multiclass neural network <br> - One-vs-all multiclass <br> - Two-class averaged perceptron <br> - Two-class boosted decision tree <br> - Two-class decision forest <br> - Two-class logistic regression <br> - Two-class neural network <br> - Two-class support vector machine | -| Model - Clustering| - K-means clustering| - K-means clustering| -| Model - Regression| - Bayesian linear regression <br> - Boosted decision tree regression <br> - Decision forest regression <br> - Fast forest quantile regression <br> - Linear regression <br> - Neural network regression <br> - Ordinal regression <br> - Poisson regression| - Boosted decision tree regression <br> - Decision forest regression <br> - Fast forest quantile regression <br> - Linear regression <br> - Neural network regression <br> - Poisson regression| +| Model ΓÇô Classification| - Multiclass decision forest <br> - Multiclass decision jungle <br> - Multiclass logistic regression <br>- Multiclass neural network <br>- One-vs-all multiclass <br>- Two-class averaged perceptron <br>- Two-class Bayes point machine <br>- Two-class boosted decision tree <br> - Two-class decision forest <br> - Two-class decision jungle <br> - Two-class locally deep SVM <br> - Two-class logistic regression <br> - Two-class neural network <br> - Two-class support vector machine | - Multiclass decision forest <br> - Multiclass boost decision tree <br> - Multiclass logistic regression <br> - Multiclass neural network <br> - One-vs-all multiclass <br> - Two-class averaged perceptron <br> - Two-class boosted decision tree <br> - Two-class decision forest <br> - Two-class logistic regression <br> - Two-class neural network <br> - Two-class support vector machine | +| Model ΓÇô Clustering| - K-means clustering| - K-means clustering| +| Model ΓÇô Regression| - Bayesian linear regression <br> - Boosted decision tree regression <br> - Decision forest regression <br> - Fast forest quantile regression <br> - Linear regression <br> - Neural network regression <br> - Ordinal regression <br> - Poisson regression| - Boosted decision tree regression <br> - Decision forest regression <br> - Fast forest quantile regression <br> - Linear regression <br> - Neural network regression <br> - Poisson regression| | Model ΓÇô Anomaly detection| - One-class SVM <br> - PCA-based anomaly detection | - PCA-based anomaly detection| | Machine Learning ΓÇô Evaluate | - Cross-validate model <br> - Evaluate model <br> - Evaluate recommender | - Cross-validate model <br> - Evaluate model <br> - Evaluate recommender| | Machine Learning ΓÇô Train| - Sweep clustering <br> - Train anomaly detection model <br> - Train clustering model <br> - Train matchbox recommender - <br> Train model <br> - Tune model hyperparameters| - Train anomaly detection model <br> - Train clustering model <br> - Train model <br> - Train PyTorch model <br> - Train SVD recommender <br> - Train wide and deep recommender <br> - Tune model hyperparameters| Consult the following table to see which modules to use while rebuilding ML Stud | Web service | - Input <br> - Output | - Input <br> - Output| | Computer vision| | - Apply image transformation <br> - Convert to image directory <br> - Init image transformation <br> - Split image directory <br> - DenseNet image classification <br> - ResNet image classification | -For more information on how to use individual designer components, see the [designer component reference](../component-reference/component-reference.md). +For more information on how to use individual designer components, see the [Algorithm & component reference](../component-reference/component-reference.md). ### What if a designer component is missing? If your migration is blocked due to missing modules in the designer, contact us ## Example migration -The following experiment migration highlights some of the differences between ML Studio (classic) and Azure Machine Learning. +The following migration example highlights some of the differences between Studio (classic) and Azure Machine Learning. ### Datasets -In ML Studio (classic), *datasets* were saved in your workspace and could only be used by Studio (classic). +In Studio (classic), *datasets* were saved in your workspace and could only be used by Studio (classic). -In Azure Machine Learning, *datasets* are registered to the workspace and can be used across all of Azure Machine Learning. For more information on the benefits of Azure Machine Learning datasets, see [Secure data access](concept-data.md). +In Azure Machine Learning, *datasets* are registered to the workspace and can be used across all of Azure Machine Learning. For more information on the benefits of Azure Machine Learning datasets, see [Data in Azure Machine Learning](concept-data.md). ### Pipeline -In ML Studio (classic), *experiments* contained the processing logic for your work. You created experiments with drag-and-drop modules. +In Studio (classic), *experiments* contained the processing logic for your work. You created experiments with drag-and-drop modules. In Azure Machine Learning, *pipelines* contain the processing logic for your work. You can create pipelines with either drag-and-drop modules or by writing code. ### Web service endpoints Studio (classic) used *REQUEST/RESPOND API* for real-time prediction and *BATCH EXECUTION API* for batch prediction or retraining. Azure Machine Learning uses *real-time endpoints* (managed endpoints) for real-time prediction and *pipeline endpoints* for batch prediction or retraining. ## Related content -In this article, you learned the high-level requirements for migrating to Azure Machine Learning. For detailed steps, see the other articles in the ML Studio (classic) migration series: +In this article, you learned the high-level requirements for migrating to Azure Machine Learning. For detailed steps, see the other articles in the Machine Learning Studio (classic) migration series: -- [Migrate dataset](migrate-register-dataset.md)-- [Rebuild a Studio (classic) training pipeline](migrate-rebuild-experiment.md)+- [Migrate a Studio (classic) dataset](migrate-register-dataset.md) +- [Rebuild a Studio (classic) experiment](migrate-rebuild-experiment.md) - [Rebuild a Studio (classic) web service](migrate-rebuild-web-service.md)-- [Integrate an Azure Machine Learning web service with client apps](migrate-rebuild-integrate-with-client-app.md).+- [Consume pipeline endpoints from client applications](migrate-rebuild-integrate-with-client-app.md). - [Migrate Execute R Script modules](migrate-execute-r-script.md) For more migration resources, see the [Azure Machine Learning Adoption Framework](https://aka.ms/mlstudio-classic-migration-repo). |
| migrate | Common Questions Appliance | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/migrate/common-questions-appliance.md | ms. Previously updated : 03/13/2024 Last updated : 04/04/2024 # Azure Migrate appliance: Common questions By default, the appliance and its installed agents are updated automatically. Th Only the appliance and the appliance agents are updated by these automatic updates. The operating system is not updated by Azure Migrate automatic updates. Use Windows Updates to keep the operating system up to date. +## How to troubleshoot Auto-update failures for Azure Migrate appliance? ++A modification was made recently to the MSI validation process, which could potentially impact the Migrate appliance auto-update process. The auto-update process might fail with the following error message: +++To fix this issue, follow these steps to ensure that your appliance can validate the digital signatures of the MSIs: ++1. Ensure that the MicrosoftΓÇÖs root certificate authority certificate is present in your applianceΓÇÖs certificate stores. + 1. Go to **Settings** and search for ΓÇÿcertificatesΓÇÖ. + 1. Select **Manage Computer Certificates**. + + :::image type="content" source="./media/common-questions-appliance/settings-inline.png" alt-text="Screenshot of Windows settings." lightbox="./media/common-questions-appliance/settings-expanded.png"::: ++ 1. In the certificate manager, you must see the entry for **Microsoft Root Certificate Authority 2011** and **Microsoft Code Signing PCA 2011** as shown in the following screenshots: + + :::image type="content" source="./media/common-questions-appliance/certificate-1-inline.png" alt-text="Screenshot of certificate 1." lightbox="./media/common-questions-appliance/certificate-1-expanded.png"::: ++ :::image type="content" source="./media/common-questions-appliance/certificate-2-inline.png" alt-text="Screenshot of certificate 2." lightbox="./media/common-questions-appliance/certificate-2-expanded.png"::: ++ 1. If these two certificates are not present, proceed to download them from the following sources: + - https://download.microsoft.com/download/2/4/8/248D8A62-FCCD-475C-85E7-6ED59520FC0F/MicrosoftRootCertificateAuthority2011.cer + - https://www.microsoft.com/pkiops/certs/MicCodSigPCA2011_2011-07-08.crt + 1. install these certificates on the appliance machine. +1. Check if there are any group policies on your machine that could be interfering with certificate validation: + 1. Go to Windows Start Menu > Run > gpedit.msc. <br>The **Local Group Policy Editor** window. Make sure that the **Network Retrieval** policies are defined as shown in the following screenshot: + + :::image type="content" source="./media/common-questions-appliance/local-group-policy-editor-inline.png" alt-text="Screenshot of local group policy editor." lightbox="./media/common-questions-appliance/local-group-policy-editor-expanded.png"::: ++1. Ensure that there are no internet access issues or firewall settings interfering with the certificate validation. ++**Verify Azure Migrate MSI Validation Readiness** ++1. To ensure that your appliance is ready to validate Azure Migrate MSIs, follow these steps: + 1. Download a sample MSI from [Microsoft Download Center](https://download.microsoft.com/download/9/b/8/9b8abdb7-a784-4a25-9da7-31ce4d80a0c5/MicrosoftAzureAutoUpdate.msi) on the appliance. + 1. Right-click on it and go to Digital Signatures tab. + + :::image type="content" source="./media/common-questions-appliance/digital-sign-inline.png" alt-text="Screenshot of digital signature tab." lightbox="./media/common-questions-appliance/digital-sign-expanded.png"::: ++ 1. Select Details and check that the Digital Signature Information for the certificate is OK as highlighted in the following screenshot: + + :::image type="content" source="./media/common-questions-appliance/digital-sign-inline.png" alt-text="Screenshot of digital signature tab." lightbox="./media/common-questions-appliance/digital-sign-expanded.png"::: + ## Can I check agent health? Yes. In the portal, go the **Agent health** page of the Azure Migrate: Discovery and assessment tool or the Migration and modernization tool. There, you can check the connection status between Azure and the discovery and assessment agents on the appliance. |
| nat-gateway | Nat Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/nat-gateway/nat-overview.md | A NAT gateway doesn't affect the network bandwidth of your compute resources. Le ### Traffic routes -* NAT gateway replaces a subnetΓÇÖs [system default route](/azure/virtual-network/virtual-networks-udr-overview#default) to the internet when configured. When NAT gateway is attached to the subnet, all traffic within the 0.0.0.0/0 prefix routes to NAT gateway before connecting outbound to the internet. +* The subnet has a [system default route](/azure/virtual-network/virtual-networks-udr-overview#default) that routes traffic with destination 0.0.0.0/0 to the internet automatically. Once NAT gateway is configured to the subnet, communication from the virtual machines existing in the subnet to the internet will prioritize using the public IP of the NAT gateway. * You can override NAT gateway as a subnetΓÇÖs system default route to the internet with the creation of a custom user-defined route (UDR) for 0.0.0.0/0 traffic. A NAT gateway doesn't affect the network bandwidth of your compute resources. Le * Outbound connectivity follows this order of precedence among different routing and outbound connectivity methods: - * Virtual appliance UDR / VPN Gateway / ExpressRoute >> NAT gateway >> Instance-level public IP address on a virtual machine >> Load balancer outbound rules >> default system route to the internet. + * UDR with Virtual appliance / VPN Gateway / ExpressRoute >> NAT gateway >> Instance-level public IP address on a virtual machine >> Load balancer outbound rules >> default system route to the internet. ### NAT gateway configurations |
| network-watcher | Vnet Flow Logs Cli | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/network-watcher/vnet-flow-logs-cli.md | -In this article, you learn how to create, change, enable, disable, or delete a VNet flow log using the Azure CLI. You can learn how to manage a VNet flow log using [PowerShell](vnet-flow-logs-powershell.md). +In this article, you learn how to create, change, enable, disable, or delete a VNet flow log using the Azure CLI. You can learn how to manage a VNet flow log using the [Azure portal](vnet-flow-logs-portal.md) or [PowerShell](vnet-flow-logs-powershell.md). > [!IMPORTANT] > The VNet flow logs feature is currently in preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). |
| network-watcher | Vnet Flow Logs Portal | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/network-watcher/vnet-flow-logs-portal.md | Last updated 04/03/2024 Virtual network flow logging is a feature of Azure Network Watcher that allows you to log information about IP traffic flowing through an Azure virtual network. For more information about virtual network flow logging, see [VNet flow logs overview](vnet-flow-logs-overview.md). -In this article, you learn how to create, change, enable, disable, or delete a VNet flow log using the Azure portal. You can also learn how to manage a VNet flow log using [Azure PowerShell](vnet-flow-logs-powershell.md) or [Azure CLI](vnet-flow-logs-cli.md). +In this article, you learn how to create, change, enable, disable, or delete a VNet flow log using the Azure portal. You can also learn how to manage a VNet flow log using [PowerShell](vnet-flow-logs-powershell.md) or [Azure CLI](vnet-flow-logs-cli.md). > [!IMPORTANT] > The VNet flow logs feature is currently in preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). |
| network-watcher | Vnet Flow Logs Powershell | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/network-watcher/vnet-flow-logs-powershell.md | -In this article, you learn how to create, change, enable, disable, or delete a VNet flow log using Azure PowerShell. You can learn how to manage a VNet flow log using the [Azure CLI](vnet-flow-logs-cli.md). +In this article, you learn how to create, change, enable, disable, or delete a VNet flow log using Azure PowerShell. You can learn how to manage a VNet flow log using the [Azure portal](vnet-flow-logs-portal.md) or [Azure CLI](vnet-flow-logs-cli.md). > [!IMPORTANT] > The VNet flow logs feature is currently in preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). |
| operator-nexus | Concepts Network Fabric Resource Update Commit | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/operator-nexus/concepts-network-fabric-resource-update-commit.md | + + Title: Update and commit Network Fabric resources +description: Learn how Nexus Network Fabric's resource update flow allows you to batch and update a set of Network Fabric resources. ++++ Last updated : 04/03/2024++#CustomerIntent: As a <type of user>, I want <what?> so that <why?>. +++# Update and commit Network Fabric resources ++Currently, Nexus Network Fabric resources require that you disable a parent resource (such as an L3Isolation domain) and reput the parent or child resource with updated values and execute the administrative post action to enable and configure the devices. Network Fabric's new resource update flow allows you to batch and update a set of Network Fabric resources via a `commitConfiguration` POST action when resources are enabled. There's no change if you choose the current workflow of disabling L3 Isolation domain, making changes and the enabling L3 Isolation domain. ++## Network Fabric resource update overview ++Any Create, Update, Delete (CUD) operation on a child resource linked to an existing enabled parent resource or an update to an enabled parent resource property is considered an **Update** operation. A few examples would be a new Internal network, or a new subnet needs to be added to an existing enabled Layer 3 Isolation domain (Internal network is a child resource of Layer 3 isolation domain). A new route policy needs to be attached to existing internal network; both these scenarios qualify for an **Update** operation. ++Any update operation carried out on supported Network Fabric resources shown in the following table puts the fabric into a pending commit state (currently **Accepted** in Configuration state) where you must initiate a fabric commit-configuration action to apply the desired changes. All updates to Network Fabric resources (including child resources) in fabric follow the same workflow. ++Commit action/updates to resources shall only be valid and applicable when the fabric is in provisioned state and Network Fabric resources are in an **enabled administrative state. Updates to parent and child resources can be batched (across various Network Fabric resources) and a `commitConfiguration` action can be performed to execute all changes in a single POST action. ++Creation of parent resources and enablement via administrative action is independent of Update/Commit Action workflow. Additionally, all administrative actions to enable / disable are independent and shall not require commitConfiguration action trigger for execution. CommitConfiguration action is only applicable to a scenario when operator wants to update any existing Azure Resource Manager resources and fabric, parent resource is in enabled state. Any automation scripts or bicep templates that were used by the operators to create Network Fabric resource and enable require no changes. ++## User workflow ++To successfully execute update resources, fabric must be in provisioned state. The following steps are involved in updating Network Fabric resources. ++1. Operator updates the required Network Fabric resources (multiple resources updates can be batched) which were already enabled (config applied to devices) using update call on Network Fabric resources via AzCli, Azure Resource Manager, Portal. (Refer to the supported scenarios, resources, and parameters' details in the following table). ++ In the following example, a new `internalnetwork` is added to an existing L3Isolation **l3domain101523-sm**. ++ ```azurecli + az networkfabric internalnetwork create --subscription 5ffad143-8f31-4e1e-b171-fa1738b14748 --resource-group "Fab3Lab-4-1-PROD" --l3-isolation-domain-name "l3domain101523-sm" --resource-name "internalnetwork101523" --vlan-id 789 --mtu 1432 --connected-ipv4-subnets "[{prefix:'10.252.11.0/24'},{prefix:'10.252.12.0/24'}] + ``` ++1. Once the Azure Resource Manager update call succeeds, the specific resource's `ConfigurationState` is set to **Accepted** and when it fails, it's set to **Rejected**. Fabric `ConfigurationState` is set to **Accepted** regardless of PATCH call success/failure. ++ If any Azure Resource Manager resource on the fabric (such as Internal Network or `RoutePolicy`) is in **Rejected** state, the Operator has to correct the configuration and ensure the specific resource's ConfigurationState is set to Accepted before proceeding further. ++2. Operator executes the commitConfiguration POST action on Fabric resource. ++ ```azurecli + az networkfabric fabric commit-configuration --subscription 5ffad143-8f31-4e1e-b171-fa1738b14748 --resource-group "FabLAB-4-1-PROD" --resource-name "nffab3-4-1-prod" + ``` ++3. Service validates if all the resource updates succeeded and validates inputs. It also validates connected logical resources to ensure consistent behavior and configuration. Once all validations succeed, the new configuration is generated and pushed to the devices. ++1. Specific resource `configurationState` is reset to **Succeeded** and Fabric `configurationState` is set to **Provisioned**. +1. If the `commitConfiguration` action fails, the service displays the appropriate error message and notifies the operator of the potential Network Fabric resource update failure. +++|State |Definition |Before Azure Resource Manager Resource Update |Before CommitConfiguration & Post Azure Resource Manager update |Post CommitConfiguration | +|||||--| +|**Administrative State** | State to represent administrative action performed on the resource | Enabled (only enabled is supported) | Enabled (only enabled is supported) |Enabled (user can disable) | +|**Configuration State** | State to represent operator actions/service driven configurations |**Resource State** - Succeeded, <br> **Fabric State** Provisioned | **Resource State** <br>- Accepted (Success)<br>- Rejected (Failure) <br>**Fabric State** <br>- Accepted | **Resource State** <br> - Accepted (Failure), <br>- Succeeded (Success)<br> **Fabric State**<br> - Provisioned | +|Provisioning State | State to represent Azure Resource Manager provisioning state of resources |Provisioned | Provisioned | Provisioned | +++## Supported Network Fabric resources and scenarios ++ Network Fabric Update Support Network Fabric resources (Network Fabric 4.1, Nexus 2310.1) ++| Network Fabric Resource | Type | Scenarios Supported | Scenarios Not Supported |Notes | +| -- | -- | | -- | -- | +| **Layer 2 Isolation Domain** | Parent | - Update to properties – MTU <br> - Addition/update tags | *Re-PUT* of resource | | +| **Layer 3 Isolation Domain** | Parent | Update to properties <br> - redistribute connected. <br>- redistribute static routes. <br>- Aggregate route configuration <br>- connected subnet route policy. <br>Addition/update tags | *Re-PUT* of resource | | +| **Internal Network** | Child (of L3 ISD) | Adding a new Internal network <br> Update to properties  <br>- MTU <br>- Addition/Update of connected IPv4/IPv6 subnets <br>- Addition/Update of IPv4/IPv6 RoutePolicy <br>- Addition/Update of Egress/Ingress ACL <br>- Update `isMonitoringEnabled` flag <br>- Addition/Update to Static routes <br>- BGP Config <br> Addition/update tags | - *Re-PUT* of resource. <br>- Deleting an Internal network when parent Layer 3 Isolation domain is enabled. | To delete the resource, the parent resource must be disabled | +| **External Network** | Child (of L3 ISD) | Update to properties  <br>- Addition/Update of IPv4/IPv6 RoutePolicy <br>- Option A properties MTU, Addition/Update of Ingress and Egress ACLs, <br>- Option A properties – BFD Configuration <br>- Option B properties – Route Targets <br> Addition/Update of tags | - *Re-PUT* of resource. <br>- Creating a new external network <br>- Deleting an External network when parent Layer 3 Isolation domain is enabled. | To delete the resource, the parent resource must be disabled.<br><br> NOTE: Only one external network is supported per ISD. | +| **Route Policy** | Parent | - Update entire statement including seq number, condition, action. <br>- Addition/update tags | - *Re-PUT* of resource. <br>- Update to Route Policy linked to a Network-to-Network Interconnect resource. | To delete the resource, the `connectedResource` (`IsolationDomain` or N-to-N Interconnect) shouldn't hold any reference. | +| **IPCommunity** | Parent | Update entire ipCommunity rule including seq number, action, community members, well known communities. | *Re-PUT* of resource | To delete the resource, the connected `RoutePolicy` Resource shouldn't hold any reference. | +| **IPPrefixes** | Parent | - Update the entire IPPrefix rule including seq number, networkPrefix, condition, subnetMask Length. <br>- Addition/update tags | *Re-PUT* of resource | To delete the resource, the connected `RoutePolicy` Resource shouldn't hold any reference. | +| **IPExtendedCommunity** | Parent | - Update entire IPExtended community rule including seq number, action, route targets. <br>- Addition/update tags | *Re-PUT* of resource | To delete the resource, the connected `RoutePolicy` Resource shouldn't hold any reference.| +| **ACLs** | Parent | - Addition/Update to match configurations and dynamic match configurations. <br>- Update to configuration type <br>- Addition/updating ACLs URL <br>- Addition/update tags | - *Re-PUT* of resource. <br>- Update to ACLs linked to a Network-to-Network Interconnect resource. | To delete the resource, the `connectedResource` (like `IsolationDomain` or N-to-N Interconnect) shouldn't hold any reference. | ++## Behavior notes and constraints ++- If a parent resource is in a **Disabled** administrative state and there are changes made to either to the parent or the child resources, the `commitConfiguration` action isn't applicable. Enabling the resource would push the configuration. The commit path for such resources is triggered only when the parent resource is in the **Enabled** administrative state. ++- If `commitConfiguration` fails, then the fabric remains in the **Accepted** in configuration state until the user addresses the issues and performs a successful `commitConfiguration`. Currently, only roll-forward mechanisms are provided when failure occurs. ++- If the Fabric configuration is in an **Accepted** state and has updates to Azure Resource Manager resources yet to be committed, then no administrative action is allowed on the resources. ++- If the Fabric configuration is in an **Accepted** state and has updates to Azure Resource Manager resources yet to be committed, then delete operation on supported resources can't be triggered. ++- Creation of parent resources is independent of `commitConfiguration` and the update flow. *Re-PUT* of resources isn't supported on any resource. ++- Network Fabric resource update is supported for both Greenfield deployments and Brownfield deployments but with some constraints. ++ - In the Greenfield deployment, the Fabric configuration state is **Accepted** once there are any updates done Network Fabric resources. Once the `commitConfiguration` action is triggered, it moves to either **Provisioned** or **Accepted** state depending on success or failure of the action. ++ - In the Brownfield deployment, the `commitConfiguration` action is supported but the supported Network Fabric resources (such as Isolation domains, Internal Networks, RoutePolicy & ACLs) must be created using general availability version of the API (2023-06-15). This temporary restriction is relaxed following the migration of all resources to the latest version. ++ - In the Brownfield deployment, the Fabric configuration state remains in a **Provisioned** state when there are changes to any supported Network Fabric resources or commitConfiguration action is triggered. This behavior is temporary until all fabrics are migrated to the latest version. ++- Route policy and other related resources (IP community, IP Extended Community, IP PrefixList) updates are considered as a list replace operation. All the existing statements are removed and only the new updated statements are configured. ++- Updating or removing existing subnets, routes, BGP configurations, and other relevant network params in Internal network or external networks configuration might cause traffic disruption and should be performed at operators discretion. ++- Update of new Route policies and ACLs might cause traffic disruption depending on the rules applied. ++- Use a list command on the specific resource type (list all resources of an internal network type) to verify the resources that are updated and aren't committed to device. The resources that have an **Accepted** or **Rejected** configuration state can be filtered and identified as resources that are yet to be committed or where the commit to device fails. ++For example: ++```azurecli +az networkfabric internalnetwork list --resource-group "example-rg" --l3domain "example-l3domain" +``` |
| payment-hsm | Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/payment-hsm/overview.md | Two host network interfaces and one management network interface are created at With the Azure Payment HSM provisioning service, customers have native access to two host network interfaces and one management interface on the payment HSM. This screenshot displays the Azure Payment HSM resources within a resource group. ## Why use Azure Payment HSM? |
| peering-service | Location Partners | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/peering-service/location-partners.md | The following table provides information on the Peering Service connectivity par | [IIJ](https://www.iij.ad.jp/en/) | Japan | | [Intercloud](https://intercloud.com/what-we-do/partners/microsoft-saas/)| Europe | | [Kordia](https://www.kordia.co.nz/cloudconnect) | Oceania |-| [LINX](https://www.linx.net/services/microsoft-azure-peering/) | Europe | +| [LINX](https://www.linx.net/services/microsoft-azure-peering/) | Europe, North America | | [Liquid Telecom](https://liquidc2.com/connect/#maps) | Africa | | [Lumen Technologies](https://www.ctl.io/microsoft-azure-peering-services/) | Asia, Europe, North America | | [MainOne](https://www.mainone.net/connectivity-services/cloud-connect/) | Africa | The following table provides information on the Peering Service connectivity par | Metro | Partners (IXPs) | |-|--| | Amsterdam | [AMS-IX](https://www.ams-ix.net/ams/service/microsoft-azure-peering-service-maps) |-| Ashburn | [Equinix IX](https://www.equinix.com/interconnection-services/internet-exchange/) | +| Ashburn | [Equinix IX](https://www.equinix.com/interconnection-services/internet-exchange/) , [LINX](https://www.linx.net/services/microsoft-azure-peering/) | | Atlanta | [Equinix IX](https://www.equinix.com/interconnection-services/internet-exchange/) | | Barcelona | [DE-CIX](https://www.de-cix.net/services/microsoft-azure-peering-service/) | | Chicago | [Equinix IX](https://www.equinix.com/interconnection-services/internet-exchange/) | The following table provides information on the Peering Service connectivity par | Kuala Lumpur | [DE-CIX](https://www.de-cix.net/services/microsoft-azure-peering-service/) | | London | [LINX](https://www.linx.net/services/microsoft-azure-peering/) | | Madrid | [DE-CIX](https://www.de-cix.net/services/microsoft-azure-peering-service/) |+| Manchester | [LINX](https://www.linx.net/services/microsoft-azure-peering/) | | Marseilles | [DE-CIX](https://www.de-cix.net/services/microsoft-azure-peering-service/) | | Mumbai | [DE-CIX](https://www.de-cix.net/services/microsoft-azure-peering-service/) | | New York | [DE-CIX](https://www.de-cix.net/services/microsoft-azure-peering-service/) | |
| postgresql | Concepts Extensions | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/postgresql/flexible-server/concepts-extensions.md | The following extensions are available in Azure Database for PostgreSQL flexible |**Extension Name** |**Description** |**Postgres 16**|**Postgres 15**|**Postgres 14**|**Postgres 13**|**Postgres 12**|**Postgres 11**| |--|--|--|--|--|--|--||-|[address_standardizer](http://postgis.net/docs/manual-2.5/Address_Standardizer.html) |Used to parse an address into constituent elements. |3.3.3 |3.1.1 |3.1.1 |3.1.1 |3.0.0 |2.5.1 | -|[address_standardizer_data_us](http://postgis.net/docs/manual-2.5/Address_Standardizer.html)|Address Standardizer US dataset example. |3.3.3 |3.1.1 |3.1.1 |3.1.1 |3.0.0 |2.5.1 | -|[amcheck](https://www.postgresql.org/docs/13/amcheck.html) |Functions for verifying the logical consistency of the structure of relations. |1.3 |1.2 |1.2 |1.2 |1.2 |1.1 | -|[anon](https://gitlab.com/dalibo/postgresql_anonymizer) |Mask or replace personally identifiable information (PII) or commercially sensitive data from a PostgreSQL database. |1.2.0 |1.2.0 |1.2.0 |1.2.0 |1.2.0 |N/A | -|[azure_ai](./generative-ai-azure-overview.md) |Azure OpenAI and Cognitive Services integration for PostgreSQL. |0.1.0 |0.1.0 |0.1.0 |0.1.0 |N/A |N/A | -|[azure_storage](../../postgresql/flexible-server/concepts-storage-extension.md) |Extension to export and import data from Azure Storage. |1.3 |1.3 |1.3 |1.3 |1.3 |N/A | -|[bloom](https://www.postgresql.org/docs/13/bloom.html) |Bloom access method - signature file based index. |1 |1 |1 |1 |1 |1 | +|[address_standardizer](http://postgis.net/docs/manual-2.5/Address_Standardizer.html) |Used to parse an address into constituent elements. |3.3.3 |3.3.1 |3.2.3 |3.2.3 |3.2.3 |3.2.3 | +|[address_standardizer_data_us](http://postgis.net/docs/manual-2.5/Address_Standardizer.html)|Address Standardizer US dataset example. |3.3.3 |3.3.1 |3.2.3 |3.2.3 |3.2.3 |3.2.3 | +|[amcheck](https://www.postgresql.org/docs/13/amcheck.html) |Functions for verifying the logical consistency of the structure of relations. |1.3 |1.3 |1.3 |1.2 |1.2 |1.1 | +|[anon](https://gitlab.com/dalibo/postgresql_anonymizer) |Mask or replace personally identifiable information (PII) or commercially sensitive data from a PostgreSQL database. |1.2.0 |1.2.0 |1.2.0 |1.2.0 |1.2.0 |1.2.0 | +|[azure_ai](./generative-ai-azure-overview.md) |Azure OpenAI and Cognitive Services integration for PostgreSQL. |1.0.0 |1.0.0 |1.0.0 |1.0.0 |N/A |N/A | +|[azure_storage](../../postgresql/flexible-server/concepts-storage-extension.md) |Extension to export and import data from Azure Storage. |1.4 |1.4 |1.4 |1.4 |1.4 |N/A | +|[bloom](https://www.postgresql.org/docs/13/bloom.html) |Bloom access method - signature file based index. |1.0 |1.0 |1.0 |1.0 |1.0 |1.0 | |[btree_gin](https://www.postgresql.org/docs/13/btree-gin.html) |Support for indexing common datatypes in GIN. |1.3 |1.3 |1.3 |1.3 |1.3 |1.3 |-|[btree_gist](https://www.postgresql.org/docs/13/btree-gist.html) |Support for indexing common datatypes in GiST. |1.7 |1.5 |1.5 |1.5 |1.5 |1.5 | +|[btree_gist](https://www.postgresql.org/docs/13/btree-gist.html) |Support for indexing common datatypes in GiST. |1.7 |1.7 |1.6 |1.5 |1.5 |1.5 | |[citext](https://www.postgresql.org/docs/13/citext.html) |Data type for case-insensitive character strings. |1.6 |1.6 |1.6 |1.6 |1.6 |1.5 |-|[cube](https://www.postgresql.org/docs/13/cube.html) |Data type for multidimensional cubes. |1.5 |1.4 |1.4 |1.4 |1.4 |1.4 | +|[cube](https://www.postgresql.org/docs/13/cube.html) |Data type for multidimensional cubes. |1.5 |1.5 |1.5 |1.4 |1.4 |1.4 | |[dblink](https://www.postgresql.org/docs/13/dblink.html) |Connect to other PostgreSQL databases from within a database. |1.2 |1.2 |1.2 |1.2 |1.2 |1.2 |-|[dict_int](https://www.postgresql.org/docs/13/dict-int.html) |Text search dictionary template for integers. |1 |1 |1 |1 |1 |1 | -|[dict_xsyn](https://www.postgresql.org/docs/13/dict-xsyn.html) |Text search dictionary template for extended synonym processing. |1 |1 |1 |1 |1 |1 | +|[dict_int](https://www.postgresql.org/docs/13/dict-int.html) |Text search dictionary template for integers. |1.0 |1.0 |1.0 |1.0 |1.0 |1.0 | +|[dict_xsyn](https://www.postgresql.org/docs/13/dict-xsyn.html) |Text search dictionary template for extended synonym processing. |1.0 |1.0 |1.0 |1.0 |1.0 |1.0 | |[earthdistance](https://www.postgresql.org/docs/13/earthdistance.html) |Calculate great-circle distances on the surface of the Earth. |1.1 |1.1 |1.1 |1.1 |1.1 |1.1 | |[fuzzystrmatch](https://www.postgresql.org/docs/13/fuzzystrmatch.html) |Determine similarities and distance between strings. |1.2 |1.1 |1.1 |1.1 |1.1 |1.1 |-|[hstore](https://www.postgresql.org/docs/13/hstore.html) |Data type for storing sets of (key, value) pairs. |1.8 |1.7 |1.7 |1.7 |1.2 |1.1.2 | -|[hypopg](https://github.com/HypoPG/hypopg) |Extension adding support for hypothetical indexes. |1.3.1 |1.3.1 |1.3.1 |1.3.1 |1.6 |1.5 | +|[hstore](https://www.postgresql.org/docs/13/hstore.html) |Data type for storing sets of (key, value) pairs. |1.8 |1.8 |1.8 |1.7 |1.6 |1.5 | +|[hypopg](https://github.com/HypoPG/hypopg) |Extension adding support for hypothetical indexes. |1.4.0 |1.4.0 |1.4.0 |1.4.0 |1.4.0 |1.4.0 | |[intagg](https://www.postgresql.org/docs/13/intagg.html) |Integer aggregator and enumerator. (Obsolete) |1.1 |1.1 |1.1 |1.1 |1.1 |1.1 |-|[intarray](https://www.postgresql.org/docs/13/intarray.html) |Functions, operators, and index support for 1-D arrays of integers. |1.5 |1.3 |1.3 |1.3 |1.2 |1.2 | -|[isn](https://www.postgresql.org/docs/13/isn.html) |Data types for international product numbering standards: EAN13, UPC, ISBN (books), ISMN (music), and ISSN (serials). |1.2 |1.2 |1.2 |1.2 |1.2 |1.2 | +|[intarray](https://www.postgresql.org/docs/13/intarray.html) |Functions, operators, and index support for 1-D arrays of integers. |1.5 |1.5 |1.5 |1.3 |1.2 |1.2 | +|[isn](https://www.postgresql.org/docs/13/isn.html) |Data types for international product numbering standards: EAN13, UPC, ISBN (books), ISMN (music), and ISSN (serials). |1.2 |1.2 |1.2 |1.2 |1.2 |1.2 | |[lo](https://www.postgresql.org/docs/13/lo.html) |Large object maintenance. |1.1 |1.1 |1.1 |1.1 |1.1 |1.1 | |[login_hook](https://github.com/splendiddata/login_hook) |Extension to execute some code on user login, comparable to Oracle's after logon trigger. |1.5 |1.4 |1.4 |1.4 |1.4 |1.4 | |[ltree](https://www.postgresql.org/docs/13/ltree.html) |Data type for hierarchical tree-like structures. |1.2 |1.2 |1.2 |1.2 |1.1 |1.1 |-|[orafce](https://github.com/orafce/orafce) |Implements in Postgres some of the functions from the Oracle database that are missing. |4.4 |3.24 |3.18 |3.18 |3.18 |3.18 | -|[pageinspect](https://www.postgresql.org/docs/13/pageinspect.html) |Inspect the contents of database pages at a low level. |1.12 |1.8 |1.8 |1.8 |1.7 |1.7 | +|[orafce](https://github.com/orafce/orafce) |Implements in Postgres some of the functions from the Oracle database that are missing. |4.4 |3.24 |3.18 |3.18 |3.18 |3.7 | +|[pageinspect](https://www.postgresql.org/docs/13/pageinspect.html) |Inspect the contents of database pages at a low level. |1.12 |1.11 |1.9 |1.8 |1.7 |1.7 | |[pg_buffercache](https://www.postgresql.org/docs/13/pgbuffercache.html) |Examine the shared buffer cache. |1.4 |1.3 |1.3 |1.3 |1.3 |1.3 |-|[pg_cron](https://github.com/citusdata/pg_cron) |Job scheduler for PostgreSQL. |1.5 |1.4 |1.4 |1.4 |1.4 |1.4 | -|[pg_failover_slots](https://github.com/EnterpriseDB/pg_failover_slots) (preview) |Logical replication slot manager for failover purposes. |1.0.1 |1.0.1 |1.0.1 |1.0.1 |1.0.1 |1.0.1 | +|[pg_cron](https://github.com/citusdata/pg_cron) |Job scheduler for PostgreSQL. |1.5 |1.4-1 |1.4-1 |1.4-1 |1.4-1 |1.4-1 | +!!!!!!!!|[pg_failover_slots](https://github.com/EnterpriseDB/pg_failover_slots) (preview) |Logical replication slot manager for failover purposes. |1.0.1 |1.0.1 |1.0.1 |1.0.1 |1.0.1 |1.0.1 | |[pg_freespacemap](https://www.postgresql.org/docs/13/pgfreespacemap.html) |Examine the free space map (FSM). |1.2 |1.2 |1.2 |1.2 |1.2 |1.2 |-|[pg_hint_plan](https://github.com/ossc-db/pg_hint_plan) |Makes it possible to tweak PostgreSQL execution plans using so-called "hints" in SQL comments. |1.6.0 |1.4 |1.4 |1.4 |1.4 |1.4 | +|[pg_hint_plan](https://github.com/ossc-db/pg_hint_plan) |Makes it possible to tweak PostgreSQL execution plans using so-called "hints" in SQL comments. |1.6.0 |1.5 |1.4 |1.3.7 |1.3.7 |1.3.7 | |[pg_partman](https://github.com/pgpartman/pg_partman) |Manage partitioned tables by time or ID. |4.7.1 |4.7.1 |4.6.1 |4.5.0 |4.5.0 |4.5.0 | |[pg_prewarm](https://www.postgresql.org/docs/13/pgprewarm.html) |Prewarm relation data. |1.2 |1.2 |1.2 |1.2 |1.2 |1.2 | |[pg_repack](https://reorg.github.io/pg_repack/) |Lets you remove bloat from tables and indexes. |1.4.7 |1.4.7 |1.4.7 |1.4.7 |1.4.7 |1.4.7 | |[pg_squeeze](https://github.com/cybertec-postgresql/pg_squeeze) |A tool to remove unused space from a relation. |1.6 |1.5 |1.5 |1.5 |1.5 |1.5 |-|[pg_stat_statements](https://www.postgresql.org/docs/13/pgstatstatements.html) |Track execution statistics of all SQL statements executed. |1.1 |1.8 |1.8 |1.8 |1.7 |1.6 | -|[pg_trgm](https://www.postgresql.org/docs/13/pgtrgm.html) |Text similarity measurement and index searching based on trigrams. |1.6 |1.5 |1.5 |1.5 |1.4 |1.4 | +|[pg_stat_statements](https://www.postgresql.org/docs/13/pgstatstatements.html) |Track execution statistics of all SQL statements executed. |1.10 |1.10 |1.9 |1.8 |1.7 |1.6 | +|[pg_trgm](https://www.postgresql.org/docs/13/pgtrgm.html) |Text similarity measurement and index searching based on trigrams. |1.6 |1.6 |1.6 |1.5 |1.4 |1.4 | |[pg_visibility](https://www.postgresql.org/docs/13/pgvisibility.html) |Examine the visibility map (VM) and page-level visibility info. |1.2 |1.2 |1.2 |1.2 |1.2 |1.2 |-|[pgaudit](https://www.pgaudit.org/) |Provides auditing functionality. |16.0 |1.7 |1.6.2 |1.5 |1.4 |1.3.1 | +|[pgaudit](https://www.pgaudit.org/) |Provides auditing functionality. |16.0 |1.7 |1.6.2 |1.5 |1.4 |1.3.2 | |[pgcrypto](https://www.postgresql.org/docs/13/pgcrypto.html) |Cryptographic functions. |1.3 |1.3 |1.3 |1.3 |1.3 |1.3 |-|[pglogical](https://github.com/2ndQuadrant/pglogical) |Logical streaming replication. |2.4.4 |2.3.2 |2.3.2 |2.3.2 |2.3.2 |2.3.2 | -|[pgrouting](https://pgrouting.org/) |Geospatial database to provide geospatial routing. |N/A |3.3.0 |3.3.0 |3.3.0 |3.3.0 |3.3.0 | +|[pglogical](https://github.com/2ndQuadrant/pglogical) |Logical streaming replication. |2.4.4 |2.4.2 |2.4.1 |2.4.1 |2.4.1 |2.4.1 | +!!!!!!!!|[pgrouting](https://pgrouting.org/) |Geospatial database to provide geospatial routing. |N/A |3.3.0 |3.3.0 |3.3.0 |3.3.0 |3.3.0 | |[pgrowlocks](https://www.postgresql.org/docs/13/pgrowlocks.html) |Show row-level locking information. |1.2 |1.2 |1.2 |1.2 |1.2 |1.2 | |[pgstattuple](https://www.postgresql.org/docs/13/pgstattuple.html) |Show tuple-level statistics. |1.5 |1.5 |1.5 |1.5 |1.5 |1.5 |-|[pgvector](https://github.com/pgvector/pgvector) |Open-source vector similarity search for Postgres. |0.6.0 |0.6.0 |0.6.0 |0.6.0 |0.6.0 |0.5.1 | -|[plpgsql](https://www.postgresql.org/docs/13/plpgsql.html) |PL/pgSQL procedural language. |1 |1 |1 |1 |1 |1 | -|[plv8](https://github.com/plv8/plv8) |Trusted JavaScript language extension. |3.1.7 |3.1.7 |3.0.0 |3.0.0 |3.2.0 |3.0.0 | -|[postgis](https://www.postgis.net/) |PostGIS geometry, geography. |3.3.3 |3.2.0 |3.2.0 |3.2.0 |3.2.0 |2.5.5 | -|[postgis_raster](https://www.postgis.net/) |PostGIS raster types and functions. |3.3.3 |3.2.0 |3.2.0 |3.2.0 |3.2.0 |N/A | -|[postgis_sfcgal](https://www.postgis.net/) |PostGIS SFCGAL functions. |3.3.3 |3.2.0 |3.2.0 |3.2.0 |3.2.0 |2.5.5 | -|[postgis_tiger_geocoder](https://www.postgis.net/) |PostGIS tiger geocoder and reverse geocoder. |3.3.3 |3.2.0 |3.2.0 |3.2.0 |3.2.0 |2.5.5 | -|[postgis_topology](https://postgis.net/docs/Topology.html) |PostGIS topology spatial types and functions. |3.3.3 |3.2.0 |3.2.0 |3.2.0 |3.2.0 |2.5.5 | -|[postgres_fdw](https://www.postgresql.org/docs/13/postgres-fdw.html) |Foreign-data wrapper for remote PostgreSQL servers. |1.1 |1 |1 |1 |1 |1 | +|[plpgsql](https://www.postgresql.org/docs/13/plpgsql.html) |PL/pgSQL procedural language. |1.0 |1.0 |1.0 |1.0 |1.0 |1.0 | +|[plv8](https://github.com/plv8/plv8) |Trusted JavaScript language extension. |3.1.7 |3.1.7 |3.0.0 |3.0.0 |3.0.0 |3.0.0 | +|[postgis](https://www.postgis.net/) |PostGIS geometry, geography. |3.3.3 |3.3.1 |3.2.3 |3.2.3 |3.2.3 |3.2.3 | +|[postgis_raster](https://www.postgis.net/) |PostGIS raster types and functions. |3.3.3 |3.3.1 |3.2.3 |3.2.3 |3.2.3 |3.2.3 | +|[postgis_sfcgal](https://www.postgis.net/) |PostGIS SFCGAL functions. |3.3.3 |3.3.1 |3.2.3 |3.2.3 |3.2.3 |3.2.3 | +|[postgis_tiger_geocoder](https://www.postgis.net/) |PostGIS tiger geocoder and reverse geocoder. |3.3.3 |3.3.1 |3.2.3 |3.2.3 |3.2.3 |3.2.3 | +|[postgis_topology](https://postgis.net/docs/Topology.html) |PostGIS topology spatial types and functions. |3.3.3 |3.3.1 |3.2.3 |3.2.3 |3.2.3 |3.2.3 | +|[postgres_fdw](https://www.postgresql.org/docs/13/postgres-fdw.html) |Foreign-data wrapper for remote PostgreSQL servers. |1.1 |1.1 |1.1 |1.0 |1.0 |1.0 | |[semver](https://pgxn.org/dist/semver/doc/semver.html) |Semantic version data type. |0.32.1 |0.32.0 |0.32.0 |0.32.0 |0.32.0 |0.32.0 | |[session_variable](https://github.com/splendiddata/session_variable) |Provides a way to create and maintain session scoped variables and constants. |3.3 |3.3 |3.3 |3.3 |3.3 |3.3 | |[sslinfo](https://www.postgresql.org/docs/13/sslinfo.html) |Information about SSL certificates. |1.2 |1.2 |1.2 |1.2 |1.2 |1.2 |-|[tablefunc](https://www.postgresql.org/docs/11/tablefunc.html) |Functions that manipulate whole tables, including crosstab. |1 |1 |1 |1 |1 |1 | +|[tablefunc](https://www.postgresql.org/docs/11/tablefunc.html) |Functions that manipulate whole tables, including crosstab. |1.0 |1.0 |1.0 |1.0 |1.0 |1.0 | |[tds_fdw](https://github.com/tds-fdw/tds_fdw) |PostgreSQL foreign data wrapper that can connect to databases that use the Tabular Data Stream (TDS) protocol, such as Sybase databases and Microsoft SQL server.|2.0.3 |2.0.3 |2.0.3 |2.0.3 |2.0.3 |2.0.3 |-|[timescaledb](https://github.com/timescale/timescaledb) |Open-source relational database for time-series and analytics. |N/A |2.5.1 |2.5.1 |2.5.1 |2.5.1 |1.7.4 | -|[tsm_system_rows](https://www.postgresql.org/docs/13/tsm-system-rows.html) |TABLESAMPLE method which accepts number of rows as a limit. |1 |1 |1 |1 |1 |1 | -|[tsm_system_time](https://www.postgresql.org/docs/13/tsm-system-time.html) |TABLESAMPLE method which accepts time in milliseconds as a limit. |1 |1 |1 |1 |1 |1 | +|[timescaledb](https://github.com/timescale/timescaledb) |Open-source relational database for time-series and analytics. |N/A |2.10.0 |2.10.0 |2.10.0 |2.10.0 |1.7.4 | +|[tsm_system_rows](https://www.postgresql.org/docs/13/tsm-system-rows.html) |TABLESAMPLE method which accepts number of rows as a limit. |1.0 |1.0 |1.0 |1.0 |1.0 |1.0 | +|[tsm_system_time](https://www.postgresql.org/docs/13/tsm-system-time.html) |TABLESAMPLE method which accepts time in milliseconds as a limit. |1.0 |1.0 |1.0 |1.0 |1.0 |1.0 | |[unaccent](https://www.postgresql.org/docs/13/unaccent.html) |Text search dictionary that removes accents. |1.1 |1.1 |1.1 |1.1 |1.1 |1.1 | |[uuid-ossp](https://www.postgresql.org/docs/13/uuid-ossp.html) |Generate universally unique identifiers (UUIDs). |1.1 |1.1 |1.1 |1.1 |1.1 |1.1 |+|[vector](https://github.com/pgvector/pgvector) |Open-source vector similarity search for Postgres (aka pgvector). |0.6.1 |0.6.1 |0.6.1 |0.6.1 |0.6.1 |0.5.1 | ## dblink and postgres_fdw For more details on restore method with Timescale enabled database, see [Timesca ### Restore a Timescale database using timescaledb-backup -While running `SELECT timescaledb_post_restore()` procedure listed above you might get permissions denied error updating timescaledb.restoring flag. This is due to limited ALTER DATABASE permission in Cloud PaaS database services. In this case you can perform alternative method using `timescaledb-backup` tool to backup and restore Timescale database. Timescaledb-backup is a program for making dumping and restoring a TimescaleDB database simpler, less error-prone, and more performant. +While running `SELECT timescaledb_post_restore()` procedure listed above you might get permissions denied error updating timescaledb.restoring flag. This is due to limited ALTER DATABASE permission in Cloud PaaS database services. In this case you can perform alternative method using `timescaledb-backup` tool to back up and restore Timescale database. Timescaledb-backup is a program for making dumping and restoring a TimescaleDB database simpler, less error-prone, and more performant. To do so, you should do following 1. Install tools as detailed [here](https://github.com/timescale/timescaledb-backup#installing-timescaledb-backup) 1. Create a target Azure Database for PostgreSQL flexible server instance and database |
| postgresql | Concepts Networking Ssl Tls | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/postgresql/flexible-server/concepts-networking-ssl-tls.md | For more on SSL\TLS configuration on the client, see [PostgreSQL documentation]( > * For connectivity to servers deployed to Azure government cloud regions (US Gov Virginia, US Gov Texas, US Gov Arizona): [DigiCert Global Root G2](https://www.digicert.com/kb/digicert-root-certificates.htm) and [Microsoft RSA Root Certificate Authority 2017](https://www.microsoft.com/pkiops/docs/repository.htm) root CA certificates, as services are migrating from Digicert to Microsoft CA. > * For connectivity to servers deployed to Azure public cloud regions worldwide : [Digicert Global Root CA](https://www.digicert.com/kb/digicert-root-certificates.htm) and [Microsoft RSA Root Certificate Authority 2017](https://www.microsoft.com/pkiops/docs/repository.htm), as services are migrating from Digicert to Microsoft CA. -### Importing Root CA Certificates in Java Key Store on the client for certificate pinning scenarios +### Downloading Root CA certificates and updating application clients in certificate pinning scenarios -Custom-written Java applications use a default keystore, called *cacerts*, which contains trusted certificate authority (CA) certificates. It's also often known as Java trust store. A certificates file named *cacerts* resides in the security properties directory, java.home\lib\security, where java.home is the runtime environment directory (the jre directory in the SDK or the top-level directory of the JavaΓäó 2 Runtime Environment). -You can use following directions to update client root CA certificates for client certificate pinning scenarios with PostgreSQL Flexible Server: -1. Make a backup copy of your custom keystore. -2. Download following certificates: +To update client applications in certificate pinning scenarios you can download certificates from following URIs: * For connectivity to servers deployed to Azure Government cloud regions (US Gov Virginia, US Gov Texas, US Gov Arizona) download Microsoft RSA Root Certificate Authority 2017 and DigiCert Global Root G2 certificates from following URIs: Microsoft RSA Root Certificate Authority 2017 https://www.microsoft.com/pkiops/certs/Microsoft%20RSA%20Root%20Certificate%20Authority%202017.crt, DigiCert Global Root G2 https://cacerts.digicert.com/DigiCertGlobalRootG2.crt.pem. * For connectivity to servers deployed in Azure public regions worldwide download Microsoft RSA Root Certificate Authority 2017 and DigiCert Global Root CA certificates from following URIs: Microsoft RSA Root Certificate Authority 2017 https://www.microsoft.com/pkiops/certs/Microsoft%20RSA%20Root%20Certificate%20Authority%202017.crt, Digicert Global Root CA https://cacerts.digicert.com/DigiCertGlobalRootCA.crt-3. Optionally, to prevent future disruption, it's also recommended to add the following roots to the trusted store: +* Optionally, to prevent future disruption, it's also recommended to add the following roots to the trusted store: Microsoft ECC Root Certificate Authority 2017 - https://www.microsoft.com/pkiops/certs/Microsoft%20ECC%20Root%20Certificate%20Authority%202017.crt-4. Generate a combined CA certificate store with both Root CA certificates are included. Example below shows using DefaultJavaSSLFactory for PostgreSQL JDBC users. - * For connectivity to servers deployed to Azure Government cloud regions (US Gov Virginia, US Gov Texas, US Gov Arizona) - ```powershell - - - keytool -importcert -alias PostgreSQLServerCACert -file D:\ DigiCertGlobalRootG2.crt.pem -keystore truststore -storepass password -noprompt --keytool -importcert -alias PostgreSQLServerCACert2 -file "D:\ Microsoft ECC Root Certificate Authority 2017.crt.pem" -keystore truststore -storepass password -noprompt -``` - * For connectivity to servers deployed in Azure public regions worldwide -```powershell -- keytool -importcert -alias PostgreSQLServerCACert -file D:\ DigiCertGlobalRootCA.crt.pem -keystore truststore -storepass password -noprompt --keytool -importcert -alias PostgreSQLServerCACert2 -file "D:\ Microsoft ECC Root Certificate Authority 2017.crt.pem" -keystore truststore -storepass password -noprompt -``` -- 5. Replace the original keystore file with the new generated one: - -```java -System.setProperty("javax.net.ssl.trustStore","path_to_truststore_file"); -System.setProperty("javax.net.ssl.trustStorePassword","password"); -``` -6. Replace the original root CA pem file with the combined root CA file and restart your application/client. --For more information on configuring client certificates with PostgreSQL JDBC driver, see this [documentation](https://jdbc.postgresql.org/documentation/ssl/) --> [!NOTE] -> Azure Database for PostgreSQL - Flexible server doesn't support [certificate based authentication](https://www.postgresql.org/docs/current/auth-cert.html) at this time. --### Get list of trusted certificates in Java Key Store --As stated above, Java, by default, stores the trusted certificates in a special file named *cacerts* that is located inside Java installation folder on the client. -Example below first reads *cacerts* and loads it into *KeyStore* object: -```java -private KeyStore loadKeyStore() { - String relativeCacertsPath = "/lib/security/cacerts".replace("/", File.separator); - String filename = System.getProperty("java.home") + relativeCacertsPath; - FileInputStream is = new FileInputStream(filename); - KeyStore keystore = KeyStore.getInstance(KeyStore.getDefaultType()); - String password = "changeit"; - keystore.load(is, password.toCharArray()); -- return keystore; -} -``` -The default password for *cacerts* is *changeit* , but should be different on real client, as administrators recommend changing password immediately after Java installation. -Once we loaded KeyStore object, we can use the *PKIXParameters* class to read certificates present. -```java -public void whenLoadingCacertsKeyStore_thenCertificatesArePresent() { - KeyStore keyStore = loadKeyStore(); - PKIXParameters params = new PKIXParameters(keyStore); - Set<TrustAnchor> trustAnchors = params.getTrustAnchors(); - List<Certificate> certificates = trustAnchors.stream() - .map(TrustAnchor::getTrustedCert) - .collect(Collectors.toList()); -- assertFalse(certificates.isEmpty()); -} -``` -### Updating Root CA certificates when using clients in Azure App Services with Azure Database for PostgreSQL - Flexible Server for certificate pinning scenarios --For Azure App services, connecting to Azure Database for PostgreSQL, we can have two possible scenarios on updating client certificates and it depends on how on you're using SSL with your application deployed to Azure App Services. --* Usually new certificates are added to App Service at platform level prior to changes in Azure Database for PostgreSQL - Flexible Server. If you are using the SSL certificates included on App Service platform in your application, then no action is needed. Consult following [Azure App Service documentation](../../app-service/configure-ssl-certificate.md) for more information. -* If you're explicitly including the path to SSL cert file in your code, then you would need to download the new cert and update the code to use the new cert. A good example of this scenario is when you use custom containers in App Service as shared in the [App Service documentation](../../app-service/tutorial-multi-container-app.md#configure-database-variables-in-wordpress) -- ### Updating Root CA certificates when using clients in Azure Kubernetes Service (AKS) with Azure Database for PostgreSQL - Flexible Server for certificate pinning scenarios --If you're trying to connect to the Azure Database for PostgreSQL using applications hosted in Azure Kubernetes Services (AKS) and pinning certificates, it's similar to access from a dedicated customers host environment. Refer to the steps [here](../../aks/ingress-tls.md). --### Updating Root CA certificates for For .NET (Npgsql) users on Windows with Azure Database for PostgreSQL - Flexible Server for certificate pinning scenarios --For .NET (Npgsql) users on Windows, connecting to Azure Database for PostgreSQL - Flexible Servers deployed in Azure Government cloud regions (US Gov Virginia, US Gov Texas, US Gov Arizona) make sure **both** Microsoft RSA Root Certificate Authority 2017 and DigiCert Global Root G2 both exist in Windows Certificate Store, Trusted Root Certification Authorities. If any certificates don't exist, import the missing certificate. --For .NET (Npgsql) users on Windows, connecting to Azure Database for PostgreSQL - Flexible Servers deployed in Azure pubiic regions worldwide make sure **both** Microsoft RSA Root Certificate Authority 2017 and DigiCert Global Root CA **both** exist in Windows Certificate Store, Trusted Root Certification Authorities. If any certificates don't exist, import the missing certificate. ----### Updating Root CA certificates for other clients for certificate pinning scenarios --For other PostgreSQL client users, you can merge two CA certificate files like this format below: --BEGIN CERTIFICATE---(Root CA1: DigiCertGlobalRootCA.crt.pem) END CERTIFICATE--BEGIN CERTIFICATE---(Root CA2: Microsoft ECC Root Certificate Authority 2017.crt.pem) END CERTIFICATE--+Detailed information on updating client applications certificate stores with new Root CA certificates has been documented in this [tutorial](../flexible-server/how-to-update-client-certificates-java.md). ### Read Replicas with certificate pinning scenarios Therefore, for clients that use **verify-ca** and **verify-full** sslmode config * For connectivity to servers deployed to Azure Government cloud regions (US Gov Virginia, US Gov Texas, US Gov Arizona): [DigiCert Global Root G2](https://www.digicert.com/kb/digicert-root-certificates.htm) and [Microsoft RSA Root Certificate Authority 2017](https://www.microsoft.com/pkiops/docs/repository.htm) root CA certificates, as services are migrating from Digicert to Microsoft CA. * For connectivity to servers deployed to Azure public cloud regions worldwide: [Digicert Global Root CA](https://www.digicert.com/kb/digicert-root-certificates.htm) and [Microsoft RSA Root Certificate Authority 2017](https://www.microsoft.com/pkiops/docs/repository.htm), as services are migrating from Digicert to Microsoft CA. +> [!NOTE] +> Azure Database for PostgreSQL - Flexible server doesn't support [certificate based authentication](https://www.postgresql.org/docs/current/auth-cert.html) at this time. -## Testing SSL\TLS Connectivity +## Testing SSL/TLS Connectivity Before trying to access your SSL enabled server from client application, make sure you can get to it via psql. You should see output similar to the following if you established an SSL connection. |
| postgresql | Generative Ai Azure Cognitive | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/postgresql/flexible-server/generative-ai-azure-cognitive.md | description: Create AI applications with sentiment analysis, summarization, or k Previously updated : 03/18/2024 Last updated : 04/05/2024 select azure_ai.set_setting('azure_cognitive.region', '<API Key>'); ### `azure_cognitive.analyze_sentiment` ```postgresql-azure_cognitive.analyze_sentiment(text text, language text, timeout_ms integer DEFAULT 3600000, throw_on_error boolean DEFAULT TRUE, disable_service_logs boolean DEFAULT false) +azure_cognitive.analyze_sentiment(text text, language text DEFAULT NULL::text, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.analyze_sentiment(text text[], language text DEFAULT NULL::text, batch_size integer DEFAULT 10, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.analyze_sentiment(text text[], language text[] DEFAULT NULL::text[], batch_size integer DEFAULT 10, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) ``` #### Arguments ##### `text` -`text` input to be processed. +`text` or `text[]` single text or array of texts, depending on the overload of the function used, with the input to be processed. ##### `language` -`text` two-letter ISO 639-1 representation of the language that the input text is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. +`text` or `text[]` single value or array of values, depending on the overload of the function used, with the two-letter ISO 639-1 representation of the language(s) that the input is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. ++#### `batch_size` ++`integer DEFAULT 10` number of records to process at a time (only available for the overload of the function for which parameter `input` is of type `text[]`). ++##### `disable_service_logs` ++`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. ##### `timeout_ms` azure_cognitive.analyze_sentiment(text text, language text, timeout_ms integer D `boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. -##### `disable_service_logs` +#### `max_attempts` -`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. +`integer DEFAULT 1` number of times the extension will retry calling the Azure Language Service endpoint for sentiment analysis if it fails with any retryable error. ++#### `retry_delay_ms` ++`integer DEFAULT 1000` amount of time (milliseconds) that the extension will wait, before calling again the Azure Language Service endpoint for sentiment analysis, when it fails with any retryable error. For more information, see Cognitive Services Compliance and Privacy notes at https://aka.ms/cs-compliance, and Microsoft Responsible AI principles at https://www.microsoft.com/ai/responsible-ai. #### Return type -`azure_cognitive.sentiment_analysis_result` a result record containing the sentiment predictions of the input text. It contains the sentiment, which can be `positive`, `negative`, `neutral`, and `mixed`; and the score for positive, neutral, and negative found in the text represented as a real number between 0 and 1. For example in `(neutral,0.26,0.64,0.09)`, the sentiment is `neutral` with `positive` score at `0.26`, neutral at `0.64` and negative at `0.09`. +`azure_cognitive.sentiment_analysis_result` or `TABLE(result azure_cognitive.sentiment_analysis_result)` a single element or a single-column table, depending on the overload of the function used, with the sentiment predictions of the input text. It contains the sentiment, which can be `positive`, `negative`, `neutral`, and `mixed`; and the score for positive, neutral, and negative found in the text represented as a real number between 0 and 1. For example in `(neutral,0.26,0.64,0.09)`, the sentiment is `neutral` with `positive` score at `0.26`, neutral at `0.64` and negative at `0.09`. ## Language detection For more information, see Cognitive Services Compliance and Privacy notes at htt ### `azure_cognitive.detect_language` ```postgresql-azure_cognitive.detect_language(text TEXT, timeout_ms INTEGER DEFAULT 3600000, throw_on_error BOOLEAN DEFAULT TRUE, disable_service_logs BOOLEAN DEFAULT FALSE) +azure_cognitive.detect_language(text text, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.detect_language(text text[], batch_size integer DEFAULT 1000, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) ``` #### Arguments ##### `text` -`text` input to be processed. +`text` or `text[]` single text or array of texts, depending on the overload of the function used, with the input to be processed. ++#### `batch_size` ++`integer DEFAULT 1000` number of records to process at a time (only available for the overload of the function for which parameter `input` is of type `text[]`). ++##### `disable_service_logs` ++`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. ##### `timeout_ms` azure_cognitive.detect_language(text TEXT, timeout_ms INTEGER DEFAULT 3600000, t `boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. -##### `disable_service_logs` +#### `max_attempts` -`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. +`integer DEFAULT 1` number of times the extension will retry calling the Azure Language Service endpoint for language detection if it fails with any retryable error. ++#### `retry_delay_ms` ++`integer DEFAULT 1000` amount of time (milliseconds) that the extension will wait, before calling again the Azure Language Service endpoint for language detection, when it fails with any retryable error. For more information, see Cognitive Services Compliance and Privacy notes at https://aka.ms/cs-compliance, and Microsoft Responsible AI principles at https://www.microsoft.com/ai/responsible-ai. #### Return type -`azure_cognitive.language_detection_result`, a result containing the detected language name, its two-letter ISO 639-1 representation, and the confidence score for the detection. For example in `(Portuguese,pt,0.97)`, the language is `Portuguese`, and detection confidence is `0.97`. +`azure_cognitive.language_detection_result` or `TABLE(result azure_cognitive.language_detection_result)` a single element or a single-column table, depending on the overload of the function used, with the detected language name, its two-letter ISO 639-1 representation, and the confidence score for the detection. For example in `(Portuguese,pt,0.97)`, the language is `Portuguese`, and detection confidence is `0.97`. ## Key phrase extraction For more information, see Cognitive Services Compliance and Privacy notes at htt ### `azure_cognitive.extract_key_phrases` ```postgresql-azure_cognitive.extract_key_phrases(text TEXT, language TEXT, timeout_ms INTEGER DEFAULT 3600000, throw_on_error BOOLEAN DEFAULT TRUE, disable_service_logs BOOLEAN DEFAULT FALSE) +azure_cognitive.extract_key_phrases(text text, language text DEFAULT NULL::text, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.extract_key_phrases(text text[], language text DEFAULT NULL::text, batch_size integer DEFAULT 10, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.extract_key_phrases(text text[], language text[] DEFAULT NULL::text[], batch_size integer DEFAULT 10, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) ``` #### Arguments ##### `text` -`text` input to be processed. +`text` or `text[]` single text or array of texts, depending on the overload of the function used, with the input to be processed. ##### `language` -`text` two-letter ISO 639-1 representation of the language that the input text is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. +`text` or `text[]` single value or array of values, depending on the overload of the function used, with the two-letter ISO 639-1 representation of the language(s) that the input is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. ++#### `batch_size` ++`integer DEFAULT 10` number of records to process at a time (only available for the overload of the function for which parameter `input` is of type `text[]`). ++##### `disable_service_logs` ++`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. ##### `timeout_ms` azure_cognitive.extract_key_phrases(text TEXT, language TEXT, timeout_ms INTEGER `boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. -##### `disable_service_logs` +#### `max_attempts` -`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. +`integer DEFAULT 1` number of times the extension will retry calling the Azure Language Service endpoint for key phrase extraction if it fails with any retryable error. ++#### `retry_delay_ms` ++`integer DEFAULT 1000` amount of time (milliseconds) that the extension will wait, before calling again the Azure Language Service endpoint for key phrase extraction, when it fails with any retryable error. For more information, see Cognitive Services Compliance and Privacy notes at https://aka.ms/cs-compliance, and Microsoft Responsible AI principles at https://www.microsoft.com/ai/responsible-ai. #### Return type -`text[]`, a collection of key phrases identified in the text. For example, if invoked with a `text` set to `'For more information, see Cognitive Services Compliance and Privacy notes.'`, and `language` set to `'en'`, it could return `{"Cognitive Services Compliance","Privacy notes",information}`. +`text[]` or `TABLE(key_phrases text[])` a single element or a single-column table, with the key phrases identified in the text. For example, if invoked with a `text` set to `'For more information, see Cognitive Services Compliance and Privacy notes.'`, and `language` set to `'en'`, it could return `{"Cognitive Services Compliance","Privacy notes",information}`. ## Entity linking For more information, see Cognitive Services Compliance and Privacy notes at htt ### `azure_cognitive.linked_entities` ```postgresql-azure_cognitive.linked_entities(text text, language text, timeout_ms integer DEFAULT 3600000, throw_on_error boolean DEFAULT true, disable_service_logs boolean DEFAULT false) +azure_cognitive.linked_entities(text text, language text DEFAULT NULL::text, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.linked_entities(text text[], language text DEFAULT NULL::text, batch_size integer DEFAULT 5, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.linked_entities(text text[], language text[] DEFAULT NULL::text[], batch_size integer DEFAULT 5, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) ``` #### Arguments ##### `text` -`text` input to be processed. +`text` or `text[]` single text or array of texts, depending on the overload of the function used, with the input to be processed. ##### `language` -`text` two-letter ISO 639-1 representation of the language that the input text is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. +`text` or `text[]` single value or array of values, depending on the overload of the function used, with the two-letter ISO 639-1 representation of the language(s) that the input is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. ++#### `batch_size` ++`integer DEFAULT 5` number of records to process at a time (only available for the overload of the function for which parameter `input` is of type `text[]`). ++##### `disable_service_logs` ++`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. ##### `timeout_ms` azure_cognitive.linked_entities(text text, language text, timeout_ms integer DEF `boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. +#### `max_attempts` ++`integer DEFAULT 1` number of times the extension will retry calling the Azure Language Service endpoint for linked identities if it fails with any retryable error. ++#### `retry_delay_ms` ++`integer DEFAULT 1000` amount of time (milliseconds) that the extension will wait, before calling again the Azure Language Service endpoint for linked identities, when it fails with any retryable error. + For more information, see Cognitive Services Compliance and Privacy notes at https://aka.ms/cs-compliance, and Microsoft Responsible AI principles at https://www.microsoft.com/ai/responsible-ai. #### Return type -`azure_cognitive.linked_entity[]`, a collection of linked entities, where each defines the name, data source entity identifier, language, data source, URL, collection of `azure_cognitive.linked_entity_match` (defining the text and confidence score) and finally a Bing entity search API identifier. For example, if invoked with a `text` set to `'For more information, see Cognitive Services Compliance and Privacy notes.'`, and `language` set to `'en'`, it could return `{"(\"Cognitive computing\",\"Cognitive computing\",en,Wikipedia,https://en.wikipedia.org/wiki/Cognitive_computing,\"{\"\"(\\\\\"\"Cognitive Services\\\\\"\",0.78)\ +`azure_cognitive.linked_entity[]` or `TABLE(entities azure_cognitive.linked_entity[])` an array or a single-column table, with the key phrases identified in the text, a collection of linked entities, where each defines the name, data source entity identifier, language, data source, URL, collection of `azure_cognitive.linked_entity_match` (defining the text and confidence score) and finally a Bing entity search API identifier. For example, if invoked with a `text` set to `'For more information, see Cognitive Services Compliance and Privacy notes.'`, and `language` set to `'en'`, it could return `{"(\"Cognitive computing\",\"Cognitive computing\",en,Wikipedia,https://en.wikipedia.org/wiki/Cognitive_computing,\"{\"\"(\\\\\"\"Cognitive Services\\\\\"\",0.78)\ "\"}\",d73f7d5f-fddb-0908-27b0-74c7db81cd8d)","(\"Regulatory compliance\",\"Regulatory compliance\",en,Wikipedia,https://en.wikipedia.org/wiki/Regulatory_compliance ,\"{\"\"(Compliance,0.28)\"\"}\",89fefaf8-e730-23c4-b519-048f3c73cdbd)","(\"Information privacy\",\"Information privacy\",en,Wikipedia,https://en.wikipedia.org/wiki /Information_privacy,\"{\"\"(Privacy,0)\"\"}\",3d0f2e25-5829-4b93-4057-4a805f0b1043)"}`. For more information, see Cognitive Services Compliance and Privacy notes at htt [Named Entity Recognition (NER) feature in Azure AI](../../ai-services/language-service/named-entity-recognition/overview.md) can identify and categorize entities in unstructured text. ```postgresql-azure_cognitive.recognize_entities(text text, language text, timeout_ms integer DEFAULT 3600000, throw_on_error boolean DEFAULT true, disable_service_logs boolean DEFAULT false) +azure_cognitive.recognize_entities(text text, language text DEFAULT NULL::text, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.recognize_entities(text text[], language text DEFAULT NULL::text, batch_size integer DEFAULT 5, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.recognize_entities(text text[], language text[] DEFAULT NULL::text[], batch_size integer DEFAULT 5, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) ``` #### Arguments ##### `text` -`text` input to be processed. +`text` or `text[]` single text or array of texts, depending on the overload of the function used, with the input to be processed. ##### `language` -`text` two-letter ISO 639-1 representation of the language that the input text is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. +`text` or `text[]` single value or array of values, depending on the overload of the function used, with the two-letter ISO 639-1 representation of the language(s) that the input is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. ++#### `batch_size` ++`integer DEFAULT 5` number of records to process at a time (only available for the overload of the function for which parameter `input` is of type `text[]`). ++##### `disable_service_logs` ++`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. ##### `timeout_ms` azure_cognitive.recognize_entities(text text, language text, timeout_ms integer `boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. -##### `disable_service_logs` +#### `max_attempts` -`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. +`integer DEFAULT 1` number of times the extension will retry calling the Azure Language Service endpoint for linked identities if it fails with any retryable error. ++#### `retry_delay_ms` ++`integer DEFAULT 1000` amount of time (milliseconds) that the extension will wait, before calling again the Azure Language Service endpoint for linked identities, when it fails with any retryable error. For more information, see Cognitive Services Compliance and Privacy notes at https://aka.ms/cs-compliance, and Microsoft Responsible AI principles at https://www.microsoft.com/ai/responsible-ai. #### Return type -`azure_cognitive.entity[]`, a collection of entities, where each defines the text identifying the entity, category of the entity and confidence score of the match. For example, if invoked with a `text` set to `'For more information, see Cognitive Services Compliance and Privacy notes.'`, and `language` set to `'en'`, it could return `{"(\"Cognitive Services\",Skill,\"\",0.94)"}`. +`azure_cognitive.entity[]` or `TABLE(entities azure_cognitive.entity[])` an array or a single-column table with entities, where each defines the text identifying the entity, category of the entity and confidence score of the match. For example, if invoked with a `text` set to `'For more information, see Cognitive Services Compliance and Privacy notes.'`, and `language` set to `'en'`, it could return `{"(\"Cognitive Services\",Skill,\"\",0.94)"}`. ## Personally Identifiable data (PII) detection For more information, see Cognitive Services Compliance and Privacy notes at htt ### `azure_cognitive.recognize_pii_entities` ```postgresql-azure_cognitive.recognize_pii_entities(text text, language text, timeout_ms integer DEFAULT 3600000, throw_on_error boolean DEFAULT true, domain text DEFAULT 'none'::text, disable_service_logs boolean DEFAULT true) +azure_cognitive.recognize_pii_entities(text text, language text DEFAULT NULL::text, domain text DEFAULT 'none'::text, disable_service_logs boolean DEFAULT true, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.recognize_pii_entities(text text[], language text DEFAULT NULL::text, domain text DEFAULT 'none'::text, batch_size integer DEFAULT 5, disable_service_logs boolean DEFAULT true, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.recognize_pii_entities(text text[], language text[] DEFAULT NULL::text[], domain text DEFAULT 'none'::text, batch_size integer DEFAULT 5, disable_service_logs boolean DEFAULT true, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) ``` #### Arguments ##### `text` -`text` input to be processed. +`text` or `text[]` single text or array of texts, depending on the overload of the function used, with the input to be processed. ##### `language` -`text` two-letter ISO 639-1 representation of the language that the input text is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. +`text` or `text[]` single value or array of values, depending on the overload of the function used, with the two-letter ISO 639-1 representation of the language(s) that the input is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. ++##### `domain` ++`text DEFAULT 'none'::text`, the personal data domain used for personal data Entity Recognition. Valid values are `none` for no domain specified and `phi` for Personal Health Information. ++#### `batch_size` ++`integer DEFAULT 5` number of records to process at a time (only available for the overload of the function for which parameter `input` is of type `text[]`). ++##### `disable_service_logs` ++`boolean DEFAULT true` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. ##### `timeout_ms` azure_cognitive.recognize_pii_entities(text text, language text, timeout_ms inte `boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. -##### `domain` +#### `max_attempts` -`text DEFAULT 'none'::text`, the personal data domain used for personal data Entity Recognition. Valid values are `none` for no domain specified and `phi` for Personal Health Information. +`integer DEFAULT 1` number of times the extension will retry calling the Azure Language Service endpoint for linked identities if it fails with any retryable error. -##### `disable_service_logs` +#### `retry_delay_ms` -`boolean DEFAULT true` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. +`integer DEFAULT 1000` amount of time (milliseconds) that the extension will wait, before calling again the Azure Language Service endpoint for linked identities, when it fails with any retryable error. For more information, see Cognitive Services Compliance and Privacy notes at https://aka.ms/cs-compliance, and Microsoft Responsible AI principles at https://www.microsoft.com/ai/responsible-ai. #### Return type -`azure_cognitive.pii_entity_recognition_result`, a result containing the redacted text, and entities as `azure_cognitive.entity[]`. Each entity contains the nonredacted text, personal data category, subcategory, and a score indicating the confidence that the entity correctly matches the identified substring. For example, if invoked with a `text` set to `'My phone number is +1555555555, and the address of my office is 16255 NE 36th Way, Redmond, WA 98052.'`, and `language` set to `'en'`, it could return `("My phone number is ***********, and the address of my office is ************************************.","{""(+1555555555,PhoneNumber,\\""\\"",0.8)"",""(\\""16255 NE 36th Way, Redmond, WA 98052\\"",Address,\\""\\"",1)""}")`. +`azure_cognitive.pii_entity_recognition_result` or `TABLE(result azure_cognitive.pii_entity_recognition_result)` a single value or a single-column table containing the redacted text, and entities as `azure_cognitive.entity[]`. Each entity contains the nonredacted text, personal data category, subcategory, and a score indicating the confidence that the entity correctly matches the identified substring. For example, if invoked with a `text` set to `'My phone number is +1555555555, and the address of my office is 16255 NE 36th Way, Redmond, WA 98052.'`, and `language` set to `'en'`, it could return `("My phone number is ***********, and the address of my office is ************************************.","{""(+1555555555,PhoneNumber,\\""\\"",0.8)"",""(\\""16255 NE 36th Way, Redmond, WA 98052\\"",Address,\\""\\"",1)""}")`. ## Document summarization For more information, see Cognitive Services Compliance and Privacy notes at htt [Document abstractive summarization](../../ai-services/language-service/summarization/overview.md) produces a summary that might not use the same words in the document but yet captures the main idea. ```postgresql-azure_cognitive.summarize_abstractive(text text, language text, timeout_ms integer DEFAULT 3600000, throw_on_error boolean DEFAULT true, sentence_count integer DEFAULT 3, disable_service_logs boolean DEFAULT false) +azure_cognitive.summarize_abstractive(text text, language text DEFAULT NULL::text, sentence_count integer DEFAULT 3, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.summarize_abstractive(text text[], language text DEFAULT NULL::text, sentence_count integer DEFAULT 3, batch_size integer DEFAULT 25, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.summarize_abstractive(text text[], language text[] DEFAULT NULL::text[], sentence_count integer DEFAULT 3, batch_size integer DEFAULT 25, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) ``` #### Arguments ##### `text` -`text` input to be processed. +`text` or `text[]` single text or array of texts, depending on the overload of the function used, with the input to be processed. ##### `language` -`text` two-letter ISO 639-1 representation of the language that the input text is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. +`text` or `text[]` single value or array of values, depending on the overload of the function used, with the two-letter ISO 639-1 representation of the language(s) that the input is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. ++##### `sentence_count` ++`integer DEFAULT 3`, maximum number of sentences that the summarization should contain. ++#### `batch_size` ++`integer DEFAULT 25` number of records to process at a time (only available for the overload of the function for which parameter `input` is of type `text[]`). ++##### `disable_service_logs` ++`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. ##### `timeout_ms` azure_cognitive.summarize_abstractive(text text, language text, timeout_ms integ `boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. -##### `sentence_count` +#### `max_attempts` -`integer DEFAULT 3`, maximum number of sentences that the summarization should contain. +`integer DEFAULT 1` number of times the extension will retry calling the Azure Language Service endpoint for linked identities if it fails with any retryable error. -##### `disable_service_logs` +#### `retry_delay_ms` -`boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. +`integer DEFAULT 1000` amount of time (milliseconds) that the extension will wait, before calling again the Azure Language Service endpoint for linked identities, when it fails with any retryable error. For more information, see Cognitive Services Compliance and Privacy notes at https://aka.ms/cs-compliance, and Microsoft Responsible AI principles at https://www.microsoft.com/ai/responsible-ai. #### Return type -`text[]`, a collection of summaries with each one not exceeding the defined `sentence_count`. For example, if invoked with a `text` set to `'PostgreSQL features transactions with atomicity, consistency, isolation, durability (ACID) properties, automatically updatable views, materialized views, triggers, foreign keys, and stored procedures. It is designed to handle a range of workloads, from single machines to data warehouses or web services with many concurrent users. It was the default database for macOS Server and is also available for Linux, FreeBSD, OpenBSD, and Windows.'`, and `language` set to `'en'`, it could return `{"PostgreSQL is a database system with advanced features such as atomicity, consistency, isolation, and durability (ACID) properties. It is designed to handle a range of workloads, from single machines to data warehouses or web services with many concurrent users. PostgreSQL was the default database for macOS Server and is available for Linux, BSD, OpenBSD, and Windows."}`. +`text[]` or `TABLE(summaries text[])` an array or a single-column table of summaries with each one not exceeding the defined `sentence_count`. For example, if invoked with a `text` set to `'PostgreSQL features transactions with atomicity, consistency, isolation, durability (ACID) properties, automatically updatable views, materialized views, triggers, foreign keys, and stored procedures. It is designed to handle a range of workloads, from single machines to data warehouses or web services with many concurrent users. It was the default database for macOS Server and is also available for Linux, FreeBSD, OpenBSD, and Windows.'`, and `language` set to `'en'`, it could return `{"PostgreSQL is a database system with advanced features such as atomicity, consistency, isolation, and durability (ACID) properties. It is designed to handle a range of workloads, from single machines to data warehouses or web services with many concurrent users. PostgreSQL was the default database for macOS Server and is available for Linux, BSD, OpenBSD, and Windows."}`. ### `azure_cognitive.summarize_extractive` [Document extractive summarization](../../ai-services/language-service/summarization/how-to/document-summarization.md) produces a summary extracting key sentences within the document. ```postgresql-azure_cognitive.summarize_extractive(text text, language text, timeout_ms integer DEFAULT 3600000, throw_on_error boolean DEFAULT true, sentence_count integer DEFAULT 3, sort_by text DEFAULT 'offset'::text, disable_service_logs boolean DEFAULT false) +azure_cognitive.summarize_extractive(text text, language text DEFAULT NULL::text, sentence_count integer DEFAULT 3, sort_by text DEFAULT 'offset'::text, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.summarize_extractive(text text[], language text DEFAULT NULL::text, sentence_count integer DEFAULT 3, sort_by text DEFAULT 'offset'::text, batch_size integer DEFAULT 25, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.summarize_extractive(text text[], language text[] DEFAULT NULL::text[], sentence_count integer DEFAULT 3, sort_by text DEFAULT 'offset'::text, batch_size integer DEFAULT 25, disable_service_logs boolean DEFAULT false, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) ``` #### Arguments ##### `text` -`text` input to be processed. +`text` or `text[]` single text or array of texts, depending on the overload of the function used, with the input to be processed. ##### `language` -`text` two-letter ISO 639-1 representation of the language that the input text is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. --##### `timeout_ms` --`integer DEFAULT 3600000` timeout in milliseconds after which the operation is stopped. --##### `throw_on_error` --`boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. +`text` or `text[]` single value or array of values, depending on the overload of the function used, with the two-letter ISO 639-1 representation of the language(s) that the input is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. ##### `sentence_count` azure_cognitive.summarize_extractive(text text, language text, timeout_ms intege `text DEFAULT ``offset``::text`, order of extracted sentences. Valid values are `rank` and `offset`. +#### `batch_size` ++`integer DEFAULT 25` number of records to process at a time (only available for the overload of the function for which parameter `input` is of type `text[]`). + ##### `disable_service_logs` `boolean DEFAULT false` the Language service logs your input text for 48 hours solely to allow for troubleshooting issues. Setting this property to `true` disables input logging and might limit our ability to investigate issues that occur. +##### `timeout_ms` ++`integer DEFAULT 3600000` timeout in milliseconds after which the operation is stopped. ++##### `throw_on_error` ++`boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. ++#### `max_attempts` ++`integer DEFAULT 1` number of times the extension will retry calling the Azure Language Service endpoint for linked identities if it fails with any retryable error. ++#### `retry_delay_ms` ++`integer DEFAULT 1000` amount of time (milliseconds) that the extension will wait, before calling again the Azure Language Service endpoint for linked identities, when it fails with any retryable error. + For more information, see Cognitive Services Compliance and Privacy notes at https://aka.ms/cs-compliance, and Microsoft Responsible AI principles at https://www.microsoft.com/ai/responsible-ai. #### Return type -`azure_cognitive.sentence[]`, a collection of extracted sentences along with their rank score. +`azure_cognitive.sentence[]` or `TABLE(sentences azure_cognitive.sentence[])` an array or a single-column table of extracted sentences along with their rank score. For example, if invoked with a `text` set to `'PostgreSQL features transactions with atomicity, consistency, isolation, durability (ACID) properties, automatically updatable views, materialized views, triggers, foreign keys, and stored procedures. It is designed to handle a range of workloads, from single machines to data warehouses or web services with many concurrent users. It was the default database for macOS Server and is also available for Linux, FreeBSD, OpenBSD, and Windows.'`, and `language` set to `'en'`, it could return `{"(\"PostgreSQL features transactions with atomicity, consistency, isolation, durability (ACID) properties, automatically updatable views, materialized views, triggers, foreign keys, and stored procedures.\",0.16)","(\"It is designed to handle a range of workloads, from single machines to data warehouses or web services with many concurrent users.\",0)","(\"It was the default database for macOS Server and is also available for Linux, FreeBSD, OpenBSD, and Windows.\",1)"}`. ## Language translation For example, if invoked with a `text` set to `'PostgreSQL features transactions ### `azure_cognitive.translate` ```postgresql-azure_cognitive.translate(text text, target_language text, timeout_ms integer DEFAULT NULL, throw_on_error boolean DEFAULT true, source_language text DEFAULT NULL, text_type text DEFAULT 'plain', profanity_action text DEFAULT 'NoAction', profanity_marker text DEFAULT 'Asterisk', suggested_source_language text DEFAULT NULL , source_script text DEFAULT NULL , target_script text DEFAULT NULL) +azure_cognitive.translate(text text, target_language text, source_language text DEFAULT NULL::text, text_type text DEFAULT 'Plain'::text, profanity_action text DEFAULT 'NoAction'::text, profanity_marker text DEFAULT 'Asterisk'::text, suggested_source_language text DEFAULT NULL::text, source_script text DEFAULT NULL::text, target_script text DEFAULT NULL::text, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.translate(text text, target_language text[], source_language text DEFAULT NULL::text, text_type text DEFAULT 'Plain'::text, profanity_action text DEFAULT 'NoAction'::text, profanity_marker text DEFAULT 'Asterisk'::text, suggested_source_language text DEFAULT NULL::text, source_script text DEFAULT NULL::text, target_script text[] DEFAULT NULL::text[], timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.translate(text text[], target_language text, source_language text DEFAULT NULL::text, text_type text DEFAULT 'Plain'::text, profanity_action text DEFAULT 'NoAction'::text, profanity_marker text DEFAULT 'Asterisk'::text, suggested_source_language text DEFAULT NULL::text, source_script text DEFAULT NULL::text, target_script text DEFAULT NULL::text, batch_size integer DEFAULT 1000, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_cognitive.translate(text text[], target_language text[], source_language text DEFAULT NULL::text, text_type text DEFAULT 'Plain'::text, profanity_action text DEFAULT 'NoAction'::text, profanity_marker text DEFAULT 'Asterisk'::text, suggested_source_language text DEFAULT NULL::text, source_script text DEFAULT NULL::text, target_script text[] DEFAULT NULL::text[], batch_size integer DEFAULT 1000, timeout_ms integer DEFAULT NULL::integer, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) ``` > [!NOTE] For more information on parameters, see [Translator API](../../ai-services/trans ##### `text` -`text` the input text to be translated +`text` or `text[]` single text or array of texts, depending on the overload of the function used, with the input to be processed. ##### `target_language` -`text` two-letter ISO 639-1 representation of the language that you want the input text to be translated to. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. --##### `timeout_ms` --`integer DEFAULT 3600000` timeout in milliseconds after which the operation is stopped. --##### `throw_on_error` --`boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. +`text` or `text[]` single value or array of values, depending on the overload of the function used, with the two-letter ISO 639-1 representation of the language(s) that the input is written in. Check [language support](../../ai-services/language-service/concepts/language-support.md) for allowed values. ##### `source_language` For more information on parameters, see [Translator API](../../ai-services/trans ##### `target_script` `text DEFAULT NULL` Specific script of the input text. +#### `batch_size` ++`integer DEFAULT 1000` number of records to process at a time (only available for the overload of the function for which parameter `text` is of type `text[]`). ++##### `timeout_ms` ++`integer DEFAULT 3600000` timeout in milliseconds after which the operation is stopped. ++##### `throw_on_error` ++`boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. ++#### `max_attempts` ++`integer DEFAULT 1` number of times the extension will retry calling the Azure Language Service endpoint for linked identities if it fails with any retryable error. ++#### `retry_delay_ms` ++`integer DEFAULT 1000` amount of time (milliseconds) that the extension will wait, before calling again the Azure Language Service endpoint for linked identities, when it fails with any retryable error. ++ #### Return type -`azure_cognitive.translated_text_result`, a json array of translated texts. Details of the response body can be found in the [response body](../../ai-services/translator/reference/v3-0-translate.md#response-body). +`azure_cognitive.translated_text_result` or `TABLE(result azure_cognitive.translated_text_result)` an array or a single-column table of translated texts. Details of the response body can be found in the [response body](../../ai-services/translator/reference/v3-0-translate.md#response-body). ## Examples |
| postgresql | Generative Ai Azure Openai | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/postgresql/flexible-server/generative-ai-azure-openai.md | Title: Generate vector embeddings with Azure OpenAI in Azure Database for Postgr description: Use vector indexes and Azure Open AI embeddings in PostgreSQL for retrieval augmented generation (RAG) patterns. Previously updated : 01/02/2024 Last updated : 04/05/2024 Invoke [Azure OpenAI embeddings](../../ai-services/openai/reference.md#embedding In the Azure OpenAI resource, under **Resource Management** > **Keys and Endpoints** you can find the endpoint and the keys for your Azure OpenAI resource. To invoke the model deployment, enable the `azure_ai` extension using the endpoint and one of the keys. ```postgresql-select azure_ai.set_setting('azure_openai.endpoint','https://<endpoint>.openai.azure.com'); +select azure_ai.set_setting('azure_openai.endpoint', 'https://<endpoint>.openai.azure.com'); select azure_ai.set_setting('azure_openai.subscription_key', '<API Key>'); ``` select azure_ai.set_setting('azure_openai.subscription_key', '<API Key>'); Invokes the Azure OpenAI API to create embeddings using the provided deployment over the given input. ```postgresql-azure_openai.create_embeddings(deployment_name text, input text, timeout_ms integer DEFAULT 3600000, throw_on_error boolean DEFAULT true) +azure_openai.create_embeddings(deployment_name text, input text, timeout_ms integer DEFAULT 3600000, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) +azure_openai.create_embeddings(deployment_name text, input text[], batch_size integer DEFAULT 100, timeout_ms integer DEFAULT 3600000, throw_on_error boolean DEFAULT true, max_attempts integer DEFAULT 1, retry_delay_ms integer DEFAULT 1000) ```- ### Arguments #### `deployment_name` azure_openai.create_embeddings(deployment_name text, input text, timeout_ms inte #### `input` -`text` input used to create embeddings. +`text` or `text[]` single text or array of texts, depending on the overload of the function used, for which embeddings are created. ++#### `batch_size` ++`integer DEFAULT 100` number of records to process at a time (only available for the overload of the function for which parameter `input` is of type `text[]`). #### `timeout_ms` azure_openai.create_embeddings(deployment_name text, input text, timeout_ms inte `boolean DEFAULT true` on error should the function throw an exception resulting in a rollback of wrapping transactions. +#### `max_attempts` ++`integer DEFAULT 1` number of times the extension will retry calling the Azure OpenAI endpoint for embedding creation if it fails with any retryable error. ++#### `retry_delay_ms` ++`integer DEFAULT 1000` amount of time (milliseconds) that the extension will wait, before calling again the Azure OpenAI endpoint for embedding creation, when it fails with any retryable error. + ### Return type -`real[]` a vector representation of the input text when processed by the selected deployment. +`real[]` or `TABLE(embedding real[])` a single element or a single-column table, depending on the overload of the function used, with vector representations of the input text, when processed by the selected deployment. ## Use OpenAI to create embeddings and store them in a vector data type |
| postgresql | How To Update Client Certificates Java | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/postgresql/flexible-server/how-to-update-client-certificates-java.md | + + Title: Updating Client SSL/TLS Certificates for Java +description: Learn about updating Java clients with Flexible Server using SSL and TLS. ++ Last updated : 04/04/2024++++++# Update Client TLS Certificates for Application Clients with Azure Database for PostgreSQL - Flexible Server ++++## Import Root CA Certificates in Java Key Store on the client for certificate pinning scenarios ++Custom-written Java applications use a default keystore, called *cacerts*, which contains trusted certificate authority (CA) certificates. It's also often known as Java trust store. A certificates file named *cacerts* resides in the security properties directory, java.home\lib\security, where java.home is the runtime environment directory (the jre directory in the SDK or the top-level directory of the JavaΓäó 2 Runtime Environment). +You can use following directions to update client root CA certificates for client certificate pinning scenarios with PostgreSQL Flexible Server: +1. Make a backup copy of your custom keystore. +2. Download [certificates](../flexible-server/concepts-networking-ssl-tls.md#downloading-root-ca-certificates-and-updating-application-clients-in-certificate-pinning-scenarios) +3. Generate a combined CA certificate store with both Root CA certificates are included. Example below shows using DefaultJavaSSLFactory for PostgreSQL JDBC users. ++ * For connectivity to servers deployed to Azure Government cloud regions (US Gov Virginia, US Gov Texas, US Gov Arizona) + ```powershell + + + keytool -importcert -alias PostgreSQLServerCACert -file D:\ DigiCertGlobalRootG2.crt.pem -keystore truststore -storepass password -noprompt ++ keytool -importcert -alias PostgreSQLServerCACert2 -file "D:\ Microsoft ECC Root Certificate Authority 2017.crt.pem" -keystore truststore -storepass password -noprompt + ``` + * For connectivity to servers deployed in Azure public regions worldwide + ```powershell ++ keytool -importcert -alias PostgreSQLServerCACert -file D:\ DigiCertGlobalRootCA.crt.pem -keystore truststore -storepass password -noprompt ++ keytool -importcert -alias PostgreSQLServerCACert2 -file "D:\ Microsoft ECC Root Certificate Authority 2017.crt.pem" -keystore truststore -storepass password -noprompt + ``` ++ 5. Replace the original keystore file with the new generated one: + + ```java + System.setProperty("javax.net.ssl.trustStore","path_to_truststore_file"); + System.setProperty("javax.net.ssl.trustStorePassword","password"); + ``` +6. Replace the original root CA pem file with the combined root CA file and restart your application/client. ++For more information on configuring client certificates with PostgreSQL JDBC driver, see this [documentation.](https://jdbc.postgresql.org/documentation/ssl/) ++++## Get list of trusted certificates in Java Key Store ++As stated above, Java, by default, stores the trusted certificates in a special file named *cacerts* that is located inside Java installation folder on the client. +Example below first reads *cacerts* and loads it into *KeyStore* object: +```java +private KeyStore loadKeyStore() { + String relativeCacertsPath = "/lib/security/cacerts".replace("/", File.separator); + String filename = System.getProperty("java.home") + relativeCacertsPath; + FileInputStream is = new FileInputStream(filename); + KeyStore keystore = KeyStore.getInstance(KeyStore.getDefaultType()); + String password = "changeit"; + keystore.load(is, password.toCharArray()); ++ return keystore; +} +``` +The default password for *cacerts* is *changeit* , but should be different on real client, as administrators recommend changing password immediately after Java installation. +Once we loaded KeyStore object, we can use the *PKIXParameters* class to read certificates present. +```java +public void whenLoadingCacertsKeyStore_thenCertificatesArePresent() { + KeyStore keyStore = loadKeyStore(); + PKIXParameters params = new PKIXParameters(keyStore); + Set<TrustAnchor> trustAnchors = params.getTrustAnchors(); + List<Certificate> certificates = trustAnchors.stream() + .map(TrustAnchor::getTrustedCert) + .collect(Collectors.toList()); ++ assertFalse(certificates.isEmpty()); +} +``` +## Update Root CA certificates when using clients in Azure App Services with Azure Database for PostgreSQL - Flexible Server for certificate pinning scenarios ++For Azure App services, connecting to Azure Database for PostgreSQL, we can have two possible scenarios on updating client certificates and it depends on how on you're using SSL with your application deployed to Azure App Services. ++* Usually new certificates are added to App Service at platform level prior to changes in Azure Database for PostgreSQL - Flexible Server. If you're using the SSL certificates included on App Service platform in your application, then no action is needed. Consult following [Azure App Service documentation](../../app-service/configure-ssl-certificate.md) for more information. +* If you're explicitly including the path to SSL cert file in your code, then you would need to download the new cert and update the code to use the new cert. A good example of this scenario is when you use custom containers in App Service as shared in the [App Service documentation](../../app-service/tutorial-multi-container-app.md#configure-database-variables-in-wordpress) ++ ## Update Root CA certificates when using clients in Azure Kubernetes Service (AKS) with Azure Database for PostgreSQL - Flexible Server for certificate pinning scenarios ++If you're trying to connect to the Azure Database for PostgreSQL using applications hosted in Azure Kubernetes Services (AKS) and pinning certificates, it's similar to access from a dedicated customers host environment. Refer to the steps [here](../../aks/ingress-tls.md). ++## Updating Root CA certificates for .NET (Npgsql) users on Windows with Azure Database for PostgreSQL - Flexible Server for certificate pinning scenarios ++For .NET (Npgsql) users on Windows, connecting to Azure Database for PostgreSQL - Flexible Servers deployed in Azure Government cloud regions (US Gov Virginia, US Gov Texas, US Gov Arizona) make sure **both** Microsoft RSA Root Certificate Authority 2017 and DigiCert Global Root G2 both exist in Windows Certificate Store, Trusted Root Certification Authorities. If any certificates don't exist, import the missing certificate. ++For .NET (Npgsql) users on Windows, connecting to Azure Database for PostgreSQL - Flexible Servers deployed in Azure public regions worldwide make sure **both** Microsoft RSA Root Certificate Authority 2017 and DigiCert Global Root CA **both** exist in Windows Certificate Store, Trusted Root Certification Authorities. If any certificates don't exist, import the missing certificate. ++++## Updating Root CA certificates for other clients for certificate pinning scenarios ++For other PostgreSQL client users, you can merge two CA certificate files like this format below. ++```azurecli +++--BEGIN CERTIFICATE-- +(Root CA1: DigiCertGlobalRootCA.crt.pem) +--END CERTIFICATE-- +--BEGIN CERTIFICATE-- +(Root CA2: Microsoft ECC Root Certificate Authority 2017.crt.pem) +--END CERTIFICATE-- +``` ++## Related content ++- Learn how to create an Azure Database for PostgreSQL flexible server instance by using the **Private access (VNet integration)** option in [the Azure portal](how-to-manage-virtual-network-portal.md) or [the Azure CLI](how-to-manage-virtual-network-cli.md). +- Learn how to create an Azure Database for PostgreSQL flexible server instance by using the **Public access (allowed IP addresses)** option in [the Azure portal](how-to-manage-firewall-portal.md) or [the Azure CLI](how-to-manage-firewall-cli.md). |
| postgresql | Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/postgresql/flexible-server/overview.md | One advantage of running your workload in Azure is global reach. Azure Database | UAE Central* | :heavy_check_mark: (v3/v4 only) | :x: | :heavy_check_mark: | :heavy_check_mark: | | UAE North | :heavy_check_mark: (v3/v4/v5 only) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | | US Gov Arizona | :heavy_check_mark: (v3/v4 only) | :x: | :heavy_check_mark: | :x: |-| US Gov Texas | :heavy_check_mark: (v3/v4 only) | :x: | :heavy_check_mark: | :heavy_check_mark: | +| US Gov Texas | :heavy_check_mark: (v3/v4 only) | :x: | :heavy_check_mark: | :x: | | US Gov Virginia | :heavy_check_mark: (v3/v4 only) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark:| | UK South | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | | UK West | :heavy_check_mark: | :x: | :heavy_check_mark: | :heavy_check_mark: | |
| postgresql | Service Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/postgresql/flexible-server/service-overview.md | Azure Database for PostgreSQL flexible server powered by the PostgreSQL communit ### Azure Database for PostgreSQL flexible server -Azure Database for PostgreSQL flexible server is a fully managed database service designed to provide more granular control and flexibility over database management functions and configuration settings. In general, the service provides more flexibility and customizations based on the user requirements. The flexible server architecture allows users to opt for high availability within single availability zone and across multiple availability zones. Azure Database for PostgreSQL flexible server provides better cost optimization controls with the ability to stop/start server and burstable compute tier, ideal for workloads that donΓÇÖt need full-compute capacity continuously. Azure Database for PostgreSQL flexible server currently supports community version of PostgreSQL 11, 12, 13 and 14, with plans to add newer versions soon. Azure Database for PostgreSQL flexible server is generally available today in a wide variety of [Azure regions](overview.md#azure-regions). +Azure Database for PostgreSQL flexible server is a fully managed database service designed to provide more granular control and flexibility over database management functions and configuration settings. In general, the service provides more flexibility and customizations based on the user requirements. The flexible server architecture allows users to opt for high availability within single availability zone and across multiple availability zones. Azure Database for PostgreSQL flexible server provides better cost optimization controls with the ability to stop/start server and burstable compute tier, ideal for workloads that donΓÇÖt need full-compute capacity continuously. Azure Database for PostgreSQL flexible server currently supports community version of PostgreSQL 11, 12, 13, 14, 15, and 16 with plans to add newer versions as they become available. Azure Database for PostgreSQL flexible server is generally available today in a wide variety of [Azure regions](overview.md#azure-regions). -Azure Database for PostgreSQL flexible server instances are best suited for +Azure Database for PostgreSQL flexible server instances are best suited for: - Application developments requiring better control and customizations - Cost optimization controls with ability to stop/start server |
| private-5g-core | Azure Private 5G Core Release Notes 2308 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/private-5g-core/azure-private-5g-core-release-notes-2308.md | The following table shows the support status for different Packet Core releases. | Release | Support Status | ||-|-| AP5GC 2308 | Supported until AP5GC 2401 released | +| AP5GC 2308 | Supported until AP5GC 2403 released | | AP5GC 2307 | Supported until AP5GC 2310 released | | AP5GC 2306 and earlier | Out of Support | |
| private-5g-core | Azure Private 5G Core Release Notes 2310 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/private-5g-core/azure-private-5g-core-release-notes-2310.md | Last updated 11/30/2023 # Azure Private 5G Core 2310 release notes -The following release notes identify the new features, critical open issues, and resolved issues for the 2308 release of Azure Private 5G Core (AP5GC). The release notes are continuously updated, with critical issues requiring a workaround added as theyΓÇÖre discovered. Before deploying this new version, review the information contained in these release notes. +The following release notes identify the new features, critical open issues, and resolved issues for the 2310 release of Azure Private 5G Core (AP5GC). The release notes are continuously updated, with critical issues requiring a workaround added as theyΓÇÖre discovered. Before deploying this new version, review the information contained in these release notes. This article applies to the AP5GC 2310 release (2310.0-8). This release is compatible with the Azure Stack Edge Pro 1 GPU and Azure Stack Edge Pro 2 running the ASE 2309 release and supports the 2023-09-01, 2023-06-01 and 2022-11-01 [Microsoft.MobileNetwork](/rest/api/mobilenetwork) API versions. The following table shows the support status for different Packet Core releases | Release | Support Status | ||-|-| AP5GC 2310 | Supported until AP5GC 2403 is released | -| AP5GC 2308 | Supported until AP5GC 2401 is released | +| AP5GC 2310 | Supported until AP5GC 2404 is released | +| AP5GC 2308 | Supported until AP5GC 2403 is released | | AP5GC 2307 and earlier | Out of Support | ## What's new |
| private-5g-core | Azure Private 5G Core Release Notes 2403 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/private-5g-core/azure-private-5g-core-release-notes-2403.md | + + Title: Azure Private 5G Core 2403 release notes +description: Discover what's new in the Azure Private 5G Core 2403 release ++++ Last updated : 04/04/2023+++# Azure Private 5G Core 2403 release notes ++The following release notes identify the new features, critical open issues, and resolved issues for the 2403 release of Azure Private 5G Core (AP5GC). The release notes are continuously updated, with critical issues requiring a workaround added as theyΓÇÖre discovered. Before deploying this new version, review the information contained in these release notes. ++This article applies to the AP5GC 2403 release (2403.0-2). This release is compatible with the Azure Stack Edge (ASE) Pro 1 GPU and Azure Stack Edge Pro 2 running the ASE 2403 release and supports the 2023-09-01, 2023-06-01 and 2022-11-01 [Microsoft.MobileNetwork](/rest/api/mobilenetwork) API versions. ++For more information about compatibility, see [Packet core and Azure Stack Edge compatibility](azure-stack-edge-packet-core-compatibility.md). ++For more information about new features in Azure Private 5G Core, see [What's New Guide](whats-new.md). ++## Support lifetime ++Packet core versions are supported until two subsequent versions are released (unless otherwise noted). You should plan to upgrade your packet core in this time frame to avoid losing support. ++### Currently supported packet core versions +The following table shows the support status for different Packet Core releases and when they're expected to no longer be supported. ++| Release | Support Status | +||-| +| AP5GC 2403 | Supported until AP5GC 2407 is released | +| AP5GC 2310 | Supported until AP5GC 2404 is released | +| AP5GC 2308 and earlier | Out of Support | ++## What's new ++### TCP Maximum Segment Size (MSS) Clamping ++TCP session initial setup messages that include a Maximum Segment Size (MSS) value, which controls the size limit of packets transmitted during the session. The packet core will now automatically set this value, where necessary, to ensure packets aren't too large for the core to transmit. This reduces packet loss due to oversized packets arriving at the core's interfaces, and reduces the need for fragmentation and reassembly, which are costly procedures. ++## Issues fixed in the AP5GC 2403 release ++The following table provides a summary of issues fixed in this release. ++ |No. |Feature | Issue | SKU Fixed In | + |--|--|--|--|-| + | 1 | Local distributed tracing | In Multi PDN session establishment/Release call flows with different DNs, the distributed tracing web GUI fails to display some of 4G NAS messages (Activate/deactivate Default EPS Bearer Context Request) and some S1AP messages (ERAB request, ERAB Release). | 2403.0-2 | + | 2 | Packet Forwarding | A slight(0.01%) increase in packet drops is observed in latest AP5GC release installed on ASE Platform Pro 2 with ASE-2309 for throughput higher than 3.0 Gbps. | 2403.0-2 | ++## Known issues in the AP5GC 2403 release +<!--**TO BE UPDATED**> + |No. |Feature | Issue | Workaround/comments | + |--|--|--|--| + | 1 | | | | +<--> ++The following table provides a summary of known issues carried over from the previous releases. ++ |No. |Feature | Issue | Workaround/comments | + |--|--|--|--| + | 1 | Local distributed tracing | When a web proxy is enabled on the Azure Stack Edge appliance that the packet core is running on and Azure Active Directory is used to authenticate access to AP5GC Local Dashboards, the traffic to Azure Active Directory doesn't transmit via the web proxy. If there's a firewall blocking traffic that doesn't go via the web proxy then enabling Azure Active Directory causes the packet core install to fail. | Disable Azure Active Directory and use password based authentication to authenticate access to AP5GC Local Dashboards instead. | ++## Next steps ++- [Upgrade the packet core instance in a site - Azure portal](upgrade-packet-core-azure-portal.md) +- [Upgrade the packet core instance in a site - ARM template](upgrade-packet-core-arm-template.md) |
| private-5g-core | Azure Stack Edge Packet Core Compatibility | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/private-5g-core/azure-stack-edge-packet-core-compatibility.md | The following table provides information on which versions of the ASE device are | Packet core version | ASE Pro GPU compatible versions | ASE Pro 2 compatible versions | |--|--|--|-! 2310 | 2309, 2312 | 2309, 2312 | +| 2403 | 2403, 2405 | 2403, 2405 | +| 2310 | 2309, 2312, 2403 | 2309, 2312, 2403 | | 2308 | 2303, 2309 | 2303, 2309 | | 2307 | 2303 | 2303 | | 2306 | 2303 | 2303 | |
| private-5g-core | Support Lifetime | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/private-5g-core/support-lifetime.md | The following table shows the support status for different Packet Core releases | Release | Support Status | ||-|-| AP5GC 2310 | Supported until AP5GC 2403 is released | -| AP5GC 2308 | Supported until AP5GC 2401 is released | -| AP5GC 2307 and earlier | Out of Support | +| AP5GC 2403 | Supported until AP5GC 2407 is released | +| AP5GC 2310 | Supported until AP5GC 2404 is released | +| AP5GC 2308 and earlier | Out of Support | |
| private-5g-core | Whats New | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/private-5g-core/whats-new.md | Last updated 12/21/2023 To help you stay up to date with the latest developments, this article covers: -- New features, improvements and fixes for the online service.+- New features, improvements, and fixes for the online service. - New releases for the packet core, referencing the packet core release notes for further information. This page is updated regularly with the latest developments in Azure Private 5G Core. +## April 2024 ++### TCP Maximum Segment Size (MSS) Clamping ++**Type:** New feature ++**Date available:** April 04, 2024 ++TCP session initial setup messages that include a Maximum Segment Size (MSS) value, which controls the size limit of packets transmitted during the session. The packet core will now automatically set this value, where necessary, to ensure packets aren't too large for the core to transmit. This reduces packet loss due to oversized packets arriving at the core's interfaces, and reduces the need for fragmentation and reassembly, which are costly procedures. + ## March 2024+ ### Azure Policy support **Type:** New feature See [Azure Policy policy definitions for Azure Private 5G Core](azure-policy-ref **Date available:** March 22, 2024 -The SUPI (subscription permanent identifier) secret needs to be encrypted before being transmitted over the radio network as a SUCI (subscription concealed identifier). The concealment is performed by the UEs on registration, and deconcealment is performed by the packet core. You can now securely manage the required private keys through the Azure Portal and provision SIMs with public keys. +The SUPI (subscription permanent identifier) secret needs to be encrypted before being transmitted over the radio network as a SUCI (subscription concealed identifier). The concealment is performed by the UEs on registration, and deconcealment is performed by the packet core. You can now securely manage the required private keys through the Azure portal and provision SIMs with public keys. For more information, see [Enable SUPI concealment](supi-concealment.md). |
| private-multi-access-edge-compute-mec | Affirmed Private Network Service Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/private-multi-access-edge-compute-mec/affirmed-private-network-service-overview.md | - Title: 'What is Affirmed Private Network Service on Azure?' -description: Learn about Affirmed Private Network Service solutions on Azure for private LTE/5G networks. ---- Previously updated : 06/16/2021----# What is Affirmed Private Network Service on Azure? --The Affirmed Private Network Service (APNS) is a managed network service offering created for managed service providers and mobile network operators to provide private LTE and private 5G solutions to enterprises. --Affirmed has combined its mobile core-technology with AzureΓÇÖs capabilities to create a complete turnkey solution for private LTE/5G networks to help carriers and enterprises take advantage of managed networks and the mobile edge. The combination of cloud management and automation allows managed service providers to deliver a fully managed infrastructure and also brings a complete end-to-end solution for operators to pick the best of breed Radio Access Network, SIM, and Azure services from a rich ecosystem of partners offered in Azure Marketplace. The solution is composed of five components: --- **Cloud-native Mobile Core**: This component is 3GPP standards compliant and supports network functions for both 4G and 5G and has virtual network probes located natively within the mobile core. The mobile core can be deployed on VMs, physical servers, or on an operator's cloud, eliminating the need for dedicated hardware.--- **Private Network Service Manager - Affirmed Networks**: Private Network Service Manager is the application that operators use to deploy, monitor, and manage private mobile core networks on the Azure platform. It features a complete set of management capabilities including simple self-activation and management of private network resources through a programmatic GUI-driven portal.--- **Azure Network Functions Manager**: Azure Network Functions Manager (NFM) is a fully managed cloud-native orchestration service that enables customers to deploy and provision network functions on Azure Stack Edge Pro with GPU for a consistent hybrid experience using the Azure portal.--- **Azure Cloud**: A public cloud computing platform with solutions including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) that can be used for services such as analytics, virtual computing, storage, networking, and much more.--- **Azure Stack Edge**: A cloud-managed, hardware-as-a-service solution shipped by Microsoft. It brings the Azure cloudΓÇÖs power to a local and robust server that can be deployed virtually anywhere local AI and advanced computing tasks need to be performed.----## Why use the Affirmed Private Network Solution? -APNS provides the following key benefits to operators and their customers: --- **Deployment Flexibility** - APNS employs Control and User Plane Separation technology and supports three types of deployment modes to address a variety of operator desired scenarios for offering to enterprises. By using the Private Network Service Manager, operators can configure the following deployment models:-- - Standalone enables operators to provide a complete standalone private network on premises by delivering the RAN, 5G core on the Azure Stack Edge and the management layer on the centralized cloud. -- - Distributed enables faster processing of data by distributing the user plane closer to the edge of the enterprise on the Azure Stack Edge while the control plane is on the cloud; an example of such a model would be manufacturing facilities. -- - All in Cloud allows for the entire 5G core to be deployed on the cloud while the RAN is on the edge, enabling dynamic allocation of cloud resources to suit the changing demands of the workloads. --- **MNO Integration** - APNS is mobile network operator integrated, which means it provides complete mobility across private and public operator networks with its distributed subscriber core. Operators have the advantage to scale the private mobile network to 1000s of enterprise edge sites.-- - Supports all Spectrum options - MNO Licensed, Private Licensed, CBRS, Shared, Unlicensed. -- - Supports isolated/standalone private networks, multi-site roaming, and macro roaming as it is MNO Integrated. -- - Can provide 99.999% service availability and inter-work with any 3GPP compliant LTE and 5G NR radio. Has Carrier-Grade resiliency for enterprises. --- **Automation and Ease of Management** - The APNS solution can be completely managed remotely through Service Manager on the Azure cloud. Through the Service Manager, end-users have access to their personalized dashboard and can manage, view, and turn on/off devices on the private mobile network. Operators can monitor the status of the networks for network issues and key parameters to ensure optimal performance.-- - Provides secure, reliable, high bandwidth, low latency private mobile networking service that runs on Azure private multi-access edge compute. -- - Supports complete remote management, without needing truck rolls. -- - Provides cloud automation to enable operators to offer managed services to enterprises or to partner with MSPs who in turn can offer managed services. --- **Smarter Network & Business Insights** - Affirmed mobile core has an embedded virtual probe/ packet brokering function that can be used to provide network insight. The operator can use these insights to better drive network decisions while their customers can use these insights to drive smarter monetization decisions.--- **Data Privacy & Security** - APNS uses Azure to deliver security and compliance across private networks and enterprise applications. Operators can confidently deploy the solution for industry use cases that require stringent data privacy laws, such as healthcare, government, public safety, and defense.--## Next steps -- Learn how to [deploy the Affirmed private Network Service solution](deploy-affirmed-private-network-service-solution.md)--- |
| private-multi-access-edge-compute-mec | Deploy Affirmed Private Network Service Solution | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/private-multi-access-edge-compute-mec/deploy-affirmed-private-network-service-solution.md | - Title: 'Deploy Affirmed Private Network Service on Azure' -description: Learn how to deploy the Affirmed Private Network Service solution on Azure ---- Previously updated : 06/16/2021---# Deploy Affirmed Private Network Service on Azure --This article provides a high-level overview of the process of deploying Affirmed Private Network Service (APNS) solution on an Azure Stack Edge device via the Microsoft Azure Marketplace. --The following diagram shows the system architecture of the Affirmed Private Network Service, including the resources required to deploy. -- --## Collect required information --To deploy APNS, you must have the following resources: --- A configured Azure Network Function Manager - Device object which serves as the digital twin of the Azure Stack Edge device. --- A fully deployed Azure Stack Edge with NetFoundry VM. --- Subscription approval for the Affirmed Management Systems VM Offer and APNS Managed Application. --- An Azure account with an active subscription and access to the following: -- - The built-in **Owner** Role for your resource group. -- - The built-in **Managed Application Contributor** role for your subscription. -- - A virtual network and subnet to join (open ports tcp/443 and tcp/8443). -- - 5 IP addresses on the virtual subnet. -- - A valid SAS Token provided by Affirmed Release Engineering. -- - An administrative username/password to program during the deployment. - -## Deploy APNS --To automatically deploy the APNS Managed application with all required resources and relevant information necessary, select the APNS Managed Application from the Microsoft Azure Marketplace. When you deploy APNS, all the required resources are automatically created for you and are contained in a Managed Resource Group. --Complete the following procedure to deploy APNS: -1. Open the Azure portal and select **Create a resource**. -2. Enter *APNS* in the search bar and press Enter. -3. Select **View Private Offers**. - > [!NOTE] - > The APNS Managed application will not appear until **View Private Offers** is selected. -4. Select **Create** from the dropdown menu of the **Private Offer**, then select the option to deploy. -5. Complete the application setup, network settings, and review and create. -6. Select **Deploy**. --## Next steps --- For information about Affirmed Private Network Service, see [What is Affirmed Private Network Service on Azure?](affirmed-private-network-service-overview.md). |
| role-based-access-control | Built In Roles | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/role-based-access-control/built-in-roles.md | The following table provides a brief description of each built-in role. Click th > | <a name='reservations-administrator'></a>[Reservations Administrator](./built-in-roles/management-and-governance.md#reservations-administrator) | Lets one read and manage all the reservations in a tenant | a8889054-8d42-49c9-bc1c-52486c10e7cd | > | <a name='reservations-reader'></a>[Reservations Reader](./built-in-roles/management-and-governance.md#reservations-reader) | Lets one read all the reservations in a tenant | 582fc458-8989-419f-a480-75249bc5db7e | > | <a name='resource-policy-contributor'></a>[Resource Policy Contributor](./built-in-roles/management-and-governance.md#resource-policy-contributor) | Users with rights to create/modify resource policy, create support ticket and read resources/hierarchy. | 36243c78-bf99-498c-9df9-86d9f8d28608 |+> | <a name='scheduled-patching-contributor'></a>[Scheduled Patching Contributor](./built-in-roles/management-and-governance.md#scheduled-patching-contributor) | Provides access to manage maintenance configurations with maintenance scope InGuestPatch and corresponding configuration assignments | cd08ab90-6b14-449c-ad9a-8f8e549482c6 | > | <a name='site-recovery-contributor'></a>[Site Recovery Contributor](./built-in-roles/management-and-governance.md#site-recovery-contributor) | Lets you manage Site Recovery service except vault creation and role assignment | 6670b86e-a3f7-4917-ac9b-5d6ab1be4567 | > | <a name='site-recovery-operator'></a>[Site Recovery Operator](./built-in-roles/management-and-governance.md#site-recovery-operator) | Lets you failover and failback but not perform other Site Recovery management operations | 494ae006-db33-4328-bf46-533a6560a3ca | > | <a name='site-recovery-reader'></a>[Site Recovery Reader](./built-in-roles/management-and-governance.md#site-recovery-reader) | Lets you view Site Recovery status but not perform other management operations | dbaa88c4-0c30-4179-9fb3-46319faa6149 | |
| role-based-access-control | Management And Governance | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/role-based-access-control/built-in-roles/management-and-governance.md | Users with rights to create/modify resource policy, create support ticket and re } ``` +## Scheduled Patching Contributor ++Provides access to manage maintenance configurations with maintenance scope InGuestPatch and corresponding configuration assignments ++[Learn more](/azure/update-manager/scheduled-patching) ++> [!div class="mx-tableFixed"] +> | Actions | Description | +> | | | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/maintenanceConfigurations/read | Read maintenance configuration. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/maintenanceConfigurations/write | Create or update maintenance configuration. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/maintenanceConfigurations/delete | Delete maintenance configuration. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/configurationAssignments/read | Read maintenance configuration assignment. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/configurationAssignments/write | Create or update maintenance configuration assignment. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/configurationAssignments/delete | Delete maintenance configuration assignment. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/configurationAssignments/maintenanceScope/InGuestPatch/read | Read maintenance configuration assignment for InGuestPatch maintenance scope. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/configurationAssignments/maintenanceScope/InGuestPatch/write | Create or update a maintenance configuration assignment for InGuestPatch maintenance scope. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/configurationAssignments/maintenanceScope/InGuestPatch/delete | Delete maintenance configuration assignment for InGuestPatch maintenance scope. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/maintenanceConfigurations/maintenanceScope/InGuestPatch/read | Read maintenance configuration for InGuestPatch maintenance scope. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/maintenanceConfigurations/maintenanceScope/InGuestPatch/write | Create or update a maintenance configuration for InGuestPatch maintenance scope. | +> | [Microsoft.Maintenance](../permissions/management-and-governance.md#microsoftmaintenance)/maintenanceConfigurations/maintenanceScope/InGuestPatch/delete | Delete maintenance configuration for InGuestPatch maintenance scope. | +> | **NotActions** | | +> | *none* | | +> | **DataActions** | | +> | *none* | | +> | **NotDataActions** | | +> | *none* | | ++```json +{ + "assignableScopes": [ + "/" + ], + "description": "Provides access to manage maintenance configurations with maintenance scope InGuestPatch and corresponding configuration assignments", + "id": "/providers/Microsoft.Authorization/roleDefinitions/cd08ab90-6b14-449c-ad9a-8f8e549482c6", + "name": "cd08ab90-6b14-449c-ad9a-8f8e549482c6", + "permissions": [ + { + "actions": [ + "Microsoft.Maintenance/maintenanceConfigurations/read", + "Microsoft.Maintenance/maintenanceConfigurations/write", + "Microsoft.Maintenance/maintenanceConfigurations/delete", + "Microsoft.Maintenance/configurationAssignments/read", + "Microsoft.Maintenance/configurationAssignments/write", + "Microsoft.Maintenance/configurationAssignments/delete", + "Microsoft.Maintenance/configurationAssignments/maintenanceScope/InGuestPatch/read", + "Microsoft.Maintenance/configurationAssignments/maintenanceScope/InGuestPatch/write", + "Microsoft.Maintenance/configurationAssignments/maintenanceScope/InGuestPatch/delete", + "Microsoft.Maintenance/maintenanceConfigurations/maintenanceScope/InGuestPatch/read", + "Microsoft.Maintenance/maintenanceConfigurations/maintenanceScope/InGuestPatch/write", + "Microsoft.Maintenance/maintenanceConfigurations/maintenanceScope/InGuestPatch/delete" + ], + "notActions": [], + "dataActions": [], + "notDataActions": [] + } + ], + "roleName": "Scheduled Patching Contributor", + "roleType": "BuiltInRole", + "type": "Microsoft.Authorization/roleDefinitions" +} +``` + ## Site Recovery Contributor Lets you manage Site Recovery service except vault creation and role assignment |
| role-based-access-control | Management And Governance | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/role-based-access-control/permissions/management-and-governance.md | Azure service: Microsoft Monitoring Insights > | Microsoft.Intune/diagnosticsettings/delete | Deleting a diagnostic setting | > | Microsoft.Intune/diagnosticsettingscategories/read | Reading a diagnostic setting categories | +## Microsoft.Maintenance ++Azure service: [Azure Maintenance](/azure/virtual-machines/maintenance-configurations), [Azure Update Manager](/azure/update-manager/overview) ++> [!div class="mx-tableFixed"] +> | Action | Description | +> | | | +> | Microsoft.Maintenance/applyUpdates/write | Write apply updates to a resource. | +> | Microsoft.Maintenance/applyUpdates/read | Read apply updates to a resource. | +> | Microsoft.Maintenance/configurationAssignments/write | Create or update maintenance configuration assignment. | +> | Microsoft.Maintenance/configurationAssignments/read | Read maintenance configuration assignment. | +> | Microsoft.Maintenance/configurationAssignments/delete | Delete maintenance configuration assignment. | +> | Microsoft.Maintenance/configurationAssignments/maintenanceScope/InGuestPatch/write | Create or update a maintenance configuration assignment for InGuestPatch maintenance scope. | +> | Microsoft.Maintenance/configurationAssignments/maintenanceScope/InGuestPatch/read | Read maintenance configuration assignment for InGuestPatch maintenance scope. | +> | Microsoft.Maintenance/configurationAssignments/maintenanceScope/InGuestPatch/delete | Delete maintenance configuration assignment for InGuestPatch maintenance scope. | +> | Microsoft.Maintenance/maintenanceConfigurations/write | Create or update maintenance configuration. | +> | Microsoft.Maintenance/maintenanceConfigurations/read | Read maintenance configuration. | +> | Microsoft.Maintenance/maintenanceConfigurations/delete | Delete maintenance configuration. | +> | Microsoft.Maintenance/maintenanceConfigurations/eventGridFilters/delete | Notifies Microsoft.Maintenance that an EventGrid Subscription for Maintenance Configuration is being deleted. | +> | Microsoft.Maintenance/maintenanceConfigurations/eventGridFilters/read | Notifies Microsoft.Maintenance that an EventGrid Subscription for Maintenance Configuration is being viewed. | +> | Microsoft.Maintenance/maintenanceConfigurations/eventGridFilters/write | Notifies Microsoft.Maintenance that a new EventGrid Subscription for Maintenance Configuration is being created. | +> | Microsoft.Maintenance/maintenanceConfigurations/maintenanceScope/InGuestPatch/write | Create or update a maintenance configuration for InGuestPatch maintenance scope. | +> | Microsoft.Maintenance/maintenanceConfigurations/maintenanceScope/InGuestPatch/read | Read maintenance configuration for InGuestPatch maintenance scope. | +> | Microsoft.Maintenance/maintenanceConfigurations/maintenanceScope/InGuestPatch/delete | Delete maintenance configuration for InGuestPatch maintenance scope. | +> | Microsoft.Maintenance/updates/read | Read updates to a resource. | + ## Microsoft.ManagedServices Azure service: [Azure Lighthouse](/azure/lighthouse/) |
| role-based-access-control | Resource Provider Operations | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/role-based-access-control/resource-provider-operations.md | Click the resource provider name in the following list to see the list of permis > | [Microsoft.Features](./permissions/management-and-governance.md#microsoftfeatures) | | [Azure Resource Manager](/azure/azure-resource-manager/) | > | [Microsoft.GuestConfiguration](./permissions/management-and-governance.md#microsoftguestconfiguration) | Audit settings inside a machine using Azure Policy. | [Azure Policy](/azure/governance/policy/) | > | [Microsoft.Intune](./permissions/management-and-governance.md#microsoftintune) | Enable your workforce to be productive on all their devices, while keeping your organization's information protected. | |+> | [Microsoft.Maintenance](./permissions/management-and-governance.md#microsoftmaintenance) | | [Azure Maintenance](/azure/virtual-machines/maintenance-configurations)<br/>[Azure Update Manager](/azure/update-manager/overview) | > | [Microsoft.ManagedServices](./permissions/management-and-governance.md#microsoftmanagedservices) | | [Azure Lighthouse](/azure/lighthouse/) | > | [Microsoft.Management](./permissions/management-and-governance.md#microsoftmanagement) | Use management groups to efficiently apply governance controls and manage groups of Azure subscriptions. | [Management Groups](/azure/governance/management-groups/) | > | [Microsoft.PolicyInsights](./permissions/management-and-governance.md#microsoftpolicyinsights) | Summarize policy states for the subscription level policy definition. | [Azure Policy](/azure/governance/policy/) | |
| role-based-access-control | Transfer Subscription | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/role-based-access-control/transfer-subscription.md | Several Azure resources have a dependency on a subscription or a directory. Depe > This section lists the known Azure services or resources that depend on your subscription. Because resource types in Azure are constantly evolving, there might be additional dependencies not listed here that can cause a breaking change to your environment. | Service or resource | Impacted | Recoverable | Are you impacted? | What you can do |-| | | | | | +| | :: | :: | | | | Role assignments | Yes | Yes | [List role assignments](#save-all-role-assignments) | All role assignments are permanently deleted. You must map users, groups, and service principals to corresponding objects in the target directory. You must re-create the role assignments. | | Custom roles | Yes | Yes | [List custom roles](#save-custom-roles) | All custom roles are permanently deleted. You must re-create the custom roles and any role assignments. | | System-assigned managed identities | Yes | Yes | [List managed identities](#list-role-assignments-for-managed-identities) | You must disable and re-enable the managed identities. You must re-create the role assignments. | Several Azure resources have a dependency on a subscription or a directory. Depe | Azure Service Fabric | Yes | No | | You must re-create the cluster. For more information, see [SF Clusters FAQ](../service-fabric/service-fabric-common-questions.md) or [SF Managed Clusters FAQ](../service-fabric/faq-managed-cluster.yml) | | Azure Service Bus | Yes | Yes | |You must delete, re-create, and attach the managed identities to the appropriate resource. You must re-create the role assignments. | | Azure Synapse Analytics Workspace | Yes | Yes | | You must update the tenant ID associated with the Synapse Analytics Workspace. If the workspace is associated with a Git repository, you must update the [workspace's Git configuration](../synapse-analytics/cicd/source-control.md#switch-to-a-different-git-repository). For more information, see [Recovering Synapse Analytics workspace after transferring a subscription to a different Microsoft Entra directory (tenant)](../synapse-analytics/how-to-recover-workspace-after-tenant-move.md). |+| Azure Databricks | Yes | No | | Currently, Azure Databricks does not support moving workspaces to a new tenant. For more information, see [Manage your Azure Databricks account](/azure/databricks/administration-guide/account-settings/#move-workspace-between-tenants-unsupported). | > [!WARNING] > If you are using encryption at rest for a resource, such as a storage account or SQL database, that has a dependency on a key vault that is being transferred, it can lead to an unrecoverable scenario. If you have this situation, you should take steps to use a different key vault or temporarily disable customer-managed keys to avoid this unrecoverable scenario. |
| search | Cognitive Search Skill Ocr | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/search/cognitive-search-skill-ocr.md | The **Optical character recognition (OCR)** skill recognizes printed and handwri An OCR skill uses the machine learning models provided by [Azure AI Vision](../ai-services/computer-vision/overview.md) API [v3.2](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2/operations/5d986960601faab4bf452005) in Azure AI services. The **OCR** skill maps to the following functionality: + For the languages listed under [Azure AI Vision language support](../ai-services/computer-vision/language-support.md#optical-character-recognition-ocr), the [Read API](../ai-services/computer-vision/overview-ocr.md) is used.-+ For Greek and Serbian Cyrillic, the [legacy OCR](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2/operations/56f91f2e778daf14a499f20d) API is used. +++ For Greek and Serbian Cyrillic, the legacy [OCR in version 3.2](https://github.com/Azure/azure-rest-api-specs/tree/master/specification/cognitiveservices/data-plane/ComputerVision/stable/v3.2) API is used. The **OCR** skill extracts text from image files. Supported file formats include: Parameters are case-sensitive. | Parameter name | Description | |--|-|-| `detectOrientation` | Detects image orientation. Valid values are `true` or `false`. </p>This parameter only applies if the [legacy OCR](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2/operations/56f91f2e778daf14a499f20d) API is used. | +| `detectOrientation` | Detects image orientation. Valid values are `true` or `false`. </p>This parameter only applies if the [legacy OCR version 3.2](https://github.com/Azure/azure-rest-api-specs/tree/master/specification/cognitiveservices/data-plane/ComputerVision/stable/v3.2) API is used. | | `defaultLanguageCode` | Language code of the input text. Supported languages include all of the [generally available languages](../ai-services/computer-vision/language-support.md#analyze-image) of Azure AI Vision. You can also specify `unk` (Unknown). </p>If the language code is unspecified or null, the language is set to English. If the language is explicitly set to `unk`, all languages found are auto-detected and returned.| | `lineEnding` | The value to use as a line separator. Possible values: "Space", "CarriageReturn", "LineFeed". The default is "Space". | The above skillset example assumes that a normalized-images field exists. To gen } ``` -- ## See also + [What is optical character recognition](../ai-services/computer-vision/overview-ocr.md) |
| search | Search Indexer Howto Access Private | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/search/search-indexer-howto-access-private.md | Shared private link is a premium feature that's billed by usage. When you set up Azure AI Search makes outbound calls to other Azure PaaS resources in the following scenarios: -+ Indexer connection requests to supported data sources -+ Indexer (skillset) connections to Azure Storage for caching enrichments or writing to a knowledge store ++ Indexer or search engine connects to Azure OpenAI for text-to-vector embeddings++ Indexer connects to supported data sources++ Indexer (skillset) connections to Azure Storage for caching enrichments, debug session sate, or writing to a knowledge store + Encryption key requests to Azure Key Vault + Custom skill requests to Azure Functions or similar resource -In service-to-service communications, Azure AI Search typically sends a request over a public internet connection. However, if your data, key vault, or function should be accessed through a [private endpoint](../private-link/private-endpoint-overview.md), you must create a *shared private link*. +Shared private links only work for Azure-to-Azure connections. If you're connecting to OpenAI or another external model, the connection must be over the public internet. ++Shared private links are for operations and data accessed through a [private endpoint](../private-link/private-endpoint-overview.md) for Azure resources or clients that run in an Azure virtual network. A shared private link is: There are two scenarios for using [Azure Private Link](../private-link/private-l + Scenario two: [configure search for a private *inbound* connection](service-create-private-endpoint.md) from clients that run in a virtual network. +Scenario one is covered in this article. + While both scenarios have a dependency on Azure Private Link, they are independent. You can create a shared private link without having to configure your own search service for a private endpoint. ### Limitations When evaluating shared private links for your scenario, remember these constraints. -+ Several of the resource types used in a shared private link are in preview. If you're connecting to a preview resource (Azure Database for MySQL, Azure Functions, or Azure SQL Managed Instance), use a preview version of the Management REST API to create the shared private link. These versions include `2020-08-01-preview` or `2021-04-01-preview`. ++ Several of the resource types used in a shared private link are in preview. If you're connecting to a preview resource (Azure Database for MySQL, Azure Functions, or Azure SQL Managed Instance), use a preview version of the Management REST API to create the shared private link. These versions include `2020-08-01-preview`, `2021-04-01-preview`, and `2024-03-01-preview`. + Indexer execution must use the private execution environment that's specific to your search service. Private endpoint connections aren't supported from the multitenant environment. The configuration setting for this requirement is covered in this article. When evaluating shared private links for your scenario, remember these constrain + An Azure AI Search at the Basic tier or higher. If you're using [AI enrichment](cognitive-search-concept-intro.md) and skillsets, the tier must be Standard 2 (S2) or higher. See [Service limits](search-limits-quotas-capacity.md#shared-private-link-resource-limits) for details. -+ An Azure PaaS resource from the following list of supported resource types, configured to run in a virtual network. ++ An Azure PaaS resource from the following list of [supported resource types](#supported-resource-types), configured to run in a virtual network.+ + Permissions on both Azure AI Search and the data source: A `202 Accepted` response is returned on success. The process of creating an out ## 2 - Approve the private endpoint connection -Approval of the private endpoint connection is granted on the Azure PaaS side. If the service consumer has a role assignment on the service provider resource, the approval will be automatic. Otherwise, manual approval is required. For details, see [Manage Azure private endpoints](/azure/private-link/manage-private-endpoint). +Approval of the private endpoint connection is granted on the Azure PaaS side. Explicit approval by the resource owner is required. The following steps cover approval using the Azure portal, but here are some links to approve the connection programmatically from the Azure PaaS side: +++ On Azure Storage, use [Private Endpoint Connections - Put](/rest/api/storagerp/private-endpoint-connections/put)++ On Azure Cosmos DB, use [Private Endpoint Connections - Create Or Update](/rest/api/cosmos-db-resource-provider/private-endpoint-connections/create-or-update) -This section assumes manual approval and the portal for this step, but you can also use the REST APIs of the Azure PaaS resource. [Private Endpoint Connections (Storage Resource Provider)](/rest/api/storagerp/privateendpointconnections) and [Private Endpoint Connections (Cosmos DB Resource Provider)](/rest/api/cosmos-db-resource-provider/2023-03-15/private-endpoint-connections) are two examples. +Using the Azure portal, perform the following steps: -1. In the Azure portal, open the **Networking** page of the Azure PaaS resource.[text](https://ms.portal.azure.com/#blade%2FHubsExtension%2FResourceMenuBlade%2Fid%2F%2Fsubscriptions%2Fa5b1ca8b-bab3-4c26-aebe-4cf7ec4791a0%2FresourceGroups%2Ftest-private-endpoint%2Fproviders%2FMicrosoft.Network%2FprivateEndpoints%2Ftest-private-endpoint) +1. Open the **Networking** page of the Azure PaaS resource.[text](https://ms.portal.azure.com/#blade%2FHubsExtension%2FResourceMenuBlade%2Fid%2F%2Fsubscriptions%2Fa5b1ca8b-bab3-4c26-aebe-4cf7ec4791a0%2FresourceGroups%2Ftest-private-endpoint%2Fproviders%2FMicrosoft.Network%2FprivateEndpoints%2Ftest-private-endpoint) 1. Find the section that lists the private endpoint connections. The following example is for a storage account. |
| search | Search Manage Azure Cli | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/search/search-manage-azure-cli.md | Last updated 02/21/2024 > * [Azure CLI](search-manage-azure-cli.md) > * [REST API](search-manage-rest.md) -You can run Azure CLI commands and scripts on Windows, macOS, Linux, or in [Azure Cloud Shell](../cloud-shell/overview.md) to create and configure Azure AI Search. The [**az search**](/cli/azure/search) module extends the [Azure CLI](/cli/) with full parity to the [Search Management REST APIs](/rest/api/searchmanagement) and the ability to perform the following tasks: +You can run Azure CLI commands and scripts on Windows, macOS, Linux, or in Azure Cloud Shell to create and configure Azure AI Search. ++Use the [**az search module**](/cli/azure/search) to perform the following tasks: > [!div class="checklist"]-> * [List search services in a subscription](#list-search-services) +> * [List search services in a subscription](#list-services-in-a-subscription) > * [Return service information](#get-search-service-information) > * [Create or delete a service](#create-or-delete-a-service) > * [Create a service with a private endpoint](#create-a-service-with-a-private-endpoint) Preview administration features are typically not available in the **az search** Azure CLI versions are [listed on GitHub](https://github.com/Azure/azure-cli/releases). -<a name="list-search-services"></a> +The [**az search**](/cli/azure/search) module extends the [Azure CLI](/cli/) with full parity to the stable versions of the [Search Management REST APIs](/rest/api/searchmanagement). ## List services in a subscription |
| search | Search Manage Powershell | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/search/search-manage-powershell.md | Title: PowerShell scripts using `Az.Search` module + Title: PowerShell scripts using Azure Search PowerShell module description: Create and configure an Azure AI Search service with PowerShell. You can scale a service up or down, manage admin and query api-keys, and query for system information. -You can run PowerShell cmdlets and scripts on Windows, Linux, or in [Azure Cloud Shell](../cloud-shell/overview.md) to create and configure Azure AI Search. The **Az.Search** module extends [Azure PowerShell](/powershell/) with full parity to the [Search Management REST APIs](/rest/api/searchmanagement) and the ability to perform the following tasks: +You can run PowerShell cmdlets and scripts on Windows, Linux, or in Azure Cloud Shell to create and configure Azure AI Search. ++Use the [**Az.Search** module](/powershell/module/az.search/) to perform the following tasks: > [!div class="checklist"] > * [List search services in a subscription](#list-search-services) You can't use tools or APIs to transfer content, such as an index, from one serv Preview administration features are typically not available in the **Az.Search** module. If you want to use a preview feature, [use the Management REST API](search-manage-rest.md) and a preview API version. +The [**Az.Search** module](/powershell/module/az.search/) extends [Azure PowerShell](/powershell/) with full parity to the stable versions of the [Search Management REST APIs](/rest/api/searchmanagement). + <a name="check-versions-and-load"></a> ## Check versions and load modules |
| search | Service Create Private Endpoint | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/search/service-create-private-endpoint.md | -In this article, learn how to secure an Azure AI Search service so that it can't be accessed over a public internet connection: +In this article, learn how to configure a private connection to Azure AI Search so that it admits requests from clients in a virtual network instead of over a public internet connection: + [Create an Azure virtual network](#create-the-virtual-network) (or use an existing one) + [Configure a search service to use a private endpoint](#create-a-search-service-with-a-private-endpoint) + [Create an Azure virtual machine in the same virtual network](#create-a-virtual-machine) + [Test using a browser session on the virtual machine](#connect-to-the-vm) +Other Azure resources that might privately connect to Azure AI Search include Azure OpenAI for "use your own data" scenarios. Azure OpenAI Studio doesn't run in a virtual network, but it can be configured on the backend to send requests over the Microsoft backbone network. Configuration for this traffic pattern is enabled by Microsoft when your request is submitted and approved. For this scenario: +++ Follow the instructions in this article to set up the private endpoint.++ [Submit a request](/azure/ai-services/openai/how-to/use-your-data-securely#disable-public-network-access-1) for Azure OpenAI Studio to connect using your private endpoint.++ Optionally, [disable public network access](#disable-public-network-access) if connections should only originate from clients in virtual network or from Azure OpenAI over a private endpoint connection.++## Key points about private endpoints + Private endpoints are provided by [Azure Private Link](../private-link/private-link-overview.md), as a separate billable service. For more information about costs, see the [pricing page](https://azure.microsoft.com/pricing/details/private-link/). -You can create a private endpoint for a search service in the Azure portal, as described in this article. Alternatively, you can use the [Management REST API version](/rest/api/searchmanagement/), [Azure PowerShell](/powershell/module/az.search), or [Azure CLI](/cli/azure/search). +Once a search service has a private endpoint, portal access to that service must be initiated from a browser session on a virtual machine inside the virtual network. See [this step](#portal-access-private-search-service) for details. -> [!NOTE] -> Once a search service has a private endpoint, portal access to that service must be initiated from a browser session on a virtual machine inside the virtual network. See [this step](#portal-access-private-search-service) for details. +You can create a private endpoint for a search service in the Azure portal, as described in this article. Alternatively, you can use the [Management REST API version](/rest/api/searchmanagement/), [Azure PowerShell](/powershell/module/az.search), or [Azure CLI](/cli/azure/search). -## Why use a Private Endpoint for secure access? +## Why use a private endpoint? [Private Endpoints](../private-link/private-endpoint-overview.md) for Azure AI Search allow a client on a virtual network to securely access data in a search index over a [Private Link](../private-link/private-link-overview.md). The private endpoint uses an IP address from the [virtual network address space](../virtual-network/ip-services/private-ip-addresses.md) for your search service. Network traffic between the client and the search service traverses over the virtual network and a private link on the Microsoft backbone network, eliminating exposure from the public internet. For a list of other PaaS services that support Private Link, check the [availability section](../private-link/private-link-overview.md#availability) in the product documentation. To work around this restriction, connect to Azure portal from a browser on a vir 1. On a virtual machine in your virtual network, open a browser and sign in to the Azure portal. The portal will use the private endpoint attached to the virtual machine to connect to your search service. +## Disable public network access ++You can lock down a search service to prevent it from admitting any request from the public internet. You can use the Azure portal for this step. ++1. In the Azure portal, on the leftmost pane of your search service page, select **Networking**. ++1. Select **Disabled** on the **Firewalls and virtual networks** tab. ++You can also use the [Azure CLI](/cli/azure/search/service?view=azure-cli-latest#az-search-service-update&preserve-view=true), [Azure PowerShell](/powershell/module/az.search/set-azsearchservice), or the [Management REST API](/rest/api/searchmanagement/services/update), setting `public-access` or `public-network-access` to `disabled`. + ## Clean up resources When you're working in your own subscription, it's a good idea at the end of a project to identify whether you still need the resources you created. Resources left running can cost you money. |
| security | Threat Modeling Tool Authentication | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/security/develop/threat-modeling-tool-authentication.md | MSAL also maintains a token cache and refreshes tokens for you when they're clos | **SDL Phase** | Build | | **Applicable Technologies** | Generic, C#, Node.JS, | | **Attributes** | N/A, Gateway choice - Azure IoT Hub |-| **References** | N/A, [Azure IoT hub with .NET](../../iot-develop/quickstart-send-telemetry-iot-hub.md?pivots=programming-language-csharp), [Getting Started with IoT hub and Node JS](../../iot-develop/quickstart-send-telemetry-iot-hub.md?pivots=programming-language-nodejs), [Securing IoT with SAS and certificates](../../iot-hub/iot-hub-dev-guide-sas.md), [Git repository](https://github.com/Azure/azure-iot-sdks/) | +| **References** | N/A, [Azure IoT hub with .NET](../../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-csharp), [Getting Started with IoT hub and Node JS](../../iot/tutorial-send-telemetry-iot-hub.md?pivots=programming-language-nodejs), [Securing IoT with SAS and certificates](../../iot-hub/iot-hub-dev-guide-sas.md), [Git repository](https://github.com/Azure/azure-iot-sdks/) | | **Steps** | <ul><li>**Generic:** Authenticate the device using Transport Layer Security (TLS) or IPSec. Infrastructure should support using pre-shared key (PSK) on those devices that cannot handle full asymmetric cryptography. Leverage Microsoft Entra ID, Oauth.</li><li>**C#:** When creating a DeviceClient instance, by default, the Create method creates a DeviceClient instance that uses the AMQP protocol to communicate with IoT Hub. To use the HTTPS protocol, use the override of the Create method that enables you to specify the protocol. If you use the HTTPS protocol, you should also add the `Microsoft.AspNet.WebApi.Client` NuGet package to your project to include the `System.Net.Http.Formatting` namespace.</li></ul>| ### Example |
| security | Threat Modeling Tool Releases 73209279 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/security/develop/threat-modeling-tool-releases-73209279.md | Title: Microsoft Threat Modeling Tool release 09/27/2022 - Azure description: Documenting the release notes for the threat modeling tool release 7.3.20927.9.--++ Last updated 09/27/2022 |
| security | Threat Modeling Tool Releases 73211082 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/security/develop/threat-modeling-tool-releases-73211082.md | Title: Microsoft Threat Modeling Tool release 11/08/2022 - Azure description: Documenting the release notes for the threat modeling tool release 7.3.21108.2.--++ Last updated 11/08/2022 |
| security | Threat Modeling Tool Releases 73306305 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/security/develop/threat-modeling-tool-releases-73306305.md | Title: Microsoft Threat Modeling Tool release 06/30/2023 - Azure description: Documenting the release notes for the threat modeling tool release 7.3.30630.5.--++ Last updated 06/30/2023 |
| security | Threat Modeling Tool Releases 73308291 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/security/develop/threat-modeling-tool-releases-73308291.md | Title: Microsoft Threat Modeling Tool release 08/30/2023 - Azure description: Documenting the release notes for the threat modeling tool release 7.3.30829.1.--++ Last updated 08/30/2023 |
| security | Threat Modeling Tool Releases 73309251 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/security/develop/threat-modeling-tool-releases-73309251.md | Title: Microsoft Threat Modeling Tool release 09/25/2023 - Azure description: Documenting the release notes for the threat modeling tool release 7.3.30925.1.--++ Last updated 09/25/2023 |
| security | Threat Modeling Tool Releases 73310263 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/security/develop/threat-modeling-tool-releases-73310263.md | Title: Microsoft Threat Modeling Tool release 10/26/2023 - Azure description: Documenting the release notes for the threat modeling tool release 7.3.31026.3.--++ Last updated 10/26/2023 |
| sentinel | Data Connectors Reference | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/sentinel/data-connectors-reference.md | Data connectors are available as part of the following offerings: ## Amazon Web Services - [Amazon Web Services](data-connectors/amazon-web-services.md)-- [Amazon Web Services S3 (preview)](data-connectors/amazon-web-services-s3.md)+- [Amazon Web Services S3](data-connectors/amazon-web-services-s3.md) ## Apache Data connectors are available as part of the following offerings: - [Threat intelligence - TAXII](data-connectors/threat-intelligence-taxii.md) - [Threat Intelligence Platforms](data-connectors/threat-intelligence-platforms.md) - [Threat Intelligence Upload Indicators API (Preview)](data-connectors/threat-intelligence-upload-indicators-api.md)-- [Windows DNS Events via AMA (Preview)](data-connectors/windows-dns-events-via-ama.md)+- [Windows DNS Events via AMA](data-connectors/windows-dns-events-via-ama.md) - [Windows Firewall](data-connectors/windows-firewall.md) - [Windows Forwarded Events](data-connectors/windows-forwarded-events.md) - [Windows Security Events via AMA](data-connectors/windows-security-events-via-ama.md) |
| sentinel | Amazon Web Services S3 | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/sentinel/data-connectors/amazon-web-services-s3.md | Title: "Amazon Web Services S3 connector for Microsoft Sentinel (preview)" + Title: "Amazon Web Services S3 connector for Microsoft Sentinel" description: "Learn how to install the connector Amazon Web Services S3 to connect your data source to Microsoft Sentinel." -# Amazon Web Services S3 connector for Microsoft Sentinel (preview) +# Amazon Web Services S3 connector for Microsoft Sentinel This connector allows you to ingest AWS service logs, collected in AWS S3 buckets, to Microsoft Sentinel. The currently supported data types are: * AWS CloudTrail |
| sentinel | Windows Dns Events Via Ama | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/sentinel/data-connectors/windows-dns-events-via-ama.md | Title: "Windows DNS Events via AMA (Preview) connector for Microsoft Sentinel" -description: "Learn how to install the connector Windows DNS Events via AMA (Preview) to connect your data source to Microsoft Sentinel." + Title: "Windows DNS Events via AMA connector for Microsoft Sentinel" +description: "Learn how to install the connector Windows DNS Events via AMA to connect your data source to Microsoft Sentinel." Previously updated : 02/28/2023 Last updated : 04/04/2024 -# Windows DNS Events via AMA (Preview) connector for Microsoft Sentinel +# Windows DNS Events via AMA connector for Microsoft Sentinel The Windows DNS log connector allows you to easily filter and stream all analytics logs from your Windows DNS servers to your Microsoft Sentinel workspace using the Azure Monitoring agent (AMA). Having this data in Microsoft Sentinel helps you identify issues and security threats such as: - Trying to resolve malicious domain names. |
| sentinel | Feature Availability | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/sentinel/feature-availability.md | This article describes the features available in Microsoft Sentinel across diffe |Feature |Feature stage |Azure commercial |Azure Government |Azure China 21Vianet | |||||| |[Amazon Web Services](connect-aws.md?tabs=ct) |GA |✅ |✅ |❌ |-|[Amazon Web Services S3 (Preview)](connect-aws.md?tabs=s3) |Public preview |✅ |✅ |❌ | +|[Amazon Web Services S3](connect-aws.md?tabs=s3) |GA|✅ |✅ |❌ | |[Microsoft Entra ID](connect-azure-active-directory.md) |GA |✅ |✅|✅ <sup>[1](#logsavailable)</sup> | |[Microsoft Entra ID Protection](connect-services-api-based.md) |GA |✅| ✅ |❌ | |[Azure Activity](data-connectors/azure-activity.md) |GA |✅| ✅|✅ | This article describes the features available in Microsoft Sentinel across diffe |[Cisco ASA](data-connectors/cisco-asa.md) |GA |✅ |✅|✅ | |[Codeless Connectors Platform](create-codeless-connector.md?tabs=deploy-via-arm-template%2Cconnect-via-the-azure-portal) |Public preview |✅ |❌|❌ | |[Common Event Format (CEF)](connect-common-event-format.md) |GA |✅ |✅|✅ |-|[Common Event Format (CEF) via AMA (Preview)](connect-cef-ama.md) |Public preview |✅|❌ |✅ | +|[Common Event Format (CEF) via AMA](connect-cef-syslog-ama.md) |GA |✅|✅ |✅ | |[DNS](data-connectors/dns.md) |Public preview |✅| ❌ |✅ | |[GCP Pub/Sub Audit Logs](connect-google-cloud-platform.md) |Public preview |✅ |✅ |❌ | |[Microsoft Defender XDR](connect-microsoft-365-defender.md?tabs=MDE) |GA |✅| ✅|❌ | This article describes the features available in Microsoft Sentinel across diffe |[Office 365](connect-services-api-based.md) |GA |✅|✅ |✅ | |[Security Events via Legacy Agent](connect-services-windows-based.md#log-analytics-agent-legacy) |GA |✅ |✅|✅ | |[Syslog](connect-syslog.md) |GA |✅| ✅|✅ |+|[Syslog via AMA](connect-cef-syslog-ama.md) |GA |✅| ✅|✅ | |[Windows DNS Events via AMA](connect-dns-ama.md) |GA |✅ |✅|✅ | |[Windows Firewall](data-connectors/windows-firewall.md) |GA |✅ |✅|✅ | |[Windows Forwarded Events](connect-services-windows-based.md) |GA |✅|✅ |✅ | |
| sentinel | Microsoft Sentinel Defender Portal | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/sentinel/microsoft-sentinel-defender-portal.md | description: Learn about changes in the Microsoft Defender portal with the integ Previously updated : 04/03/2024 Last updated : 04/04/2024 appliesto: - Microsoft Sentinel in the Microsoft Defender portal The following capabilities are only available in the Defender portal. ||| |Attack disruption for SAP | [Automatic attack disruption in the Microsoft Defender portal](/microsoft-365/security/defender/automatic-attack-disruption) | + ### Azure portal only The following capabilities are only available in the Azure portal. |Capability |Learn more | |||-|Tasks | [Use tasks to manage incidents in Microsoft Sentinel](incident-tasks.md) | |Add entities to threat intelligence from incidents | [Add entity to threat indicators](add-entity-to-threat-intelligence.md) | | Automation | Some automation procedures are available only in the Azure portal. <br><br>Other automation procedures are the same in the Defender and Azure portals, but differ in the Azure portal between workspaces that are onboarded to the unified security operations platform and workspaces that aren't. <br><br>For more information, see [Automation with the unified security operations platform](automation.md#automation-with-the-unified-security-operations-platform). |+| Hunt using bookmarks | [Bookmarks](/azure/sentinel/bookmarks) aren't supported in the advanced hunting experience in the Microsoft Defender portal. In the Defender portal, they are supported in the **Microsoft Sentinel > Threat management > Hunting**. | +|Tasks | [Use tasks to manage incidents in Microsoft Sentinel](incident-tasks.md) | ## Quick reference |
| service-fabric | Monitor Service Fabric Reference | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/service-fabric/monitor-service-fabric-reference.md | + + Title: Monitoring data reference for Azure Service Fabric +description: This article contains important reference material you need when you monitor Service Fabric. Last updated : 03/26/2024++++++++# Azure Service Fabric monitoring data reference +++See [Monitor Service Fabric](monitor-service-fabric.md) for details on the data you can collect for Azure Service Fabric and how to use it. ++Azure Monitor doesn't collect any platform metrics or resource logs for Service Fabric. You can monitor and collect: ++- Service Fabric system, node, and application events. For the full event listing, see [List of Service Fabric events](service-fabric-diagnostics-event-generation-operational.md). +- Windows performance counters on nodes and applications. For the list of performance counters, see [Performance metrics](service-fabric-diagnostics-event-generation-perf.md). +- Cluster, node, and system service health data. You can use the [FabricClient.HealthManager property](/dotnet/api/system.fabric.fabricclient.healthmanager) to get the health client to use for health related operations, like report health or get entity health. +- Metrics for the guest operating system (OS) that runs on a cluster node, through one or more agents that run on the guest OS. ++ Guest OS metrics include performance counters that track guest CPU percentage or memory usage, which are frequently used for autoscaling or alerting. You can use the agent to send guest OS metrics to Azure Monitor Logs, where you can query them by using Log Analytics. ++ > [!NOTE] + > The Azure Monitor agent replaces the previously-used Azure Diagnostics extension and Log Analytics agent. For more information, see [Overview of Azure Monitor agents](/azure/azure-monitor/agents/agents-overview). +++### Service Fabric Clusters +Microsoft.ServiceFabric/clusters ++- [AzureActivity](/azure/azure-monitor/reference/tables/AzureActivity#columns) +- [AzureMetrics](/azure/azure-monitor/reference/tables/AzureMetrics#columns) +++- [Microsoft.ServiceFabric resource provider operations](/azure/role-based-access-control/permissions/compute#microsoftservicefabric) ++## Related content ++- See [Monitor Service Fabric](monitor-service-fabric.md) for a description of monitoring Service Fabric. +- See [Monitor Azure resources with Azure Monitor](/azure/azure-monitor/essentials/monitor-azure-resource) for details on monitoring Azure resources. +- See [List of Service Fabric events](service-fabric-diagnostics-event-generation-operational.md) for the list of Service Fabric system, node, and application events. +- See [Performance metrics](service-fabric-diagnostics-event-generation-perf.md) for the list of Windows performance counters on nodes and applications. |
| service-fabric | Monitor Service Fabric | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/service-fabric/monitor-service-fabric.md | + + Title: Monitor Azure Service Fabric +description: Start here to learn how to monitor Service Fabric. Last updated : 03/26/2024++++++++# Monitor Azure Service Fabric +++## Azure Service Fabric monitoring ++Azure Service Fabric has the following layers that you can monitor: ++- Service health and performance counters for the service *infrastructure*. For more information, see [Performance metrics](service-fabric-diagnostics-event-generation-perf.md). +- Client metrics, logs, and events for the *platform* or *cluster* nodes, including container metrics. The metrics and logs are different for Linux or Windows nodes. For more information, see [Monitor the cluster](service-fabric-diagnostics-event-generation-infra.md). +- The *applications* that run on the nodes. You can monitor applications with Application Insights key or SDK, EventStore, or ASP.NET Core logging. For more information, see [Application logging](service-fabric-diagnostics-event-generation-app.md). ++You can monitor how your applications are used, the actions taken by the Service Fabric platform, your resource utilization with performance counters, and the overall health of your cluster. [Azure Monitor logs](service-fabric-diagnostics-event-analysis-oms.md) and [Application Insights](service-fabric-diagnostics-event-analysis-appinsights.md) offer built-in integration with Service Fabric. ++- For an overview of monitoring and diagnostics for Service Fabric infrastructure, platform, and applications, see [Monitoring and diagnostics for Azure Service Fabric](service-fabric-diagnostics-overview.md). +- For a tutorial that shows how to view Service Fabric events and health reports, query the EventStore APIs, and monitor performance counters, see [Tutorial: Monitor a Service Fabric cluster in Azure](service-fabric-tutorial-monitor-cluster.md). ++### Service Fabric Explorer ++[Service Fabric Explorer](service-fabric-visualizing-your-cluster.md), a desktop application for Windows, macOS, and Linux, is an open-source tool for inspecting and managing Azure Service Fabric clusters. To enable automation, every action that can be taken through Service Fabric Explorer can also be done through PowerShell or a REST API. ++### EventStore ++[EventStore](service-fabric-diagnostics-eventstore.md) is a feature that shows Service Fabric platform events in Service Fabric Explorer and programmatically through the [Service Fabric Client Library](/dotnet/api/overview/azure/service-fabric#client-library) REST API. You can see a snapshot view of what's going on in your cluster for each node, service, and application, and query based on the time of the event. ++The EventStore APIs are available only for Windows clusters running on Azure. On Windows machines, these events are fed into the Event Log, so you can see Service Fabric Events in Event Viewer. ++### Application Insights ++Application Insights integrates with Service Fabric to provide Service Fabric specific metrics and tooling experiences for Visual Studio and Azure portal. Application Insights provides a comprehensive out-of-the-box logging experience. For more information, see [Event analysis and visualization with Application Insights](service-fabric-diagnostics-event-analysis-appinsights.md). +++For more information about the resource types for Azure Service Fabric, see [Service Fabric monitoring data reference](monitor-service-fabric-reference.md). +++++### Performance counters ++Service Fabric system performance is usually measured through performance counters. These performance counters can come from various sources including the operating system, the .NET framework, or the Service Fabric platform itself. For a list of performance counters that should be collected at the infrastructure level, see [Performance metrics](service-fabric-diagnostics-event-generation-perf.md). ++Service Fabric also provides a set of performance counters for the Reliable Services and Actors programming models. For more information, see [Monitoring for Reliable Service Remoting](service-fabric-reliable-serviceremoting-diagnostics.md#performance-counters) and [Performance monitoring for Reliable Actors](service-fabric-reliable-actors-diagnostics.md#performance-counters). ++Azure Monitor Logs is recommended for monitoring cluster level events. After you configure the [Log Analytics agent](service-fabric-diagnostics-oms-agent.md) with your workspace, you can collect: ++- Performance metrics such as CPU Utilization. +- .NET performance counters such as process level CPU utilization. +- Service Fabric performance counters such as number of exceptions from a reliable service. +- Container metrics such as CPU Utilization. ++### Guest OS metrics ++Metrics for the guest operating system (OS) that runs on Service Fabric cluster nodes must be collected through one or more agents that run on the guest OS. Guest OS metrics include performance counters that track guest CPU percentage or memory usage, both of which are frequently used for autoscaling or alerting. ++A best practice is to use and configure the Azure Monitor agent to send guest OS performance metrics through the custom metrics API into the Azure Monitor metrics database. You can send the guest OS metrics to Azure Monitor Logs by using the same agent. Then you can query on those metrics and logs by using Log Analytics. ++>[!NOTE] +>The Azure Monitor agent replaces the Azure Diagnostics extension and Log Analytics agent for guest OS routing. For more information, see [Overview of Azure Monitor agents](/azure/azure-monitor/agents/agents-overview). +++## Service Fabric logs and events ++Service Fabric can collect the following logs: ++- For Windows clusters, you can set up cluster monitoring with [Diagnostics Agent](service-fabric-diagnostics-event-aggregation-wad.md) and [Azure Monitor logs](service-fabric-diagnostics-oms-setup.md). +- For Linux clusters, Azure Monitor Logs is also the recommended tool for Azure platform and infrastructure monitoring. Linux platform diagnostics require different configuration. For more information, see [Service Fabric Linux cluster events in Syslog](service-fabric-diagnostics-oms-syslog.md). +- You can configure the Azure Monitor agent to send guest OS logs to Azure Monitor Logs, where you can query on them by using Log Analytics. +- You can write Service Fabric container logs to *stdout* or *stderr* so they're available in Azure Monitor Logs. ++### Service Fabric events ++Service Fabric provides a comprehensive set of diagnostics events out of the box, which you can access through the EventStore or the operational event channel the platform exposes. These [Service Fabric events](service-fabric-diagnostics-events.md) illustrate actions done by the platform on different entities such as nodes, applications, services, and partitions. The same events are available on both Windows and Linux clusters. ++On Windows, Service Fabric events are available from a single Event Tracing for Windows (ETW) provider with a set of relevant `logLevelKeywordFilters` used to pick between Operational and Data & Messaging channels. On Linux, Service Fabric events come through LTTng and are put into one Azure Storage table, from where they can be filtered as needed. Diagnostics can be enabled at cluster creation time, which creates a Storage table where the events from these channels are sent. ++The events are sent through standard channels on both Windows and Linux and can be read by any monitoring tool that supports them, including Azure Monitor Logs. For more information, see [Azure Monitor logs integration](service-fabric-diagnostics-event-analysis-oms.md). ++### Health monitoring ++The Service Fabric platform includes a health model, which provides extensible health reporting for the status of entities in a cluster. Each node, application, service, partition, replica, or instance has a continuously updatable health status. Each time the health of a particular entity transitions, an event is also emitted. You can set up queries and alerts for health events in your monitoring tool, just like any other event. ++## Partner logging solutions ++Many events are written out through ETW providers and are extensible with other logging solutions. Examples are [Elastic Stack](https://www.elastic.co/products), especially if you're running a cluster in an offline environment, or [Dynatrace](https://www.dynatrace.com/). For a list of integrated partners, see [Azure Service Fabric Monitoring Partners](service-fabric-diagnostics-partners.md). ++++For an overview of common Service Fabric monitoring analytics scenarios, see [Diagnose common scenarios with Service Fabric](service-fabric-diagnostics-common-scenarios.md). ++++### Sample queries ++The following queries return Service Fabric Events, including actions on nodes. For other useful queries, see [Service Fabric Events](service-fabric-tutorial-monitor-cluster.md#view-service-fabric-events-including-actions-on-nodes). ++Return operational events recorded in the last hour: ++```kusto +ServiceFabricOperationalEvent +| where TimeGenerated > ago(1h) +| join kind=leftouter ServiceFabricEvent on EventId +| project EventId, EventName, TaskName, Computer, ApplicationName, EventMessage, TimeGenerated +| sort by TimeGenerated +``` ++Return Health Reports with HealthState == 3 (Error), and extract more properties from the `EventMessage` field: ++```kusto +ServiceFabricOperationalEvent +| join kind=leftouter ServiceFabricEvent on EventId +| extend HealthStateId = extract(@"HealthState=(\S+) ", 1, EventMessage, typeof(int)) +| where TaskName == 'HM' and HealthStateId == 3 +| extend SourceId = extract(@"SourceId=(\S+) ", 1, EventMessage, typeof(string)), + Property = extract(@"Property=(\S+) ", 1, EventMessage, typeof(string)), + HealthState = case(HealthStateId == 0, 'Invalid', HealthStateId == 1, 'Ok', HealthStateId == 2, 'Warning', HealthStateId == 3, 'Error', 'Unknown'), + TTL = extract(@"TTL=(\S+) ", 1, EventMessage, typeof(string)), + SequenceNumber = extract(@"SequenceNumber=(\S+) ", 1, EventMessage, typeof(string)), + Description = extract(@"Description='([\S\s, ^']+)' ", 1, EventMessage, typeof(string)), + RemoveWhenExpired = extract(@"RemoveWhenExpired=(\S+) ", 1, EventMessage, typeof(bool)), + SourceUTCTimestamp = extract(@"SourceUTCTimestamp=(\S+)", 1, EventMessage, typeof(datetime)), + ApplicationName = extract(@"ApplicationName=(\S+) ", 1, EventMessage, typeof(string)), + ServiceManifest = extract(@"ServiceManifest=(\S+) ", 1, EventMessage, typeof(string)), + InstanceId = extract(@"InstanceId=(\S+) ", 1, EventMessage, typeof(string)), + ServicePackageActivationId = extract(@"ServicePackageActivationId=(\S+) ", 1, EventMessage, typeof(string)), + NodeName = extract(@"NodeName=(\S+) ", 1, EventMessage, typeof(string)), + Partition = extract(@"Partition=(\S+) ", 1, EventMessage, typeof(string)), + StatelessInstance = extract(@"StatelessInstance=(\S+) ", 1, EventMessage, typeof(string)), + StatefulReplica = extract(@"StatefulReplica=(\S+) ", 1, EventMessage, typeof(string)) +``` ++Get Service Fabric operational events aggregated with the specific service and node: ++```kusto +ServiceFabricOperationalEvent +| where ApplicationName != "" and ServiceName != "" +| summarize AggregatedValue = count() by ApplicationName, ServiceName, Computer +``` +++### Service Fabric alert rules ++The following table lists some alert rules for Service Fabric. These alerts are just examples. You can set alerts for any metric, log entry, or activity log entry listed in the [Service Fabric monitoring data reference](monitor-service-fabric-reference.md) or the [List of Service Fabric events](service-fabric-diagnostics-event-generation-operational.md#application-events). ++| Alert type | Condition | Description | +|:|:|:| +| Node event | Node goes down | ServiceFabricOperationalEvent where EventID >= 25622 and EventID <= 25626. These Event IDs are found in the [Node events reference](service-fabric-diagnostics-event-generation-operational.md#node-events). | +| Application event | Application upgrade rollback | ServiceFabricOperationalEvent where EventID == 29623 or EventID == 29624. These Event IDs are found in the [Application events reference](service-fabric-diagnostics-event-generation-operational.md#application-events). | +| Resource health | Upgrade service unreachable/unavailable | Cluster goes to UpgradeServiceUnreachable state. | +++## Related content ++- See [Service Fabric monitoring data reference](monitor-service-fabric-reference.md) for a reference of the metrics, logs, and other important values created for Service Fabric. +- See [Monitoring Azure resources with Azure Monitor](/azure/azure-monitor/essentials/monitor-azure-resource) for general details on monitoring Azure resources. +- See the [List of Service Fabric events](service-fabric-diagnostics-event-generation-operational.md). |
| storage | Lifecycle Management Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/storage/blobs/lifecycle-management-overview.md | The following sample rule filters the account to run the actions on objects that ### Rule filters -Filters limit rule actions to a subset of blobs within the storage account. If more than one filter is defined, a logical `AND` runs on all filters. You can use a filter to specify which blobs to include. A filter provides no means to specify which blobs to exclude. +Filters limit rule actions to a subset of blobs within the storage account. If more than one filter is defined, a logical `AND` runs on all filters. You can use a filter to specify which blobs to include. A filter provides no means to specify which blobs to exclude. Filters include: | Filter name | Filter type | Notes | Is Required | |-|-|-|-|-| blobTypes | An array of predefined enum values. | The current release supports `blockBlob` and `appendBlob`. Only delete is supported for `appendBlob`, set tier isn't supported. | Yes | -| prefixMatch | An array of strings for prefixes to be matched. Each rule can define up to 10 case-sensitive prefixes. A prefix string must start with a container name. For example, if you want to match all blobs under `https://myaccount.blob.core.windows.net/sample-container/blob1/...` for a rule, the prefixMatch is `sample-container/blob1`.<br /><br />To match the container or blob name exactly, include the trailing forward slash ('/'), *e.g.*, `sample-container/` or `sample-container/blob1/`. To match the container or blob name pattern, omit the trailing forward slash, *e.g.*, `sample-container` or `sample-container/blob1`. | If you don't define prefixMatch, the rule applies to all blobs within the storage account. Prefix strings don't support wildcard matching. Characters such as `*` and `?` are treated as string literals. | No | +| blobTypes | An array of predefined enum values. | The current release supports `blockBlob` and `appendBlob`. Only the Delete action is supported for `appendBlob`; Set Tier isn't supported. | Yes | +| prefixMatch | An array of strings for prefixes to be matched. Each rule can define up to 10 case-sensitive prefixes. A prefix string must start with a container name. For example, if you want to match all blobs under `https://myaccount.blob.core.windows.net/sample-container/blob1/...`, specify the **prefixMatch** as `sample-container/blob1`. This filter will match all blobs in *sample-container* whose names begin with *blob1*.<br /><br />. | If you don't define **prefixMatch**, the rule applies to all blobs within the storage account. Prefix strings don't support wildcard matching. Characters such as `*` and `?` are treated as string literals. | No | | blobIndexMatch | An array of dictionary values consisting of blob index tag key and value conditions to be matched. Each rule can define up to 10 blob index tag condition. For example, if you want to match all blobs with `Project = Contoso` under `https://myaccount.blob.core.windows.net/` for a rule, the blobIndexMatch is `{"name": "Project","op": "==","value": "Contoso"}`. | If you don't define blobIndexMatch, the rule applies to all blobs within the storage account. | No | To learn more about the blob index feature together with known issues and limitations, see [Manage and find data on Azure Blob Storage with blob index](storage-manage-find-blobs.md). |
| storage | Secure File Transfer Protocol Support | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/storage/blobs/secure-file-transfer-protocol-support.md | The following clients have compatible algorithm support with SFTP for Azure Blob - JSCH 0.1.54+ - curl 7.85.0+ - AIX<sup>1</sup>+- MobaXterm v21.3 <sup>1</sup> Must set `AllowPKCS12KeystoreAutoOpen` option to `no`. |
| storage | File Sync Choose Cloud Tiering Policies | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/storage/file-sync/file-sync-choose-cloud-tiering-policies.md | description: Details on what to keep in mind when choosing Azure File Sync cloud Previously updated : 03/26/2024 Last updated : 04/05/2024 This article provides guidance on selecting and adjusting cloud tiering policies - Cloud tiering isn't supported on the Windows system volume. -- You can still enable cloud tiering if you have a volume-level FSRM quota. Once an FSRM quota is set, the free space query APIs that get called automatically report the free space on the volume as per the quota setting.+- If you're using File Server Resource Manager (FSRM) for quota management on server endpoints, we recommend applying the quotas at the folder level and not at the volume level. You can still enable cloud tiering if you have a volume-level FSRM quota. Once an FSRM quota is set, the free space query APIs that get called automatically report the free space on the volume as per the quota setting. However, because the free space reported by the quota setting is based on logical size, the actual amount of volume free space might be different. This could cause endless tiering if Azure File Sync thinks there isn't enough volume free space. ### Minimum file size for a file to tier |
| storage | Geo Redundant Storage For Large File Shares | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/storage/files/geo-redundant-storage-for-large-file-shares.md | description: Azure Files geo-redundancy for large file shares significantly impr Previously updated : 04/01/2024 Last updated : 04/05/2024 Azure Files geo-redundancy for large file shares is generally available in the m | North Europe | Preview | | Norway East | GA | | Norway West | GA |-| South Africa North | Preview | -| South Africa West | Preview | +| South Africa North | GA | +| South Africa West | GA | | South Central US | Preview | | South India | Preview | | Southeast Asia | GA | |
| storage | Storage Files Netapp Comparison | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/storage/files/storage-files-netapp-comparison.md | Most workloads that require cloud file storage work well on either Azure Files o | Encryption | All protocols<br><ul><li>Encryption at rest (AES-256) with customer or Microsoft-managed keys</li></ul><br>SMB<br><ul><li>Kerberos encryption using AES-256 (recommended) or RC4-HMAC</li><li>Encryption in transit</li></ul><br>REST<br><ul><li>Encryption in transit</li></ul><br> To learn more, see [Security and networking](files-nfs-protocol.md#security-and-networking). | All protocols<br><ul><li>Encryption at rest (AES-256) with Microsoft-managed keys</li><li>[Encryption at rest (AES-256) with customer-managed keys](../../azure-netapp-files/configure-customer-managed-keys.md)</li></ul><br>SMB<ul><li>Encryption in transit using AES-CCM (SMB 3.0) and AES-GCM (SMB 3.1.1)</li></ul><br>NFS 4.1<ul><li>Encryption in transit using Kerberos with AES-256</li></ul><br> To learn more, see [security FAQ](../../azure-netapp-files/faq-security.md). | | Access Options | <ul><li>Internet</li><li>Secure VNet access</li><li>VPN Gateway</li><li>ExpressRoute</li><li>Azure File Sync</li></ul><br> To learn more, see [network considerations](./storage-files-networking-overview.md). | <ul><li>Secure VNet access</li><li>VPN Gateway</li><li>ExpressRoute</li><li>[Virtual WAN](../../azure-netapp-files/configure-virtual-wan.md)</li><li>[Global File Cache](https://cloud.netapp.com/global-file-cache/azure)</li><li>[HPC Cache](../../hpc-cache/hpc-cache-overview.md)</li><li>[Standard Network Features](../../azure-netapp-files/azure-netapp-files-network-topologies.md#configurable-network-features)</li></ul><br> To learn more, see [network considerations](../../azure-netapp-files/azure-netapp-files-network-topologies.md). | | Data Protection | <ul><li>Incremental snapshots</li><li>File/directory user self-restore</li><li>Restore to new location</li><li>In-place revert</li><li>Share-level soft delete</li><li>Azure Backup integration</li></ul><br> To learn more, see [Azure Files enhances data protection capabilities](https://azure.microsoft.com/blog/azure-files-enhances-data-protection-capabilities/). | <ul><li>[Azure NetApp Files backup](../../azure-netapp-files/backup-introduction.md)</li><li>Snapshots (255/volume)</li><li>File/directory user self-restore</li><li>Restore to new volume</li><li>In-place revert</li><li>[Cross-region replication](../../azure-netapp-files/cross-region-replication-introduction.md)</li><li>[Cross-zone replication](../../azure-netapp-files/cross-zone-replication-introduction.md)</li></ul><br> To learn more, see [How Azure NetApp Files snapshots work](../../azure-netapp-files/snapshots-introduction.md). |-| Migration Tools | <ul><li>Azure Data Box</li><li>Azure File Sync</li><li>Storage Migration Service</li><li>AzCopy</li><li>Robocopy</li></ul><br> To learn more, see [Migrate to Azure file shares](./storage-files-migration-overview.md). | <ul><li>[Global File Cache](https://cloud.netapp.com/global-file-cache/azure)</li><li>[CloudSync](https://cloud.netapp.com/cloud-sync-service), [XCP](https://xcp.netapp.com/)</li><li>Storage Migration Service</li><li>AzCopy</li><li>Robocopy</li><li>Application-based (for example, HSR, Data Guard, AOAG)</li></ul> | +| Migration Tools | <ul><li>Azure Data Box</li><li>Azure File Sync</li><li>Azure Storage Mover</li><li>Storage Migration Service</li><li>AzCopy</li><li>Robocopy</li></ul><br> To learn more, see [Migrate to Azure file shares](./storage-files-migration-overview.md). | <ul><li>[Global File Cache](https://cloud.netapp.com/global-file-cache/azure)</li><li>[CloudSync](https://cloud.netapp.com/cloud-sync-service), [XCP](https://xcp.netapp.com/)</li><li>Storage Migration Service</li><li>AzCopy</li><li>Robocopy</li><li>Application-based (for example, HSR, Data Guard, AOAG)</li></ul> | | Tiers | <ul><li>Premium</li><li>Transaction Optimized</li><li>Hot</li><li>Cool</li></ul><br> To learn more, see [storage tiers](./storage-files-planning.md#storage-tiers). | <ul><li>Ultra</li><li>Premium</li><li>Standard</li></ul><br> All tiers provide sub-ms minimum latency.<br><br> To learn more, see [Service Levels](../../azure-netapp-files/azure-netapp-files-service-levels.md) and [Performance Considerations](../../azure-netapp-files/azure-netapp-files-performance-considerations.md). | | Pricing | [Azure Files Pricing](https://azure.microsoft.com/pricing/details/storage/files/) | [Azure NetApp Files Pricing](https://azure.microsoft.com/pricing/details/netapp/) | |
| storage | Storage Files Scale Targets | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/storage/files/storage-files-scale-targets.md | description: Learn about the capacity, IOPS, and throughput rates for Azure file Previously updated : 03/22/2024 Last updated : 04/05/2024 The following table indicates which targets are soft, representing the Microsoft | Resource | Target | Hard limit | |-|--|| | Storage Sync Services per region | 100 Storage Sync Services | Yes |+| Storage Sync Services per subscription | 15 Storage Sync Services | Yes | | Sync groups per Storage Sync Service | 200 sync groups | Yes | | Registered servers per Storage Sync Service | 99 servers | Yes | | Private endpoints per Storage Sync Service | 100 private endpoints | Yes | |
| stream-analytics | Functions Overview | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/stream-analytics/functions-overview.md | Azure Stream Analytics supports the following four function types: * Azure Machine Learning You can use these functions for scenarios such as real-time scoring using machine learning models, string manipulations, complex mathematical calculations, encoding and decoding data. +> [!IMPORTANT] +> C# user-defined functions for Azure Stream Analytics will be retired on September 30th 2024. After that date, it won't be possible to use the feature. ## Limitations -User-defined functions are stateless, and the return value can only be a scalar value. You cannot call out to external REST endpoints from these user-defined functions, as it will likely impact performance of your job. +User-defined functions are stateless, and the return value can only be a scalar value. You can't call out to external REST endpoints from these user-defined functions, as it will likely impact performance of your job. -Azure Stream Analytics does not keep a record of all functions invocations and returned results. To guarantee repeatability - for example, re-running your job from older timestamp produces the same results again - do not to use functions such as `Date.GetData()` or `Math.random()`, as these functions do not return the same result for each invocation. +Azure Stream Analytics doesn't keep a record of all functions invocations and returned results. To guarantee repeatability - for example, re-running your job from older timestamp produces the same results again - don't to use functions such as `Date.GetData()` or `Math.random()`, as these functions don't return the same result for each invocation. ## Resource logs -Any runtime errors are considered fatal and are surfaced through activity and resource logs. It is recommended that your function handles all exceptions and errors and return a valid result to your query. This will prevent your job from going to a [Failed state](job-states.md). +Any runtime errors are considered fatal and are surfaced through activity and resource logs. It's recommended that your function handles all exceptions and errors and return a valid result to your query. This will prevent your job from going to a [Failed state](job-states.md). ## Exception handling -Any exception during data processing is considered a catastrophic failure when consuming data in Azure Stream Analytics. User-defined functions have a higher potential to throw exceptions and cause the processing to stop. To avoid this issue, use a *try-catch* block in JavaScript or C# to catch exceptions during code execution. Exceptions that are caught can be logged and treated without causing a system failure. You are encouraged to always wrap your custom code in a *try-catch* block to avoid throwing unexpected exceptions to the processing engine. +Any exception during data processing is considered a catastrophic failure when consuming data in Azure Stream Analytics. User-defined functions have a higher potential to throw exceptions and cause the processing to stop. To avoid this issue, use a *try-catch* block in JavaScript or C# to catch exceptions during code execution. Exceptions that are caught can be logged and treated without causing a system failure. You're encouraged to always wrap your custom code in a *try-catch* block to avoid throwing unexpected exceptions to the processing engine. ## Next steps |
| stream-analytics | Visual Studio Code Custom Deserializer | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/stream-analytics/visual-studio-code-custom-deserializer.md | Last updated 01/21/2023 # Tutorial: Custom .NET deserializers for Azure Stream Analytics in Visual Studio Code (Preview) +> [!IMPORTANT] +> Custom .net deserializer for Azure Stream Analytics will be retired on September 30th 2024. After that date, it won't be possible to use the feature. + Azure Stream Analytics has built-in support for three data formats: JSON, CSV, and Avro as shown in this [doc](stream-analytics-parsing-json.md). With custom .NET deserializers, you can process data in other formats such as [Protocol Buffer](https://developers.google.com/protocol-buffers/), [Bond](https://github.com/Microsoft/bond) and other user defined formats for cloud jobs. This tutorial demonstrates how to create, test, and debug a custom .NET deserializer for an Azure Stream Analytics job using Visual Studio Code. You'll learn how to: |
| synapse-analytics | Apache Spark 24 Runtime | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/apache-spark-24-runtime.md | Azure Synapse Analytics supports multiple runtimes for Apache Spark. This docume > * Effective September 29, 2023, Azure Synapse will discontinue official support for Spark 2.4 Runtimes. > * Post September 29, we will not be addressing any support tickets related to Spark 2.4. There will be no release pipeline in place for bug or security fixes for Spark 2.4. Utilizing Spark 2.4 post the support cutoff date is undertaken at one's own risk. We strongly discourage its continued use due to potential security and functionality concerns. > * Recognizing that certain customers may need additional time to transition to a higher runtime version, we are temporarily extending the usage option for Spark 2.4, but we will not provide any official support for it.-> * We strongly advise proactively upgrading workloads to a more recent version of the runtime (e.g., [Azure Synapse Runtime for Apache Spark 3.3 (GA)](./apache-spark-33-runtime.md)). +> * **We strongly advise proactively upgrading workloads to a more recent version of the runtime (e.g., [Azure Synapse Runtime for Apache Spark 3.4 (GA)](./apache-spark-34-runtime.md)).** ## Component versions |
| synapse-analytics | Apache Spark 3 Runtime | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/apache-spark-3-runtime.md | Azure Synapse Analytics supports multiple runtimes for Apache Spark. This docume > * Effective January 26, 2024, the Azure Synapse has stopped official support for Spark 3.1 Runtimes. > * Post January 26, 2024, we will not be addressing any support tickets related to Spark 3.1. There will be no release pipeline in place for bug or security fixes for Spark 3.1. Utilizing Spark 3.1 post the support cutoff date is undertaken at one's own risk. We strongly discourage its continued use due to potential security and functionality concerns. > * Recognizing that certain customers may need additional time to transition to a higher runtime version, we are temporarily extending the usage option for Spark 3.1, but we will not provide any official support for it.-> * We strongly advise proactively upgrading workloads to a more recent version of the runtime (e.g., [Azure Synapse Runtime for Apache Spark 3.3 (GA)](./apache-spark-33-runtime.md)). +> * **We strongly advise proactively upgrading workloads to a more recent version of the runtime (e.g., [Azure Synapse Runtime for Apache Spark 3.4 (GA)](./apache-spark-34-runtime.md))**. ## Component versions |
| synapse-analytics | Apache Spark 32 Runtime | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/apache-spark-32-runtime.md | Azure Synapse Analytics supports multiple runtimes for Apache Spark. This docume > * End of Support announced for Azure Synapse Runtime for Apache Spark 3.2 has been announced July 8, 2023. > * End of Support announced runtime will not have bug and feature fixes. Security fixes will be backported based on risk assessment. > * In accordance with the Synapse runtime for Apache Spark lifecycle policy, Azure Synapse runtime for Apache Spark 3.2 will be retired and disabled as of July 8, 2024. After the End of Support date, the retired runtimes are unavailable for new Spark pools and existing workflows can't execute. Metadata will temporarily remain in the Synapse workspace.-> * We recommend that you upgrade your Apache Spark 3.2 workloads to version 3.3 at your earliest convenience. +> * **We strongly recommend that you upgrade your Apache Spark 3.2 workloads to [Azure Synapse Runtime for Apache Spark 3.4 (GA)](./apache-spark-34-runtime.md) before July 8, 2024.** ## Component versions |
| synapse-analytics | Apache Spark 33 Runtime | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/apache-spark-33-runtime.md | -## Component versions +> [!TIP] +> We strongly recommend proactively upgrading workloads to a more recent GA version of the runtime which currently is [Azure Synapse Runtime for Apache Spark 3.4 (GA)](./apache-spark-34-runtime.md). +## Component versions | Component | Version | | -- |--| | Apache Spark | 3.3.1 | The following sections present the libraries included in Azure Synapse Runtime f ## Migration between Apache Spark versions - support -For guidance on migrating from older runtime versions to Azure Synapse Runtime for Apache Spark 3.3 or 3.4 refer to [Runtime for Apache Spark Overview](./apache-spark-version-support.md). +For guidance on migrating from older runtime versions to Azure Synapse Runtime for Apache Spark 3.3 or 3.4 refer to [Runtime for Apache Spark Overview](./apache-spark-version-support.md#migration-between-apache-spark-versionssupport). |
| synapse-analytics | Apache Spark 34 Runtime | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/apache-spark-34-runtime.md | Title: Azure Synapse Runtime for Apache Spark 3.4 -description: New runtime is in Public Preview. Try it and use Spark 3.4.1, Python 3.10, Delta Lake 2.4. +description: New runtime is in GA stage. Try it and use Spark 3.4.1, Python 3.10, Delta Lake 2.4. -# Azure Synapse Runtime for Apache Spark 3.4 (Public Preview) +# Azure Synapse Runtime for Apache Spark 3.4 (GA) Azure Synapse Analytics supports multiple runtimes for Apache Spark. This document covers the runtime components and versions for the Azure Synapse Runtime for Apache Spark 3.4. ## Component versions Azure Synapse Analytics supports multiple runtimes for Apache Spark. This docume ## Libraries -The following sections present the libraries included in Azure Synapse Runtime for Apache Spark 3.4 (Public Preview). +To check the libraries included in Azure Synapse Runtime for Apache Spark 3.4 for Jav). -### Scala and Java default libraries -The following table lists all the default level packages for Java/Scala and their respective versions. --| GroupID | ArtifactID | Version | -|-||--| -| com.aliyun | aliyun-java-sdk-core | 4.5.10 | -| com.aliyun | aliyun-java-sdk-kms | 2.11.0 | -| com.aliyun | aliyun-java-sdk-ram | 3.1.0 | -| com.aliyun | aliyun-sdk-oss | 3.13.0 | -| com.amazonaws | aws-java-sdk-bundle | 1.12.1026 | -| com.chuusai | shapeless_2.12 | 2.3.7 | -| com.clearspring.analytics | stream | 2.9.6 | -| com.esotericsoftware | kryo-shaded | 4.0.2 | -| com.esotericsoftware | minlog | 1.3.0 | -| com.fasterxml.jackson | jackson-annotations | 2.13.4 | -| com.fasterxml.jackson | jackson-core | 2.13.4 | -| com.fasterxml.jackson | jackson-core-asl | 1.9.13 | -| com.fasterxml.jackson | jackson-databind | 2.13.4.1 | -| com.fasterxml.jackson | jackson-dataformat-cbor | 2.13.4 | -| com.fasterxml.jackson | jackson-mapper-asl | 1.9.13 | -| com.fasterxml.jackson | jackson-module-scala_2.12 | 2.13.4 | -| com.github.joshelser | dropwizard-metrics-hadoop-metrics2-reporter | 0.1.2 | -| com.github.luben | zstd-jni | 1.5.2-1 | -| com.github.vowpalwabbit | vw-jni | 9.3.0 | -| com.github.wendykierp | JTransforms | 3.1 | -| com.google.code.findbugs | jsr305 | 3.0.0 | -| com.google.code.gson | gson | 2.8.6 | -| com.google.crypto.tink | tink | 1.6.1 | -| com.google.flatbuffers | flatbuffers-java | 1.12.0 | -| com.google.guava | guava | 14.0.1 | -| com.google.protobuf | protobuf-java | 2.5.0 | -| com.googlecode.json-simple | json-simple | 1.1.1 | -| com.jcraft | jsch | 0.1.54 | -| com.jolbox | bonecp | 0.8.0.RELEASE | -| com.linkedin.isolation-forest | isolation-forest_3.2.0_2.12 | 2.0.8 | -| com.microsoft.azure | azure-data-lake-store-sdk | 2.3.9 | -| com.microsoft.azure | azure-eventhubs | 3.3.0 | -| com.microsoft.azure | azure-eventhubs-spark_2.12 | 2.3.22 | -| com.microsoft.azure | azure-keyvault-core | 1.0.0 | -| com.microsoft.azure | azure-storage | 7.0.1 | -| com.microsoft.azure | cosmos-analytics-spark-3.4.1-connector_2.12 | 1.8.10 | -| com.microsoft.azure | qpid-proton-j-extensions | 1.2.4 | -| com.microsoft.azure | synapseml_2.12 | 0.11.3-spark3.3 | -| com.microsoft.azure | synapseml-cognitive_2.12 | 0.11.3-spark3.3 | -| com.microsoft.azure | synapseml-core_2.12 | 0.11.3-spark3.3 | -| com.microsoft.azure | synapseml-deep-learning_2.12 | 0.11.3-spark3.3 | -| com.microsoft.azure | synapseml-internal_2.12 | 0.11.3-spark3.3 | -| com.microsoft.azure | synapseml-lightgbm_2.12 | 0.11.3-spark3.3 | -| com.microsoft.azure | synapseml-opencv_2.12 | 0.11.3-spark3.3 | -| com.microsoft.azure | synapseml-vw_2.12 | 0.11.3-spark3.3 | -| com.microsoft.azure.kusto | kusto-data | 3.2.1 | -| com.microsoft.azure.kusto | kusto-ingest | 3.2.1 | -| com.microsoft.azure.kusto | kusto-spark_3.0_2.12 | 3.1.16 | -| com.microsoft.azure.kusto | spark-kusto-synapse-connector_3.1_2.12 | 1.3.3 | -| com.microsoft.cognitiveservices.speech | client-jar-sdk | 1.14.0 | -| com.microsoft.sqlserver | msslq-jdbc | 8.4.1.jre8 | -| com.ning | compress-lzf | 1.1 | -| com.sun.istack | istack-commons-runtime | 3.0.8 | -| com.tdunning | json | 1.8 | -| com.thoughtworks.paranamer | paranamer | 2.8 | -| com.twitter | chill-java | 0.10.0 | -| com.twitter | chill_2.12 | 0.10.0 | -| com.typesafe | config | 1.3.4 | -| com.univocity | univocity-parsers | 2.9.1 | -| com.zaxxer | HikariCP | 2.5.1 | -| commons-cli | commons-cli | 1.5.0 | -| commons-codec | commons-codec | 1.15 | -| commons-collections | commons-collections | 3.2.2 | -| commons-dbcp | commons-dbcp | 1.4 | -| commons-io | commons-io | 2.11.0 | -| commons-lang | commons-lang | 2.6 | -| commons-logging | commons-logging | 1.1.3 | -| commons-pool | commons-pool | 1.5.4 | -| dev.ludovic.netlib | arpack | 2.2.1 | -| dev.ludovic.netlib | blas | 2.2.1 | -| dev.ludovic.netlib | lapack | 2.2.1 | -| io.airlift | aircompressor | 0.21 | -| io.delta | delta-core_2.12 | 2.2.0.9 | -| io.delta | delta-storage | 2.2.0.9 | -| io.dropwizard.metrics | metrics-core | 4.2.7 | -| io.dropwizard.metrics | metrics-graphite | 4.2.7 | -| io.dropwizard.metrics | metrics-jmx | 4.2.7 | -| io.dropwizard.metrics | metrics-json | 4.2.7 | -| io.dropwizard.metrics | metrics-jvm | 4.2.7 | -| io.github.resilience4j | resilience4j-core | 1.7.1 | -| io.github.resilience4j | resilience4j-retry | 1.7.1 | -| io.netty | netty-all | 4.1.74.Final | -| io.netty | netty-buffer | 4.1.74.Final | -| io.netty | netty-codec | 4.1.74.Final | -| io.netty | netty-codec-http2 | 4.1.74.Final | -| io.netty | netty-codec-http-4 | 4.1.74.Final | -| io.netty | netty-codec-socks | 4.1.74.Final | -| io.netty | netty-common | 4.1.74.Final | -| io.netty | netty-handler | 4.1.74.Final | -| io.netty | netty-resolver | 4.1.74.Final | -| io.netty | netty-tcnative-classes | 2.0.48 | -| io.netty | netty-transport | 4.1.74.Final | -| io.netty | netty-transport-classes-epoll | 4.1.87.Final | -| io.netty | netty-transport-classes-kqueue | 4.1.87.Final | -| io.netty | netty-transport-native-epoll | 4.1.87.Final-linux-aarch_64 | -| io.netty | netty-transport-native-epoll | 4.1.87.Final-linux-x86_64 | -| io.netty | netty-transport-native-kqueue | 4.1.87.Final-osx-aarch_64 | -| io.netty | netty-transport-native-kqueue | 4.1.87.Final-osx-x86_64 | -| io.netty | netty-transport-native-unix-common | 4.1.87.Final | -| io.opentracing | opentracing-api | 0.33.0 | -| io.opentracing | opentracing-noop | 0.33.0 | -| io.opentracing | opentracing-util | 0.33.0 | -| io.spray | spray-json_2.12 | 1.3.5 | -| io.vavr | vavr | 0.10.4 | -| io.vavr | vavr-match | 0.10.4 | -| jakarta.annotation | jakarta.annotation-api | 1.3.5 | -| jakarta.inject | jakarta.inject | 2.6.1 | -| jakarta.servlet | jakarta.servlet-api | 4.0.3 | -| jakarta.validation-api | | 2.0.2 | -| jakarta.ws.rs | jakarta.ws.rs-api | 2.1.6 | -| jakarta.xml.bind | jakarta.xml.bind-api | 2.3.2 | -| javax.activation | activation | 1.1.1 | -| javax.jdo | jdo-api | 3.0.1 | -| javax.transaction | jta | 1.1 | -| javax.transaction | transaction-api | 1.1 | -| javax.xml.bind | jaxb-api | 2.2.11 | -| javolution | javolution | 5.5.1 | -| jline | jline | 2.14.6 | -| joda-time | joda-time | 2.10.13 | -| mysql | mysql-connector-java | 8.0.18 | -| net.razorvine | pickle | 1.2 | -| net.sf.jpam | jpam | 1.1 | -| net.sf.opencsv | opencsv | 2.3 | -| net.sf.py4j | py4j | 0.10.9.5 | -| net.sf.supercsv | super-csv | 2.2.0 | -| net.sourceforge.f2j | arpack_combined_all | 0.1 | -| org.antlr | ST4 | 4.0.4 | -| org.antlr | antlr-runtime | 3.5.2 | -| org.antlr | antlr4-runtime | 4.8 | -| org.apache.arrow | arrow-format | 7.0.0 | -| org.apache.arrow | arrow-memory-core | 7.0.0 | -| org.apache.arrow | arrow-memory-netty | 7.0.0 | -| org.apache.arrow | arrow-vector | 7.0.0 | -| org.apache.avro | avro | 1.11.0 | -| org.apache.avro | avro-ipc | 1.11.0 | -| org.apache.avro | avro-mapred | 1.11.0 | -| org.apache.commons | commons-collections4 | 4.4 | -| org.apache.commons | commons-compress | 1.21 | -| org.apache.commons | commons-crypto | 1.1.0 | -| org.apache.commons | commons-lang3 | 3.12.0 | -| org.apache.commons | commons-math3 | 3.6.1 | -| org.apache.commons | commons-pool2 | 2.11.1 | -| org.apache.commons | commons-text | 1.10.0 | -| org.apache.curator | curator-client | 2.13.0 | -| org.apache.curator | curator-framework | 2.13.0 | -| org.apache.curator | curator-recipes | 2.13.0 | -| org.apache.derby | derby | 10.14.2.0 | -| org.apache.hadoop | hadoop-aliyun | 3.3.3.5.2-106693326 | -| org.apache.hadoop | hadoop-annotations | 3.3.3.5.2-106693326 | -| org.apache.hadoop | hadoop-aws | 3.3.3.5.2-106693326 | -| org.apache.hadoop | hadoop-azure | 3.3.3.5.2-106693326 | -| org.apache.hadoop | hadoop-azure-datalake | 3.3.3.5.2-106693326 | -| org.apache.hadoop | hadoop-client-api | 3.3.3.5.2-106693326 | -| org.apache.hadoop | hadoop-client-runtime | 3.3.3.5.2-106693326 | -| org.apache.hadoop | hadoop-cloud-storage | 3.3.3.5.2-106693326 | -| org.apache.hadoop | hadoop-openstack | 3.3.3.5.2-106693326 | -| org.apache.hadoop | hadoop-shaded-guava | 1.1.1 | -| org.apache.hadoop | hadoop-yarn-server-web-proxy | 3.3.3.5.2-106693326 | -| org.apache.hive | hive-beeline | 2.3.9 | -| org.apache.hive | hive-cli | 2.3.9 | -| org.apache.hive | hive-common | 2.3.9 | -| org.apache.hive | hive-exec | 2.3.9 | -| org.apache.hive | hive-jdbc | 2.3.9 | -| org.apache.hive | hive-llap-common | 2.3.9 | -| org.apache.hive | hive-metastore | 2.3.9 | -| org.apache.hive | hive-serde | 2.3.9 | -| org.apache.hive | hive-service-rpc | 2.3.9 | -| org.apache.hive | hive-shims-0.23 | 2.3.9 | -| org.apache.hive | hive-shims | 2.3.9 | -| org.apache.hive | hive-shims-common | 2.3.9 | -| org.apache.hive | hive-shims-scheduler | 2.3.9 | -| org.apache.hive | hive-storage-api | 2.7.2 | -| org.apache.httpcomponents | httpclient | 4.5.13 | -| org.apache.httpcomponents | httpcore | 4.4.14 | -| org.apache.httpcomponents | httpmime | 4.5.13 | -| org.apache.httpcomponents.client5 | httpclient5 | 5.1.3 | -| org.apache.iceberg | delta-iceberg | 2.2.0.9 | -| org.apache.ivy | ivy | 2.5.1 | -| org.apache.kafka | kafka-clients | 2.8.1 | -| org.apache.logging.log4j | log4j-1.2-api | 2.17.2 | -| org.apache.logging.log4j | log4j-api | 2.17.2 | -| org.apache.logging.log4j | log4j-core | 2.17.2 | -| org.apache.logging.log4j | log4j-slf4j-impl | 2.17.2 | -| org.apache.orc | orc-core | 1.7.6 | -| org.apache.orc | orc-mapreduce | 1.7.6 | -| org.apache.orc | orc-shims | 1.7.6 | -| org.apache.parquet | parquet-column | 1.12.3 | -| org.apache.parquet | parquet-common | 1.12.3 | -| org.apache.parquet | parquet-encoding | 1.12.3 | -| org.apache.parquet | parquet-format-structures | 1.12.3 | -| org.apache.parquet | parquet-hadoop | 1.12.3 | -| org.apache.parquet | parquet-jackson | 1.12.3 | -| org.apache.qpid | proton-j | 0.33.8 | -| org.apache.spark | spark-avro_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-catalyst_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-core_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-graphx_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-hadoop-cloud_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-hive_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-kvstore_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-launcher_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-mllib_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-mllib-local_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-network-common_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-network-shuffle_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-repl_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-sketch_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-sql_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-sql-kafka-0-10_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-streaming_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-streaming-kafka-0-10-assembly_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-tags_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-token-provider-kafka-0-10_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-unsafe_2.12 | 3.3.1.5.2-106693326 | -| org.apache.spark | spark-yarn_2.12 | 3.3.1.5.2-106693326 | -| org.apache.thrift | libfb303 | 0.9.3 | -| org.apache.thrift | libthrift | 0.12.0 | -| org.apache.velocity | velocity | 1.5 | -| org.apache.xbean | xbean-asm9-shaded | 4.2 | -| org.apache.yetus | audience-annotations | 0.5.0 | -| org.apache.zookeeper | zookeeper | 3.6.2.5.2-106693326 | -| org.apache.zookeeper | zookeeper-jute | 3.6.2.5.2-106693326 | -| org.apache.zookeeper | zookeeper | 3.6.2.5.2-106693326 | -| org.apache.zookeeper | zookeeper-jute | 3.6.2.5.2-106693326 | -| org.apiguardian | apiguardian-api | 1.1.0 | -| org.codehaus.janino | commons-compiler | 3.0.16 | -| org.codehaus.janino | janino | 3.0.16 | -| org.codehaus.jettison | jettison | 1.1 | -| org.datanucleus | datanucleus-api-jdo | 4.2.4 | -| org.datanucleus | datanucleus-core | 4.1.17 | -| org.datanucleus | datanucleus-rdbms | 4.1.19 | -| org.datanucleusjavax.jdo | | 3.2.0-m3 | -| org.eclipse.jetty | jetty-util | 9.4.48.v20220622 | -| org.eclipse.jetty | jetty-util-ajax | 9.4.48.v20220622 | -| org.fusesource.leveldbjni | leveldbjni-all | 1.8 | -| org.glassfish.hk2 | hk2-api | 2.6.1 | -| org.glassfish.hk2 | hk2-locator | 2.6.1 | -| org.glassfish.hk2 | hk2-utils | 2.6.1 | -| org.glassfish.hk2 | osgi-resource-locator | 1.0.3 | -| org.glassfish.hk2.external | aopalliance-repackaged | 2.6.1 | -| org.glassfish.jaxb | jaxb-runtime | 2.3.2 | -| org.glassfish.jersey.containers | jersey-container-servlet | 2.36 | -| org.glassfish.jersey.containers | jersey-container-servlet-core | 2.36 | -| org.glassfish.jersey.core | jersey-client | 2.36 | -| org.glassfish.jersey.core | jersey-common | 2.36 | -| org.glassfish.jersey.core | jersey-server | 2.36 | -| org.glassfish.jersey.inject | jersey-hk2 | 2.36 | -| org.ini4j | ini4j | 0.5.4 | -| org.javassist | javassist | 3.25.0-GA | -| org.javatuples | javatuples | 1.2 | -| org.jdom | jdom2 | 2.0.6 | -| org.jetbrains | annotations | 17.0.0 | -| org.jodd | jodd-core | 3.5.2 | -| org.json | json | 20210307 | -| org.json4s | json4s-ast_2.12 | 3.7.0-M11 | -| org.json4s | json4s-core_2.12 | 3.7.0-M11 | -| org.json4s | json4s-jackson_2.12 | 3.7.0-M11 | -| org.json4s | json4s-scalap_2.12 | 3.7.0-M11 | -| org.junit.jupiter | junit-jupiter | 5.5.2 | -| org.junit.jupiter | junit-jupiter-api | 5.5.2 | -| org.junit.jupiter | junit-jupiter-engine | 5.5.2 | -| org.junit.jupiter | junit-jupiter-params | 5.5.2 | -| org.junit.platform | junit-platform-commons | 1.5.2 | -| org.junit.platform | junit-platform-engine | 1.5.2 | -| org.lz4 | lz4-java | 1.8.0 | -| org.mlflow | mlfow-spark | 2.1.1 | -| org.objenesis | objenesis | 3.2 | -| org.openpnp | opencv | 3.2.0-1 | -| org.opentest4j | opentest4j | 1.2.0 | -| org.postgresql | postgresql | 42.2.9 | -| org.roaringbitmap | RoaringBitmap | 0.9.25 | -| org.roaringbitmap | shims | 0.9.25 | -| org.rocksdb | rocksdbjni | 6.20.3 | -| org.scalactic | scalactic_2.12 | 3.2.14 | -| org.scala-lang | scala-compiler | 2.12.15 | -| org.scala-lang | scala-library | 2.12.15 | -| org.scala-lang | scala-reflect | 2.12.15 | -| org.scala-lang.modules | scala-collection-compat_2.12 | 2.1.1 | -| org.scala-lang.modules | scala-java8-compat_2.12 | 0.9.0 | -| org.scala-lang.modules | scala-parser-combinators_2.12 | 1.1.2 | -| org.scala-lang.modules | scala-xml_2.12 | 1.2.0 | -| org.scalanlp | breeze-macros_2.12 | 1.2 | -| org.scalanlp | breeze_2.12 | 1.2 | -| org.slf4j | jcl-over-slf4j | 1.7.32 | -| org.slf4j | jul-to-slf4j | 1.7.32 | -| org.slf4j | slf4j-api | 1.7.32 | -| org.threeten | threeten-extra | 1.5.0 | -| org.tukaani | xz | 1.8 | -| org.typelevel | algebra_2.12 | 2.0.1 | -| org.typelevel | cats-kernel_2.12 | 2.1.1 | -| org.typelevel | spire_2.12 | 0.17.0 | -| org.typelevel | spire-macros_2.12 | 0.17.0 | -| org.typelevel | spire-platform_2.12 | 0.17.0 | -| org.typelevel | spire-util_2.12 | 0.17.0 | -| org.wildfly.openssl | wildfly-openssl | 1.0.7.Final | -| org.xerial.snappy | snappy-java | 1.1.8.4 | -| oro | oro | 2.0.8 | -| pl.edu.icm | JLargeArrays | 1.5 | -| stax | stax-api | 1.0.1 | --### Python libraries --The Azure Synapse Runtime for Apache Spark 3.4 is currently in Public Preview. During this phase, the Python libraries experience significant updates. Additionally, please note that some machine learning capabilities aren't yet supported, such as the PREDICT method and Synapse ML. --### R libraries --The following table lists all the default level packages for R and their respective versions. --| Library | Version | Library | Version | Library | Version | -||--|--|||| -| _libgcc_mutex | 0.1 | r-caret | 6.0_94 | r-praise | 1.0.0 | -| _openmp_mutex | 4.5 | r-cellranger | 1.1.0 | r-prettyunits | 1.2.0 | -| _r-mutex | 1.0.1 | r-class | 7.3_22 | r-proc | 1.18.4 | -| _r-xgboost-mutex | 2 | r-cli | 3.6.1 | r-processx | 3.8.2 | -| aws-c-auth | 0.7.0 | r-clipr | 0.8.0 | r-prodlim | 2023.08.28 | -| aws-c-cal | 0.6.0 | r-clock | 0.7.0 | r-profvis | 0.3.8 | -| aws-c-common | 0.8.23 | r-codetools | 0.2_19 | r-progress | 1.2.2 | -| aws-c-compression | 0.2.17 | r-collections | 0.3.7 | r-progressr | 0.14.0 | -| aws-c-event-stream | 0.3.1 | r-colorspace | 2.1_0 | r-promises | 1.2.1 | -| aws-c-http | 0.7.10 | r-commonmark | 1.9.0 | r-proxy | 0.4_27 | -| aws-c-io | 0.13.27 | r-config | 0.3.2 | r-pryr | 0.1.6 | -| aws-c-mqtt | 0.8.13 | r-conflicted | 1.2.0 | r-ps | 1.7.5 | -| aws-c-s3 | 0.3.12 | r-coro | 1.0.3 | r-purrr | 1.0.2 | -| aws-c-sdkutils | 0.1.11 | r-cpp11 | 0.4.6 | r-quantmod | 0.4.25 | -| aws-checksums | 0.1.16 | r-crayon | 1.5.2 | r-r2d3 | 0.2.6 | -| aws-crt-cpp | 0.20.2 | r-credentials | 2.0.1 | r-r6 | 2.5.1 | -| aws-sdk-cpp | 1.10.57 | r-crosstalk | 1.2.0 | r-r6p | 0.3.0 | -| binutils_impl_linux-64 | 2.4 | r-crul | 1.4.0 | r-ragg | 1.2.6 | -| bwidget | 1.9.14 | r-curl | 5.1.0 | r-rappdirs | 0.3.3 | -| bzip2 | 1.0.8 | r-data.table | 1.14.8 | r-rbokeh | 0.5.2 | -| c-ares | 1.20.1 | r-dbi | 1.1.3 | r-rcmdcheck | 1.4.0 | -| ca-certificates | 2023.7.22 | r-dbplyr | 2.3.4 | r-rcolorbrewer | 1.1_3 | -| cairo | 1.18.0 | r-desc | 1.4.2 | r-rcpp | 1.0.11 | -| cmake | 3.27.6 | r-devtools | 2.4.5 | r-reactable | 0.4.4 | -| curl | 8.4.0 | r-diagram | 1.6.5 | r-reactr | 0.5.0 | -| expat | 2.5.0 | r-dials | 1.2.0 | r-readr | 2.1.4 | -| font-ttf-dejavu-sans-mono | 2.37 | r-dicedesign | 1.9 | r-readxl | 1.4.3 | -| font-ttf-inconsolata | 3 | r-diffobj | 0.3.5 | r-recipes | 1.0.8 | -| font-ttf-source-code-pro | 2.038 | r-digest | 0.6.33 | r-rematch | 2.0.0 | -| font-ttf-ubuntu | 0.83 | r-downlit | 0.4.3 | r-rematch2 | 2.1.2 | -| fontconfig | 2.14.2 | r-dplyr | 1.1.3 | r-remotes | 2.4.2.1 | -| fonts-conda-ecosystem | 1 | r-dtplyr | 1.3.1 | r-reprex | 2.0.2 | -| fonts-conda-forge | 1 | r-e1071 | 1.7_13 | r-reshape2 | 1.4.4 | -| freetype | 2.12.1 | r-ellipsis | 0.3.2 | r-rjson | 0.2.21 | -| fribidi | 1.0.10 | r-evaluate | 0.23 | r-rlang | 1.1.1 | -| gcc_impl_linux-64 | 13.2.0 | r-fansi | 1.0.5 | r-rlist | 0.4.6.2 | -| gettext | 0.21.1 | r-farver | 2.1.1 | r-rmarkdown | 2.22 | -| gflags | 2.2.2 | r-fastmap | 1.1.1 | r-rodbc | 1.3_20 | -| gfortran_impl_linux-64 | 13.2.0 | r-fontawesome | 0.5.2 | r-roxygen2 | 7.2.3 | -| glog | 0.6.0 | r-forcats | 1.0.0 | r-rpart | 4.1.21 | -| glpk | 5 | r-foreach | 1.5.2 | r-rprojroot | 2.0.3 | -| gmp | 6.2.1 | r-forge | 0.2.0 | r-rsample | 1.2.0 | -| graphite2 | 1.3.13 | r-fs | 1.6.3 | r-rstudioapi | 0.15.0 | -| gsl | 2.7 | r-furrr | 0.3.1 | r-rversions | 2.1.2 | -| gxx_impl_linux-64 | 13.2.0 | r-future | 1.33.0 | r-rvest | 1.0.3 | -| harfbuzz | 8.2.1 | r-future.apply | 1.11.0 | r-sass | 0.4.7 | -| icu | 73.2 | r-gargle | 1.5.2 | r-scales | 1.2.1 | -| kernel-headers_linux-64 | 2.6.32 | r-generics | 0.1.3 | r-selectr | 0.4_2 | -| keyutils | 1.6.1 | r-gert | 2.0.0 | r-sessioninfo | 1.2.2 | -| krb5 | 1.21.2 | r-ggplot2 | 3.4.2 | r-shape | 1.4.6 | -| ld_impl_linux-64 | 2.4 | r-gh | 1.4.0 | r-shiny | 1.7.5.1 | -| lerc | 4.0.0 | r-gistr | 0.9.0 | r-slider | 0.3.1 | -| libabseil | 20230125 | r-gitcreds | 0.1.2 | r-sourcetools | 0.1.7_1 | -| libarrow | 12.0.0 | r-globals | 0.16.2 | r-sparklyr | 1.8.2 | -| libblas | 3.9.0 | r-glue | 1.6.2 | r-squarem | 2021.1 | -| libbrotlicommon | 1.0.9 | r-googledrive | 2.1.1 | r-stringi | 1.7.12 | -| libbrotlidec | 1.0.9 | r-googlesheets4 | 1.1.1 | r-stringr | 1.5.0 | -| libbrotlienc | 1.0.9 | r-gower | 1.0.1 | r-survival | 3.5_7 | -| libcblas | 3.9.0 | r-gpfit | 1.0_8 | r-sys | 3.4.2 | -| libcrc32c | 1.1.2 | r-gt | 0.9.0 | r-systemfonts | 1.0.5 | -| libcurl | 8.4.0 | r-gtable | 0.3.4 | r-testthat | 3.2.0 | -| libdeflate | 1.19 | r-gtsummary | 1.7.2 | r-textshaping | 0.3.7 | -| libedit | 3.1.20191231 | r-hardhat | 1.3.0 | r-tibble | 3.2.1 | -| libev | 4.33 | r-haven | 2.5.3 | r-tidymodels | 1.1.0 | -| libevent | 2.1.12 | r-hexbin | 1.28.3 | r-tidyr | 1.3.0 | -| libexpat | 2.5.0 | r-highcharter | 0.9.4 | r-tidyselect | 1.2.0 | -| libffi | 3.4.2 | r-highr | 0.1 | r-tidyverse | 2.0.0 | -| libgcc-devel_linux-64 | 13.2.0 | r-hms | 1.1.3 | r-timechange | 0.2.0 | -| libgcc-ng | 13.2.0 | r-htmltools | 0.5.6.1 | r-timedate | 4022.108 | -| libgfortran-ng | 13.2.0 | r-htmlwidgets | 1.6.2 | r-tinytex | 0.48 | -| libgfortran5 | 13.2.0 | r-httpcode | 0.3.0 | r-torch | 0.11.0 | -| libgit2 | 1.7.1 | r-httpuv | 1.6.12 | r-triebeard | 0.4.1 | -| libglib | 2.78.0 | r-httr | 1.4.7 | r-ttr | 0.24.3 | -| libgomp | 13.2.0 | r-httr2 | 0.2.3 | r-tune | 1.1.2 | -| libgoogle-cloud | 2.12.0 | r-ids | 1.0.1 | r-tzdb | 0.4.0 | -| libgrpc | 1.55.1 | r-igraph | 1.5.1 | r-urlchecker | 1.0.1 | -| libiconv | 1.17 | r-infer | 1.0.5 | r-urltools | 1.7.3 | -| libjpeg-turbo | 3.0.0 | r-ini | 0.3.1 | r-usethis | 2.2.2 | -| liblapack | 3.9.0 | r-ipred | 0.9_14 | r-utf8 | 1.2.4 | -| libnghttp2 | 1.55.1 | r-isoband | 0.2.7 | r-uuid | 1.1_1 | -| libnuma | 2.0.16 | r-iterators | 1.0.14 | r-v8 | 4.4.0 | -| libopenblas | 0.3.24 | r-jose | 1.2.0 | r-vctrs | 0.6.4 | -| libpng | 1.6.39 | r-jquerylib | 0.1.4 | r-viridislite | 0.4.2 | -| libprotobuf | 4.23.2 | r-jsonlite | 1.8.7 | r-vroom | 1.6.4 | -| libsanitizer | 13.2.0 | r-juicyjuice | 0.1.0 | r-waldo | 0.5.1 | -| libssh2 | 1.11.0 | r-kernsmooth | 2.23_22 | r-warp | 0.2.0 | -| libstdcxx-devel_linux-64 | 13.2.0 | r-knitr | 1.45 | r-whisker | 0.4.1 | -| libstdcxx-ng | 13.2.0 | r-labeling | 0.4.3 | r-withr | 2.5.2 | -| libthrift | 0.18.1 | r-labelled | 2.12.0 | r-workflows | 1.1.3 | -| libtiff | 4.6.0 | r-later | 1.3.1 | r-workflowsets | 1.0.1 | -| libutf8proc | 2.8.0 | r-lattice | 0.22_5 | r-xfun | 0.41 | -| libuuid | 2.38.1 | r-lava | 1.7.2.1 | r-xgboost | 1.7.4 | -| libuv | 1.46.0 | r-lazyeval | 0.2.2 | r-xml | 3.99_0.14 | -| libv8 | 8.9.83 | r-lhs | 1.1.6 | r-xml2 | 1.3.5 | -| libwebp-base | 1.3.2 | r-lifecycle | 1.0.3 | r-xopen | 1.0.0 | -| libxcb | 1.15 | r-lightgbm | 3.3.5 | r-xtable | 1.8_4 | -| libxgboost | 1.7.4 | r-listenv | 0.9.0 | r-xts | 0.13.1 | -| libxml2 | 2.11.5 | r-lobstr | 1.1.2 | r-yaml | 2.3.7 | -| libzlib | 1.2.13 | r-lubridate | 1.9.3 | r-yardstick | 1.2.0 | -| lz4-c | 1.9.4 | r-magrittr | 2.0.3 | r-zip | 2.3.0 | -| make | 4.3 | r-maps | 3.4.1 | r-zoo | 1.8_12 | -| ncurses | 6.4 | r-markdown | 1.11 | rdma-core | 28.9 | -| openssl | 3.1.4 | r-mass | 7.3_60 | re2 | 2023.03.02 | -| orc | 1.8.4 | r-matrix | 1.6_1.1 | readline | 8.2 | -| pandoc | 2.19.2 | r-memoise | 2.0.1 | rhash | 1.4.4 | -| pango | 1.50.14 | r-mgcv | 1.9_0 | s2n | 1.3.46 | -| pcre2 | 10.4 | r-mime | 0.12 | sed | 4.8 | -| pixman | 0.42.2 | r-miniui | 0.1.1.1 | snappy | 1.1.10 | -| pthread-stubs | 0.4 | r-modeldata | 1.2.0 | sysroot_linux-64 | 2.12 | -| r-arrow | 12.0.0 | r-modelenv | 0.1.1 | tk | 8.6.13 | -| r-askpass | 1.2.0 | r-modelmetrics | 1.2.2.2 | tktable | 2.1 | -| r-assertthat | 0.2.1 | r-modelr | 0.1.11 | ucx | 1.14.1 | -| r-backports | 1.4.1 | r-munsell | 0.5.0 | unixodbc | 2.3.12 | -| r-base | 4.2.3 | r-nlme | 3.1_163 | xorg-kbproto | 1.0.7 | -| r-base64enc | 0.1_3 | r-nnet | 7.3_19 | xorg-libice | 1.1.1 | -| r-bigd | 0.2.0 | r-numderiv | 2016.8_1.1 | xorg-libsm | 1.2.4 | -| r-bit | 4.0.5 | r-openssl | 2.1.1 | xorg-libx11 | 1.8.7 | -| r-bit64 | 4.0.5 | r-parallelly | 1.36.0 | xorg-libxau | 1.0.11 | -| r-bitops | 1.0_7 | r-parsnip | 1.1.1 | xorg-libxdmcp | 1.1.3 | -| r-blob | 1.2.4 | r-patchwork | 1.1.3 | xorg-libxext | 1.3.4 | -| r-brew | 1.0_8 | r-pillar | 1.9.0 | xorg-libxrender | 0.9.11 | -| r-brio | 1.1.3 | r-pkgbuild | 1.4.2 | xorg-libxt | 1.3.0 | -| r-broom | 1.0.5 | r-pkgconfig | 2.0.3 | xorg-renderproto | 0.11.1 | -| r-broom.helpers | 1.14.0 | r-pkgdown | 2.0.7 | xorg-xextproto | 7.3.0 | -| r-bslib | 0.5.1 | r-pkgload | 1.3.3 | xorg-xproto | 7.0.31 | -| r-cachem | 1.0.8 | r-plotly | 4.10.2 | xz | 5.2.6 | -| r-callr | 3.7.3 | r-plyr | 1.8.9 | zlib | 1.2.13 | -| | | | | zstd | 1.5.5 | --## Migration between Apache Spark versions - support --For guidance on migrating from older runtime versions to Azure Synapse Runtime for Apache Spark 3.4, refer to [Runtime for Apache Spark Overview](./apache-spark-version-support.md). +## Related content +- [Migration between Apache Spark versions - support](./apache-spark-version-support.md#migration-between-apache-spark-versionssupport) +- [Synapse runtime for Apache Spark lifecycle and supportability](./runtime-for-apache-spark-lifecycle-and-supportability.md) |
| synapse-analytics | Apache Spark Azure Log Analytics | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/apache-spark-azure-log-analytics.md | -> This feature is currently unavailable in the Spark 3.4 runtime but will be supported post-GA. +> This feature is currently unavailable in the [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) but will be supported post-GA. ## Configure workspace information |
| synapse-analytics | Apache Spark Concepts | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/apache-spark-concepts.md | You can read how to create a Spark pool and see all their properties here [Get s Spark instances are created when you connect to a Spark pool, create a session, and run a job. As multiple users may have access to a single Spark pool, a new Spark instance is created for each user that connects. -When you submit a second job, if there's capacity in the pool, the existing Spark instance also has capacity. Then, the existing instance will process the job. Otherwise, if capacity is available at the pool level, then a new Spark instance will be created. +When you submit a second job, if there's capacity in the pool, the existing Spark instance also has capacity. Then, the existing instance processes the job. Otherwise, if capacity is available at the pool level, a new Spark instance is created. Billing for the instances starts when the Azure VM(s) starts. Billing for the Spark pool instances stops when pool instances change to terminating. For more information on how Azure VMs are started and deallocated, see [States and billing status of Azure Virtual Machines](/azure/virtual-machines/states-billing). Billing for the instances starts when the Azure VM(s) starts. Billing for the Sp - You create a Spark pool called SP1; it has a fixed cluster size of 20 medium nodes - You submit a notebook job, J1 that uses 10 nodes, a Spark instance, SI1 is created to process the job - You now submit another job, J2, that uses 10 nodes because there's still capacity in the pool and the instance, the J2, is processed by SI1-- If J2 had asked for 11 nodes, there wouldn't have been capacity in SP1 or SI1. In this case, if J2 comes from a notebook, then the job will be rejected; if J2 comes from a batch job, then it will be queued.+- If J2 had asked for 11 nodes, there wouldn't have been capacity in SP1 or SI1. In this case, if J2 comes from a notebook, then the job is rejected; if J2 comes from a batch job, it is queued. - Billing starts at the submission of notebook job J1. - The Spark pool is instantiated with 20 medium nodes, each with 8 vCores, and typically takes ~3 minutes to start. 20 x 8 = 160 vCores. - Depending on the exact Spark pool start-up time, idle timeout and the runtime of the two notebook jobs; the pool is likely to run for between 18 and 20 minutes (Spark pool instantiation time + notebook job runtime + idle timeout). - Assuming 20-minute runtime, 160 x 0.3 hours = 48 vCore hours.- - Note: vCore hours are billed per second, vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing) + - Note: vCore hours are billed per minute and vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing) ### Example 2 Billing for the instances starts when the Azure VM(s) starts. Billing for the Sp - At the submission of J2, the pool autoscales by adding another 10 medium nodes, and typically takes 4 minutes to autoscale. Adding 10 x 8, 80 vCores for a total of 160 vCores. - Depending on the Spark pool start-up time, runtime of the first notebook job J1, the time to scale-up the pool, runtime of the second notebook, and finally the idle timeout; the pool is likely to run between 22 and 24 minutes (Spark pool instantiation time + J1 notebook job runtime all at 80 vCores) + (Spark pool autoscale-up time + J2 notebook job runtime + idle timeout all at 160 vCores). - 80 vCores for 4 minutes + 160 vCores for 20 minutes = 58.67 vCore hours.- - Note: vCore hours are billed per second, vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing) + - Note: vCore hours are billed per minute and vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing) ### Example 3 Billing for the instances starts when the Azure VM(s) starts. Billing for the Sp - Another Spark pool SI2 is instantiated with 20 medium nodes, each with 8 vCores, and typically takes ~3 minutes to start. 20 x 8, 160 vCores - Depending on the exact Spark pool start-up time, the ide timeout and the runtime of the first notebook job; The SI2 pool is likely to run for between 18 and 20 minutes (Spark pool instantiation time + notebook job runtime + idle timeout). - Assuming the two pools run for 20 minutes each, 160 x .03 x 2 = 96 vCore hours.- - Note: vCore hours are billed per second, vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing) + - Note: vCore hours are billed per minute and vCore pricing varies by Azure region. For more information, see [Azure Synapse Pricing](https://azure.microsoft.com/pricing/details/synapse-analytics/#pricing) ## Quotas and resource constraints in Apache Spark for Azure Synapse |
| synapse-analytics | Apache Spark External Metastore | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/apache-spark-external-metastore.md | Last updated 02/15/2022 # Use external Hive Metastore for Synapse Spark Pool > [!NOTE]-> External Hive metastores will no longer be supported in Spark 3.4 and subsequent versions in Synapse. +> External Hive metastores will no longer be supported in [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) and subsequent versions in Synapse. Azure Synapse Analytics allows Apache Spark pools in the same workspace to share a managed HMS (Hive Metastore) compatible metastore as their catalog. When customers want to persist the Hive catalog metadata outside of the workspace, and share catalog objects with other computational engines outside of the workspace, such as HDInsight and Azure Databricks, they can connect to an external Hive Metastore. In this article, you can learn how to connect Synapse Spark to an external Apache Hive Metastore. try { ``` ## Configure Spark to use the external Hive Metastore-After creating the linked service to the external Hive Metastore successfully, you need to setup a few Spark configurations to use the external Hive Metastore. You can both set up the configuration at Spark pool level, or at Spark session level. +After creating the linked service to the external Hive Metastore successfully, you need to set up a few Spark configurations to use the external Hive Metastore. You can both set up the configuration at Spark pool level, or at Spark session level. Here are the configurations and descriptions: |
| synapse-analytics | Apache Spark Version Support | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/apache-spark-version-support.md | The runtimes have the following advantages: > End of Support Notification for Azure Synapse Runtime for Apache Spark 2.4 and Apache Spark 3.1. > * Effective September 29, 2023, Azure Synapse will discontinue official support for Spark 2.4 Runtimes. > * Effective January 26, 2024, Azure Synapse will discontinue official support for Spark 3.1 Runtimes. -> * After these dates, we will not be addressing any support tickets related to Spark 2.4 or 3.1. There will be no release pipeline in place for bug or security fixes for Spark 2.4 and 3.1. Utilizing Spark 2.4 or 3.1 post the support cutoff dates is undertaken at one's own risk. We strongly discourage its continued use due to potential security and functionality concerns. +> * After these dates, we will not be addressing any support tickets related to Spark 2.4 or 3.1. There will be no release pipeline in place for bug or security fixes for Spark 2.4 and 3.1. **Utilizing Spark 2.4 or 3.1 post the support cutoff dates is undertaken at one's own risk. We strongly discourage its continued use due to potential security and functionality concerns.** > [!TIP]-> We strongly recommend proactively upgrading workloads to a more recent version of the runtime (for example, [Azure Synapse Runtime for Apache Spark 3.3 (GA)](./apache-spark-33-runtime.md)). Refer to the [Apache Spark migration guide](https://spark.apache.org/docs/latest/sql-migration-guide.html). +> We strongly recommend proactively upgrading workloads to a more recent GA version of the runtime (for example, [Azure Synapse Runtime for Apache Spark 3.4 (GA)](./apache-spark-34-runtime.md)). Refer to the [Apache Spark migration guide](https://spark.apache.org/docs/latest/sql-migration-guide.html). The following table lists the runtime name, Apache Spark version, and release date for supported Azure Synapse Runtime releases. -| Runtime name | Release date | Release stage | End of Support announcement date | End of Support effective date | -| | | | | | -| [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) | Nov 21, 2023 | Public Preview | | | -| [Azure Synapse Runtime for Apache Spark 3.3](./apache-spark-33-runtime.md) | Nov 17, 2022 | GA (as of Feb 23, 2023) | Q2/Q3 2024 | Q1 2025 | +| Runtime name | Release date | Release stage | End of Support announcement date | End of Support effective date | +| | || | | +| [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) | Nov 21, 2023 | GA (as of Apr 8, 2024) | | | +| [Azure Synapse Runtime for Apache Spark 3.3](./apache-spark-33-runtime.md) | Nov 17, 2022 | GA (as of Feb 23, 2023) | Q2/Q3 2024 | Q1 2025 | | [Azure Synapse Runtime for Apache Spark 3.2](./apache-spark-32-runtime.md) | July 8, 2022 | __End of Support Announced__ | July 8, 2023 | July 8, 2024 |-| [Azure Synapse Runtime for Apache Spark 3.1](./apache-spark-3-runtime.md) | May 26, 2021 | __End of Support__ | January 26, 2023 | January 26, 2024 | -| [Azure Synapse Runtime for Apache Spark 2.4](./apache-spark-24-runtime.md) | December 15, 2020 | __End of Support__ | __July 29, 2022__ | __September 29, 2023__ | +| [Azure Synapse Runtime for Apache Spark 3.1](./apache-spark-3-runtime.md) | May 26, 2021 | __End of Support__ | January 26, 2023 | January 26, 2024 | +| [Azure Synapse Runtime for Apache Spark 2.4](./apache-spark-24-runtime.md) | December 15, 2020 | __End of Support__ | __July 29, 2022__ | __September 29, 2023__ | ## Runtime release stages The patch policy differs based on the [runtime lifecycle stage](./runtime-for-ap - End of Support announced runtime won't have bug and feature fixes. Security fixes are backported based on risk assessment. + ## Migration between Apache Spark versions - support -General Upgrade guidelines/ FAQs: +This guide provides a structured approach for users looking to upgrade their Azure Synapse Runtime for Apache Spark workloads from versions 2.4, 3.1, 3.2, or 3.3 to [the latest GA version, such as 3.4](./apache-spark-34-runtime.md). Upgrading to the most recent version enables users to benefit from performance enhancements, new features, and improved security measures. It is important to note that transitioning to a higher version may require adjustments to your existing Spark code due to incompatibilities or deprecated features. ++### Step 1: Evaluate and plan +- **Assess Compatibility:** Start with reviewing Apache Spark migration guides to identify any potential incompatibilities, deprecated features, and new APIs between your current Spark version (2.4, 3.1, 3.2, or 3.3) and the target version (e.g., 3.4). +- **Analyze Codebase:** Carefully examine your Spark code to identify the use of deprecated or modified APIs. Pay particular attention to SQL queries and User Defined Functions (UDFs), which may be affected by the upgrade. ++### Step 2: Create a new Spark pool for testing +- **Create a New Pool:** In Azure Synapse, go to the Spark pools section and set up a new Spark pool. Select the target Spark version (e.g., 3.4) and configure it according to your performance requirements. +- **Configure Spark Pool Configuration:** Ensure that all libraries and dependencies in your new Spark pool are updated or replaced to be compatible with Spark 3.4. ++### Step 3: Migrate and test your code +- **Migrate Code:** Update your code to be compliant with the new or revised APIs in Apache Spark 3.4. This involves addressing deprecated functions and adopting new features as detailed in the official Apache Spark documentation. +- **Test in Development Environment:** Test your updated code within a development environment in Azure Synapse, not locally. This step is essential for identifying and fixing any issues before moving to production. +- **Deploy and Monitor:** After thorough testing and validation in the development environment, deploy your application to the new Spark 3.4 pool. It is critical to monitor the application for any unexpected behaviors. Utilize the monitoring tools available in Azure Synapse to keep track of your Spark applications' performance. **Question:** What steps should be taken in migrating from 2.4 to 3.X? -**Answer:** Refer to the [Apache Spark migration guide](https://spark.apache.org/docs/latest/sql-migration-guide.html). +**Answer:** Refer to the [Apache Spark migration guide](https://spark.apache.org/docs/latest/sql-migration-guide.html). **Question:** I got an error when I tried to upgrade Spark pool runtime using PowerShell cmdlet when they have attached libraries. **Answer:** Don't use PowerShell cmdlet if you have custom libraries installed in your Synapse workspace. Instead follow these steps:- 1. Recreate Spark Pool 3.3 from the ground up. - 1. Downgrade the current Spark Pool 3.3 to 3.1, remove any packages attached, and then upgrade again to 3.3. +1. Recreate Spark Pool 3.3 from the ground up. +1. Downgrade the current Spark Pool 3.3 to 3.1, remove any packages attached, and then upgrade again to 3.3. ## Related content - [Manage libraries for Apache Spark in Azure Synapse Analytics](apache-spark-azure-portal-add-libraries.md)-- [Synapse runtime for Apache Spark lifecycle and supportability](runtime-for-apache-spark-lifecycle-and-supportability.md)+- [Synapse runtime for Apache Spark lifecycle and supportability](runtime-for-apache-spark-lifecycle-and-supportability.md) |
| synapse-analytics | Azure Synapse Diagnostic Emitters Azure Eventhub | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/azure-synapse-diagnostic-emitters-azure-eventhub.md | The Synapse Apache Spark diagnostic emitter extension is a library that enables In this tutorial, you learn how to use the Synapse Apache Spark diagnostic emitter extension to emit Apache Spark applicationsΓÇÖ logs, event logs, and metrics to your Azure Event Hubs. > [!NOTE]-> This feature is currently unavailable in the Spark 3.4 runtime but will be supported post-GA. +> This feature is currently unavailable in the [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) runtime but will be supported post-GA. ## Collect logs and metrics to Azure Event Hubs |
| synapse-analytics | Azure Synapse Diagnostic Emitters Azure Storage | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/azure-synapse-diagnostic-emitters-azure-storage.md | The Synapse Apache Spark diagnostic emitter extension is a library that enables In this tutorial, you learn how to use the Synapse Apache Spark diagnostic emitter extension to emit Apache Spark applicationsΓÇÖ logs, event logs, and metrics to your Azure storage account. > [!NOTE]-> This feature is currently unavailable in the Spark 3.4 runtime but will be supported post-GA. +> This feature is currently unavailable in the [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) runtime but will be supported post-GA. ## Collect logs and metrics to storage account |
| synapse-analytics | Microsoft Spark Utilities | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/synapse-analytics/spark/microsoft-spark-utilities.md | mssparkutils.fs.fastcp('source file or directory', 'destination file or director ``` > [!NOTE]-> The method only supports in Spark 3.3 and Spark 3.4. +> The method only supports in [Azure Synapse Runtime for Apache Spark 3.3](./apache-spark-33-runtime.md) and [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md). ### Preview file content mssparkutils.notebook.runMultiple(DAG) > [!NOTE] >-> - The method only supports in Spark 3.3 and Spark 3.4. +> - The method only supports in [Azure Synapse Runtime for Apache Spark 3.3](./apache-spark-33-runtime.md) and [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md). > - The parallelism degree of the multiple notebook run is restricted to the total available compute resource of a Spark session. |
| trusted-signing | How To Sign Ci Policy | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/trusted-signing/how-to-sign-ci-policy.md | + + Title: Signing CI Policies #Required; page title is displayed in search results. Include the brand. +description: Learn how to sign new CI policies with Trusted Signing. #Required; article description that is displayed in search results. ++++ Last updated : 04/04/2024 #Required; mm/dd/yyyy format.++++# Sign CI Policies with Trusted Signing ++To sign new CI policies with the service first install several prerequisites. +++Prerequisites: +* A Trusted Signing account, Identity Validation, and Certificate Profile. +* Ensure there are proper individual or group role assignments for signing (ΓÇ£Trusted Signing Certificate Profile SignerΓÇ¥ role). +* [Azure PowerShell on Windows](https://learn.microsoft.com/powershell/azure/install-azps-windows) installed +* [Az.CodeSigning](https://learn.microsoft.com/powershell/module/az.codesigning/) module downloaded ++Overview of steps: +1. ΓüáUnzip the Az.CodeSigning module to a folder +2. ΓüáOpen Windows PowerShell [PowerShell 7](https://github.com/PowerShell/PowerShell/releases/latest) +3. In the Az.CodeSigning folder, run +``` +Import-Module .\Az.CodeSigning.psd1 +``` +4. Optionally you can create a `metadata.json` file: +``` +Endpoint "https://scus.codesigning.azure.net/" +CodeSigningAccountName "youracsaccount" +CertificateProfileName "youracscertprofile" +``` +5. [Get the root certificate](https://learn.microsoft.com/powershell/module/az.codesigning/get-azcodesigningrootcert) to be added to the trust store +``` +Get-AzCodeSigningRootCert -AccountName TestAccount -ProfileName TestCertProfile -EndpointUrl https://xxx.codesigning.azure.net/ -Destination c:\temp\root.cer +``` +Or using a metadata.json +``` +Get-AzCodeSigningRootCert -MetadataFilePath C:\temp\metadata.sample.scus.privateci.json https://xxx.codesigning.azure.net/ -Destination c:\temp\root.cer +``` +6. To get the EKU (Extended Key Usage) to insert into your policy: +``` +Get-AzCodeSigningCustomerEku -AccountName acstestcanary -ProfileName acstestcanaryCert1 -EndpointUrl https://xxx.codesigning.azure.net/ +``` +Or ++``` +Get-AzCodeSigningCustomerEku -MetadataFilePath C:\temp\metadata.sample.scus.privateci.json +``` +7. To sign your policy, you run the invoke command: +``` +Invoke-AzCodeSigningCIPolicySigning -accountName acstestcanary -profileName acstestcanaryCert1 -endpointurl "https://xxx.codesigning.azure.net/" -Path C:\Temp\defaultpolicy.bin -Destination C:\Temp\defaultpolicy_signed.bin -TimeStamperUrl: http://timestamp.acs.microsoft.com +``` + +Or use a `metadata.json` file and the following command: ++``` +Invoke-AzCodeSigningCIPolicySigning -MetadataFilePath C:\temp\metadata.sample.scus.privateci.json -Path C:\Temp\defaultpolicy.bin -Destination C:\Temp\defaultpolicy_signed.bin -TimeStamperUrl: http://timestamp.acs.microsoft.com +``` ++## Creating and Deploying a CI Policy ++For steps on creating and deploying your CI policy refer to: +* [Use signed policies to protect Windows Defender Application Control against tampering](https://learn.microsoft.com/windows/security/application-security/application-control/windows-defender-application-control/deployment/use-signed-policies-to-protect-wdac-against-tampering) +* [Windows Defender Application Control design guide](https://learn.microsoft.com/windows/security/application-security/application-control/windows-defender-application-control/design/wdac-design-guide) + |
| trusted-signing | How To Signing Integrations | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/trusted-signing/how-to-signing-integrations.md | Trusted Signing currently supports the following signing integrations: * ADO Task * PowerShell for Authenticode * Azure PowerShell - App Control for Business CI Policy-We constantly work to support more signing integrations and will update the above list if/when more are available. ++We constantly work to support more signing integrations and update the above when more become available. This article explains how to set up each of the above Trusted Signing signing integrations. The components that SignTool.exe uses to interface with Trusted Signing require ### Download and install Trusted Signing Dlib package Complete these steps to download and install the Trusted Signing Dlib package (.ZIP):-1. Download the [Trusted Signing Dlib package](https://www.nuget.org/packages/Azure.CodeSigning.Client). +1. Download the [Trusted Signing Dlib package](https://www.nuget.org/packages/Microsoft.Trusted.Signing.Client). 2. Extract the Trusted Signing Dlib zip content and install it onto your signing node in a directory of your choice. YouΓÇÖre required to install it onto the node youΓÇÖll be signing files from with SignTool.exe. Trusted Signing certificates have a 3-day validity, so timestamping is critical ## Use other signing integrations with Trusted Signing This section explains how to set up other not [SignTool](#set-up-signtool-with-trusted-signing) signing integrations with Trusting Signing. -* GitHub Action ΓÇô To use the GitHub action for Trusted Signing, visit [Azure Code Signing ┬╖ Actions ┬╖ GitHub Marketplace](https://github.com/marketplace/actions/azure-code-signing) and follow the instructions to set up and use GitHub action. +* GitHub Action ΓÇô To use the GitHub action for Trusted Signing, visit [Trusted Signing ┬╖ Actions ┬╖ GitHub Marketplace](https://github.com/azure/trusted-signing-action) and follow the instructions to set up and use GitHub action. -* ADO Task ΓÇô To use the Trusted Signing AzureDevOps task, visit [Azure Code Signing - Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=VisualStudioClient.AzureCodeSigning) and follow the instructions for setup. +* ADO Task ΓÇô To use the Trusted Signing AzureDevOps task, visit [Trusted Signing - Visual Studio Marketplace](https://marketplace.visualstudio.com/items?itemName=VisualStudioClient.TrustedSigning&ssr=false#overview) and follow the instructions for setup. -* PowerShell for Authenticode ΓÇô To use PowerShell for Trusted Signing, visit [PowerShell Gallery | AzureCodeSigning 0.2.15](https://www.powershellgallery.com/packages/AzureCodeSigning/0.2.15) to install the PowerShell module. +* PowerShell for Authenticode ΓÇô To use PowerShell for Trusted Signing, visit [PowerShell Gallery | Trusted Signing 0.3.8](https://www.powershellgallery.com/packages/TrustedSigning/0.3.8) to install the PowerShell module. -* Azure PowerShell ΓÇô App Control for Business CI Policy - App Control for Windows [link to CI policy signing tutorial]. +* Azure PowerShell: App Control for Business CI Policy ΓÇô To use Trusted Signing for CI policy signing follow the instructions at [Signing a New CI policy](./how-to-sign-ci-policy.md) and visit the [Az.CodeSigning PowerShell Module](https://learn.microsoft.com/powershell/azure/install-azps-windows). * Trusted Signing SDK ΓÇô To create your own signing integration our [Trusted Signing SDK](https://www.nuget.org/packages/Azure.CodeSigning.Sdk) is publicly available. |
| trusted-signing | Tutorial Assign Roles | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/trusted-signing/tutorial-assign-roles.md | The Identity Verified role specifically is needed to manage Identity Validation ## Assign roles in Trusting Signing Complete the following steps to assign roles in Trusted Signing.+ 1. Navigate to your Trusted Signing account on the Azure portal and select the **Access Control (IAM)** tab in the left menu. 2. Select on the **Roles** tab and search "Trusted Signing". You can see in the screenshot below the two custom roles.  -3. To assign these roles, select on the **Add** drop down and select **Add role assignment**. Follow the [Assign roles in Azure](../role-based-access-control/role-assignments-portal.md) guide to assign the relevant roles to your identities. +3. To assign these roles, select on the **Add** drop down and select **Add role assignment**. Follow the [Assign roles in Azure](https://docs.microsoft.com/azure/role-based-access-control/role-assignments-portal?tabs=current) guide to assign the relevant roles to your identities. _You'll need at least a Contributor role to create a Trusted Signing account and certificate profile._ +4. For more granular access control on the certificate profile level, you can use the Azure CLI to assign roles. The following commands can be used to assign the _Code Signing Certificate Profile Signer_ role to users/service principles to sign files. +``` +az role assignment create --assignee <objectId of user/service principle> +--role "Code Signing Certificate Profile Signer" +--scope "/subscriptions/<subscriptionId>/resourceGroups/<resource-group-name>/providers/Microsoft.CodeSigning/codeSigningAccounts/<codesigning-account-name>/certificateProfiles/<profileName>" +``` ## Related content * [What is Azure role-based access control (RBAC)?](../role-based-access-control/overview.md) |
| virtual-machines | Hibernate Resume | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/virtual-machines/hibernate-resume.md | The following Windows operating systems support hibernation: - Capacity reservations ## Prerequisites to use hibernation-- The hibernate feature is enabled for your subscription. - A persistent OS disk large enough to store the contents of the RAM, OS and other applications running on the VM is connected. - The VM size supports hibernation. - The VM OS supports hibernation. The following Windows operating systems support hibernation: - Hibernation is enabled on your VM when creating the VM. - If a VM is being created from an OS disk or a Compute Gallery image, then the OS disk or Gallery Image definition supports hibernation. -## Enabling hibernation feature for your subscription -Use the following steps to enable this feature for your subscription: --### [Portal](#tab/enablehiberPortal) -1. In your Azure subscription, go to the Settings section and select 'Preview features'. -1. Search for 'hibernation'. -1. Check the 'Hibernation Preview' item. -1. Click 'Register'. -- --### [PowerShell](#tab/enablehiberPS) -```powershell -Register-AzProviderFeature -FeatureName "VMHibernationPreview" -ProviderNamespace "Microsoft.Compute" -``` -### [CLI](#tab/enablehiberCLI) -```azurecli -az feature register --name VMHibernationPreview --namespace Microsoft.Compute -``` ---Confirm that the registration state is Registered (registration takes a few minutes) using the following command before trying out the feature. --### [Portal](#tab/checkhiberPortal) -In the Azure portal under 'Preview features', select 'Hibernation Preview'. The registration state should show as 'Registered'. -- --### [PowerShell](#tab/checkhiberPS) -```powershell -Get-AzProviderFeature -FeatureName "VMHibernationPreview" -ProviderNamespace "Microsoft.Compute" -``` -### [CLI](#tab/checkhiberCLI) -```azurecli -az feature show --name VMHibernationPreview --namespace Microsoft.Compute -``` -- ## Getting started with hibernation To hibernate a VM, you must first enable the feature while creating the VM. You can only enable hibernation for a VM on initial creation. You can't enable this feature after the VM is created. |
| vpn-gateway | Ipsec Ike Policy Howto | https://github.com/MicrosoftDocs/azure-docs/commits/main/articles/vpn-gateway/ipsec-ike-policy-howto.md | description: Learn how to configure IPsec/IKE custom policy for S2S or VNet-to-V Previously updated : 01/30/2023 Last updated : 04/04/2024 - + # Configure custom IPsec/IKE connection policies for S2S VPN and VNet-to-VNet: Azure portal This article walks you through the steps to configure IPsec/IKE policy for VPN Gateway Site-to-Site VPN or VNet-to-VNet connections using the Azure portal. The following sections help you create and configure an IPsec/IKE policy, and apply the policy to a new or existing connection. This article walks you through the steps to configure IPsec/IKE policy for VPN G The instructions in this article help you set up and configure IPsec/IKE policies as shown in the following diagram. 1. Create a virtual network and a VPN gateway. 1. Create a local network gateway for cross premises connection, or another virtual network and gateway for VNet-to-VNet connection. The following table lists the corresponding Diffie-Hellman groups supported by t [!INCLUDE [Diffie-Hellman groups](../../includes/vpn-gateway-ipsec-ike-diffie-hellman-include.md)] -Refer to [RFC3526](https://tools.ietf.org/html/rfc3526) and [RFC5114](https://tools.ietf.org/html/rfc5114) for more details. +For more information, see [RFC3526](https://tools.ietf.org/html/rfc3526) and [RFC5114](https://tools.ietf.org/html/rfc5114). ## <a name="crossprem"></a>Create S2S VPN connection with custom policy -This section walks you through the steps to create a Site-to-Site VPN connection with an IPsec/IKE policy. The following steps create the connection as shown in the following diagram: +This section walks you through the steps to create a Site-to-Site VPN connection with an IPsec/IKE policy. The following steps create the connection as shown in the following diagram. The on-premises site in this diagram represents **Site6**. ### Step 1: Create the virtual network, VPN gateway, and local network gateway for TestVNet1 -Create the following resources.For steps, see [Create a Site-to-Site VPN connection](./tutorial-site-to-site-portal.md). +Create the following resources. For steps, see [Create a Site-to-Site VPN connection](./tutorial-site-to-site-portal.md). 1. Create the virtual network **TestVNet1** using the following values. Configure a custom IPsec/IKE policy with the following algorithms and parameters The steps to create a VNet-to-VNet connection with an IPsec/IKE policy are similar to that of an S2S VPN connection. You must complete the previous sections in [Create an S2S vpn connection](#crossprem) to create and configure TestVNet1 and the VPN gateway. ### Step 1: Create the virtual network, VPN gateway, and local network gateway for TestVNet2 -Use the steps in the [Create a VNet-to-VNet connection](vpn-gateway-howto-vnet-vnet-resource-manager-portal.md) article to create TestVNet2 and create a VNet-to-VNet connection to TestVNet1. +Use the steps in the [Create a VNet-to-VNet connection](vpn-gateway-howto-vnet-vnet-resource-manager-portal.md) article to create TestVNet2, and create a VNet-to-VNet connection to TestVNet1. Example values: Example values: ### Step 2: Configure the VNet-to-VNet connection -1. From the VNet1GW gateway, add a VNet-to-VNet connection to VNet2GW, **VNet1toVNet2**. +1. From the VNet1GW gateway, add a VNet-to-VNet connection to VNet2GW named **VNet1toVNet2**. -1. Next, from the VNet2GW, add a VNet-to-VNet connection to VNet1GW, **VNet2toVNet1**. +1. Next, from the VNet2GW, add a VNet-to-VNet connection to VNet1GW named **VNet2toVNet1**. 1. After you add the connections, you'll see the VNet-to-VNet connections as shown in the following screenshot from the VNet2GW resource: Example values: 1. After you complete these steps, the connection is established in a few minutes, and you'll have the following network topology. - :::image type="content" source="./media/ipsec-ike-policy-howto/policy-diagram.png" alt-text="Diagram shows IPsec/IKE policy." border="false" lightbox="./media/ipsec-ike-policy-howto/policy-diagram.png"::: + :::image type="content" source="./media/ipsec-ike-policy-howto/policy-diagram.png" alt-text="Diagram shows IPsec/IKE policy for VNet-to-VNet and S2S VPN." lightbox="./media/ipsec-ike-policy-howto/policy-diagram.png"::: ## To remove custom policy from a connection 1. To remove a custom policy from a connection, go to the connection resource.-1. On the **Configuration** page, change the IPse /IKE policy from **Custom** to **Default**. This will remove all custom policy previously specified on the connection, and restore the Default IPsec/IKE settings on this connection. +1. On the **Configuration** page, change the IPse /IKE policy from **Custom** to **Default**. This removes all custom policy previously specified on the connection, and restore the Default IPsec/IKE settings on this connection. 1. Select **Save** to remove the custom policy and restore the default IPsec/IKE settings on the connection. ## IPsec/IKE policy FAQ To view frequently asked questions, go to the IPsec/IKE policy section of the [V ## Next steps -See [Connect multiple on-premises policy-based VPN devices](vpn-gateway-connect-multiple-policybased-rm-ps.md) for more details regarding policy-based traffic selectors. +For more information about policy-based traffic selectors, see [Connect multiple on-premises policy-based VPN devices](vpn-gateway-connect-multiple-policybased-rm-ps.md). |