Updates from: 01/01/2021 04:05:32
Service Microsoft Docs article Related commit history on GitHub Change details
active-directory https://docs.microsoft.com/en-us/azure/active-directory/app-provisioning/plan-auto-user-provisioning https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/active-directory/app-provisioning/plan-auto-user-provisioning.md
@@ -8,7 +8,7 @@ ms.service: active-directory
ms.subservice: app-provisioning ms.topic: conceptual ms.workload: identity
-ms.date: 10/17/2019
+ms.date: 12/31/2020
ms.author: kenwith ms.reviewer: arvindha, celested #customer intent: As an admin, I want to automate user provisioning to SaaS apps
@@ -325,4 +325,4 @@ Refer to the following links to troubleshoot any issues that may turn up during
* [Export or import your provisioning configuration by using Microsoft Graph API](../app-provisioning/export-import-provisioning-configuration.md)
-* [Writing expressions for attribute mappings in Azure Active directory](../app-provisioning/functions-for-customizing-application-data.md)
\ No newline at end of file
+* [Writing expressions for attribute mappings in Azure Active directory](../app-provisioning/functions-for-customizing-application-data.md)
active-directory https://docs.microsoft.com/en-us/azure/active-directory/devices/device-management-azure-portal https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/active-directory/devices/device-management-azure-portal.md
@@ -166,7 +166,7 @@ This option is a premium edition capability available through products such as A
- **Require Multi-Factor Auth to join devices** - You can choose whether users are required to provide an additional authentication factor to join or register their device to Azure AD. The default is **No**. We recommend requiring multi-factor authentication when registering or joining a device. Before you enable multi-factor authentication for this service, you must ensure that multi-factor authentication is configured for the users that register their devices. For more information on different Azure AD Multi-Factor Authentication services, see [getting started with Azure AD Multi-Factor Authentication](../authentication/concept-mfa-howitworks.md). > [!NOTE]
-> **Require Multi-Factor Auth to join devices** setting applies to devices that are either Azure AD joined (with some exceptions) or Azure AD registered. This setting does not apply to hybrid Azure AD joined devices, [Azure AD joined VMs in Azure](/howto-vm-sign-in-azure-ad-windows#enabling-azure-ad-login-in-for-windows-vm-in-azure) and Azure AD joined devices using [Windows Autopilot self-deployment mode](/mem/autopilot/self-deploying).
+> **Require Multi-Factor Auth to join devices** setting applies to devices that are either Azure AD joined (with some exceptions) or Azure AD registered. This setting does not apply to hybrid Azure AD joined devices, [Azure AD joined VMs in Azure](/azure/active-directory/devices/howto-vm-sign-in-azure-ad-windows#enabling-azure-ad-login-in-for-windows-vm-in-azure) and Azure AD joined devices using [Windows Autopilot self-deployment mode](/mem/autopilot/self-deploying).
- **Maximum number of devices** - This setting enables you to select the maximum number of Azure AD joined or Azure AD registered devices that a user can have in Azure AD. If a user reaches this quota, they are not be able to add additional devices until one or more of the existing devices are removed. The default value is **50**.
active-directory https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/access-panel-deployment-plan https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/active-directory/manage-apps/access-panel-deployment-plan.md
@@ -8,7 +8,7 @@ ms.service: active-directory
ms.subservice: app-mgmt ms.workload: identity ms.topic: conceptual
-ms.date: 09/27/2019
+ms.date: 12/31/2020
ms.author: kenwith ---
@@ -307,4 +307,4 @@ Use the least privileged role to accomplish a required task within Azure Active
You can use [Privileged Identity Management](../privileged-identity-management/pim-configure.md) to manage your roles to provide additional auditing, control, and access review for users with directory permissions. ## Next steps
-[Plan a deployment of Azure AD Multi-Factor Authentication](../authentication/howto-mfa-getstarted.md)
\ No newline at end of file
+[Plan a deployment of Azure AD Multi-Factor Authentication](../authentication/howto-mfa-getstarted.md)
active-directory https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/application-proxy-deployment-plan https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/active-directory/manage-apps/application-proxy-deployment-plan.md
@@ -2,17 +2,13 @@
title: Plan an Azure Active Directory Application Proxy Deployment description: An end-to-end guide for planning the deployment of Application proxy within your organization services: active-directory
-documentationcenter: 'azure'
author: kenwith manager: celestedg
-ms.assetid:
ms.service: active-directory ms.subservice: app-mgmt ms.workload: identity
-ms.tgt_pltfrm: na
-ms.devlang: na
ms.topic: conceptual
-ms.date: 04/04/2019
+ms.date: 12/31/2020
ms.author: kenwith ---
@@ -322,4 +318,4 @@ The following articles cover common scenarios that can also be used to create tr
* [Configure with PingAccess](./application-proxy-ping-access-publishing-guide.md) * [Can't Access this Corporate Application error](application-proxy-sign-in-bad-gateway-timeout-error.md) * [Problem installing the Application Proxy Agent Connector](application-proxy-connector-installation-problem.md)
-* [Sign-in problem](application-sign-in-problem-on-premises-application-proxy.md)
\ No newline at end of file
+* [Sign-in problem](application-sign-in-problem-on-premises-application-proxy.md)
active-directory https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/what-is-application-proxy https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/active-directory/manage-apps/what-is-application-proxy.md
@@ -8,7 +8,7 @@ ms.service: active-directory
ms.subservice: app-mgmt ms.topic: conceptual ms.workload: identity
-ms.date: 05/31/2019
+ms.date: 12/31/2020
ms.author: kenwith ms.reviewer: japere ---
@@ -200,4 +200,4 @@ Organizations should begin taking advantage of App Proxy today to take advantage
## Next steps * For information about planning, operating, and managing Azure AD Application Proxy, see [Plan an Azure AD Application Proxy deployment](application-proxy-deployment-plan.md).
-* To schedule a live demo or get a free 90-day trial for evaluation, see [Getting started with Enterprise Mobility + Security](https://www.microsoft.com/cloud-platform/enterprise-mobility-security-trial).
\ No newline at end of file
+* To schedule a live demo or get a free 90-day trial for evaluation, see [Getting started with Enterprise Mobility + Security](https://www.microsoft.com/cloud-platform/enterprise-mobility-security-trial).
active-directory https://docs.microsoft.com/en-us/azure/active-directory/saas-apps/oracle-cloud-infrastructure-console-provisioning-tutorial https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/active-directory/saas-apps/oracle-cloud-infrastructure-console-provisioning-tutorial.md new file mode 100644
@@ -0,0 +1,183 @@
+---
+title: 'Tutorial: Configure Oracle Cloud Infrastructure Console for automatic user provisioning with Azure Active Directory | Microsoft Docs'
+description: Learn how to automatically provision and de-provision user accounts from Azure AD to Oracle Cloud Infrastructure Console.
+services: active-directory
+author: zchia
+writer: zchia
+manager: CelesteDG
+ms.service: active-directory
+ms.subservice: saas-app-tutorial
+ms.workload: identity
+ms.topic: tutorial
+ms.date: 01/16/2020
+ms.author: Zhchia
+---
+
+# Tutorial: Configure Oracle Cloud Infrastructure Console for automatic user provisioning
+
+This tutorial describes the steps you need to perform in both Oracle Cloud Infrastructure Console and Azure Active Directory (Azure AD) to configure automatic user provisioning. When configured, Azure AD automatically provisions and de-provisions users and groups to [Oracle Cloud Infrastructure Console](https://www.oracle.com/cloud/free/?source=:ow:o:p:nav:0916BCButton&intcmp=:ow:o:p:nav:0916BCButton) using the Azure AD Provisioning service. For important details on what this service does, how it works, and frequently asked questions, see [Automate user provisioning and deprovisioning to SaaS applications with Azure Active Directory](../app-provisioning/user-provisioning.md).
++
+## Capabilities supported
+> [!div class="checklist"]
+> * Create users in Oracle Cloud Infrastructure Console
+> * Remove users in Oracle Cloud Infrastructure Console when they do not require access anymore
+> * Keep user attributes synchronized between Azure AD and Oracle Cloud Infrastructure Console
+> * Provision groups and group memberships in Oracle Cloud Infrastructure Console
+> * [Single sign-on](./oracle-cloud-tutorial.md) to Oracle Cloud Infrastructure Console (recommended)
+
+## Prerequisites
+
+The scenario outlined in this tutorial assumes that you already have the following prerequisites:
+
+* [An Azure AD tenant](../develop/quickstart-create-new-tenant.md)
+* A user account in Azure AD with [permission](../roles/permissions-reference.md) to configure provisioning (e.g. Application Administrator, Cloud Application administrator, Application Owner, or Global Administrator).
+* An Oracle Cloud Infrastructure Console [tenant](https://www.oracle.com/cloud/sign-in.html?intcmp=OcomFreeTier&source=:ow:o:p:nav:0916BCButton).
+* A user account in Oracle Cloud Infrastructure Console with Admin permissions.
+
+## Step 1. Plan your provisioning deployment
+1. Learn about [how the provisioning service works](../app-provisioning/user-provisioning.md).
+2. Determine who will be in [scope for provisioning](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).
+3. Determine what data to [map between Azure AD and Oracle Cloud Infrastructure Console](../app-provisioning/customize-application-attributes.md).
+
+## Step 2. Configure Oracle Cloud Infrastructure Console to support provisioning with Azure AD
+
+1. Login to Oracle Cloud Infrastructure Console's admin portal. On the top left corner of the screen navigate to **Identity > Federation**.
+
+ ![Oracle Admin](./media/oracle-cloud-infratstructure-console-provisioning-tutorial/identity.png)
+
+2. Click on the URL displayed on the page beside Oracle Identity Cloud Service Console.
+
+ ![Oracle URL](./media/oracle-cloud-infratstructure-console-provisioning-tutorial/url.png)
+
+3. Click on **Add Identity Provider** to create a new identity provider. Save the IdP id to be used as a part of tenant URL.Click on plus icon beside the **Applications** tab to create an OAuth Client and Grant IDCS Identity Domain Administrator AppRole.
+
+ ![Oracle Cloud Icon](./media/oracle-cloud-infratstructure-console-provisioning-tutorial/add.png)
+
+4. Follow the screenshots below to configure your application. Once the configuration is done click on **Save**.
+
+ ![Oracle Configuration](./media/oracle-cloud-infratstructure-console-provisioning-tutorial/configuration.png)
+
+ ![Oracle Token Issuance Policy](./media/oracle-cloud-infratstructure-console-provisioning-tutorial/token-issuance.png)
+
+5. Under the configurations tab of your application expand the **General Information** option to retrieve the client ID and client secret.
+
+ ![Oracle token generation](./media/oracle-cloud-infratstructure-console-provisioning-tutorial/general-information.png)
+
+6. To generate a secret token Base64 encode the client ID and client secret in the format **client ID:Client Secret**. Save the secret token. This value will be entered in the **Secret Token** field in the provisioning tab of your Oracle Cloud Infrastructure Console application in the Azure portal.
+
+## Step 3. Add Oracle Cloud Infrastructure Console from the Azure AD application gallery
+
+Add Oracle Cloud Infrastructure Console from the Azure AD application gallery to start managing provisioning to Oracle Cloud Infrastructure Console. If you have previously setup Oracle Cloud Infrastructure Console for SSO you can use the same application. However it is recommended that you create a separate app when testing out the integration initially. Learn more about adding an application from the gallery [here](../manage-apps/add-application-portal.md).
+
+## Step 4. Define who will be in scope for provisioning
+
+The Azure AD provisioning service allows you to scope who will be provisioned based on assignment to the application and or based on attributes of the user / group. If you choose to scope who will be provisioned to your app based on assignment, you can use the following [steps](../manage-apps/assign-user-or-group-access-portal.md) to assign users and groups to the application. If you choose to scope who will be provisioned based solely on attributes of the user or group, you can use a scoping filter as described [here](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).
+
+* When assigning users and groups to Oracle Cloud Infrastructure Console, you must select a role other than **Default Access**. Users with the Default Access role are excluded from provisioning and will be marked as not effectively entitled in the provisioning logs. If the only role available on the application is the default access role, you can [update the application manifest](../develop/howto-add-app-roles-in-azure-ad-apps.md) to add additional roles.
+
+* Start small. Test with a small set of users and groups before rolling out to everyone. When scope for provisioning is set to assigned users and groups, you can control this by assigning one or two users or groups to the app. When scope is set to all users and groups, you can specify an [attribute based scoping filter](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).
++
+## Step 5. Configure automatic user provisioning to Oracle Cloud Infrastructure Console
+
+This section guides you through the steps to configure the Azure AD provisioning service to create, update, and disable users and/or groups in TestApp based on user and/or group assignments in Azure AD.
+
+### To configure automatic user provisioning for Oracle Cloud Infrastructure Console in Azure AD:
+
+1. Sign in to the [Azure portal](https://portal.azure.com). Select **Enterprise Applications**, then select **All applications**.
+
+ ![Enterprise applications blade](common/enterprise-applications.png)
+
+2. In the applications list, select **Oracle Cloud Infrastructure Console**.
+
+ ![The Oracle Cloud Infrastructure Console link in the Applications list](common/all-applications.png)
+
+3. Select the **Provisioning** tab.
+
+ ![Screenshot of the Manage options with the Provisioning option called out.](common/provisioning.png)
+
+4. Set the **Provisioning Mode** to **Automatic**.
+
+ ![Screenshot of the Provisioning Mode dropdown list with the Automatic option called out.](common/provisioning-automatic.png)
+
+5. Under the **Admin Credentials** section, input the **Tenant URL** in the format `https://<IdP ID>.identity.oraclecloud.com/admin/v1`. For example `https://idcs-0bfd023ff2xx4a98a760fa2c31k92b1d.identity.oraclecloud.com/admin/v1`. Input the secret token value retrieved earlier in **Secret Token**. Click **Test Connection** to ensure Azure AD can connect to Oracle Cloud Infrastructure Console. If the connection fails, ensure your Oracle Cloud Infrastructure Console account has admin permissions and try again.
+
+ ![Screenshot shows the Admin Credentials dialog box, where you can enter your Tenant U R L and Secret Token.](./media/oracle-cloud-infratstructure-console-provisioning-tutorial/provisioning.png)
+
+6. In the **Notification Email** field, enter the email address of a person or group who should receive the provisioning error notifications and select the **Send an email notification when a failure occurs** check box.
+
+ ![Notification Email](common/provisioning-notification-email.png)
+
+7. Select **Save**.
+
+8. Under the **Mappings** section, select **Synchronize Azure Active Directory Users to Oracle Cloud Infrastructure Console**.
+
+9. Review the user attributes that are synchronized from Azure AD to Oracle Cloud Infrastructure Console in the **Attribute-Mapping** section. The attributes selected as **Matching** properties are used to match the user accounts in Oracle Cloud Infrastructure Console for update operations. If you choose to change the [matching target attribute](../app-provisioning/customize-application-attributes.md), you will need to ensure that the Oracle Cloud Infrastructure Console API supports filtering users based on that attribute. Select the **Save** button to commit any changes.
+
+ |Attribute|Type|
+ |---|---|
+ |displayName|String|
+ |userName|String|
+ |active|Boolean|
+ |title|String|
+ |emails[type eq "work"].value|String|
+ |preferredLanguage|String|
+ |name.givenName|String|
+ |name.familyName|String|
+ |addresses[type eq "work"].formatted|String|
+ |addresses[type eq "work"].locality|String|
+ |addresses[type eq "work"].region|String|
+ |addresses[type eq "work"].postalCode|String|
+ |addresses[type eq "work"].country|String|
+ |addresses[type eq "work"].streetAddress|String|
+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:employeeNumber|String|
+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:department|String|
+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:costCenter|String|
+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:division|String|
+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:manager|Reference|
+ |urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:organization|String|
+ |urn:ietf:params:scim:schemas:oracle:idcs:extension:user:User:bypassNotification|Boolean|
+ |urn:ietf:params:scim:schemas:oracle:idcs:extension:user:User:isFederatedUser|Boolean|
+
+10. Under the **Mappings** section, select **Synchronize Azure Active Directory Groups to Oracle Cloud Infrastructure Console**.
+
+11. Review the group attributes that are synchronized from Azure AD to Oracle Cloud Infrastructure Console in the **Attribute-Mapping** section. The attributes selected as **Matching** properties are used to match the groups in Oracle Cloud Infrastructure Console for update operations. Select the **Save** button to commit any changes.
+
+ | Attribute | Type |
+ |--|--|
+ | displayName | String |
+ | externalId | String |
+ | members | Reference |
+
+12. To configure scoping filters, refer to the following instructions provided in the [Scoping filter tutorial](../app-provisioning/define-conditional-rules-for-provisioning-user-accounts.md).
+
+13. To enable the Azure AD provisioning service for Oracle Cloud Infrastructure Console, change the **Provisioning Status** to **On** in the **Settings** section.
+
+ ![Provisioning Status Toggled On](common/provisioning-toggle-on.png)
+
+14. Define the users and/or groups that you would like to provision to Oracle Cloud Infrastructure Console by choosing the desired values in **Scope** in the **Settings** section.
+
+ ![Provisioning Scope](common/provisioning-scope.png)
+
+15. When you are ready to provision, click **Save**.
+
+ ![Saving Provisioning Configuration](common/provisioning-configuration-save.png)
+
+This operation starts the initial synchronization cycle of all users and groups defined in **Scope** in the **Settings** section. The initial cycle takes longer to perform than subsequent cycles, which occur approximately every 40 minutes as long as the Azure AD provisioning service is running.
+
+## Step 6. Monitor your deployment
+Once you've configured provisioning, use the following resources to monitor your deployment:
+
+* Use the [provisioning logs](../reports-monitoring/concept-provisioning-logs.md) to determine which users have been provisioned successfully or unsuccessfully
+* Check the [progress bar](../app-provisioning/application-provisioning-when-will-provisioning-finish-specific-user.md) to see the status of the provisioning cycle and how close it is to completion
+* If the provisioning configuration seems to be in an unhealthy state, the application will go into quarantine. Learn more about quarantine states [here](../app-provisioning/application-provisioning-quarantine-status.md).
+
+## Additional resources
+
+* [Managing user account provisioning for Enterprise Apps](../app-provisioning/configure-automatic-user-provisioning-portal.md)
+* [What is application access and single sign-on with Azure Active Directory?](../manage-apps/what-is-single-sign-on.md)
+
+## Next steps
+
+* [Learn how to review logs and get reports on provisioning activity](../app-provisioning/check-status-user-account-provisioning.md)
active-directory https://docs.microsoft.com/en-us/azure/active-directory/saas-apps/tutorial-list https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/active-directory/saas-apps/tutorial-list.md
@@ -58,8 +58,8 @@ To find more tutorials, use the table of contents on the left.
| ![logo-AskYourTeam](./media/tutorial-list/active-directory-saas-askyourteam-tutorial.png)| [AskYourTeam](askyourteam-tutorial.md)| | ![logo-Atlassian Cloud](./media/tutorial-list/active-directory-saas-atlassian-cloud-tutorial.png)| [Atlassian Cloud](atlassian-cloud-tutorial.md)| | ![logo-Box](./media/tutorial-list/active-directory-saas-box-tutorial.png)| [Box](box-tutorial.md)|
-| ![logo-Carbonite Endpoint Backup](./media/tutorial-list/active-directory-saas-carbonite-endpoint-backup-tutorial.png)| [Carbonite Endpoint Backup](carbonite-endpoint-backup-tutorial.md)|
| ![logo-CakeHR](./media/tutorial-list/active-directory-saas-cakehr-tutorial.png)| [CakeHR](cakehr-tutorial.md)|
+| ![logo-Carbonite Endpoint Backup](./media/tutorial-list/active-directory-saas-carbonite-endpoint-backup-tutorial.png)| [Carbonite Endpoint Backup](carbonite-endpoint-backup-tutorial.md)|
| ![logo-Cisco Webex](./media/tutorial-list/active-directory-saas-cisco-webex-tutorial.png)| [Cisco Webex](cisco-spark-tutorial.md)| | ![logo-Citrix ShareFile](./media/tutorial-list/active-directory-saas-sharefile-tutorial.png)| [Citrix ShareFile](sharefile-tutorial.md)| | ![logo-Concur Travel and Expense](./media/tutorial-list/active-directory-saas-concur-travel-and-expense-tutorial.png)| [Concur Travel and Expense](concur-travel-and-expense-tutorial.md)|
@@ -108,9 +108,11 @@ To find more tutorials, use the table of contents on the left.
| ![logo-Slack](./media/tutorial-list/active-directory-saas-slack-tutorial.png)| [Slack](slack-tutorial.md)| | ![logo-SmartDraw](./media/tutorial-list/active-directory-saas-smartdraw-tutorial.png)| [SmartDraw](smartdraw-tutorial.md)| | ![logo-Soloinsight-CloudGate SSO](./media/tutorial-list/active-directory-saas-soloinsight-cloudgate-sso-tutorial.png)| [Soloinsight-CloudGate SSO](soloinsight-cloudgate-sso-tutorial.md)|
+| ![logo-StatusPage](./media/tutorial-list/active-directory-saas-statuspage-tutorial.png)| [StatusPage](statuspage-tutorial.md)|
| ![logo-Tableau Online](./media/tutorial-list/active-directory-saas-tableau-online-tutorial.png)| [Tableau Online](tableauonline-tutorial.md)| | ![logo-TargetProcess](./media/tutorial-list/active-directory-saas-target-process-tutorial.png)| [TargetProcess](target-process-tutorial.md)| | ![logo-Teamphoria](./media/tutorial-list/active-directory-saas-teamphoria-tutorial.png)| [Teamphoria](teamphoria-tutorial.md)|
+| ![logo-Terraform Cloud](./media/tutorial-list/active-directory-saas-terraform-cloud-tutorial.png)| [Terraform Cloud](terraform-cloud-tutorial.md)|
| ![logo-TextMagic](./media/tutorial-list/active-directory-saas-textmagic-tutorial.png)| [TextMagic](textmagic-tutorial.md)| | ![logo-Upshotly](./media/tutorial-list/active-directory-saas-upshotly-tutorial.png)| [Upshotly](upshotly-tutorial.md)| | ![logo-Velpic SAML](./media/tutorial-list/active-directory-saas-velpicsaml-tutorial.png)| [Velpic SAML](velpicsaml-tutorial.md)|
@@ -124,6 +126,7 @@ To find more tutorials, use the table of contents on the left.
| ![logo-XaitPorter](./media/tutorial-list/active-directory-saas-xaitporter-tutorial.png)| [XaitPorter](xaitporter-tutorial.md)| | ![logo-Yodeck](./media/tutorial-list/active-directory-saas-yodeck-tutorial.png)| [Yodeck](yodeck-tutorial.md)| | ![logo-Zendesk](./media/tutorial-list/active-directory-saas-zendesk-tutorial.png)| [Zendesk](zendesk-tutorial.md)|
+| ![logo-Zoom](./media/tutorial-list/active-directory-saas-zoom-tutorial.png)| [Zoom](zoom-tutorial.md)|
| ![logo-Zscaler](./media/tutorial-list/active-directory-saas-zscaler-tutorial.png)| [Zscaler](zscaler-tutorial.md)| | ![logo-Zscaler Beta](./media/tutorial-list/active-directory-saas-zscaler-beta-tutorial.png)| [Zscaler Beta](zscaler-beta-tutorial.md)| | ![logo-Zscaler One](./media/tutorial-list/active-directory-saas-zscaler-one-tutorial.png)| [Zscaler One](zscaler-one-tutorial.md)|
aks https://docs.microsoft.com/en-us/azure/aks/troubleshooting https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/aks/troubleshooting.md
@@ -434,7 +434,7 @@ On Kubernetes versions **older than 1.15.0**, you may receive an error such as *
### Why do upgrades to Kubernetes 1.16 fail when using node labels with a kubernetes.io prefix
-As of Kubernetes [1.16](https://v1-16.docs.kubernetes.io/docs/setup/release/notes/) [only a defined subset of labels with the kubernetes.io prefix](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/0000-20170814-bounding-self-labeling-kubelets.md#proposal) can be applied by the kubelet to nodes. AKS cannot remove active labels on your behalf without consent, as it may cause downtime to impacted workloads.
+As of Kubernetes [1.16](https://v1-16.docs.kubernetes.io/docs/setup/release/notes/) [only a defined subset of labels with the kubernetes.io prefix](https://v1-18.docs.kubernetes.io/docs/concepts/overview/working-with-objects/labels/) can be applied by the kubelet to nodes. AKS cannot remove active labels on your behalf without consent, as it may cause downtime to impacted workloads.
As a result, to mitigate this issue you can:
app-service https://docs.microsoft.com/en-us/azure/app-service/faq-open-source-technologies https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/app-service/faq-open-source-technologies.md
@@ -179,24 +179,24 @@ App Service doesn't have a built-in email feature. For some good alternatives fo
If you have recently migrated to Azure, WordPress might redirect to the old domain URL. This is caused by a setting in the MySQL database.
-WordPress Buddy+ is an Azure Site Extension that you can use to update the redirection URL directly in the database. For more information about using WordPress Buddy+, see [WordPress tools and MySQL migration with WordPress Buddy+](https://sharepointforum.org/threads/wordpress-tools-and-mysql-migration-with-wordpress-buddy.82929/).
+WordPress Buddy+ is an Azure Site Extension that you can use to update the redirection URL directly in the database. For more information about using WordPress Buddy+, see [WordPress tools and MySQL migration with WordPress Buddy+](https://www.electrongeek.com/blog/2016/12/21/wordpress-buddy-site-extension-for-app-service-on-windows).
Alternatively, if you prefer to manually update the redirection URL by using SQL queries or PHPMyAdmin, see [WordPress: Redirecting to wrong URL](/archive/blogs/azureossds/wordpress-redirecting-to-wrong-url). ## How do I change my WordPress sign-in password?
-If you have forgotten your WordPress sign-in password, you can use WordPress Buddy+ to update it. To reset your password, install the WordPress Buddy+ Azure Site Extension, and then complete the steps described in [WordPress tools and MySQL migration with WordPress Buddy+](https://sharepointforum.org/threads/wordpress-tools-and-mysql-migration-with-wordpress-buddy.82929/).
+If you have forgotten your WordPress sign-in password, you can use WordPress Buddy+ to update it. To reset your password, install the WordPress Buddy+ Azure Site Extension, and then complete the steps described in [WordPress tools and MySQL migration with WordPress Buddy+](https://www.electrongeek.com/blog/2016/12/21/wordpress-buddy-site-extension-for-app-service-on-windows).
## I can't sign in to WordPress. How do I resolve this?
-If you find yourself locked out of WordPress after recently installing a plugin, you might have a faulty plugin. WordPress Buddy+ is an Azure Site Extension that can help you disable plugins in WordPress. For more information, see [WordPress tools and MySQL migration with WordPress Buddy+](https://sharepointforum.org/threads/wordpress-tools-and-mysql-migration-with-wordpress-buddy.82929/).
+If you find yourself locked out of WordPress after recently installing a plugin, you might have a faulty plugin. WordPress Buddy+ is an Azure Site Extension that can help you disable plugins in WordPress. For more information, see [WordPress tools and MySQL migration with WordPress Buddy+](https://www.electrongeek.com/blog/2016/12/21/wordpress-buddy-site-extension-for-app-service-on-windows).
## How do I migrate my WordPress database? You have multiple options for migrating the MySQL database that's connected to your WordPress website: * Developers: Use the [command prompt or PHPMyAdmin](/archive/blogs/azureossds/migrating-data-between-mysql-databases-using-kudu-console-azure-app-service)
-* Non-developers: Use [WordPress Buddy+](https://sharepointforum.org/threads/wordpress-tools-and-mysql-migration-with-wordpress-buddy.82929/)
+* Non-developers: Use [WordPress Buddy+](https://www.electrongeek.com/blog/2016/12/21/wordpress-buddy-site-extension-for-app-service-on-windows)
## How do I help make WordPress more secure?
azure-functions https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-orchestrations https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/azure-functions/durable/durable-functions-orchestrations.md
@@ -289,7 +289,7 @@ It isn't possible to pass multiple parameters to an activity function directly.
# [C#](#tab/csharp)
-In .NET you can also use [ValueTuples](/dotnet/csharp/tuples) objects. The following sample is using new features of [ValueTuples](/dotnet/csharp/tuples) added with [C# 7](/dotnet/csharp/whats-new/csharp-7#tuples):
+In .NET you can also use [ValueTuple](/dotnet/csharp/tuples) objects. The following sample is using new features of [ValueTuple](/dotnet/csharp/tuples) added with [C# 7](/dotnet/csharp/whats-new/csharp-7#tuples):
```csharp [FunctionName("GetCourseRecommendations")]
azure-monitor https://docs.microsoft.com/en-us/azure/azure-monitor/insights/container-insights-livedata-setup https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/azure-monitor/insights/container-insights-livedata-setup.md
@@ -43,7 +43,7 @@ The Azure portal prompts you to validate your login credentials for an Azure Act
To eliminate the need to apply additional configuration changes to allow the Kubernetes user role binding **clusterUser** access to the Live Data (preview) feature after [enabling Kubernetes RBAC](#configure-kubernetes-rbac-authorization) authorization, AKS has added a new Kubernetes cluster role binding called **clusterMonitoringUser**. This cluster role binding has all the necessary permissions out-of-the-box to access the Kubernetes API and the endpoints for utilizing the Live Data (preview) feature.
-In order to utilize the Live Data (preview) feature with this new user, you need to be a member of the [Contributor](../../role-based-access-control/built-in-roles.md#contributor) role on the AKS cluster resource. Azure Monitor for containers, when enabled, is configured to authenticate using this user by default. If the clusterMonitoringUser role binding does not exist on a cluster, **clusterUser** is used for authentication instead.
+In order to utilize the Live Data (preview) feature with this new user, you need to be a member of the [Azure Kubernetes Service Cluster User](../../role-based-access-control/built-in-roles.md#azure-kubernetes-service-cluster-user-role) or [Contributor](../../role-based-access-control/built-in-roles.md#contributor) role on the AKS cluster resource. Azure Monitor for containers, when enabled, is configured to authenticate using the clusterMonitoringUser by default. If the clusterMonitoringUser role binding does not exist on a cluster, **clusterUser** is used for authentication instead. Contributor gives you access to the clusterMonitoringUser (if it exists) and Azure Kuberenetes Service Cluster User gives you access to the clusterUser. Any of these two roles give sufficient access to use this feature.
AKS released this new role binding in January 2020, so clusters created before January 2020 do not have it. If you have a cluster that was created before January 2020, the new **clusterMonitoringUser** can be added to an existing cluster by performing a PUT operation on the cluster, or performing any other operation on the cluster that performs a PUT operation on the cluster, such as updating the cluster version.
azure-monitor https://docs.microsoft.com/en-us/azure/azure-monitor/log-query/logs-dedicated-clusters https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/azure-monitor/log-query/logs-dedicated-clusters.md
@@ -122,7 +122,7 @@ Content-type: application/json
Should be 200 OK and a header.
-## Check cluster provisioning status
+### Check cluster provisioning status
The provisioning of the Log Analytics cluster takes a while to complete. You can check the provisioning state in several ways:
@@ -168,127 +168,7 @@ The provisioning of the Log Analytics cluster takes a while to complete. You can
The *principalId* GUID is generated by the managed identity service for the *Cluster* resource.
-## Check workspace link status
-
-Perform Get operation on the workspace and observe if *clusterResourceId* property is present in the response under *features*. A linked workspace will have the *clusterResourceId* property.
-
-**CLI**
-
-```azurecli
-az monitor log-analytics cluster show --resource-group "resource-group-name" --name "cluster-name"
-```
-
-**PowerShell**
-
-```powershell
-Get-AzOperationalInsightsWorkspace -ResourceGroupName "resource-group-name" -Name "workspace-name"
-```
-
-**REST**
-
-```rest
-GET https://management.azure.com/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.operationalinsights/workspaces/<workspace-name>?api-version=2020-08-01
-Authorization: Bearer <token>
-```
-
-## Change cluster properties
-
-After you create your *Cluster* resource and it is fully provisioned, you can edit additional properties at the cluster level using PowerShell or REST API. Other than the properties that are available during cluster creation, additional properties can only be set after the cluster has been provisioned:
--- **keyVaultProperties**: Used to configure the Azure Key Vault used to provision an [Azure Monitor customer-managed key](../platform/customer-managed-keys.md#customer-managed-key-provisioning). It contains the following parameters: *KeyVaultUri*, *KeyName*, *KeyVersion*. -- **billingType** - The *billingType* property determines the billing attribution for the *cluster* resource and its data:
- - **Cluster** (default) - The Capacity Reservation costs for your Cluster are attributed to the *Cluster* resource.
- - **Workspaces** - The Capacity Reservation costs for your Cluster are attributed proportionately to the workspaces in the Cluster, with the *Cluster* resource being billed some of the usage if the total ingested data for the day is under the Capacity Reservation. See [Log Analytics Dedicated Clusters](../platform/manage-cost-storage.md#log-analytics-dedicated-clusters) to learn more about the Cluster pricing model.
-
-> [!NOTE]
-> The *billingType* property is not supported in PowerShell.
-> Cluster property updates might be executed asynchronous and can take a while to complete.
-
-**PowerShell**
-
-```powershell
-Update-AzOperationalInsightsCluster -ResourceGroupName {resource-group-name} -ClusterName {cluster-name} -KeyVaultUri {key-uri} -KeyName {key-name} -KeyVersion {key-version}
-```
-
-**REST**
-
-> [!NOTE]
-> You can update the *Cluster* resource *sku*, *keyVaultProperties* or *billingType* using PATCH.
-
-For example:
-
-*Call*
-
-```rst
-PATCH https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2020-08-01
-Authorization: Bearer <token>
-Content-type: application/json
-
-{
- "sku": {
- "name": "capacityReservation",
- "capacity": <capacity-reservation-amount-in-GB>
- },
- "properties": {
- "billingType": "cluster",
- "KeyVaultProperties": {
- "KeyVaultUri": "https://<key-vault-name>.vault.azure.net",
- "KeyName": "<key-name>",
- "KeyVersion": "<current-version>"
- }
- },
- "location":"<region-name>"
-}
-```
-
-"KeyVaultProperties" contains the Key Vault key identifier details.
-
-*Response*
-
-200 OK and header
-
-### Check cluster update status
-
-The propagation of the Key identifier takes a while to complete. You can check the update state in two ways:
--- Copy the Azure-AsyncOperation URL value from the response and follow the asynchronous operations status check. -
- OR
--- Send a GET request on the *Cluster* resource and look at the *KeyVaultProperties* properties. Your recently updated Key identifier details should return in the response.-
- A response to GET request on the *Cluster* resource should look like this when Key identifier update is complete:
-
- ```json
- {
- "identity": {
- "type": "SystemAssigned",
- "tenantId": "tenant-id",
- "principalId": "principle-id"
- },
- "sku": {
- "name": "capacityReservation",
- "capacity": 1000,
- "lastSkuUpdate": "Sun, 22 Mar 2020 15:39:29 GMT"
- },
- "properties": {
- "keyVaultProperties": {
- "keyVaultUri": "https://key-vault-name.vault.azure.net",
- "kyName": "key-name",
- "keyVersion": "current-version"
- },
- "provisioningState": "Succeeded",
- "billingType": "cluster",
- "clusterId": "cluster-id"
- },
- "id": "/subscriptions/subscription-id/resourceGroups/resource-group-name/providers/Microsoft.OperationalInsights/clusters/cluster-name",
- "name": "cluster-name",
- "type": "Microsoft.OperationalInsights/clusters",
- "location": "region-name"
- }
- ```
-
-## Link a workspace to the cluster
+## Link a workspace to cluster
When a workspace is linked to a dedicated cluster, new data that is ingested into the workspace is routed to the new cluster while existing data remains on the existing cluster. If the dedicated cluster is encrypted using customer-managed keys (CMK), only new data is encrypted with the key. The system is abstracting this difference from the users and the users just query the workspace as usual while the system performs cross-cluster queries on the backend.
@@ -346,22 +226,34 @@ Content-type: application/json
200 OK and header.
-### Using customer-managed keys with linking
-
+### Check workspace link status
+
If you use customer-managed keys, ingested data is stored encrypted with your managed key after the association operation, which can take up to 90 minutes to complete. You can check the workspace association state in two ways: - Copy the Azure-AsyncOperation URL value from the response and follow the asynchronous operations status check. -- Send a [Workspaces ΓÇô Get](/rest/api/loganalytics/workspaces/get) request and observe the response. The associated workspace has a clusterResourceId under "features".
+- Perform Get operation on the workspace and observe if *clusterResourceId* property is present in the response under *features*.
+
+**CLI**
-A send request looks like the following:
+```azurecli
+az monitor log-analytics cluster show --resource-group "resource-group-name" --name "cluster-name"
+```
-*Send*
+**PowerShell**
+
+```powershell
+Get-AzOperationalInsightsWorkspace -ResourceGroupName "resource-group-name" -Name "workspace-name"
+```
+
+**REST**
+
+*Call*
```rest
-GET https://management.azure.com/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.operationalInsights/workspaces/<workspace-name>?api-version=2020-08-01
+GET https://management.azure.com/subscriptions/<subscription-id>/resourcegroups/<resource-group-name>/providers/microsoft.operationalinsights/workspaces/<workspace-name>?api-version=2020-08-01
Authorization: Bearer <token> ```
@@ -397,49 +289,19 @@ Authorization: Bearer <token>
} ```
-## Unlink a workspace from a dedicated cluster
-
-You can unlink a workspace from a cluster. After unlinking a workspace from the cluster, new data associated with this workspace is not sent to the dedicated cluster. Also, the workspace billing is no longer done via the cluster.
-Old data of the unlinked workspace might be left on the cluster. If this data is encrypted using customer-managed keys (CMK), the Key Vault secrets are kept. The system is abstracts this change from Log Analytics users. Users can just query the workspace as usual. The system performs cross-cluster queries on the backend as needed with no indication to users.
-
-> [!WARNING]
-> There is a limit of two linking operations for a specific workspace within a month. Take time to consider and plan unlinking actions accordingly.
-
-## Delete a dedicated cluster
-
-A dedicated cluster resource can be deleted. You must unlink all workspaces from the cluster before deleting it. You need 'write' permissions on the *Cluster* resource to perform this operation.
-
-Once the cluster resource is deleted, the physical cluster enters a purge and deletion process. Deletion of a cluster deletes all the data that was stored on the cluster. The data could be from workspaces that were linked to the cluster in the past.
-
-A *Cluster* resource that was deleted in the last 14 days is in soft-delete state and can be recovered with its data. Since all workspaces got disassociated from the *Cluster* resource with *Cluster* resource deletion, you need to re-associate your workspaces after the recovery. The recovery operation cannot be performed by the user contact your Microsoft channel or support for recovery requests.
-
-Within the 14 days after deletion, the cluster resource name is reserved and cannot be used by other resources.
-
-> [!WARNING]
-> There is a limit of three clusters per subscription. Both active and soft-deleted clusters are counted as part of this. Customers should not create recurrent procedures that create and delete clusters. It has a significant impact on Log Analytics backend systems.
-
-**PowerShell**
-
-Use the following PowerShell command to delete a cluster:
-
- ```powershell
- Remove-AzOperationalInsightsCluster -ResourceGroupName "resource-group-name" -ClusterName "cluster-name"
- ```
-
-**REST**
-
-Use the following REST call to delete a cluster:
+## Change cluster properties
- ```rst
- DELETE https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2020-08-01
- Authorization: Bearer <token>
- ```
+After you create your *Cluster* resource and it is fully provisioned, you can edit additional properties at the cluster level using PowerShell or REST API. Other than the properties that are available during cluster creation, additional properties can only be set after the cluster has been provisioned:
- **Response**
+- **keyVaultProperties** - Updates the key in Azure Key Vault. See [Update cluster with Key identifier details](../platform/customer-managed-keys.md#update-cluster-with-key-identifier-details). It contains the following parameters: *KeyVaultUri*, *KeyName*, *KeyVersion*.
+- **billingType** - The *billingType* property determines the billing attribution for the *cluster* resource and its data:
+ - **Cluster** (default) - The Capacity Reservation costs for your Cluster are attributed to the *Cluster* resource.
+ - **Workspaces** - The Capacity Reservation costs for your Cluster are attributed proportionately to the workspaces in the Cluster, with the *Cluster* resource being billed some of the usage if the total ingested data for the day is under the Capacity Reservation. See [Log Analytics Dedicated Clusters](../platform/manage-cost-storage.md#log-analytics-dedicated-clusters) to learn more about the Cluster pricing model.
- 200 OK
+> [!NOTE]
+> The *billingType* property is not supported in PowerShell.
-## Get all clusters in a resource group
+### Get all clusters in resource group
**CLI**
@@ -497,7 +359,7 @@ Get-AzOperationalInsightsCluster -ResourceGroupName "resource-group-name"
} ```
-## Get all clusters in a subscription
+### Get all clusters in subscription
**CLI**
@@ -524,20 +386,22 @@ Authorization: Bearer <token>
The same as for 'clusters in a resource group', but in subscription scope.
-## Update capacity reservation in cluster
++
+### Update capacity reservation in cluster
When the data volume to your linked workspaces change over time and you want to update the capacity reservation level appropriately. The Capacity is specified in units of GB and can have values of 1000 GB/day or more in increments of 100 GB/day. Note that you donΓÇÖt have to provide the full REST request body but should include the sku. **CLI** ```azurecli
-az monitor log-analytics cluster update --name "cluster-name" --resource-group "resource-group-name" --sku-capacity daily-ingestion-gigabyte
+az monitor log-analytics cluster update --name "cluster-name" --resource-group "resource-group-name" --sku-capacity 1000
``` **PowerShell** ```powershell
-Update-AzOperationalInsightsCluster -ResourceGroupName "resource-group-name" -ClusterName "cluster-name" -SkuCapacity daily-ingestion-gigabyte
+Update-AzOperationalInsightsCluster -ResourceGroupName "resource-group-name" -ClusterName "cluster-name" -SkuCapacity 1000
``` **REST**
@@ -557,7 +421,7 @@ Update-AzOperationalInsightsCluster -ResourceGroupName "resource-group-name" -Cl
} ```
-## Update billingType in cluster
+### Update billingType in cluster
The *billingType* property determines the billing attribution for the cluster and its data: - *cluster* (default) -- The billing is attributed to the subscription hosting your Cluster resource
@@ -579,6 +443,63 @@ The *billingType* property determines the billing attribution for the cluster an
} ```
+### Unlink a workspace from cluster
+
+You can unlink a workspace from a cluster. After unlinking a workspace from the cluster, new data associated with this workspace is not sent to the dedicated cluster. Also, the workspace billing is no longer done via the cluster.
+Old data of the unlinked workspace might be left on the cluster. If this data is encrypted using customer-managed keys (CMK), the Key Vault secrets are kept. The system is abstracts this change from Log Analytics users. Users can just query the workspace as usual. The system performs cross-cluster queries on the backend as needed with no indication to users.
+
+> [!WARNING]
+> There is a limit of two linking operations for a specific workspace within a month. Take time to consider and plan unlinking actions accordingly.
+
+**CLI**
+
+```azurecli
+az monitor log-analytics workspace linked-service delete --resource-group "resource-group-name" --workspace-name "MyWorkspace" --name cluster
+```
+
+**PowerShell**
+
+Use the following PowerShell command to unlink a workspace from cluster:
+
+```powershell
+# Unlink a workspace from cluster
+Remove-AzOperationalInsightsLinkedService -ResourceGroupName {resource-group-name} -WorkspaceName {workspace-name} -LinkedServiceName cluster
+```
+
+### Delete cluster
+
+A dedicated cluster resource can be deleted. You must unlink all workspaces from the cluster before deleting it. You need 'write' permissions on the *Cluster* resource to perform this operation.
+
+Once the cluster resource is deleted, the physical cluster enters a purge and deletion process. Deletion of a cluster deletes all the data that was stored on the cluster. The data could be from workspaces that were linked to the cluster in the past.
+
+A *Cluster* resource that was deleted in the last 14 days is in soft-delete state and can be recovered with its data. Since all workspaces got disassociated from the *Cluster* resource with *Cluster* resource deletion, you need to re-associate your workspaces after the recovery. The recovery operation cannot be performed by the user contact your Microsoft channel or support for recovery requests.
+
+Within the 14 days after deletion, the cluster resource name is reserved and cannot be used by other resources.
+
+> [!WARNING]
+> There is a limit of three clusters per subscription. Both active and soft-deleted clusters are counted as part of this. Customers should not create recurrent procedures that create and delete clusters. It has a significant impact on Log Analytics backend systems.
+
+**PowerShell**
+
+Use the following PowerShell command to delete a cluster:
+
+ ```powershell
+ Remove-AzOperationalInsightsCluster -ResourceGroupName "resource-group-name" -ClusterName "cluster-name"
+ ```
+
+**REST**
+
+Use the following REST call to delete a cluster:
+
+ ```rst
+ DELETE https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2020-08-01
+ Authorization: Bearer <token>
+ ```
+
+ **Response**
+
+ 200 OK
+ ## Limits and constraints - The max number of cluster per region and subscription is 2
azure-monitor https://docs.microsoft.com/en-us/azure/azure-monitor/platform/customer-managed-keys https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/azure-monitor/platform/customer-managed-keys.md
@@ -46,7 +46,7 @@ After the Customer-managed key configuration, new ingested data to workspaces li
3. Dedicated Log Analytics cluster 4. Workspaces linked to *Cluster* resource
-## Encryption keys operation
+### Encryption keys operation
There are 3 types of keys involved in Storage data encryption:
@@ -62,12 +62,13 @@ The following rules apply:
- Your KEK never leaves your Key Vault and in the case of an HSM key, it never leaves the hardware. - Azure Storage uses the managed identity that's associated with the *Cluster* resource to authenticate and access to Azure Key Vault via Azure Active Directory.
-## Customer-Managed key provisioning
+### Customer-Managed key provisioning steps
1. Registering your subscription to allow cluster creation 1. Creating Azure Key Vault and storing key 1. Creating cluster 1. Granting permissions to your Key Vault
+1. Updating cluster with key identifier details
1. Linking Log Analytics workspaces Customer-Managed key configuration isn't supported in Azure portal currently and provisioning can be performed via [PowerShell](/powershell/module/az.operationalinsights/), [CLI](/cli/azure/monitor/log-analytics) or [REST](/rest/api/loganalytics/) requests.
@@ -107,7 +108,7 @@ Authorization: Bearer <token>
Use your contacts into Microsoft or open support request in Log Analytics to provide your subscriptions IDs.
-### Storing encryption key (KEK)
+## Storing encryption key (KEK)
Create or use an Azure Key Vault that you already have to generate, or import a key to be used for data encryption. The Azure Key Vault must be configured as recoverable to protect your key and the access to your data in Azure Monitor. You can verify this configuration under properties in your Key Vault, both *Soft delete* and *Purge protection* should be enabled.
@@ -118,11 +119,11 @@ These settings can be updated in Key Vault via CLI and PowerShell:
- [Soft Delete](../../key-vault/general/soft-delete-overview.md) - [Purge protection](../../key-vault/general/soft-delete-overview.md#purge-protection) guards against force deletion of the secret / vault even after soft delete
-### Create cluster
+## Create cluster
Follow the procedure illustrated in [Dedicated Clusters article](../log-query/logs-dedicated-clusters.md#creating-a-cluster).
-### Grant Key Vault permissions
+## Grant Key Vault permissions
Create access policy in Key Vault to grants permissions to your cluster. These permissions are used by the underlay Azure Monitor storage. Open your Key Vault in Azure portal and click *"Access Policies"* then *"+ Add Access Policy"* to create a policy with these settings:
@@ -133,7 +134,7 @@ Create access policy in Key Vault to grants permissions to your cluster. These p
The *Get* permission is required to verify that your Key Vault is configured as recoverable to protect your key and the access to your Azure Monitor data.
-### Update cluster with Key identifier details
+## Update cluster with key identifier details
All operations on the cluster require the `Microsoft.OperationalInsights/clusters/write` action permission. This permission could be granted via the Owner or Contributor that contains the `*/write` action or via the Log Analytics Contributor role that contains the `Microsoft.OperationalInsights/*` action.
@@ -222,14 +223,14 @@ A response to GET request should look like this when the key update is complete:
---
-### Link workspace to cluster
+## Link workspace to cluster
> [!IMPORTANT] > This step should be performed only after the completion of the Log Analytics cluster provisioning. If you link workspaces and ingest data prior to the provisioning, ingested data will be dropped and won't be recoverable. You need to have 'write' permissions to both your workspace and cluster to perform this operation, which include `Microsoft.OperationalInsights/workspaces/write` and `Microsoft.OperationalInsights/clusters/write`.
-Follow the procedure illustrated in [Dedicated Clusters article](../log-query/logs-dedicated-clusters.md#link-a-workspace-to-the-cluster).
+Follow the procedure illustrated in [Dedicated Clusters article](../log-query/logs-dedicated-clusters.md#link-a-workspace-to-cluster).
## Key revocation
@@ -361,25 +362,14 @@ Learn more about [Customer Lockbox for Microsoft Azure](../../security/fundament
## Customer-Managed key operations
-Customer-Managed key is provided on dedicated cluster and these operations are referred in dedicated cluster
+Customer-Managed key is provided on dedicated cluster and these operations are referred in [dedicated cluster article](../log-query/logs-dedicated-clusters.md#change-cluster-properties)
-- [Check cluster provisioning status](../log-query/logs-dedicated-clusters.md#check-cluster-provisioning-status)--- [Check workspace link status](../log-query/logs-dedicated-clusters.md#check-workspace-link-status)--- [Get all clusters in a resource group](../log-query/logs-dedicated-clusters.md#get-all-clusters-in-a-resource-group)
-
-- [Get all clusters in a subscription](../log-query/logs-dedicated-clusters.md#get-all-clusters-in-a-subscription)--- [Update *capacity reservation* in cluster](../log-query/logs-dedicated-clusters.md#update-capacity-reservation-in-cluster)--- [Update *billingType* in cluster](../log-query/logs-dedicated-clusters.md#update-billingtype-in-cluster)--- [Link a workspace to the cluster](../log-query/logs-dedicated-clusters.md#link-a-workspace-to-the-cluster)--- [Unlink a workspace from a dedicated cluster](../log-query/logs-dedicated-clusters.md#unlink-a-workspace-from-a-dedicated-cluster)--- [Update cluster properties](../log-query/logs-dedicated-clusters.md#change-cluster-properties)
+- Get all clusters in resource group
+- Get all clusters in subscription
+- Update *capacity reservation* in cluster
+- Update *billingType* in cluster
+- Unlink a workspace from cluster
+- Delete cluster
## Limitations and constraints
@@ -411,6 +401,44 @@ Customer-Managed key is provided on dedicated cluster and these operations are r
- If you create a cluster and get an error "<region-name> doesnΓÇÖt support Double Encryption for clusters.", you can still create the cluster without Double Encryption. Add `"properties": {"isDoubleEncryptionEnabled": false}` property in the REST request body. - Double encryption setting can not be changed after the cluster has been created.
+- Error messages
+
+ **Cluster Create**
+ - 400 -- Cluster name is not valid. Cluster name can contain characters a-z, A-Z, 0-9 and length of 3-63.
+ - 400 -- The body of the request is null or in bad format.
+ - 400 -- SKU name is invalid. Set SKU name to capacityReservation.
+ - 400 -- Capacity was provided but SKU is not capacityReservation. Set SKU name to capacityReservation.
+ - 400 -- Missing Capacity in SKU. Set Capacity value to 1000 or higher in steps of 100 (GB).
+ - 400 -- Capacity in SKU is not in range. Should be minimum 1000 and up to the max allowed capacity which is available under ΓÇÿUsage and estimated costΓÇÖ in your workspace.
+ - 400 -- Capacity is locked for 30 days. Decreasing capacity is permitted 30 days after update.
+ - 400 -- No SKU was set. Set the SKU name to capacityReservation and Capacity value to 1000 or higher in steps of 100 (GB).
+ - 400 -- Identity is null or empty. Set Identity with systemAssigned type.
+ - 400 -- KeyVaultProperties are set on creation. Update KeyVaultProperties after cluster creation.
+ - 400 -- Operation cannot be executed now. Async operation is in a state other than succeeded. Cluster must complete its operation before any update operation is performed.
+
+ **Cluster Update**
+ - 400 -- Cluster is in deleting state. Async operation is in progress . Cluster must complete its operation before any update operation is performed.
+ - 400 -- KeyVaultProperties is not empty but has a bad format. See [key identifier update](../platform/customer-managed-keys.md#update-cluster-with-key-identifier-details).
+ - 400 -- Failed to validate key in Key Vault. Could be due to lack of permissions or when key doesnΓÇÖt exist. Verify that you [set key and access policy](../platform/customer-managed-keys.md#grant-key-vault-permissions) in Key Vault.
+ - 400 -- Key is not recoverable. Key Vault must be set to Soft-delete and Purge-protection. See [Key Vault documentation](../../key-vault/general/soft-delete-overview.md)
+ - 400 -- Operation cannot be executed now. Wait for the Async operation to complete and try again.
+ - 400 -- Cluster is in deleting state. Wait for the Async operation to complete and try again.
+
+ **Cluster Get**
+ - 404 -- Cluster not found, the cluster may have been deleted. If you try to create a cluster with that name and get conflict, the cluster is in soft-delete for 14 days. You can contact support to recover it, or use another name to create a new cluster.
+
+ **Cluster Delete**
+ - 409 -- Can't delete a cluster while in provisioning state. Wait for the Async operation to complete and try again.
+
+ **Workspace link**
+ - 404 -- Workspace not found. The workspace you specified doesnΓÇÖt exist or was deleted.
+ - 409 -- Workspace link or unlink operation in process.
+ - 400 -- Cluster not found, the cluster you specified doesnΓÇÖt exist or was deleted. If you try to create a cluster with that name and get conflict, the cluster is in soft-delete for 14 days. You can contact support to recover it.
+
+ **Workspace unlink**
+ - 404 -- Workspace not found. The workspace you specified doesnΓÇÖt exist or was deleted.
+ - 409 -- Workspace link or unlink operation in process.
+ ## Troubleshooting - Behavior with Key Vault availability
azure-monitor https://docs.microsoft.com/en-us/azure/azure-monitor/platform/it-service-management-connector-secure-webhook-connections https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/azure-monitor/platform/it-service-management-connector-secure-webhook-connections.md
@@ -58,8 +58,8 @@ Start using the ITSM Connector tool with these steps:
1. Configure your partner environment. Secure Export supports connections with the following ITSM tools:
-* [ServiceNow](#connect-servicenow-to-azure-monitor)
-* [BMC Helix](#connect-bmc-helix-to-azure-monitor)
+* [ServiceNow](./itsmc-secure-webhook-connections-servicenow.md)
+* [BMC Helix](./itsmc-secure-webhook-connections-bmc.md)
## Register with Azure Active Directory
@@ -108,73 +108,5 @@ The configuration contain 2 steps:
1. Get the URI for the secure export definition. 2. Definitions according to the flow of the ITSM tool. -
-### Connect ServiceNow to Azure Monitor
-
-The following sections provide details about how to connect your ServiceNow product and Secure Export in Azure.
-
-### Prerequisites
-
-Ensure that you've met the following prerequisites:
-
-* Azure AD is registered.
-* You have the supported version of The ServiceNow Event Management - ITOM (version Orlando or later).
-
-### Configure the ServiceNow connection
-
-1.Use the link https://(instance name).service-now.com/api/sn_em_connector/em/inbound_event?source=azuremonitor the URI for the secure export definition.
-
-2. Follow the instructions according to the version:
- * [Paris](https://docs.servicenow.com/bundle/paris-it-operations-management/page/product/event-management/task/azure-events-authentication.html)
- * [Orlando](https://docs.servicenow.com/bundle/orlando-it-operations-management/page/product/event-management/task/azure-events-authentication.html)
- * [New York](https://docs.servicenow.com/bundle/newyork-it-operations-management/page/product/event-management/task/azure-events-authentication.html)
-
-### Connect BMC Helix to Azure Monitor
-
-The following sections provide details about how to connect your BMC Helix product and Secure Export in Azure.
-
-### Prerequisites
-
-Ensure that you've met the following prerequisites:
-
-* Azure AD is registered.
-* You have the supported version of BMC Helix Multi-Cloud Service Management (version 19.08 or later).
-
-### Configure the BMC Helix connection
-
-1.Use the following procedure in the BMC Helix environment in order to get the URI for the secure export:
-
- 1. Log in to Integration Studio.
- 2. Search for the **Create Incident from Azure Alerts** flow.
- 3. Copy the webhook URL .
-
- ![Screenshot of the webhook U R L in Integration Studio.](media/it-service-management-connector-secure-webhook-connections/bmc-url.png)
-
-2. Follow the instructions according to the version:
- * [Enabling prebuilt integration with Azure Monitor for version 20.02](https://docs.bmc.com/docs/multicloud/enabling-prebuilt-integration-with-azure-monitor-879728195.html).
- * [Enabling prebuilt integration with Azure Monitor for version 19.11](https://docs.bmc.com/docs/multicloudprevious/enabling-prebuilt-integration-with-azure-monitor-904157623.html).
-
-3. As a part of the configuration of the connection in BMC Helix, go into your integration BMC instance and follow these instructions:
-
- 1. Select **catalog**.
- 2. Select **Azure alerts**.
- 3. Select **connectors**.
- 4. Select **configuration**.
- 5. Select the **add new connection** configuration.
- 6. Fill in the information for the configuration section:
- - **Name**: Make up your own.
- - **Authorization type**: **NONE**
- - **Description**: Make up your own.
- - **Site**: **Cloud**
- - **Number of instances**: **2**, the default value.
- - **Check**: Selected by default to enable usage.
- - The Azure tenant ID and Azure application ID are taken from the application that you defined earlier.
-
-![Screenshot that shows BMC configuration.](media/it-service-management-connector-secure-webhook-connections/bmc-configuration.png)
---- ## Next steps- * [Create ITSM work items from Azure alerts](./itsmc-overview.md)
azure-monitor https://docs.microsoft.com/en-us/azure/azure-monitor/platform/itsmc-secure-webhook-connections-bmc https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/azure-monitor/platform/itsmc-secure-webhook-connections-bmc.md new file mode 100644
@@ -0,0 +1,53 @@
+---
+title: IT Service Management Connector - Secure Export in Azure Monitor - Configuration with BMC
+description: This article shows you how to connect your ITSM products/services with BMC on Secure Export in Azure Monitor.
+ms.subservice: logs
+ms.topic: conceptual
+author: nolavime
+ms.author: v-jysur
+ms.date: 12/31/2020
+
+---
+
+# Connect BMC Helix to Azure Monitor
+
+The following sections provide details about how to connect your BMC Helix product and Secure Export in Azure.
+
+## Prerequisites
+
+Ensure that you've met the following prerequisites:
+
+* Azure AD is registered.
+* You have the supported version of BMC Helix Multi-Cloud Service Management (version 19.08 or later).
+
+## Configure the BMC Helix connection
+
+1. Use the following procedure in the BMC Helix environment in order to get the URI for the secure export:
+
+ 1. Log in to Integration Studio.
+ 1. Search for the **Create Incident from Azure Alerts** flow.
+ 1. Copy the webhook URL .
+
+ ![Screenshot of the webhook U R L in Integration Studio.](media/it-service-management-connector-secure-webhook-connections/bmc-url.png)
+
+2. Follow the instructions according to the version:
+ * [Enabling prebuilt integration with Azure Monitor for version 20.02](https://docs.bmc.com/docs/multicloud/enabling-prebuilt-integration-with-azure-monitor-879728195.html).
+ * [Enabling prebuilt integration with Azure Monitor for version 19.11](https://docs.bmc.com/docs/multicloudprevious/enabling-prebuilt-integration-with-azure-monitor-904157623.html).
+
+3. As a part of the configuration of the connection in BMC Helix, go into your integration BMC instance and follow these instructions:
+
+ 1. Select **catalog**.
+ 2. Select **Azure alerts**.
+ 3. Select **connectors**.
+ 4. Select **configuration**.
+ 5. Select the **add new connection** configuration.
+ 6. Fill in the information for the configuration section:
+ - **Name**: Make up your own.
+ - **Authorization type**: **NONE**
+ - **Description**: Make up your own.
+ - **Site**: **Cloud**
+ - **Number of instances**: **2**, the default value.
+ - **Check**: Selected by default to enable usage.
+ - The Azure tenant ID and Azure application ID are taken from the application that you defined earlier.
+
+![Screenshot that shows BMC configuration.](media/it-service-management-connector-secure-webhook-connections/bmc-configuration.png)
azure-monitor https://docs.microsoft.com/en-us/azure/azure-monitor/platform/itsmc-secure-webhook-connections-servicenow https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/azure-monitor/platform/itsmc-secure-webhook-connections-servicenow.md new file mode 100644
@@ -0,0 +1,31 @@
+---
+title: IT Service Management Connector - Secure Export in Azure Monitor - Configuration with ServiceNow
+description: This article shows you how to connect your ITSM products/services with ServiceNow on Secure Export in Azure Monitor.
+ms.subservice: logs
+ms.topic: conceptual
+author: nolavime
+ms.author: v-jysur
+ms.date: 12/31/2020
+
+---
++
+# Connect ServiceNow to Azure Monitor
+
+The following sections provide details about how to connect your ServiceNow product and Secure Export in Azure.
+
+## Prerequisites
+
+Ensure that you've met the following prerequisites:
+
+* Azure AD is registered.
+* You have the supported version of The ServiceNow Event Management - ITOM (version Orlando or later).
+
+## Configure the ServiceNow connection
+
+1. Use the link https://(instance name).service-now.com/api/sn_em_connector/em/inbound_event?source=azuremonitor the URI for the secure export definition.
+
+2. Follow the instructions according to the version:
+ * [Paris](https://docs.servicenow.com/bundle/paris-it-operations-management/page/product/event-management/task/azure-events-authentication.html)
+ * [Orlando](https://docs.servicenow.com/bundle/orlando-it-operations-management/page/product/event-management/task/azure-events-authentication.html)
+ * [New York](https://docs.servicenow.com/bundle/newyork-it-operations-management/page/product/event-management/task/azure-events-authentication.html)
\ No newline at end of file
azure-monitor https://docs.microsoft.com/en-us/azure/azure-monitor/platform/metrics-charts https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/azure-monitor/platform/metrics-charts.md
@@ -19,6 +19,35 @@ ms.subservice: metrics
[Metrics in Azure Monitor](data-platform-metrics.md) are the series of measured values and counts that are collected and stored over time. There are standard (or ΓÇ£platformΓÇ¥) metrics, and custom metrics. The standard metrics are provided to you by the Azure platform itself. Standard metrics reflect the health and usage statistics of your Azure resources. Whereas custom metrics are sent to Azure by your applications using the [Application Insights API for custom events and metrics](../app/api-custom-events-metrics.md), [Windows Azure Diagnostics (WAD) extension](./diagnostics-extension-overview.md), or by [Azure Monitor REST API](./metrics-store-custom-rest-api.md).
+## Resource scope picker
+The resource scope picker allows you to view metrics across single and multiple resources. Below are instructions on how to use the resource scope picker.
+
+### Selecting a single resource
+Select **Metrics** from the **Azure Monitor** menu or from the **Monitoring** section of a resource's menu. Click on the "Select a scope" button to open the scope picker, which will allow you to select the resource(s) you want to see metrics for. This should already be populated if you opened metrics explorer from a resource's menu.
+
+![Screenshot of the resource scope picker](./media/metrics-charts/scope-picker.png)
+
+For certain resources, you can only view a single resourceΓÇÖs metrics at a time. These resources are under the ΓÇ£All resource typesΓÇ¥ section in the Resource types dropdown.
+
+![Screenshot of single resource](./media/metrics-charts/single-resource-scope.png)
+
+After clicking your desired resource, you will see all subscriptions and resource groups that contain that resource.
+
+![Screenshot of available resources](./media/metrics-charts/available-single-resource.png)
+
+> [!TIP]
+> If youΓÇÖd like to view multiple resourceΓÇÖs metrics at the same time, or metrics across a subscription or resource group, click the Upvote button.
+
+Once youΓÇÖre satisfied with your selection click ΓÇ£ApplyΓÇ¥.
+
+### Viewing metrics across multiple resources
+Some resource types have enabled the ability to query for metrics over multiple resources, as long as they are within the same subscription and location. These resource types can be found at the top of the ΓÇ£Resource TypesΓÇ¥ dropdown. To get more details on how to view metrics across multiple resources view [this document](metrics-dynamic-scope.md#selecting-multiple-resources).
+
+![Screenshot of cross resource types](./media/metrics-charts/multi-resource-scope.png)
+
+For multi-resource compatible types, you can also query for metrics across a subscription or multiple resource groups. To learn how to do this, view [this article](metrics-dynamic-scope.md#selecting-a-resource-group-or-subscription)
++ ## Create views with multiple metrics and charts You can create charts that plot multiple metrics lines or show multiple metric charts at once. This functionality allows you to:
azure-vmware https://docs.microsoft.com/en-us/azure/azure-vmware/faq https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/azure-vmware/faq.md
@@ -194,6 +194,9 @@ Yes.
#### Can transit connectivity be established between on-premises and Azure VMware Solution through Azure Virtual WAN over ExpressRoute Global Reach? Azure Virtual WAN doesn't provide transitive routing between two connected ExpressRoute circuits and non-virtual WAN ExpressRoute Gateway. Using ExpressRoute Global Reach allows connectivity between on-premises and Azure VMware Solution, but goes through Microsoft's global network instead of the Virtual WAN Hub.
+#### Could I use HCX through public Internet communications as a workaround for the non-supportability of HCX when using VPN S2S with vWAN for on-premises communications?
+
+Currently, the only supported method for HCX is through ExpressRoute.
## Accounts and privileges
backup https://docs.microsoft.com/en-us/azure/backup/backup-azure-restore-files-from-vm https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/backup/backup-azure-restore-files-from-vm.md
@@ -112,6 +112,7 @@ The script also requires Python and bash components to execute and connect secur
| --------------- | ---- | | bash | 4 and above | | python | 2.6.6 and above |
+| .NET | 4.6.2 and above |
| TLS | 1.2 should be supported | ## Step 4: Access requirements to successfully run the script
backup https://docs.microsoft.com/en-us/azure/backup/backup-azure-sap-hana-database https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/backup/backup-azure-sap-hana-database.md
@@ -86,6 +86,9 @@ You can also use the following FQDNs to allow access to the required services fr
When you back up an SAP HANA database running on an Azure VM, the backup extension on the VM uses the HTTPS APIs to send management commands to Azure Backup and data to Azure Storage. The backup extension also uses Azure AD for authentication. Route the backup extension traffic for these three services through the HTTP proxy. Use the list of IPs and FQDNs mentioned above for allowing access to the required services. Authenticated proxy servers aren't supported.
+> [!NOTE]
+> There is no service level proxy support. That is, traffic via the proxy from only a few or selected services (Azure backup services) isn't supported. The entire data or traffic can either be routed by proxy or not.
+ [!INCLUDE [How to create a Recovery Services vault](../../includes/backup-create-rs-vault.md)] ## Discover the databases
backup https://docs.microsoft.com/en-us/azure/backup/backup-azure-vms-troubleshoot https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/backup/backup-azure-vms-troubleshoot.md
@@ -70,6 +70,16 @@ Error message: Failed to freeze one or more mount-points of the VM to take a fil
* Run a file system consistency check on these devices by using the **fsck** command. * Mount the devices again and retry backup operation.</ol>
+If you can't un-mount the devices then you can update the VM backup configuration to ignore certain mount points. For example, if '/mnt/resource' mount point can't be un-mounted and causing the VM backup failures, you can update the VM backup configuration files with the ```MountsToSkip``` property as follows.
+
+```bash
+cat /var/lib/waagent/Microsoft.Azure.RecoveryServices.VMSnapshotLinux-1.0.9170.0/main/tempPlugin/vmbackup.conf[SnapshotThread]
+fsfreeze: True
+MountsToSkip = /mnt/resource
+SafeFreezeWaitInSeconds=600
+```
++ ### ExtensionSnapshotFailedCOM / ExtensionInstallationFailedCOM / ExtensionInstallationFailedMDTC - Extension installation/operation failed due to a COM+ error Error code: ExtensionSnapshotFailedCOM <br/>
batch https://docs.microsoft.com/en-us/azure/batch/batch-compute-node-environment-variables https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/batch/batch-compute-node-environment-variables.md
@@ -2,18 +2,18 @@
title: Task runtime environment variables description: Task runtime environment variable guidance and reference for Azure Batch Analytics. ms.topic: conceptual
-ms.date: 09/12/2019
+ms.date: 12/30/2020
--- # Azure Batch runtime environment variables The [Azure Batch service](https://azure.microsoft.com/services/batch/) sets the following environment variables on compute nodes. You can reference these environment variables in task command lines, and in the programs and scripts run by the command lines.
-For more information about using environment variables with Batch, see [Environment settings for tasks](./jobs-and-tasks.md#environment-settings-for-tasks).
+For more information about using environment variables with Batch, see [Environment settings for tasks](jobs-and-tasks.md#environment-settings-for-tasks).
## Environment variable visibility
-These environment variables are visible only in the context of the **task user**, the user account on the node under which a task is executed. You will *not* see these if you [connect remotely](./error-handling.md#connect-to-compute-nodes) to a compute node via Remote Desktop Protocol (RDP) or Secure Shell (SSH) and list the environment variables. This is because the user account that is used for remote connection is not the same as the account that is used by the task.
+These environment variables are visible only in the context of the **task user**, which is the user account on the node under which a task is executed. You won't see these variables when [connecting remotely](./error-handling.md#connect-to-compute-nodes) to a compute node via Remote Desktop Protocol (RDP) or Secure Shell (SSH) and listing environment variables. This is because the user account that is used for remote connection is not the same as the account that is used by the task.
To get the current value of an environment variable, launch `cmd.exe` on a Windows compute node or `/bin/sh` on a Linux node:
@@ -23,7 +23,7 @@ To get the current value of an environment variable, launch `cmd.exe` on a Windo
## Command-line expansion of environment variables
-The command lines executed by tasks on compute nodes do not run under a shell. Therefore, these command lines cannot natively take advantage of shell features such as environment variable expansion (this includes the `PATH`). To take advantage of such features, you must **invoke the shell** in the command line. For example, launch `cmd.exe` on Windows compute nodes or `/bin/sh` on Linux nodes:
+The command lines executed by tasks on compute nodes don't run under a shell. This means that these command lines can't natively use shell features such as environment variable expansion (including the `PATH`). To use such features, you must invoke the shell in the command line. For example, launch `cmd.exe` on Windows compute nodes or `/bin/sh` on Linux nodes:
`cmd /c MyTaskApplication.exe %MY_ENV_VAR%`
@@ -35,30 +35,31 @@ The command lines executed by tasks on compute nodes do not run under a shell. T
|-----------------------------------|--------------------------------------------------------------------------|--------------|---------| | AZ_BATCH_ACCOUNT_NAME | The name of the Batch account that the task belongs to. | All tasks. | mybatchaccount | | AZ_BATCH_ACCOUNT_URL | The URL of the Batch account. | All tasks. | `https://myaccount.westus.batch.azure.com` |
-| AZ_BATCH_APP_PACKAGE | A prefix of all the app package environment variables. For example, if Application "FOO" version "1" is installed onto a pool, the environment variable is AZ_BATCH_APP_PACKAGE_FOO_1 (on Linux) or AZ_BATCH_APP_PACKAGE_FOO#1 (on Windows). AZ_BATCH_APP_PACKAGE_FOO_1 points to the location that the package was downloaded (a folder). When using the default version of the app package, use the AZ_BATCH_APP_PACKAGE environment variable without the version numbers. If in Linux, and the application package name is "Agent-linux-x64" and the version is "1.1.46.0, the environment name is actually: AZ_BATCH_APP_PACKAGE_agent_linux_x64_1_1_46_0, using underscores and lower case. See [here](./batch-application-packages.md#execute-the-installed-applications) for more details. | Any task with an associated app package. Also available for all tasks if the node itself has application packages. | AZ_BATCH_APP_PACKAGE_FOO_1 (Linux) or AZ_BATCH_APP_PACKAGE_FOO#1 (Windows) |
+| AZ_BATCH_APP_PACKAGE | A prefix of all the app package environment variables. For example, if Application "FOO" version "1" is installed onto a pool, the environment variable is AZ_BATCH_APP_PACKAGE_FOO_1 (on Linux) or AZ_BATCH_APP_PACKAGE_FOO#1 (on Windows). AZ_BATCH_APP_PACKAGE_FOO_1 points to the location that the package was downloaded (a folder). When using the default version of the app package, use the AZ_BATCH_APP_PACKAGE environment variable without the version numbers. If in Linux, and the application package name is "Agent-linux-x64" and the version is "1.1.46.0, the environment name is actually: AZ_BATCH_APP_PACKAGE_agent_linux_x64_1_1_46_0, using underscores and lower case. For more information, see [Execute the installed applications](batch-application-packages.md#execute-the-installed-applications) for more details. | Any task with an associated app package. Also available for all tasks if the node itself has application packages. | AZ_BATCH_APP_PACKAGE_FOO_1 (Linux) or AZ_BATCH_APP_PACKAGE_FOO#1 (Windows) |
| AZ_BATCH_AUTHENTICATION_TOKEN | An authentication token that grants access to a limited set of Batch service operations. This environment variable is only present if the [authenticationTokenSettings](/rest/api/batchservice/task/add#authenticationtokensettings) are set when the [task is added](/rest/api/batchservice/task/add#request-body). The token value is used in the Batch APIs as credentials to create a Batch client, such as in the [BatchClient.Open() .NET API](/dotnet/api/microsoft.azure.batch.batchclient.open#Microsoft_Azure_Batch_BatchClient_Open_Microsoft_Azure_Batch_Auth_BatchTokenCredentials_). | All tasks. | OAuth2 access token |
-| AZ_BATCH_CERTIFICATES_DIR | A directory within the [task working directory][files_dirs] in which certificates are stored for Linux compute nodes. This environment variable does not apply to Windows compute nodes. | All tasks. | /mnt/batch/tasks/workitems/batchjob001/job-1/task001/certs |
-| AZ_BATCH_HOST_LIST | The list of nodes that are allocated to a [multi-instance task][multi_instance] in the format `nodeIP,nodeIP`. | Multi-instance primary and subtasks. | `10.0.0.4,10.0.0.5` |
-| AZ_BATCH_IS_CURRENT_NODE_MASTER | Specifies whether the current node is the master node for a [multi-instance task][multi_instance]. Possible values are `true` and `false`.| Multi-instance primary and subtasks. | `true` |
+| AZ_BATCH_CERTIFICATES_DIR | A directory within the [task working directory](files-and-directories.md) in which certificates are stored for Linux compute nodes. This environment variable does not apply to Windows compute nodes. | All tasks. | /mnt/batch/tasks/workitems/batchjob001/job-1/task001/certs |
+| AZ_BATCH_HOST_LIST | The list of nodes that are allocated to a [multi-instance task](batch-mpi.md) in the format `nodeIP,nodeIP`. | Multi-instance primary and subtasks. | `10.0.0.4,10.0.0.5` |
+| AZ_BATCH_IS_CURRENT_NODE_MASTER | Specifies whether the current node is the master node for a [multi-instance task](batch-mpi.md). Possible values are `true` and `false`.| Multi-instance primary and subtasks. | `true` |
| AZ_BATCH_JOB_ID | The ID of the job that the task belongs to. | All tasks except start task. | batchjob001 |
-| AZ_BATCH_JOB_PREP_DIR | The full path of the job preparation [task directory][files_dirs] on the node. | All tasks except start task and job preparation task. Only available if the job is configured with a job preparation task. | C:\user\tasks\workitems\jobprepreleasesamplejob\job-1\jobpreparation |
-| AZ_BATCH_JOB_PREP_WORKING_DIR | The full path of the job preparation [task working directory][files_dirs] on the node. | All tasks except start task and job preparation task. Only available if the job is configured with a job preparation task. | C:\user\tasks\workitems\jobprepreleasesamplejob\job-1\jobpreparation\wd |
-| AZ_BATCH_MASTER_NODE | The IP address and port of the compute node on which the primary task of a [multi-instance task][multi_instance] runs. Do not use the port specified here for MPI or NCCL communication - it is reserved for the Azure Batch service. Use the variable MASTER_PORT instead, either by setting it with a value passed in through command line argument (port 6105 is a good default choice), or using the value AML sets if it does so. | Multi-instance primary and subtasks. | `10.0.0.4:6000` |
+| AZ_BATCH_JOB_PREP_DIR | The full path of the job preparation [task directory](files-and-directories.md) on the node. | All tasks except start task and job preparation task. Only available if the job is configured with a job preparation task. | C:\user\tasks\workitems\jobprepreleasesamplejob\job-1\jobpreparation |
+| AZ_BATCH_JOB_PREP_WORKING_DIR | The full path of the job preparation [task working directory](files-and-directories.md) on the node. | All tasks except start task and job preparation task. Only available if the job is configured with a job preparation task. | C:\user\tasks\workitems\jobprepreleasesamplejob\job-1\jobpreparation\wd |
+| AZ_BATCH_MASTER_NODE | The IP address and port of the compute node on which the primary task of a [multi-instance task](batch-mpi.md) runs. Do not use the port specified here for MPI or NCCL communication - it is reserved for the Azure Batch service. Use the variable MASTER_PORT instead, either by setting it with a value passed in through command line argument (port 6105 is a good default choice), or using the value AML sets if it does so. | Multi-instance primary and subtasks. | `10.0.0.4:6000` |
| AZ_BATCH_NODE_ID | The ID of the node that the task is assigned to. | All tasks. | tvm-1219235766_3-20160919t172711z | | AZ_BATCH_NODE_IS_DEDICATED | If `true`, the current node is a dedicated node. If `false`, it is a [low-priority node](batch-low-pri-vms.md). | All tasks. | `true` |
-| AZ_BATCH_NODE_LIST | The list of nodes that are allocated to a [multi-instance task][multi_instance] in the format `nodeIP;nodeIP`. | Multi-instance primary and subtasks. | `10.0.0.4;10.0.0.5` |
+| AZ_BATCH_NODE_LIST | The list of nodes that are allocated to a [multi-instance task](batch-mpi.md) in the format `nodeIP;nodeIP`. | Multi-instance primary and subtasks. | `10.0.0.4;10.0.0.5` |
| AZ_BATCH_NODE_MOUNTS_DIR | The full path of the node level [file system mount](virtual-file-mount.md) location where all mount directories reside. Windows file shares use a drive letter, so for Windows, the mount drive is part of devices and drives. | All tasks including start task have access to the user, given the user is aware of the mount permissions for the mounted directory. | In Ubuntu, for example, the location is: `/mnt/batch/tasks/fsmounts` |
-| AZ_BATCH_NODE_ROOT_DIR | The full path of the root of all [Batch directories][files_dirs] on the node. | All tasks. | C:\user\tasks |
-| AZ_BATCH_NODE_SHARED_DIR | The full path of the [shared directory][files_dirs] on the node. All tasks that execute on a node have read/write access to this directory. Tasks that execute on other nodes do not have remote access to this directory (it is not a "shared" network directory). | All tasks. | C:\user\tasks\shared |
-| AZ_BATCH_NODE_STARTUP_DIR | The full path of the [start task directory][files_dirs] on the node. | All tasks. | C:\user\tasks\startup |
+| AZ_BATCH_NODE_ROOT_DIR | The full path of the root of all [Batch directories](files-and-directories.md) on the node. | All tasks. | C:\user\tasks |
+| AZ_BATCH_NODE_SHARED_DIR | The full path of the [shared directory](files-and-directories.md) on the node. All tasks that execute on a node have read/write access to this directory. Tasks that execute on other nodes do not have remote access to this directory (it is not a "shared" network directory). | All tasks. | C:\user\tasks\shared |
+| AZ_BATCH_NODE_STARTUP_DIR | The full path of the [start task directory](files-and-directories.md) on the node. | All tasks. | C:\user\tasks\startup |
| AZ_BATCH_POOL_ID | The ID of the pool that the task is running on. | All tasks. | batchpool001 |
-| AZ_BATCH_TASK_DIR | The full path of the [task directory][files_dirs] on the node. This directory contains the `stdout.txt` and `stderr.txt` for the task, and the AZ_BATCH_TASK_WORKING_DIR. | All tasks. | C:\user\tasks\workitems\batchjob001\job-1\task001 |
+| AZ_BATCH_TASK_DIR | The full path of the [task directory](files-and-directories.md) on the node. This directory contains the `stdout.txt` and `stderr.txt` for the task, and the AZ_BATCH_TASK_WORKING_DIR. | All tasks. | C:\user\tasks\workitems\batchjob001\job-1\task001 |
| AZ_BATCH_TASK_ID | The ID of the current task. | All tasks except start task. | task001 |
-| AZ_BATCH_TASK_SHARED_DIR | A directory path that is identical for the primary task and every subtask of a [multi-instance task][multi_instance]. The path exists on every node on which the multi-instance task runs, and is read/write accessible to the task commands running on that node (both the [coordination command][coord_cmd] and the [application command][app_cmd]). Subtasks or a primary task that execute on other nodes do not have remote access to this directory (it is not a "shared" network directory). | Multi-instance primary and subtasks. | C:\user\tasks\workitems\multiinstancesamplejob\job-1\multiinstancesampletask |
-| AZ_BATCH_TASK_WORKING_DIR | The full path of the [task working directory][files_dirs] on the node. The currently running task has read/write access to this directory. | All tasks. | C:\user\tasks\workitems\batchjob001\job-1\task001\wd |
-| CCP_NODES | The list of nodes and number of cores per node that are allocated to a [multi-instance task][multi_instance]. Nodes and cores are listed in the format `numNodes<space>node1IP<space>node1Cores<space>`<br/>`node2IP<space>node2Cores<space> ...`, where the number of nodes is followed by one or more node IP addresses and the number of cores for each. | Multi-instance primary and subtasks. |`2 10.0.0.4 1 10.0.0.5 1` |
-
-[files_dirs]: ./files-and-directories.md
-[multi_instance]: ./batch-mpi.md
-[coord_cmd]: ./batch-mpi.md#coordination-command
-[app_cmd]: ./batch-mpi.md#application-command
+| AZ_BATCH_TASK_SHARED_DIR | A directory path that is identical for the primary task and every subtask of a [multi-instance task](batch-mpi.md). The path exists on every node on which the multi-instance task runs, and is read/write accessible to the task commands running on that node (both the [coordination command](batch-mpi.md#coordination-command) and the [application command](batch-mpi.md#application-command). Subtasks or a primary task that execute on other nodes do not have remote access to this directory (it is not a "shared" network directory). | Multi-instance primary and subtasks. | C:\user\tasks\workitems\multiinstancesamplejob\job-1\multiinstancesampletask |
+| AZ_BATCH_TASK_WORKING_DIR | The full path of the [task working directory](files-and-directories.md) on the node. The currently running task has read/write access to this directory. | All tasks. | C:\user\tasks\workitems\batchjob001\job-1\task001\wd |
+| CCP_NODES | The list of nodes and number of cores per node that are allocated to a [multi-instance task](batch-mpi.md). Nodes and cores are listed in the format `numNodes<space>node1IP<space>node1Cores<space>`<br/>`node2IP<space>node2Cores<space> ...`, where the number of nodes is followed by one or more node IP addresses and the number of cores for each. | Multi-instance primary and subtasks. |`2 10.0.0.4 1 10.0.0.5 1` |
+
+## Next steps
+
+- Learn how to [use environment variables with Batch](jobs-and-tasks.md#environment-settings-for-tasks).
+- Learn more about [files and directories in Batch](files-and-directories.md)
+- Learn about [multi-instance-tasks](batch-mpi.md).
batch https://docs.microsoft.com/en-us/azure/batch/high-availability-disaster-recovery https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/batch/high-availability-disaster-recovery.md
@@ -2,37 +2,37 @@
title: High availability and disaster recovery description: Learn how to design your Batch application for a regional outage. ms.topic: how-to
-ms.date: 01/29/2019
+ms.date: 12/30/2020
---
-# Design your application for high availability
+# Design your Batch application for high availability
-Azure Batch is a regional service. Batch is available in all Azure regions, but when a Batch account is created it must be associated with a region. All operations for the Batch account then apply to that region. For example, pools and associated virtual machines (VMs) are created in the same region as the Batch account.
+Azure Batch is available in all Azure regions, but when a Batch account is created it must be associated with one specific region. All operations for the Batch account then apply to that region. For example, pools and associated virtual machines (VMs) are created in the same region as the Batch account.
-When designing an application that uses Batch, you must consider the possibility of Batch not being available in a region. It's possible to encounter a rare situation where there is a problem with the region as a whole, the entire Batch service in the region, or a problem with your specific Batch account.
+When designing an application that uses Batch, you must consider the possibility of Batch not being available in a region. It's possible to encounter a rare situation where there is a problem with the region as a whole, the entire Batch service in the region, or your specific Batch account.
If the application or solution using Batch always needs to be available, then it should be designed to either failover to another region or always have the workload split between two or more regions. Both approaches require at least two Batch accounts, with each account located in a different region. ## Multiple Batch accounts in multiple regions
-Using multiple Batch accounts in various regions provides the ability for your application to continue running if a Batch account in another region becomes unavailable. Using multiple accounts is especially important if your application needs to be highly available.
+Using multiple Batch accounts in various regions lets your application continue running if a Batch account in one region becomes unavailable. If your application needs to be highly available, having multiple accounts is especially important.
-In some cases, an application may be designed to always use two or more regions. For example, when you need a considerable amount of capacity, using multiple regions may be needed to handle either a large-scale application or cater for future growth.
+In some cases, applications may be designed to intentionally use two or more regions. For example, when you need a considerable amount of capacity, using multiple regions may be needed to handle either a large-scale application or cater for future growth. These applications will also require multiple Batch accounts (one per region used).
## Design considerations for providing failover
-A key point to consider when providing the ability to failover to an alternate region is that all components in a solution need to be considered; it is not sufficient to simply have a second Batch account. For example, in most Batch applications, an Azure storage account is required, with the storage account and Batch account needing to be in the same region for acceptable performance.
+When providing the ability to failover to an alternate region, all components in a solution need to be considered; it is not sufficient to simply have a second Batch account. For example, in most Batch applications, an Azure storage account is required, with the storage account and Batch account needing to be in the same region for acceptable performance.
Consider the following points when designing a solution that can failover: -- Pre-create all required accounts in each region, such as the Batch account and storage account. There often is not any charge for having accounts created, only when there is data stored or the account is used.-- Make sure quotas are set on the accounts ahead of time, so you can allocate the required number of cores using the Batch account.
+- Pre-create all required accounts in each region, such as the Batch account and storage account. There is often no charge for having accounts created, and charges accrue only when the account is used or when data is stored.
+- Make sure the appropriate [quotas](batch-quota-limit.md) are set on all accounts ahead of time, so you can allocate the required number of cores using the Batch account.
- Use templates and/or scripts to automate the deployment of the application in a region. - Keep application binaries and reference data up-to-date in all regions. Staying up-to-date will ensure the region can be brought online quickly without having to wait for the upload and deployment of files. For example, if a custom application to install on pool nodes is stored and referenced using Batch application packages, then when a new version of the application is produced, it should be uploaded to each Batch account and referenced by the pool configuration (or make the new version the default version).-- In the application calling Batch, storage, and any other services, easily switchover clients or the load to the different region.-- A best practice to ensure a failover will be successful is to frequently switchover to an alternate region as part of normal operation. For example, with two deployments in separate regions, switchover to the alternate region every month.
+- In the application calling Batch, storage, and any other services, make it easy to switch over clients or the load to different regions.
+- Consider frequently switching over to an alternate region as part of normal operation. For example, with two deployments in separate regions, switch over to the alternate region every month.
## Next steps - Learn more about creating Batch accounts with the [Azure portal](batch-account-create-portal.md), the [Azure CLI](./scripts/batch-cli-sample-create-account.md), [PowerShell](batch-powershell-cmdlets-get-started.md), or the [Batch management API](batch-management-dotnet.md).-- Default quotas are associated with a Batch account; [this article](batch-quota-limit.md) details the default quota values and describes how the quotas can be increased.
+- Learn about the [default quotas associated with a Batch account](batch-quota-limit.md) and how quotas can be increased.
batch https://docs.microsoft.com/en-us/azure/batch/large-number-tasks https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/batch/large-number-tasks.md
@@ -1,48 +1,62 @@
---
-title: Submit a large number of tasks
-description: How to efficiently submit a very large number of tasks in a single Azure Batch job
+title: Submit a large number of tasks to a Batch job
+description: Learn how to efficiently submit a very large number of tasks in a single Azure Batch job.
ms.topic: how-to
-ms.date: 08/24/2018
+ms.date: 12/30/2020
ms.custom: "devx-track-python, devx-track-csharp" --- # Submit a large number of tasks to a Batch job
-When you run large-scale Azure Batch workloads, you might want to submit tens of thousands, hundreds of thousands, or even more tasks to a single job.
+When you run large-scale Azure Batch workloads, you might want to submit tens of thousands, hundreds of thousands, or even more tasks to a single job.
-This article gives guidance and some code examples to submit large numbers of tasks with substantially increased throughput to a single Batch job. After tasks are submitted, they enter the Batch queue for processing on the pool you specify for the job.
+This article shows you how to submit large numbers of tasks with substantially increased throughput to a single Batch job. After tasks are submitted, they enter the Batch queue for processing on the pool you specify for the job.
## Use task collections
-The Batch APIs provide methods to efficiently add tasks to a job as a *collection*, in addition to one at a time. When adding a large number of tasks, you should use the appropriate methods or overloads to add tasks as a collection. Generally, you construct a task collection by defining tasks as you iterate over a set of input files or parameters for your job.
+When adding a large number of tasks, use the appropriate methods or overloads provided by the Batch APIs to add tasks as a *collection* rather than one at a time. Generally, you construct a task collection by defining tasks as you iterate over a set of input files or parameters for your job.
-The maximum size of the task collection that you can add in a single call depends on the Batch API you use:
+The maximum size of the task collection that you can add in a single call depends on the Batch API you use.
-* The following Batch APIs limit the collection to **100 tasks**. The limit could be smaller depending on the size of the tasks - for example, if the tasks have a large number of resource files or environment variables.
+### APIs allowing collections of up to 100 tasks
- * [REST API](/rest/api/batchservice/task/addcollection)
- * [Python API](/python/api/azure-batch/azure.batch.operations.TaskOperations)
- * [Node.js API](/javascript/api/@azure/batch/task)
+These Batch APIs limit the collection to 100 tasks. The limit could be smaller depending on the size of the tasks (for example, if the tasks have a large number of resource files or environment variables).
- When using these APIs, you need to provide logic to divide the number of tasks to meet the collection limit, and to handle errors and retries in case addition of tasks fails. If a task collection is too large to add, the request generates an error and should be retried again with fewer tasks.
+- [REST API](/rest/api/batchservice/task/addcollection)
+- [Python API](/python/api/azure-batch/azure.batch.operations.TaskOperations)
+- [Node.js API](/javascript/api/@azure/batch/task)
-* The following APIs support much larger task collections - limited only by RAM availability on the submitting client. These APIs transparently handle dividing the task collection into "chunks" for the lower-level APIs and retries if addition of tasks fails.
+When using these APIs, you need to provide logic to divide the number of tasks to meet the collection limit, and to handle errors and retries in case of task addition failures. If a task collection is too large to add, the request generates an error and should be retried again with fewer tasks.
- * [.NET API](/dotnet/api/microsoft.azure.batch.cloudjob.addtaskasync)
- * [Java API](/java/api/com.microsoft.azure.batch.protocol.tasks.addcollectionasync)
- * [Azure Batch CLI extension](batch-cli-templates.md) with Batch CLI templates
- * [Python SDK extension](https://pypi.org/project/azure-batch-extensions/)
+### APIs allowing collections of larger numbers of tasks
+
+Other Batch APIs support much larger task collections, limited only by RAM availability on the submitting client. These APIs transparently handle dividing the task collection into "chunks" for the lower-level APIs and retries for task addition failures.
+
+- [.NET API](/dotnet/api/microsoft.azure.batch.cloudjob.addtaskasync)
+- [Java API](/java/api/com.microsoft.azure.batch.protocol.tasks.addcollectionasync)
+- [Azure Batch CLI extension](batch-cli-templates.md) with Batch CLI templates
+- [Python SDK extension](https://pypi.org/project/azure-batch-extensions/)
## Increase throughput of task submission
-It can take some time to add a large collection of tasks to a job - for example, up to 1 minute to add 20,000 tasks via the .NET API. Depending on the Batch API and your workload, you can improve the task throughput by modifying one or more of the following:
+It can take some time to add a large collection of tasks to a job. For example, adding 20,000 tasks via the .NET API might take up to one minute. Depending on the Batch API and your workload, you can improve task throughput by modifying one or more of the following.
+
+### Task size
+
+Adding large tasks takes longer than adding smaller ones. To reduce the size of each task in a collection, you can simplify the task command line, reduce the number of environment variables, or handle requirements for task execution more efficiently.
+
+For example, instead of using a large number of resource files, install task dependencies using a [start task](jobs-and-tasks.md#start-task) on the pool, or use an [application package](batch-application-packages.md) or [Docker container](batch-docker-container-workloads.md).
+
+### Number of parallel operations
-* **Task size** - Adding large tasks takes longer than adding smaller ones. To reduce the size of each task in a collection, you can simplify the task command line, reduce the number of environment variables, or handle requirements for task execution more efficiently. For example, instead of using a large number of resource files, install task dependencies using a [start task](jobs-and-tasks.md#start-task) on the pool or use an [application package](batch-application-packages.md) or [Docker container](batch-docker-container-workloads.md).
+Depending on the Batch API, you can increase throughput by increasing the maximum number of concurrent operations by the Batch client. Configure this setting using the [BatchClientParallelOptions.MaxDegreeOfParallelism](/dotnet/api/microsoft.azure.batch.batchclientparalleloptions.maxdegreeofparallelism) property in the .NET API, or the `threads` parameter of methods such as [TaskOperations.add_collection](/python/api/azure-batch/azure.batch.operations.TaskOperations) in the Batch Python SDK extension. (This property is not available in the native Batch Python SDK.)
-* **Number of parallel operations** - Depending on the Batch API, increase throughput by increasing the maximum number of concurrent operations by the Batch client. Configure this setting using the [BatchClientParallelOptions.MaxDegreeOfParallelism](/dotnet/api/microsoft.azure.batch.batchclientparalleloptions.maxdegreeofparallelism) property in the .NET API, or the `threads` parameter of methods such as [TaskOperations.add_collection](/python/api/azure-batch/azure.batch.operations.TaskOperations) in the Batch Python SDK extension. (This property is not available in the native Batch Python SDK.) By default, this property is set to 1, but set it higher to improve throughput of operations. You trade off increased throughput by consuming network bandwidth and some CPU performance. Task throughput increases by up to 100 times the `MaxDegreeOfParallelism` or `threads`. In practice, you should set the number of concurrent operations below 100.
-
- The Azure Batch CLI extension with Batch templates increases the number of concurrent operations automatically based on the number of available cores, but this property is not configurable in the CLI.
+By default, this property is set to 1, but you can set it higher to improve throughput of operations. You trade off increased throughput by consuming network bandwidth and some CPU performance. Task throughput increases by up to 100 times the `MaxDegreeOfParallelism` or `threads`. In practice, you should set the number of concurrent operations to below 100.
-* **HTTP connection limits** - The number of concurrent HTTP connections can throttle the performance of the Batch client when it is adding large numbers of tasks. The number of HTTP connections is limited with certain APIs. When developing with the .NET API, for example, the [ServicePointManager.DefaultConnectionLimit](/dotnet/api/system.net.servicepointmanager.defaultconnectionlimit) property is set to 2 by default. We recommend that you increase the value to a number close to or greater than the number of parallel operations.
+ The Azure Batch CLI extension with Batch templates increases the number of concurrent operations automatically based on the number of available cores, but this property is not configurable in the CLI.
+
+### HTTP connection limits
+
+Having many concurrent HTTP connections can throttle the performance of the Batch client when it is adding large numbers of tasks. Some APIs limit the number of HTTP connections. When developing with the .NET API, for example, the [ServicePointManager.DefaultConnectionLimit](/dotnet/api/system.net.servicepointmanager.defaultconnectionlimit) property is set to 2 by default. We recommend that you increase the value to a number close to or greater than the number of parallel operations.
## Example: Batch .NET
@@ -57,6 +71,7 @@ BatchClientParallelOptions parallelOptions = new BatchClientParallelOptions()
}; ... ```+ Add a task collection to the job using the appropriate overload of the [AddTaskAsync](/dotnet/api/microsoft.azure.batch.cloudjob.addtaskasync) or [AddTask](/dotnet/api/microsoft.azure.batch.cloudjob.addtask ) method. For example:
@@ -67,12 +82,11 @@ List<CloudTask> tasksToAdd = new List<CloudTask>(); // Populate with your tasks
await batchClient.JobOperations.AddTaskAsync(jobId, tasksToAdd, parallelOptions); ``` - ## Example: Batch CLI extension Using the Azure Batch CLI extensions with [Batch CLI templates](batch-cli-templates.md), create a job template JSON file that includes a [task factory](https://github.com/Azure/azure-batch-cli-extensions/blob/master/doc/taskFactories.md). The task factory configures a collection of related tasks for a job from a single task definition.
-The following is a sample job template for a one-dimensional parametric sweep job with a large number of tasks - in this case, 250,000. The task command line is a simple `echo` command.
+The following is a sample job template for a one-dimensional parametric sweep job with a large number of tasks (in this case, 250,000). The task command line is a simple `echo` command.
```json {
@@ -109,6 +123,7 @@ The following is a sample job template for a one-dimensional parametric sweep jo
} } ```+ To run a job with the template, see [Use Azure Batch CLI templates and file transfer](batch-cli-templates.md). ## Example: Batch Python SDK extension
@@ -131,7 +146,6 @@ client = batch.BatchExtensionsClient(
Create a collection of tasks to add to a job. For example: - ```python tasks = list() # Populate the list with your tasks
@@ -195,5 +209,6 @@ except Exception as e:
## Next steps
-* Learn more about using the Azure Batch CLI extension with [Batch CLI templates](batch-cli-templates.md).
-* Learn more about the [Batch Python SDK extension](https://pypi.org/project/azure-batch-extensions/).
+- Learn more about using the Azure Batch CLI extension with [Batch CLI templates](batch-cli-templates.md).
+- Learn more about the [Batch Python SDK extension](https://pypi.org/project/azure-batch-extensions/).
+- Read about [best practices for Azure Batch](best-practices.md).
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/Bing-Autosuggest/includes/quickstarts/autosuggest-client-library-csharp https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/Bing-Autosuggest/includes/quickstarts/autosuggest-client-library-csharp.md
@@ -14,7 +14,7 @@ Get started with the Bing Autosuggest client library for .NET. Follow these step
Use the Bing Autosuggest client library for .NET to get search suggestions based on partial query strings.
-[Reference documentation](/dotnet/api/overview/azure/cognitiveservices/client/bingautosuggest?view=azure-dotnet) | [Library source code](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/cognitiveservices/Search.BingAutoSuggest) | [Package (NuGet)](https://www.nuget.org/packages/Microsoft.Azure.CognitiveServices.Search.AutoSuggest/) | [Sample code](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/dotnet/BingAutoSuggest/Program.cs)
+[Reference documentation](/dotnet/api/overview/azure/cognitiveservices/bing-autosuggest-readme?view=azure-dotnet) | [Library source code](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/cognitiveservices/Search.BingAutoSuggest) | [Package (NuGet)](https://www.nuget.org/packages/Microsoft.Azure.CognitiveServices.Search.AutoSuggest/) | [Sample code](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/dotnet/BingAutoSuggest/Program.cs)
## Prerequisites
@@ -211,4 +211,4 @@ If you want to clean up and remove a Cognitive Services subscription, you can de
## See also - [What is Bing Autosuggest?](../../get-suggested-search-terms.md)-- [Bing Autosuggest dotnet reference](/dotnet/api/overview/azure/cognitiveservices/client/bingautosuggest?view=azure-dotnet)\ No newline at end of file
+- [Bing Autosuggest dotnet reference](/dotnet/api/overview/azure/cognitiveservices/bing-autosuggest-readme?view=azure-dotnet)
\ No newline at end of file
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/Bing-Custom-Search/includes/quickstarts/custom-search-client-library-csharp https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/Bing-Custom-Search/includes/quickstarts/custom-search-client-library-csharp.md
@@ -15,7 +15,7 @@ Get started with the Bing Custom Search client library for C#. Follow these step
Use the Bing Custom Search client library for C# to: * Find search results on the web, from your Bing Custom Search instance.
-[Reference documentation](/dotnet/api/overview/azure/cognitiveservices/client/bingcustomsearch?view=azure-dotnet) | [Library source code](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/cognitiveservices/Search.BingCustomSearch) | [Package (NuGet)](https://www.nuget.org/packages/Microsoft.Azure.CognitiveServices.Search.CustomSearch/1.2.0) | [Samples](https://github.com/Azure-Samples/cognitive-services-dotnet-sdk-samples)
+[Reference documentation](/dotnet/api/overview/azure/cognitiveservices/bing-custom-search-readme?view=azure-dotnet) | [Library source code](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/cognitiveservices/Search.BingCustomSearch) | [Package (NuGet)](https://www.nuget.org/packages/Microsoft.Azure.CognitiveServices.Search.CustomSearch/1.2.0) | [Samples](https://github.com/Azure-Samples/cognitive-services-dotnet-sdk-samples)
## Prerequisites
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/Bing-Spell-Check/sdk-quickstart-spell-check https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/Bing-Spell-Check/sdk-quickstart-spell-check.md
@@ -106,4 +106,4 @@ Build and run your project. If you're using Visual Studio, press **F5** to debug
> [Create a single page web-app](tutorials/spellcheck.md) - [What is the Bing Spell Check API?](overview.md)-- [Bing Spell Check C# SDK reference guide](/dotnet/api/overview/azure/cognitiveservices/client/bingspellcheck?view=azure-dotnet)\ No newline at end of file
+- [Bing Spell Check C# SDK reference guide](/dotnet/api/overview/azure/cognitiveservices/bing-spell-check-readme?view=azure-dotnet)
\ No newline at end of file
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/Custom-Vision-Service/includes/quickstarts/rest-tutorial https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/Custom-Vision-Service/includes/quickstarts/rest-tutorial.md
@@ -9,7 +9,7 @@ ms.topic: include
Get started with the Custom Vision REST API. Follow these steps to call the API and build an image classification model. You'll create a project, add tags, train the project, and use the project's prediction endpoint URL to programmatically test it. Use this example as a template for building your own image recognition app. > [!NOTE]
-> If you want to build and train a classification model _without_ writing code, see the [browser-based guidance](../../getting-started-build-a-classifier.md) instead.
+> Custom Vision is most easily used through a client library SDK or through the [browser-based guidance](../../get-started-build-detector.md).
Use the Custom Vision client library for .NET to:
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/Custom-Vision-Service/select-domain https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/Custom-Vision-Service/select-domain.md
@@ -14,25 +14,25 @@ ms.author: shono
# Select a domain for a Custom Vision project
-From the settings blade for your Custom Vision project, you can select a domain for your project. Choose the domain that is closest to your scenario.
+From the settings tab of your Custom Vision project, you can select a domain for your project. Choose the domain that is closest to your scenario. If you're accessing Custom Vision through a client library or REST API, you'll need to specify a domain ID when creating the project. You can get a list of domain IDs with [Get Domains](https://westus2.dev.cognitive.microsoft.com/docs/services/Custom_Vision_Training_3.3/operations/5eb0bcc6548b571998fddeab), or use the table below.
## Image Classification |Domain|Purpose| |---|---|
-|__Generic__| Optimized for a broad range of image classification tasks. If none of the other domains are appropriate, or you're unsure of which domain to choose, select the Generic domain.|
-|__Food__|Optimized for photographs of dishes as you would see them on a restaurant menu. If you want to classify photographs of individual fruits or vegetables, use the Food domain.|
-|__Landmarks__|Optimized for recognizable landmarks, both natural and artificial. This domain works best when the landmark is clearly visible in the photograph. This domain works even if the landmark is slightly obstructed by people in front of it.|
-|__Retail__|Optimized for images that are found in a shopping catalog or shopping website. If you want high precision classifying between dresses, pants, and shirts, use this domain.|
+|__General__| Optimized for a broad range of image classification tasks. If none of the other domains are appropriate, or you're unsure of which domain to choose, select the General domain. ID: `ee85a74c-405e-4adc-bb47-ffa8ca0c9f31`|
+|__Food__|Optimized for photographs of dishes as you would see them on a restaurant menu. If you want to classify photographs of individual fruits or vegetables, use the Food domain. ID: `c151d5b5-dd07-472a-acc8-15d29dea8518`|
+|__Landmarks__|Optimized for recognizable landmarks, both natural and artificial. This domain works best when the landmark is clearly visible in the photograph. This domain works even if the landmark is slightly obstructed by people in front of it. ID: `ca455789-012d-4b50-9fec-5bb63841c793`|
+|__Retail__|Optimized for images that are found in a shopping catalog or shopping website. If you want high precision classifying between dresses, pants, and shirts, use this domain. ID: `b30a91ae-e3c1-4f73-a81e-c270bff27c39`|
|__Compact domains__| Optimized for the constraints of real-time classification on edge devices.| ## Object Detection |Domain|Purpose| |---|---|
-|__General__| Optimized for a broad range of object detection tasks. If none of the other domains are appropriate, or you are unsure of which domain to choose, select the Generic domain.|
-|__Logo__|Optimized for finding brand logos in images.|
-|__Products on shelves__|Optimized for detecting and classifying products on shelves.|
+|__General__| Optimized for a broad range of object detection tasks. If none of the other domains are appropriate, or you are unsure of which domain to choose, select the General domain. ID: `da2e3a8a-40a5-4171-82f4-58522f70fbc1`|
+|__Logo__|Optimized for finding brand logos in images. ID: `1d8ffafe-ec40-4fb2-8f90-72b3b6cecea4`|
+|__Products on shelves__|Optimized for detecting and classifying products on shelves. ID: `3780a898-81c3-4516-81ae-3a139614e1f3`|
|__Compact domains__| Optimized for the constraints of real-time object detection on edge devices.| ## Compact domains
@@ -41,11 +41,11 @@ The models generated by compact domains can be exported to run locally. In the C
Model performance varies by selected domain. In the table below, we report the model size and inference time on Intel Desktop CPU and NVidia GPU \[1\]. These numbers don't include preprocessing and postprocessing time.
-|Task|Domain|Model Size|CPU inference time|GPU inference time|
-|---|---|---|---|---|
-|Classification|General (compact)|5 MB|13 ms|5 ms|
-|Object Detection|General (compact)|45 MB|35 ms|5 ms|
-|Object Detection|General (compact) [S1]|14 MB|27 ms|7 ms|
+|Task|Domain|ID|Model Size|CPU inference time|GPU inference time|
+|---|---|---|---|---|---|
+|Classification|General (compact)|`0732100f-1a38-4e49-a514-c9b44c697ab5`|5 MB|13 ms|5 ms|
+|Object Detection|General (compact)|`a27d5ca5-bb19-49d8-a70a-fec086c47f5b`|45 MB|35 ms|5 ms|
+|Object Detection|General (compact) [S1]|`7ec2ac80-887b-48a6-8df9-8b1357765430`|14 MB|27 ms|7 ms|
>[!NOTE] >__General (compact)__ domain for Object Detection requires special postprocessing logic. For the detail, please see an example script in the exported zip package. If you need a model without the postprocessing logic, use __General (compact) [S1]__.
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/Ink-Recognizer/language-support https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/Ink-Recognizer/language-support.md
@@ -36,8 +36,8 @@ This article explains which languages are supported for the Ink Recognizer API.
| Dutch (Netherlands) | `nl-NL` | | English (Australia) | `en-AU` | | English (Canada) | `en-CA` |
-| English (United Kingdom) | `en-GB` |
| English (India) | `en-IN` |
+| English (United Kingdom) | `en-GB` |
| English (United States) | `en-US` | | Finnish | `fi-FI` | | French (France) | `fr-FR` |
@@ -62,23 +62,23 @@ This article explains which languages are supported for the Ink Recognizer API.
| Polish | `pl-PL` | | Portuguese (Brazil) | `pt-BR` | | Portuguese (Portugal) | `pt-PT` |
-| Romansh | `rm-CH` |
| Romanian | `ro-RO` |
+| Romansh | `rm-CH` |
| Russian | `ru-RU` | | Scottish Gaelic | `gd-GB` |
-| Sesotho sa Leboa | `nso-ZA` |
| Serbian (Cyrillic, Bosnia and Herzegovina) | `sr-Cyrl-BA` | | Serbian (Cyrillic, Montenegro) | `sr-Cyrl-ME` | | Serbian (Cyrillic, Serbia) | `sr-Cyrl-RS` | | Serbian (Latin, Bosnia and Herzegovina) | `sr-Latn-BA` | | Serbian (Latin, Montenegro) | `sr-Latn-ME` | | Serbian (Latin, Serbia) | `sr-Latn-RS` |
+| Sesotho sa Leboa | `nso-ZA` |
| Setswana (South Africa) | `tn-ZA` | | Slovak | `sk-SK` | | Slovenian | `sl-SI` | | Spanish (Argentina) | `es-AR` |
-| Spanish (Spain) | `es-ES` |
| Spanish (Mexico) | `es-MX` |
+| Spanish (Spain) | `es-ES` |
| Swedish (Sweden) | `sv-SE` | | Turkish | `tr-TR` | | Welsh | `cy-GB` |
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/LUIS/luis-language-support https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/LUIS/luis-language-support.md
@@ -27,12 +27,12 @@ LUIS understands utterances in the following languages:
| Language |Locale | Prebuilt domain | Prebuilt entity | Phrase list recommendations | **[Text analytics](../text-analytics/language-support.md)<br>(Sentiment and<br>Keywords)| |--|--|:--:|:--:|:--:|:--:|
-| English (United States) |`en-US` | Γ£ö | Γ£ö |Γ£ö|Γ£ö|
| Arabic (preview - modern standard Arabic) |`ar-AR`|-|-|-|-| | *[Chinese](#chinese-support-notes) |`zh-CN` | Γ£ö | Γ£ö |Γ£ö|-| | Dutch |`nl-NL` |Γ£ö|-|-|Γ£ö|
-| French (France) |`fr-FR` |Γ£ö| Γ£ö |Γ£ö |Γ£ö|
+| English (United States) |`en-US` | Γ£ö | Γ£ö |Γ£ö|Γ£ö|
| French (Canada) |`fr-CA` |-|-|-|Γ£ö|
+| French (France) |`fr-FR` |Γ£ö| Γ£ö |Γ£ö |Γ£ö|
| German |`de-DE` |Γ£ö| Γ£ö |Γ£ö |Γ£ö| | Gujarati | `gu-IN`|-|-|-|-| | Hindi | `hi-IN`|-|Γ£ö|-|-|
@@ -41,8 +41,8 @@ LUIS understands utterances in the following languages:
| Korean |`ko-KR` |Γ£ö|-|-|Key phrase only| | Marathi | `mr-IN`|-|-|-|-| | Portuguese (Brazil) |`pt-BR` |Γ£ö| Γ£ö |Γ£ö |not all sub-cultures|
-| Spanish (Spain) |`es-ES` |Γ£ö| Γ£ö |Γ£ö|Γ£ö|
| Spanish (Mexico)|`es-MX` |-|-|Γ£ö|Γ£ö|
+| Spanish (Spain) |`es-ES` |Γ£ö| Γ£ö |Γ£ö|Γ£ö|
| Tamil | `ta-IN`|-|-|-|-| | Telugu | `te-IN`|-|-|-|-| | Turkish | `tr-TR` |Γ£ö|Γ£ö|-|Sentiment only|
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/Speech-Service/faq-stt https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/Speech-Service/faq-stt.md
@@ -50,9 +50,9 @@ You can deploy baseline and customized models in the portal and then run accurac
The other results are likely worse and might not have full capitalization and punctuation applied. These results are most useful in special scenarios such as giving users the option to pick corrections from a list or handling incorrectly recognized commands.
-**Q: What's the difference between the Search and Dictation model and the Conversational model?**
+**Q: Why are there different base models?**
-**A**: You can choose from more than one baseline model in the Speech service. The Conversational model is useful for recognizing speech that is spoken in a conversational style. This model is ideal for transcribing phone calls. The Search and Dictation model is ideal for voice-triggered apps. The Universal model is a new model that aims to address both scenarios. The Universal model is currently at or above the quality level of the Conversational model in most locales.
+**A**: You can choose from more than one base model in the Speech service. Each model name contains the date when it was added. When you start training a custom model, use the latest model to get the best accuracy. Older base models are still available for some time when a new model is made available. You can continue using the model that you have worked with until it is retired (see [Model lifecycle](custom-speech-overview.md#model-lifecycle)). It is still recommended to switch to the latest base model for better accuracy.
**Q: Can I update my existing model (model stacking)?**
@@ -60,15 +60,17 @@ The other results are likely worse and might not have full capitalization and pu
The old dataset and the new dataset must be combined in a single .zip file (for acoustic data) or in a .txt file (for language data). When adaptation is finished, the new, updated model needs to be redeployed to obtain a new endpoint
-**Q: When a new version of a baseline is available, is my deployment automatically updated?**
+**Q: When a new version of a base model is available, is my deployment automatically updated?**
**A**: Deployments will NOT be automatically updated.
-If you have adapted and deployed a model with baseline V1.0, that deployment will remain as is. Customers can decommission the deployed model, readapt using the newer version of the baseline and redeploy.
+If you have adapted and deployed a model, that deployment will remain as is. You can decommission the deployed model, readapt using the newer version of the base model and redeploy for better accuracy.
+
+Both base models and custom models will be retired after some time (see [Model lifecycle](custom-speech-overview.md#model-lifecycle)).
**Q: Can I download my model and run it locally?**
-**A**: Models can't be downloaded and executed locally.
+**A**: You can run a custom model locally in a [Docker container](speech-container-howto.md?tabs=cstt).
**Q: Are my requests logged?**
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/Speech-Service/how-to-develop-custom-commands-application https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/Speech-Service/how-to-develop-custom-commands-application.md
@@ -480,7 +480,7 @@ To add a confirmation, you use the `SetTemperature` command. To achieve confirma
1. Leave the existing **Confirmation was denied** condition. 1. Add a new condition: **Type** > **Required parameters** > **Temperature**. 1. Add a new action: **Type** > **Send speech response** > **No problem. What temperature then?**.
- 1. Leave the default **Post-execution state** value as **Wait for user's input**.
+ 1. Change the default **Post-execution state** value to **Wait for user's input**.
> [!IMPORTANT] > In this article, you use the built-in confirmation capability. You can also manually add interaction rules one by one.
@@ -494,7 +494,7 @@ Try out the changes by selecting **Train**. When the training finishes, select *
- **Input**: *72 degrees* - **Output**: are you sure you want to set the temperature as 72 degrees? - **Input**: *Yes*-- **Output**: OK, setting temperature to 83 degrees
+- **Output**: OK, setting temperature to 72 degrees
### Implement corrections in a command
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/bing-visual-search/includes/quickstarts/visual-search-client-library-csharp https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/bing-visual-search/includes/quickstarts/visual-search-client-library-csharp.md
@@ -13,7 +13,7 @@ ms.custom: devx-track-csharp
Use this quickstart to begin getting image insights from the Bing Visual Search service, using the C# client library. While Bing Visual Search has a REST API compatible with most programming languages, the client library provides an easy way to integrate the service into your applications. The source code for this sample can be found on [GitHub](https://github.com/Azure-Samples/cognitive-services-dotnet-sdk-samples/tree/master/BingSearchv7/BingVisualSearch).
-[Reference documentation](/dotnet/api/overview/azure/cognitiveservices/client/bingvisualsearch?view=azure-dotnet) | [Library source code](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/cognitiveservices/Search.BingVisualSearch) | [Package (NuGet)](https://www.nuget.org/packages/Microsoft.Azure.CognitiveServices.Search.VisualSearch/) | [Samples](https://github.com/Azure-Samples/cognitive-services-dotnet-sdk-samples/)
+[Reference documentation](/dotnet/api/overview/azure/cognitiveservices/bing-visual-search-readme?view=azure-dotnet) | [Library source code](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/cognitiveservices/Search.BingVisualSearch) | [Package (NuGet)](https://www.nuget.org/packages/Microsoft.Azure.CognitiveServices.Search.VisualSearch/) | [Samples](https://github.com/Azure-Samples/cognitive-services-dotnet-sdk-samples/)
## Prerequisites
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/build-training-data-set https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/build-training-data-set.md
@@ -69,10 +69,8 @@ If you add the following content to the request body, the API will train with do
Now that you've learned how to build a training data set, follow a quickstart to train a custom Form Recognizer model and start using it on your forms.
-* [Train a model and extract form data using the client library](./quickstarts/client-library.md)
-* [Train a model and extract form data using the REST API and Python](./quickstarts/python-train-extract.md)
+* [Train a model and extract form data using the client library or REST API](./quickstarts/client-library.md)
* [Train with labels using the sample labeling tool](./quickstarts/label-tool.md)
-* [Train with labels using the REST API and Python](./quickstarts/python-labeled-data.md)
## See also
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/concept-business-cards https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/concept-business-cards.md
@@ -371,7 +371,7 @@ The "readResults" node contains all of the recognized text. Text is organized by
} ```
-Follow the [Extract business card data](./QuickStarts/python-business-cards.md) quickstart to implement business card data extraction using Python and the REST API.
+Follow the [quickstart](./QuickStarts/client-library.md) quickstart to implement business card data extraction using Python and the REST API.
## Customer Scenarios
@@ -382,11 +382,11 @@ The data extracted with the Business Card API can be used to perform a variety o
* Keep track of sales leads. * Extract contact info in bulk from existing business card images.
-The Business Card API also powers the [AIBuilder Business Card Processing feature](/ai-builder/prebuilt-business-card).
+The Business Card API also powers the [AI Builder Business Card Processing feature](/ai-builder/prebuilt-business-card).
## Next steps -- Follow the [Business Cards API Python quickstart](./quickstarts/python-business-cards.md) to get started recognizing business cards.
+- Follow the [quickstart](./quickstarts/client-library.md) to get started recognizing business cards.
## See also
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/concept-invoices https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/concept-invoices.md
@@ -30,7 +30,7 @@ To try out the Form Recognizer Invoice Service, go to the online Sample UI Tool:
> [!div class="nextstepaction"] > [Try Prebuilt Models](https://fott-preview.azurewebsites.net/)
-You will need an Azure subscription ([create one for free](https://azure.microsoft.com/free/cognitive-services)) and a [Form Recognzier resource](https://ms.portal.azure.com/#create/Microsoft.CognitiveServicesFormRecognizer) endpoint and key to try out the Form Recognizer Invoice service.
+You will need an Azure subscription ([create one for free](https://azure.microsoft.com/free/cognitive-services)) and a [Form Recognizer resource](https://ms.portal.azure.com/#create/Microsoft.CognitiveServicesFormRecognizer) endpoint and key to try out the Form Recognizer Invoice service.
![Analyzed invoice example](./media/analyze-invoice.png)
@@ -76,7 +76,7 @@ The Invoice service will extract the text, tables and 26 invoice fields. Followi
| CustomerName | string | Customer being invoiced | Microsoft Corp | | | CustomerId | string | Reference ID for the customer | CID-12345 | | | PurchaseOrder | string | A purchase order reference number | PO-3333 | | |
-| InvoiceId | string | Id for this specific invoice (often "Invoice Number") | INV-100 | | |
+| InvoiceId | string | ID for this specific invoice (often "Invoice Number") | INV-100 | | |
| InvoiceDate | date | Date the invoice was issued | 11/15/2019 | | DueDate | date | Date payment for this invoice is due | 12/15/2019 | 2019-12-15 | 2019-11-15 | | VendorName | string | Vendor who has created this invoice | CONTOSO LTD. | |
@@ -104,8 +104,8 @@ The Invoice service will extract the text, tables and 26 invoice fields. Followi
## Next steps - Try your own invoices and samples in the [Form Recognizer Sample UI](https://fott-preview.azurewebsites.net/).-- Complete a [Form Recognizer client library quickstart](quickstarts/client-library.md) to get started writing an invoice processing app with Form Recognizer in the language of your choice.-- Or, follow the [Extract invoice data](./quickstarts/python-invoices.md) quickstart to implement invoice data extraction using Python and the REST API.
+- Complete a [Form Recognizer quickstart](quickstarts/client-library.md) to get started writing an invoice processing app with Form Recognizer in the language of your choice.
+ ## See also * [What is Form Recognizer?](./overview.md)
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/concept-layout https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/concept-layout.md
@@ -86,8 +86,7 @@ Layout also extracts selection marks from documents. Selection marks extracted i
## Next steps - Try your own layout extraction using the [Form Recognizer Sample UI](https://fott-preview.azurewebsites.net/)-- Complete a [Form Recognizer client library quickstart](quickstarts/client-library.md) to get started extracting layouts in the language of your choice.-- Or, follow the [Extract Layout data](./QuickStarts/python-layout.md) quickstart to implement layout data extraction using Python and the REST API.
+- Complete a [Form Recognizer quickstart](quickstarts/client-library.md) to get started extracting layouts in the language of your choice.
## See also
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/concept-receipts https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/concept-receipts.md
@@ -453,12 +453,11 @@ The Receipt output is also useful for general book-keeping for business or perso
Receipts contain useful data which you can use to analyze consumer behavior and shopping trends.
-The Receipt API also powers the [AIBuilder Receipt Processing feature](/ai-builder/prebuilt-receipt-processing).
+The Receipt API also powers the [AI Builder Receipt Processing feature](/ai-builder/prebuilt-receipt-processing).
## Next steps -- Complete a [Form Recognizer client library quickstart](quickstarts/client-library.md) to get started writing a receipt processing app with Form Recognizer in the language of your choice.-- Or, follow the [Receipt API Python quickstart](./quickstarts/python-receipts.md) to recognize receipts using the REST API.
+- Complete a [Form Recognizer quickstart](quickstarts/client-library.md) to get started writing a receipt processing app with Form Recognizer in the language of your choice.
## See also
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/includes/quickstarts/rest-api https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/includes/quickstarts/rest-api.md
@@ -11,6 +11,9 @@ ms.date: 12/15/2020
ms.author: pafarley ---
+> [!NOTE]
+> This guide uses cURL to execute REST API calls. There is also [sample code on GitHub](https://github.com/Azure-Samples/cognitive-services-quickstart-code/tree/master/python/FormRecognizer/rest) that illustrates how to call the REST APIs with Python.
+ ## Prerequisites * [cURL](https://curl.haxx.se/windows/) installed.
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/language-support https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/language-support.md
@@ -19,12 +19,12 @@ This table lists the human languages supported by the Form Recognizer service.
|Language| Language code | Form Recognizer v2.0 | Form Recognizer v2.1 preview| |:-----|:----:|:-----:|:---:|
-|English (printed & handwritten) | `en` | Γ£ö | Γ£ö|
|Chinese (Simplified) | `zh-Hans`| | Γ£ö |
-|Japanese | `ja` | | Γ£ö|
|Dutch | `nl` | | Γ£ö |
+|English (printed & handwritten) | `en` | Γ£ö | Γ£ö|
|French | `fr` | | Γ£ö | |German | `de` | | Γ£ö | |Italian | `it` | | Γ£ö |
+|Japanese | `ja` | | Γ£ö|
|Portuguese | `pt` | | Γ£ö | |Spanish | `es` | | Γ£ö |
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/overview https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/overview.md
@@ -105,21 +105,21 @@ The Business Cards model enables you to extract information such as the person's
Use the [Sample Form Recognizer tool](https://fott.azurewebsites.net/) or follow a quickstart to get started extracting data from your forms. We recommend that you use the free service when you're learning the technology. Remember that the number of free pages is limited to 500 per month.
-* [Client library quickstarts](./quickstarts/client-library.md) (all languages, multiple scenarios)
+* [Client library / REST API quickstart](./quickstarts/client-library.md) (all languages, multiple scenarios)
* Web UI quickstarts * [Train with labels - sample labeling tool](quickstarts/label-tool.md)
-* REST quickstarts
+* REST samples (GitHub)
* Extract text, selection marks and table structure from documents
- * [Extract layout data - Python](quickstarts/python-layout.md)
+ * [Extract layout data - Python](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/python/FormRecognizer/rest/python-layout.md)
* Train custom models and extract form data
- * [Train without labels - Python](quickstarts/python-train-extract.md)
- * [Train with labels - Python](quickstarts/python-labeled-data.md)
+ * [Train without labels - Python](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/python/FormRecognizer/rest/python-train-extract.md)
+ * [Train with labels - Python](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/python/FormRecognizer/rest/python-labeled-data.md)
* Extract data from invoices
- * [Extract invoice data - Python](quickstarts/python-invoices.md)
+ * [Extract invoice data - Python](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/python/FormRecognizer/rest/python-invoices.md)
* Extract data from sales receipts
- * [Extract receipt data - Python](quickstarts/python-receipts.md)
+ * [Extract receipt data - Python](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/python/FormRecognizer/rest/python-receipts.md)
* Extract data from business cards
- * [Extract business card data - Python](quickstarts/python-business-cards.md)
+ * [Extract business card data - Python](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/python/FormRecognizer/rest/python-business-cards.md)
### Review the REST APIs
@@ -157,4 +157,4 @@ As with all the cognitive services, developers using the Form Recognizer service
## Next steps
-Complete a [client library quickstart](quickstarts/client-library.md) to get started writing a forms processing app with Form Recognizer in the language of your choice.
\ No newline at end of file
+Complete a [quickstart](quickstarts/client-library.md) to get started writing a forms processing app with Form Recognizer in the language of your choice.
\ No newline at end of file
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/quickstarts/label-tool https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/quickstarts/label-tool.md
@@ -102,7 +102,7 @@ You'll use the Docker engine to run the sample labeling tool. Follow these steps
This command will make the sample labeling tool available through a web browser. Go to `http://localhost:3000`. > [!NOTE]
-> You can also label documents and train models using the Form Recognizer REST API. To train and Analyze with the REST API, see [Train with labels using the REST API and Python](./python-labeled-data.md).
+> You can also label documents and train models using the Form Recognizer REST API. To train and Analyze with the REST API, see [Train with labels using the REST API and Python](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/python/FormRecognizer/rest/python-labeled-data.md).
## Set up input data
@@ -274,7 +274,7 @@ Click the Train icon on the left pane to open the Training page. Then click the
After training finishes, examine the **Average Accuracy** value. If it's low, you should add more input documents and repeat the steps above. The documents you've already labeled will remain in the project index. > [!TIP]
-> You can also run the training process with a REST API call. To learn how to do this, see [Train with labels using Python](./python-labeled-data.md).
+> You can also run the training process with a REST API call. To learn how to do this, see [Train with labels using Python](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/python/FormRecognizer/rest/python-labeled-data.md).
## Compose trained models
@@ -298,7 +298,7 @@ Click on the "Compose" button. In the pop up, name your new composed model and c
Click on the Predict (light bulb) icon on the left to test your model. Upload a form document that you haven't used in the training process. Then click the **Predict** button on the right to get key/value predictions for the form. The tool will apply tags in bounding boxes and will report the confidence of each tag. > [!TIP]
-> You can also run the Analyze API with a REST call. To learn how to do this, see [Train with labels using Python](./python-labeled-data.md).
+> You can also run the Analyze API with a REST call. To learn how to do this, see [Train with labels using Python](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/python/FormRecognizer/rest/python-labeled-data.md).
## Improve results
@@ -325,7 +325,7 @@ Finally, go to the main page (house icon) and click Open Cloud Project. Then sel
In this quickstart, you've learned how to use the Form Recognizer sample labeling tool to train a model with manually labeled data. If you'd like to build your own utility to label training data, use the REST APIs that deal with labeled data training. > [!div class="nextstepaction"]
-> [Train with labels using Python](./python-labeled-data.md)
+> [Train with labels using Python](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/python/FormRecognizer/rest/python-labeled-data.md)
* [What is Form Recognizer?](../overview.md)
-* [Form Recognizer client library quickstarts](client-library.md)
+* [Form Recognizer quickstart](client-library.md)
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/quickstarts/python-business-cards https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/quickstarts/python-business-cards.md deleted file mode 100644
@@ -1,254 +0,0 @@
-title: "Quickstart: Extract business card data using Python - Form Recognizer"
-titleSuffix: Azure Cognitive Services
-description: In this quickstart, you'll use the Form Recognizer REST API with Python to extract data from images of business cards in English.
-author: PatrickFarley
-manager: nitinme
-
-ms.service: cognitive-services
-ms.subservice: forms-recognizer
-ms.topic: quickstart
-ms.date: 11/23/2020
-ms.author: pafarley
-ms.custom: devx-track-python
-#Customer intent: As a developer or data scientist familiar with Python, I want to learn how to use a prebuilt Form Recognizer model to extract my business card data.
-
-# Quickstart: Extract business card data using the Form Recognizer REST API with Python
-
-In this quickstart, you'll use the Azure Form Recognizer REST API with Python to extract and identify relevant information on business cards.
-
-If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
-
-## Prerequisites
-
-To complete this quickstart, you must have:
-- [Python](https://www.python.org/downloads/) installed (if you want to run the sample locally).-- An image of a business card. You can use a [sample image](../media/business-card-english.jpg) for this quickstart.-
-> [!NOTE]
-> This quickstart uses a local file. To use a remote business card image accessed by URL instead, see the [reference documentation](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1-preview-2/operations/AnalyzeBusinessCardAsync).
-
-## Create a Form Recognizer resource
-
-[!INCLUDE [create resource](../includes/create-resource.md)]
-
-## Analyze a business card
-
-To start analyzing a business card, you call the **[Analyze Business Card](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1-preview-2/operations/AnalyzeBusinessCardAsync)** API using the Python script below. Before you run the script, make these changes:
-
-1. Replace `<endpoint>` with the endpoint that you obtained with your Form Recognizer subscription.
-1. Replace `<path to your business card>` with the local path to your business card image or PDF.
-1. Replace `<subscription key>` with the subscription key you copied from the previous step.
-1. Replace `<file type>` with 'image/jpeg', 'image/png', 'application/pdf', or 'image/tiff'.
-
- ```python
- ########### Python Form Recognizer Async Business Cards #############
-
- import json
- import time
- from requests import get, post
-
- # Endpoint URL
- endpoint = r"<endpoint>"
- apim_key = "<subscription key>"
- post_url = endpoint + "/formrecognizer/v2.1-preview.2/prebuilt/businessCard/analyze"
- source = r"<path to your business card>"
- content_type = "<file type>"
-
- headers = {
- # Request headers
- 'Content-Type': content_type,
- 'Ocp-Apim-Subscription-Key': apim_key,
- }
-
- params = {
- "includeTextDetails": True # True to output all recognized text
- }
-
- with open(source, "rb") as f:
- data_bytes = f.read()
-
- try:
- resp = post(url = post_url, data = data_bytes, headers = headers, params = params)
- if resp.status_code != 202:
- print("POST analyze failed:\n%s" % resp.text)
- quit()
- print("POST analyze succeeded:\n%s" % resp.headers)
- get_url = resp.headers["operation-location"]
- except Exception as e:
- print("POST analyze failed:\n%s" % str(e))
- quit()
- ```
-
-1. Save the code in a file with a .py extension. For example, *form-recognizer-businesscards.py*.
-1. Open a command prompt window.
-1. At the prompt, use the `python` command to run the sample. For example, `python form-recognizer-businesscards.py`.
-
-You'll receive a `202 (Success)` response that includes an **Operation-Location** header, which the script will print to the console. This header contains a result ID that you can use to query the status of the long running operation and get the results. In the following example value, the string after `operations/` is the result ID.
-
-```console
-https://cognitiveservice/formrecognizer/v2.1-preview.2/prebuilt/businessCard/analyzeResults/54f0b076-4e38-43e5-81bd-b85b8835fdfb
-```
-
-## Get the business card results
-
-After you've called the **Analyze Business Card** API, you call the **[Get Analyze Business Card Result](https://westcentralus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1-preview-2/operations/GetAnalyzeBusinessCardResult)** API to get the status of the operation and the extracted data. Add the following code to the bottom of your Python script. This uses the result ID value in a new API call. This script calls the API at regular intervals until the results are available. We recommend an interval of one second or more.
-
-```python
-n_tries = 10
-n_try = 0
-wait_sec = 6
-while n_try < n_tries:
- try:
- resp = get(url = get_url, headers = {"Ocp-Apim-Subscription-Key": apim_key})
- resp_json = json.loads(resp.text)
- if resp.status_code != 200:
- print("GET Business card results failed:\n%s" % resp_json)
- quit()
- status = resp_json["status"]
- if status == "succeeded":
- print("Business card Analysis succeeded:\n%s" % resp_json)
- quit()
- if status == "failed":
- print("Analysis failed:\n%s" % resp_json)
- quit()
- # Analysis still running. Wait and retry.
- time.sleep(wait_sec)
- n_try += 1
- except Exception as e:
- msg = "GET analyze results failed:\n%s" % str(e)
- print(msg)
- quit()
-```
-
-1. Save the script.
-1. Again use the `python` command to run the sample. For example, `python form-recognizer-businesscards.py`.
-
-### Examine the response
-![A business card from Contoso company](../media/business-card-english.jpg)
-
-This sample illustrates the JSON output returned by Form Recognizer. It has been truncated for readability.
-
-```json
-{
- "status": "succeeded",
- "createdDateTime": "2020-06-04T08:19:29Z",
- "lastUpdatedDateTime": "2020-06-04T08:19:35Z",
- "analyzeResult": {
- "version": "2.1.1",
- "readResults": [
- {
- "page": 1,
- "angle": -17.0956,
- "width": 4032,
- "height": 3024,
- "unit": "pixel"
- }
- ],
- "documentResults": [
- {
- "docType": "prebuilt:businesscard",
- "pageRange": [
- 1,
- 1
- ],
- "fields": {
- "ContactNames": {
- "type": "array",
- "valueArray": [
- {
- "type": "object",
- "valueObject": {
- "FirstName": {
- "type": "string",
- "valueString": "Avery",
- "text": "Avery",
- "boundingBox": [
- 703,
- 1096,
- 1134,
- 989,
- 1165,
- 1109,
- 733,
- 1206
- ],
- "page": 1
- },
- "text": "Dr. Avery Smith",
- "boundingBox": [
- 419.3,
- 1154.6,
- 1589.6,
- 877.9,
- 1618.9,
- 1001.7,
- 448.6,
- 1278.4
- ],
- "confidence": 0.993
- }
- ]
- },
- "Emails": {
- "type": "array",
- "valueArray": [
- {
- "type": "string",
- "valueString": "avery.smith@contoso.com",
- "text": "avery.smith@contoso.com",
- "boundingBox": [
- 2107,
- 934,
- 2917,
- 696,
- 2935,
- 764,
- 2126,
- 995
- ],
- "page": 1,
- "confidence": 0.99
- }
- ]
- },
- "Websites": {
- "type": "array",
- "valueArray": [
- {
- "type": "string",
- "valueString": "https://www.contoso.com/",
- "text": "https://www.contoso.com/",
- "boundingBox": [
- 2121,
- 1002,
- 2992,
- 755,
- 3014,
- 826,
- 2143,
- 1077
- ],
- "page": 1,
- "confidence": 0.995
- }
- ]
- }
- }
- }
- ]
- }
-}
-```
-
-The script will print responses to the console until the **Analyze Business Card** operation completes.
-The `"readResults"` node contains all of the recognized text. Text is organized by page, then by line, then by individual words. The `"documentResults"` node contains the business-card-specific values that the model discovered. This is where you'll find useful contact information like the company name, first name, last name, phone number, and so on.
--
-## Next steps
-
-In this quickstart, you used the Form Recognizer REST API with Python to extract the content of a business card. Next, see the reference documentation to explore the Form Recognizer API in more depth.
-
-> [!div class="nextstepaction"]
-> [REST API reference documentation](https://westcentralus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1-preview-2/operations/AnalyzeBusinessCardAsync)
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/quickstarts/python-invoices https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/quickstarts/python-invoices.md deleted file mode 100644
@@ -1,294 +0,0 @@
-title: "Quickstart: Extract invoice data using Python - Form Recognizer"
-titleSuffix: Azure Cognitive Services
-description: In this quickstart, you'll use the Form Recognizer REST API with Python to extract data from invoices
-author: PatrickFarley
-manager: nitinme
-
-ms.service: cognitive-services
-ms.subservice: forms-recognizer
-ms.topic: quickstart
-ms.date: 11/12/2020
-ms.author: pafarley
-ms.custom: devx-track-python
-#Customer intent: As a developer or data scientist familiar with Python, I want to learn how to use a prebuilt Form Recognizer model to extract my invoice data.
-
-# Quickstart: Extract invoice data using the Form Recognizer REST API with Python
-
-In this quickstart, you'll use the Azure Form Recognizer REST API with Python to extract and identify relevant information in invoices.
-
-If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/cognitive-services/) before you begin.
-
-## Prerequisites
-
-To complete this quickstart, you must have:
-- [Python](https://www.python.org/downloads/) installed (if you want to run the sample locally).-- An invoice document. You can use the [sample invoice](../media/sample-invoice.jpg) for this quickstart.-
-> [!NOTE]
-> This quickstart uses a local file. To use a invoice document accessed by URL instead, see the [reference documentation](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync).
-
-## Create a Form Recognizer resource
-
-[!INCLUDE [create resource](../includes/create-resource.md)]
-
-## Analyze an invoice
-
-To start analyzing an invoice, call the **[Analyze Invoice](https://westcentralus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1-preview-2/operations/5ed8c9843c2794cbb1a96291)** API using the Python script below.
-Before you run the script, make these changes:
-
-1. Replace `<Endpoint>` with the endpoint that you obtained with your Form Recognizer subscription.
-1. Replace `<subscription key>` with the subscription key you copied from the previous step.
-1. Replace `<path to your invoice>` with the local path where you have a sample invoice saved.
-
- ```python
- ########### Python Form Recognizer Async Invoice #############
-
- import json
- import time
- from requests import get, post
-
- # Endpoint URL
- endpoint = r"<Endpoint>"
- apim_key = "<subscription key>"
- post_url = endpoint + "/formrecognizer/v2.1-preview.2/prebuilt/invoice/analyze"
- source = r"<path to your invoice>"
-
- headers = {
- # Request headers
- 'Content-Type': '<file type>',
- 'Ocp-Apim-Subscription-Key': apim_key,
- }
-
- params = {
- "includeTextDetails": True
- "locale": "en-US"
- }
-
- with open(source, "rb") as f:
- data_bytes = f.read()
-
- try:
- resp = post(url = post_url, data = data_bytes, headers = headers, params = params)
- if resp.status_code != 202:
- print("POST analyze failed:\n%s" % resp.text)
- quit()
- print("POST analyze succeeded:\n%s" % resp.headers)
- get_url = resp.headers["operation-location"]
- except Exception as e:
- print("POST analyze failed:\n%s" % str(e))
- quit()
- ```
-
-1. Save the code in a file with a .py extension. For example, *form-recognizer-invoice.py*.
-1. Open a command prompt window.
-1. At the prompt, use the `python` command to run the sample. For example, `python form-recognizer-invoice.py`.
-
-You'll receive a `202 (Success)` response that includes an **Operation-Location** header, which the script will print to the console. This header contains an operation ID that you can use to query the status of the asynchronous operation and get the results. In the following example value, the string after `operations/` is the operation ID.
-
-## Get the invoice results
-
-After you've called the **Analyze Invoice** API, you call the **[Get Analyze Invoice Result](https://westcentralus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1-preview-2/operations/5ed8c9acb78c40a2533aee83)** API to get the status of the operation and the extracted data. Add the following code to the bottom of your Python script. This uses the operation ID value in a new API call. This script calls the API at regular intervals until the results are available. We recommend an interval of one second or more.
-
-```python
-n_tries = 10
-n_try = 0
-wait_sec = 6
-while n_try < n_tries:
- try:
- resp = get(url = get_url, headers = {"Ocp-Apim-Subscription-Key": apim_key})
- resp_json = json.loads(resp.text)
- if resp.status_code != 200:
- print("GET Invoice results failed:\n%s" % resp_json)
- quit()
- status = resp_json["status"]
- if status == "succeeded":
- print("Invoice Analysis succeeded:\n%s" % resp_json)
- quit()
- if status == "failed":
- print("Invoice Analysis failed:\n%s" % resp_json)
- quit()
- # Analysis still running. Wait and retry.
- time.sleep(wait_sec)
- n_try += 1
- except Exception as e:
- msg = "GET analyze results failed:\n%s" % str(e)
- print(msg)
- quit()
-```
-
-1. Save the script.
-1. Again use the `python` command to run the sample. For example, `python form-recognizer-invoice.py`.
-
-### Examine the response
-
-The script will print responses to the console until the **Analyze Invoice** operation completes. Then, it will print the extracted text data in JSON format. The `"readResults"` field contains every line of text that was extracted from the invoice, the `"pageResults"` includes the tables and selections marks extracted from the invoice and the `"documentResults"` field contains key/value information for the most relevant parts of the invoice.
-
-See the following invoice document and its corresponding JSON output:
-
-* [Sample invoice](https://github.com/Azure-Samples/cognitive-services-REST-api-samples/tree/master/curl/form-recognizer/sample-invoice.pdf)
-* [Sample invoice JSON output](https://github.com/Azure-Samples/cognitive-services-REST-api-samples/tree/master/curl/form-recognizer/sample-invoice-output.json)
--
-### Sample Python script to extract invoice or a batch of invoices into a CSV file
-
-This sample python script shows you how to get started using the Invoice API. It can run with single invoice as a parameter or folder and will output the JSON file ".invoice.json" and a CSV file _invoiceResutls.csv_ with the extracted values results. When running on a folder, it will scan through all "pdf","jpg","jpeg","png","bmp","tif","tiff" files and run them with the API.
-
-```python
-########### Python Form Recognizer Async Invoice #############
-
-import json
-import time
-import os
-import ntpath
-import sys
-from requests import get, post
-import csv
--
-def analyzeInvoice(filename):
- invoiceResultsFilename = filename + ".invoice.json"
-
- # do not run analyze if .invoice.json file is present on disk
- if os.path.isfile(invoiceResultsFilename):
- with open(invoiceResultsFilename) as json_file:
- return json.load(json_file)
-
- # Endpoint URL
- #endpoint = r"<Endpoint>"
- #apim_key = "<subscription key>"
- post_url = endpoint + "/formrecognizer/v2.1-preview.2/prebuilt/invoice/analyze"
- headers = {
- # Request headers
- 'Content-Type': 'application/octet-stream',
- 'Ocp-Apim-Subscription-Key': apim_key,
- }
-
- params = {
- "includeTextDetails": True
- }
-
- with open(filename, "rb") as f:
- data_bytes = f.read()
-
- try:
- resp = post(url = post_url, data = data_bytes, headers = headers, params = params)
- if resp.status_code != 202:
- print("POST analyze failed:\n%s" % resp.text)
- return None
- print("POST analyze succeeded: %s" % resp.headers["operation-location"])
- get_url = resp.headers["operation-location"]
- except Exception as e:
- print("POST analyze failed:\n%s" % str(e))
- return None
-
- n_tries = 50
- n_try = 0
- wait_sec = 6
-
- while n_try < n_tries:
- try:
- resp = get(url = get_url, headers = {"Ocp-Apim-Subscription-Key": apim_key})
- resp_json = json.loads(resp.text)
- if resp.status_code != 200:
- print("GET Invoice results failed:\n%s" % resp_json)
- return None
- status = resp_json["status"]
- if status == "succeeded":
- print("Invoice analysis succeeded.")
- with open(invoiceResultsFilename, 'w') as outfile:
- json.dump(resp_json, outfile, indent=4)
- return resp_json
- if status == "failed":
- print("Analysis failed:\n%s" % resp_json)
- return None
- # Analysis still running. Wait and retry.
- time.sleep(wait_sec)
- n_try += 1
- except Exception as e:
- msg = "GET analyze results failed:\n%s" % str(e)
- print(msg)
- return None
-
- return resp_json
-
-def parseInvoiceResults(resp_json):
- docResults = resp_json["analyzeResult"]["documentResults"]
- invoiceResult = {}
- for docResult in docResults:
- for fieldName, fieldValue in sorted(docResult["fields"].items()):
- valueFields = list(filter(lambda item: ("value" in item[0]) and ("valueString" not in item[0]), fieldValue.items()))
- invoiceResult[fieldName] = fieldValue["text"]
- if len(valueFields) == 1:
- print("{0:26} : {1:50} NORMALIZED VALUE: {2}".format(fieldName , fieldValue["text"], valueFields[0][1]))
- invoiceResult[fieldName + "_normalized"] = valueFields[0][1]
- else:
- print("{0:26} : {1}".format(fieldName , fieldValue["text"]))
- print("")
- return invoiceResult
-
-def main(argv):
- if (len(argv) != 2):
- print("ERROR: Please provide invoice filename or root directory with invoice PDFs/images as an argument to the python script")
- return
-
- # list of invoice to analyze
- invoiceFiles = []
- csvPostfix = '-invoiceResults.csv'
- if os.path.isfile(argv[1]):
- # Single invoice
- invoiceFiles.append(argv[1])
- csvFileName = argv[1] + csvPostfix
- else:
- # Folder of invoices
- supportedExt = ['.pdf', '.jpg','.jpeg','.tif','.tiff','.png','.bmp']
- invoiceDirectory = argv[1]
- csvFileName = os.path.join(invoiceDirectory, os.path.basename(os.path.abspath(invoiceDirectory)) + csvPostfix)
- for root, directories, filenames in os.walk(invoiceDirectory):
- for invoiceFilename in filenames:
- ext = os.path.splitext(invoiceFilename)[-1].lower()
- if ext in supportedExt:
- fullname = os.path.join(root, invoiceFilename)
- invoiceFiles.append(fullname)
-
- with open(csvFileName, mode='w', newline='\n', encoding='utf-8') as csv_file:
- fieldnames = ['Filename',
- 'FullFilename','InvoiceTotal','InvoiceTotal_normalized','AmountDue','AmountDue_normalized','SubTotal','SubTotal_normalized','TotalTax','TotalTax_normalized','CustomerName','VendorName',
- 'InvoiceId','CustomerId','PurchaseOrder','InvoiceDate','InvoiceDate_normalized','DueDate','DueDate_normalized',
- 'VendorAddress','VendorAddressRecipient','BillingAddress','BillingAddressRecipient','ShippingAddress','ShippingAddressRecipient','CustomerAddress','CustomerAddressRecipient','ServiceAddress','ServiceAddressRecipient','RemittanceAddress','RemittanceAddressRecipient', 'ServiceStartDate','ServiceStartDate_normalized','ServiceEndDate','ServiceEndDate_normalized','PreviousUnpaidBalance','PreviousUnpaidBalance_normalized']
- writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
- writer.writeheader()
- counter = 0
- for invoiceFullFilename in invoiceFiles:
- counter = counter + 1
- invoiceFilename = ntpath.basename(invoiceFullFilename)
- print("----- Processing {0}/{1} : {2} -----".format(counter, len(invoiceFiles),invoiceFullFilename))
-
- resp_json = analyzeInvoice(invoiceFullFilename)
-
- if (resp_json is not None):
- invoiceResults = parseInvoiceResults(resp_json)
- invoiceResults["FullFilename"] = invoiceFullFilename
- invoiceResults["Filename"] = invoiceFilename
- writer.writerow(invoiceResults)
-
-if __name__ == '__main__':
- main(sys.argv)
-```
-
-1. Save the code in a file with a .py extension. For example, *form-recognizer-invoice-to-csv.py*.
-1. Replace `<Endpoint>` with the endpoint that you obtained with your Form Recognizer subscription.
-1. Replace `<subscription key>` with the subscription key you copied from the previous step.
-1. Open a command prompt window.
-1. At the prompt, use the `python` command to run the sample. For example, `python form-recognizer-invoice.py {file name or folder name}`
-The Python script can run with a single invoice or a folder as the parameter and will output the JSON file ".invoice.json" and the values extracted from the invoices into a CSV file "-invoiceResults.csv" with the results. When running on a folder, it will scan through all "pdf","jpg","jpeg","png","bmp","tif","tiff" files and run them with the API.
-
-## Next steps
-
-In this quickstart, you used the Form Recognizer REST API with Python to extract the content from invoices. Next, see the reference documentation to explore the Form Recognizer API in more depth.
-
-> [!div class="nextstepaction"]
-> [REST API reference documentation](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync)
-
-
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/quickstarts/python-labeled-data https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/quickstarts/python-labeled-data.md deleted file mode 100644
@@ -1,758 +0,0 @@
-title: "Quickstart: Train with labels using the REST API and Python - Form Recognizer"
-titleSuffix: Azure Cognitive Services
-description: Learn how to use the Form Recognizer labeled data feature with the REST API and Python to train a custom model.
-author: PatrickFarley
-manager: nitinme
-
-ms.service: cognitive-services
-ms.subservice: forms-recognizer
-ms.topic: quickstart
-ms.date: 10/05/2020
-ms.author: pafarley
-ms.custom: devx-track-python
-
-# Train a Form Recognizer model with labels using REST API and Python
-
-In this quickstart, you'll use the Form Recognizer REST API with Python to train a custom model with manually labeled data. See the [Train with labels](../overview.md#train-with-labels) section of the overview to learn more about this feature.
-
-If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/cognitive-services/) before you begin.
-
-## Prerequisites
-
-To complete this quickstart, you must have:
-- [Python](https://www.python.org/downloads/) installed (if you want to run the sample locally).-- A set of at least six forms of the same type. You'll use this data to train the model and test a form. You can use a [sample data set](https://go.microsoft.com/fwlink/?linkid=2090451) for this quickstart. Download and extract *sample_data.zip*. Upload the training files to the root of a blob storage container in a standard-performance-tier Azure Storage account.-
-> [!NOTE]
-> This quickstart uses remote documents accessed by URL. To use local files instead, see the [reference documentation for v2.0](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/TrainCustomModelAsync) and [reference documentation for v2.1](https://westcentralus.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2-1-preview-2/operations/TrainCustomModelAsync).
-
-## Create a Form Recognizer resource
-
-[!INCLUDE [create resource](../includes/create-resource.md)]
-
-## Set up training data
-
-Next you'll need to set up the required input data. The labeled data feature has special input requirements beyond what's needed to train a custom model without labels.
-
-Make sure all the training documents are of the same format. If you have forms in multiple formats, organize them into subfolders based on common format. When you train, you'll need to direct the API to a subfolder.
-
-In order to train a model using labeled data, you'll need the following files as inputs in the sub-folder. You will learn how to create these files below.
-
-* **Source forms** ΓÇô the forms to extract data from. Supported types are JPEG, PNG, PDF, or TIFF.
-* **OCR layout files** - these are JSON files that describe the sizes and positions of all readable text in each source form. You'll use the Form Recognizer Layout API to generate this data.
-* **Label files** - these are JSON files that describe the data labels that a user has entered manually.
-
-All of these files should occupy the same sub-folder and be in the following format:
-
-* input_file1.pdf
-* input_file1.pdf.ocr.json
-* input_file1.pdf.labels.json
-* input_file2.pdf
-* input_file2.pdf.ocr.json
-* input_file2.pdf.labels.json
-* ...
-
-> [!TIP]
-> When you label forms using the Form Recognizer [sample labeling tool](./label-tool.md), the tool creates these label and OCR layout files automatically.
-
-### Create the OCR output files
-
-You need OCR result files in order for the service to consider the corresponding input files for labeled training. To obtain OCR results for a given source form, follow the steps below:
-
-1. Call the **[Analyze Layout](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync)** API on the read Layout container with the input file as part of the request body. Save the ID found in the response's **Operation-Location** header.
-1. Call the **[Get Analyze Layout Result](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/GetAnalyzeLayoutResult)** API, using the operation ID from the previous step.
-1. Get the response and write the content to a file. For each source form, the corresponding OCR file should have the original file name appended with `.ocr.json`. The OCR JSON output should have the following format. See the [sample OCR file](https://github.com/Azure-Samples/cognitive-services-REST-api-samples/blob/master/curl/form-recognizer/Invoice_1.pdf.ocr.json) for a full example.
-
- # [v2.0](#tab/v2-0)
- ```json
- {
- "status": "succeeded",
- "createdDateTime": "2019-11-12T21:18:12Z",
- "lastUpdatedDateTime": "2019-11-12T21:18:17Z",
- "analyzeResult": {
- "version": "2.0.0",
- "readResults": [
- {
- "page": 1,
- "language": "en",
- "angle": 0,
- "width": 8.5,
- "height": 11,
- "unit": "inch",
- "lines": [
- {
- "language": "en",
- "boundingBox": [
- 0.5384,
- 1.1583,
- 1.4466,
- 1.1583,
- 1.4466,
- 1.3534,
- 0.5384,
- 1.3534
- ],
- "text": "Contoso",
- "words": [
- {
- "boundingBox": [
- 0.5384,
- 1.1583,
- 1.4466,
- 1.1583,
- 1.4466,
- 1.3534,
- 0.5384,
- 1.3534
- ],
- "text": "Contoso",
- "confidence": 1
- }
- ]
- },
- ...
- ```
- # [v2.1 preview](#tab/v2-1)
- ```json
- {
- "status": "succeeded",
- "createdDateTime": "2019-11-12T21:18:12Z",
- "lastUpdatedDateTime": "2019-11-12T21:18:17Z",
- "analyzeResult": {
- "version": "2.1.0",
- "readResults": [
- {
- "page": 1,
- "language": "en",
- "angle": 0,
- "width": 8.5,
- "height": 11,
- "unit": "inch",
- "lines": [
- {
- "language": "en",
- "boundingBox": [
- 0.5384,
- 1.1583,
- 1.4466,
- 1.1583,
- 1.4466,
- 1.3534,
- 0.5384,
- 1.3534
- ],
- "text": "Contoso",
- "words": [
- {
- "boundingBox": [
- 0.5384,
- 1.1583,
- 1.4466,
- 1.1583,
- 1.4466,
- 1.3534,
- 0.5384,
- 1.3534
- ],
- "text": "Contoso",
- "confidence": 1
- }
- ]
- },
- ...
- ```
--
- ---
---
-### Create the label files
-
-Label files contain key-value associations that a user has entered manually. They are needed for labeled data training, but not every source file needs to have a corresponding label file. Source files without labels will be treated as ordinary training documents. We recommend five or more labeled files for reliable training. You can use a UI tool like the [sample labeling tool](./label-tool.md) to generate these files.
-
-When you create a label file, you can optionally specify regions&mdash;exact positions of values on the document. This will give the training even higher accuracy. Regions are formatted as a set of eight values corresponding to four X,Y coordinates: top-left, top-right, bottom-right, and bottom-left. Coordinate values are between zero and one, scaled to the dimensions of the page.
-
-For each source form, the corresponding label file should have the original file name appended with `.labels.json`. The label file should have the following format. See the [sample label file](https://github.com/Azure-Samples/cognitive-services-REST-api-samples/blob/master/curl/form-recognizer/Invoice_1.pdf.labels.json) for a full example.
-
-```json
-{
- "document": "Invoice_1.pdf",
- "labels": [
- {
- "label": "Provider",
- "key": null,
- "value": [
- {
- "page": 1,
- "text": "Contoso",
- "boundingBoxes": [
- [
- 0.06334117647058823,
- 0.1053,
- 0.17018823529411767,
- 0.1053,
- 0.17018823529411767,
- 0.12303636363636362,
- 0.06334117647058823,
- 0.12303636363636362
- ]
- ]
- }
- ]
- },
- {
- "label": "For",
- "key": null,
- "value": [
- {
- "page": 1,
- "text": "Microsoft",
- "boundingBoxes": [
- [
- 0.6122941176470589,
- 0.1374,
- 0.6841764705882353,
- 0.1374,
- 0.6841764705882353,
- 0.14682727272727272,
- 0.6122941176470589,
- 0.14682727272727272
- ]
- ]
- },
- {
- "page": 1,
- "text": "1020",
- "boundingBoxes": [
- [
- 0.6121882352941176,
- 0.156,
- 0.6462941176470588,
- 0.156,
- 0.6462941176470588,
- 0.1653181818181818,
- 0.6121882352941176,
- 0.1653181818181818
- ]
- ]
- },
- ...
-```
-
-> [!IMPORTANT]
-> You can only apply one label to each text element, and each label can only be applied once per page. You cannot apply a label across multiple pages.
--
-## Train a model using labeled data
-
-To train a model with labeled data, call the **[Train Custom Model](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/TrainCustomModelAsync)** API by running the following python code. Before you run the code, make these changes:
-
-1. Replace `<Endpoint>` with the endpoint URL for your Form Recognizer resource.
-1. Replace `<SAS URL>` with the Azure Blob storage container's shared access signature (SAS) URL. [!INCLUDE [get SAS URL](../includes/sas-instructions.md)]
-
- :::image type="content" source="../media/quickstarts/get-sas-url.png" alt-text="SAS URL retrieval":::
-1. Replace `<Blob folder name>` with the folder name in your blob container where the input data is located. Or, if your data is at the root, leave this blank and remove the `"prefix"` field from the body of the HTTP request.
-
-# [v2.0](#tab/v2-0)
-```python
-########### Python Form Recognizer Labeled Async Train #############
-import json
-import time
-from requests import get, post
-
-# Endpoint URL
-endpoint = r"<Endpoint>"
-post_url = endpoint + r"/formrecognizer/v2.0/custom/models"
-source = r"<SAS URL>"
-prefix = "<Blob folder name>"
-includeSubFolders = False
-useLabelFile = True
-
-headers = {
- # Request headers
- 'Content-Type': 'application/json',
- 'Ocp-Apim-Subscription-Key': '<subsription key>',
-}
-
-body = {
- "source": source,
- "sourceFilter": {
- "prefix": prefix,
- "includeSubFolders": includeSubFolders
- },
- "useLabelFile": useLabelFile
-}
-
-try:
- resp = post(url = post_url, json = body, headers = headers)
- if resp.status_code != 201:
- print("POST model failed (%s):\n%s" % (resp.status_code, json.dumps(resp.json())))
- quit()
- print("POST model succeeded:\n%s" % resp.headers)
- get_url = resp.headers["location"]
-except Exception as e:
- print("POST model failed:\n%s" % str(e))
- quit()
-```
-# [v2.1 preview](#tab/v2-1)
-```python
-########### Python Form Recognizer Labeled Async Train #############
-import json
-import time
-from requests import get, post
-
-# Endpoint URL
-endpoint = r"<Endpoint>"
-post_url = endpoint + r"/formrecognizer/v2.1-preview.2/custom/models"
-source = r"<SAS URL>"
-prefix = "<Blob folder name>"
-includeSubFolders = False
-useLabelFile = True
-
-headers = {
- # Request headers
- 'Content-Type': 'application/json',
- 'Ocp-Apim-Subscription-Key': '<subsription key>',
-}
-
-body = {
- "source": source,
- "sourceFilter": {
- "prefix": prefix,
- "includeSubFolders": includeSubFolders
- },
- "useLabelFile": useLabelFile
-}
-
-try:
- resp = post(url = post_url, json = body, headers = headers)
- if resp.status_code != 201:
- print("POST model failed (%s):\n%s" % (resp.status_code, json.dumps(resp.json())))
- quit()
- print("POST model succeeded:\n%s" % resp.headers)
- get_url = resp.headers["location"]
-except Exception as e:
- print("POST model failed:\n%s" % str(e))
- quit()
-```
----
-## Get training results
-
-After you've started the train operation, you use the returned ID to get the status of the operation. Add the following code to the bottom of your Python script. This uses the ID value from the training call in a new API call. The training operation is asynchronous, so this script calls the API at regular intervals until the training status is completed. We recommend an interval of one second or more.
-
-```python
-n_tries = 15
-n_try = 0
-wait_sec = 5
-max_wait_sec = 60
-while n_try < n_tries:
- try:
- resp = get(url = get_url, headers = headers)
- resp_json = resp.json()
- if resp.status_code != 200:
- print("GET model failed (%s):\n%s" % (resp.status_code, json.dumps(resp_json)))
- quit()
- model_status = resp_json["modelInfo"]["status"]
- if model_status == "ready":
- print("Training succeeded:\n%s" % json.dumps(resp_json))
- quit()
- if model_status == "invalid":
- print("Training failed. Model is invalid:\n%s" % json.dumps(resp_json))
- quit()
- # Training still running. Wait and retry.
- time.sleep(wait_sec)
- n_try += 1
- wait_sec = min(2*wait_sec, max_wait_sec)
- except Exception as e:
- msg = "GET model failed:\n%s" % str(e)
- print(msg)
- quit()
-print("Train operation did not complete within the allocated time.")
-```
-
-When the training process is completed, you'll receive a `201 (Success)` response with JSON content like the following. The response has been shortened for simplicity.
-
-```json
-{
- "modelInfo":{
- "status":"ready",
- "createdDateTime":"2019-10-08T10:20:31.957784",
- "lastUpdatedDateTime":"2019-10-08T14:20:41+00:00",
- "modelId":"1cfb372bab404ba3aa59481ab2c63da5"
- },
- "trainResult":{
- "trainingDocuments":[
- {
- "documentName":"invoices\\Invoice_1.pdf",
- "pages":1,
- "errors":[
-
- ],
- "status":"succeeded"
- },
- {
- "documentName":"invoices\\Invoice_2.pdf",
- "pages":1,
- "errors":[
-
- ],
- "status":"succeeded"
- },
- {
- "documentName":"invoices\\Invoice_3.pdf",
- "pages":1,
- "errors":[
-
- ],
- "status":"succeeded"
- },
- {
- "documentName":"invoices\\Invoice_4.pdf",
- "pages":1,
- "errors":[
-
- ],
- "status":"succeeded"
- },
- {
- "documentName":"invoices\\Invoice_5.pdf",
- "pages":1,
- "errors":[
-
- ],
- "status":"succeeded"
- }
- ],
- "errors":[
-
- ]
- },
- "keys":{
- "0":[
- "Address:",
- "Invoice For:",
- "Microsoft",
- "Page"
- ]
- }
-}
-```
-
-Copy the `"modelId"` value for use in the following steps.
-
-[!INCLUDE [analyze forms](../includes/python-custom-analyze.md)]
-
-When the process is completed, you'll receive a `202 (Success)` response with JSON content in the following format. The response has been shortened for simplicity. The main key/value associations are in the `"documentResults"` node. The `"selectionMarks"` node (in v2.1 preview) shows every selection mark (checkbox, radio mark) and whether its status is "selected" or "unselected". The Layout API results (the content and positions of all the text in the document) are in the `"readResults"` node.
-
-# [v2.0](#tab/v2-0)
-```json
-{
- "status": "succeeded",
- "createdDateTime": "2020-08-21T02:16:28Z",
- "lastUpdatedDateTime": "2020-08-21T02:16:35Z",
- "analyzeResult": {
- "version": "2.0.0",
- "readResults": [
- {
- "page": 1,
- "language": "en",
- "angle": 0,
- "width": 8.5,
- "height": 11,
- "unit": "inch",
- "lines": [
- {
- "boundingBox": [
- 0.5826,
- 0.4411,
- 2.3387,
- 0.4411,
- 2.3387,
- 0.7969,
- 0.5826,
- 0.7969
- ],
- "text": "Contoso, Ltd.",
- "words": [
- {
- "boundingBox": [
- 0.5826,
- 0.4411,
- 1.744,
- 0.4411,
- 1.744,
- 0.7969,
- 0.5826,
- 0.7969
- ],
- "text": "Contoso,",
- "confidence": 1
- },
- {
- "boundingBox": [
- 1.8448,
- 0.4446,
- 2.3387,
- 0.4446,
- 2.3387,
- 0.7631,
- 1.8448,
- 0.7631
- ],
- "text": "Ltd.",
- "confidence": 1
- }
- ]
- },
- ...
- ]
- }
- ]
- }
- ],
- "pageResults": [
- {
- "page": 1,
- "tables": [
- {
- "rows": 5,
- "columns": 5,
- "cells": [
- {
- "rowIndex": 0,
- "columnIndex": 0,
- "text": "Training Date",
- "boundingBox": [
- 0.5133,
- 4.2167,
- 1.7567,
- 4.2167,
- 1.7567,
- 4.4492,
- 0.5133,
- 4.4492
- ],
- "elements": [
- "#/readResults/0/lines/14/words/0",
- "#/readResults/0/lines/14/words/1"
- ]
- },
- ...
- ]
- },
- ]
- }
- ],
- "documentResults": [
- {
- "docType": "custom:form",
- "pageRange": [
- 1,
- 1
- ],
- "fields": {
- "Receipt No": {
- "type": "string",
- "valueString": "9876",
- "text": "9876",
- "page": 1,
- "boundingBox": [
- 7.615,
- 1.245,
- 7.915,
- 1.245,
- 7.915,
- 1.35,
- 7.615,
- 1.35
- ],
- "confidence": 1,
- "elements": [
- "#/readResults/0/lines/3/words/3"
- ]
- },
- ...
- }
- }
- ],
- "errors": []
- }
-}
-```
-# [v 2](#tab/v2-1)
-```json
-{
- "status": "succeeded",
- "createdDateTime": "2020-08-21T02:29:42Z",
- "lastUpdatedDateTime": "2020-08-21T02:29:50Z",
- "analyzeResult": {
- "version": "2.1.0",
- "readResults": [
- {
- "page": 1,
- "angle": 0,
- "width": 8.5,
- "height": 11,
- "unit": "inch",
- "lines": [
- {
- "boundingBox": [
- 0.5826,
- 0.4411,
- 2.3387,
- 0.4411,
- 2.3387,
- 0.7969,
- 0.5826,
- 0.7969
- ],
- "text": "Contoso, Ltd.",
- "words": [
- {
- "boundingBox": [
- 0.5826,
- 0.4411,
- 1.744,
- 0.4411,
- 1.744,
- 0.7969,
- 0.5826,
- 0.7969
- ],
- "text": "Contoso,",
- "confidence": 1
- },
- {
- "boundingBox": [
- 1.8448,
- 0.4446,
- 2.3387,
- 0.4446,
- 2.3387,
- 0.7631,
- 1.8448,
- 0.7631
- ],
- "text": "Ltd.",
- "confidence": 1
- }
- ]
- },
- ...
- ],
- "selectionMarks": [
- {
- "boundingBox": [
- 3.9737,
- 3.7475,
- 4.1693,
- 3.7475,
- 4.1693,
- 3.9428,
- 3.9737,
- 3.9428
- ],
- ...
- ]
- }
- ],
- "pageResults": [
- {
- "page": 1,
- "tables": [
- {
- "rows": 5,
- "columns": 5,
- "cells": [
- {
- "rowIndex": 0,
- "columnIndex": 0,
- "text": "Training Date",
- "boundingBox": [
- 0.5133,
- 4.2167,
- 1.7567,
- 4.2167,
- 1.7567,
- 4.4492,
- 0.5133,
- 4.4492
- ],
- "elements": [
- "#/readResults/0/lines/12/words/0",
- "#/readResults/0/lines/12/words/1"
- ]
- },
- ...
- ]
- }
- ]
- }
- ],
- "documentResults": [
- {
- "docType": "custom:e1073364-4f3d-4797-8cc4-4bdbcd0dab6b",
- "modelId": "e1073364-4f3d-4797-8cc4-4bdbcd0dab6b",
- "pageRange": [
- 1,
- 1
- ],
- "fields": {
- "ID #": {
- "type": "string",
- "valueString": "5554443",
- "text": "5554443",
- "page": 1,
- "boundingBox": [
- 2.315,
- 2.43,
- 2.74,
- 2.43,
- 2.74,
- 2.515,
- 2.315,
- 2.515
- ],
- "confidence": 1,
- "elements": [
- "#/readResults/0/lines/8/words/1"
- ]
- },
- ...
- },
- "docTypeConfidence": 1
- }
- ],
- "errors": []
- }
-}
-```
---
-## Improve results
-
-Examine the `"confidence"` values for each key/value result under the `"documentResults"` node. You should also look at the confidence scores in the `"readResults"` node, which correspond to the Layout operation. The confidence of the layout results does not affect the confidence of the key/value extraction results, so you should check both.
-* If the confidence scores for the Layout operation are low, try to improve the quality of your input documents (see [Input requirements](../overview.md#input-requirements)).
-* If the confidence scores for the key/value extraction operation are low, ensure that the documents being analyzed are of the same type as documents used in the training set. If the documents in the training set have variations in appearance, consider splitting them into different folders and training one model for each variation.
-
-### Avoid cluttered labels
-
-Sometimes when you apply different labels within the same line of text, the service may merge those labels into one field. For example, in an address, you might label the city, state, and zip code as different fields, but during prediction those fields are not recognized separately.
-
-We understand this scenario is essential for our customers, and we are working on improving this in the future. Currently, we recommend our users to label multiple cluttered fields as one field, and then separate the terms in a post-processing of the extraction results.
-
-## Next steps
-
-In this quickstart, you learned how to use the Form Recognizer REST API with Python to train a model with manually labeled data. Next, see the API reference documentation to explore the Form Recognizer API in more depth.
-
-> [!div class="nextstepaction"]
-> [REST API reference documentation](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeWithCustomForm)
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/quickstarts/python-layout https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/quickstarts/python-layout.md deleted file mode 100644
@@ -1,410 +0,0 @@
-title: "Quickstart: Extract text and layout information using Python - Form Recognizer"
-titleSuffix: Azure Cognitive Services
-description: In this quickstart, you'll use the Form Recognizer Layout REST API with Python to read text and table data from your forms.
-author: PatrickFarley
-manager: nitinme
-
-ms.service: cognitive-services
-ms.subservice: forms-recognizer
-ms.topic: quickstart
-ms.date: 10/05/2020
-ms.author: pafarley
-ms.custom: devx-track-python
-
-# Quickstart: Extract text and layout information using the Form Recognizer REST API with Python
-
-In this quickstart, you'll use the Azure Form Recognizer REST API with Python to extract text layout information and table data from form documents.
-
-If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/cognitive-services/) before you begin.
-
-## Prerequisites
-
-To complete this quickstart, you must have:
-- [Python](https://www.python.org/downloads/) installed (if you want to run the sample locally).-- A form document. You can download an image from the [sample data set](https://go.microsoft.com/fwlink/?linkid=2090451) (download and extract *sample_data.zip*) for this quickstart.-
-> [!NOTE]
-> This quickstart uses a locally stored document. To use learn how to use remote files accessed by URL, see the [reference documentation](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync).
--
-## Create a Form Recognizer resource
-
-[!INCLUDE [create resource](../includes/create-resource.md)]
-
-## Analyze the form layout
-
-To start analyzing the layout, you call the **[Analyze Layout](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync)** API using the Python script below. Before you run the script, make these changes:
-
-1. Replace `<Endpoint>` with the endpoint that you obtained with your Form Recognizer subscription.
-1. Replace `<path to your form>` with the path to your local form document.
-1. Replace `<subscription key>` with the subscription key you copied from the previous step.
-
- # [v2.0](#tab/v2-0)
- ```python
- ########### Python Form Recognizer Async Layout #############
-
- import json
- import time
- from requests import get, post
-
- # Endpoint URL
- endpoint = r"<Endpoint>"
- apim_key = "<Subscription Key>"
- post_url = endpoint + "/formrecognizer/v2.0/Layout/analyze"
- source = r"<path to your form>"
-
- headers = {
- # Request headers
- # Change Content-Type as appropriate
- 'Content-Type': 'application/pdf',
- 'Ocp-Apim-Subscription-Key': apim_key,
- }
- with open(source, "rb") as f:
- data_bytes = f.read()
-
- try:
- resp = post(url = post_url, data = data_bytes, headers = headers)
- if resp.status_code != 202:
- print("POST analyze failed:\n%s" % resp.text)
- quit()
- print("POST analyze succeeded:\n%s" % resp.headers)
- get_url = resp.headers["operation-location"]
- except Exception as e:
- print("POST analyze failed:\n%s" % str(e))
- quit()
- ```
- # [v2.1 preview](#tab/v2-1)
- ```python
- ########### Python Form Recognizer Async Layout #############
-
- import json
- import time
- from requests import get, post
-
- # Endpoint URL
- endpoint = r"<Endpoint>"
- apim_key = "<Subscription Key>"
- post_url = endpoint + "/formrecognizer/v2.1-preview.2/Layout/analyze"
- source = r"<path to your form>"
-
- headers = {
- # Request headers
- # Change Content-Type as appropriate
- 'Content-Type': 'application/pdf',
- 'Ocp-Apim-Subscription-Key': apim_key,
- }
- with open(source, "rb") as f:
- data_bytes = f.read()
-
- try:
- resp = post(url = post_url, data = data_bytes, headers = headers)
- if resp.status_code != 202:
- print("POST analyze failed:\n%s" % resp.text)
- quit()
- print("POST analyze succeeded:\n%s" % resp.headers)
- get_url = resp.headers["operation-location"]
- except Exception as e:
- print("POST analyze failed:\n%s" % str(e))
- quit()
- ```
-
-
- ---
--
-1. Save the code in a file with a .py extension. For example, *form-recognizer-layout.py*.
-1. Open a command prompt window.
-1. At the prompt, use the `python` command to run the sample. For example, `python form-recognizer-layout.py`.
-
-You'll receive a `202 (Success)` response that includes an **Operation-Location** header, which the script will print to the console. This header contains an operation ID that you can use to query the status of the asynchronous operation and get the results. In the following example value, the string after `operations/` is the operation ID.
-
-# [v2.0](#tab/v2-0)
-```console
-https://cognitiveservice/formrecognizer/v2.0/layout/operations/54f0b076-4e38-43e5-81bd-b85b8835fdfb
-```
-# [v2.1 preview](#tab/v2-1)
-```console
-https://cognitiveservice/formrecognizer/v2.1-preview.2/layout/operations/54f0b076-4e38-43e5-81bd-b85b8835fdfb
-```
-
-
---
-## Get the layout results
-
-After you've called the **Analyze Layout** API, you call the **[Get Analyze Layout Result](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/GetAnalyzeLayoutResult)** API to get the status of the operation and the extracted data. Add the following code to the bottom of your Python script. This code uses the operation ID value in a new API call. This script calls the API at regular intervals until the results are available. We recommend an interval of one second or more.
-
-```python
-n_tries = 10
-n_try = 0
-wait_sec = 6
-while n_try < n_tries:
- try:
- resp = get(url = get_url, headers = {"Ocp-Apim-Subscription-Key": apim_key})
- resp_json = json.loads(resp.text)
- if resp.status_code != 200:
- print("GET Layout results failed:\n%s" % resp_json)
- quit()
- status = resp_json["status"]
- if status == "succeeded":
- print("Layout Analysis succeeded:\n%s" % resp_json)
- quit()
- if status == "failed":
- print("Layout Analysis failed:\n%s" % resp_json)
- quit()
- # Analysis still running. Wait and retry.
- time.sleep(wait_sec)
- n_try += 1
- except Exception as e:
- msg = "GET analyze results failed:\n%s" % str(e)
- print(msg)
- quit()
-```
-
-1. Save the script.
-1. Again use the `python` command to run the sample. For example, `python form-recognizer-layout.py`.
-
-### Examine the response
-
-The script will print responses to the console until the **Analyze Layout** operation completes. Then, it will print the extracted data in JSON format. The `"readResults"` node contains every line of text with its respective bounding box placement on the page. The `"selectionMarks"` node (in v2.1 preview) shows every selection mark (checkbox, radio mark) and whether its status is "selected" or "unselected". The `"pageResults"` field shows every piece of text within tables, each with its row-column coordinate.
-
-See the following invoice image and its corresponding JSON output. The output has been shortened for simplicity.
-
-:::image type="content" source="../media/contoso-invoice.png" alt-text="Contoso project statement document with a table.":::
-
-# [v2.0](#tab/v2-0)
-```json
-{
- "status": "succeeded",
- "createdDateTime": "2020-08-20T20:36:52Z",
- "lastUpdatedDateTime": "2020-08-20T20:36:58Z",
- "analyzeResult": {
- "version": "2.0.0",
- "readResults": [
- {
- "page": 1,
- "language": "en",
- "angle": 0,
- "width": 8.5,
- "height": 11,
- "unit": "inch",
- "lines": [
- {
- "boundingBox": [
- 0.5826,
- 0.4411,
- 2.3387,
- 0.4411,
- 2.3387,
- 0.7969,
- 0.5826,
- 0.7969
- ],
- "text": "Contoso, Ltd.",
- "words": [
- {
- "boundingBox": [
- 0.5826,
- 0.4411,
- 1.744,
- 0.4411,
- 1.744,
- 0.7969,
- 0.5826,
- 0.7969
- ],
- "text": "Contoso,",
- "confidence": 1
- },
- {
- "boundingBox": [
- 1.8448,
- 0.4446,
- 2.3387,
- 0.4446,
- 2.3387,
- 0.7631,
- 1.8448,
- 0.7631
- ],
- "text": "Ltd.",
- "confidence": 1
- }
- ]
- },
- ...
- ]
- }
- ],
- "pageResults": [
- {
- "page": 1,
- "tables": [
- {
- "rows": 5,
- "columns": 5,
- "cells": [
- {
- "rowIndex": 0,
- "columnIndex": 0,
- "text": "Training Date",
- "boundingBox": [
- 0.5133,
- 4.2167,
- 1.7567,
- 4.2167,
- 1.7567,
- 4.4492,
- 0.5133,
- 4.4492
- ],
- "elements": [
- "#/readResults/0/lines/14/words/0",
- "#/readResults/0/lines/14/words/1"
- ]
- },
- ...
- ]
- },
- ...
- ]
- }
- ]
- }
-}
-```
-
-# [v2.1 preview](#tab/v2-1)
-```json
-{
- "status": "succeeded",
- "createdDateTime": "2020-08-20T20:40:50Z",
- "lastUpdatedDateTime": "2020-08-20T20:40:55Z",
- "analyzeResult": {
- "version": "2.1.0",
- "readResults": [
- {
- "page": 1,
- "angle": 0,
- "width": 8.5,
- "height": 11,
- "unit": "inch",
- "lines": [
- {
- "boundingBox": [
- 0.5826,
- 0.4411,
- 2.3387,
- 0.4411,
- 2.3387,
- 0.7969,
- 0.5826,
- 0.7969
- ],
- "text": "Contoso, Ltd.",
- "words": [
- {
- "boundingBox": [
- 0.5826,
- 0.4411,
- 1.744,
- 0.4411,
- 1.744,
- 0.7969,
- 0.5826,
- 0.7969
- ],
- "text": "Contoso,",
- "confidence": 1
- },
- {
- "boundingBox": [
- 1.8448,
- 0.4446,
- 2.3387,
- 0.4446,
- 2.3387,
- 0.7631,
- 1.8448,
- 0.7631
- ],
- "text": "Ltd.",
- "confidence": 1
- }
- ]
- },
- ...
- ]
- }
- ],
- "selectionMarks": [
- {
- "boundingBox": [
- 3.9737,
- 3.7475,
- 4.1693,
- 3.7475,
- 4.1693,
- 3.9428,
- 3.9737,
- 3.9428
- ],
- "confidence": 0.989,
- "state": "selected"
- },
- ...
- ]
- }
- ],
- "pageResults": [
- {
- "page": 1,
- "tables": [
- {
- "rows": 5,
- "columns": 5,
- "cells": [
- {
- "rowIndex": 0,
- "columnIndex": 0,
- "text": "Training Date",
- "boundingBox": [
- 0.5133,
- 4.2167,
- 1.7567,
- 4.2167,
- 1.7567,
- 4.4492,
- 0.5133,
- 4.4492
- ],
- "elements": [
- "#/readResults/0/lines/12/words/0",
- "#/readResults/0/lines/12/words/1"
- ]
- },
- ...
- ]
- },
- ...
- ]
- }
- ]
- }
-}
-```
----
-## Next steps
-
-In this quickstart, you used the Form Recognizer REST API with Python to extract the text layout of an invoice. Next, see the reference documentation to explore the Form Recognizer API in more depth.
-
-> [!div class="nextstepaction"]
-> [REST API reference documentation](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeLayoutAsync)
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/quickstarts/python-receipts https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/quickstarts/python-receipts.md deleted file mode 100644
@@ -1,521 +0,0 @@
-title: "Quickstart: Extract receipt data using Python - Form Recognizer"
-titleSuffix: Azure Cognitive Services
-description: In this quickstart, you'll use the Form Recognizer REST API with Python to extract data from images of USA sales receipts.
-author: PatrickFarley
-manager: nitinme
-
-ms.service: cognitive-services
-ms.subservice: forms-recognizer
-ms.topic: quickstart
-ms.date: 10/05/2020
-ms.author: pafarley
-ms.custom: devx-track-python
-#Customer intent: As a developer or data scientist familiar with Python, I want to learn how to use a prebuilt Form Recognizer model to extract my receipt data.
-
-# Quickstart: Extract receipt data using the Form Recognizer REST API with Python
-
-In this quickstart, you'll use the Azure Form Recognizer REST API with Python to extract and identify relevant information in USA sales receipts.
-
-If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/cognitive-services/) before you begin.
-
-## Prerequisites
-
-To complete this quickstart, you must have:
-- [Python](https://www.python.org/downloads/) installed (if you want to run the sample locally).-- An image of a receipt. You can use a [sample image](https://raw.githubusercontent.com/Azure-Samples/cognitive-services-REST-api-samples/master/curl/form-recognizer/contoso-allinone.jpg) for this quickstart.-
-> [!NOTE]
-> This quickstart uses a local file. To use a receipt image accessed by URL instead, see the [reference documentation](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeReceiptAsync).
-
-## Create a Form Recognizer resource
-
-[!INCLUDE [create resource](../includes/create-resource.md)]
-
-## Analyze a receipt
-
-To start analyzing a receipt, you call the **[Analyze Receipt](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeReceiptAsync)** API using the Python script below. Before you run the script, make these changes:
-
-1. Replace `<Endpoint>` with the endpoint that you obtained with your Form Recognizer subscription.
-1. Replace `<path to your receipt>` with the path to your local form document.
-1. Replace `<subscription key>` with the subscription key you copied from the previous step.
-
-# [v2.0](#tab/v2-0)
-
-```python
- ########### Python Form Recognizer Async Receipt #############
-
- import json
- import time
- from requests import get, post
-
- # Endpoint URL
- endpoint = r"<Endpoint>"
- apim_key = "<subscription key>"
- post_url = endpoint + "/formrecognizer/v2.0/prebuilt/receipt/analyze"
- source = r"<path to your receipt>"
-
- headers = {
- # Request headers
- 'Content-Type': '<file type>',
- 'Ocp-Apim-Subscription-Key': apim_key,
- }
-
- params = {
- "includeTextDetails": True
- }
-
- with open(source, "rb") as f:
- data_bytes = f.read()
-
- try:
- resp = post(url = post_url, data = data_bytes, headers = headers, params = params)
- if resp.status_code != 202:
- print("POST analyze failed:\n%s" % resp.text)
- quit()
- print("POST analyze succeeded:\n%s" % resp.headers)
- get_url = resp.headers["operation-location"]
- except Exception as e:
- print("POST analyze failed:\n%s" % str(e))
- quit()
-```
-
-# [v2.1-preview.2](#tab/v2-1)
-```python
- ########### Python Form Recognizer Async Receipt #############
-
- import json
- import time
- from requests import get, post
-
- # Endpoint URL
- endpoint = r"<Endpoint>"
- apim_key = "<subscription key>"
- post_url = endpoint + "/formrecognizer/v2.1-preview.2/prebuilt/receipt/analyze"
- source = r"<path to your receipt>"
-
- headers = {
- # Request headers
- 'Content-Type': '<file type>',
- 'Ocp-Apim-Subscription-Key': apim_key,
- }
-
- params = {
- "includeTextDetails": True
- "locale": "en-US"
- }
-
- with open(source, "rb") as f:
- data_bytes = f.read()
-
- try:
- resp = post(url = post_url, data = data_bytes, headers = headers, params = params)
- if resp.status_code != 202:
- print("POST analyze failed:\n%s" % resp.text)
- quit()
- print("POST analyze succeeded:\n%s" % resp.headers)
- get_url = resp.headers["operation-location"]
- except Exception as e:
- print("POST analyze failed:\n%s" % str(e))
- quit()
-```
-
-> [!NOTE]
-> **Language input**
->
-> The Analzye Receipt 2.1 release operation has an optional request parameter for language, locale of the receipt. Supported locales include: en-AU, en-CA, en-GB, en-IN, en-US.
--
-1. Save the code in a file with a .py extension. For example, *form-recognizer-receipts.py*.
-1. Open a command prompt window.
-1. At the prompt, use the `python` command to run the sample. For example, `python form-recognizer-receipts.py`.
-
-You'll receive a `202 (Success)` response that includes an **Operation-Location** header, which the script will print to the console. This header contains an operation ID that you can use to query the status of the asynchronous operation and get the results. In the following example value, the string after `operations/` is the operation ID.
-
-# [v2.0](#tab/v2-0)
-```console
-https://cognitiveservice/formrecognizer/v2.0/prebuilt/receipt/operations/54f0b076-4e38-43e5-81bd-b85b8835fdfb
-```
-# [v2.1-preview.2](#tab/v2-1)
-```console
-https://cognitiveservice/formrecognizer/v2.1-preview.2/prebuilt/receipt/operations/54f0b076-4e38-43e5-81bd-b85b8835fdfb
-```
-
-## Get the receipt results
-
-After you've called the **Analyze Receipt** API, you call the **[Get Analyze Receipt Result](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/GetAnalyzeReceiptResult)** API to get the status of the operation and the extracted data. Add the following code to the bottom of your Python script. This uses the operation ID value in a new API call. This script calls the API at regular intervals until the results are available. We recommend an interval of one second or more.
-
-```python
-n_tries = 10
-n_try = 0
-wait_sec = 6
-while n_try < n_tries:
- try:
- resp = get(url = get_url, headers = {"Ocp-Apim-Subscription-Key": apim_key})
- resp_json = json.loads(resp.text)
- if resp.status_code != 200:
- print("GET Receipt results failed:\n%s" % resp_json)
- quit()
- status = resp_json["status"]
- if status == "succeeded":
- print("Receipt Analysis succeeded:\n%s" % resp_json)
- quit()
- if status == "failed":
- print("Analysis failed:\n%s" % resp_json)
- quit()
- # Analysis still running. Wait and retry.
- time.sleep(wait_sec)
- n_try += 1
- except Exception as e:
- msg = "GET analyze results failed:\n%s" % str(e)
- print(msg)
- quit()
-```
-
-1. Save the script.
-1. Again use the `python` command to run the sample. For example, `python form-recognizer-receipts.py`.
-
-### Examine the response
-
-The script will print responses to the console until the **Analyze Receipt** operation completes. Then, it will print the extracted text data in JSON format. The `"readResults"` field contains every line of text that was extracted from the receipt, and the `"documentResults"` field contains key/value information for the most relevant parts of the receipt.
-
-See the following receipt image and its corresponding JSON output. The output has been shortened for readability.
-
-![A receipt from Contoso store](../media/contoso-allinone.jpg)
-
-The `"readResults"` node contains all of the recognized text. Text is organized by page, then by line, then by individual words. The `"documentResults"` node contains the receipt-specific values that the model discovered. This is where you'll find useful key/value pairs like the tax, total, merchant address, and so on.
-
-```json
-{
- "status":"succeeded",
- "createdDateTime":"2019-12-17T04:11:24Z",
- "lastUpdatedDateTime":"2019-12-17T04:11:32Z",
- "analyzeResult":{
- "version":"2.0.0",
- "readResults":[
- {
- "page":1,
- "angle":0.6893,
- "width":1688,
- "height":3000,
- "unit":"pixel",
- "language":"en",
- "lines":[
- {
- "text":"Contoso",
- "boundingBox":[
- 635,
- 510,
- 1086,
- 461,
- 1098,
- 558,
- 643,
- 604
- ],
- "words":[
- {
- "text":"Contoso",
- "boundingBox":[
- 639,
- 510,
- 1087,
- 461,
- 1098,
- 551,
- 646,
- 604
- ],
- "confidence":0.955
- }
- ]
- },
- ...
- ]
- }
- ],
- "documentResults":[
- {
- "docType":"prebuilt:receipt",
- "pageRange":[
- 1,
- 1
- ],
- "fields":{
- "ReceiptType":{
- "type":"string",
- "valueString":"Itemized",
- "confidence":0.692
- },
- "MerchantName":{
- "type":"string",
- "valueString":"Contoso Contoso",
- "text":"Contoso Contoso",
- "boundingBox":[
- 378.2,
- 292.4,
- 1117.7,
- 468.3,
- 1035.7,
- 812.7,
- 296.3,
- 636.8
- ],
- "page":1,
- "confidence":0.613,
- "elements":[
- "#/readResults/0/lines/0/words/0",
- "#/readResults/0/lines/1/words/0"
- ]
- },
- "MerchantAddress":{
- "type":"string",
- "valueString":"123 Main Street Redmond, WA 98052",
- "text":"123 Main Street Redmond, WA 98052",
- "boundingBox":[
- 302,
- 675.8,
- 848.1,
- 793.7,
- 809.9,
- 970.4,
- 263.9,
- 852.5
- ],
- "page":1,
- "confidence":0.99,
- "elements":[
- "#/readResults/0/lines/2/words/0",
- "#/readResults/0/lines/2/words/1",
- "#/readResults/0/lines/2/words/2",
- "#/readResults/0/lines/3/words/0",
- "#/readResults/0/lines/3/words/1",
- "#/readResults/0/lines/3/words/2"
- ]
- },
- "MerchantPhoneNumber":{
- "type":"phoneNumber",
- "valuePhoneNumber":"+19876543210",
- "text":"987-654-3210",
- "boundingBox":[
- 278,
- 1004,
- 656.3,
- 1054.7,
- 646.8,
- 1125.3,
- 268.5,
- 1074.7
- ],
- "page":1,
- "confidence":0.99,
- "elements":[
- "#/readResults/0/lines/4/words/0"
- ]
- },
- "TransactionDate":{
- "type":"date",
- "valueDate":"2019-06-10",
- "text":"6/10/2019",
- "boundingBox":[
- 265.1,
- 1228.4,
- 525,
- 1247,
- 518.9,
- 1332.1,
- 259,
- 1313.5
- ],
- "page":1,
- "confidence":0.99,
- "elements":[
- "#/readResults/0/lines/5/words/0"
- ]
- },
- "TransactionTime":{
- "type":"time",
- "valueTime":"13:59:00",
- "text":"13:59",
- "boundingBox":[
- 541,
- 1248,
- 677.3,
- 1261.5,
- 668.9,
- 1346.5,
- 532.6,
- 1333
- ],
- "page":1,
- "confidence":0.977,
- "elements":[
- "#/readResults/0/lines/5/words/1"
- ]
- },
- "Items":{
- "type":"array",
- "valueArray":[
- {
- "type":"object",
- "valueObject":{
- "Quantity":{
- "type":"number",
- "text":"1",
- "boundingBox":[
- 245.1,
- 1581.5,
- 300.9,
- 1585.1,
- 295,
- 1676,
- 239.2,
- 1672.4
- ],
- "page":1,
- "confidence":0.92,
- "elements":[
- "#/readResults/0/lines/7/words/0"
- ]
- },
- "Name":{
- "type":"string",
- "valueString":"Cappuccino",
- "text":"Cappuccino",
- "boundingBox":[
- 322,
- 1586,
- 654.2,
- 1601.1,
- 650,
- 1693,
- 317.8,
- 1678
- ],
- "page":1,
- "confidence":0.923,
- "elements":[
- "#/readResults/0/lines/7/words/1"
- ]
- },
- "TotalPrice":{
- "type":"number",
- "valueNumber":2.2,
- "text":"$2.20",
- "boundingBox":[
- 1107.7,
- 1584,
- 1263,
- 1574,
- 1268.3,
- 1656,
- 1113,
- 1666
- ],
- "page":1,
- "confidence":0.918,
- "elements":[
- "#/readResults/0/lines/8/words/0"
- ]
- }
- }
- },
- ...
- ]
- },
- "Subtotal":{
- "type":"number",
- "valueNumber":11.7,
- "text":"11.70",
- "boundingBox":[
- 1146,
- 2221,
- 1297.3,
- 2223,
- 1296,
- 2319,
- 1144.7,
- 2317
- ],
- "page":1,
- "confidence":0.955,
- "elements":[
- "#/readResults/0/lines/13/words/1"
- ]
- },
- "Tax":{
- "type":"number",
- "valueNumber":1.17,
- "text":"1.17",
- "boundingBox":[
- 1190,
- 2359,
- 1304,
- 2359,
- 1304,
- 2456,
- 1190,
- 2456
- ],
- "page":1,
- "confidence":0.979,
- "elements":[
- "#/readResults/0/lines/15/words/1"
- ]
- },
- "Tip":{
- "type":"number",
- "valueNumber":1.63,
- "text":"1.63",
- "boundingBox":[
- 1094,
- 2479,
- 1267.7,
- 2485,
- 1264,
- 2591,
- 1090.3,
- 2585
- ],
- "page":1,
- "confidence":0.941,
- "elements":[
- "#/readResults/0/lines/17/words/1"
- ]
- },
- "Total":{
- "type":"number",
- "valueNumber":14.5,
- "text":"$14.50",
- "boundingBox":[
- 1034.2,
- 2617,
- 1387.5,
- 2638.2,
- 1380,
- 2763,
- 1026.7,
- 2741.8
- ],
- "page":1,
- "confidence":0.985,
- "elements":[
- "#/readResults/0/lines/19/words/0"
- ]
- }
- }
- }
- ]
- }
-}
-```
-
-## Next steps
-
-In this quickstart, you used the Form Recognizer REST API with Python to extract the content of a sales receipt. Next, see the reference documentation to explore the Form Recognizer API in more depth.
-
-> [!div class="nextstepaction"]
-> [REST API reference documentation](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeReceiptAsync)
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/quickstarts/python-train-extract https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/quickstarts/python-train-extract.md deleted file mode 100644
@@ -1,540 +0,0 @@
-title: "Quickstart: Train a model and extract form data using the REST API with Python - Form Recognizer"
-titleSuffix: Azure Cognitive Services
-description: In this quickstart, you'll use the Form Recognizer REST API with Python to train a model and extract data from forms.
-author: PatrickFarley
-manager: nitinme
-
-ms.service: cognitive-services
-ms.subservice: forms-recognizer
-ms.topic: quickstart
-ms.date: 10/05/2020
-ms.author: pafarley
-ms.custom: devx-track-python
-#Customer intent: As a developer or data scientist familiar with Python, I want to learn how to use Form Recognizer to extract my form data.
-
-# Quickstart: Train a Form Recognizer model and extract form data by using the REST API with Python
-
-In this quickstart, you'll use the Azure Form Recognizer REST API with Python to train and score forms to extract key-value pairs and tables.
-
-If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/cognitive-services/) before you begin.
-
-## Prerequisites
-
-To complete this quickstart, you must have:
-- [Python](https://www.python.org/downloads/) installed (if you want to run the sample locally).-- A set of at least five forms of the same type. You will use this data to train the model. Your forms can be of different file types but must be the same type of document. You can use a [sample data set](https://go.microsoft.com/fwlink/?linkid=2090451) (download and extract *sample_data.zip*) for this quickstart. Upload the training files to the root of a blob storage container in an standard-performance-tier Azure Storage account.-
-> [!NOTE]
-> This quickstart uses remote documents accessed by URL. To use local files instead, see the [reference documentation](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/TrainCustomModelAsync).
--
-## Create a Form Recognizer resource
-
-[!INCLUDE [create resource](../includes/create-resource.md)]
-
-## Train a Form Recognizer model
-
-First, you'll need a set of training data in an Azure Storage blob container. You should have a minimum of five filled-in forms (PDF documents and/or images) of the same type/structure as your main input data. Or, you can use a single empty form with two filled-in forms. The empty form's file name needs to include the word "empty." See [Build a training data set for a custom model](../build-training-data-set.md) for tips and options for putting together your training data.
-
-> [!NOTE]
-> You can use the labeled data feature to manually label some or all of your training data beforehand. This is a more complex process but results in a better trained model. See the [Train with labels](../overview.md#train-with-labels) section of the overview to learn more.
-
-To train a Form Recognizer model with the documents in your Azure blob container, call the **[Train Custom Model](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/TrainCustomModelAsync)** API by running the following python code. Before you run the code, make these changes:
-
-1. Replace `<SAS URL>` with the Azure Blob storage container's shared access signature (SAS) URL. [!INCLUDE [get SAS URL](../includes/sas-instructions.md)]
-
- :::image type="content" source="../media/quickstarts/get-sas-url.png" alt-text="SAS URL retrieval":::
-1. Replace `<subscription key>` with the subscription key you copied from the previous step.
-1. Replace `<endpoint>` with the endpoint URL for your Form Recognizer resource.
-1. Replace `<Blob folder name>` with the path to the folder in blob storage where your forms are located. If your forms are at the root of your container, leave this string empty.
-1. Optionally replace `<your model name>` with the friendly name you'd like to give to your model.
-
- # [v2.0](#tab/v2-0)
- ```python
- ########### Python Form Recognizer Labeled Async Train #############
- import json
- import time
- from requests import get, post
-
- # Endpoint URL
- endpoint = r"<endpoint>"
- post_url = endpoint + r"/formrecognizer/v2.0/custom/models"
- source = r"<SAS URL>"
- prefix = "<Blob folder name>"
- includeSubFolders = False
- useLabelFile = False
-
- headers = {
- # Request headers
- 'Content-Type': 'application/json',
- 'Ocp-Apim-Subscription-Key': '<subsription key>',
- }
-
- body = {
- "source": source,
- "sourceFilter": {
- "prefix": prefix,
- "includeSubFolders": includeSubFolders
- },
- "modelName":"<your model name>",
- "useLabelFile": useLabelFile
- }
-
- try:
- resp = post(url = post_url, json = body, headers = headers)
- if resp.status_code != 201:
- print("POST model failed (%s):\n%s" % (resp.status_code, json.dumps(resp.json())))
- quit()
- print("POST model succeeded:\n%s" % resp.headers)
- get_url = resp.headers["location"]
- except Exception as e:
- print("POST model failed:\n%s" % str(e))
- quit()
- ```
- # [v2.1 preview](#tab/v2-1)
- ```python
- ########### Python Form Recognizer Labeled Async Train #############
- import json
- import time
- from requests import get, post
-
- # Endpoint URL
- endpoint = r"<endpoint>"
- post_url = endpoint + r"/formrecognizer/v2.1-preview.2/custom/models"
- source = r"<SAS URL>"
- prefix = "<Blob folder name>"
- includeSubFolders = False
- useLabelFile = False
-
- headers = {
- # Request headers
- 'Content-Type': 'application/json',
- 'Ocp-Apim-Subscription-Key': '<subsription key>',
- }
-
- body = {
- "source": source,
- "sourceFilter": {
- "prefix": prefix,
- "includeSubFolders": includeSubFolders
- },
- "useLabelFile": useLabelFile
- }
-
- try:
- resp = post(url = post_url, json = body, headers = headers)
- if resp.status_code != 201:
- print("POST model failed (%s):\n%s" % (resp.status_code, json.dumps(resp.json())))
- quit()
- print("POST model succeeded:\n%s" % resp.headers)
- get_url = resp.headers["location"]
- except Exception as e:
- print("POST model failed:\n%s" % str(e))
- quit()
- ```
--
- ---
--
-1. Save the code in a file with a .py extension. For example, *form-recognizer-train.py*.
-1. Open a command prompt window.
-1. At the prompt, use the `python` command to run the sample. For example, `python form-recognizer-train.py`.
-
-## Get training results
-
-After you've started the train operation, you use the returned ID to get the status of the operation. Add the following code to the bottom of your Python script. This uses the ID value from the training call in a new API call. The training operation is asynchronous, so this script calls the API at regular intervals until the training status is completed. We recommend an interval of one second or more.
-
-```python
-n_tries = 15
-n_try = 0
-wait_sec = 5
-max_wait_sec = 60
-while n_try < n_tries:
- try:
- resp = get(url = get_url, headers = headers)
- resp_json = resp.json()
- if resp.status_code != 200:
- print("GET model failed (%s):\n%s" % (resp.status_code, json.dumps(resp_json)))
- quit()
- model_status = resp_json["modelInfo"]["status"]
- if model_status == "ready":
- print("Training succeeded:\n%s" % json.dumps(resp_json))
- quit()
- if model_status == "invalid":
- print("Training failed. Model is invalid:\n%s" % json.dumps(resp_json))
- quit()
- # Training still running. Wait and retry.
- time.sleep(wait_sec)
- n_try += 1
- wait_sec = min(2*wait_sec, max_wait_sec)
- except Exception as e:
- msg = "GET model failed:\n%s" % str(e)
- print(msg)
- quit()
-print("Train operation did not complete within the allocated time.")
-```
-
-When the training process is completed, you'll receive a `201 (Success)` response with JSON content like the following:
-
-```json
-{
- "modelInfo":{
- "status":"ready",
- "createdDateTime":"2019-10-08T10:20:31.957784",
- "lastUpdatedDateTime":"2019-10-08T14:20:41+00:00",
- "modelId":"1cfb372bab404ba3aa59481ab2c63da5"
- },
- "trainResult":{
- "trainingDocuments":[
- {
- "documentName":"invoices\\Invoice_1.pdf",
- "pages":1,
- "errors":[
-
- ],
- "status":"succeeded"
- },
- {
- "documentName":"invoices\\Invoice_2.pdf",
- "pages":1,
- "errors":[
-
- ],
- "status":"succeeded"
- },
- {
- "documentName":"invoices\\Invoice_3.pdf",
- "pages":1,
- "errors":[
-
- ],
- "status":"succeeded"
- },
- {
- "documentName":"invoices\\Invoice_4.pdf",
- "pages":1,
- "errors":[
-
- ],
- "status":"succeeded"
- },
- {
- "documentName":"invoices\\Invoice_5.pdf",
- "pages":1,
- "errors":[
-
- ],
- "status":"succeeded"
- }
- ],
- "errors":[
-
- ]
- },
- "keys":{
- "0":[
- "Address:",
- "Invoice For:",
- "Microsoft",
- "Page"
- ]
- }
-}
-```
-
-Copy the `"modelId"` value for use in the following steps.
-
-[!INCLUDE [analyze forms](../includes/python-custom-analyze.md)]
-
-When the process is completed, you'll receive a `200 (Success)` response with JSON content in the following format. The response has been shortened for simplicity. The main key/value pair associations and tables are in the `"pageResults"` node. If you also specified plain text extraction through the *includeTextDetails* URL parameter, then the `"readResults"` node will show the content and positions of all the text in the document.
--
-This sample JSON output has been shortened for simplicity.
-
-# [v2.0](#tab/v2-0)
-```JSON
-{
- "status": "succeeded",
- "createdDateTime": "2020-08-21T00:46:25Z",
- "lastUpdatedDateTime": "2020-08-21T00:46:32Z",
- "analyzeResult": {
- "version": "2.0.0",
- "readResults": [
- {
- "page": 1,
- "angle": 0,
- "width": 8.5,
- "height": 11,
- "unit": "inch",
- "lines": [
- {
- "text": "Project Statement",
- "boundingBox": [
- 5.0153,
- 0.275,
- 8.0944,
- 0.275,
- 8.0944,
- 0.7125,
- 5.0153,
- 0.7125
- ],
- "words": [
- {
- "text": "Project",
- "boundingBox": [
- 5.0153,
- 0.275,
- 6.2278,
- 0.275,
- 6.2278,
- 0.7125,
- 5.0153,
- 0.7125
- ]
- },
- {
- "text": "Statement",
- "boundingBox": [
- 6.3292,
- 0.275,
- 8.0944,
- 0.275,
- 8.0944,
- 0.7125,
- 6.3292,
- 0.7125
- ]
- }
- ]
- },
- ...
- ]
- }
- ],
- "pageResults": [
- {
- "page": 1,
- "keyValuePairs": [
- {
- "key": {
- "text": "Date:",
- "boundingBox": [
- 6.9722,
- 1.0264,
- 7.3417,
- 1.0264,
- 7.3417,
- 1.1931,
- 6.9722,
- 1.1931
- ],
- "elements": [
- "#/readResults/0/lines/2/words/0"
- ]
- },
- "confidence": 1
- },
- ...
- ],
- "tables": [
- {
- "rows": 4,
- "columns": 5,
- "cells": [
- {
- "text": "Training Date",
- "rowIndex": 0,
- "columnIndex": 0,
- "boundingBox": [
- 0.6931,
- 4.2444,
- 1.5681,
- 4.2444,
- 1.5681,
- 4.4125,
- 0.6931,
- 4.4125
- ],
- "confidence": 1,
- "rowSpan": 1,
- "columnSpan": 1,
- "elements": [
- "#/readResults/0/lines/15/words/0",
- "#/readResults/0/lines/15/words/1"
- ],
- "isHeader": true,
- "isFooter": false
- },
- ...
- ]
- }
- ],
- "clusterId": 0
- }
- ],
- "documentResults": [],
- "errors": []
- }
-}
-```
-# [v2.1 preview](#tab/v2-1)
-```JSON
-{
- "status": "succeeded",
- "createdDateTime": "2020-08-21T01:13:28Z",
- "lastUpdatedDateTime": "2020-08-21T01:13:42Z",
- "analyzeResult": {
- "version": "2.1.0",
- "readResults": [
- {
- "page": 1,
- "angle": 0,
- "width": 8.5,
- "height": 11,
- "unit": "inch",
- "lines": [
- {
- "text": "Project Statement",
- "boundingBox": [
- 5.0444,
- 0.3613,
- 8.0917,
- 0.3613,
- 8.0917,
- 0.6718,
- 5.0444,
- 0.6718
- ],
- "words": [
- {
- "text": "Project",
- "boundingBox": [
- 5.0444,
- 0.3587,
- 6.2264,
- 0.3587,
- 6.2264,
- 0.708,
- 5.0444,
- 0.708
- ]
- },
- {
- "text": "Statement",
- "boundingBox": [
- 6.3361,
- 0.3635,
- 8.0917,
- 0.3635,
- 8.0917,
- 0.6396,
- 6.3361,
- 0.6396
- ]
- }
- ]
- },
- ...
- ]
- }
- ],
- "pageResults": [
- {
- "page": 1,
- "keyValuePairs": [
- {
- "key": {
- "text": "Date:",
- "boundingBox": [
- 6.9833,
- 1.0615,
- 7.3333,
- 1.0615,
- 7.3333,
- 1.1649,
- 6.9833,
- 1.1649
- ],
- "elements": [
- "#/readResults/0/lines/2/words/0"
- ]
- },
- "value": {
- "text": "9/10/2020",
- "boundingBox": [
- 7.3833,
- 1.0802,
- 7.925,
- 1.0802,
- 7.925,
- 1.174,
- 7.3833,
- 1.174
- ],
- "elements": [
- "#/readResults/0/lines/3/words/0"
- ]
- },
- "confidence": 1
- },
- ...
- ],
- "tables": [
- {
- "rows": 5,
- "columns": 5,
- "cells": [
- {
- "text": "Training Date",
- "rowIndex": 0,
- "columnIndex": 0,
- "boundingBox": [
- 0.6944,
- 4.2779,
- 1.5625,
- 4.2779,
- 1.5625,
- 4.4005,
- 0.6944,
- 4.4005
- ],
- "confidence": 1,
- "rowSpan": 1,
- "columnSpan": 1,
- "elements": [
- "#/readResults/0/lines/15/words/0",
- "#/readResults/0/lines/15/words/1"
- ],
- "isHeader": true,
- "isFooter": false
- },
- ...
- ]
- }
- ],
- "clusterId": 0
- }
- ],
- "documentResults": [],
- "errors": []
- }
-}
-```
---
-## Improve results
-
-[!INCLUDE [improve results](../includes/improve-results-unlabeled.md)]
-
-## Next steps
-
-In this quickstart, you used the Form Recognizer REST API with Python to train a model and run it in a sample scenario. Next, see the reference documentation to explore the Form Recognizer API in more depth.
-
-> [!div class="nextstepaction"]
-> [REST API reference documentation](https://westus2.dev.cognitive.microsoft.com/docs/services/form-recognizer-api-v2/operations/AnalyzeWithCustomForm)
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/whats-new https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/form-recognizer/whats-new.md
@@ -206,7 +206,7 @@ The JSON responses for all API calls have new formats. Some keys and values have
## Next steps
-Complete a [client library quickstart](quickstarts/client-library.md) to get started writing a forms processing app with Form Recognizer in the language of your choice.
+Complete a [quickstart](quickstarts/client-library.md) to get started writing a forms processing app with Form Recognizer in the language of your choice.
## See also
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/metrics-advisor/includes/quickstarts/javascript https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/metrics-advisor/includes/quickstarts/javascript.md
@@ -11,7 +11,7 @@ ms.date: 11/09/2020
ms.author: mbullwin ---
-[Reference documentation](/javascript/api/overview/azure/ai-metrics-advisor-readme-pre?preserve-view=true&view=azure-node-preview) | [Library source code](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/metricsadvisor/ai-metrics-advisor/README.md) | [Package (npm)](https://www.npmjs.com/package/@azure/ai-metrics-advisor) | [Samples](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/metricsadvisor/ai-metrics-advisor/samples)
+[Reference documentation](/java/api/overview/azure/ai-metricsadvisor-readme?view=azure-java-preview) | [Library source code](https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/metricsadvisor/ai-metrics-advisor/README.md) | [Package (npm)](https://www.npmjs.com/package/@azure/ai-metrics-advisor) | [Samples](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/metricsadvisor/ai-metrics-advisor/samples)
## Prerequisites
cognitive-services https://docs.microsoft.com/en-us/azure/cognitive-services/text-analytics/language-support https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/cognitive-services/text-analytics/language-support.md
@@ -55,9 +55,9 @@ ms.author: aahi
| Language | Language code | v2.1 support | v3 support | Starting with v3 model version: | Notes | |:-----------------------|:-------------:|:----------:|:----------:|:-------------------------------:|:------------------:| | Arabic | `ar` | Γ£ô | | | |
-| Czech | `cs` | Γ£ô | | | |
| Chinese-Simplified | `zh-hans` | Γ£ô | | | `zh` also accepted | | Chinese-Traditional | `zh-hant` | Γ£ô | | | |
+| Czech | `cs` | Γ£ô | | | |
| Danish | `da` | Γ£ô | | | | | Dutch | `nl` | Γ£ô | | | | | English | `en` | Γ£ô | Γ£ô | 2019-10-01 | |
@@ -71,8 +71,8 @@ ms.author: aahi
| Korean | `ko` | ✓ | | | | | Norwegian (Bokmål) | `no` | ✓ | | | `nb` also accepted | | Polish | `pl` | ✓ | | | |
-| Portuguese (Portugal) | `pt-PT` | Γ£ô | | | `pt` also accepted |
| Portuguese (Brazil) | `pt-BR` | Γ£ô | | | |
+| Portuguese (Portugal) | `pt-PT` | Γ£ô | | | `pt` also accepted |
| Russian | `ru` | Γ£ô | | | | | Spanish | `es` | Γ£ô | Γ£ô | 2020-04-01 | | | Swedish | `sv` | Γ£ô | | | |
@@ -85,21 +85,21 @@ ms.author: aahi
| Language | Language code | v2 support | v3 support | Available starting with v3 model version: | Notes | |:----------------------|:-------------:|:----------:|:----------:|:-----------------------------------------:|:------------------:|
-| Dutch | `nl` | Γ£ô | Γ£ô | 2019-10-01 | |
-| English | `en` | Γ£ô | Γ£ô | 2019-10-01 | |
-| Finnish | `fi` | Γ£ô | Γ£ô | 2019-10-01 | |
-| French | `fr` | Γ£ô | Γ£ô | 2019-10-01 | |
-| German | `de` | Γ£ô | Γ£ô | 2019-10-01 | |
-| Italian | `it` | Γ£ô | Γ£ô | 2019-10-01 | |
-| Japanese | `ja` | Γ£ô | Γ£ô | 2019-10-01 | |
-| Korean | `ko` | Γ£ô | Γ£ô | 2019-10-01 | |
-| Norwegian (Bokmål) | `no` | ✓ | ✓ | 2019-10-01 | `nb` also accepted |
-| Polish | `pl` | Γ£ô | Γ£ô | 2019-10-01 | |
-| Portuguese (Portugal) | `pt-PT` | Γ£ô | Γ£ô | 2019-10-01 | `pt` also accepted |
-| Portuguese (Brazil) | `pt-BR` | Γ£ô | Γ£ô | 2019-10-01 | |
-| Russian | `ru` | Γ£ô | Γ£ô | 2019-10-01 | |
-| Spanish | `es` | Γ£ô | Γ£ô | 2019-10-01 | |
-| Swedish | `sv` | Γ£ô | Γ£ô | 2019-10-01 | |
+| Dutch                 |     `nl`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| English               |     `en`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| Finnish               |     `fi`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| French                |     `fr`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| German                |     `de`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| Italian               |     `it`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| Japanese              |     `ja`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| Korean                |     `ko`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| Norwegian  (Bokmål)   |     `no`      |     ✓      |     ✓      |                2019-10-01                 | `nb` also accepted |
+| Polish                |     `pl`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| Portuguese (Brazil)   |    `pt-BR`    |     ✓      |     ✓      |                2019-10-01                 |                    |
+| Portuguese (Portugal) |    `pt-PT`    |     ✓      |     ✓      |                2019-10-01                 | `pt` also accepted |
+| Russian               |     `ru`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| Spanish               |     `es`      |     ✓      |     ✓      |                2019-10-01                 |                    |
+| Swedish               |     `sv`      |     ✓      |     ✓      |                2019-10-01                 |                    |
#### [Entity linking](#tab/entity-linking)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/architecture https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/architecture.md
@@ -4,7 +4,7 @@ description: Learn about Azure Defender for IoT architecture and information flo
services: defender-for-iot ms.service: defender-for-iot documentationcenter: na
-author: rkarlin
+author: shhazam-ms
manager: rkarlin editor: ''
@@ -12,8 +12,8 @@ ms.devlang: na
ms.topic: conceptual ms.tgt_pltfrm: na ms.workload: na
-ms.date: 10/08/2020
-ms.author: rkarlin
+ms.date: 12/02/2020
+ms.author: shhazam
--- # Azure Defender for IoT architecture
@@ -38,18 +38,17 @@ Azure Defender for IoT includes the following components:
- Azure Defender for IoT sensor VM or appliance - On-premises management console for local site management -
-![Defender for IoT architecture](./media/architecture/defender-iot-security-architecture.png)
+:::image type="content" source="./media/architecture/defender-iot-security-architecture-v3.png" alt-text="The architecture for Defender for IoT.":::
### Azure Defender for IoT sensors
-Defender for IoT sensors discover and continuously monitor network devices. Sensors collect ICS network traffic using passive (agentless) monitoring on IoT and OT devices.
+Defender for IoT sensors discovers and continuously monitor network devices. Sensors collect ICS network traffic using passive (agentless) monitoring on IoT and OT devices.
Purpose-built for IoT and OT networks, the agentless technology delivers deep visibility into IoT and OT risk within minutes of being connected to the network. It has zero performance impact on the network and network devices due to its non-invasive, Network Traffic Analysis (NTA) approach. Leveraging patented, IoT and OT-aware behavioral analytics and Layer-7 Deep Packet Inspection (DPI), it allows you to analyze beyond traditional signature-based solutions to immediately detect advanced IoT and OT threats (such as fileless malware) based on anomalous or unauthorized activity.
-Defender for IoT sensors connect to a SPAN port or network TAP and immediately begins performing DPI on IoT and OT network traffic.
+Defender for IoT sensors connects to a SPAN port or network TAP and immediately begins performing DPI on IoT and OT network traffic.
Data collection, processing, analysis, and alerting takes place directly on the sensor. This makes it ideally suited for locations with low bandwidth or high latency connectivity, because only metadata is transferred to the management console.
@@ -62,14 +61,13 @@ The protocol violation detection engine identifies the use of packet structures
Using machine learning, the policy violation detection engine alerts users of any deviation from baseline behavior, such as unauthorized use of specific function codes, access to specific objects, or changes to device configuration. For example: DeltaV software version changed, and Unauthorized PLC programming alerts. Specifically, the policy violation engine models the ICS networks as deterministic sequences of states and transitionsΓÇöusing a patented technique called Industrial Finite State Modeling (IFSM). The policy violation detection engine establishes a baseline of the ICS networks, so that the platform requires a shorter learning period to build a baseline of the network than generic mathematical approaches or analytics, which were originally developed for IT rather than OT networks. #### Industrial malware detection engine
-The industrial malware detection engine identifies behaviors that indicate the presence of known malware, such as Conficker, Black Energy, Havex, WannaCry,NotPetya, and Triton.
+The industrial malware detection engine identifies behaviors that indicate the presence of known malware, such as Conficker, Black Energy, Havex, WannaCry, NotPetya, and Triton.
#### Anomaly detection engine
-The anomaly detection engine detects unusual machine-to-machine (M2M) communications and behaviors. By modeling ICS networks as deterministic sequences of states and transitions, the platform requires a shorter learning period than generic mathematical approaches or analytics originally developed for IT rather than OT. It also detects anomalies faster, with minimal false positives. Anomaly detection engine alerts include Excessive SMB sign-in attempts, and PLC Scan Detected alerts.
+The anomaly detection engine detects unusual machine-to-machine (M2M) communications and behaviors. By modeling ICS networks as deterministic sequences of states and transitions, the platform requires a shorter learning period than generic mathematical approaches or analytics originally developed for IT rather than OT. It also detects anomalies faster, with minimal false positives. Anomaly detection engine alerts include Excessive SMB sign in attempts, and PLC Scan Detected alerts.
#### Operational incident detection
-The operational incident detection detects operational issues such as intermittent connectivity that can indicate early signs of equipment failure. For example, the device is suspected to be disconnected (unresponsive), and Siemens S7 stop PLC command was sent alerts.
-
+The operational incident detection detects operational issues such as intermittent connectivity that can indicate early signs of equipment failure. For example, the device is thought to be disconnected (unresponsive), and Siemens S7 stop PLC command was sent alerts.
### Management consoles Managing Azure Defender for IoT across hybrid environments is accomplished via two management portals:
@@ -78,9 +76,9 @@ Managing Azure Defender for IoT across hybrid environments is accomplished via t
- The Azure portal #### Sensor console
-Sensor detections are displayed in the sensor console, where they can be viewed, investigated, and analyzed in a network map, asset inventory, and in an extensive range of reports, for example risk assessment reports, data mining queries and attack vectors. You can also use the console to view and handle threats detected by sensor engines, forward information to third-party systems, manage users, and more.
+Sensor detections are displayed in the sensor console, where they can be viewed, investigated, and analyzed in a network map, asset inventory, and in an extensive range of reports, for example risk assessment reports, data mining queries and attack vectors. You can also use the console to view and handle threats detected by sensor engines, forward information to partner systems, manage users, and more.
-![Defender for IoT sensor console](./media/architecture/sensor-console.png)
+:::image type="content" source="./media/architecture/sensor-console-v2.png" alt-text="Defender for IoT sensor console":::
#### On-premises management console The on-premises management console enables security operations center (SOC) operators to manage and analyze alerts aggregated from multiple sensors into one single dashboard and provides an overall view of the health of the OT networks.
@@ -89,41 +87,47 @@ This architecture provides a comprehensive unified view of the network at a SOC
In addition to multi-tenancy, monitoring, data analysis, and centralized sensor remote control, the management console provides additional system maintenance tools (such as alert exclusion) and fully customized reporting features for each of the remote appliances. This scalable architecture supports both local management at a site level, zone level, and global management within the SOC.
-The management console can be deployed for high-availability configuration, which provides a backup console that periodically receives backups of all configuration files required for recovery. If the master console fails, the local site management appliances will automatically fail over to synchronize with the backup console to maintain availability without interruption.
+The management console can be deployed for high-availability configuration, which provides a backup console that periodically receives backups of all configuration files required for recovery. If the primary console fails, the local site management appliances will automatically fail over to synchronize with the backup console to maintain availability without interruption.
+
+Tightly integrated with your SOC workflows and run books, it enables easy prioritization of mitigation activities and cross-site correlation of threats.
+
+- Holistic - reduce complexity with a single unified platform for asset management, risk and vulnerability management, as well as threat monitoring with incident response.
+
+- Aggregation and correlation ΓÇô display, aggregate, and analyze data and alerts collected from all sites.
+
+- Control all sensors ΓÇô configure and monitor all sensors from a single location.
+
+ :::image type="content" source="media/updates/alerts-and-site-management-v2.png" alt-text="Manage all of your alerts and information.":::
#### Azure portal The Defender for IoT portal in Azure is used to help you:
-┬╖ Purchase solution appliances
-┬╖ Install and update software
-┬╖ Onboard sensors to Azure
-┬╖ Update Threat Intelligence packages
+
+- Purchase solution appliances
+- Install and update software
+- Onboard sensors to Azure
+- Update Threat Intelligence packages
## Embedded security agent: Built-in mode
-In **Built-in** mode, Defender for IoT is enabled when you elect to turn on the **Security** option in your IoT Hub. Offering real-time monitoring, recommendations and alerts, Built-in mode offers single-step device visibility and unmatched security. Build-in mode does not require agent installation on any devices and uses advanced analytics on logged activities to analyze and protect your field device and IoT hub.
+In **Built-in** mode, Defender for IoT is enabled when you elect to turn on the **Security** option in your IoT hub. Offering real-time monitoring, recommendations and alerts, built-in mode offers single-step device visibility and unmatched security. Build-in mode does not require agent installation on any devices and uses advanced analytics on logged activities to analyze and protect your field device and IoT hub.
## Embedded security agent: Enhanced mode
-In **Enhanced** mode, after turning on the **Security** option in your IoT Hub and installing Defender for IoT device agents on your devices, the agents collect, aggregate, and analyze raw security events from your devices. Raw security events can include IP connections, process creation, user logins, and other security-relevant information. Defender for IoT device agents also handle event aggregation to help avoid high network throughput. The agents are highly customizable, allowing you to use them for specific tasks, such as sending only important information at the fastest SLA, or for aggregating extensive security information and context into larger segments, avoiding higher service costs.
+In **Enhanced** mode, after turning on the **Security** option in your IoT hub and installing Defender for IoT device agents on your devices, the agents collect, aggregate, and analyze raw security events from your devices. Raw security events can include IP connections, process creation, user logins, and other security-relevant information. Defender for IoT device agents also handles event aggregation to help avoid high network throughput. The agents are highly customizable, allowing you to use them for specific tasks, such as sending only important information at the fastest SLA, or for aggregating extensive security information and context into larger segments, avoiding higher service costs.
-Device agents, and other applications use the **Azure send security message SDK** to send security information into Azure IoT Hub. IoT Hub gets this information and forwards it to the Defender for IoT service.
+Device agents, and other applications use the **Azure send security message SDK** to send security information into Azure IoT hub. IoT hub gets this information and forwards it to the Defender for IoT service.
-Once the Defender for IoT service is enabled, in addition to the forwarded data, IoT Hub also sends out all of its internal data for analysis by Defender for IoT. This data includes device-cloud operation logs, device identities, and Hub configuration. All of this information helps to create the Defender for IoT analytics pipeline.
+Once the Defender for IoT service is enabled, in addition to the forwarded data, IoT hub also sends out all of its internal data for analysis by Defender for IoT. This data includes device-cloud operation logs, device identities, and hub configuration. All of this information helps to create the Defender for IoT analytics pipeline.
-Defender for IoT analytics pipeline also receives additional threat intelligence streams from various sources within Microsoft and Microsoft partners. The Defender for IoT entire analytics pipeline works with every customer configuration made on the service (such as custom alerts and use of the send security message SDK).
+Defender for IoT analytics pipeline also receives additional threat intelligence streams from various sources within Microsoft and Microsoft partners. The Defender for IoT entire analytics pipeline works with every customer configuration made on the service (such as custom alerts and use of the send security message SDK).
-Using the analytics pipeline, Defender for IoT combines all of the streams of information to generate actionable recommendations and alerts. The pipeline contains both custom rules created by security researchers and experts as well as machine learning models searching for deviation from standard device behavior and risk analysis.
+Using the analytics pipeline, Defender for IoT combines all of the streams of information to generate actionable recommendations and alerts. The pipeline contains both custom rules created by security researchers and experts as well as machine learning models searching for deviation from standard device behavior and risk analysis.
-Defender for IoT recommendations and alerts (analytics pipeline output) is written to the Log Analytics workspace of each customer. Including the raw events in the workspace as well as the alerts and recommendations enables deep dive investigations and queries using the exact details of the suspicious activities detected.
+Defender for IoT recommendations and alerts (analytics pipeline output) is written to the Log Analytics workspace of each customer. Including the raw events in the workspace as well as the alerts and recommendations enables deep dive investigations and queries using the exact details of the suspicious activities detected.
-## Next steps
+## See also
-In this article, you learned about the basic architecture and workflow of Defender for IoT solution. To learn more about prerequisites, how to get started and enable your security solution in IoT Hub, see the following articles:
+[Defender for IoT FAQ](resources-frequently-asked-questions.md)
-- [Service prerequisites](service-prerequisites.md)-- [Getting started](getting-started.md)-- [Configure your solution](quickstart-configure-your-solution.md)-- [Enable security in IoT Hub](quickstart-onboard-iot-hub.md)-- [Defender for IoT FAQ](resources-frequently-asked-questions.md)-- [Defender for IoT security alerts](concept-security-alerts.md)
+[System prerequisites](quickstart-system-prerequisites.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/concept-key-concepts https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/concept-key-concepts.md new file mode 100644
@@ -0,0 +1,128 @@
+---
+title: Key advantages
+description: Learn about basic Defender for IoT concepts.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/13/2020
+ms.topic: article
+ms.service: azure
+---
+
+# Basic concepts
+
+This article describes key advantages of Azure Defender for IoT.
+
+## Rapid non-invasive deployment and passive monitoring
+
+Defender for IoT sensors connect to a SPAN port or network TAP and immediately begin collecting ICS network traffic via passive (agentless) monitoring. Deep packet inspection (DPI) is used to dissect traffic from both serial and Ethernet control network equipment. Defender for IoT has zero impact on OT networks because it isn't placed in the data path and doesn't actively scan OT devices.
+
+To deliver instant snapshots of detailed asset information, Defender for IoT sensor supplements passive monitoring with an optional active component. This component uses safe, vendor-approved commands to query both Windows and controller devices for asset details, as often or as infrequently as you want.
+
+## Embedded knowledge of ICS protocols, devices, and applications
+
+DPI alone is not enough to identify protocol anomalies and identify device at a granular level. The Defender for IoT sensor addresses some of the largest and most complex environments. More than 1,300 OT networks have been analyzed to date, across all industrial sectors.
+
+## Analytics and self-learning engines
+
+Engines identify security issues via continuous monitoring and five analytics engines that incorporate self-learning to eliminate the need for updating signatures or defining rules. The engines use ICS-specific behavioral analytics and data science to continuously analyze OT network traffic for anomalies. The five engines are:
+
+- **Protocol violation detection**: Identifies the use of packet structures and field values that violate ICS protocol specifications.
+
+- **Policy violation detection**: Identifies policy violations such as unauthorized use of function codes, access to specific objects, or changes to asset configuration.
+
+- **Industrial malware detection**: Identifies behaviors that indicate the presence of known malware such as Conficker, Black Energy, Havex, WannaCry, and NotPetya.
+
+- **Anomaly detection**: Detects unusual machine-to-machine (M2M) communications and behaviors. By modeling ICS networks as deterministic sequences of states and transitions, the engine uses a patented technique called Industrial Finite State Modeling (IFSM). The solution requires a shorter learning period than generic mathematical approaches or analytics, which were originally developed for IT rather than OT. It also detects anomalies faster, with minimal false positives.
+
+- **Operational incident detection**: Identifies operational issues such as intermittent connectivity that can indicate early signs of equipment failure.
+
+## Network Traffic Analysis for risk and vulnerability assessment
+
+Unique in the industry, Defender for IoT uses proprietary Network Traffic Analysis (NTA) algorithms to passively identify all network and endpoint vulnerabilities, such as:
+
+- Unauthorized remote access connections
+- Rogue or undocumented devices
+- Weak authentication
+- Vulnerable devices (based on unpatched CVEs)
+- Unauthorized bridges between subnets
+- Weak firewall rules
+
+## Data mining for investigations, forensics, and threat hunting
+
+The platform provides an intuitive data-mining interface for granular searching of historical traffic across all relevant dimensions. Examples include time period, IP address, MAC address, and ports. You can also make protocol-specific queries based on function codes, protocol services, and modules. Full-fidelity PCAPs are available for further drill-down analysis.
+
+## Sensor Cloud Management mode
+
+The Sensor Cloud Management mode determines where device, alert, and other information that the sensor detects is displayed.
+
+For **cloud-connected sensors**, information that the sensor detects is displayed in the sensor console. Alert information is delivered through an IoT hub and can be shared with other Azure services, such as Azure Sentinel.
+
+For **locally connected sensors**, information that the sensor detects is displayed in the sensor console. Detection information is also shared with the on-premises management console if the sensor is connected to it.
+
+## Air-gapped networks
+
+If you're working in an air-gapped environment, the on-premises management console in Defender for IoT delivers a real-time view of key IoT and OT risk indicators and alerts across all of your facilities. Tightly integrated with your SOC workflows and runbooks, it enables easy prioritization of mitigation activities and cross-site correlation of threats.
+
+Defender for IoT provides a consolidated view of all your devices. It also provides critical information about the devices, such as type (PLC, RTU, DCS, and more), manufacturer, model, and firmware revision level, as well as alert information.
+
+Defender for IoT enables the effective management of multiple deployments and a comprehensive unified view of the network. Defender for IoT optimizes alert handling and control of operational network security.
+
+The on-premises management console is a web-based administrative platform that lets you monitor and control the activities of global sensor installations. In addition to managing the data received from deployed sensors, the on-premises management console seamlessly integrates data from a variety of business resources: CMDBs, DNS, firewalls, Web APIs, and more.
+
+:::image type="content" source="media/concept-air-gapped-networks/site-management-alert-screen.png" alt-text="On-premises management console display.":::
+
+We recommend that you familiarize yourself with the concepts, capabilities, and features available to sensors before working with the on-premises management console.
+
+## Integrations
+
+You can expand the capabilities of Defender for IoT by sharing both device and alert information with partner systems. Integrations help enterprises bridge previously siloed security solutions to significantly enhance device visibility and threat intelligence. Integrations also help enterprises accelerate the system-wide responses and mitigate risks faster.
+
+Integrations reduce complexity and eliminate IT and OT silos by integrating them into your existing SOC workflows and security stack. For example:
+
+- SIEMs such as IBM QRadar, Splunk, ArcSight, LogRhythm, and RSA NetWitness
+
+- Security orchestration and ticketing systems such as ServiceNow and IBM Resilient
+
+- Secure remote access solutions such as CyberArk Privileged Session Manager (PSM) and BeyondTrust
+
+- Secure network access control (NAC) systems such as Aruba ClearPass and Forescout CounterACT
+
+- Firewalls such as Fortinet and Check Point
+
+## Complete protocol support
+
+In addition to embedded protocol support, you can secure IoT and ICS devices running proprietary and custom protocols, or protocols that deviate from any standard. By using the Horizon Open Development Environment (ODE) SDK, developers can create dissector plug-ins that decode network traffic based on defined protocols. Services analyze traffic to provide complete monitoring, alerting, and reporting. Use Horizon to:
+
+- Expand visibility and control without the need to upgrade to new versions.
+
+- Secure proprietary information by developing on-site as an external plug-in.
+
+- Localize text for alerts, events, and protocol parameters.
+
+In addition, you can use proprietary protocol alerts to communicate information:
+
+- About traffic detections based on protocols and underlying protocols in a proprietary Horizon plug-in.
+
+- About a combination of protocol fields from all protocol layers. For example, in an environment running MODBUS, you might want to generate an alert when the sensor detects a write command to a memory register on a specific IP address and Ethernet destination. Or you might want to generate an alert when any access is performed to a specific IP address.
+
+Alerts are triggered when Horizon alert rule conditions are met.
+
+In addition, working with Horizon custom alerts lets you write your own alert titles and messages. Resolved protocol fields and values can be embedded in the alert message text.
+
+Using custom, condition-based alert triggering and messaging helps pinpoint specific network activity and effectively update your security, IT, and operational teams.
++
+## High availability
+
+Increase the resilience of your Defender for IoT deployment by installing a high-availability appliance in the on-premises management console. High-availability deployments ensure that your managed sensors continuously report to an active on-premises management console.
+
+This deployment is implemented with an on-premises management console pair that includes a primary and secondary appliance.
+
+## Localization
+
+Many console features support an extensive range of languages.
+
+## Next step
+
+[Getting started with Defender for IoT](getting-started.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/concept-pricing https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/concept-pricing.md
@@ -4,7 +4,7 @@ description: Learn about the costs associated with Defender for IoT, and how to
services: defender-for-iot ms.service: defender-for-iot documentationcenter: na
-author: mlottner
+author: shhazam-ms
manager: rkarlin editor: ''
@@ -12,8 +12,8 @@ ms.devlang: na
ms.topic: conceptual ms.tgt_pltfrm: na ms.workload: na
-ms.date: 09/04/2020
-ms.author: mlottner
+ms.date: 12/08/2020
+ms.author: shhazam
--- # Pricing and associated costs
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/event-aggregation https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/event-aggregation.md
@@ -93,6 +93,6 @@ To continue getting started with Defender for IoT deployment, use the following
- Understand [Security agent authentication methods](concept-security-agent-authentication-methods.md) - Select and deploy a [security agent](how-to-deploy-agent.md)-- Review Defender for IoT [service prerequisites](service-prerequisites.md)
+- Review Defender for IoT [System prerequisites](quickstart-system-prerequisites.md)
- Learn how to [Enable Defender for IoT service in your IoT Hub](quickstart-onboard-iot-hub.md) - Learn more about the service from the [Defender for IoT FAQ](resources-frequently-asked-questions.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/getting-started https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/getting-started.md
@@ -1,69 +1,129 @@
---
-title: Deployment options
-description: Get started in understanding the basic workflow of Defender for IoT features and service.
+title: Getting started
+description: Get started with understanding the basic workflow for Defender for IoT deployment.
services: defender-for-iot ms.service: defender-for-iot documentationcenter: na
-author: mlottner
+author: shhazam-ms
manager: rkarlin editor: ''-- ms.devlang: na ms.topic: conceptual ms.tgt_pltfrm: na ms.workload: na
-ms.date: 09/09/2020
-ms.author: mlottner
+ms.date: 12/26/2020
+ms.author: shhazam
---
-# Getting started with Azure Defender for IoT
+# Getting started with Defender for IoT
+
+This article provides an overview of the steps you'll take to set up Azure Defender for IoT. The process requires that you:
+
+- Register your subscription and sensors on the Azure Defender for IoT portal.
+- Install the sensor and on-premises management console software.
+- Perform initial activation of the sensor and management console.
+
+## Permission requirements
+
+Some of the setup steps require specific user permissions.
+
+Administrative user permissions are required to activate the sensor and management console, upload SSL/TLS certificates, and generate new passwords.
+
+The following table describes user access permissions to Azure Defender for IoT portal tools:
+
+| Permission | Security reader | Security administrator | Subscription contributor | Subscription owner |
+|--|--|--|--|--|
+| View all Defender for IoT screens and data | Γ£ô | Γ£ô | Γ£ô | Γ£ô |
+| Onboard a sensor | | Γ£ô | Γ£ô | Γ£ô |
+| Update pricing | | Γ£ô | Γ£ô | Γ£ô |
+| Recover password | Γ£ô | Γ£ô | Γ£ô | Γ£ô |
+
+## 1. Identify the solution infrastructure
+
+**Clarify your network setup needs**
+
+Research your network architecture, monitored bandwidth, and other network details. For more information, see [About Azure Defender for IoT network setup](how-to-set-up-your-network.md).
+
+**Clarify which sensors and management console appliances are required to handle the network load**
+
+Azure Defender for IoT supports both physical and virtual deployments. For the physical deployments, you can purchase various certified appliances. For more information, see [Identify required appliances](how-to-identify-required-appliances.md).
+
+We recommend that you calculate the approximate number of devices that will be monitored. Later, when you register your Azure subscription to the portal, you'll be asked to enter this number. Numbers can be added in intervals of 1,000 seconds. The numbers of monitored devices are called *committed devices*.
+
+## 2. Register with Azure Defender for IoT
+
+Registration includes:
+
+- Onboarding your Azure subscriptions to Defender for IoT.
+- Defining committed devices.
+- Downloading an activation file for the on-premises management console.
+
+To register:
+
+1. Go to the Azure Defender for IoT portal.
+1. Select **Onboard subscription**.
+1. On the **Pricing** page, select a subscription or create a new one, and add the number of committed devices.
+1. Select the **Download the on-premises management console** tab and save the downloaded activation file. This file contains the aggregate committed devices that you defined. The file will be uploaded to the management console after initial sign-in.
+
+## 3. Install and set up the on-premises management console
+
+After you acquire your on-premises management console appliance:
+
+- Download the ISO package from the Azure Defender for IoT portal.
+- Install the software.
+- Activate and carry out initial management console setup.
+
+To install and set up:
-This article describes the deployment and onboarding processes necessary to get Azure Defender for IoT running. Additional steps are also required. It is recommended that you understand these steps and familiarize yourself with information in accompanying documents.
+1. Select **Getting Started** from the Defender for IoT portal.
+1. Select the **On-premises management console** tab.
+1. Choose a version and select **Download**.
+1. Install the on-premises management console software. For more information, see [Defender for IoT installation](how-to-install-software.md).
+1. Activate and set up the management console. For more information, see [Activate and set up your on-premises management console](how-to-activate-and-set-up-your-on-premises-management-console.md).
-Once you complete all the steps, Azure Defender for IoT sensors will monitor your network. Depending on how you set up your solution, detections can also be sent to the on-premises management console, or to the IoT Hub.
+## 4. Onboard a sensor
-Complete the following steps to get Azure Defender for IoT up and running.
+Onboard a sensor by registering it with Azure Defender for IoT and downloading a sensor activation file:
-## 1. Set up Azure
+1. Define a sensor name and associate it with a subscription.
+1. Choose a sensor management mode:
-- Set up an Azure Account. For more information, see [Create an Azure account](/learn/modules/create-an-azure-account/).
+ - **Cloud connected sensors**: Information that sensors detect is displayed in the sensor console. In addition, alert information is delivered through an IoT hub and can be shared with other Azure services, such as Azure Sentinel.
-- Firewall or proxy: If you have a firewall or similar intervening network device that is configured to allow specific connections verify that either *.azure-devices.net:443 is opened to the firewall or proxy. If wildcards are not supported or you want more control, the specific IoT Hub FQDN should be opened in your FW or proxy. For more information, see [Reference - IoT Hub endpoints](../iot-hub/iot-hub-devguide-endpoints.md).
+ - **Locally managed sensors**: Information that sensors detect is displayed in the sensor console. If you're working in an air-gapped network and want a unified view of all information detected by multiple locally managed sensors, work with the on-premises management console.
-## 2. Deploy hardware, software, and onboard to sensor
+1. Download a sensor activation file.
-- Purchase sensor hardware and install software. Follow the steps outlined here, and for more information, see this article and the [Defender for IoT Hardware Guide](https://aka.ms/AzureDefenderforIoTBareMetalAppliance) and the [Installation Guide](https://aka.ms/AzureDefenderforIoTInstallSensorISO).
+For more information, see [Onboard and manage sensors in the Defender for IoT portal](how-to-manage-sensors-on-the-cloud.md).
- - After you install your sensor, securely record the sensor sign-in credentials. You'll need the credentials to upload the activation file to the sensor.
+## 5. Install and set up the sensor
- - If you are working with sensors that are locally managed, securely record the IP address of the sensor or the sensor name defined in the installation. You may want to use it when creating a sensor name during sensor registration in the Defender for IoT portal. You can use them later to ensure easier tracking and consistent naming between the registration name in the Azure Defender for IoT portal and the IP address of the deployed sensor displayed in the sensor console.
+Download the ISO package from the Azure Defender for IoT portal, install the software, and set up the sensor.
-- Register the sensor with the Defender for IoT portal and download a sensor activation file.
+1. Select **Getting Started** from the Defender for IoT portal.
+1. Select **Set up sensor**.
+1. Choose a version and select **Download**.
+1. Install the sensor software. For more information, see [Defender for IoT installation](how-to-install-software.md).
+1. Activate and set up your sensor. For more information, see [Sign in and activate a sensor](how-to-activate-and-set-up-your-sensor.md).
-- Upload the activation file to your sensor.
+## 6. Connect sensors to an on-premises management console
-## 3. Perform network setup for sensor monitoring and management
+Connect sensors to the management console to ensure that:
-- Connect your sensor to the network. Described in the [Network setup guide](https://aka.ms/AzureDefenderForIoTNetworkSetup).
+- Sensors send alert and device inventory information to the on-premises management console.
-## 4. Start discovering your network
+- The on-premises management console can perform sensor backups, manage alerts that sensors detect, investigate sensor disconnections, and carry out other activity on connected sensors.
-- Tweak system settings in the sensor console.
+We recommend that you group multiple sensors monitoring the same networks in one zone. Doing this will coalesce information collected by multiple sensors.
-- Connect sensors to an on-premises management console.
+For more information, see [Connect sensors to the on-premises management console](how-to-activate-and-set-up-your-on-premises-management-console.md#connect-sensors-to-the-on-premises-management-console).
-For more information, see the [Azure Defender for IoT Sensor User Guide](https://aka.ms/AzureDefenderforIoTUserGuide) and the [Defender for IoT on-premises management console user's guide](https://aka.ms/DefenderForIoTManagementConsole).
+## 7. Populate Azure Sentinel with alert information (optional)
-## 5. Populate Azure Sentinel with alert information
+Send alert information to Azure Sentinel by configuring Azure Sentinel. See [Connect your data from Defender for IoT to Azure Sentinel](how-to-configure-with-sentinel.md).
-- To send alert information to Azure Sentinel, configure Azure Sentinel: [Connect your data from Defender for IoT to Azure Sentinel](how-to-configure-with-sentinel.md).
-
+## See also
-## Next steps
+- [Welcome to Azure Defender for IoT](overview.md)
-- Enable [Defender for IoT](quickstart-onboard-iot-hub.md)-- Configure your [solution](quickstart-configure-your-solution.md)-- [Create security modules](quickstart-create-security-twin.md)-- Configure [custom alerts](quickstart-create-custom-alerts.md)-- [Deploy a security agent](how-to-deploy-agent.md)\ No newline at end of file
+- [Azure Defender for IoT architecture](architecture.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-accelerate-alert-incident-response https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-accelerate-alert-incident-response.md new file mode 100644
@@ -0,0 +1,125 @@
+---
+title: Accelerate alert workflows
+description: Improve alert and incident workflows.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/02/2020
+ms.service: azure
+ms.topic: how-to
+---
++
+# Accelerate alert workflows
+
+This article describes how to accelerate alert workflows by using alert comments, alert groups, and custom alert rules in Azure Defender for IoT. These tools help you:
+
+- Analyze and manage the large volume of alert events detected in your network.
+
+- Pinpoint and handle specific network activity.
+
+## Accelerate incident workflows by using alert comments
+
+Work with alert comments to improve communication between individuals and teams during the investigation of an alert event.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/suspicion-of malicious-activity-screen.png" alt-text="Screenshot that shows malicious activity.":::
+
+Use alert comments to improve:
+
+- **Workflow steps**: Provide alert mitigation steps.
+
+- **Workflow follow-up**: Notify that steps were taken.
+
+- **Workflow guidance**: Provide recommendations, insights, or warnings about the event.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/alert-comment-screen.png" alt-text="Screenshot that shows alert comments.":::
+
+The list of available options appears in each alert. Users can select one or several messages.
+
+To add alert comments:
+
+1. On the side menu, select **System Settings**.
+
+2. In the **System Setting** window, select **Alert Comments**.
+
+3. In the **Add comments** box, enter the comment text. Use up to 50 characters. Commas are not permissible.
+
+4. Select **Add**.
+
+## Accelerate incident workflows by using alert groups
+
+Alert groups let SOC teams view and filter alerts in their SIEM solutions and then manage these alerts based on enterprise security policies and business priorities. For example, alerts about new detections are organized in a discovery group. This group includes alerts that deal with the detection of new devices, new VLANs, new user accounts, new MAC addresses, and more.
+
+Alert groups are applied when you create forwarding rules for the following partner solutions:
+
+ - Syslog servers
+
+ - QRadar
+
+ - ArcSight
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/create-forwarding-rule.png" alt-text="Screenshot of creating a forwarding rule.":::
+
+The relevant alert group appears in partner output solutions.
+
+### Requirements
+
+The alert group will appear in supported partner solutions with the following prefixes:
+
+ - **cat** for QRadar, ArcSight, Syslog CEF, Syslog LEEF
+
+ - **Alert Group** for Syslog text messages
+
+ - **alert_group** for Syslog objects
+
+These fields should be configured in the partner solution to display the alert group name. If there is no alert associated with an alert group, the field in the partner solution will display **NA**.
+
+### Default alert groups
+
+The following alert groups are automatically defined:
+| | | |
+|--|--|--|
+| Abnormal communication behavior | Custom alerts | Remote access |
+| Abnormal HTTP communication behavior | Discovery | Restart and stop commands |
+| Authentication | Firmware change | Scan |
+| Unauthorized communication behavior | Illegal commands | Sensor traffic |
+| Bandwidth anomalies | Internet access | Suspicion of malware |
+| Buffer overflow | Operation failures | Suspicion of malicious activity |
+| Command failures | Operational issues | |
+| Configuration changes | Programming | |
+
+Alert groups are predefined. For details about alerts associated with alert groups, and about creating custom alert groups, contact [Microsoft Support](https://support.microsoft.com/supportforbusiness/productselection?sapId=82c88f35-1b8e-f274-ec11-c6efdd6dd099).
+
+## Customize alert rules
+
+You can add custom alert rules based on information that individual sensors detect. For example, define a rule that instructs a sensor to trigger an alert based on a source IP, destination IP, or command (within a protocol). When the sensor detects the traffic defined in the rule, an alert or event is generated.
+
+The alert message indicates that a user-defined rule triggered the alert.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/customized-alerts-screen.png" alt-text="Screenshot that shows customized alerts.":::
+
+To create a custom alert rule:
+
+1. Select **Custom Alerts** from the side menu of a sensor.
+1. Select the plus sign (**+**) to create a rule.
+
+ :::image type="content" source="media/how-to-work-with-alerts-sensor/user-defined-rule.png" alt-text="Screenshot that shows a user-defined rule.":::
+
+1. Define a rule name.
+1. Select a category or protocol from the **Categories** pane.
+1. Define a specific source and destination IP or MAC address, or choose any address.
+1. Add a condition. A list of conditions and their properties is unique for each category. You can select more than one condition for each alert.
+1. Indicate if the rule triggers an **Alarm** or **Event**.
+1. Assign a severity level to the alert.
+1. Indicate if the alert will include a PCAP file.
+1. Select **Save**.
+
+The rule is added to the **Customized Alerts Rules** list, where you can review basic rule parameters, the last time the rule was triggered, and more. You can also enable and disable the rule from the list.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/customized-alerts-screen.png" alt-text="Screenshot of a user-added customized rule.":::
+
+### See also
+
+[View information provided in alerts](how-to-view-information-provided-in-alerts.md)
+
+[Manage the alert event](how-to-manage-the-alert-event.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-activate-and-set-up-your-on-premises-management-console https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-activate-and-set-up-your-on-premises-management-console.md new file mode 100644
@@ -0,0 +1,314 @@
+---
+title: Activate and set up your on-premises management console
+description: Management console activation and setup ensures that sensors are registered with Azure and send information to the on-premises management console, and that the on-premises management console carries out management tasks on connected sensors.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/24/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Activate and set up your on-premises management console
+
+Activation and setup of the on-premises management console ensures that:
+
+- Network devices that you're monitoring through connected sensors are registered with an Azure account.
+
+- Sensors send information to the on-premises management console.
+
+- The on-premises management console carries out management tasks on connected sensors.
+
+- You have installed an SSL certificate.
+
+## Sign in for the first time
+
+To sign in to the management console:
+
+- Open a web browser and enter the IP address and password that you received for the on-premises management console during the system installation. If you forgot your password, select **Recover Password** and see [Password recovery](how-to-manage-the-on-premises-management-console.md#password-recovery).
+
+## Upload an activation file
+
+After first-time sign-in, activate the on-premises management console by downloading an activation file from the **Pricing** page of the Azure Defender for IoT portal. This file contains the aggregate committed devices defined during the onboarding process. **Committed devices** indicates the number of devices that Defender for IoT will monitor per subscription.
+
+To upload an activation file:
+
+1. Go to the Defender for IoT **Pricing** page.
+1. Select the **Download the activation file for the management console** tab. The activation file is downloaded.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/cloud_download_opm_activation_file.png" alt-text="Download the activation file.":::
+
+1. Select **System Settings** from the management console.
+1. Select **Activation**.
+1. Select **Choose a File** and select the file that you saved.
+
+After initial activation, the number of monitored devices might exceed the number of committed devices defined during onboarding. This might happen, for example, if you connect more sensors to the management console. If there's a discrepancy between the number of monitored devices and the number of committed devices, a warning appears in the management console. If this happens, you should upload a new activation file.
+
+## Set up a certificate
+
+Following installation of the management console, a local self-signed certificate is generated and used to access the console. After an administrator signs in to the management console for the first time, that user is prompted to onboard an SSL/TLS certificate. We recommend that you work with a trusted CA-signed certificate and not use the locally generated self-signed certificate.
+
+Two levels of security are available:
+
+- Meet specific certificate and encryption requirements requested by your organization by uploading the CA-signed certificate.
+- Allow validation between the management console and connected sensors. Validation is evaluated against a certificate revocation list and the certificate expiration date. *If validation fails, communication between the management console and the sensor is halted and a validation error is presented in the console.* This option is enabled by default after installation.
+
+The console supports the following types of certificates:
+
+- Private and Enterprise Key Infrastructure (private PKI)
+- Public Key Infrastructure (public PKI)
+- Locally generated on the appliance (locally self-signed)
+
+ > [!IMPORTANT]
+ > We recommend that you don't use a self-signed certificate. The certificate is not secure and should be used for test environments only. The owner of the certificate can't be validated, and the security of your system can't be maintained. Never use this option for production networks.
+
+To upload a certificate:
+
+1. When you're prompted after sign-in, define a certificate name.
+1. Upload the CRT and key files.
+1. Enter a passphrase and upload a PEM file if required.
+
+You might need to refresh your screen after you upload the CA-signed certificate.
+
+To disable validation between the management console and connected sensors:
+
+1. Select **Next**.
+1. Turn off the **Enable system-wide validation** toggle.
+
+For information about uploading a new certificate, supported certificate files, and related items, see [Manage the on-premises management console](how-to-manage-the-on-premises-management-console.md).
+
+## Connect sensors to the on-premises management console
+
+You must ensure that sensors send information to the on-premises management console, and that the on-premises management console can perform backups, manage alerts, and carry out other activity on the sensors. To do that, use the following procedures to verify that you make an initial connection between sensors and the on-premises management console.
+
+Two options are available for connecting Azure Defender for IoT sensors to the on-premises management console:
+
+- Connect from the sensor console
+
+- Connect by using tunneling
+
+After connecting, you must set up a site with these sensors.
+
+### Connect sensors from the sensor console
+
+To connect specific sensors to the on-premises management console from the sensor console:
+
+1. On the left pane of the sensor console, select **System Settings**.
+
+2. Select **Connection to Management**.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/connection-status-window-not-connected.png" alt-text="Screenshot of the status window of an on-premises management console, showing Unconnected.":::
+
+3. In the **Address** text box, enter the IP address of the on-premises management console to which you want to connect.
+
+4. Select **Connect**. The status changes:
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/connection-status-window-connected.png" alt-text="Screenshot of the status window of an on-premises management console, showing Connected.":::
+
+### Connect sensors by using tunneling
+
+Enable a secured tunneling connection between organizational sensors and the on-premises management console. This setup circumvents interaction with the organizational firewall, and as a result reduces the attack surface.
+
+Using tunneling allows you to connect to the on-premises management console from its IP address and a single port (that is, 9000) to any sensor.
+
+To set up tunneling at the on-premises management console:
+
+- Sign in to the on-premises management console and run the following commands:
+
+ ```bash
+ cyberx-management-tunnel-enable
+ service apache2 reload
+ sudo cyberx-management-tunnel-add-xsense --xsenseuid <sensorIPAddress> --xsenseport 9000
+ service apache2 reload
+ ```
+
+To set up tunneling on the sensor:
+
+1. Open TCP port 9000 on the sensor (network.properties) manually. If the port is not open, the sensor will reject the connection from the on-premises management console.
+
+2. Sign in to each sensor and run the following commands:
+
+ ```bash
+ sudo cyberx-xsense-management-connect -ip <centralmanagerIPAddress>
+ sudo cyberx-xsense-management-tunnel
+ sudo vi /var/cyberx/properties/network.properties
+ opened_tcp_incoming_ports=22,80,443,102,9000
+ sudo cyberx-xsense-network-validation
+ sudo /etc/network/if-up.d/iptables-recover
+ sudo iptables -nvL
+ ```
+
+## Set up a site
+
+The default enterprise map provides an overall view of your assets according to several levels of geographical locations.
+
+The view of your assets might be required where the organizational structure and user permissions are complex. In these cases, site setup might be determined by a global organizational structure, in addition to the standard site or zone structure.
+
+To support this environment, you need to create a global business topology that's based on your organization's business units, regions, sites, and zones. You also need to define user access permissions around these entities by using access groups.
+
+Access groups enable better control over where users manage and analyze assets in the Defender for IoT platform.
+
+### How it works
+
+For each site, you can define a business unit and a region. Then you can add zones, which are logical entities in your network.
+
+For each zone, you should assign at least one sensor. The five-level model provides the flexibility and granularity required to deliver the protection system that reflects the structure of your organization.
+
+You can edit your sites directly from any of the map views. When you're opening a site from a map view, the number of open alerts appears next to each zone.
+
+:::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/console-map-with-data-overlay-v2.png" alt-text="Screenshot of an on-premises management console map with Berlin data overlay.":::
+
+:::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/diagram-of-sensor-showing-relationships.png" alt-text="Diagram showing sensors and regional relationship.":::
+
+To set up a site:
+
+1. Add new business units to reflect your organization's logical structure.
+
+2. Add new regions to reflect your organization's regions.
+
+3. Add a site.
+
+4. Add zones to a site.
+
+5. Connect the sensors.
+
+6. Assign sensor to site zones.
+
+To add business units:
+
+1. From the Enterprise view, select **All Sites** > **Manage Business Units**.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/manage-business-unit-screen.png" alt-text="Screenshot showing the Manage Business Units view.":::
+
+2. Enter the new business unit name and select **ADD**.
+
+To add a new region:
+
+1. From the Enterprise view, select **All Regions** > **Manage Regions**.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/manage-regions-screen.png" alt-text="Screenshot showing the Manage Regions view.":::
+
+2. Enter the new region name and select **ADD**.
+
+To add a new site:
+
+1. From the Enterprise view, select :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/new-site-icon.png" border="false"::: on the top bar. Your cursor appears as a plus sign (**+**).
+
+2. Position the **+** at the location of the new site and select it. The **Create New Site** dialog box opens.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/create-new-site-screen.png" alt-text="Screenshot of the Create New Site view.":::
+
+3. Define the name and the physical address for the new site and select **SAVE**. The new site appears on the site map.
+
+To delete a site:
+
+1. In the **Site Management** window, select :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/expand-view-icon.png" border="false"::: from the bar that contains the site name, and then select **Delete Site**. The confirmation box appears, verifying that you want to delete the site.
+
+2. In the confirmation box, select **YES**. The confirmation box closes, and the **Site Management** window appears without the site that you've deleted.
+
+## Create enterprise zones
+
+Zones are logical entities that enable you to divide assets within a site into groups according to various characteristics. For example, you can create groups for production lines, substations, site areas, or types of assets. You can define zones based on any characteristic that's suitable for your organization.
+
+You configure zones as a part of the site configuration process.
+
+:::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/site-management-zones-screen-v2.png" alt-text="Screenshot of the Site Management Zones view.":::
+
+The following table describes the parameters in the **Site Management** window.
+
+| Parameter | Description |
+|--|--|
+| Name | The name of the sensor. You can change this name only from the sensor. For more information, see the Defender for IoT user guide. |
+| IP | The sensor IP address. |
+| Version | The sensor version. |
+| Connectivity | The sensor connectivity status. The status can be **Connected** or **Disconnected**. |
+| Last Upgrade | The date of the last upgrade. |
+| Upgrade Progress | The progress bar shows the status of the upgrade process, as follows:<br />- Uploading package<br />- Preparing to install<br />- Stopping processes<br />- Backing up data<br />- Taking snapshot<br />- Updating configuration<br />- Updating dependencies<br />- Updating libraries<br />- Patching databases<br />- Starting processes<br />- Validating system sanity<br />- Validation succeeded<br />- Success<br />- Failure<br />- Upgrade started<br />- Starting installation<br /></br >For details about upgrading, refer to [Microsoft Support](https://support.microsoft.com/) for help. |
+| Assets | The number of OT assets that the sensor monitors. |
+| Alerts | The number of alerts on the sensor. |
+| :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/assign-icon.png" border="false"::: | Enables assigning a sensor to zones. |
+| :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/delete-icon.png" border="false":::| Enables deleting a disconnected sensor from the site. |
+| :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/sensor-icon.png" border="false"::: | Indicates how many sensors are currently connected to the zone. |
+| :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/ot-assets-icon.png" border="false"::: | Indicates how many OT assets are currently connected to the zone. |
+| :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/number-of-alerts-icon.png" border="false"::: | Indicates the number of alerts sent by sensors that are assigned to the zone. |
+| :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/unassign-sensor-icon.png" border="false"::: | Unassigns sensors from zones. |
+
+To add a zone to a site:
+
+1. In the **Site Management** window, select :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/expand-view-icon.png" border="false"::: from the bar that contains the site name, and then select **Add Zone**. The **Create New Zone** dialog box appears.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/create-new-zone-screen.png" alt-text="Screenshot of the Create New Zone view.":::
+
+2. Enter the zone name.
+
+3. Enter a description for the new zone that clearly states the characteristics that you used to divide the site into zones.
+
+4. Select **SAVE**. The new zone appears in the **Site Management** window under the site that this zone belongs to.
+
+To edit a zone:
+
+1. In the **Site Management** window, select :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/expand-view-icon.png" border="false"::: from the bar that contains the zone name, and then select **Edit Zone**. The **Edit Zone** dialog box appears.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/zone-edit-screen.png" alt-text="Screenshot that shows the Edit Zone dialog box.":::
+
+2. Edit the zone parameters and select **SAVE**.
+
+To delete a zone:
+
+1. In the **Site Management** window, select :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/expand-view-icon.png" border="false"::: from the bar that contains the zone name, and then select **Delete Zone**.
+
+2. In the confirmation box, select **YES**.
+
+To filter according to the connectivity status:
+
+- From the upper-left corner, select :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/down-pointing-icon.png" border="false"::: next to **Connectivity**, and then select one of the following options:
+
+ - **All**: Presents all the sensors that report to this on-premises management console.
+
+ - **Connected**: Presents only connected sensors.
+
+ - **Disconnected**: Presents only disconnected sensors.
+
+To filter according to the upgrade status:
+
+- From the upper-left corner, select :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/down-pointing-icon.png" border="false"::: next to **Upgrade Status** and select one of the following options:
+
+ - **All**: Presents all the sensors that report to this on-premises management console.
+
+ - **Valid**: Presents sensors with a valid upgrade status.
+
+ - **In Progress**: Presents sensors that are in the process of upgrade.
+
+ - **Failed**: Presents sensors whose upgrade process has failed.
+
+## Assign sensors to zones
+
+For each zone, you need to assign sensors that perform local traffic analysis and alerting. You can assign only the sensors that are connected to the on-premises management console.
+
+To assign a sensor:
+
+1. Select **Site Management**. The unassigned sensors appear in the upper-left corner of the dialog box.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/unassigned-sensors-view.png" alt-text="Screenshot of the Unassigned Sensors view.":::
+
+2. Verify that the **Connectivity** status is connected. If not, see [Connect sensors to the on-premises management console](#connect-sensors-to-the-on-premises-management-console) for details about connecting.
+
+3. Select :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/assign-icon.png" border="false"::: for the sensor that you want to assign.
+
+4. In the **Assign Sensor** dialog box, select the business unit, region, site, and zone to assign.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-on-premises-management-console/assign-sensor-screen.png" alt-text="Screenshot of the Assign Sensor view.":::
+
+5. Select **ASSIGN**.
+
+To unassign and delete a sensor:
+
+1. Disconnect the sensor from the on-premises management console. See [Connect sensors to the on-premises management console](#connect-sensors-to-the-on-premises-management-console) for details.
+
+2. In the **Site Management** window, select the sensor and select :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/unassign-sensor-icon.png" border="false":::. The sensor appears in the list of unassigned sensors after a few moments.
+
+3. To delete the unassigned sensor from the site, select the sensor from the list of unassigned sensors and select :::image type="icon" source="media/how-to-activate-and-set-up-your-on-premises-management-console/delete-icon.png" border="false":::.
+
+## See also
+
+[Troubleshoot the sensor and on-premises management console](how-to-troubleshoot-the-sensor-and-on-premises-management-console.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-activate-and-set-up-your-sensor https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-activate-and-set-up-your-sensor.md new file mode 100644
@@ -0,0 +1,216 @@
+---
+title: Activate and set up your sensor
+description: This article describes how to sign in and activate a sensor console.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/26/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Activate and set up your sensor
+
+This article describes how to activate a sensor and perform initial setup.
+
+Administrator users carry out activation when signing in for the first time and when activation management is required. Setup ensures that the sensor is configured to optimally detect and alert.
+
+Security analysts and read-only users can't activate a sensor or generate a new password.
+
+## Sign-in and activation for administrator users
+
+Administrators who sign in for the first time should verify that they have access to activation and password recovery files that were downloaded during sensor onboarding. If not, they need Azure security administrator, subscription contributor, or subscription owner permissions to generate these files on the Azure Defender for IoT portal.
+
+### First-time sign-in and activation checklist
+
+Before signing in to the sensor console, administrator users should have access to:
+
+- The sensor IP address that was defined during the installation.
+
+- User sign-in credentials for the sensor. If you downloaded an ISO for the sensor, use the default credentials that you received during the installation. We recommend that you create a new *Administrator* user after activation.
+
+- An initial password. If you purchased a preconfigured sensor from Arrow, you need to generate a password when signing in for the first time.
+
+- The activation file associated with this sensor. The file was generated and downloaded during sensor onboarding on the Defender for IoT portal.
+
+- An SSL/TLS CA-signed certificate that your company requires.
+
+### About activation files
+
+Your sensor was onboarded to Azure Defender for IoT in a specific management mode:
+
+| Mode type | Description |
+|--|--|
+| **Cloud connected mode** | Information that the sensor detects is displayed in the sensor console. Alert information is also delivered through the IoT hub and can be shared with other Azure services, such as Azure Sentinel. |
+| **Locally connected mode** | Information that the sensor detects is displayed in the sensor console. Detection information is also shared with the on-premises management console, if the sensor is connected to it. |
+
+A locally connected or cloud-connected activation file was generated and downloaded for this sensor during onboarding. The activation file contains instructions for the management mode of the sensor. *A unique activation file should be uploaded to each sensor you deploy.* The first time you sign in, you need to upload the relevant activation file for this sensor.
+
+:::image type="content" source="media/how-to-activate-and-set-up-your-sensor/azure-defender-for-iot-activation-file-download-button.png" alt-text="Azure Defender for IoT portal, onboard sensor.":::
+
+### About certificates
+
+Following sensor installation, a local self-signed certificate is generated and used to access the sensor console. After an administrator signs in to the console for the first time, that user is prompted to onboard an SSL/TLS certificate.
+
+Two levels of security are available:
+
+- Meet specific certificate and encryption requirements requested by your organization, by uploading the CA-signed certificate.
+- Allow validation between the management console and connected sensors. Validation is evaluated against a certificate revocation list and the certificate expiration date. *If validation fails, communication between the management console and the sensor is halted and a validation error appears in the console.* This option is enabled by default after installation.
+
+The console supports the following certificate types:
+
+- Private and Enterprise Key Infrastructure (private PKI)
+- Public Key Infrastructure (public PKI)
+- Locally generated on the appliance (locally self-signed)
+
+ > [IMPORTANT]
+ > We recommend that you don't use the default self-signed certificate. The certificate is not secure and should be used for test environments only. The owner of the certificate can't be validated, and the security of your system can't be maintained. Never use this option for production networks.
+
+### Sign in and activate the sensor
+
+To sign in and activate:
+
+1. Go to the sensor console from your browser by using the IP defined during the installation. The sign-in dialog box opens.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-sensor/azure-defender-for-iot-sensor-log-in-screen.png" alt-text="Azure Defender for IoT sensor.":::
+
+1. Enter the credentials defined during the sensor installation. If you purchased a preconfigured sensor from Arrow, generate a password first. For more information on password recovery, see [Investigate password failure at initial sign-in](how-to-troubleshoot-the-sensor-and-on-premises-management-console.md#investigate-password-failure-at-initial-sign-in).
+
+1. After you sign in, the **Activation** dialog box opens. Select **Upload** and go to the activation file that you downloaded during sensor onboarding.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-sensor/activation-upload-screen-with-upload-button.png" alt-text="Select Upload and go to the activation file.":::
+
+1. Select the **Sensor Network Configuration** link if you want to change the sensor network configuration before activation. See [Update sensor network configuration before activation](#update-sensor-network-configuration-before-activation).
+
+1. Accept the terms and conditions.
+
+1. Select **Activate**. The SSL/TLS certificate dialog box opens.
+
+1. Define a certificate name.
+1. Upload the CRT and key files.
+1. Enter a passphrase and upload a PEM file if required.
+1. Select **Next**. The validation screen opens. By default, validation between the management console and connected sensors is enabled.
+1. Turn off the **Enable system-wide validation** toggle to disable validation. We recommend that you enable validation.
+1. Select **Save**.
+
+You might need to refresh your screen after uploading the CA-signed certificate.
+
+For information about uploading a new certificate, supported certificate parameters, and working with CLI certificate commands, see [Manage individual sensors](how-to-manage-individual-sensors.md).
+
+#### Update sensor network configuration before activation
+
+The sensor network configuration parameters were defined during the software installation or when you purchased a preconfigured sensor. The following parameters were defined:
+
+- IP address
+- DNS
+- Default gateway
+- Subnet mask
+- Host name
+
+You might want to update this information before activating the sensor. For example, you might need to change the preconfigured parameters defined by Arrow. You can also define proxy settings before activating your sensor.
+
+To update sensor network configuration parameters:
+
+1. Select the **Sensor Network Configuration** link form the **Activation** dialog box.
+
+ :::image type="content" source="media/how-to-activate-and-set-up-your-sensor/editable-network-configuration-screen-v2.png" alt-text="Sensor Network Configuration.":::
+
+2. The parameters defined during installation are displayed. The option to define the proxy is also available. Update any settings as required and select **Save**.
+
+### Subsequent sign-ins
+
+After first-time activation, the Azure Defender for IoT sensor console opens after sign-in without requiring an activation file. You need only your sign-in credentials.
+
+After your sign in, the Azure Defender for IoT console opens.
+
+:::image type="content" source="media/how-to-activate-and-set-up-your-sensor/azure-defender-for-iot-log-in-screen-dashboard-v2.png" alt-text="Azure Defender for IoT console.":::
+
+## Initial setup and learning (for administrators)
+
+After your first sign-in, the Azure Defender for IoT sensor starts to monitor your network automatically. Network assets will appear in the asset map and asset inventory sections. Azure Defender for IoT will begin to detect and alert you on all security and operational incidents that occur in your network. You can then create reports and queries based on the detected information.
+
+Initially this activity is carried out in the learning mode, which instructs your sensor to learn your network's usual activity. For example, the sensor learns assets discovered in your network, protocols detected in the network, and file transfers that occur between specific assets. This activity becomes your network's baseline activity.
+
+### Review and update basic system settings
+
+Review the sensor's system settings to make sure the sensor is configured to optimally detect and alert.
+
+Define the sensor's system settings. For example:
+
+- Define ICS (or IoT) and segregated subnets.
+
+- Define port aliases for site-specific protocols.
+
+- Define VLANs and names that are in use.
+
+- If DHCP is in use, define legitimate DHCP ranges.
+
+- Define integration with Active Directory and mail servers.
+
+### Disable learning mode
+
+After adjusting the system settings, you can let the Azure Defender for IoT sensor run in learning mode until you feel that system detections accurately reflect your network activity.
+
+The learning mode should run for about 2 to 6 weeks, depending on your network size and complexity. After you disable learning mode, any activity that differs from your baseline activity will trigger an alert.
+
+To disable learning mode:
+
+- Select **System Settings** and turn off the **Learning** option.
+
+## First-time sign-in for security analysts and read-only users
+
+Before you sign in, verify that you have:
+
+- The sensor IP address.
+- Sign-in credentials that your administrator provided.
+
+## Console tools: Overview
+
+You access console tools from the side menu.
+
+**Navigation**
+
+| Window | Icon | Description |
+| -----------|--|--|
+| Dashboard | :::image type="icon" source="media/concept-sensor-console-overview/dashboard-icon-azure.png" border="false"::: | View an intuitive snapshot of the state of the network's security. |
+| Device map | :::image type="icon" source="media/concept-sensor-console-overview/asset-map-icon-azure.png" border="false"::: | View the network devices, device connections, and device properties in a map. Various zooms, highlight, and filter options are available to display your network. |
+| Device inventory | :::image type="icon" source="media/concept-sensor-console-overview/asset-inventory-icon-azure.png" border="false"::: | The device inventory displays an extensive range of device attributes that this sensor detects. Options are available to: <br /> - Filter the information according to the table fields, and see the filtered information displayed. <br /> - Export information to a CSV file. <br /> - Import Windows registry details.|
+| Alerts | :::image type="icon" source="media/concept-sensor-console-overview/alerts-icon-azure.png" border="false"::: | Display alerts when policy violations occur, deviations from the baseline behavior occur, or any type of suspicious activity in the network is detected. |
+| Reports | :::image type="icon" source="media/concept-sensor-console-overview/reports-icon-azure.png" border="false"::: | View reports that are based on data-mining queries. |
+
+**Analysis**
+
+| Window| Icon | Description |
+|---|---|---|
+| Event timeline | :::image type="icon" source="media/concept-sensor-console-overview/event-timeline-icon-azure.png" border="false"::: | View a timeline with information about alerts, network events (informational), and user operations, such as user sign-ins and user deletions.|
+
+**Navigation**
+
+| Window | Icon | Description |
+|---|---|---|
+| Data mining | :::image type="icon" source="media/concept-sensor-console-overview/data-mining-icon-azure.png" border="false"::: | Generate comprehensive and granular information about your network's devices at various layers. |
+| Trends and statistics | :::image type="icon" source="media/concept-sensor-console-overview/trends-and-statistics-icon-azure.jpg" border="false"::: | View trends and statistics in an extensive range of widgets. |
+| Risk Assessment | :::image type="icon" source="media/concept-sensor-console-overview/vulnerabilities-icon-azure.png" border="false"::: | Display the **Vulnerabilities** window. |
+
+**Admin**
+
+| Window | Icon | Description |
+|---|---|---|
+| Users | :::image type="icon" source="media/concept-sensor-console-overview/users-icon-azure.png" border="false"::: | Define users and roles with various access levels. |
+| Forwarding | :::image type="icon" source="media/concept-sensor-console-overview/forwarding-icon-azure.png" border="false"::: | Forward alert information to partners integrating with Defender for IoT, to email addresses, to webhook servers, and more. <br /> See [Forward alert information](how-to-forward-alert-information-to-partners.md) for details. |
+| System settings | :::image type="icon" source="media/concept-sensor-console-overview/system-settings-icon-azure.png" border="false"::: | Configure the system settings. For example, define DHCP settings, provide mail server details, or create port aliases. |
+| Import settings | :::image type="icon" source="media/concept-sensor-console-overview/import-settings-icon-azure.png" border="false"::: | Display the **Import Settings** window. You can perform manual changes in a device's information.<br /> See [Import device information](how-to-import-device-information.md) for details. |
+
+**Support**
+
+| Window| Icon | Description |
+|----|---|---|
+| Support | :::image type="icon" source="media/concept-sensor-console-overview/support-icon-azure.png" border="false"::: | Contact [Microsoft Support](https://support.microsoft.com/) for help. |
+
+### See also
+
+[Onboard a sensor](getting-started.md#4-onboard-a-sensor)
+
+[Manage sensor activation files](how-to-manage-individual-sensors.md#manage-sensor-activation-files)
+
+[Control what traffic is monitored](how-to-control-what-traffic-is-monitored.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-configure-with-sentinel https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-configure-with-sentinel.md
@@ -1,5 +1,5 @@
---
-title: Configure Azure Sentinel for Defender for IoT (preview)
+title: Configure Azure Sentinel for Defender for IoT
description: Explains how to configure Azure Sentinel to receive data from your Defender for IoT solution. services: defender-for-iot ms.service: defender-for-iot
@@ -10,14 +10,11 @@ ms.devlang: na
ms.topic: how-to ms.tgt_pltfrm: na ms.workload: na
-ms.date: 12/16/2020
+ms.date: 12/28/2020
ms.author: shhazam ---
-# Connect your data from Defender for IoT to Azure Sentinel (preview)
-
-> [!IMPORTANT]
-> The Defender for IoT data connector is currently in public preview. This feature is provided without a service level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
+# Connect your data from Defender for IoT to Azure Sentinel
Use the Defender for IoT connector to stream all your Defender for IoT events into Azure Sentinel.
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-control-what-traffic-is-monitored https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-control-what-traffic-is-monitored.md new file mode 100644
@@ -0,0 +1,301 @@
+---
+title: Control what traffic is monitored
+description: Sensors automatically perform deep packet detection for IT and OT traffic and resolve information about network devices, such as device attributes and network behavior. Several tools are available to control the type of traffic that each sensor detects.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/07/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Control what traffic is monitored
+
+Sensors automatically perform deep packet detection for IT and OT traffic and resolve information about network devices, such as device attributes and behavior. Several tools are available to control the type of traffic that each sensor detects.
+
+## Learning and Smart IT Learning modes
+
+The Learning mode instructs your sensor to learn your network's usual activity. Examples are devices discovered in your network, protocols detected in the network, file transfers between specific devices, and more. This activity becomes your network baseline.
+
+The Learning mode is automatically enabled after installation and will remain enabled until turned off. The approximate learning mode period is between two to six weeks, depending on the network size and complexity. After this period, when the learning mode is disabled, any new activity detected will trigger alerts. Alerts are triggered when the policy engine discovers deviations from your learned baseline.
+
+After the learning period is complete and the Learning mode is disabled, the sensor might detect an unusually high level of baseline changes that are the result of normal IT activity, such as DNS and HTTP requests. The activity is called nondeterministic IT behavior. The behavior might also trigger unnecessary policy violation alerts and system notifications. To reduce the amount of these alerts and notifications, you can enable the **Smart IT Learning** function.
+
+When Smart IT Learning is enabled, the sensor tracks network traffic that generates nondeterministic IT behavior based on specific alert scenarios.
+
+The sensor monitors this traffic for seven days. If it detects the same nondeterministic IT traffic within the seven days, it continues to monitor the traffic for another seven days. When the traffic is not detected for a full seven days, Smart IT Learning is disabled for that scenario. New traffic detected for that scenario will only then generate alerts and notifications.
+
+Working with Smart IT Learning helps you reduce the number of unnecessary alerts and notifications caused by noisy IT scenarios.
+
+If your sensor is controlled by the on-premises management console, you can't disable the learning modes. In cases like this, the learning mode can only be disabled from the management console.
+
+The learning capabilities (Learning and Smart IT Learning) are enabled by default.
+
+To enable or disable learning:
+
+- Select **System Settings** and toggle the **Learning** and **Smart IT Learning** options.
+
+:::image type="content" source="media/concept-learning-modes/toggle-options-for-learning-and-smart-it-learning.png" alt-text="System settings toggle screen.":::
+
+## Configure subnets
+
+Subnet configurations affect how you see devices in the device map.
+
+By default, the sensor discovers your subnet setup and populates the **Subnet Configuration** dialog box with this information.
+
+To enable focus on the OT devices, IT devices are automatically aggregated by subnet in the device map. Each subnet is presented as a single entity on the map, including an interactive collapsing/expanding capability to "drill down" into an IT subnet and back.
+
+When you're working with subnets, select the ICS subnets to identify the OT subnets. You can then focus the map view on OT and ICS networks and collapse to a minimum the presentation of IT network elements. This effort reduces the total number of the devices shown on the map and provides a clear picture of the OT and ICS network elements.
+
+:::image type="content" source="media/how-to-control-what-traffic-is-monitored/expand-network-option.png" alt-text="Screenshot that shows filtering to a network.":::
+
+You can change the configuration or change the subnet information manually by exporting the discovered data, changing it manually, and then importing back the list of subnets that you manually defined. For more information about export and import, see [Import device information](how-to-import-device-information.md).
+
+In some cases, such as environments that use public ranges as internal ranges, you can instruct the sensor to resolve all subnets as internal subnets by selecting the **Do Not Detect Internet Activity** option. When you select that option:
+
+- Public IP addresses will be treated as local addresses.
+
+- No alerts will be sent about unauthorized internet activity, which reduces notifications and alerts received on external addresses.
+
+To configure subnets:
+
+1. On the side menu, select **System Settings**.
+
+2. In the **System Setting** window, select **Subnets**.
+
+ :::image type="content" source="media/how-to-control-what-traffic-is-monitored/edit-subnets-configuration-screen.png" alt-text="Screenshot that shows the subnet configuration screen.":::
+
+3. To add subnets automatically when new devices are discovered, keep **Auto Subnets Learning** selected.
+
+4. To resolve all subnets as internal subnets, select **Don't Detect Internet Activity**.
+
+5. Select **Add network** and define the following parameters for each subnet:
+
+ - The subnet IP address.
+ - The subnet mask address.
+ - The subnet name. We recommend that you name each subnet with a meaningful name that you can easily identify, so you can differentiate between IT and OT networks. The name can be up to 60 characters.
+
+6. To mark this subnet as an OT subnet, select **ICS Subnet**.
+
+7. To present the subnet separately when you're arranging the map according to the Purdue level, select **Segregated**.
+
+8. To delete a subnet, select :::image type="icon" source="media/how-to-control-what-traffic-is-monitored/delete-icon.png" border="false":::.
+
+9. To delete all subnets, select **Clear All**.
+
+10. To export configured subnets, select **Export**. The subnet table is downloaded to your workstation.
+
+11. Select **Save**.
+
+### Importing information
+
+To import subnet information, select **Import** and select a CSV file to import. The subnet information is updated with the information that you imported. If you important an empty field, you'll lose your data.
+
+## Detection engines
+
+Self-learning analytics engines eliminate the need for updating signatures or defining rules. The engines use ICS-specific behavioral analytics and data science to continuously analyze OT network traffic for anomalies, malware, operational problems, protocol violations, and baseline network activity deviations.
+
+> [!NOTE]
+> We recommend that you enable all the security engines.
+
+When an engine detects a deviation, an alert is triggered. You can view and manage alerts from the alert screen or from a partner system.
+
+:::image type="content" source="media/how-to-control-what-traffic-is-monitored/deviation-alert-screen.png" alt-text="Screenshot that shows detection of deviation.":::
+
+The name of the engine that triggered the alert appears under the alert title.
+
+### Protocol violation engine
+
+A protocol violation occurs when the packet structure or field values don't comply with the protocol specification.
+
+Example scenario:
+*"Illegal MODBUS Operation (Function Code Zero)"* alert. This alert indicates that a primary device sent a request with function code 0 to a secondary device. This action is not allowed according to the protocol specification, and the secondary device might not handle the input correctly.
+
+### Policy violation engine
+
+A policy violation occurs with a deviation from baseline behavior defined in learned or configured settings.
+
+Example scenario:
+*"Unauthorized HTTP User Agent"* alert. This alert indicates that an application that was not learned or approved by policy is used as an HTTP client on a device. This might be a new web browser or application on that device.
+
+### Malware engine
+
+The Malware engine detects malicious network activity.
+
+Example scenario:
+*"Suspicion of Malicious Activity (Stuxnet)"* alert. This alert indicates that the sensor detected suspicious network activity known to be related to the Stuxnet malware. This malware is an advanced persistent threat aimed at industrial control and SCADA networks.
+
+### Anomaly engine
+
+The Anomaly engine detects anomalies in network behavior.
+
+Example scenario:
+*"Periodic Behavior in Communication Channel"* alert. The component inspects network connections and finds periodic and cyclic behavior of data transmission. This behavior is common in industrial networks.
+
+### Operational engine
+
+The Operational engine detects operational incidents or malfunctioning entities.
+
+Example scenario:
+*"Device is Suspected to be Disconnected (Unresponsive)"* alert. This alert is raised when a device is not responding to any kind of request for a predefined period. This alert might indicate a device shutdown, disconnection, or malfunction.
+
+### Enable and disable engines
+
+When you disable a policy engine, information that the engine generates won't be available to the sensor. For example, if you disable the Anomaly engine, you won't receive alerts on network anomalies. If you created a forwarding rule, anomalies that the engine learns won't be sent. To enable or disable a policy engine, select **Enabled** or **Disabled** for the specific engine.
+
+The overall score is displayed in the lower-right corner of the **System Settings** screen. The score indicates the percentage of available protection enabled through the threat protection engines. Each engine is 20 percent of available protection.
+
+:::image type="content" source="media/how-to-control-what-traffic-is-monitored/protection-score-screen.png" alt-text="Screenshot that shows a score.":::
+
+## Configure DHCP address ranges
+
+Your network might consist of both static and dynamic IP addresses. Typically, static addresses are found on OT networks through historians, controllers, and network infrastructure devices such as switches and routers. Dynamic IP allocation is typically implemented on guest networks with laptops, PCs, smartphones, and other portable equipment (using Wi-Fi or LAN physical connections in different locations).
+
+If you're working with dynamic networks, you handle IP address changes that occur when new IP addresses are assigned. You do this by defining DHCP address ranges.
+
+Changes might happen, for example, when a DHCP server assigns IP addresses.
+
+Defining dynamic IP addresses on each sensor enables comprehensive, transparent support in instances of IP address changes. This ensures comprehensive reporting for each unique device.
+
+The sensor console presents the most current IP address associated with the device and indicates which devices are dynamic. For example:
+
+- The Data Mining report and Device Inventory report consolidate all activity learned from the device as one entity, regardless of the IP address changes. These reports indicate which addresses were defined as DHCP addresses.
+
+ :::image type="content" source="media/how-to-control-what-traffic-is-monitored/populated-device-inventory-screen-v2.png" alt-text="Screenshot that shows device inventory.":::
+
+- The **Device Properties** window indicates if the device was defined as a DHCP device.
+
+To set a DHCP address range:
+
+1. On the side menu, select **DHCP Ranges** from the **System Settings** window.
+
+ :::image type="content" source="media/how-to-control-what-traffic-is-monitored/dhcp-address-range-screen.png" alt-text="Screenshot that shows the selection of DHCP Ranges.":::
+
+2. Define a new range by setting **From** and **To** values.
+
+3. Optionally: Define the range name, up to 256 characters.
+
+4. To export the ranges to a CSV file, select **Export**.
+
+5. To manually add multiple ranges from a CSV file, select **Import** and then select the file.
+
+ > [!NOTE]
+ > The ranges that you import from a CSV file overwrite the existing range settings.
+
+6. Select **Save**.
+
+## Configure DNS servers for reverse lookup resolution
+
+To enhance asset enrichment, you can configure multiple DNS servers to carryout reverse lookups. You can resolve host names or FQDNs associated with the IP addresses detected in network subnets. For example, if a sensor discovers an IP address, it might query multiple DNS servers to resolve the host name.
+
+All CIDR formats are supported.
+
+The host name appears in the asset inventory and asset map, as well as in reports.
+
+You can schedule reverse lookup resolution schedules for specific hourly intervals, such as every 12 hours. Or you can schedule a specific time.
+
+To define DNS servers:
+
+1. Select **System Settings** and then select **DNS Settings**.
+
+2. Select **Add DNS Server**.
+
+ :::image type="content" source="media/how-to-enrich-asset-information/dns-reverse-lookup-configuration-screen.png" alt-text="Screenshot that shows the selection of Add DNS Server.":::
+
+3. In the **Schedule reverse DNS lookup** field, choose either:
+
+ - Intervals (per hour).
+
+ - A specific time. Use European formatting. For example, use **14:30** and not **2:30 PM**.
+
+4. In the **DNS Server Address** field, enter the DNS IP address.
+
+5. In the **DNS Server Port** field, enter the DNS port.
+
+6. Resolve the network IP addresses to asset FQDNs. In the **Number of Labels** field, add the number of domain labels to display. Up to 30 characters are displayed from left to right.
+
+7. In the **Subnets** field, enter the subnets that you want the DNS server to query.
+
+8. Select the **Enable** toggle if you want to initiate the reverse lookup.
+
+### Test the DNS configuration
+
+By using a test asset, verify that the settings you defined work properly:
+
+1. Enable the **DNS Lookup** toggle.
+
+2. Select **Test**.
+
+3. Enter an address in **Lookup Address** for the **DNS reverse lookup test for server** dialog box.
+
+ :::image type="content" source="media/how-to-enrich-asset-information/dns-reverse-looup-test-screen.png" alt-text="Screenshot that shows the Lookup Address area.":::
+
+4. Select **Test**.
+
+## Configure Windows Endpoint Monitoring
+
+With the Windows Endpoint Monitoring capability, you can configure Azure Defender for IoT to selectively probe Windows systems. This provides you with more focused and accurate information about your devices, such as service pack levels.
+
+You can configure probing with specific ranges and hosts, and configure it to be performed only as often as desired. You accomplish selective probing by using the Windows Management Instrumentation (WMI), which is Microsoft's standard scripting language for managing Windows systems.
+
+> [!NOTE]
+> - You can run only one scan at a time.
+> - You get the best results with users who have domain or local administrator privileges.
+> - Before you begin the WMI configuration, configure a firewall rule that opens outgoing traffic from the sensor to the scanned subnet by using UDP port 135 and all TCP ports above 1024.
+
+When the probe is finished, a log file with all the probing attempts is available from the option to export a log. The log contains all the IP addresses that were probed. For each IP address, the log shows success and failure information. There's also an error code, which is a free string derived from the exception. The scan of the last log only is kept in the system.
+
+You can perform scheduled scans or manual scans. When a scan is finished, you can view the results in a CSV file.
+
+**Prerequisites**
+
+Configure a firewall rule that opens outgoing traffic from the sensor to the scanned subnet by using UDP port 135 and all TCP ports above 1024.
+
+To configure an automatic scan:
+
+1. On the side menu, select **System Settings**.
+
+2. Select **Windows Endpoint Monitoring** :::image type="icon" source="media/how-to-control-what-traffic-is-monitored/windows-endpoint-monitoring-icon-v2.png" border="false":::.
+
+ :::image type="content" source="media/how-to-control-what-traffic-is-monitored/windows-endpoint-monitoring-screen-v2.png" alt-text="Screenshot that shows the selection of Windows Endpoint Monitoring.":::
+
+3. On the **Scan Schedule** pane, configure options as follows:
+
+ - **By fixed intervals (in hours)**: Set the scan schedule according to intervals in hours.
+
+ - **By specific times**: Set the scan schedule according to specific times and select **Save Scan**.
+
+ :::image type="content" source="media/how-to-control-what-traffic-is-monitored/schedule-a-scan-screen-v2.png" alt-text="Screenshot that shows the Save Scan button.":::
+
+4. To define the scan range, select **Set scan ranges**.
+
+5. Set the IP address range and add your user and password.
+
+ :::image type="content" source="media/how-to-control-what-traffic-is-monitored/edit-scan-range-screen.png" alt-text="Screenshot that shows adding a user and password.":::
+
+6. To exclude an IP range from a scan, select **Disable** next to the range.
+
+7. To remove a range, select :::image type="icon" source="media/how-to-control-what-traffic-is-monitored/remove-scan-icon.png" border="false"::: next to the range.
+
+8. Select **Save**. The **Edit Scan Ranges Configuration** dialog box closes, and the number of ranges appears in the **Scan Ranges** pane.
+
+To perform a manual scan:
+
+1. On the side menu, select **System Settings**.
+
+2. Select **Windows Endpoint Monitoring** :::image type="icon" source="media/how-to-control-what-traffic-is-monitored/windows-endpoint-monitoring-icon-v2.png" border="false":::.
+
+ :::image type="content" source="media/how-to-control-what-traffic-is-monitored/windows-endpoint-monitoring-screen-v2.png" alt-text="Screenshot that shows the Windows Endpoint Monitoring setup screen.":::
+
+3. In the **Actions** pane, select **Start scan**. A status bar appears on the **Actions** pane and shows the progress of the scanning process.
+
+ :::image type="content" source="media/how-to-control-what-traffic-is-monitored/started-scan-screen-v2.png" alt-text="Screenshot that shows the Start scan button.":::
+
+To view scan results:
+
+1. When the scan is finished, on the **Actions** pane, select **View Scan Results**. The CSV file with the scan results is downloaded to your computer.
+
+## See also
+
+[Investigate sensor detections in a device inventory](how-to-investigate-sensor-detections-in-a-device-inventory.md)
+[Investigate sensor detections in the device map](how-to-work-with-the-sensor-device-map.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-create-and-manage-users https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-create-and-manage-users.md new file mode 100644
@@ -0,0 +1,200 @@
+---
+title: Create and manage users
+description: Create and manage users of sensors and the on-premises management console. Users can be assigned the role of administrator, security analyst, or read-only user.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/21/2020
+ms.topic: article
+ms.service: azure
+---
+
+# About Defender for IoT console users
+
+This article describes how to create and manage users of sensors and the on-premises management console. User roles include administrator, security analyst, or read-only user. Each role is associated with a range of permissions to tools for the sensor or on-premises management console. Roles are designed to facilitate granular, secure access to Azure Defender for IoT.
+
+Features are also available to track user activity and enable Active Directory sign-in.
+
+By default, each sensor and on-premises management console is installed with a *cyberx and support* user. These users have access to advanced tools for troubleshooting and setup. Administrator users should sign in with these user credentials, create an admin user, and then create additional users for security analysts and read-only users.
+
+## Role-based permissions
+The following user roles are available:
+
+- **Read only**: Read-only users perform tasks such as viewing alerts and devices on the device map. These users have access to options displayed under **Navigation**.
+
+- **Security analyst**: Security analysts have read-only user permissions. They can also perform actions on devices, acknowledge alerts, and use investigation tools. These users have access to options displayed under **Navigation** and **Analysis**.
+
+- **Administrator**: Administrators have access to all tools, including defining system configurations, creating and managing users, and more. These users have access to options displayed under **Navigation**, **Analysis**, and **Administration**.
+
+### Role-based permissions to on-premises management console tools
+
+This section describes permissions available to administrators, security analysts, and read-only users for the on-premises management console.
+
+| Permission | Read only | Security analyst | Administrator |
+|--|--|--|--|
+| View and filter the enterprise map | Γ£ô | Γ£ô | Γ£ô |
+| Build a site | | | Γ£ô |
+| Manage a site (add and edit zones) | | | Γ£ô |
+| View and filter device inventory | Γ£ô | Γ£ô | Γ£ô |
+| View and manage alerts: acknowledge, learn, and pin | Γ£ô | Γ£ô | Γ£ô |
+| Generate reports | | Γ£ô | Γ£ô |
+| View risk assessment reports | | Γ£ô | Γ£ô |
+| Set alert exclusions | | Γ£ô | Γ£ô |
+| View or define access groups | | | Γ£ô |
+| Manage system settings | | | Γ£ô |
+| Manage users | | | Γ£ô |
+| Send alert data to partners | | | Γ£ô |
+| Manage certificates | | | Γ£ô |
+| Session timeout when users are not active | 30 minutes | 30 minutes | 30 minutes |
+
+#### Assign users to access groups
+
+Administrators can enhance user access control in Defender for IoT by assigning users to specific *access groups*. Access groups are assigned to zones, sites, regions, and business units where a sensor is located. By assigning users to access groups, administrators gain specific control over where users manage and analyze device detections.
+
+Working this way accommodates large organizations where user permissions can be complex or determined by a global organizational security policy. For more information, see [Define global access control](how-to-define-global-user-access-control.md).
+
+### Role-based permissions to sensor tools
+
+This section describes permissions available to sensor administrators, security analysts, and read-only users.
+
+| Permission | Read only | Security analyst | Administrator |
+|--|--|--|--|
+| View the dashboard | Γ£ô | Γ£ô | Γ£ô |
+| Control map zoom views | | | Γ£ô |
+| View alerts | Γ£ô | Γ£ô | Γ£ô |
+| Manage alerts: acknowledge, learn, and pin | | Γ£ô | Γ£ô |
+| View events in a timeline | | Γ£ô | Γ£ô |
+| Authorize devices, known scanning devices, programming devices | | Γ£ô | Γ£ô |
+| View investigation data | Γ£ô | Γ£ô | Γ£ô |
+| Manage system settings | | | Γ£ô |
+| Manage users | | | Γ£ô |
+| DNS servers for reverse lookup | | | Γ£ô |
+| Send alert data to partners | | Γ£ô | Γ£ô |
+| Create alert comments | | Γ£ô | Γ£ô |
+| View programming change history | Γ£ô | Γ£ô | Γ£ô |
+| Create customized alert rules | | Γ£ô | Γ£ô |
+| Manage multiple notifications simultaneously | | Γ£ô | Γ£ô |
+| Manage certificates | | | Γ£ô |
+| Session timeout when users are not active | 30 minutes | 30 minutes | 30 minutes |
+
+## Define users
+
+This section describes how to define users. Cyberx, support, and administrator users can add, remove, and update other user definitions.
+
+To define a user:
+
+1. From the left pane for the sensor or the on-premises management console, select **Users**.
+2. In the **Users** window, select **Create User**.
+3. On the **Create User** pane, define the following parameters:
+
+ - **Username**: Enter a username.
+ - **Email**: Enter the user's email address.
+ - **First Name**: Enter the user's first name.
+ - **Last Name**: Enter the user's last name.
+ - **Role**: Define the user's role. See [Role-based permissions](#role-based-permissions).
+ - **Access Group**: If you're creating a user for the on-premises management console, define the user's access group. See [Define global access control](how-to-define-global-user-access-control.md).
+ - **Password**: Select the user type as follows:
+ - **Local User**: Define a password for the user of a sensor or an on-premises management console. The password must include at least six characters and must include letters and numbers.
+ - **Active Directory User**: You can allow users to sign in to the sensor or management console by using Active Directory credentials. Defined Active Directory groups can be associated with specific permission levels. For example, configure a specific Active Directory group and assign all users in the group to the read-only user type.
+
+:::image type="content" source="media/how-to-create-azure-for-defender-users-and-roles/manage-user-views.png" alt-text="Manage your users.":::
+
+## User session timeout
+
+If users are not active at the keyboard or mouse for a specific time, they're signed out of their session and must sign in again.
+
+When users have not worked with their console mouse or keyboard for a period of 30 minutes, a session sign-out is forced.
+
+This feature is enabled by default and on upgrade but can be disabled. In addition, session counting times can be updated. Session times are defined in seconds. Definitions are applied per sensor and on-premises management console.
+
+A session timeout message appears at the console when the inactivity timeout has passed.
+
+### Control inactivity sign-out
+
+Administrator users can enable and disable inactivity sign-out and adjust the inactivity thresholds.
+
+To access the command:
+
+1. Sign in to the CLI for the sensor or on-premises management console by using Defender for IoT administrative credentials.
+
+2. Enter `sudo nano /var/cyberx/properties/authentication`.
+
+```azurecli-interactive
+ infinity_session_expiration = true
+ session_expiration_default_seconds = 0
+ # half an hour in seconds (comment)
+ session_expiration_admin_seconds = 1800
+ # a day in seconds
+ session_expiration_security_analyst_seconds = 1800
+ # a week in seconds
+ session_expiration_read_only_users_seconds = 1800
+```
+
+To disable the feature, change `infinity_session_expiration = true` to `infinity_session_expiration = false`.
+
+To update sign-out counting periods, adjust the `= <number>` value to the required time.
++
+## Track user activity
+
+You can track user activity in the event timeline on each sensor. The timeline displays the event or affected device, and the time and date that the user carried out the activity.
+
+To view user activity:
+
+1. Sign in to the sensor.
+1. In the event timeline, enable the **User Operations** option.
+
+ :::image type="content" source="media/how-to-create-azure-for-defender-users-and-roles/User-login-attempts.png" alt-text="View a user's activity.":::
+
+## Integrate with Active Directory servers
+
+Configure the sensor or on-premises management console to work with Active Directory. This allows Active Directory users to access the Defender for IoT consoles by using their Active Directory credentials.
+
+Two types of LDAP-based authentication are supported:
+
+- **Full authentication**: User details are retrieved from the LDAP server. Examples are the first name, last name, email, and user permissions.
+
+- **Trusted user**: Only the user password is retrieved. Other user details that are retrieved are based on users defined in the sensor.
+
+### Active Directory and Defender for IoT permissions
+
+You can associate Active Directory groups defined here with specific permission levels. For example, configure a specific Active Directory group and assign RO permissions to all users in the group. See [Create and manage users](how-to-create-and-manage-users.md) for details.
+
+To configure Active Directory:
+
+1. From the left pane, select **System Settings**.
+
+ :::image type="content" source="media/how-to-setup-active-directory/ad-system-settings-v2.png" alt-text="View your Active Directory system settings.":::
+
+2. On the **System Settings** pane, select **Active Directory**.
+
+ :::image type="content" source="media/how-to-setup-active-directory/ad-configurations-v2.png" alt-text="Edit your Active Directory configurations.":::
+
+3. In the **Edit Active Directory Configuration** dialog box, select **Active Directory Integration Enabled** > **Save**. The **Edit Active Directory Configuration** dialog box expands, and you can now enter the parameters to configure Active Directory.
+
+ :::image type="content" source="media/how-to-setup-active-directory/ad-integration-enabled-v2.png" alt-text="Enter the parameters to configure Active Directory.":::
+
+ > [!NOTE]
+ > - You must define the LDAP parameters here exactly as they appear in Active Directory.
+ > - For all the Active Directory parameters, use lowercase only. Use lowercase even when the configurations in Active Directory use uppercase.
+ > - You can't configure both LDAP and LDAPS for the same domain. You can, however, use both for different domains at the same time.
+
+4. Set the Active Directory server parameters, as follows:
+
+ | Server parameter | Description |
+ |--|--|
+ | Domain controller FQDN | Set the fully qualified domain name (FQDN) exactly as it appears on your LDAP server. For example, enter `host1.subdomain.domain.com`. |
+ | Domain controller port | Define the port on which your LDAP is configured. |
+ | Primary domain | Set the domain name (for example, `subdomain.domain.com`) and the connection type according to your LDAP configuration. |
+ | Active Directory groups | Enter the group names that are defined in your Active Directory configuration on the LDAP server. |
+ | Trusted domains | To add a trusted domain, add the domain name and the connection type of a trusted domain. <br />You can configure trusted domains only for users who were defined under users. |
+
+5. Select **Save**.
+
+6. To add a trusted server, select **Add Server** and configure another server.
+
+## See also
+
+[Activate and set up your sensor](how-to-activate-and-set-up-your-sensor.md)
+[Activate and set up your on-premises management console](how-to-activate-and-set-up-your-on-premises-management-console.md)
+[Track sensor activity](how-to-track-sensor-activity.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-define-global-user-access-control https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-define-global-user-access-control.md new file mode 100644
@@ -0,0 +1,88 @@
+---
+title: Define global user access control
+description: In large organizations, user permissions can be complex and might be determined by a global organizational structure, in addition to the standard site and zone structure.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/08/2020
+ms.topic: article
+ms.service: azure
+---
+
+# Define global access control
+
+In large organizations, user permissions can be complex and might be determined by a global organizational structure, in addition to the standard site and zone structure.
+
+To support the demand for user access permissions that are global and more complex, you can create a global business topology that's based on business units, regions, and sites. Then you can define user access permissions around these entities.
+
+Working with access tools for business topology helps organizations implement zero-trust strategies by better controlling where users manage and analyze assets in the Azure Defender for IoT platform.
+
+## About access groups
+
+Global access control is established through the creation of user access groups. Access groups consist of rules regarding which users can access specific business entities. Working with groups lets you control view and configuration access to Defender for IoT for specific user roles at relevant business units, regions, and sites.
+
+For example, allow security analysts from an Active Directory group to access all West European automotive and glass production lines, along with a plastics line in one region.
+
+:::image type="content" source="media/how-to-define-global-user-access-control/sa-diagram.png" alt-text="Diagram of the Security Analyst Active Directory group.":::
+
+Before you create access groups, we recommend that you:
+
+- Carefully set up your business topology. For more information about business topology, see [Work with site map views](how-to-gain-insight-into-global-regional-and-local-threats.md#work-with-site-map-views).
+
+- Plan which users are associated with the access groups that you create. Two options are available for assigning users to access groups:
+
+ - **Assign groups of Active Directory groups**: Verify that you set up an Active Directory instance to integrate with the on-premises management console.
+
+ - **Assign local users**: Verify that you created users. For more information, see [Define users](how-to-create-and-manage-users.md#define-users).
+
+Admin users can't be assigned to access groups. These users have access to all business topology entities by default.
+
+## Create access groups
+
+This section describes how to create access groups. Default global business units and regions are created for the first group that you create. You can edit the default entities when you define your first group.
+
+To create groups:
+
+1. Select **Access Groups** from the side menu of the on-premises management console.
+
+2. Select :::image type="icon" source="media/how-to-define-global-user-access-control/add-icon.png" border="false":::. In the **Add Access Group** dialog box, enter a name for the access group. The console supports 64 characters. Assign the name in a way that will help you easily distinguish this group from other groups.
+
+ :::image type="content" source="media/how-to-define-global-user-access-control/add-access-group.png" alt-text="The Add Access Group dialog box where you create access groups.":::
+
+3. If the **Assign an Active Directory Group** option appears, you can assign one Active Directory group of users to this access group.
+
+ :::image type="content" source="media/how-to-define-global-user-access-control/add-access-group.png" alt-text="Assign an Active Directory group in the Create Access Group dialog box.":::
+
+ If the option doesn't appear, and you want to include Active Directory groups in access groups, select **System Settings**. On the **Integrations** pane, define the groups. Enter a group name exactly as it appears in the Active Directory configurations, and in lowercase.
+
+5. On the **Users** pane, assign as many users as required to the group. You can also assign users to different groups. If you work this way, you must create and save the access group and rules, and then assign users to the group from the **Users** pane.
+
+ :::image type="content" source="media/how-to-define-global-user-access-control/role-management.png" alt-text="Manage your users' roles and assign them as needed.":::
+
+6. Create rules in the **Add Rules for *name*** dialog box based on your business topology's access requirements. Options that appear here are based on the topology that you designed in the **Enterprise View** and **Site Management** windows.
+
+ You can create more than one rule per group. You might need to create more than one rule per group when you're working with complex access granularity at multiple sites.
+
+ :::image type="content" source="media/how-to-define-global-user-access-control/add-rule.png" alt-text="Add a rule to your system.":::
+
+The rules that you create appear in the **Add Access Group** dialog box. There, you can delete or edit them.
+
+:::image type="content" source="media/how-to-define-global-user-access-control/edit-access-groups.png" alt-text="View and edit all of your access groups from this window.":::
+
+### About rules
+
+When you're creating rules, be aware of the following information:
+
+- When an access group contains several rules, the rule logic aggregates all rules. For example, the rules use AND logic, not OR logic.
+
+- For a rule to be successfully applied, you must assign sensors to zones in the **Site Management** window.
+
+- You can assign only one element per rule. For example, you can assign one business unit, one region, and one site for each rule. Create more rules for the group if, for example, you want users in one Active Directory group to have access to different business units in different regions.
+
+- If you change an entity and the change affects the rule logic, the rule will be deleted. If changes made to a topology entity affect the rule logic such that all rules are deleted, the access group remains but the users can't sign in to the on-premises management console. Users are notified to contact their administrator to sign in.
+
+- If no business unit or region is selected, users will have access to all defined business units and regions.
+
+## See also
+
+[About Defender for IoT console users](how-to-create-and-manage-users.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-deploy-edge https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-deploy-edge.md
@@ -49,7 +49,7 @@ Use the following steps to deploy an Defender for IoT security module for IoT Ed
1. Select **Internet of Things**, then search for **Defender for IoT** and select it.
- ![Select Defender for IoT](media/howto/edge-onboarding-8.png)
+ :::image type="content" source="media/howto/edge-onboarding-8.png" alt-text="Select Defender for IoT":::
1. Click **Create** to configure the deployment.
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-deploy-management-console-appliance https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-deploy-management-console-appliance.md deleted file mode 100644
@@ -1,58 +0,0 @@
-title: Deploy management console in Azure Defender for IoT
-description: Learn about how to deploy the management console in Azure Defender for IoT.
-services: defender-for-iot
-ms.service: defender-for-iot
-documentationcenter: na
-author: rkarlin
-manager: rkarlin
-editor: ''
-
-ms.devlang: na
-ms.topic: conceptual
-ms.tgt_pltfrm: na
-ms.workload: na
-ms.date: 10/14/2020
-ms.author: rkarlin
-
-# Deploy the management console
-This article describes how to deploy the Azure Defender for IoT on-premises management console.
-
-## The on-premises management console
-
-The on-premises management console provides a consolidated view of all network assets and delivers a real-time view of key IoT and OT risk indicators and alerts across all your facilities. Tightly integrated with your SOC workflows and run books, it enables easy prioritization of mitigation activities and cross-site correlation of threats.
--- Holistic - reduce complexity with a single unified platform for asset management, risk and vulnerability management, as well as threat monitoring with incident response.--- Aggregation and correlation ΓÇô display, aggregate and analyze data and alerts collected from all sites.--- Control all sensors ΓÇô configure and monitor all sensors from a single location.-
- ![Azure Defender for IoT Site Management view](media/updates/image2.png)
-
-## Deploy the on-premises management console appliance
-
-This process requires acquiring your own hardware and installing software. The solution runs on certified appliances. Refer to the [Azure Defender for IoT hardware specifications guide](https://aka.ms/AzureDefenderforIoTBareMetalAppliance) as a reference when purchasing your certified appliance.
-
-Refer to the [Azure Defender for IoT Installation Guide](https://aka.ms/AzureDefenderforIoTInstallSensorISO) for details about downloading the ISO image and installing the sensor software.
-
-**To deploy the on-premises management console:**
-
-1. Navigate to Microsoft Azure Defender for IoT.
-
-2. Select **on-premises management console**.
-
- ![Azure Defender for IoT On-Premises Management Console view](media/updates/image15.png)
-
-3. Select version 3.1 from the **Select Version** menu.
-
-4. Select **Download** and save the file.
-
-5. After the software is installed, carry out network setup tasks. Refer to the [Azure Defender for IoT Network Setup Guide](https://aka.ms/AzureDefenderForIoTNetworkSetup) for details.
-
-## Next steps
-
-To learn more about configuration options, continue to the how-to guide for module configuration.
-> [!div class="nextstepaction"]
-> [Module configuration how-to guide](./how-to-agent-configuration.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-deploy-view-sensors https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-deploy-view-sensors.md deleted file mode 100644
@@ -1,65 +0,0 @@
-title: View and manage onboarded sensors in Azure Defender for IoT
-description: This article describes how to view and delete onboarded sensors, as well as download new activation files for onboarded sensors in Azure Defender for IoT.
-author: rkarlin
-ms.author: rkarlin
-ms.date: 10/10/2020
-ms.topic: article
-ms.service: azure
-
-# View and manage onboarded sensors
-
-This article describes how to view and delete onboarded sensors, as well as download new activation files for onboarded sensors.
-
-## Update sensor management mode
-
-You may want to update the mode that your sensor is managed. When you do this, you need to remove the current sensor from the sensor management page and upload a new Activation file.
--- **Work in the Cloud Managed mode instead of Locally Managed mode**: Update the activation file for your locally managed sensor with an activation file for a Cloud Managed sensor. After reactivation, sensor detections are displayed in both the sensor and Azure Defender for IoT portal. After the reactivation file is successfully uploaded newly detected alert information is sent to Azure.--- **Work in the Locally Managed mode instead of Cloud Managed mode**: Update the activation file for a Cloud Managed sensor with an activation file for a Locally Managed sensor. After reactivation, sensor detection information is only displayed in the sensor.--- **Associate the sensor to a new IoT Hub**: You may want to associate your sensor with a new IoT Hub. To do this, re-register then sensor and upload a new activation file.-
-**To reactivate the sensor:**
-
-1. Navigate to the Azure Defender for IoT, **Sensor Management** page.
-
-2. Select the sensor for which you want to upload a new activation file.
-
-3. Delete it.
-
-4. Onboard the sensor again from the **Onboarding** page in the new mode or with a new IoT Hub.
-
-5. Download the activation file from the **Download activation file** page.
-
-6. Log in to the Azure Defender for IoT sensor console.
-
-7. In the sensor console, select **System Settings** and then select **Reactivation**.
-
- ![Screenshot of Reactivation view](media/updates/image14.png)
-
-8. Select **Upload** and select the file you saved.
-
-9. Select **Activate**.
-
-## Delete sensors
-
-If you delete a cloud managed sensor, information will not be sent to the IoT Hub.
-
-Delete locally managed sensors when you are no longer working with them.
-
-**To delete sensors:**
-
-1. Navigate to the Azure Defender for IoT **Sensor Management** page.
-
-2. Select the sensors you want to delete.
-
-3. Select **Delete sensor**.
-
-## Next steps
-
-To learn more about configuration options, continue to the how-to guide for module configuration.
-> [!div class="nextstepaction"]
-> [Module configuration how-to guide](./how-to-agent-configuration.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-enhance-port-and-vlan-name-resolution https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-enhance-port-and-vlan-name-resolution.md new file mode 100644
@@ -0,0 +1,85 @@
+---
+title: Enhance port and VLAN name resolution
+description: Customize port and VLAN names on your sensors to enrich device resolution.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/13/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Enhance port and VLAN name resolution
+
+You can customize port and VLAN names on your sensors to enrich device resolution.
+
+## Customize port names
+
+Azure Defender for IoT automatically assigns names to most universally reserved ports, such as DHCP or HTTP. You can customize port names for other ports that Defender for IoT detects. For example, assign a name to a non-reserved port because that port shows unusually high activity.
+
+These names appear when:
+
+ - You select **Device groups** from the device map.
+
+ - You create widgets that provide port information.
+
+### View custom port names in the device map
+
+Ports that include a name defined by users appear in the device map, in the **Known Applications** group.
+
+:::image type="content" source="media/how-to-enrich-asset-information/applications-v2.png" alt-text="Screenshot of the device map, showing the Known Applications group.":::
+
+### View custom port names in widgets
+
+Port names that you defined appear in the widgets that cover traffic by port.
+
+:::image type="content" source="media/how-to-enrich-asset-information/traffic-v2.png" alt-text="Cover traffic.":::
+
+To define custom port names:
+
+1. Select **System Settings** and then select **Standard Aliases**.
+
+2. Select **Add Port Alias**.
+
+ :::image type="content" source="media/how-to-enrich-asset-information/edit-aliases.png" alt-text="Add Port Alias.":::
+
+3. Enter the port number, select **TCP/UDP**, or select **Both**, and add the name.
+
+4. Select **Save**.
+
+## Configure VLAN names
+
+You can enrich device inventory data with device VLAN numbers and tags. In addition to data enrichment, you can view the number of devices per VLAN, and view bandwidth by VLAN widgets.
+
+VLANs support is based on 802.1q (up to VLAN ID 4094).
+
+Two methods are available for retrieving VLAN information:
+
+- **Automatically discovered**: By default, the sensor automatically discovers VLANs. VLANs detected with traffic are displayed on the VLAN configuration screen, in data-mining reports, and in other reports that contain VLAN information. Unused VLANs are not displayed. You can't edit or delete these VLANs.
+
+ You should add a unique name to each VLAN. If you don't add a name, the VLAN number appears in all the locations where the VLAN is reported.
+
+- **Manually added**: You can add VLANs manually. You must add a unique name for each VLAN that was manually added, and you can edit or delete these VLANs.
+
+VLAN names can contain up to 50 ASCII characters.
+
+> [!NOTE]
+> VLAN names are not synchronized between the sensor and the management console. You need to define the name on the management console as well.
+For Cisco switches, add the following line to the span configuration: `monitor session 1 destination interface XX/XX encapsulation dot1q`. In that command, *XX/XX* is the name and number of the port.
+
+To configure VLANs:
+
+1. On the side menu, select **System Settings**.
+
+2. In the **System Settings** window, select **VLAN**.
+
+ :::image type="content" source="media/how-to-enrich-asset-information/edit-vlan.png" alt-text="Use the system settings to edit your VLANs.":::
+
+3. Add a unique name next to each VLAN ID.
+
+## Next steps
+
+View enriched device information in the reports for device inventory and data mining:
+
+- [Investigate sensor detections in a device inventory](how-to-investigate-sensor-detections-in-a-device-inventory.md)
+- [Generate reports](how-to-generate-reports.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-forward-alert-information-to-partners https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-forward-alert-information-to-partners.md new file mode 100644
@@ -0,0 +1,203 @@
+---
+title: Forward alert information
+description: You can send alert information to partner systems by working with forwarding rules.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/02/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Forward alert information
+
+You can send alert information to partners who are integrating with Azure Defender for IoT, to syslog servers, to email addresses, and more. Working with forwarding rules lets you quickly deliver alert information to security stakeholders.
+
+Syslog and other default forwarding actions are delivered with your system. More forwarding actions might become available when you integrate with partner vendors, such as Microsoft Azure Sentinel, ServiceNow, or Splunk.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/alert-information-screen.png" alt-text="Alert information.":::
+
+Defender for IoT administrators have permission to use forwarding rules.
+
+## About forwarded alert information
+
+Alerts provide information about an extensive range of security and operational events. For example:
+
+ - Date and time of the alert
+
+ - Engine that detected the event
+
+ - Alert title and descriptive message
+
+ - Alert severity
+
+ - Source and destination name and IP address
+
+ - Suspicious traffic detected
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/address-scan-detected-screen.png" alt-text="Address scan detected.":::
+
+Relevant information is sent to partner systems when forwarding rules are created.
+
+## Create forwarding rules
+
+To create a new forwarding rule:
+
+1. Select **Forwarding** on the side menu.
+
+ ::image type="content" source="media/how-to-work-with-alerts-sensor/create-forwarding-rule-screen.png" alt-text="Create a Forwarding Rule icon.":::
+
+2. Select **Create Forwarding Rule**.
+
+ :::image type="content" source="media/how-to-work-with-alerts-sensor/create-a-forwardong-rule.png" alt-text="Create a new forwarding rule.":::
+
+3. Enter the name of the forwarding rule.
+
+### Forwarding rule criteria
+
+Define criteria by which to trigger a forwarding rule. Working with forwarding rule criteria helps pinpoint and manage the volume of information sent from the sensor to external systems. The following options are available:
+
+**Protocols**: Only trigger the forwarding rule if the traffic detected was running over specific protocols. Select the required protocols from the drop-down list or choose them all.
+
+**Engines**: Select the required engines or choose them all. Alerts from selected engines will be sent.
+
+**Severity levels**: This is the minimum incident to forward, in terms of severity level. For example, if you select **Minor**, minor alerts and any alert above this severity level will be forwarded. Levels are predefined.
+
+### Forwarding rule actions
+
+Forwarding rule actions instruct the sensor to forward alert information to partner vendors or servers. You can create multiple actions for each forwarding rule.
+
+In addition to the forwarding actions delivered with your system, other actions might become available when you integrate with partner vendors.
+
+#### Email address action
+
+Send mail that includes the alert information. You can enter one email address per rule.
+
+To define email for the forwarding rule:
+
+1. Enter a single email address. If more than one mail needs to be sent, create another action.
+
+2. Enter the time zone for the time stamp for the alert detection at the SIEM.
+
+3. Select **Submit**.
+
+#### Syslog server actions
+
+The following formats are supported:
+
+- Text messages
+
+- CEF messages
+
+- LEEF messages
+
+- Object messages
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/create-actions-rule.png" alt-text="Create forwarding rule actions.":::
+
+Enter the following parameters:
+
+- Syslog host name and port.
+
+- Protocol TCP and UDP.
+
+- Time zone for the time stamp for the alert detection at the SIEM.
+
+- TLS encryption certificate file and key file for CEF servers (optional).
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/configure-encryption.png" alt-text="Configure your encryption for your forwarding rule.":::
+
+| Syslog text message output fields | Description |
+|--|--|
+| Date and time | Date and time that the syslog server machine received the information. |
+| Priority | User.Alert |
+| Hostname | Sensor IP address |
+| Protocol | TCP or UDP |
+| Message | Sensor: The sensor name.<br /> Alert: The title of the alert.<br /> Type: The type of the alert. Can be **Protocol Violation**, **Policy Violation**, **Malware**, **Anomaly**, or **Operational**.<br /> Severity: The severity of the alert. Can be **Warning**, **Minor**, **Major**, or **Critical**.<br /> Source: The source device name.<br /> Source IP: The source device IP address.<br /> Destination: The destination device name.<br /> Destination IP: The IP address of the destination device.<br /> Message: The message of the alert.<br /> Alert group: The alert group associated with the alert. |
++
+| Syslog object output | Description |
+|--|--|
+| Date and Time | Date and time that the syslog server machine received the information. |
+| Priority | User.Alert |
+| Hostname | Sensor IP |
+| Message | Sensor name: The name of the appliance. <br /> Alert time: The time that the alert was detected: Can vary from the time of the syslog server machine, and depends on the time-zone configuration of the forwarding rule. <br /> Alert title: The title of the alert. <br /> Alert message: The message of the alert. <br /> Alert severity: The severity of the alert: **Warning**, **Minor**, **Major**, or **Critical**. <br /> Alert type: **Protocol Violation**, **Policy Violation**, **Malware**, **Anomaly**, or **Operational**. <br /> Protocol: The protocol of the alert. <br /> **Source_MAC**: IP address, name, vendor, or OS of the source device. <br /> Destination_MAC: IP address, name, vendor, or OS of the destination. If data is missing, the value will be **N/A**. <br /> alert_group: The alert group associated with the alert. |
++
+| Syslog CEF output format | Description |
+|--|--|
+| Date and time | Date and time that the syslog server machine received the information. |
+| Priority | User.Alert |
+| Hostname | Sensor IP address |
+| Message | CEF:0 <br />Azure Defender for IoT <br />Sensor name: The name of the sensor appliance. <br />Sensor version <br />Alert title: The title of the alert. <br />msg: The message of the alert. <br />protocol: The protocol of the alert. <br />severity: **Warning**, **Minor**, **Major**, or **Critical**. <br />type: **Protocol Violation**, **Policy Violation**, **Malware**, **Anomaly**, or **Operational**. <br /> start: The time that the alert was detected. <br />Might vary from the time of the syslog server machine, and depends on the time-zone configuration of the forwarding rule. <br />src_ip: IP address of the source device. <br />dst_ip: IP address of the destination device.<br />cat: The alert group associated with the alert. |
+
+| Syslog LEEF output format | Description |
+|--|--|
+| Date and time | Date and time that the syslog server machine received the information. |
+| Priority | User.Alert |
+| Hostname | Sensor IP |
+| Message | Sensor name: The name of the Azure Defender for IoT appliance. <br />LEEF:1.0 <br />Azure Defender for IoT <br />Sensor <br />Sensor version <br />Azure Defender for IoT Alert <br />title: The title of the alert. <br />msg: The message of the alert. <br />protocol: The protocol of the alert.<br />severity: **Warning**, **Minor**, **Major**, or **Critical**. <br />type: The type of the alert: **Protocol Violation**, **Policy Violation**, **Malware**, **Anomaly**, or **Operational**. <br />start: The time of the alert. Note that it might be different from the time of the syslog server machine. (This depends on the time-zone configuration.) <br />src_ip: IP address of the source device.<br />dst_ip: IP address of the destination device. <br />cat: The alert group associated with the alert. |
+
+After you enter all the information, select **Submit**.
+
+#### NetWitness action
+
+Send alert information to a NetWitness server.
+
+To define NetWitness forwarding parameters:
+
+1. Enter NetWitness **Hostname** and **Port** information.
+
+2. Enter the time zone for the time stamp for the alert detection at the SIEM.
+
+ :::image type="content" source="media/how-to-work-with-alerts-sensor/add-timezone.png" alt-text="Add a time zone to your forwarding rule.":::
+
+3. Select **Submit**.
+
+#### Integrated vendor actions
+
+You might have integrated your system with a security, device management, or other industry vendor. These integrations let you:
+
+ - Send alert information.
+
+ - Send device inventory information.
+
+ - Communicate with vendor-side firewalls.
+
+Integrations help bridge previously siloed security solutions, enhance device visibility, and accelerate system-wide response to more rapidly mitigate risks.
+
+Use the actions section to enter the credentials and other information required to communicate with integrated vendors.
+
+For details about setting up forwarding rules for the integrations, refer to the relevant partner integration articles.
+
+### Test forwarding rules
+
+Test the connection between the sensor and the partner server that's defined in your forwarding rules:
+
+1. Select the rule from the **Forwarding rule** dialog box.
+
+2. Select the **More** box.
+
+3. Select **Send Test Message**.
+
+4. Go to your partner system to verify that the information sent by the sensor was received.
+
+### Edit and delete forwarding rules
+
+To edit a forwarding rule:
+
+- On the **Forwarding Rule** screen, select **Edit** under the **More** drop-down menu. Make the desired changes and select **Submit**.
+
+To remove a forwarding rule:
+
+- On the **Forwarding Rule** screen, select **Remove** under the **More** drop-down menu. In the **Warning** dialog box, select **OK**.
+
+### Forwarding rules and alert exclusion rules
+
+The administrator might have defined alert exclusion rules. These rules help administrators achieve more granular control over alert triggering by instructing the sensor to ignore alert events based on various parameters. These parameters might include device addresses, alert names, or specific sensors.
+
+This means that the forwarding rules you define might be ignored based on exclusion rules that your administrator has created. Exclusion rules are defined in the on-premises management console.
+
+## See also
+
+[Accelerate alert workflows](how-to-accelerate-alert-incident-response.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-gain-insight-into-global-regional-and-local-threats https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-gain-insight-into-global-regional-and-local-threats.md new file mode 100644
@@ -0,0 +1,80 @@
+---
+title: Gain insight into global, regional, and local threats
+description: Gain insight into global, regional, and local threats by using the site map in the on-premises management console.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/07/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Gain insight into global, regional, and local threats
+
+The site map in the on-premises management console helps you achieve full security coverage by dividing your network into geographical and logical segments that reflect your business topology:
+
+- **Geographical facility level**: A site reflects a number of devices grouped according to a geographical location presented on the map. By default, Azure Defender for IoT provides you with a world map. You update the map to reflect your organizational or business structure. For example, use a map that reflects sites across a specific country, city, or industrial campus. When the site color changes on the map, it provides the SOC team with an indication of critical system status in the facility.
+
+ The map is interactive and enables opening each site and delving into this site's information.
+
+- **Global logical layer**: A business unit is a way to divide your enterprise into logical segments according to specific industries. When you do this, your business topology is reflected on the map.
+
+ For example, a global company that contains glass factories, plastic factories, and automobile factories can be managed as three different business units. A physical site located in Toronto includes three different glass production lines, a plastic production line, and a truck engine production line. So, this site has representatives of all three business units.
+
+- **Geographical region level**: Create regions to divide a global enterprise into geographical regions. For example, the company that we described might use the regions North America, Western Europe, and Eastern Europe. North America has factories from all three business units. Western Europe has automobile factories and glass factories, and Eastern Europe has only plastic factories.
+
+- **Local logical segment level**: A zone is a logical segment within a site that defines, for example, a functional area or production line. Working with zones allows enforcement of security policies that are relevant to the zone definition. For example, a site that contains five production lines can be segmented into five zones.
+
+- **Local view level**: A local view of a single sensor installation provides insight into the operational and security status of connected devices.
+
+## Work with site map views
+
+The on-premises management console provides an overall view of your industrial network in a context-related map. The general map view presents the global map of your organization with the geographical location of each site.
+
+:::image type="content" source="media/how-to-work-with-maps/enterprise.png" alt-text="Screenshot of the Enterprise Map view.":::
+
+### Color-coded map views
+
+**Green**: The number of security events is below the threshold that Defender for IoT has defined for your system. No action is needed.
+
+**Yellow**: The number of security events is equal to the threshold that Defender for IoT has defined for your system. Consider investigating the events.
+
+**Red**: The number of security events is beyond the threshold that Defender for IoT has defined for your system. Take immediate action.
+
+### Risk-level map views
+
+**Risk Assessment**: The Risk Assessment view displays information on site risks. Risk information helps you prioritize mitigation
+and build a road map to plan security improvements.
+
+**Incident Response**: Get a centralized view of all unacknowledged alerts on each site across the enterprise. You can drill down and manage alerts detected in a specific site.
+
+:::image type="content" source="media/how-to-work-with-maps/incident-responses.png" alt-text="Screenshot of the Enterprise Map view with Incident Response.":::
+
+**Malicious Activity**: If malware was detected, the site appears in red. This indicates that you should take immediate action.
+
+:::image type="content" source="media/how-to-work-with-maps/malicious-activity.png" alt-text="Screenshot of the Enterprise Map view with Malicious Activity.":::
+
+**Operational Alerts**: This map view for OT systems provides a better understanding of which OT system might experience operational incidents, such as PLC stops, firmware upload, and program upload.
+
+:::image type="content" source="media/how-to-work-with-maps/operational-alerts.png" alt-text="Screenshot of the Enterprise Map view with Operational Alerts.":::
+
+To choose a map view:
+
+1. Select **Default View** from the map.
+2. Select a view.
+
+:::image type="content" source="media/how-to-work-with-maps/map-views.png" alt-text="Screenshot of the site map default view.":::
+
+## Update the site map image
+
+Defender for IoT provides a default world map. You can change it to reflect your organization: a country map or a city map, for example.
+
+To replace the map:
+
+1. On the left pane, select **System Settings**.
+
+2. Select the **Change Site Map** and upload the graphic file to replace the default map.
+
+## Next step
+
+[View alerts](how-to-view-alerts.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-generate-reports https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-generate-reports.md new file mode 100644
@@ -0,0 +1,489 @@
+---
+title: Generate reports
+description: Gain insight into network activity, risks, attacks, and trends by using Defender for IoT reporting tools.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/17/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Generate reports
+
+Gain insight into network activity, risks, attacks, and trends by using Azure Defender for IoT reporting tools. Tools are available to generate reports:
+
+- Based on activity detected by individual sensors.
+- Based on activity detected by all sensors.
+
+These reports are generated from the on-premises management console.
+
+## Reports for sensor risk assessment
+
+Risk assessment reporting lets you generate a security score for each network device, as well as an overall network security score. The overall score represents the percentage of 100 percent security.
+
+This score is based on results from packet inspection, behavioral modeling engines, and a SCADA-specific state machine design. An extensive range of other information is available, such as:
+
+- Configuration issues
+
+- Device vulnerability prioritized by security level
+
+- Network security issues
+
+- Network operational issues
+
+- Attack vectors
+
+- Connections to ICS networks
+
+- Internet connections
+
+- Industrial malware indicators
+
+- Protocol issues
+
+ - Secure Devices: Devices with a security score above 90%.
+
+- **Secure Devices**: Devices with a security score above 90 percent.
+
+- **Vulnerable Devices**: Devices with a security score below 70 percent.
+
+- **Devices Needing Improvement**: Devices with a security score between 70 percent and 89 percent.
+
+To create a report:
+
+1. Select **Risk Assessment** on the side menu.
+2. Select a sensor from the **Select Sensor** drop-down list.
+3. Select **Generate Report**.
+4. Select **Download** from the Archived Reports section.
+
+:::image type="content" source="media/how-to-generate-reports/risk-assessment.png" alt-text="A view of the risk assessment.":::
+
+To import a company logo:
+
+To import a company logo:
+
+- Select **Import Logo**.
+
+## Reports for sensor attack vector
+
+Attack vector reports provide a graphical representation of a vulnerability chain of exploitable devices. These vulnerabilities can give an attacker access to key network devices. The Attack Vector Simulator calculates attack vectors in real time and analyzes all attack vectors for a specific target.
+
+Working with the attack vector lets you evaluate the effect of mitigation activities in the attack sequence. You can then determine, for example, if a system upgrade disrupts the attacker's path by breaking the attack chain, or if an alternate attack path remains. This information helps you prioritize remediation and mitigation activities.
+
+:::image type="content" source="media/how-to-generate-reports/control-center.png" alt-text="View your alerts in the control center.":::
+
+> [!NOTE]
+> Administrators and security analysts can perform the procedures described in this section.
+
+To create an attack vector simulation:
+
+1. Select :::image type="content" source="media/how-to-generate-reports/plus.png" alt-text="Plus sign":::on the side menu to add a Simulation.
+
+ :::image type="content" source="media/how-to-generate-reports/vector.png" alt-text="The attack vector simulation.":::
+
+2. Enter simulation properties:
+
+ - **Name**: Simulation name.
+
+ - **Maximum vectors**: The maximum number of vectors in a single simulation.
+
+ - **Show in Device map**: Show the attack vector as a filter on the device map.
+
+ - **All Source devices**: The attack vector will consider all devices as an attack source.
+
+ - **Attack Source**: The attack vector will consider only the specified devices as an attack source.
+
+ - **All Target devices**: The attack vector will consider all devices as an attack target.
+
+ - **Attack Target**: The attack vector will consider only the specified devices as an attack target.
+
+ - **Exclude devices**: Specified devices will be excluded from the attack vector simulation.
+
+ - **Exclude Subnets**: Specified subnets will be excluded from the attack vector simulation.
+
+3. Select **Add Simulation**. The simulation will be added to the simulations list.
+
+ :::image type="content" source="media/how-to-generate-reports/new-simulation.png" alt-text="Add a new simulation.":::
+
+4. Select :::image type="icon" source="media/how-to-generate-reports/edit-a-simulation-icon.png" border="false"::: if you want to edit the simulation.
+
+ Select :::image type="icon" source="media/how-to-generate-reports/delete-simulation-icon.png" border="false"::: if you want to delete the simulation.
+
+ Select :::image type="icon" source="media/how-to-generate-reports/make-a-favorite-icon.png" border="false"::: if you want to mark the simulation as a favorite.
+
+5. A list of attack vectors appears and includes vector score (out of 100), attack source device, and attack target device. Select a specific attack for graphical depiction of attack vectors.
+
+ :::image type="content" source="media/how-to-generate-reports/sample-attack-vectors.png" alt-text="Attack vectors.":::
+
+## Sensor data-mining queries
+
+Data-mining tools let you generate comprehensive and granular information about your network devices at various layers. For example, you can create a query based on:
+
+- Time periods
+
+- Connections to the internet
+
+- Ports
+
+- Protocols
+
+- Firmware versions
+
+- Programming commands
+
+- Inactivity of the device
+
+You can fine-tune the report based on filters. For example, you can query a specific subnet in which firmware was updated.
+
+:::image type="content" source="media/how-to-generate-reports/active-device-list-v2.png" alt-text="List of active devices.":::
+
+Various tools are available to manage queries. For example, you can export and save.
+
+> [!NOTE]
+> Administrators and security analysts have access to data-mining options.
+
+### Dynamic updates
+
+Data-mining queries that you create are dynamically updated each time you open them. For example:
+
+- If you create a report for firmware versions on devices on June 1 and open the report again on June 10, this report will be updated with information that's accurate for June 10.
+
+- If you create a report to detect new devices discovered over the last 30 days on June 1 and open the report on June 30, the results will be displayed for the last 30 days.
+
+### Data-mining use cases
+
+You can use queries to handle an extensive range of security needs for various security teams:
+
+- **SOC incident response**: Generate a report in real time to help deal with immediate incident response. For example, generate a report for a list of devices that might require patching.
+
+- **Forensics**: Generate a report based on historical data for investigative reports.
+
+- **IT Network Integrity**: Generate a report that helps improve overall network security. For example, generate a report that lists devices with weak authentication credentials.
+
+- **Visibility**: Generate a report that covers all query items to view all baseline parameters of your network.
+
+### Data-mining storage
+
+Data mining information is saved and stored continuously, except for when a device is deleted. Data mining results can be exported and stored externally to a secure server. In addition, the sensor performs automatic daily backups to ensure system continuity and preservation of data.
+
+### Data mining and reports
+
+Regular reports, accessed from the **Reports** option, are predefined data-mining reports. They're not dynamic queries as are available in data mining, but a static representation of the data-mining query results.
+
+Data-mining query results are not available to RO users. Administrators and security analysts who want RO users to have access to the information generated by data mining queries should save the information as report.
+
+### Predefined queries
+
+The following predefined queries are available. These queries are generated in real time.
+
+- **CVEs**: A list of devices detected with known vulnerabilities within the last 24 hours.
+
+- **Excluded CVEs**: A list of all the CVEs that were manually excluded. To achieve more accurate results in VA reports and attack vectors, you can customize the CVE list manually by including and excluding CVEs.
+
+- **Internet activity**: Devices that are connected to the internet.
+
+- **Nonactive devices**: Devices that have not communicated for the past seven days.
+
+- **Active devices**: Active network devices within the last 24 hours.
+
+- **Remote access**: Devices that communicate through remote session protocols.
+
+- **Programming commands**: Devices that send industrial programming.
+
+These reports are automatically accessible from the **Reports** screen, where RO users and other users can view them. RO users can't access data-mining reports.
+
+:::image type="content" source="media/how-to-generate-reports/data-mining-screeshot-v2.png" alt-text="The data mining screen.":::
+
+### Create a data-mining report
+
+To create a data-mining report:
+
+1. Select **Data Mining** from the side menu. Predefined suggested reports appear automatically.
+
+ :::image type="content" source="media/how-to-generate-reports/data-mining-screeshot-v2.png" alt-text="Select data mining from side pane.":::
+
+2. Select :::image type="icon" source="media/how-to-generate-reports/plus-icon.png" border="false":::.
+
+3. Select **New Report** and define the report.
+
+ :::image type="content" source="media/how-to-generate-reports/create-new-report-screen.png" alt-text="Create a new report by filling out this screen.":::
+
+ The following parameters are available:
+
+ - Provide a report name and description.
+
+ - For categories, select either:
+
+ - **Categories (All)** to view all report results about all devices in your network.
+
+ - **Generic** to choose standard categories.
+
+ - Select specific report parameters of interest to you.
+
+ - Choose a sort order (**Order by**). Sort results based on activity or category.
+
+ - Select **Save to Report Pages** to save the report results as a report that's accessible from the **Report** page. This will enable RO users to run the report that you created.
+
+ - Select **Filters (Add)** to add more filters. Wildcard requests are supported.
+
+ - Specify a device group (defined in the device map).
+
+ - Specify an IP address.
+
+ - Specify a port.
+
+ - Specify a MAC address.
+
+4. Select **Save**. Report results open on the **Data Mining** page.
+
+:::image type="content" source="media/how-to-generate-reports/data-mining-page.png" alt-text="Report results as seen on the Data Mining page.":::
+
+### Manage data-mining reports
+
+The following table describes management options for data mining:
+
+| Icon image | Description |
+|--|--|
+| :::image type="icon" source="media/how-to-generate-reports/edit-a-simulation-icon.png" border="false"::: | Edit the report parameters. |
+| :::image type="icon" source="media/how-to-generate-reports/export-as-pdf-icon.png" border="false"::: | Export as PDF. |
+| :::image type="icon" source="media/how-to-generate-reports/csv-export-icon.png" border="false"::: |Export as CSV. |
+| :::image type="icon" source="media/how-to-generate-reports/information-icon.png" border="false"::: | Show additional information such as the date last modified. Use this feature to create a query result snapshot. You might need to do this for further investigation with team leaders or SOC analysts, for example. |
+| :::image type="icon" source="media/how-to-generate-reports/pin-icon.png" border="false"::: | Display on the **Reports** page or hide on the **Reports** page. :::image type="content" source="media/how-to-generate-reports/hide-reports-page.png" alt-text="Hide or reveal your reports."::: |
+| :::image type="icon" source="media/how-to-generate-reports/delete-simulation-icon.png" border="false"::: | Delete the report. |
+
+#### Create customized directories
+
+You can organize the extensive information for data-mining queries by creating directories for categories. For example, you can create directories for protocols or locations.
+
+To create a new directory:
+
+1. Select :::image type="icon" source="media/how-to-generate-reports/plus-icon.png" border="false"::: to add a new directory.
+
+2. Select **New Directory** to display the new directory form.
+
+3. Name the new directory.
+
+4. Drag the required reports into the new directory. At any time, you can drag the report back to the main view.
+
+5. Right-click the new directory to open, edit, or delete it.
+
+#### Create snapshots of report results
+
+You might need to save certain query results for further investigation. For example, you might need to share results with various security teams.
+
+Snapshots are saved within the report results and don't generate dynamic queries.
+
+:::image type="content" source="media/how-to-generate-reports/report-results-report.png" alt-text="Snapshots.":::
+
+To create a snapshot:
+
+1. Open the required report.
+
+2. Select the information icon :::image type="icon" source="media/how-to-generate-reports/information-icon.png" border="false":::.
+
+3. Select **Take New**.
+
+4. Enter a name for the snapshot and select **Save**.
+
+:::image type="content" source="media/how-to-generate-reports/take-a-snapshot.png" alt-text="Take a snapshot.":::
+
+#### Customize the CVE list
+
+You can manually customize the CVE list as follows:
+
+ - Include/exclude CVEs
+
+ - Change the CVE score
+
+To perform manual changes in the CVE report:
+
+1. From the side menu, select **Data Mining**.
+
+2. Select :::image type="icon" source="media/how-to-generate-reports/plus-icon.png" border="false"::: in the upper-left corner of the **Data Mining** window. Then select **New Report**.
+
+ :::image type="content" source="media/how-to-generate-reports/create-a-new-report-screen.png" alt-text="Create a new report.":::
+
+3. From the left pane, select one of the following options:
+
+ - **Known Vulnerabilities**: Selects both options and presents results in the report's two tables, one with CVEs and the other with excluded CVEs.
+
+ - **CVEs**: Select this option to present a list of all the CVEs.
+
+ - **Excluded CVEs**: Select this option to presents a list of all the excluded CVEs.
+
+4. Fill in the **Name** and **Description** information and select **Save**. The new report appears in the **Data Mining** window.
+
+5. To exclude CVEs, open the data-mining report for CVEs. The list of all the CVEs appears.
+
+ :::image type="content" source="media/how-to-generate-reports/cves.png" alt-text="C V E report.":::
+
+6. To enable selecting items in the list, select :::image type="icon" source="media/how-to-generate-reports/enable-selecting-icon.png" border="false"::: and select the CVEs that you want to customize. The **Operations** bar appears on the bottom.
+
+ :::image type="content" source="media/how-to-generate-reports/operations-bar-appears.png" alt-text="Screenshot of the data-mining Operations bar.":::
+
+7. Select the CVEs that you want to exclude, and then select **Delete Records**. The CVEs that you've selected don't appear in the list of CVEs and will appear in the list of excluded CVEs when you generate one.
+
+8. To include the excluded CVEs in the list of CVEs, generate the report for excluded CVEs and delete from that list the items that you want to include back in the list of CVEs.
+
+9. Select the CVEs in which you want to change the score, and then select **Update CVE Score**.
+
+ :::image type="content" source="media/how-to-generate-reports/set-new-score-screen.png" alt-text="Update the CVE score.":::
+
+10. Enter the new score and select **OK**. The updated score appears in the CVEs that you selected.
+
+### Reports based on data mining
+
+Reports reflect information generated by data mining query results. This includes default data mining reports, which are available in the Reports view. Administrator and security analysts can also generate custom data mining queries, and save them as reports. These reports are available for RO users as well.
+
+To generate a report:
+
+1. Select **Reports** on the side menu.
+
+2. Choose the required report to display. The choice can be **Custom** or **Auto-Generated** reports, such as **Programming Commands** and **Remote Access**.
+
+3. You can export the report by selecting one of the icons on the upper right of the screen:
+
+ :::image type="icon" source="media/how-to-generate-reports/export-to-pdf-icon.png" border="false"::: Export to a PDF file.
+
+ :::image type="icon" source="media/how-to-generate-reports/export-to-csv-icon.png" border="false"::: Export to a CSV file.
+
+> [!NOTE]
+> The RO user can see only reports created for them.
+
+:::image type="content" source="media/how-to-generate-reports/select-a-report-screen.png" alt-text="Select the report to generate.":::
+
+:::image type="content" source="media/how-to-generate-reports/remote-access-report.png" alt-text="Remote access report generated.":::
+
+## Reports for sensor trends and statistics
+
+You can create widget graphs and pie charts to gain insight into network trends and statistics. Widgets can be grouped under user-defined dashboards.
+
+> [!NOTE]
+> Only administrators and security analysts can perform the procedures in this section.
+
+To see dashboards and widgets, select **Trends & Statistics** on the side menu.
+
+:::image type="content" source="media/how-to-generate-reports/investigation-screenshot.png" alt-text="Screenshot of an investigation.":::
+
+The dashboard consists of widgets that graphically describe the following types of information:
+
+- Traffic by port
+- Channel bandwidth
+- Total bandwidth
+- Active TCP connection
+- Devices:
+ - New devices
+ - Busy devices
+ - Devices by vendor
+ - Devices by OS
+ - Disconnected devices
+- Connectivity drop by hours
+- Alerts for incidents by type
+- Database table access
+- Protocol dissection widgets
+- Ethernet and IP address:
+ - Ethernet and IP address traffic by CIP service
+ - Ethernet and IP address traffic by CIP class
+ - Ethernet and IP address traffic by command
+- OPC:
+ - OPC top management operations
+ - OPC top I/O operations
+- Siemens S7:
+ - S7 traffic by control function
+ - S7 traffic by subfunction
+- SRTP:
+ - SRTP traffic by service code
+ - SRTP errors by day
+- SuiteLink:
+ - SuiteLink top queried tags
+ - SuiteLink numeric tag behavior
+- IEC-60870 traffic by ASDU
+- DNP3 traffic by function
+- MMS traffic by service
+- Modbus traffic by function
+- OPC-UA traffic by service
+
+> [!NOTE]
+> The time in the widgets is set according to the sensor time.
+
+## Reports for the on-premises management console
+
+The on-premises management console lets you generate reports for each sensor that's connected to it. Reports are based on sensor data-mining queries that are performed.
+
+You can generate the following reports:
+
+- **Active Devices (Last 24 Hours)**: Presents a list of devices that show network activity within a period of 24 hours.
+
+- **Non-Active Devices (Last 7 Days)**: Presents a list of devices that show no network activity in the last seven days.
+
+- **Programming Commands**: Presents a list of devices that sent programming commands within the last 24 hours.
+
+- **Remote Access**: Presents a list of devices that remote sources accessed within the last 24 hours.
+
+:::image type="content" source="media/how-to-generate-reports/reports-view.png" alt-text="Screenshot of the reports view.":::
+
+When you choose the sensor from the on-premises management console, all the custom reports configured on that sensor appear in the list of reports. For each sensor, you can generate a default report or a custom report configured on that sensor.
+
+To generate a report:
+
+1. On the left pane, select **Reports**. The **Reports** window appears.
+
+2. From the **Sensors** drop-down list, select the sensor for which you want to generate the report.
+
+ :::image type="content" source="media/how-to-generate-reports/sensor-drop-down-list.png" alt-text="Screenshot of sensors view.":::
+
+3. From the right drop-down list, select the report that you want to generate.
+
+4. To create a PDF of the report results, select :::image type="icon" source="media/how-to-generate-reports/pdf-report-icon.png" border="false":::.
+
+## Risk assessment reports for the on-premises management console
+
+Generate a risk assessment report for each sensor that's connected to your on-premises management console. This report provides insight into each of the networks that your sensors are monitoring.
+
+Risk assessment reports let you generate a security score for each network device, as well as an overall network security score. The overall score is based on deep packet inspection, behavioral modeling engines, and a SCADA-specific state machine design. An extensive range of other information is available. For example:
+
+- Configuration issues
+
+- Device vulnerability prioritized by security level
+
+- Network security issues
+
+- Network operational issues
+
+- Attack vectors
+
+- Connections to ICS networks
+
+- Internet connections
+
+- Industrial malware indicators
+
+- Protocol issues
+
+The report provides mitigation recommendations that will help you improve your security score.
+
+- Secure devices: Devices with a security score above 90%.
+
+- **Vulnerable devices**: Devices with a security score below 70 percent.
+
+- **Devices needing improvement**: Devices with a security score between 70 percent and 89 percent.
+
+To create a report:
+
+1. Select **Risk Assessment** on the side menu.
+
+2. Select a sensor from the **Select sensor** drop-down list.
+
+3. Select **Generate Report**.
+
+4. Select **Download** from the **Archived Reports** section.
+
+To import a company logo:
+
+- Select **Import Logo**.
+
+:::image type="content" source="media/how-to-generate-reports/import-logo-screenshot.png" alt-text="Import your logo through the risk assessment view.":::
+
+## See also
+[Work with site map views](how-to-gain-insight-into-global-regional-and-local-threats.md#work-with-site-map-views)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-identify-required-appliances https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-identify-required-appliances.md new file mode 100644
@@ -0,0 +1,279 @@
+---
+title: Identify required appliances
+description: Learn about hardware and virtual appliances for certified Defender for IoT sensors and the on-premises management console.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/21/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Identify required appliances
+
+This article provides information on certified Defender for IoT sensor appliances. Defender fort IoT can be deployed on physical and virtual appliances.
+
+This includes certified *pre-configured* appliances, on which software is already installed, as well as non-configured certified appliances on which you can download and install required software.
+
+The article also provides specifications for an on-premises management console appliance. The on-premises management console is not available as a preconfigured appliance.
+
+- If you want to purchase a pre-configured sensor, review the models available in the [Sensor appliances](#sensor-appliances) section and then proceed with the purchase.
+
+- If you want to purchase your own appliance, review the models available in the [Sensor appliances](#sensor-appliances) section and in the [Additional certified appliances](#additional-certified-appliances) section. After you acquire the appliance, you can download and install the software.
+
+- If you want to purchase the on-premises management console, review the information in the [On-premises management console appliance](#on-premises-management-console-appliance) section. After you acquire the device, you can download and install the software.
+
+After you've completed the tasks here, you can install the software and set up your network.
+
+## Sensor appliances
+
+Defender for IoT supports both physical and virtual deployments.
+
+### Physical sensors
+
+This section provides an overview of physical sensor models that are available. You can purchase sensors with preconfigured software or purchase sensors that are not preconfigured.
+
+| Deployment type | Corporate | Enterprise | SMB |
+|--|--|--|--|
+| Image | :::image type="content" source="media/how-to-prepare-your-network/corporate-hpe-proliant-dl360-v2.png" alt-text="The corporate-level model."::: | :::image type="content" source="media/how-to-prepare-your-network/enterprise-and-smb-hpe-proliant-dl20-v2.png" alt-text="The enterprise-level model."::: | :::image type="content" source="media/how-to-prepare-your-network/enterprise-and-smb-hpe-proliant-dl20-v2.png" alt-text="The SMB-level model."::: |
+| Model | HPE ProLiant DL360 | HPE ProLiant DL20 | HPE ProLiant DL20 |
+| Monitoring ports | Up to 15 RJ45 or 8 OPT | Up to 8 RJ45 or 6 OPT | 4 RJ45 |
+| Maximum bandwidth [1](#anchortext) | 3 Gb per second | 1 Gb per second | 200 Mb per second |
+| Maximum protected devices | 30,000 | 15,000 | 1,000 |
+
+See [Appliance specifications](#appliance-specifications) for vendor details.
+
+About preconfigured sensors: Microsoft has partnered with Arrow to provide preconfigured sensors. To purchase a preconfigured sensor, contact Arrow at the following address: <hardware.sales@arrow.com>
+
+About bringing your own appliance: Review supported models described here. After you've acquired your appliance, go to **Defender for IoT** > **Network Sensors ISO** > **Installation** to download the software.
+
+:::image type="content" source="media/how-to-prepare-your-network/azure-defender-for-iot-sensor-download-software-screen.png" alt-text="Network sensors ISO.":::
+
+<a id="anchortext">1</a> Bandwidth capacity can vary, depending on the distribution of protocols.
+
+### Virtual sensors
+
+This section provides an overview of the virtual sensors that are available.
+
+| Deployment type | Corporate | Enterprise | SMB |
+|--|--|--|--|
+| Maximum bandwidth | 2.5 Gb/sec | 800 Mb/sec | 160 Mb/sec |
+| Maximum protected devices | 30,000 | 10,000 | 2,500 |
+
+## On-premises management console appliance
+
+The management console is available as a virtual deployment.
+
+| Deployment type | Enterprise |
+|--|--|
+| Appliance type | HPE DL20, VM |
+| Number of managed sensors | Up to 300 |
+
+After you acquire an on-premises management console, go to **Defender for IoT** > **On-premises management console** > **ISO Installation** to download the ISO.
+
+:::image type="content" source="media/how-to-prepare-your-network/azure-defender-for-iot-iso-download-screen.png" alt-text="On-premises management console.":::
+
+## Appliance specifications
+
+This section describes hardware specifications for the following appliances:
+
+- Corporate deployment: HPE ProLiant DL360
+
+- Enterprise deployment: HPE ProLiant DL20
+
+- SMB deployment: HPE ProLiant DL20
+
+- Virtual appliance specifications
+
+## Corporate deployment: HPE ProLiant DL360
+
+| Component | Technical specifications |
+|--|--|
+| Chassis | 1U rack server |
+| Dimensions | 42.9 x 43.46 x 70.7 (cm)/1.69" x 17.11" x 27.83" (in) |
+| Weight | Max 16.27 kg (35.86 lb) |
+| Processor | Intel Xeon Silver 4215 R 3.2 GHz, 11M cache, 8c/16T, 130 W |
+| Chipset | Intel C621 |
+| Memory | 32 GB = 2 x 16-GB 2666MT/s DDR4 ECC UDIMM |
+| Storage | 6 x 1.2-TB SAS 12G Enterprise 10K SFF (2.5 in) in Hot-Plug Hard Drive - RAID 5 |
+| Network controller | On-board: 2 x 1-Gb Broadcom BCM5720<br>On-board LOM: iDRAC Port Card 1-Gb Broadcom BCM5720<br><br>External: 1 x Intel Ethernet i350 QP 1-Gb Server Adapter, Low Profile |
+| Management | HPE iLO Advanced |
+| Device access | Two rear USB 3.0<br>One front USB 2.0<br>One internal USB 3.0 |
+| Power | 2 x HPE 500 W Flex Slot Platinum Hot Plug Low Halogen Power Supply Kit |
+| Rack support | HPE 1U Gen10 SFF Easy Install Rail Kit |
+
+### Appliance BOM
+
+| PN | Description | Quantity |
+|--|--|--|
+| P19766-B21 | HPE DL360 Gen10 8SFF NC CTO Server | 1 |
+| P19766-B21 | Europe - Multilingual Localization | 1 |
+| P24479-L21 | Intel Xeon-S 4215 R FIO Kit for DL360 G10 | 1 |
+| P24479-B21 | Intel Xeon-S 4215 R Kit for DL360 Gen10 | 1 |
+| P00922-B21 | HPE 16-GB 2Rx8 PC4-2933Y-R Smart Kit | 2 |
+| 872479-B21 | HPE 1.2-TB SAS 10K SFF SC DS HDD | 6 |
+| 811546-B21 | HPE 1-GbE 4-p BASE-T I350 Adapter | 1 |
+| P02377-B21 | HPE Smart Hybrid Capacitor w\_ 145 mm Cable | 1 |
+| 804331-B21 | HPE Smart Array P408i-a SR Gen10 Controller | 1 |
+| 665240-B21 | HPE 1-GbE 4-p FLR-T I350 Adapter | 1 |
+| 871244-B21 | HPE DL360 Gen10 High Performance Fan Kit | 1 |
+| 865408-B21 | HPE 500-W FS Plat Hot Plug LH Power Supply Kit | 2 |
+| 512485-B21 | HPE iLO Adv 1-Server License 1 Year Support | 1 |
+| 874543-B21 | HPE 1U Gen10 SFF Easy Install Rail Kit | 1 |
+
+## Enterprise deployment: HPE ProLiant DL20
+
+| Component | Technical specifications |
+|--|--|
+| Chassis | 1U rack server |
+| Dimensions (height x width x depth) | 4.32 x 43.46 x 38.22 cm/1.70 x 17.11 x 15.05 inch |
+| Weight | 7.9 kg/17.41 lb |
+| Processor | Intel Xeon E-2234, 3.6 GHz, 4C/8T, 71 W |
+| Chipset | Intel C242 |
+| Memory | 2 x 16-GB Dual Rank x8 DDR4-2666 |
+| Storage | 3 x 1-TB SATA 6G Midline 7.2 K SFF (2.5 in) ΓÇô RAID 5 with Smart Array P408i-a SR Controller |
+| Network controller | On-board: 2 x 1 Gb <br>On-board: iLO Port Card 1 Gb <br>External: 1 x HPE Ethernet 1-Gb 4-port 366FLR Adapter |
+| Management | HPE iLO Advanced |
+| Device access | Front: 1 x USB 3.0, 1 x USB iLO Service Port <br>Rear: 2 x USB 3.0 <br>Internal: 1 x USB 3.0 |
+| Power | Dual Hot Plug Power Supplies 500 W |
+| Rack support | HPE 1U Short Friction Rail Kit |
+
+### Appliance BOM
+
+| PN | Description: high end | Quantity |
+|--|--|--|
+| P06963-B21 | HPE DL20 Gen10 4SFF CTO Server | 1 |
+| P06963-B21 | HPE DL20 Gen10 4SFF CTO Server | 1 |
+| P17104-L21 | HPE DL20 Gen10 E-2234 FIO Kit | 1 |
+| 879507-B21 | HPE 16-GB 2Rx8 PC4-2666V-E STND Kit | 2 |
+| 655710-B21 | HPE 1-TB SATA 7.2 K SFF SC DS HDD | 3 |
+| P06667-B21 | HPE DL20 Gen10 x8x16 FLOM Riser Kit | 1 |
+| 665240-B21 | HPE Ethernet 1-Gb 4-port 366FLR Adapter | 1 |
+| 782961-B21 | HPE 12-W Smart Storage Battery | 1 |
+| 869081-B21 | HPE Smart Array P408i-a SR G10 LH Controller | 1 |
+| 865408-B21 | HPE 500-W FS Plat Hot Plug LH Power Supply Kit | 2 |
+| 512485-B21 | HPE iLO Adv 1-Server License 1 Year Support | 1 |
+| P06722-B21 | HPE DL20 Gen10 RPS Enablement FIO Kit | 1 |
+| 775612-B21 | HPE 1U Short Friction Rail Kit | 1 |
+
+## SMB deployment: HPE ProLiant DL20
+
+| Component | Technical specifications |
+|--|--|
+| Chassis | 1U rack server |
+| Dimensions (height x width x depth) | 4.32 x 43.46 x 38.22 cm/1.70 x 17.11 x 15.05 inch |
+| Weight | 7.88 kg/17.37 lb |
+| Processor | Intel Xeon E-2224, 3.4 GHz, 4C, 71 W |
+| Chipset | Intel C242 |
+| Memory | 1 x 8-GB Dual Rank x8 DDR4-2666 |
+| Storage | 2 x 1-TB SATA 6G Midline 7.2 K SFF (2.5 in) ΓÇô RAID 1 with Smart Array P208i-a |
+| Network controller | On-board: 2 x 1 Gb <br>On-board: iLO Port Card 1 Gb <br>External: 1 x HPE Ethernet 1-Gb 4-port 366FLR Adapter |
+| Management | HPE iLO Advanced |
+| Device access | Front: 1 x USB 3.0, 1 x USB iLO Service Port <br>Rear: 2 x USB 3.0 <br>Internal: 1 x USB 3.0 |
+| Power | Hot Plug Power Supply 290 W |
+| Rack support | HPE 1U Short Friction Rail Kit |
+
+### Appliance BOM
+
+| PN | Description | Quantity |
+|--|--|--|
+| P06961-B21 | HPE DL20 Gen10 NHP 2LFF CTO Server | 1 |
+| P06961-B21 | HPE DL20 Gen10 NHP 2LFF CTO Server | 1 |
+| P17102-L21 | HPE DL20 Gen10 E-2224 FIO Kit | 1 |
+| 879505-B21 | HPE 8-GB 1Rx8 PC4-2666V-E STND Kit | 1 |
+| 801882-B21 | HPE 1-TB SATA 7.2 K LFF RW HDD | 2 |
+| P06667-B21 | HPE DL20 Gen10 x8x16 FLOM Riser Kit | 1 |
+| 665240-B21 | HPE Ethernet 1-Gb 4-port 366FLR Adapter | 1 |
+| 869079-B21 | HPE Smart Array E208i-a SR G10 LH Controller | 1 |
+| P21649-B21 | HPE DL20 Gen10 Plat 290 W FIO PSU Kit | 1 |
+| P06683-B21 | HPE DL20 Gen10 M.2 SATA/LFF AROC Cable Kit | 1 |
+| 512485-B21 | HPE iLO Adv 1-Server License 1 Year Support | 1 |
+| 775612-B21 | HPE 1U Short Friction Rail Kit | 1 |
+
+## Virtual appliance specifications
+
+### Sensors
+
+| Type | Corporate | Enterprise | SMB |
+|--|--|--|--|
+| vCPU | 32 | 8 | 4 |
+| Memory | 32 GB | 32 GB | 8 GB |
+| Storage | 5.6 TB | 1.8 TB | 500 GB |
+
+### On-premises management console appliance
+
+| Type | Enterprise |
+|--|--|
+| Description | Virtual appliance for enterprise deployment types |
+| vCPU | 8 |
+| Memory | 32 GB |
+| Storage | 1.8 TB |
+
+Supported hypervisors: VMware ESXi version 5.0 and later, Hyper-V
+
+## Additional certified appliances
+
+This section details additional appliances that were certified by Microsoft but are not offered as preconfigured appliances.
+
+| Deployment type | Enterprise |
+|--|--|
+| Image | :::image type="content" source="media/how-to-prepare-your-network/deployment-type-enterprise-for-azure-defender-for-iot-v2.png" alt-text="Enterprise deployment type."::: |
+| Model | Dell PowerEdge R340 XL |
+| Monitoring ports | Up to nine RJ45 or six OPT |
+| Max bandwidth [1](#anchortext2)| 1G Mb/sec |
+| Max protected devices | 10,000 |
+
+<a id="anchortext2">One</a> Bandwidth capacity can vary, depending on protocols distribution.
+
+After you purchase the appliance, go to **Defender for IoT** > **Network Sensors ISO** > **Installation** to download the software.
+
+:::image type="content" source="media/how-to-prepare-your-network/azure-defender-for-iot-sensor-download-software-screen.png" alt-text="Network sensors ISO.":::
+
+## Enterprise deployment: Dell PowerEdge R340 XL
+
+| Component | Technical specifications |
+|--|--|
+| Chassis | 1U rack server |
+| Dimensions | 42.8 x 434.0 x 596 (mm) /1.67" x 17.09" x 23.5" (in) |
+| Weight | Max 29.98 lb/13.6 kg |
+| Processor | Intel Xeon E-2144G 3.6 GHz, 8M cache, 4C/8T, turbo (71 W) |
+| Chipset | Intel C246 |
+| Memory | 32 GB = 2 x 16-GB 2666MT/s DDR4 ECC UDIMM |
+| Storage | 3 x 2-TB 7.2 K RPM SATA 6-Gbps 512n 3.5-in Hot-Plug Hard Drive - RAID 5 |
+| Network controller | On-board: 2 x 1-Gb Broadcom BCM5720<br>On-board LOM: iDRAC Port Card 1-Gb Broadcom BCM5720 <br><br>External: 1 x Intel Ethernet i350 QP 1-Gb Server Adapter, Low Profile |
+| Management | iDRAC nine Enterprise |
+| Device access | Two rear USB 3.0 <br> One front USB 3.0 |
+| Power | Dual Hot Plug Power Supplies 350 W |
+| Rack support | ReadyRails II sliding rails for tool-less mounting in 4-post racks with square or unthreaded round holes or tooled mounting in 4-post threaded hole racks, with support for optional tool-less cable management arm. |
+
+## Dell R340 BOM
+
+:::image type="content" source="media/how-to-prepare-your-network/enterprise-deployment-for-azure-defender-for-iot-dell-r340-bom.png" alt-text="Dell R340 BOM.":::
+
+## SMB deployment: Neousys Nuvo-5006LP
+
+| Component | Technical specifications |
+|--|--|
+| Construction | Aluminum, fanless and dust-proof design |
+| Dimensions | 240 mm (W) x 225 mm (D) x 77 mm (H) |
+| Weight | 3.1 kg (including CPU, memory, and HDD) |
+| CPU | Intel Core i5-6500TE (6M Cache, up to 3.30 GHz) S1151 |
+| Chipset | Intel Q170 Platform Controller Hub |
+| Memory | 8-GB DDR4 2133 MHz Wide Temperature SODIMM |
+| Storage | 128-GB 3ME3 Wide Temperature mSATA SSD |
+| Network controller | 6x Gigabit Ethernet ports by Intel I219 |
+| Device access | 4 USBs: Two fronts, two rears, one internal |
+| Power adapter | 120/240VAC-20VDC/6A |
+| Mounting | Mounting kit, DIN rail |
+| Operating Temperature | \-25┬░C ~ 70┬░C |
+| Storage Temperature | \-40┬░C ~ 85┬░C |
+| Humidity | 10% ~ 90%, non-condensing |
+| Vibration | Operating, 5 Grms, 5-500 Hz, 3 axes <br>(w/ SSD, according to IEC60068-2-64) |
+| Shock | Operating, 50 Grms, half-sine 11-ms duration (w/ SSD, according to IEC60068-2-27) |
+| EMC | CE/FCC Class A, according to EN 55022, EN 55024, and EN 55032 |
+
+## Next steps
+
+[About Azure Defender for IoT installation](how-to-install-software.md)
+
+[About Azure Defender for IoT network setup](how-to-set-up-your-network.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-import-device-information https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-import-device-information.md new file mode 100644
@@ -0,0 +1,190 @@
+---
+title: Import device information
+description: Defender for IoT sensors monitor and analyze mirrored traffic. In these cases, you might want to import data to enrich information on devices already detected.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/06/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Import device information to a sensor
+
+An Azure Defender for IoT sensor monitors and analyzes mirrored traffic. In some cases, because of organization-specific network configuration policies, some information might not be transmitted.
+
+In these cases, you might want to import data to enrich information on devices that are already detected. Two options are available for importing information to sensors:
+
+- **Import from the Map**: Update the device name, type, group, or Purdue layer to the map.
+
+- **Import from Import Settings**: Import device OS, IP address, patch level, or authorization status.
+
+## Import from the map
+
+This section describes how to import device names, types, groups, or Purdue layers to the device map. You do this from the map.
+
+Here are the import requirements:
+
+- **Names**: Can be up to 30 characters.
+
+- **Type** or **Purdue Layer**: Use the options that appear in the **Device Properties** dialog box. (Right-click the device and select **View Properties**.)
+
+- **Device Group**: Create a new group of up to 30 characters.
+
+> [!NOTE]
+> To avoid conflicts, don't import the data that you exported from one sensor to another sensor.
+
+To import:
+
+1. On the side menu, select **Devices**.
+
+2. In the upper-right corner of the **Devices** window, select :::image type="icon" source="media/how-to-import-device-information/file-icon.png" border="false":::.
+
+ :::image type="content" source="media/how-to-import-device-information/device-window-v2.png" alt-text="Screenshot of the device window.":::
+
+3. Select **Export Devices**. An extensive range of information appears in the exported file. This information includes protocols that the device uses and the device authorization status.
+
+ :::image type="content" source="media/how-to-import-device-information/sample-exported-file.png" alt-text="The information in the exported file.":::
+
+4. In the CSV file, change only the device name, type, group, and Purdue layer. Then save the file.
+
+ Use capitalization standards shown in the exported file. For example, for the Purdue layer, use all first-letter capitalization.
+
+5. From the **Import/Export** drop-down menu in the **Device** window, select **Import Devices**.
+
+ :::image type="content" source="media/how-to-import-device-information/import-assets-v2.png" alt-text="Import devices through the device window.":::
+
+6. Select **Import Devices** and select the CSV file that you want to import. The import status messages appear on the screen until the **Import Devices** dialog box closes.
+
+## Import from import settings
+
+This section describes how to import the device IP address, OS, patch level, or authorization status to the device map. You do this from the **Import Settings** dialog box.
+
+To import the IP address, OS, and patch level:
+
+1. Download the [assets_info_2.2.8 and up.csv](https://cyberx-labs.zendesk.com/hc/en-us/articles/360008658272-How-To-Import-Data) file from the [Help Center](https://cyberx-labs.zendesk.com/hc/en-us) and enter the information as follows:
+
+ - **IP Address**: Enter the device IP address.
+
+ - **Operating System**: Select from the drop-down list.
+
+ - **Last Update**: Use the YYYY-MM-DD format.
+
+ :::image type="content" source="media/how-to-import-device-information/last-update-screen.png" alt-text="The options screen.":::
+
+2. On the side menu, select **Import Settings**.
+
+ :::image type="content" source="media/how-to-import-device-information/import-settings-screen-v2.png" alt-text="Import your settings.":::
+
+3. To upload the required configuration, in the **Device Info** section, select **Add** and upload the CSV file that you prepared.
+
+To import the authorization status:
+
+1. Download and save the [authorized_assets.csv](https://cyberx-labs.zendesk.com/hc/en-us/articles/360008658272-How-To-Import-Data) file from the Defender for IoT help center. Verify that you saved the file as a CSV.
+
+2. Enter the information as:
+
+ - **IP Address**: The device IP address.
+
+ - **Name**: The authorized device name. Make sure that names are accurate. Names given to the devices in the imported list overwrite names shown in the device map.
+
+ :::image type="content" source="media/how-to-import-device-information/device-map-file.png" alt-text="Excel files with imported device list.":::
+
+3. On the side menu, select **Import Settings**.
+
+4. In the **Authorized Devices** section, select **Add** and upload the CSV file that you saved.
+
+When the information is imported, you receive alerts about unauthorized devices for all the devices that don't appear on this list.
+
+## Import device information to the sensor
+
+The sensor monitors and analyzes mirrored traffic. In some cases, because of organization-specific network configuration policies, some information might not be transmitted.
+
+In these cases, you might want to import data to enrich device information on devices that are already detected. Two options are available for importing information to sensors:
+
+- **Import from the Map**: Update the device name, type, group, or Purdue layer to the map.
+
+- **Import from Import Settings**: Import device OS, IP address, patch level, or authorization status.
+
+### Import from the map
+
+This section describes how to import device names, types, groups, or Purdue layers to the device map. You do this from the map.
+
+Here are the import requirements:
+
+- **Names**: Can be up to 30 characters.
+
+- **Type** or **Purdue Layer**: Use the options that appear in the **Device Properties** dialog box. (Right-click the device and select **View Properties**.)
+
+- **Device Group**: Create a new group of up to 30 characters.
+
+> [!NOTE]
+> To avoid conflicts, don't import the data that you exported from one sensor to another sensor.
+
+To import:
+
+1. On the side menu, select **Devices**.
+
+2. In the upper-right corner of the **Devices** window, select :::image type="icon" source="media/how-to-import-device-information/file-icon.png" border="false":::.
+
+ :::image type="content" source="media/how-to-import-device-information/device-window-v2.png" alt-text="The window to pick your device from.":::
+
+3. Select **Export Devices**. An extensive range of information appears in the exported file. Examples include protocols that the device uses and the device authorization status.
+
+ :::image type="content" source="media/how-to-import-device-information/sample-exported-file.png" alt-text="The information in the exported file.":::
+
+4. In the CSV file, only change the device name, type, group, and Purdue layer. Then save the file.
+
+ Use capitalization standards shown in the exported file. For example, for the Purdue layer, use all first-letter capitalization.
+
+5. From the **Import/Export** drop-down menu in the **device** window, select **Import Devices**.
+
+ :::image type="content" source="media/how-to-import-device-information/import-assets-v2.png" alt-text="Import your devices.":::
+
+6. Select **Import Devices** and select the CSV file that you want to import. The import status messages appear on the screen until the **Import Devices** dialog box closes.
+
+### Import from import settings
+
+This section describes how to import the device IP address, OS, patch level, or authorization status to the device map. You do this from the **Import Settings** dialog box.
+
+To import the IP address, OS, and patch level:
+
+1. Download the [assets_info_2.2.8 and up.csv](https://cyberx-labs.zendesk.com/hc/en-us/articles/360008658272-How-To-Import-Data) file from the [Help Center](https://cyberx-labs.zendesk.com/hc/en-us) and enter the information as follows:
+
+ - **IP Address**: The device IP address.
+
+ - **Operating System**: Select from the drop-down list.
+
+ - **Last Update**: Use the YYYY-MM-DD format.
+
+ :::image type="content" source="media/how-to-import-device-information/last-update-screen.png" alt-text="The content on the screen.":::
+
+2. On the side menu, select **Import Settings**.
+
+ :::image type="content" source="media/how-to-import-device-information/import-settings-screen-v2.png" alt-text="Fill out the import settings screen.":::
+
+3. To upload the required configuration, in the **Device Info** section, select **Add** and upload the CSV file that you've prepared.
+
+To import the authorization status:
+
+1. Download and save the [authorized_assets.csv](https://cyberx-labs.zendesk.com/hc/en-us/articles/360008658272-How-To-Import-Data) file from the Defender for IoT help center. Verify that you saved the file as a CSV.
+
+2. Enter the information as:
+
+ - **IP Address**: The device IP address.
+
+ - **Name**: The authorized device name. Verify that names are accurate. Names given to the devices in the imported list overwrite names shown in the device map.
+
+ :::image type="content" source="media/how-to-import-device-information/device-map-file.png" alt-text="The import list to the device map.":::
+
+3. On the side menu, select **Import Settings**.
+
+4. In the **Authorized Devices** section, select **Add** and upload the CSV file that you saved.
+
+When the information is imported, you receive alerts about unauthorized devices for all the devices that don't appear on this list.
+
+## See also
+
+[Control what traffic is monitored](how-to-control-what-traffic-is-monitored.md)
+
+[Investigate sensor detections in a device inventory](how-to-investigate-sensor-detections-in-a-device-inventory.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-install-software https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-install-software.md new file mode 100644
@@ -0,0 +1,1071 @@
+---
+title: Defender for IoT installation
+description: Learn how to install a sensor and the on-premises management console for Azure Defender for IoT.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/2/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Defender for IoT installation
+
+This article describes how to install the following elements of Azure Defender for IoT:
+
+- **Sensor**: Defender for IoT sensors collects ICS network traffic by using passive (agentless) monitoring. Passive and nonintrusive, the sensors have zero impact on OT and IoT networks and devices. The sensor connects to a SPAN port or network TAP and immediately begins monitoring your network. Detections appear in the sensor console. There, you can view, investigate, and analyze them in a network map, device inventory, and an extensive range of reports. Examples include risk assessment reports, data mining queries, and attack vectors. Read more about sensor capabilities in the [Defender for IoT Sensor User Guide (direct download)](https://aka.ms/AzureDefenderforIoTUserGuide).
+
+- **On-premises management console**: The on-premises management console lets you carry out device management, risk management, and vulnerability management. You can also use it to carry out threat monitoring and incident response across your enterprise. It provides a unified view of all network devices, key IoT, and OT risk indicators and alerts detected in facilities where sensors are deployed. Use the on-premises management console to view and manage sensors in air-gapped networks.
+
+This article covers the following installation information:
+
+ - **Hardware:** Dell and HPE physical appliance details.
+
+ - **Software:** Sensor and on-premises management console software installation.
+
+ - **Virtual Appliances:** Virtual machine details and software installation.
+
+After installation, connect your sensor to your network.
+
+## About Defender for IoT appliances
+
+The following sections provide information about Defender for IoT sensor appliances and the appliance for the Defender for IoT on-premises management console.
+
+### Physical appliances
+
+The Defender for IoT appliance sensor connects to a SPAN port or network TAP and immediately begins collecting ICS network traffic by using passive (agentless) monitoring. This process has zero impact on OT networks and devices because it isn't placed in the data path and doesn't actively scan OT devices.
+
+The following rack mount appliances are available:
+
+| **Deployment type** | **Corporate** | **Enterprise** | **SMB** | |
+|--|--|--|--|--|
+| **Model** | HPE ProLiant DL360 | Dell PowerEdge R340 XL | HPE ProLiant DL20 | HPE ProLiant DL20 |
+| **Monitoring ports** | up to 15 RJ45 or 8 OPT | up to 9 RJ45 or 6 OPT | up to 8 RJ45 or 6 OPT | 4 RJ45 |
+| **Max Bandwidth\*** | 3 Gb/Sec | 1 Gb/Sec | 1 Gb/Sec | 100 Mb/Sec |
+| **Max Protected Devices** | 30,000 | 10,000 | 15,000 | 1,000 |
+
+*Maximum bandwidth capacity might vary depending on protocol distribution.
+
+### Virtual appliances
+
+The following virtual appliances are available:
+
+| **Deployment type** | **Enterprise** | **SMB** | **Line** |
+|--|--|--|--|
+| **Description** | Virtual appliance for enterprise deployments | Virtual appliance for SMB deployments | Virtual appliance for line deployments |
+| **Max Bandwidth\*** | 150 Mb/sec | 15 Mb/sec | 3 Mb/sec |
+| **Max protected devices** | 3,000 | 300 | 100 |
+| **Deployment Type** | Enterprise | SMB | Line |
+| **Description** | Virtual appliance for enterprise deployments | Virtual appliance for SMB deployments | Virtual appliance for line deployments |
+
+*Maximum bandwidth capacity might vary depending on protocol distribution.
+
+### Hardware specifications for the on-premises management console
+
+ | Item | Description |
+ |----|--|
+ **Description** | In a multitier architecture, the on-premises management console delivers visibility and control across geographically distributed sites. It integrates with SOC security stacks, including SIEMs, ticketing systems, next-generation firewalls, secure remote access platforms, and the Defender for IoT ICS malware sandbox. |
+ **Deployment type** | Enterprise |
+ **Appliance type** | Dell R340, VM |
+ **Number of managed sensors** | Unlimited |
+
+## Prepare for the installation
+
+### Access the ISO installation image
+
+The installation image is accessible from the Defender for IoT portal.
+
+To access the file:
+
+1. Sign in to your Defender for IoT account.
+
+2. Go to the **Network sensor** or **On-premises management console** page and select a version to download.
+
+### Install from DVD
+
+Before the installation, ensure you have:
+
+- A portable DVD drive with the USB connector.
+
+- An ISO installer image.
+
+To install:
+
+1. Burn the image to a DVD or prepare a disk on a key. Connect a portable DVD drive to your computer, right-click the ISO image, and select **Burn to disk**.
+
+1. Connect the DVD or disk on a key and configure the appliance to boot from DVD or disk on a key.
+
+### Install from disk on a key
+
+Before the installation, ensure you have:
+
+ - Rufus installed.
+
+ - A disk on key with USB version 3.0 and later. The minimum size is 4 GB.
+
+ - An ISO installer image file.
+
+The disk on a key will be erased in this process.
+
+To prepare a disk on a key:
+
+1. Run Rufus and select **SENSOR ISO**.
+
+2. Connect the disk on a key to the front panel.
+
+3. Set the BIOS of the server to boot from the USB.
+
+## Dell PowerEdgeR340XL installation
+
+Before installing the software on the Dell appliance, you need to adjust the appliance's BIOS configuration:
+
+ - [Dell PowerEdge R340 Front Panel](#dell-poweredge-r340-front-panel) and [Dell PowerEdge R340 Back Panel](#dell-poweredge-r340-back-panel) contains the description of front and back panels, along with information required for installation, such as drivers and ports.
+
+ - [Dell BIOS Configuration](#dell-bios-configuration) provides information about how to connect to the Dell appliance management interface and configure the BIOS.
+
+ - [Software Installation (Dell R340)](#software-installation-dell-r340) describes the procedure required to install the Defender for IoT sensor software.
+
+### Dell PowerEdge R340XL requirements
+
+To install the Dell PowerEdge R340XL appliance, you need:
+
+- Enterprise license for Dell Remote Access Controller (iDrac)
+
+- BIOS configuration XML
+
+- Server firmware versions:
+
+ - BIOS version 2.1.6
+
+ - iDrac version 3.23.23.23
+
+### Dell PowerEdge R340 front panel
+
+:::image type="content" source="media/tutorial-install-components/view-of-dell-poweredge-r340-front-panel.jpg" alt-text="Dell PowerEdge R340 front panel.":::
+
+ 1. Left control panel
+ 2. Optical drive (optional)
+ 3. Right control panel
+ 4. Information tag
+ 5. Drives
+
+### Dell PowerEdge R340 back panel
+
+:::image type="content" source="media/tutorial-install-components/view-of-dell-poweredge-r340-back-panel.jpg" alt-text="Dell PowerEdge R340 back panel.":::
+
+1. Serial port
+2. NIC port (Gb 1)
+3. NIC port (Gb 1)
+4. Half-height PCIe
+5. Full-height PCIe expansion card slot
+6. Power supply unit 1
+7. Power supply unit 2
+8. System identification
+9. System status indicator cable port (CMA) button
+10. USB 3.0 port (2)
+11. iDRAC9 dedicated network port
+12. VGA port
+
+### Dell BIOS configuration
+
+Dell BIOS configuration is required to adjust the Dell appliance to work with the software.
+
+The BIOS configuration is performed through a predefined configuration. The file is accessible from the [Help Center](https://help.cyberx-labs.com/).
+
+Import the configuration file to the Dell appliance. Before using the configuration file, you need to establish the communication between the Dell appliance and the management computer.
+
+The Dell appliance is managed by an integrated iDRAC with Lifecycle Controller (LC). The LC is embedded in every Dell PowerEdge server and provides functionality that helps you deploy, update, monitor, and maintain your Dell PowerEdge appliances.
+
+To establish the communication between the Dell appliance and the management computer, you need to define the iDRAC IP address and the management computer's IP address on the same subnet.
+
+When the connection is established, the BIOS is configurable.
+
+To configure Dell BIOS:
+
+1. [Configure the iDRAC IP address](#configure-idrac-ip-address)
+
+2. [Import the BIOS configuration file](#import-the-bios-configuration-file)
+
+#### Configure iDRAC IP address
+
+1. Power up the sensor.
+
+2. If the OS is already installed, select the F2 key to enter the BIOS configuration.
+
+3. Select **iDRAC Settings**.
+
+4. Select **Network**.
+
+ > [!NOTE]
+ > During the installation, you must configure the default iDRAC IP address and password mentioned in the following steps. After the installation, you change these definitions.
+
+5. Change the static IPv4 address to **10.100.100.250**.
+
+6. Change the static subnet mask to **255.255.255.0**.
+
+ :::image type="content" source="media/tutorial-install-components/idrac-network-settings-screen-v2.png" alt-text="Screenshot that shows the static subnet mask.":::
+
+7. Select **Back** > **Finish**.
+
+#### Import the BIOS configuration file
+
+This article describes how to configure the BIOS by using the configuration file.
+
+1. Plug in a PC with a static preconfigured IP address **10.100.100.200** to the **iDRAC** port.
+
+ :::image type="content" source="media/tutorial-install-components/idrac-port.png" alt-text="Screenshot of the preconfigured IP address port.":::
+
+2. Open a browser and enter **10.100.100.250** to connect to iDRAC web interface.
+
+3. Sign in with Dell default administrator privileges:
+
+ - Username: **root**
+
+ - Password: **calvin**
+
+4. The appliance's credentials are:
+
+ - Username: **cyberx**
+
+ - Password: **xhxvhttju,@4338**
+
+ The import server profile operation is initiated.
+
+ > [!NOTE]
+ > Before you import the file, make sure:
+ > - You're the only user who is currently connected to iDRAC.
+ > - The system is not in the BIOS menu.
+
+5. Go to **Configuration** > **Server Configuration Profile**. Set the following parameters:
+
+ :::image type="content" source="media/tutorial-install-components/configuration-screen.png" alt-text="Screenshot that shows the configuration of your server profile.":::
+
+ | Parameter | Configuration |
+ |--|--|
+ | Location Type | Select **Local**. |
+ | File Path | Select **Choose File** and add the configuration XML file. |
+ | Import Components | Select **BIOS, NIC, RAID**. |
+ | Maximum wait time | Select **20 minutes**. |
+
+6. Select **Import**.
+
+7. To monitor the process, go to **Maintenance** > **Job Queue**.
+
+ :::image type="content" source="media/tutorial-install-components/view-the-job-queue.png" alt-text="Screenshot that shows Job Queue.":::
+
+#### Manually configuring BIOS
+
+You need to manually configure the appliance BIOS if:
+
+- You did not purchase your appliance from Arrow.
+
+- You have an appliance, but do not have access to the XML configuration file.
+
+After you access the BIOS, go to **Device Settings**.
+
+To manually configure:
+
+1. Access the appliance BIOS directly by using a keyboard and screen, or use iDRAC.
+
+ - If the appliance is not a Defender for IoT appliance, open a browser and go to the IP address that was configured before. Sign in with the Dell default administrator privileges. Use **root** for the username and **calvin** for the password.
+
+ - If the appliance is a Defender for IoT appliance, sign in by using **cyberx** for the username and **xhxvhttju,@4338** for the password.
+
+2. After you access the BIOS, go to **Device Settings**.
+
+3. Choose the RAID-controlled configuration by selecting **Integrated RAID controller 1: Dell PERC\<PERC H330 Adapter\> Configuration Utility**.
+
+4. Select **Configuration Management**.
+
+5. Select **Create Virtual Disk**.
+
+6. In the **Select RAID Level** field, select **RAID5**. In the **Virtual Disk Name** field, enter **ROOT** and select **Physical Disks**.
+
+7. Select **Check All** and then select **Apply Changes**
+
+8. Select **Ok**.
+
+9. Scroll down and select **Create Virtual Disk**.
+
+10. Select the **Confirm** check box and select **Yes**.
+
+11. Select **OK**.
+
+12. Return to the main screen and select **System BIOS**.
+
+13. Select **Boot Settings**.
+
+14. For the **Boot Mode** option, select **BIOS**.
+
+15. Select **Back**, and then select **Finish** to exit the BIOS settings.
+
+### Software installation (Dell R340)
+
+The installation process takes about 20 minutes. After the installation, the system is restarted several times.
+
+To install:
+
+1. Verify that the version media is mounted to the appliance in one of the following ways:
+
+ - Connect the external CD or disk on a key with the release.
+
+ - Mount the ISO image by using iDRAC. After signing in to iDRAC, select the virtual console, and then select **Virtual Media**.
+
+2. In the **Map CD/DVD** section, select **Choose File**.
+
+3. Choose the version ISO image file for this version from the dialog box that opens.
+
+4. Select the **Map Device** button.
+
+ :::image type="content" source="media/tutorial-install-components/mapped-device-on-virtual-media-screen-v2.png" alt-text="Screenshot that shows a mapped device.":::
+
+5. The media is mounted. Select **Close**.
+
+6. Start the appliance. When you're using iDRAC, you can restart the servers by selecting the **Consul Control** button. Then, on the **Keyboard Macros**, select the **Apply** button, which will start the Ctrl+Alt+Delete sequence.
+
+7. Select **English**.
+
+8. Select **SENSOR-RELEASE-\<version\> Enterprise**.
+
+ :::image type="content" source="media/tutorial-install-components/sensor-version-select-screen-v2.png" alt-text="Screenshot that shows version selection.":::
+
+9. Define the appliance profile and network properties:
+
+ :::image type="content" source="media/tutorial-install-components/appliance-profile-screen-v2.png" alt-text="Screenshot that shows the appliance profile.":::
+
+ | Parameter | Configuration |
+ |--|--|
+ | **Hardware profile** | **enterprise** |
+ | **Management interface** | **eno1** |
+ | **Network parameters (provided by the customer)** | - |
+ |**management network IP address:** | - |
+ | **subnet mask:** | - |
+ | **appliance hostname:** | - |
+ | **DNS:** | - |
+ | **default gateway IP address:** | - |
+ | **input interfaces:** | The system generates the list of input interfaces for you. To mirror the input interfaces, copy all the items presented in the list with a comma separator. Note that there's no need to configure the bridge interface. This option is used for special use cases only. |
+
+10. After about 10 minutes, the two sets of credentials appear. One is for a **CyberX** user, and one is for a **Support** user.
+
+11. Save the appliance ID and passwords. You'll need these credentials to access the platform the first time you use it.
+
+12. Select **Enter** to continue.
+
+## HPE ProLiant DL20 installation
+
+This article describes the HPE ProLiant DL20 installation process, which includes the following steps:
+
+ - Enable remote access and update the default administrator password.
+ - Configure BIOS and RAID settings.
+ - Install the software.
+
+### About the installation
+
+ - Enterprise and SMB appliances can be installed. The installation process is identical for both appliance types, except for the array configuration.
+ - A default administrative user is provided. We recommend that you change the password during the network configuration process.
+ - During the network configuration process, you'll configure the iLO port on network port 1.
+ - The installation process takes about 20 minutes. After the installation, the system is restarted several times.
+
+### HPE ProLiant DL20 front panel
+
+:::image type="content" source="media/tutorial-install-components/hpe-proliant-dl20-front-panel-v2.png" alt-text="HPE ProLiant DL20 front panel.":::
+
+### HPE ProLiant DL20 back panel
+
+:::image type="content" source="media/tutorial-install-components/hpe-proliant-dl20-back-panel-v2.png" alt-text="The back panel of the HPE ProLiant DL20.":::
+
+### Enable remote access and update the password
+
+Use the following procedure to set up network options and update the default password.
+
+To enable and update the password:
+
+1. Connect a screen and a keyboard to the HP appliance, turn on the appliance, and press **F9**.
+
+ :::image type="content" source="media/tutorial-install-components/hpe-proliant-screen-v2.png" alt-text="Screenshot that shows the HPE ProLiant window.":::
+
+2. Go to **System Utilities** > **System Configuration** > **iLO 5 Configuration Utility** > **Network Options**.
+
+ :::image type="content" source="media/tutorial-install-components/system-configuration-window-v2.png" alt-text="Screenshot that shows the System Configuration window.":::
+
+ 1. Select **Shared Network Port-LOM** from the **Network Interface Adapter** field.
+
+ 1. Disable DHCP.
+
+ 1. Enter the IP address, subnet mask, and gateway IP address.
+
+3. Select **F10: Save**.
+
+4. Select **Esc** to get back to the **iLO 5 Configuration Utility**, and then select **User Management**.
+
+5. Select **Edit/Remove User**. The administrator is the only default user defined.
+
+6. Change the default password and select **F10: Save**.
+
+### Configure the HPE BIOS
+
+The following procedure describes how to configure the HPE BIOS for the enterprise and SMB appliances.
+
+To configure the HPE BIOS:
+
+1. Select **System Utilities** > **System Configuration** > **BIOS/Platform Configuration (RBSU)**.
+
+2. In the **BIOS/Platform Configuration (RBSU)** form, select **Boot Options**.
+
+3. Change **Boot Mode** to **Legacy BIOS Mode**, and then select **F10: Save**.
+
+4. Select **Esc** twice to close the **System Configuration** form.
+
+#### For the enterprise appliance
+
+1. Select **Embedded RAID 1: HPE Smart Array P408i-a SR Gen 10** > **Array Configuration** > **Create Array**.
+
+2. In the **Create Array** form, select all the options. Three options are available for the **Enterprise** appliance.
+
+#### For the SMB appliance
+
+1. Select **Embedded RAID 1: HPE Smart Array P208i-a SR Gen 10** > **Array Configuration** > **Create Array**.
+
+2. Select **Proceed to Next Form**.
+
+3. In the **Set RAID Level** form, set the level to **RAID 5** for enterprise deployments and **RAID 1** for SMB deployments.
+
+4. Select **Proceed to Next Form**.
+
+5. In the **Logical Drive Label** form, enter **Logical Drive 1**.
+
+6. Select **Submit Changes**.
+
+7. In the **Submit** form, select **Back to Main Menu**.
+
+8. Select **F10: Save** and then press **Esc** twice.
+
+9. In the **System Utilities** window, select **One-Time Boot Menu**.
+
+10. In the **One-Time Boot Menu** form, select **Legacy BIOS One-Time Boot Menu**.
+
+11. The **Booting in Legacy** and **Boot Override** windows appear. Choose a boot override option; for example, to a CD-ROM, USB, HDD, or UEFI shell.
+
+ :::image type="content" source="media/tutorial-install-components/boot-override-window-one-v2.png" alt-text="Screenshot that shows the first Boot Override window.":::
+
+ :::image type="content" source="media/tutorial-install-components/boot-override-window-two-v2.png" alt-text="Screenshot that shows the second Boot Override window.":::
+### Software installation (HPE ProLiant DL20 appliance)
+
+The installation process takes about 20 minutes. After the installation, the system is restarted several times.
+
+To install the software:
+
+1. Connect the screen and keyboard to the appliance, and then connect to the CLI.
+
+2. Connect an external CD or disk on the key with the ISO image that you downloaded from the **Updates** page in the Defender for IoT portal.
+
+3. Start the appliance.
+
+4. Select **English**.
+
+ :::image type="content" source="media/tutorial-install-components/select-english-screen.png" alt-text="Selection of English in the CLI window.":::
+
+5. Select **SENSOR-RELEASE-<version> Enterprise**.
+
+ :::image type="content" source="media/tutorial-install-components/sensor-version-select-screen-v2.png" alt-text="Screenshot of the screen for selecting a version.":::
+
+6. In the Installation Wizard, define the appliance profile and network properties:
+
+ :::image type="content" source="media/tutorial-install-components/installation-wizard-screen-v2.png" alt-text="Screenshot that shows the Installation Wizard.":::
+
+ | Parameter | Configuration |
+ | ----------| ------------- |
+ | **Hardware profile** | Select **Enterprise** or **Office** for SMB deployments. |
+ | **Management interface** | **eno2** |
+ | **Default network parameters (usually the parameters are provided by the customer)** | **management network IP address:** <br/> <br/>**appliance hostname:** <br/>**DNS:** <br/>**the default gateway IP address:**|
+ | **input interfaces:** | The system generates the list of input interfaces for you.<br/><br/>To mirror the input interfaces, copy all the items presented in the list with a comma separator: **eno5, eno3, eno1, eno6, eno4**<br/><br/>**For HPE DL20: Do not list eno1, enp1s0f4u4 (iLo interfaces)**<br/><br/>**BRIDGE**: There's no need to configure the bridge interface. This option is used for special use cases only. Press **Enter** to continue. |
+
+7. After about 10 minutes, the two sets of credentials appear. One is for a **CyberX** user, and one is for a **Support** user.
+
+8. Save the appliance's ID and passwords. You'll need the credentials to access the platform for the first time.
+
+9. Select **Enter** to continue.
+
+## HPE ProLiant DL360 installation
+
+ - A default administrative user is provided. We recommend that you change the password during the network configuration.
+
+ - During the network configuration, you'll configure the iLO port.
+
+ - The installation process takes about 20 minutes. After the installation, the system is restarted several times.
+
+### HPE ProLiant DL360 front panel
+
+:::image type="content" source="media/tutorial-install-components/hpe-proliant-dl360-front-panel.png" alt-text="HPE ProLiant DL360 front panel.":::
+
+### HPE ProLiant DL360 back panel
+
+:::image type="content" source="media/tutorial-install-components/hpe-proliant-dl360-back-panel.png" alt-text="HPE ProLiant DL360 back panel.":::
+
+### Enable remote access and update the password
+
+Refer to the preceding sections for HPE ProLiant DL20 installation:
+
+ - "Enable remote access and update the password"
+
+ - "Configure the HPE BIOS"
+
+The enterprise configuration is identical.
+
+> [!Note]
+> In the array form, verify that you select all the options.
+
+### iLO remote installation (from a virtual drive)
+
+This procedure describes the iLO installation from a virtual drive.
+
+To install:
+
+1. Sign in to the iLO console, and then right-click the servers' screen.
+
+2. Select **HTML5 Console**.
+
+3. In the console, select the CD icon, and choose the CD/DVD option.
+
+4. Select **Local ISO file**.
+
+5. In the dialog box, choose the relevant ISO file.
+
+6. Go to the left icon, select **Power**, and the select **Reset**.
+
+7. The appliance will restart and run the sensor installation process.
+
+### Software installation (HPE DL360)
+
+The installation process takes about 20 minutes. After the installation, the system is restarted several times.
+
+To install:
+
+1. Connect the screen and keyboard to the appliance, and then connect to the CLI.
+
+2. Connect an external CD or disk on a key with the ISO image that you downloaded from the **Updates** page in the Defender for IoT portal.
+
+3. Start the appliance.
+
+4. Select **English**.
+
+5. Select **SENSOR-RELEASE-<version> Enterprise**.
+
+ :::image type="content" source="media/tutorial-install-components/sensor-version-select-screen-v2.png" alt-text="Screenshot that shows selecting the version.":::
+
+6. In the Installation Wizard, define the appliance profile and network properties.
+
+ :::image type="content" source="media/tutorial-install-components/installation-wizard-screen-v2.png" alt-text="Screenshot that shows the Installation Wizard.":::
+
+ | Parameter | Configuration |
+ | ----------| ------------- |
+ | **Hardware profile** | Select **corporate**. |
+ | **Management interface** | **eno2** |
+ | **Default network parameters (provided by the customer)** | **management network IP address:** <br/>**subnet mask:** <br/>**appliance hostname:** <br/>**DNS:** <br/>**the default gateway IP address:**|
+ | **input interfaces:** | The system generates a list of input interfaces for you.<br/><br/>To mirror the input interfaces, copy all the items presented in the list with a comma separator.<br/><br/>Note that there's no need to configure the bridge interface. This option is used for special use cases only. |
+
+7. After about 10 minutes, the two sets of credentials appear. One is for a **CyberX** user, and one is for a **support** user.
+
+8. Save the appliance's ID and passwords. You'll need these credentials to access the platform for the first time.
+
+9. Select **Enter** to continue.
+
+## Sensor installation for the virtual appliance
+
+You can deploy the virtual machine for the Defender for IoT sensor in the following architectures:
++
+| Architecture | Specifications | Usage | Comments |
+|---|---|---|---|
+| **Enterprise** | CPU: 8<br/>Memory: 32G RAM<br/>HDD: 1800 GB | Production environment | Default and most common |
+| **Small Business** | CPU: 4 <br/>Memory: 8G RAM<br/>HDD: 500 GB | Test or small production environments | - |
+| **Office** | CPU: 4<br/>Memory: 8G RAM<br/>HDD: 100 GB | Small test environments | - |
+
+### Prerequisites
+
+The on-premises management console supports both VMware and Hyper-V deployment options. Before you begin the installation, make sure you have the following items:
+
+ - VMware (ESXi 5.5 or later) or Hyper-V hypervisor (Windows 10 Pro or Enterprise) installed and operational
+
+ - Available hardware resources for the virtual machine
+
+ - ISO installation file for the Azure Defender for IoT sensor
+
+Make sure the hypervisor is running.
+
+### Create the virtual machine (ESXi)
+
+1. Sign in to the ESXi, choose the relevant **datastore**, and select **Datastore Browser**.
+
+2. **Upload** the image and select **Close**.
+
+3. Go to **Virtual Machines**, and then select **Create/Register VM**.
+
+4. Select **Create new virtual machine**, and then select **Next**.
+
+5. Add a sensor name and choose:
+
+ - Compatibility: **&lt;latest ESXi version&gt;**
+
+ - Guest OS family: **Linux**
+
+ - Guest OS version: **Ubuntu Linux (64-bit)**
+
+6. Select **Next**.
+
+7. Choose the relevant datastore and select **Next**.
+
+8. Change the virtual hardware parameters according to the required architecture.
+
+9. For **CD/DVD Drive 1**, select **Datastore ISO file** and choose the ISO file that you uploaded earlier.
+
+10. Select **Next** > **Finish**.
+
+### Create the virtual machine (Hyper-V)
+
+This procedure describes how to create a virtual machine by using Hyper-V.
+
+To create a virtual machine:
+
+1. Create a virtual disk in Hyper-V Manager.
+
+2. Select **format = VHDX**.
+
+3. Select **type = Dynamic Expanding**.
+
+4. Enter the name and location for the VHD.
+
+5. Enter the required size (according to the architecture).
+
+6. Review the summary and select **Finish**.
+
+7. On the **Actions** menu, create a new virtual machine.
+
+8. Enter a name for the virtual machine.
+
+9. Select **Specify Generation** > **Generation 1**.
+
+10. Specify the memory allocation (according to the architecture) and select the check box for dynamic memory.
+
+11. Configure the network adaptor according to your server network topology.
+
+12. Connect the VHDX created previously to the virtual machine.
+
+13. Review the summary and select **Finish**.
+
+14. Right-click the new virtual machine and select **Settings**.
+
+15. Select **Add Hardware** and add a new network adapter.
+
+16. Select the virtual switch that will connect to the sensor management network.
+
+17. Allocate CPU resources (according to the architecture).
+
+18. Connect the management console's ISO image to a virtual DVD drive.
+
+19. Start the virtual machine.
+
+20. On the **Actions** menu, select **Connect** to continue the software installation.
+
+### Software installation (ESXi and Hyper-V)
+
+This section describes the ESXi and Hyper-V software installation.
+
+To install:
+
+1. Open the virtual machine console.
+
+2. The VM will start from the ISO image, and the language selection screen will appear. Select **English**.
+
+3. Select the required architecture.
+
+4. Define the appliance profile and network properties:
+
+ | Parameter | Configuration |
+ | ----------| ------------- |
+ | **Hardware profile** | &lt;required architecture&gt; |
+ | **Management interface** | **ens192** |
+ | **Network parameters (provided by the customer)** | **management network IP address:** <br/>**subnet mask:** <br/>**appliance hostname:** <br/>**DNS:** <br/>**default gateway:** <br/>**input interfaces:**|
+ | **bridge interfaces:** | There's no need to configure the bridge interface. This option is for special use cases only. |
+
+5. Enter **Y** to accept the settings.
+
+6. Sign-in credentials are automatically generated and presented. Copy the username and password in a safe place, because they're required for sign-in and administration.
+
+ - **Support**: The administrative user for user management.
+
+ - **CyberX**: The equivalent of root for accessing the appliance.
+
+7. The appliance restarts.
+
+8. Access the management console via the IP address previously configured: `https://ip_address`.
+
+ :::image type="content" source="media/tutorial-install-components/defender-for-iot-sign-in-screen.png" alt-text="Screenshot that shows access to the management console.":::
+
+## Virtual appliance: On-premises management console installation
+
+The on-premises management console VM supports the following architectures:
+
+| Architecture | Specifications | Usage |
+|--|--|--|
+| Enterprise <br/>(Default and most common) | CPU: 8 <br/>Memory: 32G RAM<br/> HDD: 1.8 TB | Large production environments |
+| Enterprise | CPU: 4 <br/> Memory: 8G RAM<br/> HDD: 500 GB | Large production environments |
+| Enterprise | CPU: 4 <br/>Memory: 8G RAM <br/> HDD: 100 GB | Small test environments |
+
+### Prerequisites
+
+The on-premises management console supports both VMware and Hyper-V deployment options. Before you begin the installation, verify the following:
+
+- VMware (ESXi 5.5 or later) or Hyper-V hypervisor (Windows 10 Pro or Enterprise) is installed and operational.
+
+- The hardware resources are available for the virtual machine.
+
+- You have the ISO installation file for the on-premises management console.
+
+- The hypervisor is running.
+
+### Create the virtual machine (ESXi)
+
+To a create virtual machine (ESXi):
+
+1. Sign in to the ESXi, choose the relevant **datastore**, and select **Datastore Browser**.
+
+2. Upload the image and select **Close**.
+
+3. Go to **Virtual Machines**.
+
+4. Select **Create/Register VM**.
+
+5. Select **Create new virtual machine** and select **Next**.
+
+6. Add a sensor name and choose:
+
+ - Compatibility: \<latest ESXi version>
+
+ - Guest OS family: Linux
+
+ - Guest OS version: Ubuntu Linux (64-bit)
+
+7. Select **Next**.
+
+8. Choose relevant datastore and select **Next**.
+
+9. Change the virtual hardware parameters according to the required architecture.
+
+10. For **CD/DVD Drive 1**, select **Datastore ISO file** and choose the ISO file that you uploaded earlier.
+
+11. Select **Next** > **Finish**.
+
+### Create the virtual machine (Hyper-V)
+
+To create a virtual machine by using Hyper-V:
+
+1. Create a virtual disk in Hyper-V Manager.
+
+2. Select the format **VHDX**.
+
+3. Select **Next**.
+
+4. Select the type **Dynamic expanding**.
+
+5. Select **Next**.
+
+6. Enter the name and location for the VHD.
+
+7. Select **Next**.
+
+8. Enter the required size (according to the architecture).
+
+9. Select **Next**.
+
+10. Review the summary and select **Finish**.
+
+11. On the **Actions** menu, create a new virtual machine.
+
+12. Select **Next**.
+
+13. Enter a name for the virtual machine.
+
+14. Select **Next**.
+
+15. Select **Generation** and set it to **Generation 1**.
+
+16. Select **Next**.
+
+17. Specify the memory allocation (according to the architecture) and select the check box for dynamic memory.
+
+18. Select **Next**.
+
+19. Configure the network adaptor according to your server network topology.
+
+20. Select **Next**.
+
+21. Connect the VHDX created previously to the virtual machine.
+
+22. Select **Next**.
+
+23. Review the summary and select **Finish**.
+
+24. Right-click the new virtual machine, and then select **Settings**.
+
+25. Select **Add Hardware** and add a new adapter for **Network Adapter**.
+
+26. For **Virtual Switch**, select the switch that will connect to the sensor management network.
+
+27. Allocate CPU resources (according to the architecture).
+
+28. Connect the management console's ISO image to a virtual DVD drive.
+
+29. Start the virtual machine.
+
+30. On the **Actions** menu, select **Connect** to continue the software installation.
+
+### Software installation (ESXi and Hyper-V)
+
+Starting the virtual machine will start the installation process from the ISO image.
+
+To install the software:
+
+1. Select **English**.
+
+2. Select the required architecture for your deployment.
+
+3. Define the network interface for the sensor management network: interface, IP, subnet, DNS server, and default gateway.
+
+4. Sign-in credentials are automatically generated and presented. Keep these credentials in a safe place, because they're required for sign-in and administration.
+
+ - **Support**: The administrative user for user management.
+
+ - **CyberX**: The equivalent of root for accessing the appliance.
+
+5. The appliance restarts.
+
+6. Access the management console via the IP address previously configured: `<https://ip_address>`.
+
+ :::image type="content" source="media/tutorial-install-components/defender-for-iot-management-console-sign-in-screen.png" alt-text="Screenshot that shows the management console's sign-in screen.":::
+
+## Post-installation validation
+
+To validate the installation of a physical appliance, you need to perform a number of tests. The same validation process applies to all the appliance types.
+
+Perform the validation by using the GUI or the CLI. The validation is available to the user **Support** and the user **CyberX**.
+
+Post-installation validation must include the following tests:
+
+ - **Sanity test**: Verify that the system is running.
+
+ - **Version**: Verify that the version is correct.
+
+ - **ifconfig**: Verify that all the input interfaces configured during the installation process are running.
+
+### Checking system health by using the GUI
+
+:::image type="content" source="media/tutorial-install-components/system-health-check-screen.png" alt-text="Screenshot that shows the system health check.":::
+
+#### Sanity
+
+- **Appliance**: Runs the appliance sanity check. You can perform the same check by using the CLI command `system-sanity`.
+
+- **Version**: Displays the appliance version.
+
+- **Network Properties**: Displays the sensor network parameters.
+
+#### Redis
+
+- **Memory**: Provides the overall picture of memory usage, such as how much memory was used and how much remained.
+
+- **Longest Key**: Displays the longest keys that might cause extensive memory usage.
+
+#### System
+
+- **Core Log**: Provides the last 500 rows of the core log, enabling you to view the recent log rows without exporting the entire system log.
+
+- **Task Manager**: Translates the tasks that appear in the table of processes to the following layers:
+
+ - Persistent layer (Redis)
+ - Cash layer (SQL)
+
+- **Network Statistics**: Displays your network statistics.
+
+- **TOP**: Shows the table of processes. It's a Linux command that provides a dynamic real-time view of the running system.
+
+- **Backup Memory Check**: Provides the status of the backup memory, checking the following:
+ - The location of the backup folder
+ - The size of the backup folder
+ - The limitations of the backup folder
+ - When the last backup happened
+ - How much space there is for the additional backup files
+
+- **ifconfig**: Displays the parameters for the appliance's physical interfaces.
+
+- **CyberX nload**: Displays network traffic and bandwidth by using the six-second tests.
+
+- **Errors from Core, log**: Displays errors from the core log file.
+
+To access the tool:
+
+1. Sign in to the sensor with the **Support** user credentials.
+
+2. Select **System Statistics** from the **System Settings** window.
+
+ :::image type="icon" source="media/tutorial-install-components/system-statistics-icon.png" border="false":::
+
+### Checking system health by using the CLI
+
+**Test 1: Sanity**
+
+Verify that the system is up and running:
+
+1. Connect to the CLI with the Linux terminal (for example, PuTTY) and the user **Support**.
+
+2. Enter `system sanity`.
+
+3. Check that all the services are green (running).
+
+ :::image type="content" source="media/tutorial-install-components/support-screen.png" alt-text="Screenshot that shows running services.":::
+
+4. Verify that **System is UP! (prod)** appears at the bottom.
+
+**Test 2: Version check**
+
+Verify that the correct version is used:
+
+1. Connect to the CLI with the Linux terminal (for example, PuTTY) and the user **Support**.
+
+2. Enter `system version`.
+
+3. Check that the correct version appears.
+
+**Test 3: Network validation**
+
+Verify that all the input interfaces configured during the installation process are running:
+
+1. Connect to the CLI with the Linux terminal (for example, PuTTY) and the user **Support**.
+
+2. Enter `network list` (the equivalent of the Linux command `ifconfig`).
+
+3. Validate that the required input interfaces appear. For example, if two quad Copper NICs are installed, there should be 10 interfaces in the list.
+
+ :::image type="content" source="media/tutorial-install-components/interface-list-screen.png" alt-text="Screenshot that shows the list of interfaces.":::
+
+**Test 4: Management access to the UI**
+
+Verify that you can access the console web GUI:
+
+1. Connect a laptop with an Ethernet cable to the management port (**Gb1**).
+
+2. Define the laptop NIC address to be in the same range as the appliance.
+
+ :::image type="content" source="media/tutorial-install-components/access-to-ui.png" alt-text="Screenshot that shows management access to the UI.":::
+
+3. Ping the appliance's IP address from the laptop to verify connectivity (default: 10.100.10.1).
+
+4. Open the Chrome browser in the laptop and enter the appliance's IP address.
+
+5. In the **Your connection is not private** window, select **Advanced** and proceed.
+
+6. The test is successful when the Defender for IoT sign-in screen appears.
+
+ :::image type="content" source="media/tutorial-install-components/defender-for-iot-sign-in-screen.png" alt-text="Screenshot that shows access to management console.":::
+
+## Troubleshooting
+
+### You can't connect by using a web interface
+
+1. Verify that the computer that you're trying to connect is on the same network as the appliance.
+
+2. Verify that the GUI network is connected to the management port.
+
+3. Ping the appliance's IP address. If there is no ping:
+
+ 1. Connect a monitor and a keyboard to the appliance.
+
+ 1. Use the **Support** user and password to sign in.
+
+ 1. Use the command `network list` to see the current IP address.
+
+ :::image type="content" source="media/tutorial-install-components/network-list.png" alt-text="Screenshot that shows the network list.":::
+
+4. If the network parameters are misconfigured, use the following procedure to change them:
+
+ 1. Use the command `network edit-settings`.
+
+ 1. To change the management network IP address, select **Y**.
+
+ 1. To change the subnet mask, select **Y**.
+
+ 1. To change the DNS, select **Y**.
+
+ 1. To change the default gateway IP address, select **Y**.
+
+ 1. For the input interface change (sensor only), select **N**.
+
+ 1. To apply the settings, select **Y**.
+
+5. After restart, connect with the support user credentials and use the `network list` command to verify that the parameters were changed.
+
+6. Try to ping and connect from the GUI again.
+
+### The appliance isn't responding
+
+1. Connect a monitor and keyboard to the appliance, or use PuTTY to connect remotely to the CLI.
+
+2. Use the **Support** user's credentials to sign in.
+
+3. Use the `system sanity` command and check that all processes are running.
+
+ :::image type="content" source="media/tutorial-install-components/system-sanity-screen.png" alt-text="Screenshot that shows the system sanity command.":::
+
+For any other issues, contact [Microsoft Support](https://support.microsoft.com/en-us/supportforbusiness/productselection?sapId=82c88f35-1b8e-f274-ec11-c6efdd6dd099).
+
+## Appendix A: Mirroring port on vSwitch (ESXi)
+
+### Configure a SPAN port on an existing vSwitch
+
+A vSwitch does not have mirroring capabilities, but you can use a simple workaround to implement a SPAN port.
+
+To configure a SPAN port:
+
+1. Open vSwitch properties.
+
+2. Select **Add**.
+
+3. Select **Virtual Machine** > **Next**.
+
+4. Insert a network label **SPAN Network**, select **VLAN ID** > **All**, and then select **Next**.
+
+5. Select **Finish**.
+
+6. Select **SPAN Network** > **Edit*.
+
+7. Select **Security**, and verify that the **Promiscuous Mode** policy is set to **Accept** mode.
+
+8. Select **OK**, and then select **Close** to close the vSwitch properties.
+
+9. Open the **XSense VM** properties.
+
+10. For **Network Adapter 2**, select the **SPAN** network.
+
+11. Select **OK**.
+
+12. Connect to the sensor and verify that mirroring works.
+
+## Appendix B: Access sensors from the on-premises management console
+
+You can enhance system security by preventing direct user access to the sensor. Instead, use proxy tunneling to let users access the sensor from the on-premises management console with a single firewall rule. This technique narrows the possibility of unauthorized access to the network environment beyond the sensor. The user's experience when signing in to the sensor remains the same.
+
+:::image type="content" source="media/tutorial-install-components/sensor-system-graph.png" alt-text="Screenshot that shows access to the sensor.":::
+
+To enable tunneling:
+
+1. Sign in to the on-premises management console's CLI with **CyberX** or **Support** user credentials.
+
+2. Enter `sudo cyberx-management-tunnel-enable`.
+
+3. Select **Enter**.
+
+4. Enter `--port 10000`.
+
+### Next steps
+
+[Set up your network](how-to-set-up-your-network.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-investigate-all-enterprise-sensor-detections-in-a-device-inventory https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-investigate-all-enterprise-sensor-detections-in-a-device-inventory.md new file mode 100644
@@ -0,0 +1,225 @@
+---
+title: Learn about devices discovered by all enterprise sensors
+description: Use the device inventory in the on-premises management console to get a comprehensive view of device information from connected sensors. Use import, export, and filtering tools to manage this information.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/02/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Investigate all enterprise sensor detections in the device inventory
+
+You can view device information from connected sensors by using the *device inventory* in the on-premises management console. This feature gives you a comprehensive view of all network information. Use import, export, and filtering tools to manage this information. The status information about the connected sensor versions also appears.
+
+:::image type="content" source="media/how-to-work-with-asset-inventory-information/device-inventory-data-table.png" alt-text="Screenshot of the device inventory data table.":::
+
+The following table describes the table columns in the device inventory.
+
+| Parameter | Description |
+|--|--|
+| **Unacknowledged Alerts** | The number of unhandled alerts associated with this device. |
+| **Business Unit** | The business unit that contains this device. |
+| **Region** | The region that contains this device. |
+| **Site** | The site that contains this device. |
+| **Zone** | The zone that contains this device. |
+| **Appliance** | The Azure Defender for IoT sensor that protects this device. |
+| **Name** | The name of this device as Defender for IoT discovered it. |
+| **Type** | The type of device, such as PLC or HMI. |
+| **Vendor** | The name of the device's vendor, as defined in the MAC address. |
+| **Operating System** | The OS of the device. |
+| **Firmware** | The device's firmware. |
+| **IP Address** | The IP address of the device. |
+| **VLAN** | The VLAN of the device. |
+| **MAC Address** | The MAC address of the device. |
+| **Protocols** | The protocols that the device uses. |
+| **Unacknowledged Alerts** | The number of unhandled alerts associated with this device. |
+| **Is Authorized** | The authorization status of the device:<br />- **True**: The device has been authorized.<br />- **False**: The device has not been authorized. |
+| **Is Known as Scanner** | Whether this device performs scanning-like activities in the network. |
+| **Is Programming Device** | Whether this is a programming device:<br />- **True**: The device performs programming activities for PLCs, RTUs, and controllers, which are relevant to engineering stations.<br />- **False**: The device is not a programming device. |
+| **Groups** | Groups in which this device participates. |
+| **Last Activity** | The last activity that the device performed. |
+| **Discovered** | When this device was first seen in the network. |
+
+## Integrate data into the enterprise device inventory
+
+Data integration capabilities let you enhance the data in the device inventory with information from other enterprise resources. These sources include CMDBs, DNS, firewalls, and Web APIs.
+
+You can use this information to learn. For example:
+
+- Device purchase dates and end-of-warranty dates
+
+- Users responsible for each device
+
+- Opened tickets for devices
+
+- The last date when firmware was upgraded
+
+- Devices allowed access to the internet
+
+- Devices running active antivirus applications
+
+- Users signed in to devices
+
+:::image type="content" source="media/how-to-work-with-asset-inventory-information/asset-inventory-screen-with-items-highlighted-v2.png" alt-text="Data table on the asset inventory screen.":::
+
+You can integrate data by either:
+
+- Adding it manually
+
+- Running customized scripts that Defender for IoT provides
+
+:::image type="content" source="media/how-to-work-with-asset-inventory-information/enterprise-data-integrator-graph.png" alt-text="Diagram of the enterprise data integrator.":::
+
+You can work with Defender for IoT technical support to set up your system to receive Web API queries.
+
+To add data manually:
+
+1. On the side menu, select **Device Inventory** and then select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/menu-icon.png" border="false":::.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/asset-inventory-settings-v2.png" alt-text="Edit your device's inventory settings.":::
+
+2. In the **Device Inventory Settings** dialog box, select **ADD CUSTOM COLUMN**.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/add-custom-column.png" alt-text="Add a custom column to your inventory.":::
+
+3. In the **Add Custom Column** dialog box, add the new column name (up to 250 characters UTF), select **Manual**, and select **SAVE**. The new item appears in the **Device Inventory Settings** dialog box.
+
+4. In the upper-right corner of the **Device Inventory** window, select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/menu-icon-device-inventory.png" border="false"::: and select **Export All Device Inventory**. The CSV file is generated.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/sample-exported-csv-file.png" alt-text="The exported CSV file.":::
+
+5. Manually add the information to the new column and save the file.
+
+6. In the upper-right corner of the **Device Inventory** window, select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/menu-icon-device-inventory.png" border="false":::, select **Import Manual Input Columns**, and browse to the CSV file. The new data appears in the **Device Inventory** table.
+
+To integrate data from other enterprise entities:
+
+1. In the upper-right corner of the **Device Inventory** window, select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/menu-icon-device-inventory.png" border="false"::: and select **Export All Device Inventory**.
+
+2. In the **Device Inventory Settings** dialog box, select **ADD CUSTOM COLUMN**.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/add-custom-column.png" alt-text="Add a custom column to your inventory.":::
+
+3. In the **Add Custom Column** dialog box, add the new column name (up to 250 characters UTF), and then select **Automatic**. The **UPLOAD SCRIPT** and **TEST SCRIPT** options appear.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/add-custom-column-automatic.png" alt-text="Automatically add custom columns.":::
+
+4. Upload and test the script that you received from [Microsoft Support](https://support.serviceshub.microsoft.com/supportforbusiness/create?sapId=82c88f35-1b8e-f274-ec11-c6efdd6dd099).
+
+## Retrieve information from the device inventory
+
+You can retrieve an extensive range of device information detected by managed sensors and integrate that information with partner systems. For example, you can retrieve sensor, zone, site ID, IP address, MAC address, firmware, protocol, and vendor information. Filter information that you retrieve based on:
+
+- Authorized and unauthorized devices.
+
+- Devices associated with specific sites.
+
+- Devices associated with specific zones.
+
+- Devices associated with specific sensors.
+
+Work with Defender for IoT API commands to retrieve and integrate this information. For more information, see [Defender for IoT API sensor and management console APIs](references-work-with-defender-for-iot-apis.md).
+
+## Filter the device inventory
+
+You can filter the device inventory to show columns of interest. For example, you can view PLC device information.
+
+:::image type="content" source="media/how-to-work-with-asset-inventory-information/asset-inventory-view-v2.png" alt-text="Screenshot of the asset inventory.":::
+
+The filter is cleared when you leave the window.
+
+To use the same filter multiple times, you can save a filter or a combination of filters that you need. You can open a left pane and view the filters that you've saved:
+
+:::image type="content" source="media/how-to-work-with-asset-inventory-information/view-your-asset-inventories-v2.png" alt-text="Asset inventories screen.":::
+
+To filter the device inventory:
+
+1. In the column that you want to filter, select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/filter-a-column-icon.png" border="false":::.
+
+2. In the **Filter** dialog box, select the filter type:
+
+ - **Equals**: The exact value according to which you want to filter the column. For example, if you filter the protocol column according to **Equals** and `value=ICMP`, the column will present devices that use the ICMP protocol only.
+
+ - **Contains**: The value that's contained among other values in the column. For example, if you filter the protocol column according to **Contains** and `value=ICMP`, the column will present devices that use the ICMP protocol as a part of the list of protocols that the device uses.
+
+3. To organize the column information according to alphabetical order, select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/alphabetical-order-icon.png" border="false":::. Arrange the order by selecting the :::image type="icon" source="media/how-to-work-with-asset-inventory-information/alphabetical-a-z-order-icon.png" border="false"::: and :::image type="icon" source="media/how-to-work-with-asset-inventory-information/alphabetical-z-a-order-icon.png" border="false"::: arrows.
+
+4. To save a new filter, define the filter and select **Save As**.
+
+5. To change the filter definitions, change the definitions and select **Save Changes**.
+
+## View device information per zone
+
+You can learn the following information about devices in a zone.
+
+### View a device map
+
+To view a sensor device map for a selected zone:
+
+- In the **Site Management** window, select **View Zone Map** from the bar that contains the zone name.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/default-region-to-default-business-unit-v2.png" alt-text="Default region to default business unit.":::
+
+The **Device Map** window appears. It shows all the network elements related to the selected zone, including the sensors, the devices connected to them, and other information.
+
+:::image type="content" source="media/how-to-work-with-asset-inventory-information/zone-map-screenshot.png" alt-text="Screenshot of the zone map.":::
+
+The following tools are available for viewing devices and device information from the map. For details about each of these features, see the *Defender for IoT platform user guide*.
+
+- **Map zoom views**: Simplified View, Connections View, and Detailed View. The displayed map view varies depending on the map's zoom level. You switch between map views by adjusting the zoom levels.
+
+ :::image type="icon" source="media/how-to-work-with-asset-inventory-information/zoom-icon.png" border="false":::
+
+- **Map search and layout tools**: Tools used to display varied network segments, devices, device groups, or layers.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/search-and-layout-tools.png" alt-text="Screenshot of the Search and Layout Tools view.":::
+
+- **Labels and indicators on devices:** For example, the number of devices grouped in a subnet in an IT network. In this example, it's 8.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/labels-and-indicators.png" alt-text="Screenshot of labels and indicators.":::
+
+- **View device properties**: For example, the sensor that's monitoring the device and basic device properties. Right-click the device to view the device properties.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/asset-properties-v2.png" alt-text="Screenshot of the Asset Properties view.":::
+
+- **Alert associated with a device:** Right-click the device to view related alerts.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/show-alerts.png" alt-text="Screenshot of the Show Alerts view.":::
+
+### View alerts associated with a zone
+
+To view alerts associated with a specific zone:
+
+- Select the alert icon form the **Zone** window.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/business-unit-view-v2.png" alt-text="The default Business Unit view with examples.":::
+
+For more information, see [Overview: Working with alerts](how-to-work-with-alerts-on-premises-management-console.md).
+
+### View the device inventory of a zone
+
+To view the device inventory associated with a specific zone:
+
+- Select **View Device Inventory** from the **Zone** window.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/default-business-unit.png" alt-text="The device inventory screen will appear.":::
+
+For more information, see [Investigate all enterprise sensor detections in a device inventory](how-to-investigate-all-enterprise-sensor-detections-in-a-device-inventory.md).
+
+### View additional zone information
+
+The following additional zone information is available:
+
+- **Zone details**: View the number of devices, alerts, and sensors associated with the zone.
+
+- **Sensor details**: View the name, IP address, and version of each sensor assigned to the zone.
+
+- **Connectivity status**: If a sensor is disconnected, connect from the sensor. See [Connect sensors to the on-premises management console](how-to-activate-and-set-up-your-on-premises-management-console.md#connect-sensors-to-the-on-premises-management-console).
+
+- **Update progress**: If the connected sensor is being upgraded, upgrade statuses will appear. During upgrade, the on-premises management console does not receive device information from the sensor.
+
+## See also
+
+[Investigate sensor detections in a device inventory](how-to-investigate-sensor-detections-in-a-device-inventory.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-investigate-sensor-detections-in-a-device-inventory https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-investigate-sensor-detections-in-a-device-inventory.md new file mode 100644
@@ -0,0 +1,225 @@
+---
+title: Gain insight into devices discovered by a specific sensor
+description: The device inventory displays an extensive range of device attributes that a sensor detects.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/06/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Investigate sensor detections in a device inventory
+
+The device inventory displays an extensive range of device attributes that a sensor detects. Options are available to:
+
+ - Easily filter the information.
+
+ - Export information to a CSV file.
+
+ - Import Windows registry details.
+
+ - Create groups for display in the device map.
+
+## View device attributes in the device inventory
+
+The following attributes appear in the device inventory table.
+
+| Parameter | Description |
+|--|--|
+| Name | The name of the device as the sensor discovered it. |
+| Type | The type of device. |
+| Vendor | The name of the device's vendor, as defined in the MAC address. |
+| Operating System | The OS of the device. |
+| Firmware | The device's firmware. |
+| IP Address | The IP address of the device. |
+| VLAN | The VLAN of the device. For details about instructing the sensor to discover VLANs, see [Define VLAN names](how-to-manage-the-on-premises-management-console.md#define-vlan-names).(how-to-define-management-console-network-settings.md#define-vlan-names). |
+| MAC Address | The MAC address of the device. |
+| Protocols | The protocols that the device uses. |
+| Unacknowledged Alerts | The number of unacknowledged alerts associated with this device. |
+| Is Authorized | The authorization status defined by the user:<br />- **True**: The device has been authorized.<br />- **False**: The device has not been authorized. |
+| Is Known as Scanner | Defined as a scanning device by the user. |
+| Is Programming device | Defined as an authorized programming device by the user. <br />- **True**: The device performs programming activities for PLCs, RTUs, and controllers, which are relevant to engineering stations. <br />- **False**: The device is not a programming device. |
+| Groups | The groups that this device participates in. |
+| Last Activity | The last activity that the device performed. |
+| Discovered | When this device was first seen in the network. |
+
+To view the device inventory:
+
+1. In the left pane, select **Devices**. The **Devices** pane opens on the right.
+
+2. In the **Devices** pane, select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/device-pane-icon.png" border="false":::.
+
+To hide and display columns, customize the device inventory table:
+
+1. On the upper-right menu of the device inventory, select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/settings-icon.png" border="false":::.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/device-inventory-settings-screens-v2.png" alt-text="Device inventory settings screen.":::
+
+2. In the **Device Inventory Settings** window, select the columns that you want to display in the device inventory table.
+
+3. Change the location of the columns in the table by using arrows.
+
+4. Select **Save**. The **Device Inventory Settings** window closes, and the new settings appear in the table.
+
+### Create temporary device inventory filters
+
+You can set a filter that defines what information the table displays. For example, you can decide that you want to view only the PLC device's information.
+
+:::image type="content" source="media/how-to-work-with-asset-inventory-information/devices-learning-v2.png" alt-text="Devices learning.":::
+
+The filter is not saved when you leave the inventory.
+
+### Save device inventory filters
+
+You can save a filter or a combination of filters that you need and reapply them in the device inventory. Create broader filters based on a certain device type, or more narrow filters based on a specific type and a specific protocol.
+
+The filters that you save are also saved as device map groups. This feature provides an additional level of granularity in viewing network devices on the map.
+
+To create filters:
+
+1. In the column that you want to filter, select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/filter-icon.png" border="false":::.
+
+2. In the **Filter** dialog box, select the filter type:
+
+ - **Equals**: The exact value according to which you want to filter the column. For example, if you filter the protocol column according to **Equals** and `value=ICMP`, the column will present devices that use the ICMP protocol only.
+
+ - **Contains**: The value that's contained among other values in the column. For example, if you filter the protocol column according to **Contains** and `value=ICMP`, the column will present devices that use the ICMP protocol as a part of the list of protocols that the device uses.
+
+3. To organize the column information according to alphabetical order, select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/alphabetical-order-icon.png" border="false":::. Arrange the order by selecting the :::image type="icon" source="media/how-to-work-with-asset-inventory-information/alphabetical-a-z-order-icon.png" border="false"::: and :::image type="icon" source="media/how-to-work-with-asset-inventory-information/alphabetical-z-a-order-icon.png" border="false"::: arrows.
+
+4. To save a new filter, define the filter and select **Save As**.
+
+5. To change the filter definitions, change the definitions and select **Save Changes**.
+
+To view filters:
+
+- Open the left pane and view the filters that you've saved:
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/filters-from-left-pane-v2.png" alt-text="View the filters from the left-side pane.":::
+
+### View filtered information as a map group
+
+When you switch to the map view, the filtered devices are highlighted and filtered. The filter group that you saved appears in the side menu under the **Device Inventory Filters** group.
+
+:::image type="content" source="media/how-to-work-with-asset-inventory-information/filters-in-the-map-view-v2.png" alt-text="View filters when in the map view.":::
+
+## Learn Windows registry details
+
+In addition to learning OT devices, you can discover IT devices, including Microsoft Windows workstations and servers. These devices are also displayed in device inventory. After you learn devices, you can enrich the device inventory with detailed Windows information, such as:
+
+- Windows version installed
+
+- Applications installed
+
+- Patch-level information
+
+- Open ports
+
+- More robust information on OS versions
+
+Two options are available for retrieving this information:
+
+- Active polling by using scheduled WMI scans.
+
+- Local surveying by distributing and running a script on the device. Working with local scripts bypasses the risks of running WMI polling on an endpoint. It's also useful for regulated networks with waterfalls and one-way elements.
+
+This article describes how to locally survey the Windows endpoint registry with a script. This information will be used for generating alerts, notifications, data mining reports, risk assessments, and attack vector reports.
+
+:::image type="content" source="media/how-to-work-with-asset-inventory-information/data-mining-screen.png" alt-text="Data mining screenshot.":::
+
+You can survey the following Windows operating systems:
+
+- Windows XP
+
+- Windows 2000
+
+- Windows NT
+
+- Windows 7
+
+- Windows 10
+
+- Windows Server 2003/2008/2012/2016
+
+### Before you begin
+
+To work with the script, you need to meet the following requirements:
+
+- Administrator permissions are required to run the script on the device.
+
+- The sensor should have already learned the Windows device. This means that if the device already exists, the script will retrieve its information.
+
+- A sensor is monitoring the network that the Windows PC is connected to.
+
+### Acquire the script
+
+To receive the script, [contact customer support](mailto:support.microsoft.com).
+
+### Deploy the script
+
+You can deploy the script once or schedule ongoing queries by using standard automated deployment methods and tools.
+
+### About the script
+
+- The script is run as a utility and not an installed program. Running the script does not affect the endpoint.
+
+- The files that the script generates remain on the local drive until you delete them.
+
+- The files that the script generates are located next to each other. Don't separate them.
+
+- If you run the script again in the same location, these files are overwritten.
+
+To run the script:  
+
+1. Copy the script to a local drive and unzip it. The following files appear:
+
+ - start.bat
+
+ - settings.json
+
+ - data.bin
+
+ - run.bat
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/files-in-file-explorer.png" alt-text="View of the files in File Explorer.":::
+
+2. Run the `run.bat` file.
+
+3. After the registry is probed, the CX-snapshot file appears with the registry information.
+
+4. The file name indicates the system name and date and time of the snapshot. An example file name is `CX-snaphot_SystemName_Month_Year_Time`.
+
+### Import device details
+
+Information learned on each endpoint should be imported to the sensor.
+
+Files generated from the queries can be placed in one folder that you can access from sensors. Use standard, automated methods and tools to move the files from each Windows endpoint to the location where you'll be importing them to the sensor.
+
+Don't update file names.
+
+To import:
+
+1. Select **Import Settings** from the **Import Windows Configuration** dialog box.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/import-windows-configuration-v2.png" alt-text="Import your Windows configurations.":::
+
+2. Select **Add**, and then select all the files (Ctrl+A).
+
+3. Select **Close**. The device registry information is imported. If there's a problem uploading one of the files, you'll be informed which file upload failed.
+
+ :::image type="content" source="media/how-to-work-with-asset-inventory-information/add-new-file.png" alt-text="Upload of added files was successful.":::
+
+## Export device inventory information
+
+You can export device inventory information to an Excel file. Imported information overwrites current information.
+
+To export a CSV file:
+
+- On the upper-right menu of the device inventory, select :::image type="icon" source="media/how-to-work-with-asset-inventory-information/csv-excel-export-icon.png" border="false":::. The CSV report is generated and downloaded.
+
+## See also
+
+[Investigate all enterprise sensor detections in a device inventory](how-to-investigate-all-enterprise-sensor-detections-in-a-device-inventory.md)
+
+[Work with site map views](how-to-gain-insight-into-global-regional-and-local-threats.md#work-with-site-map-views)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-manage-individual-sensors https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-manage-individual-sensors.md new file mode 100644
@@ -0,0 +1,460 @@
+---
+title: Manage individual sensors
+description: Learn how to manage individual sensors, including managing activation files, performing backups, and updating a standalone sensor.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/22/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Manage individual sensors
+
+This article describes how to manage individual sensors. Tasks include managing activation files, performing backups, and updating a standalone sensor.
+
+You can also do certain sensor management tasks from the on-premises management console, where multiple sensors can be managed simultaneously.
+
+You use the Azure portal for sensor onboarding and registration.
+
+## Manage sensor activation files
+
+Your sensor was onboarded with Azure Defender for IoT from the Azure portal. Each sensor was onboarded as either a locally connected sensor or a cloud-connected sensor.
+
+A unique activation file is uploaded to each sensor that you deploy. For more information about when and how to use a new file, see [Upload new activation files](#upload-new-activation-files). If you can't upload the file, see [Troubleshoot activation file upload](#troubleshoot-activation-file-upload).
+
+### About activation files for locally connected sensors
+
+Locally connected sensors are associated with an Azure subscription. The activation file for your locally connected sensors contains an expiration date. One month before this date, a warning message appears at the top of the sensor console. The warning remains until after you've updated the activation file.
+
+:::image type="content" source="media/how-to-manage-individual-sensors/system-setting-screenshot.png" alt-text="The screenshot of the system settings.":::
+
+You can continue to work with Defender for IoT features even if the activation file has expired.
+
+### About activation files for cloud-connected sensors
+
+Sensors that are cloud connected are associated with the Defender for IoT hub. These sensors are not limited by time periods for the activation file. The activation file for cloud-connected sensors is used to ensure connection to the Defender for IoT hub.
+
+### Upload new activation files
+
+You might need to upload a new activation file for an onboarded sensor when:
+
+- An activation file expires on a locally connected sensor.
+
+- You want to work in a different sensor management mode.
+
+- You want to assign a new Defender for IoT hub to a cloud-connected sensor.
+
+To add a new activation file:
+
+1. Go to the **Sensor Management** page.
+
+2. Select the sensor for which you want to upload a new activation file.
+
+3. Delete it.
+
+4. Onboard the sensor again from the **Onboarding** page in the new mode or with a new Defender for IoT hub.
+
+5. Download the activation file from the **Download Activation File** page.
+
+6. Save the file.
+
+ :::image type="content" source="media/how-to-manage-individual-sensors/download-activation-file.png" alt-text="Download the activation file from the Defender for IoT hub.":::
+
+7. Sign in to the Defender for IoT sensor console.
+
+8. In the sensor console, select **System Settings** > **Reactivation**.
+
+ :::image type="content" source="media/how-to-manage-individual-sensors/reactivation.png" alt-text="Reactivation selection on the System Settings screen.":::
+
+9. Select **Upload** and select the file that you saved.
+
+ :::image type="content" source="media/how-to-manage-individual-sensors/upload-the-file.png" alt-text="Upload the file you saved.":::
+
+10. Select **Activate**.
+
+### Troubleshoot activation file upload
+
+You'll receive an error message if the activation file could not be uploaded. The following events might have occurred:
+
+- **For locally connected sensors**: The activation file is not valid. If the file is not valid, go to the Defender for IoT portal. On the **Sensor Management** page, select the sensor with the invalid file, and download a new activation file.
+
+- **For cloud-connected sensors**: The sensor can't connect to the internet. Check the sensor's network configuration. If your sensor needs to connect through a web proxy to access the internet, verify that your proxy server is configured correctly on the **Sensor Network Configuration** screen. Verify that \*.azure-devices.net:443 is allowed in the firewall and/or proxy. If wildcards are not supported or you want more control, the FQDN for your specific Defender for IoT hub should be opened in your firewall and/or proxy. For details, see [Reference - IoT Hub endpoints](https://docs.microsoft.com/azure/iot-hub/iot-hub-devguide-endpoints).
+
+- **For cloud-connected sensors**: The activation file is valid but Defender for IoT rejected it. If you can't resolve this problem, you can download another activation from the **Sensor Management** page of the Defender for IoT portal. If this doesn't work, contact Microsoft Support.
+
+## Manage certificates
+
+Following sensor installation, a local self-signed certificate is generated and used to access the sensor web application. When logging in to the sensor for the first time, Administrator users are prompted to provide an SSL/TLS certificate. For more information about first time setup, see [Sign in and activate a sensor](how-to-activate-and-set-up-your-sensor.md).
+
+This article provides information on updating certificates, working with certificate CLI commands, and supported certificates and certificate parameters.
+
+### About certificates
+
+Azure Defender for IoT uses SSL/TLS certificates to:
+
+1. Meet specific certificate and encryption requirements requested by your organization by uploading the CA-signed certificate.
+
+1. Allow validation between the management console and connected sensors, and between a management console and a High Availability management console. Validations is evaluated against a Certificate Revocation List, and the certificate expiration date. **If validation fails, communication between the management console and the sensor is halted and a validation error is presented in the console. This option is enabled by default after installation.**
+
+ Third party Forwarding rules, for example alert information sent to SYSLOG, Splunk or ServiceNow; or communication with Active Directory are not validated.
+
+### Update certificates
+
+Sensor Administrator users can update certificates.
+
+To update a certificate:
+
+1. Select **System Settings**.
+1. Select **SSL/TLS Certificates.**
+1. Delete or edit the certificate and add a new one.
+ - Add a certificate name.
+ - Upload a CRT file and key file and enter a passphrase.
+ - Upload a PEM file if required.
+
+To change the validation setting:
+
+1. Enable or Disable the **Enable Certificate Validation** toggle.
+1. Select **Save**.
+
+If the option is enabled and validation fails, communication between the management console and the sensor is halted and a validation error is presented in the console.
+
+### Certificate Support
+
+The following certificates are supported:
+
+- Private / Enterprise Key Infrastructure (Private PKI)
+- Public Key Infrastructure (Public PKI)
+- Locally generated on the appliance (locally self-signed). **Using self-signed certificates is not recommended.** This connection is *insecure* and should be used for test environments only. The owner of the certificate cannot be validated, and the security of your system cannot be maintained. Self-signed certificates should never be used for production networks.
+
+The following parameters are supported.
+Certificate CRT
+
+- The primary certificate file for your domain name
+- Signature Algorithm = SHA256RSA
+- Signature Hash Algorithm = SHA256
+- Valid from = Valid past date
+- Valid To = Valid future date
+- Public Key = RSA 2048bits (Minimum) or 4096bits
+- CRL Distribution Point = URL to .crl file
+- Subject CN = URL, can be a wildcard certificate e.g. example.contoso.com or *.contoso.com**
+- Subject (C)ountry = defined, e.g. US
+- Subject (OU) Org Unit = defined, e.g. Contoso Labs
+- Subject (O)rganization = defined, e.g. Contoso Inc.
+
+Key File
+
+- The key file generated when you created CSR
+- RSA 2048bits (Minimum) or 4096bits
+
+Certificate Chain
+
+- The intermediate certificate file (if any) that was supplied by your CA
+- The CA certificate that issued the server's certificate should be first in the file, followed by any others up to the root.
+- Can include Bag attributes.
+
+Passphrase
+
+- 1 key supported
+- Setup when importing the certificate
+
+Certificates with other parameters may work but cannot be supported by Microsoft.
+
+#### Encryption key artifacts
+
+**.pem – Certificate Container File**
+
+The name is from Privacy Enhanced Mail (PEM), an historic method for secure email but the container format it used lives on, and is a base64 translation of the x509 ASN.1 keys. 
+
+Defined in RFCs 1421 to 1424: a container format that may include just the public certificate (such as with Apache installs, and CA certificate files /etc/ssl/certs), or may include an entire certificate chain including public key, private key, and root certificates.
+
+It may also encode a CSR as the PKCS10 format can be translated into PEM.
+
+**.cert .cer .crt – Certificate Container File**
+
+A .pem (or rarely .der) formatted file with a different extension. It is recognized by Windows Explorer as a certificate. The .pem file is not recognized by Windows Explorer.
+
+**.key – Private Key File**
+
+A KEY file is the same "format" as a PEM file, but it has a different extension.
+##### Use CLI commands to deploy certificates
+
+Use the *cyberx-xsense-certificate-import* CLI command to import certificates. To use this tool, certificate files need to be uploaded to the device (using tools such as winscp or wget).
+
+The command supports the following input flags:
+
+-h Show the command line help syntax
+
+--crt Path to certificate file (CRT extension)
+
+--key *.key file, key length should be minimum 2048 bits
+
+--chain Path to certificate chain file (optional)
+
+--pass Passphrase used to encrypt the certificate (optional)
+
+--passphrase-set Default = False, unused. Set to TRUE to use previous passphrase supplied with previous certificate (optional)
+
+When using the CLI command:
+
+- Verify the certificate files are readable on the appliance.
+
+- Verify that the domain name and IP in the certificate match the configuration planned by the IT department.
++
+## Connect a sensor to the management console
+
+This section describes how to ensure connection between the sensor and the on-premises management console. You need to do this if you're working in an air-gapped network and want to send asset and alert information to the management console from the sensor. This connection also allows the management console to push system settings to the sensor and perform other management tasks on the sensor.
+
+To connect:
+
+1. Sign in to the on-premises management console.
+
+2. Select **System Settings**.
+
+3. In the **Sensor Setup ΓÇô Connection String** section, copy the automatically generated connection string.
+
+ :::image type="content" source="media/how-to-manage-individual-sensors/connection-string-screen.png" alt-text="Copy the connection string from this screen.":::
+
+4. Sign in to the sensor console.
+
+5. On the left pane, select **System Settings**.
+
+6. Select **Management Console Connection**.
+
+ :::image type="content" source="media/how-to-manage-individual-sensors/management-console-connection-screen.png" alt-text="Screenshot of the Management Console Connection dialog box.":::
+
+7. Paste the connection string in the **Connection string** box and select **Connect**.
+
+8. In the on-premises management console, in the **Site Management** window, assign the sensor to a zone.
+
+## Change the name of a sensor
+
+You can change the name of your sensor console. The new name will appear in the console web browser, in various console windows, and in troubleshooting logs.
+
+The process for changing sensor names varies for locally connected sensors and cloud-connected sensors. The default name is **sensor**.
+
+### Change the name of a locally connected sensor
+
+To change the name:
+
+1. In the bottom of the left pane of the console, select the current sensor label.
+
+ :::image type="content" source="media/how-to-change-the-name-of-your-azure-consoles/label-name.png" alt-text="Screenshot that shows the sensor label.":::
+
+1. In the **Edit sensor name** dialog box, enter a name.
+
+1. Select **Save**. The new name is applied.
+
+### Change the name of a cloud-connected sensor
+
+If your sensor was registered as a cloud-connected sensor, the sensor name is defined by the name assigned during the registration. The name is included in the activation file that you uploaded when signing in for the first time. To change the name of the sensor, you need to upload a new activation file.
+
+To change the name:
+
+1. In the Azure Defender for IoT portal, go to the **Sensor Management** page.
+
+1. Delete the sensor from the **Sensor Management** window.
+
+1. Register with the new name.
+
+1. Download and new activation file.
+
+1. Sign in to the sensor and upload the new activation file.
+
+## Update the sensor network configuration
+
+The sensor network configuration was defined during the sensor installation. You can change configuration parameters. You can also set up a proxy configuration.
+
+If you create a new IP address, you might be required to sign in again.
+
+To change the configuration:
+
+1. On the side menu, select **System Settings**.
+
+2. In the **System Settings** window, select **Network**.
+
+ :::image type="content" source="media/how-to-manage-individual-sensors/edit-network-configuration-screen.png" alt-text="Configure your network settings.":::
+
+3. Set the parameters as follows:
+
+ | Parameter | Description |
+ |--|--|
+ | IP address | The sensor IP address |
+ | Subnet mask | The mask address |
+ | Default gateway | The default gateway address |
+ | DNS | The DNS server address |
+ | Hostname | The sensor hostname |
+ | Proxy | Proxy host and port name |
+
+4. Select **Save**.
+
+## Synchronize time zones on the sensor
+
+You can configure the sensor's time and region so that all the users see the same time and region.
+
+:::image type="content" source="media/how-to-manage-individual-sensors/time-and-region.png" alt-text="Configure the time and region.":::
+
+| Parameter | Description |
+|--|--|
+| Timezone | The time zone definition for:<br />- Alerts<br />- Trends and statistics widgets<br />- Data mining reports<br /> -Risk assessment reports<br />- Attack vectors |
+| Date format | Select one of the following format options:<br />- dd/MM/yyyy HH:mm:ss<br />- MM/dd/yyyy HH:mm:ss<br />- yyyy/MM/dd HH:mm:ss |
+| Date and time | Displays the current date and local time in the format that you selected.<br />For example, if your actual location is America and New York, but the time zone is set to Europe and Berlin, the time is displayed according to Berlin local time. |
+
+To configure the sensor time:
+
+1. On the side menu, select **System Settings**.
+
+2. In the **System Settings** window, select **Time & Regional**.
+
+3. Set the parameters and select **Save**.
+
+## Set up backup and restore files
+
+System backup is performed automatically at 3:00 AM daily. The data is saved on a different disk in the sensor. The default location is `/var/cyberx/backups`.
+
+You can automatically transfer this file to the internal network.
+
+> [!NOTE]
+> - The backup and restore procedure can be performed between the same versions only.
+> - In some architectures, the backup is disabled. You can enable it in the `/var/cyberx/properties/backup.properties` file.
+
+When you control a sensor by using the on-premises management console, you can use the sensor's backup schedule to collect these backups and store them on the management console or on an external backup server.
+
+**What is backed up:** Configurations and data.
+
+**What is not backed up:** PCAP files and logs. You can manually back up and restore PCAPs and logs.
+
+Sensor backup files are automatically named through the following format: `<sensor name>-backup-version-<version>-<date>.tar`. An example is `Sensor_1-backup-version-2.6.0.102-2019-06-24_09:24:55.tar`.
+
+To configure backup:
+
+- Sign in to an administrative account and enter `$ sudo cyberx-xsense-system-backup`.
+
+To restore the latest backup file:
+
+- Sign in to an administrative account and enter `$ sudo cyberx-xsense-system-restore`.
+
+To save the backup to an external SMB server:
+
+1. Create a shared folder in the external SMB server.
+
+ Get the folder path, username, and password required to access the SMB server.
+
+2. In the sensor, make a directory for the backups:
+
+ - `sudo mkdir /<backup_folder_name_on_cyberx_server>`
+
+ - `sudo chmod 777 /<backup_folder_name_on_cyberx_server>/`
+
+3. Edit `fstab`:
+
+ - `sudo nano /etc/fstab`
+
+ - `add - //<server_IP>/<folder_path> /<backup_folder_name_on_cyberx_server> cifsrw,credentials=/etc/samba/user,vers=X.X,uid=cyberx,gid=cyberx,file_mode=0777,dir_mode=0777 0 0`
+
+4. Edit and create credentials to share for the SMB server:
+
+ `sudo nano /etc/samba/user`
+
+5. Add:
+
+ - `username=&gt:user name&lt:`
+
+ - `password=<password>`
+
+6. Mount the directory:
+
+ `sudo mount -a`
+
+7. Configure a backup directory to the shared folder on the Defender for IoT sensor:ΓÇ»
+
+ - `sudo nano /var/cyberx/properties/backup.properties`
+
+ - `set backup_directory_path to <backup_folder_name_on_cyberx_server>`
+
+### Restore sensors
+
+You can restore backups from the sensor console and by using the CLI.
+
+**To restore from the console:**
+
+- Select **Restore Image** from the sensor's **System Settings** window.
+
+:::image type="content" source="media/how-to-manage-individual-sensors/restore-image-screen.png" alt-text="Restore your image by selecting the button.":::
+
+The console will display restore failures.
+
+**To restore by using the CLI:**
+
+- Sign in to an administrative account and enter `$ sudo cyberx-management-system-restore`.
+
+## Update a standalone sensor version
+
+The following procedure describes how to update a standalone sensor by using the sensor console. The update process takes about 30 minutes.
+
+1. Go to the [Azure portal](https://portal.azure.com/).
+
+2. Go to Defender for IoT.
+
+3. Go to the **Updates** page.
+
+ :::image type="content" source="media/how-to-manage-individual-sensors/updates-page.png" alt-text="Screenshot of the Updates page of Defender for IoT.":::
+
+4. Select **Download** from the **Sensors** section and save the file.
+
+5. In the sensor console's sidebar, select **System Settings**.
+
+6. On the **Version Upgrade** pane, select **Upgrade**.
+
+ :::image type="content" source="media/how-to-manage-individual-sensors/upgrade-pane-v2.png" alt-text="Screenshot of the upgrade pane.":::
+
+7. Select the file that you downloaded from the Defender for IoT **Updates** page.
+
+8. The update process starts, during which time the system is rebooted twice. After the first reboot (before the completion of the update process), the system opens with the sign-in window. After you sign in, the upgrade version appears at the lower left of the sidebar.
+
+ :::image type="content" source="media/how-to-manage-individual-sensors/defender-for-iot-version.png" alt-text="Screenshot of the upgrade version that appears after you sign in.":::
+
+## Forward sensor failure alerts
+
+You can forward alerts to third parties to provide details about:
+
+- Disconnected sensors
+
+- Remote backup failures
+
+:::image type="content" source="media/how-to-work-with-system-notifications/image81.png" alt-text="Screenshot of the Management System Status Mail view.](media/image80.png) ![Screenshot of Management System Status Mail view":::
+
+This information is sent when you create a forwarding rule for system notifications.
+
+> [!NOTE]
+> Administrators can send system notifications.
+
+To send notifications:
+
+1. Sign in to the on-premises management console.
+1. Select **Forwarding** from the side menu.
+1. Create a forwarding rule.
+1. Select **Report System Notifications**.
+
+For more information about forwarding rules, see [Forward alert information](how-to-forward-alert-information-to-partners.md).
+
+## Adjust system properties
+
+System properties control various operations and settings in the sensor. Editing or modifying them might damage the operation of the sensor console.
+
+Consult with [Microsoft Support](https://support.microsoft.com/) before you change your settings.
+
+To access system properties:
+
+1. Sign in to the on-premises management console or the sensor.
+
+2. Select **System Settings**.
+
+3. Select **System Properties** from the **General** section.
+
+## See also
+
+[Threat intelligence research and packages](how-to-work-with-threat-intelligence-packages.md)
+
+[Manage sensors from the management console](how-to-manage-sensors-from-the-on-premises-management-console.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-manage-sensors-from-the-on-premises-management-console https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-manage-sensors-from-the-on-premises-management-console.md new file mode 100644
@@ -0,0 +1,315 @@
+---
+title: Manage sensors from the on-premises management console
+description: Learn how to manage sensors from the management console, including updating sensor versions, pushing system settings to sensors, and enabling and disabling engines on sensors.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/07/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Manage sensors from the management console
+
+This article describes how to manage sensors from the management console, including:
+
+- Push system settings to sensors
+
+- Enable and disable engines on sensors
+
+- Update sensor versions
+
+## Push configurations
+
+You can define various system settings and automatically apply them to sensors that are connected to the management console. This saves time and helps ensure streamlined settings across your enterprise sensors.
+
+You can define the following sensor system settings from the management console:
+
+- Mail server
+
+- SNMP MIB monitoring
+
+- Active Directory
+
+- DNS settings
+
+- Subnets
+
+- Port aliases
+
+To apply system settings:
+
+1. On the console's left pane, select **System Settings**.
+
+2. On the **Configure Sensors** pane, select one of the options.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/sensor-system-setting-options.png" alt-text="The system setting options for a sensor.":::
+
+ The following example describes how to define mail server parameters for your enterprise sensors.
+
+3. Select **Mail Server**.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/edit-system-settings-screen.png" alt-text="Select your mail server from the System Settings screen.":::
+
+4. Select a sensor on the left.
+
+5. Set the mail server parameters and select **Duplicate**. Each item in the sensor tree appears with a check box next to it.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/check-off-each-sensor.png" alt-text="Ensure the check boxes are selected for your sensors.":::
+
+6. In the sensor tree, select the items to which you want to apply the configuration.
+
+7. Select **Save**.
+
+## Update versions
+
+You can update several sensors simultaneously from the on-premises management console.
+
+To update several sensors:
+
+1. Go to the [Azure portal](https://portal.azure.com/).
+
+2. Go to Azure Defender for IoT.
+
+3. Go to the **Updates** page.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/update-screen.png" alt-text="Screenshot of the Updates dashboard view.":::
+
+4. Select **Download** from the **Sensors** section and save the file.
+
+5. Sign in to the management console and select **System Settings**.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/admin-system-settings.png" alt-text="Screenshot of the Administration menu to select System Settings.":::
+
+6. Mark the sensors that you want to update in the **Sensor Engine Configuration** section, and then select **Automatic Updates**.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/sensors-select.png" alt-text="Two sensors showing learning mode and automatic updates.":::
+
+7. Select **Save Changes**.
+
+8. On the **Sensors Version upgrade** pane, select :::image type="icon" source="media/how-to-manage-sensors-from-the-on-premises-management-console/plus-icon.png" border="false":::.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/display-files.png" alt-text="Sensor version upgrade screen to display files.":::
+
+9. An **Upload File** dialog box opens. Upload the file that you downloaded from the **Updates** page.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/upload-file.png" alt-text="Select the Browse button to upload your file.":::
+
+10. During the update process, the update status of each sensor appears in the **Site Management** window.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/progress.png" alt-text="Observe the progress of your update.":::
+
+## Update threat intelligence packages
+
+The data package for threat intelligence is provided with each new Defender for IoT version, or if needed between releases. The package contains signatures (including malware signatures), CVEs, and other security content.
+
+You can manually upload this file from the Defender for IoT portal's **Updates** page and automatically update it to sensors.
+
+To update the threat intelligence data:
+
+1. Go to the Defender for IoT **Updates** page.
+
+2. Download and save the file.
+
+3. Sign in to the management console.
+
+4. On the side menu, select **System Settings**.
+
+5. Select the sensors that should receive the update in the **Sensor Engine Configuration** section.
+
+6. In the **Select Threat Intelligence Data** section, select the plus sign (**+**).
+
+7. Upload the package that you downloaded from the Defender for IoT **Updates** page.
+
+## Understand sensor disconnection events
+
+The **Site Manager** window displays disconnection information if sensors disconnect from their assigned on-premises management console. The following sensor disconnection information is available:
+
+- "The on-premises management console cannot process data received from the sensor."
+
+- "Times drift detected. The on-premises management console has been disconnected from sensor."
+
+- "Sensor not communicating with on-premises management console. Check network connectivity or certificate validation."
+
+:::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/edit-system-settings-screen.png" alt-text="Screenshot of the zone 1 view.":::
+
+You can send alerts to third parties with information about disconnected sensors. For more information, see [Forward sensor failure alerts](how-to-manage-individual-sensors.md#forward-sensor-failure-alerts).
+
+## Enable or disable sensors
+
+Sensors are protected by five Defender for IoT engines. You can enable or disable the engines for connected sensors.
+
+| Engine | Description | Example scenario |
+|--|--|--|
+| Protocol violation engine | A protocol violation occurs when the packet structure or field values don't comply with the protocol specification. | "Illegal MODBUS Operation (Function Code Zero)" alert. This alert indicates that a primary device sent a request with function code 0 to a secondary device. This is not allowed according to the protocol specification, and the secondary device might not handle the input correctly. |
+| Policy violation engine | A policy violation occurs with a deviation from baseline behavior defined in the learned or configured policy. | "Unauthorized HTTP User Agent" alert. This alert indicates that an application that was not learned or approved by the policy is used as an HTTP client on a device. This might be a new web browser or application on that device. |
+| Malware engine | The malware engine detects malicious network activity. | "Suspicion of Malicious Activity (Stuxnet)" alert. This alert indicates that the sensor found suspicious network activity known to be related to the Stuxnet malware, which is an advanced persistent threat aimed at industrial control and SCADA networks. |
+| Anomaly engine | The malware engine detects an anomaly in network behavior. | "Periodic Behavior in Communication Channel." This is a component that inspects network connections and finds periodic or cyclic behavior of data transmission, which is common in industrial networks. |
+| Operational engine | This engine detects operational incidents or malfunctioning entities. | "Asset is Suspected to be Disconnected (Unresponsive)" alert. This alert triggered when a device is not responding to any requests for a predefined period. It might indicate a device shutdown, disconnection, or malfunction.
+|
+
+To enable or disable engines for connected sensors:
+
+1. In the console's left pane, select **System Settings**.
+
+2. In the **Sensor Engine Configuration** section, select **Enable** or **Disable** for the engines.
+     
+3. Select **SAVE CHANGES**.
+
+ A red exclamation mark appears if there's a mismatch of enabled engines on one of your enterprise sensors. The engine might have been disabled directly from the sensor.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/red-exclamation-example.png" alt-text="Mismatch of enabled engines.":::
+
+## Define sensor backup schedules
+
+You can schedule sensor backups and perform on-demand sensor backups from the on-premises management console. This helps protect against hard drive failures and data loss.
+
+- What is backed up: Configurations and data.
+
+- What isn't backed up: PCAP files and logs. You can manually back up and restore PCAPs and logs.
+
+By default, sensors are automatically backed up at 3:00 AM daily. The backup schedule feature for sensors lets you collect these backups and store them on the on-premises management console or on an external backup server. Copying files from sensors to the on-premises management console happens over an encrypted channel.
+
+:::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/sensor-backup-schedule-screen.png" alt-text="A view of the sensor backup screen.":::
+
+When the default sensor backup location is changed, the on-premises management console automatically retrieves the files from the new location on the sensor or an external location, provided that the console has permission to access the location.
+
+When the sensors are not registered with the on-premises management console, the **Sensor Backup Schedule** dialog box indicates that no sensors are managed.
+
+The restore process is the same regardless of where the files are stored.
+
+### Backup storage for sensors
+
+You can use the on-premises management console to maintain up to nine backups for each managed sensor, provided that the backed-up files don't exceed the maximum backup space that's allocated.
+
+The available space is calculated based on the management console model you're working with:
+
+- **Production model**: Default storage is 40 GB; limit is 100 GB.
+
+- **Medium model**: Default storage is 20 GB; limit is 50 GB.
+
+- **Laptop model**: Default storage is 10 GB; limit is 25 GB.
+
+- **Thin model**: Default storage is 2 GB; limit is 4 GB.
+
+- **Rugged model**: Default storage is 10 GB; limit is 25 GB.
+
+The default allocation is displayed in the **Sensor Backup Schedule** dialog box.
+
+:::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/edit-mail-server-configuration.png" alt-text="The Edit Mail Server Configuration screen.":::
+
+There is no storage limit when you're backing up to an external server. You must, however, define an upper allocation limit in the **Sensor Backup Schedule** > **Custom Path** field. The following numbers and characters are supported: `/, a-z, A-Z, 0-9, and _`.
+
+Here's information about exceeding allocation storage limits:
+
+- If you exceed the allocated storage space, the sensor is not backed up.
+
+- If you're backing up more than one sensor, the management console tries to retrieve sensor files for the managed sensors.
+
+- If the retrieval from one sensor exceeds the limit, the management console tries to retrieve backup information from the next sensor.
+
+When you exceed the retained number of backups defined, the oldest backed-up file is deleted to accommodate the new one.
+
+Sensor backup files are automatically named in the following format: `<sensor name>-backup-version-<version>-<date>.tar`. For example: `Sensor_1-backup-version-2.6.0.102-2019-06-24_09:24:55.tar`.
+
+To back up sensors:
+
+1. Select **Schedule Sensor Backup** from the **System Settings** window. Sensors that your on-premises management console manages appear in the **Sensor Backup Schedule** dialog box.
+
+2. Enable the **Collect Backups** toggle.
+
+3. Select a calendar interval, date, and time zone. The time format is based on a 24-hour clock. For example, enter 6:00 PM as **18:00**.
+
+4. In the **Backup Storage Allocation** field, enter the storage that you want to allocate for your backups. You're notified if you exceed the maximum space.
+
+5. In the **Retain Last** field, indicate the number of backups per sensor you want to retain. When the limit is exceeded, the oldest backup is deleted.
+
+6. Choose a backup location:
+
+ - To back up to the on-premises management console, disable the **Custom Path** toggle. The default location is `/var/cyberx/sensor-backups`.
+
+ - To back up to an external server, enable the **Custom Path** toggle and enter a location. The following numbers and characters are supported: `/, a-z, A-Z, 0-9, and, _`.
+
+7. Select **Save**.
+
+To back up immediately:
+
+- Select **Back Up Now**. The on-premises management console creates and collects sensor backup files.
+
+### Receiving backup notifications for sensors
+
+The **Sensor Backup Schedule** dialog box and the backup log automatically list information about backup successes and failures.
+
+:::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/sensor-location.png" alt-text="View your sensors and where they are located and all relevant information.":::
+
+Failures might occur because:
+
+- No backup file is found.
+
+- A file was found but can't be retrieved.
+
+- There's a network connection failure.
+
+- There's not enough room allocated to the on-premises management console to complete the backup.
+
+You can send an email notification, syslog updates, and system notifications when a failure occurs. To do this, create a forwarding rule in **System Notifications**.
+
+### Restoring sensors
+
+You can restore backups from the on-premises management console and by using the CLI.
+
+To restore from the console:
+
+- Select **Restore Image** from the **Sensor System** setting window.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/restore.png" alt-text="Restore a backup of your image.":::
+
+ The console then displays restore failures.
+
+To restore by using the CLI:
+
+- Sign in to an administrative account and enter `$ sudo cyberx-xsense-system-restore`.
+
+### Save a sensor backup to an external SMB server
+
+To set up an SMB server so you can save a sensor backup to an external drive:
+
+1. Create a shared folder in the external SMB server.
+
+2. Get the folder path, username, and password required to access the SMB server.
+
+3. In Defender for IoT, make a directory for the backups:
+
+ `sudo mkdir /<backup_folder_name_on_server>`
+
+ `sudo chmod 777 /<backup_folder_name_on_server>/`
+
+4. Edit fstab:ΓÇ»
+
+ `sudo nano /etc/fstab`
+
+ `add - //<server_IP>/<folder_path> /<backup_folder_name_on_cyberx_server> cifs rw,credentials=/etc/samba/user,vers=3.0,uid=cyberx,gid=cyberx,file_mode=0777,dir_mode=0777 0 0`
+
+5. Edit or create credentials to share. These are the credentials for the SMB server:
+
+ `sudo nano /etc/samba/user`
+
+6. Add:ΓÇ»
+
+ `username=<user name>`
+
+ `password=<password>`
+
+7. Mount the directory:
+
+ `sudo mount -a`
+
+8. Configure a backup directory to the shared folder on the Defender for IoT sensor:ΓÇ»
+
+ `sudo nano /var/cyberx/properties/backup.properties`
+
+9. Set `Backup.shared_location` to `<backup_folder_name_on_cyberx_server>`.
+
+## See also
+
+[Manage individual sensors](how-to-manage-individual-sensors.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-manage-sensors-on-the-cloud https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-manage-sensors-on-the-cloud.md new file mode 100644
@@ -0,0 +1,117 @@
+---
+title: Onboard and manage sensors in the Defender for IoT portal
+description: Learn how to onboard, view, and manage sensors in the Defender for IoT portal.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/27/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Onboard and manage sensors in the Defender for IoT portal
+
+This article describes how to onboard, view, and manage sensors in the Defender for IoT portal.
+
+## Onboard sensors
+
+You onboard a sensor by registering it with Azure Defender for IoT and downloading a sensor activation file.
+
+### Register the sensor
+
+To register:
+
+1. Go to the **Welcome** page in the Defender for IoT portal.
+1. Select **Onboard sensor**.
+1. Create a sensor name. We recommend that you include the IP address of the sensor you installed as part of the name, or use an easily identifiable name. This will ensure easier tracking and consistent naming between the registration name in the Azure Defender for IoT portal and the IP of the deployed sensor displayed in the sensor console.
+1. Associate the sensor with an Azure subscription.
+1. Choose a sensor management mode by using the **Cloud connected** toggle. If the toggle is on, the sensor is cloud connected. If the toggle is off, the sensor is locally managed.
+
+ - **Cloud-connected sensors**: Information that the sensor detects is displayed in the sensor console. Alert information is delivered through an IoT hub and can be shared with other Azure services, such as Azure Sentinel.
+
+ - **Locally managed sensors**: Information that sensors detect is displayed in the sensor console. If you're working in an air-gapped network and want a unified view of all information detected by multiple locally managed sensors, work with the on-premises management console.
+
+ For cloud-connected sensors, the name defined during onboarding is the name that appears in the sensor console. You can't change this name from the console directly. For locally managed sensors, the name applied during onboarding will be stored in Azure and can be updated in the sensor console.
+
+1. Choose an IoT hub to serve as a gateway between this sensor and Azure Defender for IoT.
+1. If the sensor is cloud connected, associate it with an IoT hub, and then define a site name and zone. You can also add descriptive tags. The site name, zone, and tags are descriptive entries on the [Sites and Sensors page](#view-onboarded-sensors).
+
+### Download the sensor activation file
+
+The sensor activation file contains instructions about the management mode of the sensor. You download a unique activation file for each sensor that you deploy. A user who signs in to the sensor console for the first time uploads the activation file to the sensor.
+
+To download an activation file:
+
+1. On the **Onboard Sensor** page, select **download activation file**.
+1. Make the file accessible to the user who's signing in to the sensor console for the first time.
+
+## View onboarded sensors
+
+On the Defender for IoT portal, you can view basic information about onboarded sensors.
+
+1. Select **Sites and Sensors**.
+1. On the **Sites and Sensors** page, use filter and search tools to find sensor information that you need.
+
+The available information includes:
+
+- How many sensors were onboarded
+- The number of sensors that are cloud connected and locally managed
+- The hub associated with an onboarded sensor
+- Details added about a sensor, such as the name assigned to the sensor during onboarding, the zone associated with the sensor, or other descriptive information added with tags
+
+## Manage onboarded sensors
+
+You use the Defender for IoT portal for management tasks related to sensors.
+
+### Export
+
+To export onboarded sensor information, select the **Export** icon on the top of the **Sites and Sensors** page.
+
+### Edit
+
+Use the **Sites and Sensors** editing tools to add and edit the site name, zone, and tags.
+
+### Delete
+
+If you delete a cloud-connected sensor, information won't be sent to the IoT hub. Delete locally connected sensors when you're no longer working with them.
+
+To delete a sensor:
+
+1. Select the ellipsis (**...**) for the sensor you want to delete.
+1. Confirm the deletion.
+
+### Reactivate
+
+You might want to update the mode that your sensor is managed in. For example:
+
+- **Work in cloud-connected mode instead of locally managed mode**: To do this, update the activation file for your locally connected sensor with an activation file for a cloud-connected sensor. After reactivation, sensor detections are displayed in both the sensor and the Defender for IoT portal. After the reactivation file is successfully uploaded, newly detected alert information is sent to Azure.
+
+- **Work in locally connected mode instead of cloud-connected mode**: To do this, update the activation file for a cloud-connected sensor with an activation file for a locally managed sensor. After reactivation, sensor detection information is displayed only in the sensor.
+
+- **Associate the sensor to a new IoT hub**: To do this, re-register then sensor and upload a new activation file.
+
+To reactivate a sensor:
+
+1. Go to **Sites and Sensors** page on the Defender for IoT portal.
+
+2. Select the sensor for which you want to upload a new activation file.
+
+3. Delete the sensor.
+
+4. Onboard the sensor again from the **Onboarding** page in the new mode or with a new IoT hub.
+
+5. Download the activation file from the **Download Activation File** page.
+
+6. Sign in to the Defender for IoT sensor console.
+
+7. In the sensor console, select **System Settings** and then select **Reactivation**.
+
+ :::image type="content" source="media/how-to-manage-sensors-on-the-cloud/reactivate.png" alt-text="Upload your activation file to reactivate the sensor.":::
+
+8. Select **Upload** and select the file you saved.
+
+9. Select **Activate**.
+
+## See also
+
+[Activate and set up your sensor](how-to-activate-and-set-up-your-sensor.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-manage-the-alert-event https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-manage-the-alert-event.md new file mode 100644
@@ -0,0 +1,108 @@
+---
+title: Manage alert events
+description: Manage alert events detected in your network.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/07/2020
+ms.service: azure
+ms.topic: how-to
+---
+
+# Manage alert events
+
+The following options are available for managing alert events:
+
+ | Action | Description |
+ |--|--|
+ | **Learn** | Authorize the detected event. For more information, see [About learning and unlearning events](#about-learning-and-unlearning-events). |
+ | **Acknowledge** | Hide the alert once for the detected event. The alert will be triggered again if the event is detected again. For more information, see [About acknowledging and unacknowledging events](#about-acknowledging-and-unacknowledging-events). |
+ | **Mute** | Continuously ignore activity with identical devices and comparable traffic. For more information, see [About muting and unmuting events](#about-muting-and-unmuting-events). |
+
+## About learning and unlearning events
+
+Events that indicate deviations of the learned network might reflect valid network changes. Examples might include a new authorized device that joined the network or an authorized firmware update.
+
+When you select **Learn**, the sensor considers traffic, configurations, or activity valid. This means alerts will no longer be triggered for the event. It also means the event won't be calculated when the sensor generates risk assessment, attack vector, and other reports.
+
+For example, you receive an alert that indicates address scanning activity on a device that a network scanner didn't previously define. If this device was added to the network for the purpose of scanning, you can instruct the sensor to learn the device as a scanning device.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/detected.png" alt-text="The Address Detected Scan window.":::
+
+Learned events can be unlearned. When the sensor unlearns events, it will retrigger alerts related to this event.
+
+## About acknowledging and unacknowledging events
+
+In certain situations, you might not want a sensor to learn a detected event, or the option might not be available. Instead, the incident might require mitigation. For example:
+
+- **Mitigate a network configuration or device**: You receive an alert indicating that a new device was detected on the network. When investigating, you discover that the device is an unauthorized network device. You handle the incident by disconnecting the device from the network.
+- **Update a sensor configuration**: You receive an alert indicating that a server initiated an excessive number of remote connections. This alert was triggered because the sensor anomaly thresholds were defined to trigger alerts above a certain number of sessions within one minute. You handle the incident by updating the thresholds.
+
+After you carry out mitigation or investigation, you can instruct the sensor to hide the alert by selecting **Acknowledge**. If the event is detected again, the alert will be retriggered.
+
+To hide the alert:
+
+ - Select **Acknowledge**.
+
+To view the alert again:
+
+ - Access the alert and select **Unacknowledge**.
+
+Unacknowledge alerts if further investigation is required.
+
+## About muting and unmuting events
+
+Under certain circumstances, you might want to instruct your sensor to ignore a specific scenario on your network. For example:
+
+ - The **Anomaly** engine triggers an alert on a spike in bandwidth between two devices, but the spike is valid for these devices.
+
+ - The **Protocol Violation** engine triggers an alert on a protocol deviation detected between two devices, but the deviation is valid between the devices.
+
+In these situations, learning is not available. When learning can't be carried out and you want to suppress the alert and remove the device when calculating risks and attack vectors, you can mute the alert event instead.
+
+> [!NOTE]
+> You can't mute events in which an internet device is defined as the source or destination.
+
+### What traffic is muted?
+
+A muted scenario includes the network devices and traffic detected for an event. The alert title describes the traffic that's being muted.
+
+The device or devices being muted will be displayed as an image in the alert. If two devices are shown, the traffic between them will be muted.
+
+**Example 1**
+
+When an event is muted, it's is ignored any time the primary (source) sends the secondary (destination) an illegal function code as defined by the vendor.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/secondary-device-connected.png" alt-text="Secondary device received.":::
+
+**Example 2**
+
+When an event is muted, it's ignored any time the source sends an HTTP header with illegal content, regardless of the destination.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/illegal-http-header-content.png" alt-text="Screenshot of illegal HTTP header content.":::
+
+**After an event is muted:**
+
+- The alert will be accessible in the **Acknowledged** alert view until it's unmuted.
+
+- The mute action will appear in the **Event Timeline**.
+
+ :::image type="content" source="media/how-to-work-with-alerts-sensor/muted-event-notification-screenshot.png" alt-text="Event detected and muted.":::
+
+- The sensor will recalculate devices when generating risk assessment, attack vector, and other reports. For example, if you muted an alert that detected malicious traffic on a device, that device will not be calculated in the risk assessment report.
+
+**To mute and unmute an alert:**
+
+- Select the **Mute** icon on the alert.
+
+**To view muted alerts:**
+
+1. Select the **Acknowledged** option form the **Alerts** main screen.
+
+2. Hover over an alert to see if it's muted.
+
+## See also
+
+[Generate reports](how-to-generate-reports.md)
+
+[Control what traffic is monitored](how-to-control-what-traffic-is-monitored.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-manage-the-on-premises-management-console https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-manage-the-on-premises-management-console.md new file mode 100644
@@ -0,0 +1,301 @@
+---
+title: Manage the on-premises management console
+description: Learn about on-premises management console options like backup and restore, defining the host name, and setting up a proxy to sensors.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/12/2020
+ms.topic: article
+ms.service: azure
+---
+
+# Manage the on-premises management console
+
+This article covers on-premises management console options like backup and restore, downloading committee device activation file, updating certificates, and setting up a proxy to sensors.
+
+You onboard the on-premises management console from the Azure portal.
+
+## Upload an activation file
+
+When you first sign in, an activation file for the on-premises management console is downloaded. This file contains the aggregate committed devices that are defined during the onboarding process. The list includes sensors associated with multiple subscriptions.
+
+After initial activation, the number of monitored devices might exceed the number of committed devices defined during onboarding. This event might happen, for example, if you connect more sensors to the management console. If there's a discrepancy between the number of monitored devices and the number of committed devices, a warning appears in the management console. If this event occurs, you should upload a new activation file.
+
+To upload an activation file:
+
+1. Go to the Azure Defender for IoT **Pricing** page.
+1. Select the **Download the activation file for the management console** tab. The activation file is downloaded.
+
+ :::image type="content" source="media/how-to-manage-sensors-from-the-on-premises-management-console/cloud_download_opm_activation_file.png" alt-text="Download the activation file.":::
+
+1. Select **System Settings** from the management console.
+1. Select **Activation**.
+1. Select **Choose a File** and select the file that you saved.
+
+## Manage certificates
+
+After installation of the on-premises management console, a local self-signed certificate is generated and used to access the management console's web application. When administrator users sign in to the management console for the first time, they're prompted to provide an SSL/TLS certificate. For more information about first-time setup, see [Activate and set up your on-premises management console](how-to-activate-and-set-up-your-on-premises-management-console.md).
+
+The following sections provide information on updating certificates, working with certificate CLI commands, and supported certificates and certificate parameters.
+
+### About certificates
+
+Azure Defender for IoT uses SSL and TLS certificates to:
+
+- Meet specific certificate and encryption requirements requested by your organization by uploading the CA-signed certificate.
+
+- Allow validation between the management console and connected sensors, and between a management console and a high-availability management console. Validation is evaluated against a certificate revocation list and the certificate expiration date. *If validation fails, communication between the management console and the sensor is halted and a validation error appears in the console.* This option is enabled by default after installation.
+
+Third-party forwarding rules aren't validated. Examples are alert information sent to SYSLOG, Splunk, or ServiceNow; and communication with Active Directory.
+
+### Update certificates
+
+Administrator users of the on-premises management console can update certificates.
+
+To update a certificate:
+
+1. Select **System Settings**.
+1. Select **SSL/TLS Certificates**.
+1. Delete or edit the certificate and add a new one.
+
+ - Add a certificate name.
+ - Upload a CRT file and key file, and enter a passphrase.
+ - Upload a PEM file if necessary.
+
+To change the validation setting:
+
+1. Turn on or turn off the **Enable Certificate Validation** toggle.
+1. Select **Save**.
+
+If the option is enabled and validation fails, communication between the management console and the sensor is halted and a validation error appears in the console.
+
+### Certificate support
+
+The following certificates are supported:
+
+- Private and Enterprise Key Infrastructure (Private PKI)
+- Public Key Infrastructure (Public PKI)
+- Locally generated on the appliance (locally self-signed)
+
+ > [!IMPORTANT]
+ > We don't recommend that you use self-signed certificates. This connection is not secure and should be used for test environments only. The owner of the certificate can't be validated, and the security of your system can't be maintained. Self-signed certificates should never be used for production networks.
+
+The following parameters are supported:
+
+**Certificate CRT**
+
+- The primary certificate file for your domain name
+- Signature Algorithm = SHA256RSA
+- Signature Hash Algorithm = SHA256
+- Valid from = Valid past date
+- Valid To = Valid future date
+- Public Key = RSA 2048 bits (Minimum) or 4096 bits
+- CRL Distribution Point = URL to .crl file
+- Subject CN = URL, can be a wildcard certificate; for example, www.contoso.com or \*.contoso.com
+- Subject (C)ountry = defined, for example, US
+- Subject (OU) Org Unit = defined; for example, Contoso Labs
+- Subject (O)rganization = defined; for example, Contoso Inc
+
+**Key file**
+
+- The key file generated when you created the CSR
+- RSA 2048 bits (minimum) or 4096 bits
+
+**Certificate chain**
+
+- The intermediate certificate file (if any) that was supplied by your CA.
+- The CA certificate that issued the server's certificate should be first in the file, followed by any others up to the root.
+- The chain can include bag attributes.
+
+**Passphrase**
+
+- One key is supported.
+- Set up when you're importing the certificate.
+
+Certificates with other parameters might work, but Microsoft doesn't support them.
+
+#### Encryption key artifacts
+
+**.pem: certificate container file**
+
+The name is from Privacy Enhanced Mail (PEM), a historic method for secure email. The container format is a Base64 translation of the x509 ASN.1 keys. 
+
+This file is defined in RFCs 1421 to 1424: a container format that might include just the public certificate (such as with Apache installations, CA certificate files, and ETC SSL certificates). Or it might include an entire certificate chain, including a public key, a private key, and root certificates.
+
+It might also encode a CSR, because the PKCS10 format can be translated into PEM.
+
+**.certΓÇ».cerΓÇ».crt: certificate container file**
+
+This is a .pem (or rarely, .der) formatted file with a different extension. Windows File Explorer recognizes it as a certificate. File Explorer doesn't recognize the .pem file.
+
+**.key: private key file**
+
+A key file is the same format as a PEM file, but it has a different extension.
+
+#### CLI commands
+
+Use the `cyberx-xsense-certificate-import` CLI command to import certificates. To use this tool, you need to upload certificate files to the device (by using tools such as winscp or wget).
+
+The command supports the following input flags:
+
+- `-h`: Shows the command-line help syntax.
+
+- `--crt`: Path to a certificate file (.crt extension).
+
+- `--key`: \*.key file. Key length should be a minimum of 2,048 bits.
+
+- `--chain`: Path to a certificate chain file (optional).
+
+- `--pass`: Passphrase used to encrypt the certificate (optional).
+
+- `--passphrase-set`: Default = `False`, unused. Set to `True` to use the previous passphrase supplied with the previous certificate (optional).
+
+When you're using the CLI command:
+
+- Verify that the certificate files are readable on the appliance.
+
+- Verify that the domain name and IP in the certificate match the configuration that the IT department has planned.
+
+## Define backup and restore settings
+
+The on-premises management console system backup is performed automatically, daily. The data is saved on a different disk. The default location is `/var/cyberx/backups`.
+
+You can automatically transfer this file to the internal network.
+
+> [!NOTE]
+> You can perform the backup and restore procedure on the same version only.
+
+To back up the on-premises management console machine:
+
+- Sign in to an administrative account and enter `sudo cyberx-management-backup -full`.
+
+To restore the latest backup file:
+
+- Sign in to an administrative account and enter `$ sudo cyberx-management-system-restore`.
+
+To save the backup to an external SMB server:
+
+1. Create a shared folder in the external SMB server.
+
+ Get the folder path, username, and password required to access the SMB server.
+
+2. In Defender for IoT, make a directory for the backups:
+
+ - `sudo mkdir /<backup_folder_name_on_ server>`
+
+ - `sudo chmod 777 /<backup_folder_name_on_c_server>/`
+
+3. Edit fstab:ΓÇ»
+
+ - `sudo nano /etc/fstab`
+
+ - `add - //<server_IP>/<folder_path> /<backup_folder_name_on_server> cifs rw,credentials=/etc/samba/user,vers=3.0,uid=cyberx,gid=cyberx,file_mode=0777,dir_mode=0777 0 0`
+
+4. Edit or create credentials for the SMB server to share:
+
+ - `sudo nano /etc/samba/user`
+
+5. Add:
+
+ - `username=<user name>`
+
+ - `password=<password>`
+
+6. Mount the directory:
+
+ - `sudo mount -a`
+
+7. Configure a backup directory to the shared folder on the Defender for IoT on-premises management console:ΓÇ»
+
+ - `sudo nano /var/cyberx/properties/backup.properties`
+
+ - `set Backup.shared_location to <backup_folder_name_on_server>`
+
+## Edit the host name
+
+To edit the management console's host name configured in the organizational DNS server:
+
+1. In the management console's left pane, select **System Settings**.
+
+2. In the console's networking section, select **Network**.
+
+3. Enter the host name configured in the organizational DNS server.
+
+4. Select **Save**.
+
+## Define VLAN names
+
+VLAN names are not synchronized between the sensor and the management console. You should define identical names on components.
+
+In the networking area, select **VLAN** and add names to the discovered VLAN IDs. Then select **Save**.
+
+## Define a proxy to sensors
+
+Enhance system security by preventing user sign-in directly to the sensor. Instead, use proxy tunneling to let users access the sensor from the on-premises management console with a single firewall rule. This enhancement narrows the possibility of unauthorized access to the network environment beyond the sensor.
+
+Use a proxy in environments where there's no direct connectivity to sensors.
+
+:::image type="content" source="media/how-to-access-sensors-using-a-proxy/proxy-diagram.png" alt-text="User.":::
+
+The following procedure connects a sensor to the on-premises management console and enables tunneling on that console:
+
+1. Sign in to the on-premises management console appliance CLI with administrative credentials.
+
+2. Type `sudo cyberx-management-tunnel-enable` and select **Enter**.
+
+4. Type `--port 10000` and select **Enter**.
+
+## Adjust system properties
+
+System properties control various operations and settings in the management console. Editing or modifying them might damage the management console's operation. Consult with [Microsoft Support](https://support.microsoft.com) before changing your settings.
+
+To access system properties:
+
+1. Sign in to the on-premises management console or the sensor.
+
+2. Select **System Settings**.
+
+3. Select **System Properties** from the **General** section.
+
+## Change the name of the on-premises management console
+
+You can change the name of the on-premises management console. The new names appear in the console web browser, in various console windows, and in troubleshooting logs. The default name is **management console**.
+
+To change the name:
+
+1. In the bottom of the left pane, select the current name.
+
+ :::image type="content" source="media/how-to-change-the-name-of-your-azure-consoles/console-name.png" alt-text="Screenshot of the on-premises management console version.":::
+
+2. In the **Edit management console configuration** dialog box, enter the new name. The name can't be longer than 25 characters.
+
+ :::image type="content" source="media/how-to-change-the-name-of-your-azure-consoles/edit-management-console-configuration.png" alt-text="Screenshot of editing the Defender for IoT platform configuration.":::
+
+3. Select **Save**. The new name is applied.
+
+ :::image type="content" source="media/how-to-change-the-name-of-your-azure-consoles/name-changed.png" alt-text="Screenshot that shows the changed name of the console.":::
+
+## Password recovery
+
+Password recovery for your on-premises management console is tied to the subscription that the device is attached to. You can't recover a password if you don't know which subscription a device is attached to.
+
+To reset your password:
+
+1. Go to the on-premises management console's sign-in page.
+1. Select **Password Recovery**.
+1. Copy the unique identifier.
+1. Go to the Defender for IoT **Sites and sensors** page and select the **Recover my password** tab.
+1. Enter the unique identifier and select **Recover**. The activation file is downloaded.
+1. Go to the **Password Recovery** page and upload the activation file.
+1. Select **Next**.
+
+ You're now given your username and a new system-generated password.
+
+> [!NOTE]
+> The sensor is linked to the subscription that it was originally connected to. You can recover the password only by using the same subscription that it's attached to.
+
+## See also
+
+[Manage sensors from the management console](how-to-manage-sensors-from-the-on-premises-management-console.md)
+
+[Manage individual sensors](how-to-manage-individual-sensors.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-set-up-high-availability https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-set-up-high-availability.md new file mode 100644
@@ -0,0 +1,148 @@
+---
+title: Set up high availability
+description: Increase the resiliency of your Defender for IoT deployment by installing a on-premises management console high availability appliance. High availability deployments ensure your managed sensors continuously report to an active on-premises management console.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/07/2020
+ms.topic: how-to
+ms.service: azure
+---
+# About high availability
+
+Increase the resiliency of your Defender for IoT deployment by installing a on-premises management console high availability appliance. High availability deployments ensure your managed sensors continuously report to an active on-premises management console.
+
+This deployment is implemented with an on-premises management console pair that includes a primary and secondary appliance.
+
+## About primary and secondary communication
+
+When a primary and secondary on-premises management console is paired:
+
+- An on-premises management console SLL certificate is applied to create a secure connection between the primary and secondary appliances. The SLL may be the self-signed certificate installed by default or a certificate installed by the customer.
+
+- The primary on-premises management console data is automatically backed up to the secondary on-premises management console every 10 minutes. The on-premises management console configurations and device data are backed up. PCAP files and logs are not included in the backup. You can back up and restore of PCAPs and logs manually.
+
+- The primary setup at the management console is duplicated on the secondary; for example, system settings. If these settings are updated on the primary, they are also updated on the secondary.
+
+- Before the license of the secondary expires, you should define it as the primary in order to update the license.
+
+## About failover and failback
+
+If a sensor cannot connect to the primary on-premises management console, it automatically connects to the secondary. Your system will be supported by both the primary and secondary simultaneously, if less than half of the sensors are communicating with the secondary. The secondary takes over when more than half of the sensors are communicating with it. Fail over from the primary to the secondary takes approximately three minutes. When the failover occurs, the primary on-premises management console freezes. When this happens, you can sign in to the secondary using the same sign-in credentials.
+
+During failover, sensors continue attempting to communicate with the primary appliance. When more than half the managed sensors succeed to communicate with the primary, the primary is restored. The following message appears at the secondary console when the primary is restored.
+
+:::image type="content" source="media/how-to-set-up-high-availability/secondary-console-message.png" alt-text="Screenshot of a message that appears at the secondary console when the primary is restored.":::
+
+Sign back in to the primary appliance after redirection.
+
+## High availability setup overview
+
+The installation and configuration procedures are performed in four main stages:
+
+1. Install an on-premises management console primary appliance.
+
+2. Configure the on-premises management console primary appliance. For example, scheduled backup settings, VLAN settings. See the on-premises management console user guide for details. All settings are applied to the secondary appliance automatically after pairing.
+
+3. Install a on-premises management console secondary appliance. For more information see, [About the Defender for IoT Installation](how-to-install-software.md).
+
+4. Pair the primary and secondary on-premises management console appliances as described [here](/create-the-primary-and-secondary-pair.md). The primary on-premises management console must manage at least two sensors in order to carry out the setup.
+
+## High availability requirements
+
+Verify that you have met the following high availability requirements:
+
+- Certificate requirements
+
+- Software and hardware requirements
+
+- Network access requirements
+
+### Software and hardware requirements
+
+- Both the primary and secondary on-premises management console appliances must be running identical hardware models and software versions.
+
+- The high availability system can be set up by Defender for IoT users only, using CLI tools.
+
+### Network access requirements
+
+You need to verify that your organizational security policy allows you access to the following services on the primary and secondary on-premises management console. These services also allow the connection between the sensors and secondary on-premises management console:
+
+|Port|Service|Description|
+|----|-------|-----------|
+|**443 or TCP**|HTTPS|Grants access to the on-premises management console web console.|
+|**22 or TCP**|SSH|Syncs the data between the primary and secondary on-premises management console appliances|
+|**123 or UDP**|NTP| The on-premises management console's NTP time sync. Verify that the active and passive appliances are defined with the same timezone.|
+
+## Create the primary and secondary pair
+
+Verify that both the primary and secondary on-premises management console appliances are powered on before starting the procedure.  
+
+### On the primary
+
+1. Sign in to the CLI as a Defender for IoT user.
+
+2. Run the following command on the primary:
+
+```azurecli-interactive
+sudo cyberx-management-trusted-hosts-add -ip <Secondary IP>
+```
+
+>[!NOTE]
+>In this document, the principal on-premises management console is referred to as the primary, and the agent is referred to as the secondary.
+
+3. Enter the IP address of the secondary appliance in the ```<Secondary ip>``` field and select Enter. The IP address is then validated, and the SSL certificate is downloaded to the primary. Entering the IP address also associates the sensors to the secondary appliance.
+
+4. Run the following command on the primary to verify that the certificate is installed properly:
+
+```azurecli-interactive
+sudo cyberx-management-trusted-hosts-apply
+```
+
+5. Run the following command on the primary. **Do not run with sudo.**
+
+```azurecli-interactive
+cyberx-management-deploy-ssh-key <Secondary IP>
+```
+
+This allows the connection between the primary and secondary appliances for backup and restoration purposes between them.
+
+6. Enter the IP address of the secondary and select Enter.
+
+### On the secondary
+
+1. Sign in to the CLI as an Defender for IoT user.
+
+2. Run the following command on the secondary. **Do not run with sudo**:
+
+```azurecli-interactive
+cyberx-management-deploy-ssh-key <Primary ip>
+```
+
+This allows the connection between the Primary and Secondary appliances for backup and restore purposes between them.
+
+3. Enter the IP address of the primary and press Enter.
+
+### Track high availability activity
+
+The core application logs can be exported to the Defender for IoT support team to handle any high availability issues.  
+
+To access the core logs:
+
+1. Select **Export** from the **System Settings** window.
+
+## Update the on-premises management console with high availability
+
+Perform the high availability update in the following order. Make sure each step is complete before you begin a new step.
+
+To update with high availability:
+
+1. Update the primary on-premises management console.
+
+2. Update the secondary on-premises management console.
+
+3. Update the sensors.
+
+## See also
+
+[Activate and set up your on-premises management console](how-to-activate-and-set-up-your-on-premises-management-console.md)
\ No newline at end of file
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-set-up-snmp-mib-monitoring https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-set-up-snmp-mib-monitoring.md new file mode 100644
@@ -0,0 +1,84 @@
+---
+title: Set up SNMP MIB monitoring
+description: You can perform sensor health monitoring by using SNMP. The sensor responds to SNMP queries sent from an authorized monitoring server.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/14/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Set up SNMP MIB monitoring
+
+You can perform sensor health monitoring by using Simple Network Management Protocol (SNMP). The sensor responds to SNMP queries sent from an authorized monitoring server. The SNMP monitor polls the sensor OIDs periodically (up to 50 times a second).
+
+The SNMP supported versions are SNMP v2 or SNMP v3. SNMP uses UDP as its transport protocol with port 161 (SNMP).
+
+Before you begin configuring SNMP monitoring, you need to open the port UDP 161 in the firewall.
+
+## OIDs
+
+| Management console and sensor | OID | Format | Description |
+|--|--|--|--|
+| Appliance name | 1.3.6.1.2.1.1.5.0 | DISPLAYSTRING | Appliance name for the on-premises management console |
+| Vendor | 1.3.6.1.2.1.1.4.0 | DISPLAYSTRING | Microsoft Support (support.microsoft.com) |
+| Platform | 1.3.6.1.2.1.1.1.0 | DISPLAYSTRING | Sensor or on-premises management console |
+| Serial number | 1.3.6.1.4.1.9.9.53313.1 | DISPLAYSTRING | String that the license uses |
+| Software version | 1.3.6.1.4.1.9.9.53313.2 | DISPLAYSTRING | Xsense full-version string and management full-version string |
+| CPU usage | 1.3.6.1.4.1.9.9.53313.3.1 | GAUGE32 | Indication for zero to 100 |
+| CPU temperature | 1.3.6.1.4.1.9.9.53313.3.2 | DISPLAYSTRING | Celsius indication for zero to 100 based on Linux input |
+| Memory usage | 1.3.6.1.4.1.9.9.53313.3.3 | GAUGE32 | Indication for zero to 100 |
+| Disk Usage | 1.3.6.1.4.1.9.9.53313.3.4 | GAUGE32 | Indication for zero to 100 |
+| Service Status | 1.3.6.1.4.1.9.9.53313.5 | DISPLAYSTRING | Online or offline if one of the four crucial components is down |
+| Bandwidth | Out of scope for 2.4 | | The bandwidth received on each monitor interface in Xsense |
+
+ - Non-existing keys respond with null, HTTP 200, based on [Stack Overflow](https://stackoverflow.com/questions/51419026/querying-for-non-existing-record-returns-null-with-http-200).
+
+ - Hardware-related MIBs (CPU usage, CPU temperature, memory usage, disk usage) should be tested on all architectures and physical sensors. CPU temperature on virtual machines is expected to be not applicable.
+
+You can download the log that contains all the SNMP queries that the sensor receives, including the connection data and raw data from the packet.
+
+To define SNMP v2 health monitoring:
+
+1. On the side menu, select **System Settings**.
+
+2. In the **Active Discovery** pane, select **SNMP MIB Monitoring** :::image type="icon" source="media/how-to-set-up-snmp-mib-monitoring/snmp-icon.png" border="false":::.
+
+ :::image type="content" source="media/how-to-set-up-snmp-mib-monitoring/edit-snmp.png" alt-text="Edit your SNMP window.":::
+
+3. In the **Allowed Hosts** section, select **Add host** and enter the IP address of the server that performs the system health monitoring.
+
+ :::image type="content" source="media/how-to-set-up-snmp-mib-monitoring/snmp-allowed-ip-addess.png" alt-text="Enter the IP address for allowed hosts.":::
+
+4. In the **Authentication** section, in the **SNMP v2 Community String** box, enter the string. The SNMP community string can contain up to 32 characters and include any combination of alphanumeric characters (uppercase letters, lowercase letters, and numbers). Spaces are not allowed.
+
+5. Select **Save**.
+
+To define SNMP v3 health monitoring:
+
+1. On the side menu, select **System Settings**.
+
+2. On the **Active Discovery** pane, select **SNMP MIB Monitoring** :::image type="icon" source="media/how-to-set-up-snmp-mib-monitoring/snmp-icon.png" border="false":::.
+
+ :::image type="content" source="media/how-to-set-up-snmp-mib-monitoring/edit-snmp.png" alt-text="Edit your SNMP window.":::
+
+3. In the **Allowed Hosts** section, select **Add host** and enter the IP address of the server that performs the system health monitoring.
+
+ :::image type="content" source="media/how-to-set-up-snmp-mib-monitoring/snmp-allowed-ip-addess.png" alt-text="Enter the IP address for the allowed hosts.":::
+
+4. In the **Authentication** section, set the following parameters:
+
+ | Parameter | Description |
+ |--|--|
+ | **Username** | The SNMP username can contain up to 32 characters and include any combination of alphanumeric characters (uppercase letters, lowercase letters, and numbers). Spaces are not allowed. <br /> <br />The username for the SNMP v3 authentication must be configured on the system and on the SNMP server. |
+ | **Password** | Enter a case-sensitive authentication password. The authentication password can contain 8 to 12 characters and include any combination of alphanumeric characters (uppercase letters, lowercase letters, and numbers). <br /> <br/>The username for the SNMP v3 authentication must be configured on the system and on the SNMP server. |
+ | **Auth Type** | Select MD5 or SHA. |
+ | **Encryption** | Select DES or AES. |
+ | **Secret Key** | The key must contain exactly eight characters and include any combination of alphanumeric characters (uppercase letters, lowercase letters, and numbers). |
+
+5. Select **Save**.
+
+## See also
+
+[Export troubleshooting logs](how-to-troubleshoot-the-sensor-and-on-premises-management-console.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-set-up-your-network https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-set-up-your-network.md new file mode 100644
@@ -0,0 +1,694 @@
+---
+title: Set up your network
+description: Learn about solution architecture, network preparation, prerequisites, and other information needed to ensure that you successfully set up your network to work with Azure Defender for IoT appliances.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/06/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# About Azure Defender for IoT network setup
+
+Azure Defender for IoT delivers continuous ICS threat monitoring and device discovery. The platform includes the following components:
+
+**Defender for IoT sensors:** Sensors collect ICS network traffic by using passive (agentless) monitoring. Passive and nonintrusive, the sensors have zero performance impact on OT and IoT networks and devices. The sensor connects to a SPAN port or network TAP and immediately begins monitoring your network. Detections are displayed in the sensor console. There, you can view, investigate, and analyze them in a network map, a device inventory, and an extensive range of reports. Examples include risk assessment reports, data mining queries, and attack vectors.
+
+**Defender for IoT on-premises management console**: The on-premises management console provides a consolidated view of all network devices. It delivers a real-time view of key OT and IoT risk indicators and alerts across all your facilities. Tightly integrated with your SOC workflows and playbooks, it enables easy prioritization of mitigation activities and cross-site correlation of threats.
+
+**Defender for IoT for IoT portal:** The Defender for IoT application can help you purchase solution appliances, install and update software, and update TI packages.
+
+This article provides information about solution architecture, network preparation, prerequisites, and more to help you successfully set up your network to work with Defender for IoT appliances. Readers working with the information in this article should be experienced in operating and managing OT and IoT networks. Examples include automation engineers, plant managers, OT network infrastructure service providers, cybersecurity teams, CISOs, or CIOs.
+
+For assistance or support, contact [Microsoft Support](https://support.microsoft.com/en-us/supportforbusiness/productselection?sapId=82c88f35-1b8e-f274-ec11-c6efdd6dd099).
+
+## On-site deployment tasks
+
+Site deployment tasks include:
+
+- [Collect site information](#collect-site-information)
+
+- [Prepare a configuration workstation](#prepare-a-configuration-workstation)
+
+- [Planning rack installation](#planning-rack-installation)
+
+### Collect site information
+
+Record site information such as:
+
+- Sensor management network information.
+
+- Site network architecture.
+
+- Physical environment.
+
+- System integrations.
+
+- Planned user credentials.
+
+- Configuration workstation.
+
+- SSL certificates (optional).
+
+- SMTP authentication (optional). To use the SMTP server with authentication, prepare the credentials required for your server.
+
+- DNS servers (optional). Prepare your DNS server's IP and host name.
+
+For a detailed list and description of important site information, see [Example site book](#example-site-book).
+
+#### Successful monitoring guidelines
+
+To find the optimal place to connect the appliance in each of your production networks, we recommend that you follow this procedure:
+
+:::image type="content" source="media/how-to-set-up-your-network/production-network-diagram.png" alt-text="Diagram example of production network setup.":::
+
+### Prepare a configuration workstation
+
+Prepare a Windows workstation, including the following:
+
+- Connectivity to the sensor management interface.
+
+- A supported browser
+
+- Terminal software, such as PuTTY.
+
+Make sure the required firewall rules are open on the workstation. See [Network access requirements](#network-access-requirements) for details.
+
+#### Supported browsers
+
+The following browsers are supported for the sensors and on-premises management console web applications:
+
+- Chrome 32+
+
+- Microsoft Edge 86+
+
+- Internet Explorer 10+
+
+### Network access requirements
+
+Verify that your organizational security policy allows access to the following:
+
+| **Purpose** | **Protocol** | **Transport** | **In or out** | **Port** | **Category** |
+| ----------- | ----------- | ------------ | ---------- | -------- | ------------ |
+| **Access to the web console** | HTTPS | TCP | In or out | 443 | On-premises management console for the Defender for IoT platform |
+| **Access to the CLI** | SSH | TCP | In or out | 22 | CLI |
+| **Connection between the Defender for IoT platform and the on-premises management console** | SSL | TCP | In or out | 443 | Sensor and on-premises management console|
+| **On-premises management console used as NTP to the sensor** | NTP | UDP| In to CM | 123 | Time sync |
+| **Sensor connected to external NTP server (if relevant)** | NTP | UDP | In or out| 123 | Time sync |
+| **Connection between the Defender for IoT platform and management platform and the mail server (if relevant)** | SMTP | TCP | Out of sensor management | 25 | Email |
+| **Logs that send from the on-premises management console to the Syslog server (if relevant)** | Syslog | UDP | Out of sensor management| 514 | LEEF |
+| **DNS server port (if relevant)** | DNS | N/A | In or out| 53 | DNS |
+| **Connection between the Defender for IoT platform and on-premises management console to Active Directory (if relevant)** | LDAPS | TCP | In or out | 636 <br />389 | Active Directory |
+| **Remote SNMP collectors (if relevant)** | SNMP | UDP | Out of sensor management| 161 | Monitoring |
+| **Windows endpoint monitoring (if relevant)** | WMI | UDP | Out of sensor management| 135 | Monitoring |
+| **Windows endpoint monitoring (if relevant)** | WMI | TCP | Out of sensor management| 1024 and above | Monitoring |
+| **Tunneling (if relevant)** | Tunneling | TCP | IN to CM | 9000<br />in addition to port 443<br />From the end user to the on-premises management console <br />Port 22 from the sensor to the on-premises management console | Monitoring |
+| **Outbound to the Defender for IoT hub** | HTTPS | TCP | Out of sensor management| **URL**<br />*.azure-devices.net:443<br />or if wildcards are not supported<br />{your IoT hub name}.azure-devices.net:443 |
+
+### Planning rack installation
+
+To plan your rack installation:
+
+1. Prepare a monitor and a keyboard for your appliance network settings.
+2. Allocate the rack space for the appliance.
+3. Have AC power available for the appliance.
+4. Prepare the LAN cable for connecting the management to the network switch.
+5. Prepare the LAN cables for connecting switch SPAN (mirror) ports and or network taps to the Defender for IoT appliance.
+6. Configure, connect, and validate SPAN ports in the mirrored switches as described in the architecture review session.
+7. Connect the configured SPAN port to a computer running Wireshark and verify that the port is configured correctly.
+8. Open all the relevant firewall ports.
+
+## About passive network monitoring
+
+The appliance receives traffic from multiple sources, either by switch mirror ports (SPAN ports) or by network TAPs. The management port is connected to the business, corporate, or sensor management network with connectivity to an on-premises management console or the Defender for IoT portal.
+
+:::image type="content" source="media/how-to-set-up-your-network/switch-with-port-mirroring.png" alt-text="Diagram of a managed switch with port mirroring.":::
+
+### Purdue model
+
+The following sections describe Purdue levels.
+
+:::image type="content" source="media/how-to-set-up-your-network/purdue-model.png" alt-text="Diagram of the Purdue model.":::
+
+#### Level 0: Cell and area
+
+Level 0 consists of a wide variety of sensors, actuators, and devices involved in the basic manufacturing process. These devices perform the basic functions of the industrial automation and control system, such as:
+
+- Driving a motor.
+- Measuring variables.
+- Setting an output.
+- Performing key functions, such as painting, welding, and bending.
+
+#### Level 1: Process control
+
+Level 1 consists of embedded controllers that control and manipulate the manufacturing process whose key function is to communicate with the Level 0 devices. In discrete manufacturing, those devices are programmable logic controllers (PLCs) or remote telemetry units (RTUs). In process manufacturing, the basic controller is called a distributed control system (DCS).
+
+#### Level 2: Supervisory
+
+Level 2 represents the systems and functions associated with the runtime supervision and operation of an area of a production facility. These usually include the following:
+
+- Operator interfaces or HMIs
+
+- Alarms or alerting systems
+
+- Process historian and batch management systems
+
+- Control room workstations
+
+These systems communicate with the PLCs and RTUs in Level 1. In some cases, they communicate or share data with the site or enterprise (Level 4 and Level 5) systems and applications. These systems are primarily based on standard computing equipment and operating systems (Unix or Microsoft Windows).
+
+#### Levels 3 and 3.5: Site-level and industrial perimeter network
+
+The site level represents the highest level of industrial automation and control systems. The systems and applications that exist at this level manage site-wide industrial automation and control functions. Levels 0 through 3 are considered critical to site operations. The systems and functions that exist at this level might include the following:
+
+- Production reporting (for example, cycle times, quality index, predictive maintenance)
+
+- Plant historian
+
+- Detailed production scheduling
+
+- Site-level operations management
+
+- Device and material management
+
+- Patch launch server
+
+- File server
+
+- Industrial domain, Active Directory, terminal server
+
+These systems communicate with the production zone and share data with the enterprise (Level 4 and Level 5) systems and applications.
+
+#### Levels 4 and 5: Business and enterprise networks
+
+Level 4 and Level 5 represent the site or enterprise network where the centralized IT systems and functions exist. The IT organization directly manages the services, systems, and applications at these levels.
+
+## Planning for network monitoring
+
+The following examples present different types of topologies for industrial control networks, along with considerations for optimal monitoring and placement of sensors.
+
+### What should be monitored?
+
+Traffic that goes through layers 1 and 2 should be monitored.
+
+### What should the Defender for IoT appliance connect to?
+
+The Defender for IoT appliance should connect to the managed switches that see the industrial communications between layers 1 and 2 (in some cases also layer 3).
+
+The following diagram is a general abstraction of a multilayer, multitenant network, with an expansive cybersecurity ecosystem typically operated by an SOC and MSSP.
+
+Typically, NTA sensors are deployed in layers 0 to 3 of the OSI model.
+
+:::image type="content" source="media/how-to-set-up-your-network/osi-model.png" alt-text="Diagram of the OSI model.":::
+
+#### Example: Ring topology
+
+The ring network is a topology in which each switch or node connects to exactly two other switches, forming a single continuous pathway for the traffic.
+
+:::image type="content" source="media/how-to-set-up-your-network/ring-topology.PNG" alt-text="Diagram of the ring topology.":::
+
+#### Example: Linear bus and star topology
+
+In a star network, every host is connected to a central hub. In its simplest form, one central hub acts as a conduit to transmit messages. In the following example, lower switches are not monitored, and traffic that remains local to these switches will not be seen. Devices might be identified based on ARP messages, but connection information will be missing.
+
+:::image type="content" source="media/how-to-set-up-your-network/linear-bus-star-topology.PNG" alt-text="Diagram of the linear bus and star topology.":::
+
+#### Multisensor deployment
+
+Here are some recommendations for deploying multiple sensors:
+
+| **Number **| **Meters** | **Dependency** | **Number of sensors** |
+|--|--|--|--|
+| The maximum distance between switches | 80 meters | Prepared Ethernet cable | More than 1 |
+| Number of OT networks | More than 1 | No physical connectivity | More than 1 |
+| Number of switches | Can use RSPAN configuration | Up to 8 switches with local span close to the sensor by cabling distance | More than 1 |
+
+#### Traffic mirroring
+
+To see only relevant information for traffic analysis, you need to connect the Defender for IoT platform to a mirroring port on a switch or a TAP that includes only industrial ICS and SCADA traffic.
+
+:::image type="content" source="media/how-to-set-up-your-network/switch.jpg" alt-text="Use this switch for your setup.":::
+
+You can monitor switch traffic by using the following methods:
+
+- [Switch SPAN port](#switch-span-port)
+
+- [Remote SPAN (RSPAN)](#remote-span-rspan)
+
+- [Active and passive aggregation TAP](#active-and-passive-aggregation-tap)
+
+SPAN and RSPAN are Cisco terminology. Other brands of switches have similar functionality but might use different terminology.
+
+#### Switch SPAN port
+
+A switch port analyzer mirrors local traffic from interfaces on the switch to interface on the same switch. Here are some considerations:
+
+- Verify that the relevant switch supports the port mirroring function.
+
+- The mirroring option is disabled by default.
+
+- We recommend that you configure all of the switch's ports, even if no data is connected to them. Otherwise, a rogue device might be connected to an unmonitored port, and it would not be alerted on the sensor.
+
+- On OT networks that utilize broadcast or multicast messaging, configure the switch to mirror only RX (Receive) transmissions. Otherwise, multicast messages will be repeated for as many active ports, and the bandwidth is multiplied.
+
+The following configuration examples are for reference only and are based on a Cisco 2960 switch (24 ports) running IOS. They are typical examples only, so don't use them as instructions. Mirror ports on other Cisco operating systems and other brands of switches are configured differently.
+
+:::image type="content" source="media/how-to-set-up-your-network/span-port-configuration-terminal-v2.png" alt-text="Diagram of a SPAN port configuration terminal.":::
+:::image type="content" source="media/how-to-set-up-your-network/span-port-configuration-mode-v2.png" alt-text="Diagram of SPAN port configuration mode.":::
+
+##### Monitoring multiple VLANs
+
+Defender for IoT allows monitoring VLANs configured in your network. No configuration of the Defender for IoT system is required. The user needs to ensure that the switch in your network is configured to send VLAN tags to Defender for IoT.
+
+The following example shows the required commands that must be configured on the Cisco switch to enable monitoring VLANs in Defender for IoT:
+
+**Monitor session**: This command is responsible for the process of sending VLANs to the SPAN port.
+
+- monitor session 1 source interface Gi1/2
+
+- monitor session 1 filter packet type good Rx
+
+- monitor session 1 destination interface fastEthernet1/1 encapsulation dot1q
+
+**Monitor Trunk Port F.E. Gi1/1**: VLANs are configured on the trunk port.
+
+- interface GigabitEthernet1/1
+
+- switchport trunk encapsulation dot1q
+
+- switchport mode trunk
+
+#### Remote SPAN (RSPAN)
+
+The remote SPAN session mirrors traffic from multiple distributed source ports into a dedicated remote VLAN.
+
+:::image type="content" source="media/how-to-set-up-your-network/remote-span.png" alt-text="Diagram of remote SPAN.":::
+
+The data in the VLAN is then delivered through trunked ports across multiple switches to a specific switch that contains the physical destination port. This port connects to the Defender for IoT platform.
+
+##### More about RSPAN
+
+- RSPAN is an advanced feature that requires a special VLAN to carry the traffic that SPAN monitors between switches. RSPAN is not supported on all switches. Verify that the switch supports the RSPAN function.
+
+- The mirroring option is disabled by default.
+
+- The remote VLAN must be allowed on the trunked port between the source and destination switches.
+
+- All switches that connect the same RSPAN session must be from the same vendor.
+
+> [!NOTE]
+> Make sure that the trunk port that's sharing the remote VLAN between the switches is not defined as a mirror session source port.
+>
+> The remote VLAN increases the bandwidth on the trunked port by the size of the mirrored session's bandwidth. Verify that the switch's trunk port supports that.
+
+:::image type="content" source="media/how-to-set-up-your-network/remote-vlan.jpg" alt-text="Diagram of remote VLAN.":::
+
+#### RSPAN configuration examples
+
+RSPAN: Based on Cisco catalyst 2960 (24 ports).
+
+**Source switch configuration example:**
+
+:::image type="content" source="media/how-to-set-up-your-network/rspan-configuration.png" alt-text="Screenshot of RSPAN configuration.":::
+
+1. Enter global configuration mode.
+
+1. Create a dedicated VLAN.
+
+1. Identify the VLAN as the RSPAN VLAN.
+
+1. Return to "configure terminal" mode.
+
+1. Configure all 24 ports as session sources.
+
+1. Configure the RSPAN VLAN to be the session destination.
+
+1. Return to privileged EXEC mode.
+
+1. Verify the port mirroring configuration.
+
+**Destination switch configuration example:**
+
+1. Enter global configuration mode.
+
+1. Configure the RSPAN VLAN to be the session source.
+
+1. Configure physical port 24 to be the session destination.
+
+1. Return to privileged EXEC mode.
+
+1. Verify the port mirroring configuration.
+
+1. Save the configuration.
+
+#### Active and passive aggregation TAP
+
+An active or passive aggregation TAP is installed inline to the network cable. It duplicates both RX and TX to the monitoring sensor.
+
+The terminal access point (TAP) is a hardware device that allows network traffic to flow from port A to port B, and from port B to port A, without interruption. It creates an exact copy of both sides of the traffic flow, continuously, without compromising network integrity. Some TAPs aggregate transmit and receive traffic by using switch settings if desired. If aggregation is not supported, each TAP uses two sensor ports to monitor send and receive traffic.
+
+TAPs are advantageous for a variety of reasons. They're hardware-based and can't be compromised. They pass all traffic, even damaged messages, which switches often drop. They're not processor sensitive, so packet timing is exact where switches handle the mirror function as a low-priority task that can affect the timing of the mirrored packets. For forensic purposes, a TAP is the best device.
+
+TAP aggregators can also be used for port monitoring. These devices are processor based and are not as intrinsically secure as hardware TAPs. They might not reflect exact packet timing.
+
+:::image type="content" source="media/how-to-set-up-your-network/active-passive-tap-v2.PNG" alt-text="Diagram of active and passive TAPs.":::
+
+##### Common TAP models
+
+These models have been tested for compatibility. Other vendors and models might also be compatible.
+
+| Image | Model |
+| -- | -- |
+| :::image type="content" source="media/how-to-set-up-your-network/garland-p1gccas-v2.png" alt-text="Screenshot of Garland P1GCCAS."::: | Garland P1GCCAS |
+| :::image type="content" source="media/how-to-set-up-your-network/ixia-tpa2-cu3-v2.png" alt-text="Screenshot of IXIA TPA2-CU3."::: | IXIA TPA2-CU3 |
+| :::image type="content" source="media/how-to-set-up-your-network/us-robotics-usr-4503-v2.png" alt-text="Screenshot of US Robotics USR 4503."::: | US Robotics USR 4503 |
+
+##### Special TAP configuration
+
+| Garland TAP | US Robotics TAP |
+| ----------- | --------------- |
+| Make sure jumpers are set as follows:<br />:::image type="content" source="media/how-to-set-up-your-network/jumper-setup-v2.jpg" alt-text="Screenshot of US Robotics switch.":::| Make sure Aggregation mode is active. |
+
+## Deployment validation
+
+### Engineering self-review
+
+Reviewing your OT and ICS network diagram is the most efficient way to define the best place to connect to, where you can get the most relevant traffic for monitoring.
+
+The site engineers know what their network looks like. Having a review session with the local network and operational teams will usually clarify expectations and define the best place to connect the appliance.
+
+Relevant information:
+
+- List of known devices (spreadsheet)
+
+- Estimated number of devices
+
+- Vendors and industrial protocols
+
+- Model of switches, to verify that port mirroring option is available
+
+- Information about who manages the switches (for example, IT) and whether they're external resources
+
+- List of OT networks at the site
+
+#### Common questions
+
+- What are the overall goals of the implementation? Are a complete inventory and accurate network map important?
+
+- Are there multiple or redundant networks in the ICS? Are all the networks being monitored?
+
+- Are there communications between the ICS and the enterprise (business) network? Are these communications being monitored?
+
+- Are VLANs configured in the network design?
+
+- How is maintenance of the ICS performed, with fixed or transient devices?
+
+- Where are firewalls installed in the monitored networks?
+
+- Is there any routing in the monitored networks?
+
+- What OT protocols are active on the monitored networks?
+
+- If we connect to this switch, will we see communication between the HMI and the PLCs?
+
+- What is the physical distance between the ICS switches and the enterprise firewall?
+
+- Can unmanaged switches be replaced with managed switches, or is the use of network TAPs an option?
+
+- Is there any serial communication in the network? If yes, show it on the network diagram.
+
+- If the Defender for IoT appliance should be connected to that switch, is there physical available rack space in that cabinet?
+
+#### Additional considerations
+
+The purpose of the Defender for IoT appliance is to monitor traffic from layers 1 and 2.
+
+For some architectures, the Defender for IoT appliance will also monitor layer 3, if OT traffic exists on this layer. While you're reviewing the site architecture and deciding whether to monitor a switch, consider the following variables:
+
+- What is the cost/benefit versus the importance of monitoring this switch?
+
+- If a switch is unmanaged, will it be possible to monitor the traffic from a higher-level switch?
+
+ If the ICS architecture is a ring topology, only one switch in this ring needs to be monitored.
+
+- What is the security or operational risk in this network?
+
+- Is it possible to monitor the switch's VLAN? Is that VLAN visible in another switch that we can monitor?
+
+#### Technical validation
+
+Receiving a sample of recorded traffic (PCAP file) from the switch SPAN (or mirror) port can help to:
+
+- Validate if the switch is configured properly.
+
+- Confirm if the traffic that goes through the switch is relevant for monitoring (OT traffic).
+
+- Identify bandwidth and the estimated number of devices in this switch.
+
+You can record a sample PCAP file (a few minutes) by connecting a laptop to an already configured SPAN port through the Wireshark application.
+
+:::image type="content" source="media/how-to-set-up-your-network/laptop-connected-to-span.jpg" alt-text="Screenshot of a laptop connected to a SPAN port.":::
+:::image type="content" source="media/how-to-set-up-your-network/sample-pcap-file.PNG" alt-text="Screenshot of the recording of a sample PCAP file.":::
+
+#### Wireshark validation
+
+- Check that *Unicast packets* are present in the recording traffic. Unicast is from one address to another. If most of the traffic is ARP messages, then the switch setup is incorrect.
+
+- Go to **Statistics** > **Protocol Hierarchy**. Verify that industrial OT protocols are present.
+
+:::image type="content" source="media/how-to-set-up-your-network/wireshark-validation.png" alt-text="Screenshot of Wireshark validation.":::
+
+## Troubleshooting
+
+Use these sections for troubleshooting issues:
+
+- [Can't connect by using a web interface](#cant-connect-by-using-a-web-interface)
+
+- [Appliance is not responding](#appliance-is-not-responding)
+
+### Can't connect by using a web interface
+
+1. Verify that the computer that you're trying to connect is on the same network as the appliance.
+
+2. Verify that the GUI network is connected to the management port on the sensor.
+
+3. Ping the appliance IP address. If there is no response to ping:
+
+ 1. Connect a monitor and a keyboard to the appliance.
+
+ 1. Use the **support** user and password to sign in.
+
+ 1. Use the command **network list** to see the current IP address.
+
+ :::image type="content" source="media/how-to-set-up-your-network/list-of-network-commands.png" alt-text="Screenshot of the network list command.":::
+
+4. If the network parameters are misconfigured, use the following procedure to change it:
+
+ 1. Use the command **network edit-settings**.
+
+ 1. To change the management network IP address, select **Y**.
+
+ 1. To change the subnet mask, select **Y**.
+
+ 1. To change the DNS, select **Y**.
+
+ 1. To change the default gateway IP address, select **Y**.
+
+ 1. For the input interface change (for sensor only), select **Y**.
+
+ 1. For the bridge interface, select **N**.
+
+ 1. To apply the settings, select **Y**.
+
+5. After you restart, connect with user support and use the **network list** command to verify that the parameters were changed.
+
+6. Try to ping and connect from the GUI again.
+
+### Appliance is not responding
+
+1. Connect with a monitor and keyboard to the appliance, or use PuTTY to connect remotely to the CLI.
+
+2. Use the support credentials to sign in.
+
+3. Use the **system sanity** command and check that all processes are running.
+
+ :::image type="content" source="media/how-to-set-up-your-network/system-sanity-command.png" alt-text="Screenshot of the system sanity command.":::
+
+For any other issues, contact [Microsoft Support](https://support.microsoft.com/en-us/supportforbusiness/productselection?sapId=82c88f35-1b8e-f274-ec11-c6efdd6dd099).
+
+## Example site book
+
+Use the example site book to retrieve and review important information that you need for network setup.
+
+### Site checklist
+
+Review this list before site deployment:
+
+| **#** | **Task or activity** | **Status** | **Comments** |
+|--|--|--|--|
+| 1 | Provide global. | ΓÿÉ | |
+| 3 | Order appliances. | ΓÿÉ | |
+| 4 | Prepare a list of subnets in the network. | ΓÿÉ | |
+| 5 | Provide a VLAN list of the production networks. | ΓÿÉ | |
+| 6 | Provide a list of switch models in the network. | ΓÿÉ | |
+| 7 | Provide a list of vendors and protocols of the industrial equipment. | ΓÿÉ | |
+| 8 | Provide network details for sensors (IP address, subnet, D-GW, DNS). | ΓÿÉ | |
+| 9 | Create necessary firewall rules and the access list. | ΓÿÉ | |
+| 10 | Create spanning ports on switches for port monitoring, or configure network taps as desired. | ΓÿÉ | |
+| 11 | Prepare rack space for sensor appliances. | ΓÿÉ | |
+| 12 | Prepare a workstation for personnel. | ΓÿÉ | |
+| 13 | Provide a keyboard, monitor, and mouse for the Defender for IoT rack devices. | ΓÿÉ | |
+| 14 | Rack and cable the appliances. | ΓÿÉ | |
+| 15 | Allocate site resources to support deployment. | ΓÿÉ | |
+| 16 | Create Active Directory groups or local users. | ΓÿÉ | |
+| 17 | Set up training (self-learning). | ΓÿÉ | |
+| 18 | Go or no-go. | ΓÿÉ | |
+| 19 | Schedule the deployment date. | ΓÿÉ | |
++
+| **Date** | **Note** | **Deployment date** | **Note** |
+|--|--|--|--|
+| Defender for IoT | | Site name* | |
+| Name | | Name | |
+| Position | | Position | |
+
+#### Architecture review
+
+An overview of the industrial network diagram will allow you to define the proper location for the Defender for IoT equipment.
+
+1. View a global network diagram of the industrial OT environment. For example:
+
+ :::image type="content" source="media/how-to-set-up-your-network/ot-global-network-diagram.png" alt-text="Diagram of the industrial OT environment for the global network.":::
+
+ > [!NOTE]
+ > The Defender for IoT appliance should be connected to a lower-level switch that sees the traffic between the ports on the switch.
+
+2. Provide the approximate number of devices in the networks (optional).
+
+3. Provide a subnet list for the production networks and a description (optional).
+
+ | **#** | **Subnet name** | **Description** |
+ |--| --------------- | --------------- |
+ | 1 | |
+ | 2 | |
+ | 3 | |
+ | 4 | |
+
+4. Provide a VLAN list of the production networks.
+
+ | **#** | **VLAN Name** | **Description** |
+ |--|--|--|
+ | 1 | | |
+ | 2 | | |
+ | 3 | | |
+ | 4 | | |
+
+5. To verify that the switches have port mirroring capability, provide the switch model numbers that the Defender for IoT platform should connect to:
+
+ | **#** | **Switch** | **Model** | **Traffic mirroring support (SPAN, RSPAN, or none)** |
+ |--|--|--|--|
+ | 1 | | |
+ | 2 | | |
+ | 3 | | |
+ | 4 | | |
+
+ Does a third party manage the switches? Y or N
+
+ If yes, who? __________________________________
+
+ What is their policy? __________________________________
+
+ For example:
+
+ - Siemens
+
+ - Rockwell automation ΓÇô Ethernet or IP
+
+ - Emerson ΓÇô DeltaV, Ovation
+
+6. Are there devices that communicate via a serial connection in the network? Yes or No
+
+ If yes, specify which serial communication protocol: ________________
+
+ If yes, mark on the network diagram what devices communicate with serial protocols, and where they are:
+
+ <Add your network diagram with marked serial connection>
+
+7. For QoS, the default setting of the sensor is 1.5 Mbps. Specify if you want to change it: ________________
+
+ Business unit (BU): ________________
+
+### Specifications for site equipment
+
+#### Network
+
+The sensor appliance is connected to switch SPAN port through a network adapter. It's connected to the customer's corporate network for management through another dedicated network adapter.
+
+Provide address details for the sensor NIC that will be connected in the corporate network:
+
+| Item | Appliance 1 | Appliance 2 | Appliance 3 |
+| --------------- | ------------- | ------------- | ------------- |
+| Appliance IP address | | | |
+| Subnet | | | |
+| Default gateway | | | |
+| DNS | | | |
+| Host name | | | |
+
+#### iDRAC/iLO/Server management
+
+| Item | Appliance 1 | Appliance 2 | Appliance 3 |
+| --------------- | ------------- | ------------- | ------------- |
+| Appliance IP address | | | |
+| Subnet | | | |
+| Default gateway | | | |
+| DNS | | | |
+
+#### On-premises management console
+
+| Item | Active | Passive (when using HA) |
+| --------------- | ------ | ----------------------- |
+| IP address | | |
+| Subnet | | |
+| Default gateway | | |
+| DNS | | |
+
+#### SNMP
+
+| Item | Details |
+| --------------- | ------ |
+| IP | |
+| IP address | |
+| Username | |
+| Password | |
+| Authentication type | MD5 or SHA |
+| Encryption | DES or AES |
+| Secret key | |
+| SNMP v2 community string |
+
+### CM SSL certificate
+
+Are you planning to use an SSL certificate? Yes or No
+
+If yes, what service will you use to generate it? What attributes will you include in the certificate (for example, domain or IP address)?
+
+### SMTP authentication
+
+Are you planning to use SMTP to forward alerts to an email server? Yes or No
+
+If yes, what authentication method you will use?
+
+### Active Directory or local users
+
+Contact an Active Directory administrator to create an Active Directory site user group or create local users. Be sure to have your users ready for the deployment day.
+
+### IoT device types in the network
+
+| Device type | Number of devices in the network | Average bandwidth |
+| --------------- | ------ | ----------------------- |
+| Camera | |
+| X-ray machine | |
+
+## See also
+
+[About the Defender for IoT installation](how-to-install-software.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-track-sensor-activity https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-track-sensor-activity.md new file mode 100644
@@ -0,0 +1,89 @@
+---
+title: Track sensor activity
+description: The event timeline presents a timeline of activity detected on your network, including alerts and alert management actions, network events, and user operations such as user sign-in and user deletion.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/10/2020
+ms.topic: article
+ms.service: azure
+---
+
+# Track sensor activity
+
+## Event timeline
+
+The event timeline presents a timeline of activity that your sensor has detected. For example:
+
+ - Alerts and alert management actions
+
+ - Network events
+
+ - User operations such as user sign-in and user deletion
+
+The event timeline provides a chronological view of events that happened in the network. The event timeline allows understanding and analyses of the chain of events that preceded and followed an attack or incident, which assists in the investigation and forensics.
+
+> [!NOTE]
+> *Administrators* and *security analysts* can perform the procedures described in this section.
+
+To view the event logs:
+
+- From the side menu, select **Event Timeline**.
+
+ :::image type="content" source="media/how-to-track-sensor-activity/event-timeline.png" alt-text="View your events on the event timeline.":::
+
+In addition to viewing the events that the sensor has detected, you can manually add events to the timeline. This process is useful if the event happened in an external system but has an impact on your network, and it's important to record the event and present it as a part of the timeline.
+
+To add events manually:
+
+- Select **Create Event**.
+
+To export event log information into a CSV file:
+
+- Select **Export**.
+
+## Filter the event timeline
+
+Filter the timeline to display devices and events of interest to you.
+
+To filter the timeline:
+
+1. Select **Advanced Filters**.
+
+ :::image type="content" source="media/how-to-track-sensor-activity/advance-filters.png" alt-text="Use the Events Advanced Filters window to filter your events.":::
+
+2. Set event filters, as follows:
+
+ - **Include Address**: Display specific event devices.
+
+ - **Exclude Address**: Hide specific event devices.
+
+ - **Include Event Types**: Display specific event types.
+
+ - **Exclude Event Types**: Hide specific event types.
+
+ - **Device Group**: Select a device group, as it was defined in the device map. Only the events of this group are presented.
+
+3. Select **Clear All** to clear all the selected filters.
+
+4. Search by **Alerts Only**, **Alerts and Notices**, or **All Events**.
+
+5. Select **Select Date** to choose a specific date. Choose a day, hour, and minute. Events from the selected time frame are shown.
+
+6. Select **User Operations** to include or exclude user operation events.
+
+7. Select the arrow (**V**) to view more information about the event:
+
+ - Select the related alerts (if any) to display a detailed description of the alert.
+
+ - Select the device to display the device on the map.
+
+ - Select **Filter events by related devices** if you want to filter by related devices.
+
+ - Select **PCAP File** to download the PCAP file (if it exists) that contains a packet capture of the whole network at a specific time.
+
+ The PCAP file contains technical information that can help engineers determine exactly where the event was and what's happening there. You can analyze the PCAP file with a network protocol analyzer such as Wireshark, a free application.
+
+## See also
+
+[View alerts](how-to-view-alerts.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-troubleshoot-the-sensor-and-on-premises-management-console https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-troubleshoot-the-sensor-and-on-premises-management-console.md new file mode 100644
@@ -0,0 +1,276 @@
+---
+title: Troubleshoot the sensor and on-premises management console
+description: Troubleshoot your sensor and on-premises management console to eliminate any problems you might be having.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/12/2020
+ms.topic: article
+ms.service: azure
+---
+# Troubleshoot the sensor and on-premises management console
+
+This article describes basic troubleshooting tools for the sensor and the on-premises management console. In addition to the items described here, you can check the health of your system in the following ways:
+
+**Alerts**: An alert is created when the sensor interface that monitors the traffic is down.
+
+**SNMP**: Sensor health is monitored through SNMP. Azure Defender for IoT responds to SNMP queries sent from an authorized monitoring server.
+
+**System notifications**: When a management console controls the sensor, you can forward alerts about failed sensor backups and disconnected sensors.
+
+## Sensor troubleshooting tools
+
+### Investigate password failure at initial sign-in
+
+When you're signing in to a preconfigured Arrow sensor for the first time, you'll need to perform the following password recovery:
+
+1. On the Defender for IoT sign-in screen, select the **Password Recovery** option.
+
+ The **Password Recovery** screen opens. There, you're prompted to select the user and subscription, and you're given a unique identifier.
+
+1. Go to the Defender for IoT **Sites and sensors** page and select the **Recover my password** tab.
+
+1. Enter the unique identifier that you received on the **Password Recovery** screen and select **Recover**. The `password_recovery.zip` file
+ is downloaded.
+
+ > [!NOTE]
+ > Don't alter the activation file. It's a signed file and won't work if you tamper with it.
+
+1. On the **Password Recovery** screen, upload the `password_recovery.zip` file and select **Next**.
+
+You then receive your system-generated password for your management console.
+
+### Investigate a lack of traffic
+
+An indicator appears at the top of the console when the sensor recognizes that there's no traffic on one of the configured ports. This indicator is visible to all users.
+
+:::image type="content" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/no-traffic-detected.png" alt-text="Screenshot of the alert that no traffic was detected.":::
+
+When this message appears, you can investigate where there's no traffic. Make sure the span cable is connected and there was no change in the span architecture.
+
+For support and troubleshooting information, contact [Microsoft Support](https://support.serviceshub.microsoft.com/supportforbusiness/create?sapId=82c88f35-1b8e-f274-ec11-c6efdd6dd099).
+
+### Check system performance
+
+When a new sensor is deployed or, for example, the sensor is working slowly or not showing any alerts, you can check system performance.
+
+To check system performance:
+
+1. In the dashboard, make sure that `PPS > 0`.
+
+ :::image type="content" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/dashboard-view-v2.png" alt-text="Screenshot of a sample dashboard.":::
+
+2. From the side menu, select **Devices**.
+
+3. In the **Devices** window, make sure devices are being discovered.
+
+ :::image type="content" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/discovered-devices.png" alt-text="Ensure that devices are discovered.":::
+
+4. From the side menu, select **Data Mining**.
+
+5. In the **Data Mining** window, select **ALL** and generate a report.
+
+ :::image type="content" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/new-report-generated.png" alt-text="Generate a new report by using data mining.":::
+
+6. Make sure the report contains data.
+
+ :::image type="content" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/new-report-generated.png" alt-text="Ensure that the report contains data.":::
+
+7. From the side menu, select **Trends & Statistics**.
+
+8. In the **Trends & Statistics** window, select **Add Widget**.
+
+ :::image type="content" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/add-widget.png" alt-text="Add a widget by selecting it.":::
+
+9. Add a widget and make sure it shows data.
+
+ :::image type="content" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/widget-data.png" alt-text="Ensure that the widget is showing data.":::
+
+10. From the side menu, select **Alerts**. The **Alerts** window appears.
+
+11. Make sure the alerts were created.
+
+ :::image type="content" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/alerts-created.png" alt-text="Ensure that alerts were created.":::
++
+### Investigate a lack of expected alerts
+
+If the **Alerts** window doesn't show an alert that you expected, verify the following:
+
+- Check if the same alert already appears in the **Alerts** window as a reaction to a different security instance. If yes, and this alert has not been handled yet, the sensor console does not show a new alert.
+
+- Make sure you did not exclude this alert by using the **Alert Exclusion** rules in the management console.
+
+### Investigate widgets that show no data
+
+When the widgets in the **Trends & Statistics** window show no data, do the following:
+
+- [Check system performance](#check-system-performance).
+
+- Make sure the time and region settings are properly configured and not set to a future time.
+
+### Investigate a device map that shows only broadcasting devices
+
+When devices shown on the map appear not connected to each other, something might be wrong with the SPAN port configuration. That is, you might be seeing only broadcasting devices and no unicast traffic.
+
+:::image type="content" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/broadcasting-devices.png" alt-text="View your broadcasting devices.":::
+
+In such a case, you need to validate that you can see only the broadcast traffic. Then ask the network engineer to fix the SPAN port configuration so that you can see the unicast traffic.
+
+To validate that you're seeing only the broadcast traffic:
+
+- On the **Data Mining** screen, create a report by using the **All** option. Then see if only broadcast and multicast traffic (and no unicast traffic) appears in the report.
+
+Or:
+
+- Record a PCAP directly from the switch, or connect a laptop by using Wireshark.
+
+### Connect the sensor to NTP
+
+You can configure a standalone sensor and a management console, with the sensors related to it, to connect to NTP.
+
+To connect a standalone sensor to NTP:
+
+- [Contact the Support team for assistance](https://support.microsoft.com/en-us/supportforbusiness/productselection?sapId=82c88f35-1b8e-f274-ec11-c6efdd6dd099).
+
+To connect a sensor controlled by the management console to NTP:
+
+- The connection to NTP is configured on the management console. All the sensors that the management console controls get the NTP connection automatically.
+
+### Investigate when devices aren't shown on the map, or you have multiple internet-related alerts
+
+Sometimes ICS devices are configured with external IP addresses. These ICS devices are not shown on the map. Instead of the devices, an internet cloud appears on the map. The IP addresses of these devices are included in the cloud image.
+
+Another indication of the same problem is when multiple internet-related alerts appear.
+
+:::image type="content" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/alert-problems.png" alt-text="Multiple internet-related alerts.":::
+
+To fix the configuration:
+
+1. Right-click the cloud icon on the device map and select **Export IP Addresses**. Copy the public ranges that are private, and add them to the subnet list. For more information, see [Configure subnets](how-to-control-what-traffic-is-monitored.md#configure-subnets).
+
+2. Generate a new data-mining report for internet connections.
+
+3. In the data-mining report, select :::image type="icon" source="media/how-to-troubleshoot-the-sensor-and-on-premises-management-console/administrator-mode.png" border="false"::: to enter the administrator mode and delete the IP addresses of your ICS devices.
+
+### Tweak the sensor's quality of service
+
+To save your network resources, you can limit the interface bandwidth that the sensor uses for day-to-day procedures.
+
+To limit the interface bandwidth, use the `cyberx-xsense-limit-interface` CLI tool that needs to be run with sudo permissions. The tool gets the following arguments:
+
+ - `* -i`: interfaces (example: eth0).
+
+ - `* -l`: limit (example: 30 kbit / 1 mbit). You can use the following bandwidth units: kbps, mbps, kbit, mbit, or bps.
+
+ - `* -c`: clear (to clear the interface bandwidth limitation).
+
+To tweak the quality of service:
+
+1. Sign in to the sensor CLI as a Defender for IoT user, and enter `sudo cyberx-xsense-limit-interface-I eth0 -l value`.
+
+ For example: `sudo cyberx-xsense-limit-interface -i eth0 -l 30kbit`
+
+ > [!NOTE]
+ > For a physical appliance, use the em1 interface.
+
+2. To clear interface limitation, enter `sudo cyberx-xsense-limit-interface -i eth0 -l 1mbps -c`.
+
+## On-premises management console troubleshooting tools
+
+### Investigate a lack of expected alerts
+
+If an expected alert is not shown in the **Alerts** window, verify the following:
+
+- Check if the same alert already appears in the **Alerts** window as a reaction to a different security instance. If yes, and this alert has not been handled yet, a new alert is not shown.
+
+- Verify that you did not exclude this alert by using the **Alert Exclusion** rules in the on-premises management console.
+
+### Tweak the quality of service
+
+To save your network resources, you can limit the number of alerts sent to external systems (such as emails or SIEM) in one sync operation between an appliance and the on-premises management console.
+
+The default is 50. This means that in one communication session between an appliance and the on-premises management console, there will be no more than 50 alerts to external systems.
+
+To limit the number of alerts, use the `notifications.max_number_to_report` property available in `/var/cyberx/properties/management.properties`. No restart is needed after you change this property.
+
+To tweak the quality of service:
+
+1. Sign in as a Defender for IoT user.
+
+2. Verify the default values:
+
+ ```bash
+ grep \"notifications\" /var/cyberx/properties/management.properties
+ ```
+
+ The following default values appear:
+
+ ```bash
+ notifications.max_number_to_report=50
+ notifications.max_time_to_report=10 (seconds)
+ ```
+
+3. Edit the default settings:
+
+ ```bash
+ sudo nano /var/cyberx/properties/management.properties
+ ```
+
+4. Edit the settings of the following lines:
+
+ ```bash
+ notifications.max_number_to_report=50
+ notifications.max_time_to_report=10 (seconds)
+ ```
+
+5. Save the changes. No restart is required.
+
+## Export information for troubleshooting
+
+In addition to tools for monitoring and analyzing your network, you can send information to the support team for further investigation. When you export logs, the sensor will automatically generate a one-time password (OTP), unique for the exported logs, in a separate text file.
+
+To export logs:
+
+1. On the left pane, select **System Settings**.
+
+2. Select **Export Logs**.
+
+ :::image type="content" source="media/how-to-export-information-for-troubleshooting/export-a-log.png" alt-text="Export a log to system support.":::
+
+3. In the **File Name** box, enter the file name that you want to use for the log export. The default is the current date.
+
+4. To define what data you want to export, select the data categories:
+
+ | Export category | Description |
+ |--|--|
+ | **Operating System Logs** | Select this option to get information about the operating system state. |
+ | **Installation/Upgrade logs** | Select this option for investigation of the installation and upgrade configuration parameters. |
+ | **System Sanity Output** | Select this option to check system performance. |
+ | **Dissection Logs** | Select this option to allow advanced inspection of protocol dissection. |
+ | **OS Kernel Dumps** | Select this option to export your kernel memory dump. A kernel memory dump contains all the memory that the kernel is using at the time of the problem that occurred in this kernel. The size of the dump file is smaller than the complete memory dump. Typically, the dump file is around one-third the size of the physical memory on the system. |
+ | **Forwarding logs** | Select this option for investigation of the forwarding rules. |
+ | **SNMP Logs** | Select this option to receive SNMP health check information. |
+ | **Core Application Logs** | Select this option to export data about the core application configuration and operation. |
+ | **Communication with CM logs** | Select this option if there are continuous problems or interruptions of connection with the management console. |
+ | **Web Application Logs** | Select this option to get information about all the requests sent from the application's web interface. |
+ | **System Backup** | Select this option to export a backup of all the system data for investigating the exact state of the system. |
+ | **Dissection Statistics** | Select this option to allow advanced inspection of protocol statistics. |
+ | **Database Logs** | Select this option to export logs from the system database. Investigating system logs assists in identifying system problems. |
+ | **Configuration** | Select this option to export information about all the configurable parameters to make sure everything was configured correctly. |
+
+5. To select all the options, select **Select All** next to **Choose Categories**.
+
+6. Select **Export Logs**.
+
+The exported logs are added to the **Archived Logs** list. Send the OTP to the support team in a separate message and medium from the exported logs. The support team will be able to extract exported logs only by using the unique OTP that's used to encrypt the logs.
+
+The list of archived logs can contain up to five items. If the number of items in the list goes beyond that number, the earliest item is deleted.
+
+## See also
+
+- [View alerts](how-to-view-alerts.md)
+
+- [Set up SNMP MIB monitoring](how-to-set-up-snmp-mib-monitoring.md)
+
+- [Understand sensor disconnection events](how-to-manage-sensors-from-the-on-premises-management-console.md#understand-sensor-disconnection-events)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-view-alerts https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-view-alerts.md new file mode 100644
@@ -0,0 +1,120 @@
+---
+title: View alerts
+description: View alerts according to various categories, and use search features to help you find alerts of interest.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/02/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# View alerts
+
+This article describes how to view alerts triggered by your sensor and manage them with alert tools.
+
+You can view alerts based on various categories, such as alerts that have been archived or pinned. Or you can search for alerts of interest, such as alerts based on an IP or MAC address.
+
+You can also view alerts from the sensor dashboard.
+
+To view alerts:
+
+- Select **Alerts** from the side menu. The **Alerts** window displays the alerts that your sensor has detected.
+
+ :::image type="content" source="media/how-to-work-with-alerts-sensor/alerts-screen.png" alt-text="View of the Alerts screen.":::
+
+## View alerts by category
+
+You can view alerts according to various categories from the **Alerts** main view. Select an alert to review details and manage the event.
+
+| Sort by type | Description |
+|--|--|
+| **Important Alerts** | Alerts sorted by importance. |
+| **Pinned Alerts** | Alerts that the user pinned for further investigation. Pinned alerts are not archived and are stored for 14 days in the pinned folder. |
+| **Recent Alerts** | Alerts sorted by time. |
+| **Acknowledged Alerts** | Alerts that were acknowledged and unhandled, or that were muted and unmuted. |
+| **Archived Alerts** | Alerts that the system archived automatically. Only the administrator user can access them. |
+
+## Search for alerts of interest
+
+The **Alerts** main view provides various search features to help you find alerts of interest.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/main-alerts-view.png" alt-text="Alerts learning screenshot.":::
+
+### Text search
+
+Use the **Free Search** option to search for alerts by text, numbers, or characters.
+
+To search:
+
+- Type the required text in the **Free Search** field and press Enter on your keyboard.
+
+To clear the search:
+
+- Delete the text in the **Free Search** field and press Enter on your keyboard.
+
+### Device group or device IP address search
+
+Search for alerts that reference specific IP addresses or device groups. Device groups are created in the device map.
+
+To use advanced filters:
+
+1. Select **Advanced Filters** from the **Alerts** main view.
+
+2. Choose a device group or a device.
+
+3. Select **Confirm**.
+
+4. To clear the search, select **Clear All**.
+
+### Security versus operational alert search
+
+Switch between viewing operational and security alerts:
+
+- **Security** alerts represent potential malware traffic, network anomalies, policy violations, and protocol violations.
+
+- **Operational** alerts represent network anomalies, policy violations, and protocol violations.
+
+When none of the options are selected, all the alerts are displayed.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/alerts-security.png" alt-text="Security on the alerts screen.":::
+
+## Alert window options
+
+Alert messages provide the following actions:
+
+- Select :::image type="icon" source="media/how-to-work-with-alerts-sensor/acknowledge-an-alert-icon.png" border="false"::: to acknowledge an alert.
+
+- Select :::image type="icon" source="media/how-to-work-with-alerts-sensor/unacknowledge-an-alert-icon.png" border="false"::: to unacknowledge an alert.
+
+- Select :::image type="icon" source="media/how-to-work-with-alerts-sensor/pin-an-alert-icon.png" border="false"::: to pin an alert.
+
+- Select :::image type="icon" source="media/how-to-work-with-alerts-sensor/unpin-an-alert-icon.png" border="false"::: to unpin an alert.
+
+- Select :::image type="icon" source="media/how-to-work-with-alerts-sensor/acknowledge-all-alerts-icon.png" border="false"::: to acknowledge all alerts.
+
+- Select :::image type="icon" source="media/how-to-work-with-alerts-sensor/learn-and-acknowledge-all-alerts.png" border="false"::: to learn and acknowledge all alerts.
+
+- Select :::image type="icon" source="media/how-to-work-with-alerts-sensor/export-to-csv.png" border="false"::: to export the alert list to a CSV file and select the export option. Choose **Alert Export** for the regular export-to-CSV option. Or choose **Extended Alert Export** for the possibility to add separate rows for additional information about an alert in the CSV file.
+
+- Select the :::image type="icon" source="media/how-to-work-with-alerts-sensor/export-to-pdf.png" border="false"::: icon to download an alert report as a PDF file.
+
+- Select the :::image type="icon" source="media/how-to-work-with-alerts-sensor/download-pcap.png" border="false"::: icon to download the PCAP file. The file is viewable with Wireshark, the free network protocol analyzer.
+
+- Select :::image type="icon" source="media/how-to-work-with-alerts-sensor/download-filtered-pcap.png" border="false"::: to download a filtered PCAP file that contains only the relevant alert packets, thereby reducing output file size and allowing a more focused analysis. You can view the file by using Wireshark.
+
+- Select the :::image type="icon" source="media/how-to-work-with-alerts-sensor/show-alert-in-timeline.png" border="false"::: icon to show the alert in the event timeline.
+
+- Select the :::image type="icon" source="media/how-to-work-with-alerts-sensor/pin-an-alert-icon.png" border="false"::: icon to pin the alert.
+
+- Select the :::image type="icon" source="media/how-to-work-with-alerts-sensor/unpin-an-alert-icon.png" border="false"::: icon to unpin the alert.
+
+- Select :::image type="icon" source="media/how-to-work-with-alerts-sensor/learn-icon.png" border="false"::: to approve the traffic (security analysts and administrators only).
+
+- Select a device to display it in the device map.
+
+## Next steps
+
+[Manage the alert event](how-to-manage-the-alert-event.md)
+
+[Accelerate alert workflows](how-to-accelerate-alert-incident-response.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-view-information-provided-in-alerts https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-view-information-provided-in-alerts.md new file mode 100644
@@ -0,0 +1,92 @@
+---
+title: View information in alerts
+description: Select an alert from the Alerts window to review details.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/03/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# View information in alerts
+
+Select an alert from the **Alerts** window to review alert details. Details include the following information:
+
+- Alert metadata
+
+- Information about traffic, devices, and the event
+
+- Links to connected devices in the device map
+
+- Comments defined by security analysts and administrators
+
+- Recommendations for investigating the event
+
+## Alert metadata
+
+Alert details include the following alert metadata:
+
+ - Alert ID
+
+ - Policy engine that triggered the alert
+
+ - Date and time that the alert was triggered
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/internet-connectivity-detection-unauthorized.png" alt-text="Unauthorized internet connectivity detected.":::
+
+## Information about devices, traffic, and the event
+
+The alert message provides information about:
+
+ - The detected devices.
+
+ - The traffic detected between the devices, such as protocols and function codes.
+
+ - Insights into the implications of the event.
+
+You can use this information when deciding how to manage the alert event.
+
+## Links to connected devices in the device map
+
+To learn more about devices connected to the detected devices, you can select a device image in the alert and view connected devices in the map.
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/rcp-operation-failed.png" alt-text="RCP operation failed.":::
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/devices-screen-populated.png" alt-text="Devices screenshot.":::
+
+The map filters to the device that you selected, and other devices connected to it. The map also displays the **Quick Properties** dialog box for the devices detected in the alerts.
+
+## Comments defined by security analysts and administrators
+
+Alerts might include a list of predefined comments. For example, comments can be instructions for mitigation actions to take, or names of individuals to contact about the event.
+
+When you're managing an alert event, you can choose the comment or comments that best reflect the event status or the steps you've taken to investigate the alert.
+
+Selected comments are saved in the alert message. Working with comments enhances communication between individuals and teams during the investigation of an alert event. As a result, comments can accelerate incident response time.
+
+An administrator or security analyst predefines comments. Selected comments are not forwarded to partner systems defined in the forwarding rules.
+
+After you review the information in an alert, you can carry out various forensic steps to guide you in managing the alert event. For example:
+
+- Analyze recent device activity (data-mining report).
+
+- Analyze other events that occurred at the same time (event timeline).
+
+- Analyze comprehensive event traffic (PCAP file).
+
+## PCAP files
+
+In some cases, you can access a PCAP file from the alert message. This might be useful if you want more detailed information about the associated network traffic.
+
+To download a PCAP file, select :::image type="content" source="media/how-to-work-with-alerts-sensor/download-pcap.png" alt-text="Download icon."::: at the upper right of the **Alert details** dialog box.
+
+## Recommendations for investigating an event
+
+The **Recommendation** area of an alert displays information that might help you better understand an event. Review this information before managing the alert event or taking action on the device or the network.
+
+## See also
+
+[Accelerate Alert workflows](how-to-accelerate-alert-incident-response.md)
+
+[Manage the alert event](how-to-manage-the-alert-event.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-work with-the-sensor-console-dashboard https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-work with-the-sensor-console-dashboard.md new file mode 100644
@@ -0,0 +1,100 @@
+---
+title: Work with the sensor console dashboard
+description: The dashboard allows you to quickly view the security status of your network. It provides a high-level overview of threats to your whole system on a timeline along with information about related devices.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 11/03/2020
+ms.topic: article
+ms.service: azure
+---
+
+# The dashboard
+
+The dashboard allows you to quickly view the security status of your network. It provides a high-level overview of threats to your whole system on a timeline along with information about related devices, including:
+
+- Alerts at different severity levels:
+
+- Critical
+
+- Major
+
+- Minor
+
+- Warnings
+
+- The two gauges in the center of the page indicate the Packets per Second (PPS), and Unacknowledged Alerts (UA). **PPS** is the number of packets acknowledged by the system per second. **UA** is the number of alerts that have not been acknowledged yet.
+
+- List of unacknowledged alerts with their description.
+
+- Timeline with the alert description.
+
+:::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/dashboard-alert-screen-v2.png" alt-text="Dashboard":::
+
+## Viewing the latest alerts
+
+The Unacknowledged Alerts (UA) gauge in the center of the page indicates the number of such alerts. To view a list of alerts, select **More Alerts** at the bottom of the dashboard page or select **Alerts** on the side menu.
+
+:::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/unhandled-alerts-list.png" alt-text="Unacknowledged Alerts":::
+
+## Status boxes
+
+Each status box is described in this section.
+
+| Status Box and Gauges | Description |
+| -------------- | -------------- |
+| :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/critical-alert-status-box-v2.png" alt-text="Critical Alerts"::: | **Critical Alerts** - the box at the top middle of the page indicates the number of critical alerts. Select this box to display descriptions of the alerts on the timeline and on the list under the gauges, if any. |
+| :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/major-alert-status-box-v2.png" alt-text="Major Alerts"::: | **Major Alerts** - the box at the top right of the page indicates the number of major alerts. Select this box to display descriptions of the alerts on the timeline and on the list under the gauges, if any. |
+| :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/minor-alert-status-box-v2.png" alt-text="Minor Alerts"::: | **Minor Alerts** - the box at the bottom left of the page indicates the number of minor alerts. Select this box to display descriptions of the alerts on the timeline and on the list under the gauges, if any. |
+| :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/warnings-alert-status-box-v2.png" alt-text="Warning Alerts"::: | **Warning Alerts** - the box at the bottom middle of the page indicates the number of warning alerts. Select this box to display descriptions of the alerts on the timeline and on the list under the gauges, if any. |
+| :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/all-alert-status-box-v2.png" alt-text="All Alerts"::: | **All Alerts** - the box at the bottom right of the page indicates the total number of critical, major, minor, and warning alerts. Select this box to display descriptions of the alerts on the timeline and on the list under the gauges, if any. |
+| :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/packets-per-second-gauge-v2.png" alt-text="Packets Per Second"::: | **Packets Per Second (PPS)** - the PPS metric is an indicator of the performance of the network. |
+| :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/unacknowledged-events-gauge-v2.png" alt-text="Unacknowledged Events (UA)"::: | **Unacknowledged Events** - this metric indicates the number of unacknowledged events.
+
+## Using the timeline
+
+The alerts are displayed along a vertical timeline that includes date and time information.
+
+The Timeline graphically displays:
+
+- Critical Alerts
+
+- Major Alerts
+
+- Minor Alerts
+
+- Warning Alerts
+
+:::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/timeline-of-events.png" alt-text="Timeline graph":::
+
+## Viewing alerts
+
+Select the down arrow **V** at the bottom of an alert box to display the alert entry and devices information.
+
+:::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/extended-alert-screen.png" alt-text="Alert entry and devices information":::
+
+- Select the device or **Show Devices** to display the physical mode map. The subjected devices are highlighted.
+
+- Select :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/excel-icon.png" alt-text="Excel"::: to export a CSV file about the alert.
+
+- Administrators and Security Analysts Only ΓÇö Select :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/approve-all-icon.png" alt-text="Acknowledge all"::: to **Acknowledge All** associated alerts.
+
+- Select the alert entry to view the type and the description of the alert:
+
+- Select :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/pdf-icon.png" alt-text="PDF":::to download an alert report as a PDF file.
+
+- Select :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/pin-icon.png" alt-text="Pin":::to pin or unpin the alert.
+
+- Select :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/download-icon.png" alt-text="Download"::: to investigate the alert by downloading the PCAP file containing a network protocol analysis.
+
+- Select :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/cloud-download-icon.png" alt-text="Cloud"::: to download a filtered PCAP file that contains only the alert-relevant packets, thereby reducing output file size and allowing a more focused analysis. You can view it using [Wireshark](https://www.wireshark.org/).
+
+- Select :::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/navigate-icon.png" alt-text="Navigation"::: to navigate to the event timeline at the time of the requested alert.
+
+- Administrators and Security Analysts only - change the status of the alert from unacknowledged to acknowledged. Select Learn to approve detected activity.
+
+:::image type="content" source="media/how-to-work with-the-sensor-console-dashboard/unauthorized-internet-connectivity-detection-v3.png" alt-text="Unauthorized Internet connectivity detected":::
+
+## See also
+
+[Work with alerts on your sensor](how-to-work-with-alerts-on-your-sensor.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-work-with-alerts-on-premises-management-console https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-work-with-alerts-on-premises-management-console.md new file mode 100644
@@ -0,0 +1,225 @@
+---
+title: Work with alerts on the on-premises management console
+description: Use the on-premises management console to get an enterprise view of recent threats in your network and better understand how sensor users are handling them.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/06/2020
+ms.service: azure
+ms.topic: how-to
+---
+
+# Work with alerts on the on-premises management console
+
+You can do the following from the **Alerts** window in the management console:
+
+- Work with alert filters
+
+- Work with alert counters
+
+- View alert information
+
+- Manage alert events
+
+- Create alert exclusion rules
+
+- Trigger alert exclusion rules from external systems
+
+- Accelerate incident workflow with alert groups
+
+## View alerts in the on-premises management console
+
+The on-premises management console aggregates alerts from all connected sensors. This provides an enterprise view of recent threats in your network and helps you better understand how sensor users are handling them.
+
+:::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/alerts-with-samples.png" alt-text="Screenshot of the Alerts window.":::
+
+### Work with alert filters
+
+The **Alerts** window displays the alerts generated by sensors connected to your on-premises management console. You can view all the alerts for connected sensors or present the alerts sent from a specific:
+
+- Site
+
+- Zone
+
+- Device
+
+- Sensor
+
+Select **Clear Filters** to view all alerts.
+
+:::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/clear-filters.png" alt-text="Clear your filters by selecting the Clear Filters button.":::
+
+### Work with alert counters
+
+Alert counters provide a breakdown of alerts by severity and the acknowledged state.
+
+:::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/number-of-alerts.png" alt-text="The alert counter shows how many alerts you have.":::
+
+The following severity levels appear in the alert counter:
+
+- **Critical**: Indicates a malicious attack that should be handled immediately.
+
+- **Major**: Indicates a security threat that's important to address.
+
+- **Minor**: Indicates some deviation from the baseline behavior that might contain a security threat.
+
+- **Warning**: Indicates some deviation from the baseline behavior with no security threats.
+
+Severity levels are predefined.
+
+You can adjust the counter to provide numbers based on acknowledged and unacknowledged alerts. Unacknowledged alerts were triggered at Defender for IoT sensors but haven't yet been reviewed by operators at the sensor.
+
+When the **Show Acknowledged Alerts** option is selected, all the acknowledged alerts appear in the **Alerts** window.
+
+:::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/show-acknowledged-alerts.png" alt-text="View your acknowledged alerts.":::
+
+### View alert information
+
+The alert presents the following information:
+
+- A summary of the alert event.
+
+- Alert severity.
+
+- A link to the alert in the sensor that detected it.
+
+- An alert UUID. The UUID consists of the alert ID that's associated with the alert event detected on the sensor, separated by a hyphen and followed by a unique system ID number.
+
+**On-premises management console Alert UUID**
+
+:::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/unauthorized-internet-connectivity.png" alt-text="A device is connected but not authorized.":::
+
+**Sensor alert ID**
+
+:::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/internet-connectivity-alert.png" alt-text="Sensor alert ID.":::
+
+Working with UUIDs ensures that each alert displayed in the on-premises management console is searchable and identifiable by a unique number. This is required because alerts generated from multiple sensors might produce the same alert ID.
+
+> [!NOTE]
+> By default, UUIDs are displayed in the following partner systems when forwarding rules are defined: ArcSight, syslog servers, QRadar, Sentinel, and NetWitness. No setup is required.
+
+To view alert information:
+
+- From the alert list, select an alert.
+
+ :::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/alert-information.png" alt-text="Screenshot of alert information.":::
+
+To view the alert in the sensor:
+
+- Select **OPEN SENSOR** from the alert.
+
+To view the devices in a zone map:
+
+- To view the device map with a focus on the alerted device and all the devices connected to it, select **SHOW DEVICES**.
+
+## Manage alert events
+
+You can manage alert events detected by organizational sensors as follows:
+
+- Learn or acknowledge alert events. Select **Learn & Acknowledge** to learn all alert events that can be authorized and to acknowledge all alert events that are currently not acknowledged.
+
+ :::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/learn-and-acknowledge.png" alt-text="Select Learn & Acknowledge to learn all.":::
+
+- Mute and unmute alert events.
+
+## Create alert exclusion rules
+
+Instruct Defender for IoT to ignore alert triggers based on:
+
+- Time zones and time periods
+
+- Device address (IP, MAC, subnet)
+
+- Alert names
+
+- A specific sensor
+
+Create alert exclusion rules when you want Defender for IoT to ignore activity that will trigger an alert.
+
+For example, if you know that all the OT devices monitored by a specific sensor will be going through maintenance procedures for two days, you can define an exclusion rule that instructs Defender for IoT to suppress alerts detected by this sensor during the predefined period.
+
+### Alert exclusion logic
+
+Alert rule logic is `AND` based. This means an alert will be triggered only when all the rule conditions are met.
+
+If a rule condition is not defined, the condition will include all options. For example, if you don't include the name of a sensor in the rule, it will be applied to all sensors.
+
+:::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/create-alert-exclusion-v2.png" alt-text="Screenshot of the Create Exclusion Rule view.":::
+
+Rule summaries appear in the **Exclusion Rule** window.
+
+:::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/exclusion-summary-v2.png" alt-text="Screenshot of the Exclusion Rule Summary view.":::
+
+In addition to working with exclusion rules, you can suppress alerts by muting them.
+
+### Create exclusion rules
+
+To create exclusion rules:
+
+1. From the left pane of the on-premises management console, select **Alert Exclusion**. Define a new exclusion rule by selecting the **Add** icon :::image type="icon" source="media/how-to-work-with-alerts-on-premises-management-console/add-icon.png" border="false"::: in the upper-right corner of the window that opens. The **Create Exclusion Rule** dialog box opens.
+
+ :::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/create-alert-exclusion-view.png" alt-text="Create an alert exclusion by filling in the information here.":::
+
+2. Enter a rule name in the **Name** field. The name can't contain quotes (`"`).
+
+3. In the **By Time Zone/Period** section, enter a time period within a specific time zone. Use this feature when an exclusion rule is created for a specific time period in one time zone, but should be implemented at the same time in other time zones. For example, you might need to apply an exclusion rule between 8:00 AM and 10:00 AM in three different time zones. In this case, create three separate exclusion rules that use the same time period and the relevant time zone.
+
+4. Select **ADD**. During the exclusion period, no alerts are created on the connected sensors.
+
+ :::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/by-the-time-period.png" alt-text="Screenshot of the By Time Period view.":::
+
+5. In the **By Device Address** section, define the:
+
+ - Device IP address, MAC address, or subnet address that you want to exclude.
+
+ - Traffic direction for the excluded devices, source, and destination.
+
+6. Select **ADD**.
+
+7. In the **By Alert Title** section, start typing the alert title. From the drop-down list, select the alert title or titles to be excluded.
+
+ :::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/alert-title.png" alt-text="Screenshot of the By Alert Title view.":::
+
+8. Select **ADD**.
+
+9. In the **By Sensor Name** section, start typing the sensor name. From the drop-down list, select the sensor or sensors that you want to exclude.
+
+10. Select **ADD**.
+
+11. Select **SAVE**. The new rule appears in the list of rules.
+
+You can suppress alerts by either muting them or creating alert exclusion rules. This section describes potential use cases for both features.
+
+- **Exclusion rule**. Write an exclusion rule when:
+
+ - You know ahead of time that you want to exclude the event from the database. For example, you know that the scenario detected at a certain sensor will trigger irrelevant alerts. For example, you'll be carrying out maintenance work on organizational PLCs on a specific site and want to suppress alerts related to PLCs for this site.
+
+ - You want Defender for IoT to ignore events for a specific range of time (for system maintenance tasks).
+
+ - You want to ignore events in a specific subnet.
+
+ - You want to control alert events generated from several sensors with one rule.
+
+ - You don't want to track the alert exclusion as an event in the event log.
+
+- **Mute**. Mute an alert when:
+
+ - Items that need to be muted are not planned. You don't know ahead of time which events will be irrelevant.
+
+ - You want to suppress the alert from the **Alerts** window, but you still want to track it in the event log.
+
+ - You want to ignore events on a specific channel.
+
+### Trigger alert exclusion rules from external systems
+
+Trigger alert exclusion rules from external systems. For example, manage exclusion rules from enterprise ticketing systems or systems that manage network maintenance processes.
+
+Define the sensors, engines, start time, and end time to apply the rule. For more information, see [Defender for IoT API sensor and management console APIs](references-work-with-defender-for-iot-apis.md).
+
+Rules that you create by using the API appear in the **Exclusion Rule** window as RO.
+
+:::image type="content" source="media/how-to-work-with-alerts-on-premises-management-console/edit-exclusion-rule-screen.png" alt-text="Screenshot of the Edit Exclusion Rule view.":::
+
+## See also
+
+[Work with alerts on your sensor](how-to-work-with-alerts-on-your-sensor.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-work-with-alerts-on-your-sensor https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-work-with-alerts-on-your-sensor.md new file mode 100644
@@ -0,0 +1,71 @@
+---
+title: Work with alerts on your sensor
+description: Work with alerts to help you enhance the security and operation of your network.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 11/30/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Work with alerts on your sensor
+
+Work with alerts to help you enhance the security and operation of your network. Alerts provide you with information about:
+
+- Deviations from authorized network activity
+
+- Protocol and operational anomalies
+
+- Suspected malware traffic
+
+:::image type="content" source="media/how-to-work-with-alerts-sensor/address-scan-detected-screen.png" alt-text="Detect address scan.":::
+
+Alert management options let users:
+
+- Instruct sensors to learn activity detected as authorized traffic.
+
+- Acknowledge reviewing the alert.
+
+- Instruct sensors to mute events detected with identical devices and comparable traffic.
+
+Additional tools are available that help you enhance and expedite the alert investigation. For example:
+
+ - Add instructional comments for alert reviewers.
+
+ - Create alert groups for display at SOC solutions.
+
+ - Search for specific alerts; review related PCAP files; view the detected devices and other connected devices in the device map or send alert details to partner systems.
+
+ - Forward alerts to partner vendors: SIEM systems, MSSP systems, and more.
+
+## Alerts and engines
+
+Alerts are triggered when sensor engines detect changes in network traffic and behavior that need your attention. This article describes the kind of alerts that each engine triggers.
+
+| Alert type | Description |
+|-|-|
+| Policy violation alerts | Triggered when the Policy Violation engine detects a deviation from traffic previously learned. For example: <br /> - A new device is detected.  <br /> - A new configuration is detected on a device. <br /> - A device not defined as a programming device carries out a programming change. <br /> - A firmware version changed. |
+| Protocol violation alerts | Triggered when the Protocol Violation engine detects packet structures or field values that don't comply with the protocol specification. |
+| Operational alerts | Triggered when the Operational engine detects network operational incidents or a device malfunctioning. For example, a network device was stopped through a Stop PLC command, or an interface on a sensor stopped monitoring traffic. |
+| Malware alerts | Triggered when the Malware engine detects malicious network activity. For example, the engine detects a known attack such as Conficker. |
+| Anomaly alerts | Triggered when the Anomaly engine detects a deviation. For example, a device is performing network scans but is not defined as a scanning device. |
+
+Tools are available to enable and disable sensor engines. Alerts are not triggered from engines that are disabled. See [Control what traffic is monitored](how-to-control-what-traffic-is-monitored.md).
+
+## Alerts and sensor reporting
+
+Activity reflected in alerts can be calculated when you're generating Data Mining, Risk Assessment, and Attack Vector reports. When you manage these events, the sensor updates the reports accordingly.
+
+For example:
+
+ - Unauthorized connectivity between a device in a defined subnet and devices located outside the subnet (public) will appear in the Data Mining *Internet Activity* report and the Risk Assessment *Internet Connections* section. After these devices are authorized (learned), they're calculated in generating these reports.
+
+ - Malware events detected on network devices are reported in Risk Assessment reports. When alerts about malware events are *muted*, affected devices won't be calculated in the Risk Assessment report.
+
+## See also
+
+- [Learning and Smart IT Learning modes](how-to-control-what-traffic-is-monitored.md#learning-and-smart-it-learning-modes)
+- [View information provided in alerts](how-to-view-information-provided-in-alerts.md)
+- [Manage the alert event](how-to-manage-the-alert-event.md)
+- [Accelerate alert workflows](how-to-accelerate-alert-incident-response.md)
defender-for-iot https://docs.microsoft.com/en-us/azure/defender-for-iot/how-to-work-with-device-notifications https://github.com/MicrosoftDocs/azure-docs/commits/master/articles/defender-for-iot/how-to-work-with-device-notifications.md new file mode 100644
@@ -0,0 +1,98 @@
+---
+title: Work with device notifications
+description: Notifications provide information about network activity that might require your attention, along with recommendations for handling this activity.
+author: shhazam-ms
+manager: rkarlin
+ms.author: shhazam
+ms.date: 12/12/2020
+ms.topic: how-to
+ms.service: azure
+---
+
+# Work with device notifications
+
+Notifications provide information about network activity that might require your attention, along with recommendations for handling this activity. For example, you might receive a notification about:
+
+- An inactive device. If the device is no longer a part of your network, you can remove it. If the device is inactive, for example because it's mistakenly disconnected from the network, reconnect the device and dismiss the notification.
+
+- An IP address was detected on a device that's currently identified by a MAC address only. Respond by authorizing the IP address for the device.
+
+Responding to notifications improves the information provided in the device map, device inventory, and data-mining queries and reports. It also provides insight into legitimate network changes and potential network misconfigurations.
+
+To access notifications:
+
+- Select **System Settings** and then select **Data Enhancement**.
+
+## Notifications vs. alerts
+
+In addition to receiving notifications on network activity, you might receive *alerts*. Notifications provide information about network changes or unresolved device properties that don't present a threat. Alerts provide information about network deviations and changes that might present a threat to the network.
+
+To view notifications:
+
+1. Select **Device Map** from the console's left menu pane.
+
+2. Select the **Notifications** icon. The red number above the icon indicates the number of notifications.
+
+ :::image type="content" source="media/how-to-enrich-asset-information/notifications-alert-screenshot.png" alt-text="Notification icon.":::
+
+ The **Notifications** window displays all notifications that the sensor has detected.
+
+ :::image type="content" source="media/how-to-enrich-asset-information/notification-screen.png" alt-text="Notifications.":::
+
+## Filter the Notifications window
+
+Use search filters to display notifications of interest to you.
+
+| Filter by | Description |
+|--|--|
+| Filter by type | View notifications that cover a specific area of interest. For example, view only notifications for inactive devices. |
+| Filter by date range | Display notifications that cover a specific time range. For example, view notifications sent over the last week only. |
+| Search for specific information | Search for specific notifications. |
+
+## Notification events and responses
+
+The following table describes the notification event types you might receive, along with the options for handling them. You can update the device information with a recommended value or dismiss the notification. When you dismiss a notification, the device information is not updated with the recommended information. If traffic is detected again, the notification will be re-sent.
+
+| Notification event types | Description | Responses |
+|--|--|--|
+| New IP addresses | A new IP address is associated with the device. Five scenarios might be detected: <br /><br /> An additional IP address was associated with a device. This device is also associated with an existing MAC address.<br /><br /> A new IP address was detected for a device that's using an existing MAC address. Currently the device does not communicate by using an IP address.<br /> <br /> A new IP address was detected for a device that's using a NetBIOS name. <br /><br /> An IP address was detected as the management interface for a device associated with a MAC address. <br /><br /> A new IP address was detected for a device that's using a virtual IP address. | **Set Additional IP to Device** (merge devices) <br /> <br />**Replace Existing IP** <br /> <br /> **Dismiss**<br /> Remove the notification. |
+| Inactive devices | Traffic was not detected on a device for more than 60 days. | **Delete** <br /> If this device is not part of your network, remove it. <br /><br />**Dismiss** <br /> Remove the notification if the device is part of your network. If the device is inactive (for example, because it's mistakenly disconnected from the network), dismiss the notification and reconnect the device. |
+| New OT device | A subnet includes an OT device that's not defined in an ICS subnet. <br /><br /> Each subnet that contains at least one OT device can be defined as an ICS subnet. This helps differentiate between OT and IT devices on the map. | **Set as ICS Subnet** <br /> <br /> **Dismiss** <br />Remove the notification if the device is not part of the subnet. |
+| No subnets configured | No subnets are currently configured in your network. <br /><br /> Configure subnets for better representation in the map and the ability to differentiate between OT and IT devices. | **Open Subnets Configuration** and configure subnets. <br /><br />**Dismiss** <br /> Remove the notification. |
+| Operating system changes | One or more new operating systems have been associated with the device. | Select the name of the new OS that you want to associate with the device.<br /><br /> **Dismiss** <br /> Remove the notification. |
+| Subnets were detected | New subnets were discovered. | **Learn**<br />Automatically add the subnet.<br />**Open Subnet Configuration**<br />Add all missing subnet information.<br />**Dismiss**<br />Remove the notification. |
+| Device type change was detected | A new device type has been associated with the device. | **Set as {…}**<br />Associate the new type with the device.<br />**Dismiss**<br />Remove the notification. |
+
+## Respond to many notifications simultaneously
+
+You might need to handle several notifications simultaneously. For example:
+
+- If IT did an OS upgrade to a large set of network servers, you can instruct the sensor to learn the new server versions for all upgraded servers. 
+
+- If a group of devices in a certain line was phased out and isn't active anymore, you can instruct the sensor to remove these devices from the console.
+
+You can instruct the sensor to apply newly detected information to multiple devices or ignore it.
+
+To display notifications and handle notifications:
+
+1. Use the **filter by type, date range** option or the **Select All** option. Deselect notifications as required.
+
+2. Instruct the sensor to apply newly detected information to selected devices by selecting **LEARN**. Or, instruct the sensor to ignore newly detected information by selecting **DISMISS**. The number of notifications that you can simultaneously learn and dismiss, along with the number of notifications you must handle individually, is shown.
+
+**New IPs** and **No Subnets** configured events can't be handled simultaneously. They require manual confirmation.
+
+## Improve device OS classification: data enhancement
+
+The sensor continuously autodiscovers new OT devices. It also autodiscovers changes to previously discovered devices, including operating system types.
+
+Under certain circumstances, conflicts might be detected in discovered operating systems. This can happen because you have an OS version that refers to either desktop or server systems. If it happen