-1. Assign the **Cognitive Service OpenAI User** role to your deployed App Service resource

- 1. Go to the Azure portal and select the newly created Azure App Service resource

- 1. Go to **settings** -> **identity** -> **enable system assigned identity**

- 1. Select **Azure role assignments** and then **add role assignments**. Specify the following parameters:

- * Scope: resource group

- * Subscription: the subscription of your Azure OpenAI resource

- * Resource group of your Azure OpenAI resource

- * Role: **Cognitive Service OpenAI user**

-* Azure Machine Learning

-* Azure Machine Learning registries

-* Azure Storage (all sub resource types)

-* Azure Container Registry

-* Azure Key Vault

-* Azure AI services

-* Azure SQL Server

-* Azure Data Factory

-* Azure Event Hubs

-* Azure Redis Cache

-* Azure Databricks

-* Azure Database for PostgreSQL Single Server

-* Azure SQL Managed Instance

-* Azure API Management

-As a Python developer, you might also be interested in one of the following articles:

-| Getting started | Concepts| Scenarios and samples |

-|--|--|--|

-| <ul><li>[Create Python functions by using Visual Studio Code](./create-first-function-vs-code-python.md?pivots=python-mode-configuration)</li><li>[Create Python functions by using a terminal or command prompt](./create-first-function-cli-python.md?pivots=python-mode-configuration)</li></ul> | <ul><li>[Developer guide](functions-reference.md)</li><li>[Hosting options](functions-scale.md)</li><li>[Performance&nbsp;considerations](functions-best-practices.md)</li></ul> | <ul><li>[Image classification with PyTorch](machine-learning-pytorch.md)</li><li>[Azure Automation sample](/samples/azure-samples/azure-functions-python-list-resource-groups/azure-functions-python-sample-list-resource-groups/)</li><li>[Machine learning with TensorFlow](functions-machine-learning-tensorflow.md)</li><li>[Browse Python samples](/samples/browse/?products=azure-functions&languages=python)</li></ul> |

-| Getting started | Concepts| Samples |

-| | | |

-| <ul><li>[Create Python functions by using Visual Studio Code](./create-first-function-vs-code-python.md?pivots=python-mode-decorators)</li><li>[Create Python functions by using a terminal or command prompt](./create-first-function-cli-python.md?pivots=python-mode-decorators)</li></ul> | <ul><li>[Developer guide](functions-reference.md)</li><li>[Hosting options](functions-scale.md)</li><li>[Performance&nbsp;considerations](functions-best-practices.md)</li></ul> | <li>[Code Examples](functions-bindings-triggers-python.md)</li> |

-You can define a data collection rule to send data from multiple machines to multiple Log Analytics workspaces, including workspaces in a different region or tenant. Create the data collection rule in the *same region* as your Analytics workspace.

-1. On the **Destination** tab, add one or more destinations for the data source. You can select multiple destinations of the same or different types. For instance, you can select multiple Log Analytics workspaces, which is also known as multihoming.

-If using applicationinsights npm package.

-When using [Azure App Services autoinstrumentation](./azure-web-apps-nodejs.md) The `APPLICATIONINSIGHTS_AUTHENTICATION_STRING` environment variable lets Application Insights authenticate to Microsoft Entra ID and send telemetry.

-> [!NOTE]

-> Microsoft Entra authentication is only available for Python v2.7, v3.6, and v3.7. Support for Microsoft Entra ID in the Application Insights OpenCensus Python SDK

-is included starting with beta version [opencensus-ext-azure 1.1b0](https://pypi.org/project/opencensus-ext-azure/1.1b0/).

-> [!NOTE]

-> [OpenCensus Python SDK is deprecated](https://opentelemetry.io/blog/2023/sunsetting-opencensus/), but Microsoft supports it until retirement on September 30, 2024. We now recommend the [OpenTelemetry-based Python offering](./opentelemetry-enable.md?tabs=python) and provide [migration guidance](./opentelemetry-python-opencensus-migrate.md?tabs=aspnetcore).

-Construct the appropriate [credentials](/python/api/overview/azure/identity-readme#credentials) and pass them into the constructor of the Azure Monitor exporter. Make sure your connection string is set up with the instrumentation key and ingestion endpoint of your resource.

-The `OpenCensus` Azure Monitor exporters support these authentication types. We recommend using managed identities in production environments.

-#### System-assigned managed identity

-```python

-from azure.identity import ManagedIdentityCredential

-from opencensus.ext.azure.trace_exporter import AzureExporter

-from opencensus.trace.samplers import ProbabilitySampler

-from opencensus.trace.tracer import Tracer

-credential = ManagedIdentityCredential()

-tracer = Tracer(

- exporter=AzureExporter(credential=credential, connection_string="InstrumentationKey=<your-instrumentation-key>;IngestionEndpoint=<your-ingestion-endpoint>"),

- sampler=ProbabilitySampler(1.0)

-)

-...

-```

-#### User-assigned managed identity

-```python

-from azure.identity import ManagedIdentityCredential

-from opencensus.ext.azure.trace_exporter import AzureExporter

-from opencensus.trace.samplers import ProbabilitySampler

-from opencensus.trace.tracer import Tracer

-credential = ManagedIdentityCredential(client_id="<client-id>")

-tracer = Tracer(

- exporter=AzureExporter(credential=credential, connection_string="InstrumentationKey=<your-instrumentation-key>;IngestionEndpoint=<your-ingestion-endpoint>"),

- sampler=ProbabilitySampler(1.0)

-)

-...

-```

-When a request is made to the authorized URL, the client\_id is the application ID from your Microsoft Entra app, copied from the app's properties menu. The redirect\_uri is the homepage/login URL from the same Microsoft Entra app. When a request is successful, this endpoint redirects you to the sign-in page you provided at sign-up with the authorization code appended to the URL. See the following example:

-Internal logs could be turned on by using the following setup. After they're enabled, error logs will be shown in the console, including any error related to Microsoft Entra integration. Examples include failure to generate the token when the wrong credentials are supplied or errors when the ingestion endpoint fails to authenticate by using the provided credentials.

- - Dependency Agent supports the same [Windows versions that Azure Monitor Agent supports](../agents/agents-overview.md#supported-operating-systems), except Windows Server 2008 SP2, Windows Server 2022, and Azure Stack HCI.

- - For Linux, see [Dependency Agent Linux support](#dependency-agent-linux-support).

-* If you use the standard storage with cool access, see [Manage Azure NetApp Files standard storage with cool access](manage-cool-access.md#considerations) for more considerations.

-By `Default` (unless cool access retrieval policy is configured otherwise), data blocks on the cool tier that are read randomly again become "warm" and are moved back to the hot tier. Once marked as _warm_, the data blocks are again subjected to the temperature scan. However, large sequential reads (such as index and antivirus scans) on inactive data in the cool tier don't "warm" the data nor do they trigger inactive data to be moved back to the hot tier. Additionally, sequential reads for Azure NetApp Files, cross-region replication, or cross-zone replication do ***not*** "warm" the data.

- * Sequential reads from Azure NetApp Files backup, cross-zone replication, and cross-zone replication do not impact the temperature of the data.

- * If you're using a third-party backup service, configure it to use NDMP instead of the CIFS or NFS protocols. NDMP reads do not affect the temperature of the data.

-* Considerations for using cool access with [cross-region replication](cross-region-replication-requirements-considerations.md) (CRR) and [cross-zone replication](cross-zone-replication-introduction.md):

- * If the volume is in a CRR relationship as a source volume, you can enable cool access on it only if the [mirror state](cross-region-replication-display-health-status.md#display-replication-status) is `Mirrored`. Enabling cool access on the source volume automatically enables cool access on the destination volume.

- * If the volume is in a CRR relationship as a destination volume (data protection volume), enabling cool access isn't supported for the volume.

- * The cool access setting is updated automatically on the destination volume to be the same as the source volume. When you update the cool access setting on the source volume, the same setting is applied at the destination volume.

-* Considerations for using cool access with [Azure NetApp Files backup](backup-requirements-considerations.md):

- * When a backup is in progress for a volume, you canΓÇÖt enable cool access on the volume.

- * If a volume already contains cool-tiered data, you canΓÇÖt enable backup for the volume.

- * If backup is already enabled on a volume, you can enable cool access only if the baseline backup is complete.

-Defender for Cloud considers a resource sensitive if a Sensitive Information Type (SIT) is detected in it and the customer has configured the SIT to be considered sensitive. Defender for Cloud detects SITs that are considered sensitive by default.

-| Defender CSPM | CspmMonitorAws | To discover AWS resources permissions, read all resources except:<br> "consolidatedbilling:*"<br> "freetier:*"<br> "invoicing:*"<br> "payments:*"<br> "billing:*"<br> "tax:*"<br> "cur:*" |

-| Defender CSPM <br><br> Defender for Servers | DefenderForCloud-AgentlessScanner | To create and clean up disk snapshots (scoped by tag) ΓÇ£CreatedByΓÇ¥: "Microsoft Defender for Cloud" Permissions:<br> "ec2:DeleteSnapshot" "ec2:ModifySnapshotAttribute"<br> "ec2:DeleteTags"<br> "ec2:CreateTags"<br> "ec2:CreateSnapshots"<br> "ec2:CopySnapshot"<br> "ec2:CreateSnapshot"<br> "ec2:DescribeSnapshots"<br> "ec2:DescribeInstanceStatus"<br> Permission to EncryptionKeyCreation "kms:CreateKey"<br> "kms:ListKeys"<br> Permissions to EncryptionKeyManagement "kms:TagResource"<br> "kms:GetKeyRotationStatus"<br> "kms:PutKeyPolicy"<br> "kms:GetKeyPolicy"<br> "kms:CreateAlias"<br> "kms:TagResource"<br> "kms:ListResourceTags"<br> "kms:GenerateDataKeyWithoutPlaintext"<br> "kms:DescribeKey"<br> "kms:RetireGrant"<br> "kms:CreateGrant"<br> "kms:ReEncryptFrom" |

-| Defender CSPM <br><br> Defender for Storage | SensitiveDataDiscovery | Permissions to discover S3 buckets in the AWS account, permission for the Defender for Cloud scanner to access data in the S3 buckets.<br> S3 read only; KMS decrypt "kms:Decrypt" |

-| CIEM | DefenderForCloud-Ciem <br> DefenderForCloud-OidcCiem | Permissions for Ciem Discovery<br> "sts:AssumeRole"<br> "sts:AssumeRoleWithSAML"<br> "sts:GetAccessKeyInfo"<br> "sts:GetCallerIdentity"<br> "sts:GetFederationToken"<br> "sts:GetServiceBearerToken"<br> "sts:GetSessionToken"<br> "sts:TagSession" |

-| Defender for Servers | DefenderForCloud-DefenderForServers | Permissions to configure JIT Network Access: <br>"ec2:RevokeSecurityGroupIngress"<br> "ec2:AuthorizeSecurityGroupIngress"<br> "ec2:DescribeInstances"<br> "ec2:DescribeSecurityGroupRules"<br> "ec2:DescribeVpcs"<br> "ec2:CreateSecurityGroup"<br> "ec2:DeleteSecurityGroup"<br> "ec2:ModifyNetworkInterfaceAttribute"<br> "ec2:ModifySecurityGroupRules"<br> "ec2:ModifyInstanceAttribute"<br> "ec2:DescribeSubnets"<br> "ec2:DescribeSecurityGroups" |

-| Defender for Containers | DefenderForCloud-Containers-K8s | Permissions to List EKS clusters and Collect Data from EKS clusters. <br>"eks:UpdateClusterConfig"<br> "eks:DescribeCluster" |

-| Defender for Containers | DefenderForCloud-DataCollection | Permissions to CloudWatch Log Group created by Defender for Cloud <br>ΓÇ£logs:PutSubscriptionFilter"<br> "logs:DescribeSubscriptionFilters"<br> "logs:DescribeLogGroups" autp "logs:PutRetentionPolicy"<br><br> Permissions to use SQS queue created by Defender for Cloud <br>"sqs:ReceiveMessage"<br> "sqs:DeleteMessage" |

-| Defender for Containers | DefenderForCloud-Containers-K8s-cloudwatch-to-kinesis | Permissions to access Kinesis Data Firehose delivery stream created by Defender for Cloud<br> "firehose:*" |

-| Defender for Containers | DefenderForCloud-Containers-K8s-kinesis-to-s3 | Permissions to access S3 bucket created by Defender for Cloud <br> "s3:GetObject"<br> "s3:GetBucketLocation"<br> "s3:AbortMultipartUpload"<br> "s3:GetBucketLocation"<br> "s3:GetObject"<br> "s3:ListBucket"<br> "s3:ListBucketMultipartUploads"<br> "s3:PutObject" |

-| Defender for Servers | DefenderForCloud-ArcAutoProvisioning | Permissions to install Azure Arc on all EC2 instances using SSM <br>"ssm:CancelCommand"<br> "ssm:DescribeInstanceInformation"<br> "ssm:GetCommandInvocation"<br> "ssm:UpdateServiceSetting"<br> "ssm:GetServiceSetting"<br> "ssm:GetAutomationExecution"<br> "ec2:DescribeIamInstanceProfileAssociations"<br> "ec2:DisassociateIamInstanceProfile"<br> "ec2:DescribeInstances"<br> "ssm:StartAutomationExecution"<br> "iam:GetInstanceProfile"<br> "iam:ListInstanceProfilesForRole"<br> "ssm:GetAutomationExecution"<br> "ec2:DescribeIamInstanceProfileAssociations"<br> "ec2:DisassociateIamInstanceProfile"<br> "ec2:DescribeInstances"<br> "ssm:StartAutomationExecution"<br> "iam:GetInstanceProfile"<br> "iam:ListInstanceProfilesForRole" |

-| Defender CSPM | DefenderForCloud-DataSecurityPostureDB | Permission to Discover RDS instances in AWS account, create RDS instance snapshot, <br> - List all RDS DBs/clusters <br> - List all DB/Cluster snapshots <br> - Copy all DB/cluster snapshots <br> - Delete/update DB/cluster snapshot with prefixΓÇ»*defenderfordatabases* <br> - List all KMS keys <br> - Use all KMS keys only for RDS on source account <br> - List KMS keys with tag prefixΓÇ»*DefenderForDatabases* <br> - Create alias for KMS keys <br><br> Permissions required to discover, RDS instances<br> "rds:DescribeDBInstances"<br> "rds:DescribeDBClusters"<br> "rds:DescribeDBClusterSnapshots"<br> "rds:DescribeDBSnapshots"<br> "rds:CopyDBSnapshot"<br> "rds:CopyDBClusterSnapshot"<br> "rds:DeleteDBSnapshot"<br> "rds:DeleteDBClusterSnapshot"<br> "rds:ModifyDBSnapshotAttribute"<br> "rds:ModifyDBClusterSnapshotAttribute" "rds:DescribeDBClusterParameters"<br> "rds:DescribeDBParameters"<br> "rds:DescribeOptionGroups"<br> "kms:CreateGrant"<br> "kms:ListAliases"<br> "kms:CreateKey"<br> "kms:TagResource"<br> "kms:ListGrants"<br> "kms:DescribeKey"<br> "kms:PutKeyPolicy"<br> "kms:Encrypt"<br> "kms:CreateGrant"<br> "kms:EnableKey"<br> "kms:CancelKeyDeletion"<br> "kms:DisableKey"<br> "kms:ScheduleKeyDeletion"<br> "kms:UpdateAlias"<br> "kms:UpdateKeyDescription" |

-| Defender CSPM | MDCCspmCustomRole | To discover GCP resources <br> resourcemanager.folders.getIamPolicy<br> resourcemanager.folders.list<br> resourcemanager.organizations.get<br> resourcemanager.organizations.getIamPolicy<br> storage.buckets.getIamPolicy resourcemanager.folders.get<br> resourcemanager.projects.get<br> resourcemanager.projects.list<br> serviceusage.services.enable<br> iam.roles.create<br> iam.roles.list<br> iam.serviceAccounts.actAs<br> compute.projects.get<br> compute.projects.setCommonInstanceMetadata" |

-| Defender for Servers | microsoft-defender-for-servers <br> azure-arc-for-servers-onboard | Read-only access to get and list Compute Engine <br> resources roles/compute.viewer<br> roles/iam.serviceAccountTokenCreator<br> roles/osconfig.osPolicyAssignmentAdmin<br> roles/osconfig.osPolicyAssignmentReportViewer |

-| Defender for Database | defender-for-databases-arc-ap | Permissions to Defender for databases ARC auto provisioning <br> roles/compute.viewer <br> roles/iam.workloadIdentityUser <br> roles/iam.serviceAccountTokenCreator<br> roles/osconfig.osPolicyAssignmentAdmin<br> roles/osconfig.osPolicyAssignmentReportViewer |

-| Defender CSPM <br><br> Defender for Containers | mdc-containers-artifact-assess | Permission to Scan images from GAR and GCR. <br> Roles/artifactregistry.readerΓÇ»<br> Roles/storage.objectViewer |

-| Defender for Containers | mdc-containers-k8s-operator | Permissions to Collect Data from GKE clusters. Update GKE clusters to support IP restriction. <br> Roles/container.viewerΓÇ»<br> MDCGkeClusterWriteRole container.clusters.update* |

-**Setting email at the lab:**

-

-**Setting email at the VM:**

-

-### User object ID

-DevTest Labs uses the user object ID to show month-over-month cost trends and cost by resource information to lab admins. It allows them to track costs and manage thresholds for their Lab.

-**Estimated cost trend for the current calendar month:**

-

-**Estimated month-to-date cost by resource:**

-

-The DevTest Labs service uses the personal data for operational purposes. This data is critical for the service to deliver key features. If you set a retention policy on the user email address, lab users do not receive timely auto shutdown email notifications after their email address is deleted from our system. Similarly, the lab admin can't view month-over-month cost trends and cost by resource for machines in their labs if the user object IDs are deleted based on a retention policy. Therefore, this data needs to be retained for as long as the user's resource is active in the Lab.

-## How can I have the system to forget my personal data?

-As a lab user, if you like to have this personal data deleted, you can do so by deleting the corresponding resource in the Lab. The DevTest Labs service anonymizes the deleted personal data 30 days after it's deleted by the user.

-For example, If you delete your VM, or removed your email address, the DevTest Labs service anonymizes this data 30 days after the resource is deleted. The 30-day retention policy after deletion is to make sure that we provide an accurate month-over-month cost projection to the lab admin.

-You can export personal and lab usage data by using the Azure portal or PowerShell. The data is exported as two different CSV files:

-### Azure portal

-As a lab user, you can request an export on the personal data that the DevTest Labs service stores. To request for an export, navigate to the **Personal data** option on the **Overview** page of your lab. Select the **Request export** button kicks off the creation of a downloadable excel file in your Lab admin's storage account. You can then contact your lab admin to view this data.

-1. Select **Personal data** on the left menu.

- 

-2. Select the **resource group** that contains the lab.

- 

-3. Select the **storage account** in the resource group.

-4. On the **Storage account** page, select **Blobs**.

- 

-5. Select the container named **labresourceusage** in the list of containers.

- 

-6. Select the **folder** named after your lab. You find **csv** files for **disks** and **virtual machines** in your lab in this folder. You can download these csv files, filter the content for the lab user requesting an access, and share it with them.

- 

- [Parameter (Mandatory=$true, HelpMessage="The storage account name where to store usage data")]

- [string] $storageAccountName,

- [Parameter (Mandatory=$true, HelpMessage="The storage account key")]

- [string] $storageKey,

- [Parameter (Mandatory=$true, HelpMessage="The DevTest Lab name to get usage data from")]

- [string] $labName,

- [Parameter (Mandatory=$true, HelpMessage="The DevTest Lab subscription")]

- [string] $labSubscription

- )

-#Login

-Login-AzureRmAccount

-# Set the subscription for the lab

-Get-AzureRmSubscription -SubscriptionId $labSubscription | Select-AzureRmSubscription

-# DTL will create this container in the storage when invoking the action, cannot be changed currently

-$containerName = "labresourceusage"

-# Get the storage context

-$Ctx = New-AzureStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageKey

-$SasToken = New-AzureStorageAccountSASToken -Service Blob, File -ResourceType Container, Service, Object -Permission rwdlacup -Protocol HttpsOnly -Context $Ctx

-# Generate the storage blob uri

-$blobUri = $Ctx.BlobEndPoint + $SasToken

-# blobStorageAbsoluteSasUri and usageStartDate are required

- 'blobStorageAbsoluteSasUri' = $blobUri

-$startdate = (Get-Date).AddDays(-7)

-# Get the lab resource group

-$resourceGroupName = (Find-AzureRmResource -ResourceType 'Microsoft.DevTestLab/labs' | Where-Object { $_.Name -eq $labName}).ResourceGroupName

-

-# Create the lab resource id

-$resourceId = "/subscriptions/" + $labSubscription + "/resourceGroups/" + $resourceGroupName + "/providers/Microsoft.DevTestLab/labs/" + $labName + "/"

-# !!!!!!! this is the new resource action to get the usage data.

-$result = Invoke-AzureRmResourceAction -Action 'exportLabResourceUsage' -ResourceId $resourceId -Parameters $actionParameters -Force

-# Finish up cleanly

-if ($result.Status -eq "Succeeded") {

- Write-Output "Telemetry successfully downloaded for " $labName

- return 0

-}

-else

-{

- Write-Output "Failed to download lab: " + $labName

- Write-Error $result.toString()

- return -1

-}

-The key components in the above sample are:

-

- Invoke-AzureRmResourceAction -Action 'exportLabResourceUsage' -ResourceId $resourceId -Parameters $actionParameters -Force

- - **blobStorageAbsoluteSasUri** - The storage account URI with the Shared Access Signature (SAS) token. In the PowerShell script, this value could be passed in instead of the storage key.

- - **usageStartDate** - The beginning date to pull data, with the end date being the current date on which the action is executed. The granularity is at the day level, so even if you add time information, it will be ignored.

-### Exported data - a closer look

-Now letΓÇÖs take a closer look at the exported data. As mentioned earlier, once the data are successfully exported, there will be two CSV files.

-The **virtualmachines.csv** contains the following data columns:

-| Column name | Description |

-| -- | -- |

-| SubscriptionId | The subscription identifier that the lab exists in. |

-| LabUId | Unique GUID identifier for the lab. |

-| LabName | Name of the lab. |

-| LabResourceId | Fully qualified lab resource ID. |

-| ResourceGroupName | Name of the resource group that contains the VM |

-| ResourceId | Fully qualified resource ID for the VM. |

-| ResourceUId | GUID for the VM |

-| Name | Virtual machine name. |

-| CreatedTime | The date-time at which the VM was created. |

-| DeletedDate | The date-time at which the VM was deleted. If it's empty, deletion hasn't occurred, yet. |

-| ResourceOwner | Owner of the VM. If the value is empty, then it's either a claimable VM or created by a service principal. |

-| PricingTier | Pricing tier of the VM |

-| ResourceStatus | Availability state of the VM. Active, if still exists or Inactive, if the VM has been deleted. |

-| ComputeResourceId | Fully qualified virtual machine compute resource identifier. |

-| Claimable | Set to true if the VM is a claimable VM |

-| EnvironmentId | The environment resource identifier within which the Virtual machine was created in. It's empty when the VM wasn't created as part of an environment resource. |

-| ExpirationDate | Expiration date for the VM. It's set to empty, if an expiration date hasn't been set.

-| GalleryImageReferenceVersion | Version of the VM base image. |

-| GalleryImageReferenceOffer | Offer of the VM base image. |

-| GalleryImageReferencePublisher | Publisher of the VM base image. |

-| GalleryImageReferenceSku | Sku of the VM base image |

-| GalleryImageReferenceOsType | OS type of the VM base image |

-| CustomImageId | Fully qualified ID of the VM base custom image. |

-The data columns contained in **disks.csv** are listed below:

-| Column name | Description |

-| -- | -- |

-| SubscriptionId | ID of the subscription that contains the lab |

-| LabUId | GUID for the lab |

-| LabName | Name of the lab |

-| LabResourceId | Fully qualified resource ID for the lab |

-| ResourceGroupName | Name of the resource group that contains the lab |

-| ResourceId | Fully qualified resource ID for the VM. |

-| ResourceUId | GUID for the VM |

- |Name | The name of the attached disk |

-| CreatedTime |The date and time at which the data disk was created. |

-| DeletedDate | The date and time at which the data disk was deleted. |

-| ResourceStatus | Status of the resource. Active, if the resource exists. Inactive, when deleted. |

-| DiskBlobName | Blob name for the data disk. |

-| DiskSizeGB | The size of the data disk. |

-| DiskType | Type of the data disk. 0 for Standard, 1 for Premium. |

-| LeasedByVmId | Resource ID of the VM to which the data disk has been attached. |

-> [!NOTE]

-> If you are dealing with multiple labs and want to get overall information, the two key columns are the **LabUID** and the **ResourceUId**, which are the unique ids across subscriptions.

-The exported data can be manipulated and visualized using tools, like SQL Server, Power BI, etc. This feature is especially useful when you want to report usage of your lab to your management team that may not be using the same Azure subscription as you do.

-Once the export is complete, there will be multiple CSV files in the blob storage with the different resource information.

-

-Currently there are two CSV files:

-* *virtualmachines.csv* - contains information about the virtual machines in the lab

-* *disks.csv* - contains information about the different disks in the lab

-These files are stored in the *labresourceusage* blob container. The files are under the lab name, lab unique ID, date executed, and either `full` or the start date of the export request. An example blob structure is:

-* `labresourceusage/labname/1111aaaa-bbbb-cccc-dddd-2222eeee/<End>DD26-MM6-2019YYYY/full/virtualmachines.csv`

-* `labresourceusage/labname/1111aaaa-bbbb-cccc-dddd-2222eeee/<End>DD-MM-YYYY/26-6-2019/20-6-2019<Start>DD-MM-YYYY/virtualmachines.csv`

-description: Learn how to create, retrieve, and delete certificates from an Azure key vault using the JavaScript client library

-Get started with the Azure Key Vault certificate client library for JavaScript. [Azure Key Vault](../general/overview.md) is a cloud service that provides a secure store for certificates. You can securely store keys, passwords, certificates, and other secrets. Azure key vaults may be created and managed through the Azure portal. In this quickstart, you learn how to create, retrieve, and delete certificates from an Azure key vault using the JavaScript client library

-This application is using key vault name as an environment variable called `KEY_VAULT_NAME`.

-set KEY_VAULT_NAME=<your-key-vault-name>

-$Env:KEY_VAULT_NAME="<your-key-vault-name>"

-export KEY_VAULT_NAME=<your-key-vault-name>

-In this code, the name of your key vault is used to create the key vault URI, in the format `https://<your-key-vault-name>.vault.azure.net`. For more information about authenticating to key vault, see [Developer's Guide](/azure/key-vault/general/developers-guide#authenticate-to-key-vault-in-code).

-1. Create new text file and paste the following code into the **index.js** file.

- const keyVaultName = process.env["KEY_VAULT_NAME"];

- if(!keyVaultName) throw new Error("KEY_VAULT_NAME is empty");

- const url = "https://" + keyVaultName + ".vault.azure.net";

- const client = new CertificateClient(url, credential);

- "id": "https://YOUR-KEY-VAULT-NAME.vault.azure.net/certificates/YOUR-CERTIFICATE-NAME/YOUR-CERTIFICATE-VERSION",

- "vaultUrl": "https://YOUR-KEY-VAULT-NAME.vault.azure.net",

-description: Learn how to create, retrieve, and delete keys from an Azure key vault using the JavaScript client library

-Get started with the Azure Key Vault key client library for JavaScript. [Azure Key Vault](../general/overview.md) is a cloud service that provides a secure store for cryptographic keys. You can securely store keys, passwords, certificates, and other secrets. Azure key vaults may be created and managed through the Azure portal. In this quickstart, you learn how to create, retrieve, and delete keys from an Azure key vault using the JavaScript key client library

-This application is using key vault name as an environment variable called `KEY_VAULT_NAME`.

-set KEY_VAULT_NAME=<your-key-vault-name>

-$Env:KEY_VAULT_NAME="<your-key-vault-name>"

-export KEY_VAULT_NAME=<your-key-vault-name>

-In this code, the name of your key vault is used to create the key vault URI, in the format `https://<your-key-vault-name>.vault.azure.net`. For more information about authenticating to key vault, see [Developer's Guide](/azure/key-vault/general/developers-guide#authenticate-to-key-vault-in-code).

-1. Create new text file and paste the following code into the **index.js** file.

- const keyVaultName = process.env["KEY_VAULT_NAME"];

- if(!keyVaultName) throw new Error("KEY_VAULT_NAME is empty");

- const url = "https://" + keyVaultName + ".vault.azure.net";

- const client = new KeyClient(url, credential);

- "kid": "https://YOUR-KEY-VAULT-NAME.vault.azure.net/keys/YOUR-KEY-NAME/YOUR-KEY-VERSION",

- "id": "https://YOUR-KEY-VAULT-NAME.vault.azure.net/keys/YOUR-KEY-NAME/YOUR-KEY-VERSION",

- "vaultUrl": "https://YOUR-KEY-VAULT-NAME.vault.azure.net",

- "id": "https://YOUR-KEY-VAULT-NAME.vault.azure.net/keys/YOUR-KEY-NAME/YOUR-KEY-VERSION"

-description: Learn how to create, retrieve, and delete secrets from an Azure key vault using the JavaScript client library

-Get started with the Azure Key Vault secret client library for JavaScript. [Azure Key Vault](../general/overview.md) is a cloud service that provides a secure store for secrets. You can securely store keys, passwords, certificates, and other secrets. Azure key vaults may be created and managed through the Azure portal. In this quickstart, you learn how to create, retrieve, and delete secrets from an Azure key vault using the JavaScript client library

-This application is using key vault name as an environment variable called `KEY_VAULT_NAME`.

-set KEY_VAULT_NAME=<your-key-vault-name>

-$Env:KEY_VAULT_NAME="<your-key-vault-name>"

-export KEY_VAULT_NAME=<your-key-vault-name>

-In this code, the name of your key vault is used to create the key vault URI, in the format `https://<your-key-vault-name>.vault.azure.net`. For more information about authenticating to key vault, see [Developer's Guide](/azure/key-vault/general/developers-guide#authenticate-to-key-vault-in-code).

-1. Create new text file and paste the following code into the **index.js** file.

- const keyVaultName = process.env["KEY_VAULT_NAME"];

- if(!keyVaultName) throw new Error("KEY_VAULT_NAME is empty");

- const url = "https://" + keyVaultName + ".vault.azure.net";

- const client = new SecretClient(url, credential);

- "id": "https: //YOUR-KEYVAULT-NAME.vault.azure.net/secrets/secret1637692472606/YOUR-VERSION",

- "vaultUrl": "https: //YOUR-KEYVAULT-NAME.vault.azure.net",

- "id": "https: //YOUR-KEYVAULT-NAME.vault.azure.net/secrets/secret1637692472606/YOUR-VERSION",

- "vaultUrl": "https: //YOUR-KEYVAULT-NAME.vault.azure.net",

-> Ensure you have the latest `azure-fsspec` and `mltable` python libraries installed in your Python environment:

-> pip install -U azureml-fsspec mltable

-The in-product data that Azure Machine Learning stores is available for export and deletion. You can export and delete data with Azure Machine Learning studio, the CLI, and the SDK. Additionally, you can access telemetry data through the Azure Privacy portal.

-An Azure workspace relies on a **resource group** to hold the related resources for an Azure solution. When you create a workspace, you have the opportunity to use an existing resource group, or to create a new one. See [this page](../azure-resource-manager/management/manage-resource-groups-portal.md) to learn more about Azure resource groups.

-> If the resource is configured for soft delete, the data won't actually delete unless you optionally select to delete the resource permanently. For more information, see the following articles:

-> * [Workspace soft-deletion](concept-soft-delete.md).

-> * [Soft delete for blobs](../storage/blobs/soft-delete-blob-overview.md).

-> * [Soft delete in Azure Container Registry](../container-registry/container-registry-soft-delete-policy.md).

-> * [Azure log analytics workspace](../azure-monitor/logs/delete-workspace.md).

-> * [Azure Key Vault soft-delete](../key-vault/general/soft-delete-overview.md).

-Job history documents might contain personal user information. These documents are stored in the storage account in blob storage, in `/azureml` subfolders. You can download and delete the data from the portal.

-Azure Machine Learning studio provides a unified view of your machine learning resources - for example, notebooks, data assets, models, and jobs. Azure Machine Learning studio emphasizes preservation of a record of your data and experiments. You can delete computational resources - pipelines and compute resources - right in the browser. For these resources, navigate to the resource in question, and choose **Delete**.

-You can unregister data assets and archive jobs, but these operations don't delete the data. To entirely remove the data, data assets and job data require deletion at the storage level. Storage level deletion happens in the portal, as described earlier. Azure Machine Learning studio can handle individual deletion. Job deletion deletes the data of that job.

-Azure Machine Learning studio can handle training artifact downloads from experimental jobs. Choose the relevant **Job**. Choose **Output + logs**, and navigate to the specific artifacts you wish to download. Choose **...** and **Download**, or select **Download all**.

-To download a registered model, navigate to the **Model** and choose **Download**.

-You can download the outputs of a particular job using:

-The following machine learning resources can be deleted using the Python SDK:

-| Type | Function Call | Notes |

-| `Workspace` | [`delete`](/python/api/azureml-core/azureml.core.workspace.workspace#delete-delete-dependent-resources-false--no-wait-false-) | Use `delete-dependent-resources` to cascade the delete |

-| `Model` | [`delete`](/python/api/azureml-core/azureml.core.model%28class%29#delete--) | |

-| `ComputeTarget` | [`delete`](/python/api/azureml-core/azureml.core.computetarget#delete--) | |

-| `WebService` | [`delete`](/python/api/azureml-core/azureml.core.webservice%28class%29) | |

-Learn more about [Managing a workspace](how-to-manage-workspace.md).

-If you have a Microsoft Entra ID P2 or Microsoft Entra ID Governance license, [Microsoft Entra Privileged Identity Management (PIM)](/entra/id-governance/privileged-identity-management/pim-configure) is integrated into role assignment steps. For example, you can assign roles to users for a limited period of time. You can also make users eligible for role assignments so that they must activate to use the role, such as request approval. Eligible role assignments provide just-in-time access to a role for a limited period of time. You can't create eligible role assignments for applications, service principals, or managed identities because they can't perform the activation steps. This capability is being deployed in stages, so it might not be available yet in your tenant or your interface might look different.

-| Azure Files | Yes | Yes | | You must re-create any ACLs. |

-> This skill is in public preview under [supplemental terms of use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). Preview REST APIs support this skill.

-The **AML** skill is a preview feature, but depending on the endpoint, you can call it in a skillset that targets a stable API version. For example, a skillset that's created using 2023-11-01 stable API can include an **AML** skill even though it's a preview feature.

-PUT https://[servicename].search.windows.net/skillsets/[skillset name]?api-version=2023-11-01

- PUT https://[servicename].search.windows.net/skillsets/[skillset name]?api-version=2023-11-01

-> To lower the cost of skillset processing, enable [incremental enrichment (preview)](cognitive-search-incremental-indexing-conceptual.md) to cache and reuse any enrichments that are unaffected by changes made to a skillset. Caching requires Azure Storage (see [pricing](https://azure.microsoft.com/pricing/details/storage/blobs/) but the cumulative cost of skillset execution is lower if existing enrichments can be reused, especially for skillsets that use image extraction and analysis.

-In this case, you can use the [Reset Skills](/rest/api/searchservice/preview-api/reset-skills) to force reprocessing of a particular skill, including any downstream skills that have a dependency on that skill's output. This API accepts a POST request with a list of skills that should be invalidated and marked for reprocessing. After Reset Skills, follow with a [Run Indexer](/rest/api/searchservice/indexers/run) request to invoke the pipeline processing.

-+ [Create or Update Skillset (api-version=2023-11-01)](/rest/api/searchservice/skillsets/create-or-update) (New URI parameter on the request)

-+ [Create or Update Data Source (api-version=2023-11-01)](/rest/api/searchservice/data-sources/create-or-update), when called with a preview API version, provides a new parameter named "ignoreResetRequirement", which should be set to true when your update action shouldn't invalidate the cache. Use "ignoreResetRequirement" sparingly as it could lead to unintended inconsistency in your data that won't be detected easily.

-> [!IMPORTANT]

-> This feature is in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). The [2023-10-01-preview REST API](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2023-10-01-preview&preserve-view=true) supports the first iteration of this feature. The [2024-05-01-preview REST API](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) adds more properties and supports more text embedding models on Azure OpenAI.

-The [Import and vectorize data wizard](search-get-started-portal-import-vectors.md) in the Azure portal uses the **Azure OpenAI Embedding** skill to vectorize content. You can run the wizard and review the generated skillset to see how the wizard builds the skill for the text-embedding-ada-002 model.

-| `modelName` | This property is required if your skillset is created using the 2024-05-01-preview REST API. Set this property to the deployment name of an Azure OpenAI embedding model deployed on the provider specified through `resourceUri` and identified through `deploymentId`. Currently, the supported values are `text-embedding-ada-002`, `text-embedding-3-large`, and `text-embedding-3-small`. |

-| `pageOverlapLength` | Only applies if `textSplitMode` is set to `pages`. Each page starts with this number of characters from the end of the previous page. If this parameter is set to 0, there's no overlapping text on successive pages. This parameter is supported in [2023-10-01-Preview](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2023-10-01-preview&tabs=HTTP#splitskill&preserve-view=true) and newer preview REST APIs, and in Azure SDK beta packages that have been updated to support integrated vectorization. This [example](#example-for-chunking-and-vectorization) includes the parameter. |

-| `maximumPagesToTake` | Only applies if `textSplitMode` is set to `pages`. Number of pages to return. The default is 0, which means to return all pages. You should set this value if only a subset of pages are needed. This parameter is supported in [2023-10-01-Preview](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2023-10-01-preview&tabs=HTTP#splitskill&preserve-view=true) and newer preview REST APIs, and in Azure SDK beta packages that have been updated to support integrated vectorization. This [example](#example-for-chunking-and-vectorization) includes the parameter.|

-This example is for integrated vectorization, currently in preview. It adds preview-only parameters to the sample definition, and shows the resulting output.

-# Azure AI Vision multimodal embeddings skill

-> This feature is in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). The [2024-05-01-Preview REST API](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-Preview&preserve-view=true) supports this feature.

-| `modelVersion` | (Required) The model version to be passed to the Azure AI Vision multimodal embeddings API for generating embeddings. It is important that all embeddings stored in a given index field are generated using the same `modelVersion`. For information about version support for this model refer to [multimodal embeddings](../ai-services/computer-vision/concept-image-retrieval.md#what-are-vector-embeddings).|

-For mapping image embeddings to the index, you will need to use the [Index Projections](index-projections-concept-intro.md) feature. The payload for `indexProjections` might look something like this:

-POST {{baseUrl}}/datasources?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/skillsets?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexes?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexers?api-version=2023-11-01 HTTP/1.1

-GET {{baseUrl}}/indexers/cog-search-demo-idxr/status?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexes/cog-search-demo-idx/docs/search?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexes/cog-search-demo-idx/docs/search?api-version=2023-11-01 HTTP/1.1

-To improve relevance:

-+ New parameters [maxTextRecallSize and countAndFacetMode(preview)](#set-maxtextrecallsize-and-countandfacetmode-preview) give you more control over text inputs into a hybrid query.

-+ New [vector weighting](vector-search-how-to-query.md#vector-weighting-preview) lets you set the relative weight of the vector query. This feature is particularly useful in complex queries where two or more distinct result sets need to be combined, as is the case for hybrid search.

-+ (Optional) If you want text-to-vector conversion of a query string (currently in preview), [create and assign a vectorizer](vector-search-how-to-configure-vectorizer.md) to vector fields in the search index.

-+ [**2023-11-01**](/rest/api/searchservice/documents/search-post) stable version

-+ [**2023-10-01-preview**](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2023-10-01-preview&preserve-view=true), adds integrated vectorization to the vector side of a hybrid query

-+ [**2024-03-01-preview**](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2024-03-01-preview&preserve-view=true), adds narrow data types and scalar quantization to the vector side of a hybrid query

-+ [**2024-05-01-preview**](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2024-05-01-preview&preserve-view=true) adds `maxTextRecallSize` and `countAndFacetMode` specifially for hybrid search

-1. In [Search Explorer](search-explorer.md), make sure the API version is **2023-10-01-preview** or later.

- A hybrid query has a text query specified in `search`, and a vectory query specified under `vectorQueries.vector`.

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-11-01

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-11-01

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-11-01

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-11-01

-This section explains how to adjust the inputs to a hybrid query by controlling the amount BM25-ranked results that flow to the hybrid ranking model. Controlling over the BM25-ranked input gives you more options for relevance tuning in hybrid scenarios.

-> Another option to consider is a supplemental or replacement technique, is [vector weighting](vector-search-how-to-query.md#vector-weighting-preview), which increases the importance of vector queries in the request.

-1. Reduce `maxTextRecallSize` if vector similarity search is generally outperforming the text-side of the the hybrid query.

-POST https://{{searchServiceName}}.search.windows.net/indexes/hotels-vector-quickstart/docs/search?api-version=2023-11-01

-Starting in 2024-05-01-preview, you can [weight vector queries](vector-search-how-to-query.md#vector-weighting-preview) to increase or decrease their importance in a hybrid query.

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-11-01

-GET /indexes/hotels/docs?search+inn&scoringProfile=geo&scoringParameter=currentLocation--122.123,44.77233&api-version=2023-11-01

-POST /indexes/hotels/docs&api-version=2023-11-01

-POST /indexes/myxboxgames/docs/autocomplete?search&api-version=2023-11-01

-> [!Important]

-> Index projections are in public preview under [supplemental terms of use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). It's available through the Azure portal, preview REST APIs, Azure portal, and beta client libraries that have been updated to include the feature.

-You can use `2023-10-01-Preview` or newer preview REST APIs to create index projections through additions to a skillset. We recommend the latest preview API.

-+ [Create Skillset (api-version=2024-05-01-preview)](/rest/api/searchservice/skillsets/create?view=rest-searchservice-2024-05-01-preview&preserve-view=true)

-+ [Create or Update Skillset (api-version=2024-05-01-preview)](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true)

- PUT [service-name].search.windows.net/indexes/[index-name]?api-version=2023-11-01&allowIndexDowntime=true

-PUT [service-name].search.windows.net/indexes/[index name]?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexes/hotels/docs/search?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexes/hotels/docs/search?api-version=2023-11-01

-+ [Storage Explorer](../storage/blobs/quickstart-storage-explorer.md) or Storage browser (preview) in Azure portal to view enriched document structure and content. Consider this as your baseline tool for viewing knowledge store contents.

-1. In the storage account's left navigation pane, select **Storage browser (preview)** to view the new tables.

- POST {{baseUrl}}/indexes?api-version=2023-11-01 HTTP/1.1

- POST {{baseUrl}}/datasources?api-version=2023-11-01 HTTP/1.1

- POST {{baseUrl}}/skillsets?api-version=2023-11-01 HTTP/1.1

- POST {{baseUrl}}/indexers?api-version=2023-11-01 HTTP/1.1

-GET {{baseUrl}}/indexers/hotel-reviews-kstore-idxr/status?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexes/hotel-reviews-kstore-idxr/docs/search?api-version=2023-11-01 HTTP/1.1

-1. In Azure AI Search, [enable enrichment caching (preview)](search-howto-incremental-index.md) in the indexer and then [run the indexer](search-howto-run-reset-indexers.md) to execute the skillset and populate the cache. This is a preview feature, so be sure to use the preview REST API (api-version=2020-06-30-preview or later) on the indexer request. Once the cache is populated, you can modify projection definitions in a knowledge store free of charge (as long as the skills themselves aren't modified).

-> When developing projections, [enable enrichment caching (preview)](search-howto-incremental-index.md) so that you can reuse existing enrichments while editing projection definitions. Enrichment caching is a preview feature, so be sure to use the preview REST API (api-version=2020-06-30-preview or later) on the indexer request. Without caching, simple edits to a projection will result in a full reprocess of enriched content. By caching the enrichments, you can iterate over projections without incurring any skillset processing charges.

-| OperationVersion | String | The api-version used on the request. For example: `2023-11-01` |

-| Query_s | String | The query parameters used in the request. For example: `?search=beach access&$count=true&api-version=2023-11-01` |

-POST /indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST https://{{service-name}}.search.windows.net/indexes/hotel-rooms-sample/docs/search?api-version=2023-11-01

- ¹ The generally available functionality of [vector support](vector-search-overview.md) requires that you call other libraries or models for data chunking and vectorization. However, [integrated vectorization (preview)](vector-search-integrated-vectorization.md) embeds these steps. For code samples showing both approaches, see [azure-search-vectors repo](https://github.com/Azure/azure-search-vector-samples).

-The following code is copied from the [retrievethenread.py](https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/retrievethenread.py) file from a demo site. It produces `content` for the LLM from hybrid query search results. You can write a simpler query, but this example is inclusive of vector search and keyword search with semantic reranking and spell check. In the demo, this query is used to get initial content.

-# Alternatively simply use search_client.search(q, top=3) if not using semantic ranking

-POST /indexes/myxboxgames/docs/autocomplete?search&api-version=2023-11-01

-+ [`2023-11-01`](/rest/api/searchservice/search-service-api-versions#2023-11-01) is the most recent stable version.

-Upgrade instructions focus on code changes that get you through breaking changes from previous versions so that existing code runs the same as before, but on the newer API version. Once your code is in working order, you can decide whether to adopt newer features. To learn more about preview features, see [vector code samples](https://github.com/Azure/azure-search-vector-samples) and [What's New](whats-new.md).

-The `api-version` parameter is specified in the request header. In your application code that makes direct calls to the REST APIs, search for all instances of the existing version and then replace it with the new version. For more information about structuring a REST call, see [Quickstart: using REST](search-get-started-rest.md#set-up-visual-studio-code).

-If you're using an Azure SDK, those packages target specific versions of the REST API. Package updates might coincide with a REST API update, but each SDK is on it's own release schedule that ships independently of Azure AI Search REST API versions. Check the change log of your SDK package to determine whether a package release targets the latest REST API version.

-+ If you need to retrieve connection strings of another Azure resource such as Azure Storage or Azure Cosmos DB, use the APIs of that resource and published guidance to obtain the information.

-1. If the property is explicitly set, no action is required. If you used the default, be aware that the new default behavior is to filter before query execution. If you want post-query filtering, explicitly set `vectorFilterMode` to postfilter to retain the old behavior.

-This article identifies all data plane and control plane features in public preview. This list is helpful for checking feature status. It also provides usage guidance and reminders to always upgrade to newer preview API versions as they roll out.

-Preview API versions are cumulative and roll up to newer Preview versions. We recommend always using the latest preview APIs for full access to all preview features.

-| [**Scalar quantization**](vector-search-how-to-configure-compression-storage.md#option-1-configure-scalar-quantization) | Index | Compress vector index size in memory and on disk using built-in scalar quantization. | [Create or Update Index (preview)](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) to add a `compressions` section to a vector profile. |

-| [**Narrow data types**](vector-search-how-to-configure-compression-storage.md#option-2-assign-narrow-data-types-to-vector-fields) | Index | Assign a smaller data type on vector fields, assuming incoming data is of that data type. | [Create or Update Index (preview)](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) to specify a vector field definition. [Binary vector support](vector-search-how-to-index-binary-data.md) is added in 2024-05-01-preview.|

-| [**stored property**](vector-search-how-to-configure-compression-storage.md#option-3-set-the-stored-property-to-remove-retrievable-storage) | Index | Boolean that reduces storage of vector indexes by *not* storing retrievable vectors. | [Create or Update Index (preview)](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) to set `stored` on a vector field. |

-| [**Vectorizers**](vector-search-integrated-vectorization.md) | Queries | Text-to-vector conversion during query execution. | [Create or Update Index (preview)](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) to define a `vectorizer`. [Search POST (preview)](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2024-05-01-preview&preserve-view=true) for `vectorQueries`. Vectorizers should be paired with an equivalent skill that supports integrated vectorization during indexing. Skills used for embeddings during indexing include AzureOpenAIEmbedding, Azure AI Vision multimodal, AML for models in the Azure AI Studio model catalog. There are vectorizers that correspond to each one of these embedding skills. Always use the same embedding model for both queries and indexing. |

-| [**Integrated vectorization**](vector-search-integrated-vectorization.md) | Index, skillset | Skills-driven data chunking and embedding during indexing. | [Create or Update Skillset (preview)](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) for AzureOpenAIEmbedding skill and the data chunking properties of the Text Split skill. |

-| [**Import and vectorize data**](search-get-started-portal-import-vectors.md) | Azure portal | A wizard that creates a full indexing pipeline that includes data chunking and vectorization. The wizard creates all of the objects and configuration settings. | Available on all search services, in all regions. |

-| [**AzureOpenAIEmbedding skill**](cognitive-search-skill-azure-openai-embedding.md) | Applied AI (skills) | A new skill type that calls Azure OpenAI embedding model to generate embeddings during queries and indexing. | [Create or Update Skillset (preview)](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true). Also available in the portal through the [Import and vectorize data wizard](search-get-started-portal-import-vectors.md). |

-| [**Text Split skill**](cognitive-search-skill-textsplit.md) | Applied AI (skills) | Text Split has two new chunking-related properties in preview: `maximumPagesToTake`, `pageOverlapLength`. | [Create or Update Skillset (preview)](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) adds support for the preview properties. These properties are also used in the portal through the [Import and vectorize data wizard](search-get-started-portal-import-vectors.md). |

-| [**Incremental enrichment**](cognitive-search-incremental-indexing-conceptual.md) | Applied AI (skills) | Adds caching to an enrichment pipeline, allowing you to reuse existing output if a targeted modification, such as an update to a skillset or another object, doesn't change the content. Caching applies only to enriched documents produced by a skillset.| [Create or Update Indexer (preview)](/rest/api/searchservice/indexers/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true). |

-| [**Index projections**](index-projections-concept-intro.md) | Applied AI (skills) | A component of a skillset definition that defines the shape of a secondary index, supporting a one-to-many index pattern, where content from an enrichment pipeline can target multiple indexes.| [Create or Update Skillset (preview)](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true). Also available in the portal through the [Import and vectorize data wizard](search-get-started-portal-import-vectors.md). |

-+ [Change log for Azure SDK for .NET](https://github.com/Azure/azure-sdk-for-net/blob/Azure.Search.Documents_11.5.0-beta.5/sdk/search/Azure.Search.Documents/CHANGELOG.md)

-+ [Change log for Azure SDK for Java](https://github.com/Azure/azure-sdk-for-jav)

-+ [Change log for Azure SDK for JavaScript](https://github.com/Azure/azure-sdk-for-js/blob/%40azure/search-documents_11.3.3/sdk/search/search-documents/CHANGELOG.md)

-+ [Change log for Azure SDK for Python](https://github.com/Azure/azure-sdk-for-python/blob/azure-search-documents_11.3.0/sdk/search/azure-search-documents/CHANGELOG.md).

-PUT /indexers/[indexer name]?api-version=2023-11-01

-1. On the index page, at the top, select **Create demo app (preview)** to start the wizard.

- POST /indexes?api-version=2023-11-01

- POST /indexers?api-version=2023-11-01

-GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexes/hotels/docs/search?api-version=2023-11-01

-> [!IMPORTANT]

-> Image vectors are supported in stable API versions, but the wizard and vectorizers are in preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). By default, the wizard targets the [2024-05-01-Preview REST API](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true).

-+ Azure Storage to store image files as blobs. Use Azure Blob Storage, a standard performance (general-purpose v2) account. Access tiers can be hot, cool, and cold.

- Don't use Azure Data Lake Storage Gen2 (a storage account with a hierarchical namespace). This version of the wizard doesn't support Data Lake Storage Gen2.

-+ An index with vector fields, text fields, vectorizers, vector profiles, and vector algorithms. You can't modify the default index during the wizard workflow. Indexes conform to the [2024-05-01-preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true).

-> [!IMPORTANT]

-> The **Import and vectorize data** wizard is in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). By default, it targets the [2024-05-01-Preview REST API](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true).

-This quickstart helps you get started with [integrated vectorization (preview)](vector-search-integrated-vectorization.md) by using the **Import and vectorize data** wizard in the Azure portal. The wizard chunks your content and calls an embedding model to vectorize content during indexing and for queries.

-+ Source data is either Azure Blob Storage or OneLake files and shortcuts.

-+ [Azure Blob Storage](/azure/storage/common/storage-account-overview) or a [OneLake lakehouse](search-how-to-index-onelake-files.md).

- Azure Storage must be a standard performance (general-purpose v2) account. Access tiers can be hot, cool, and cold. Don't use Azure Data Lake Storage Gen2 (a storage account with a hierarchical namespace). This version of the wizard doesn't support Data Lake Storage Gen2.

- + You're prompted to choose either **Native blob soft delete** or **Soft delete using custom data**.

- + Your blob container must have deletion detection enabled before you run the wizard.

- + [Enable soft delete](/azure/storage/blobs/soft-delete-blob-overview) in Azure Storage, or [add custom metadata](search-howto-index-changed-deleted-blobs.md#soft-delete-strategy-using-custom-metadata) to your blobs that indexing recognizes as a deletion flag.

- + If you choose **Soft delete using custom data**, you're prompted to provide the metadata property name-value pair.

- + Do not skip this option. A connection error occurs during indexing if the wizard can't connect to Azure Storage.

- $url = "https://<YOUR-SEARCH-SERVICE-NAME>.search.windows.net/indexes?api-version=2023-11-01&`$select=name"

- $url = "https://<YOUR-SEARCH-SERVICE>.search.windows.net/indexes/hotels-quickstart?api-version=2023-11-01"

- $url = "https://<YOUR-SEARCH-SERVICE>.search.windows.net/indexes/hotels-quickstart/docs/index?api-version=2023-11-01"

- $url = 'https://<YOUR-SEARCH-SERVICE>.search.windows.net/indexes/hotels-quickstart/docs?api-version=2023-11-01&search=*&$count=true'

-$url = 'https://<YOUR-SEARCH-SERVICE>.search.windows.net/indexes/hotels-quickstart/docs?api-version=2023-11-01&search=restaurant wifi&$count=true&$select=HotelName,Description,Tags'

-$url = 'https://<YOUR-SEARCH-SERVICE>.search.windows.net/indexes/hotels-quickstart/docs?api-version=2023-11-01&search=*&$filter=Rating gt 4&$select=HotelName,Rating'

-$url = 'https://<YOUR-SEARCH-SERVICE>.search.windows.net/indexes/hotels-quickstart/docs?api-version=2023-11-01&search=boutique&$top=2&$select=HotelName,Category'

-$url = 'https://<YOUR-SEARCH-SERVICE>.search.windows.net/indexes/hotels-quickstart/docs?api-version=2023-11-01&search=pool&$orderby=Address/City asc&$select=HotelName, Address/City, Tags, Rating'

- GET {{baseUrl}}/indexes?api-version=2023-11-01&$select=name HTTP/1.1

- GET {{baseUrl}}/indexes?api-version=2023-11-01&$select=name HTTP/1.1

- POST {{baseUrl}}/indexes?api-version=2023-11-01 HTTP/1.1

- POST {{baseUrl}}/indexes/hotels-quickstart/docs/index?api-version=2023-11-01 HTTP/1.1

- POST {{baseUrl}}/indexes/hotels-quickstart/docs/search?api-version=2023-11-01 HTTP/1.1

- GET {{baseUrl}}/indexes/hotels-quickstart/stats?api-version=2023-11-01 HTTP/1.1

-DELETE {{baseUrl}}/indexes/hotels-quickstart?api-version=2023-11-01 HTTP/1.1

-> The stable **2023-11-01** REST API version depends on external solutions for data chunking and embedding. If you want to evaluate the [built-in data chunking and vectorization (public preview)](vector-search-integrated-vectorization.md) features, try the [**Import and vectorize data** wizard](search-get-started-portal-import-vectors.md) for an end-to-end walkthrough.

-+ Vector content is vectorized using an [external embedding model](vector-search-how-to-generate-embeddings.md) or [integrated vectorization (preview)](vector-search-integrated-vectorization.md) using Azure AI Search features that integrate with applied AI.

- You can follow these links to review the workflow: [Quickstart: Create an Azure AI Search index](search-get-started-portal.md) and [Quickstart: Integrated vectorization (preview)](search-get-started-portal-import-vectors.md).

- POST https://[service name].search.windows.net/indexes/hotels-sample-index/docs/index?api-version=2023-11-01

- GET https://[service name].search.windows.net/indexes/hotel-sample-index/docs/1111?api-version=2023-11-01

- POST https://myservice.search.windows.net/datasources?api-version=2023-11-01

- POST https://[service name].search.windows.net/indexes?api-version=2023-11-01

- POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2023-11-01

-POST https://myservice.search.windows.net/datasources?api-version=2023-11-01

-POST https://myservice.search.windows.net/datasources?api-version=2023-11-01

-If you need to clear the high water mark to reindex in full, you can use [Reset Indexer](/rest/api/searchservice/reset-indexer). For more selective reindexing, use [Reset Skills](/rest/api/searchservice/preview-api/reset-skills) or [Reset Documents](/rest/api/searchservice/preview-api/reset-documents). Through the reset APIs, you can clear internal state, and also flush the cache if you enabled [incremental enrichment](search-howto-incremental-index.md). For more background and comparison of each reset option, see [Run or reset indexers, skills, and documents](search-howto-run-reset-indexers.md).

-GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2023-11-01

-PUT /indexers/[indexer name]?api-version=2023-11-01

-+ [Native blob soft delete](#native-blob-soft-delete), applies to Blob Storage only

-+ You must use the [REST API (`api-version=2023-11-01`)](/rest/api/searchservice/search-service-api-versions) or newer version, or the indexer Data Source configuration in the Azure portal, to configure support for soft delete.

-In Azure AI Search, set a native blob soft deletion detection policy on the data source. You can do this either from the Azure portal, by using REST API (`api-version=2023-11-01`). The following instructions explain how to set the delete detection policy in Azure portal or through REST APIs.

-PUT https://[service name].search.windows.net/datasources/blob-datasource?api-version=2023-11-01

- PUT https://[service name].search.windows.net/datasources/file-datasource?api-version=2023-11-01

- POST https://[service name].search.windows.net/datasources?api-version=2023-11-01

- POST https://[service name].search.windows.net/indexes?api-version=2023-11-01

- POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2023-11-01

-POST https://[service name].search.windows.net/datasources?api-version=2023-11-01

-POST https://[service name].search.windows.net/datasources?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-PUT https://[service name].search.windows.net/indexers/[indexer name]?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

- POST https://[service name].search.windows.net/indexes?api-version=2023-11-01

- POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2023-11-01

-PUT /indexers/[indexer name]?api-version=2023-11-01

- POST https://[service name].search.windows.net/datasources?api-version=2023-11-01

- POST https://[service name].search.windows.net/indexes?api-version=2023-11-01

- POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2023-11-01

-POST https://[service name].search.windows.net/datasources?api-version=2023-11-01

-POST https://[service name].search.windows.net/datasources?api-version=2023-11-01

-POST https://[service name].search.windows.net/datasources?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexes?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexers?api-version=2023-11-01

-POST https://[service name].search.windows.net/datasources?api-version=2023-11-01

-POST https://[service name].search.windows.net/datasources?api-version=2023-11-01

-GET https://[service name].search.windows.net/indexers/[indexer name]/status?api-version=2023-11-01

- * For the URL, enter a sample query for your search index (`search=*` returns all documents, `$select=` lets you choose fields). The API version is required. Fully specified, a URL might look like this: `mydemo.search.windows.net/indexes/hotels-sample-index/docs?search=*&$select=HotelName,Description,Address/City&api-version=2023-11-01`. Omit the `https://` prefix.

-1. For *api-version*: Set `2023-11-01` as the **default value**, set **required** to *True*, and set **visibility** as *internal*.

- - {name: api-version, in: query, required: true, type: string, default: '2023-11-01',

-GET {{baseUrl}}/indexes/hotels-vector-quickstart/docs('1')?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexes/hotels-vector-quickstart/docs/search.index?api-version=2023-11-01 HTTP/1.1

-GET {{baseUrl}}/indexes/hotels-vector-quickstart/docs('1')?api-version=2023-11-01 HTTP/1.1

- GET https://[service name].search.windows.net/indexes/hotel-sample-index/docs/1111?api-version=2023-11-01

- GET https://[service name].search.windows.net/indexes/cog-search-demo/docs/aHR0cHM6Ly9oZWlkaWJsb2JzdG9yYWdlMi5ibG9iLmNvcmUud2luZG93cy5uZXQvY29nLXNlYXJjaC1kZW1vL2d1dGhyaWUuanBn0?api-version=2023-11-01

- POST https://[service name].search.windows.net/indexes/hotels-sample-index/docs/index?api-version=2023-11-01

- PUT /indexers/<indexer-name>?api-version=2023-11-01

-PUT /indexers?api-version=2023-11-01

-PUT /indexers/blob-indexer?api-version=2023-11-01

- POST https://myservice.search.windows.net/datasources?api-version=2023-11-01

- POST https://myservice.search.windows.net/indexers?api-version=2023-11-01

-az rest --method get --uri https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/contoso/providers/Microsoft.Search/searchServices/contoso-search/sharedPrivateLinkResources/blob-pe?api-version=2023-11-01

-PUT https://[service name].search.windows.net/indexers/[indexer name]?api-version=2023-11-01

-PUT https://[service name].search.windows.net/indexers/[indexer name]?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST /indexes/hotels/docs/search?api-version=2023-11-01

-POST /indexes/hotels/docs/search?api-version=2023-11-01

- | where Query_s != "?api-version=2023-11-01&search=*"

-1. Optionally, set a Column filter on *Query_s* to search over a specific syntax or string. For example, you could filter over *is equal to* `?api-version=2023-11-01&search=*&%24filter=HotelName`.

-POST /indexes/hotels-sample-index/docs/search?api-version=2023-11-01

- POST /indexes/good-books/docs/search?api-version=2023-11-01

- POST /indexes/good-books/docs/search?api-version=2023-11-01

-POST /indexes/good-books/docs/search?api-version=2023-11-01

-POST /indexes/good-books/docs/search?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-https://{{service-name}}.search.windows.net/indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST /indexes/hotel-samples-index/docs/search?api-version=2023-11-01

-POST /indexes/hotel-samples-index/docs/search?api-version=2023-11-01

-POST /indexes/hotel-samples-index/docs/search?api-version=2023-11-01

-POST /indexes/hotel-samples-index/docs/search?api-version=2023-11-01

-POST /indexes/hotel-samples-index/docs/search?api-version=2023-11-01

-POST /indexes/hotel-samples-index/docs/search?api-version=2023-11-01

-POST https://{{searchServiceName}}.search.windows.net/indexes/hotels-vector-quickstart/docs/search?api-version=2024-05-01-preview

-https://{{service-name}}.search.windows.net/indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST /indexes/hotel-samples-index/docs/search?api-version=2023-11-01

-GET /indexes/hotels-sample-index/docs/41?api-version=2023-11-01

-POST /indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST /indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST /indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST /indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST /indexes/v/docs/search?api-version=2023-11-01

-POST /indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST /indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST /indexes/hotels-sample-index/docs/search?api-version=2023-11-01

-POST {{baseUrl}}/indexes?api-version=2023-11-01 HTTP/1.1

-GET /indexes/my-new-index/docs?search=*&api-version=2023-11-01&api-key={{queryApiKey}}

-$uri= 'https://<YOUR-SEARCH-SERVICE>.search.windows.net/indexes?api-version=2023-11-01&$select=name'

-$uri= 'https://<YOUR-SEARCH-SERVICE>.search.windows.net/indexes/<YOUR-INDEX-NAME>?api-version=2023-11-01'

-$uri= 'https://<YOUR-SEARCH-SERVICE>.search.windows.net/synonyms?api-version=2023-11-01&$select=name'

-$uri= 'https://<YOUR-SEARCH-SERVICE>.search.windows.net/synonyms/<YOUR-SYNONYM-MAP-NAME>?api-version=2023-11-01'

-+ For indexers, data sources, and skillsets, only those fields that store connection strings, descriptions, keys, and user inputs are encrypted. For example, skillsets have Azure AI services keys, and some skills accept user inputs, such as custom entities. In both cases, keys and user inputs into skills are encrypted.

- POST https://{{baseUrl}}/indexes/{{index-name}}/docs/search?api-version=2023-11-01 HTTP/1.1

- POST https://{{baseUrl}}/indexes/{{index-name}}/docs/search?api-version=2023-11-01 HTTP/1.1

- POST https://[search service].search.windows.net/indexes/securedfiles/docs/index?api-version=2023-11-01

-POST https://[search service].search.windows.net/indexes/securedfiles/docs/index?api-version=2023-11-01

-POST https://[service name].search.windows.net/indexes/securedfiles/docs/search?api-version=2023-11-01

-POST {{baseUrl}}/datasources?api-version=2023-11-01 HTTP/1.1

-Location: https://free-demo-search-svc.search.windows.net:443/datasources('ny-philharmonic-ds')?api-version=2023-11-01

-POST {{baseUrl}}/indexes?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexers?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexes/ny-philharmonic-index/docs/search?api-version=2023-11-01 HTTP/1.1

- "@odata.nextLink": "https://free-demo-search-svc.search.windows.net/indexes/ny-philharmonic-index/docs/search?api-version=2023-11-01"

-POST {{baseUrl}}/indexes/ny-philharmonic-index/docs/search?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexers/ny-philharmonic-indexer/reset?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexers/ny-philharmonic-indexer/run?api-version=2023-11-01 HTTP/1.1

-GET {{baseUrl}}/indexers/ny-philharmonic-indexer/status?api-version=2023-11-01 HTTP/1.1

-url = "https://{}.search.windows.net/indexes/{}/docs/search?api-version=2023-11-01".format(search_service, search_index)

-POST /synonymmaps?api-version=2023-11-01

-POST /indexes?api-version=2023-11-01

-+ Rich indexing with [integrated data chunking and vectorization (preview)](vector-search-integrated-vectorization.md), [lexical analysis](search-analyzers.md) for text, and [optional applied AI](cognitive-search-concept-intro.md) for content extraction and transformation

-+ [2023-11-01 (REST)](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2023-11-01&preserve-view=true)

-This article explains how to invoke the semantic ranker on queries. You can apply semantic ranking to text queries, hybrid queries, and vector queries if your search documents contain string fields and the [vector query has a text representation](vector-search-how-to-query.md#query-with-integrated-vectorization-preview).

- POST https://[service name].search.windows.net/indexes/hotels-sample-index/docs/search?api-version=2023-11-01     

- POST {{baseUrl}}/indexes?api-version=2023-11-01 HTTP/1.1

- POST {{baseUrl}}/indexes/phone-numbers-index/docs/index?api-version=2023-11-01 HTTP/1.1

- GET {{baseUrl}}/indexes/phone-numbers-index/docs/search?api-version=2023-11-01&search=(425) 555-0100 HTTP/1.1

- GET {{baseUrl}}/indexes/phone-numbers-index/docs/search?api-version=2023-11-01&search=4255550100 HTTP/1.1

-POST {{baseUrl}}/indexes/phone-numbers-index/analyze?api-version=2023-11-01 HTTP/1.1

- DELETE {{baseUrl}}/indexes/phone-numbers-index?api-version=2023-11-01 HTTP/1.1

- POST {{baseUrl}}/indexes?api-version=2023-11-01 HTTP/1.1

-POST {{baseUrl}}/indexes/tutorial-first-analyzer/analyze?api-version=2023-11-01 HTTP/1.1

-You can set a [**vector filter modes on a vector query**](vector-search-how-to-query.md) to specify whether you want filtering before or after query execution.

-This article describes each filter mode and provides guidance on when to use each one.

-# Chunking large documents for vector search solutions in Azure AI Search

-This article describes several approaches for data chunking. Chunking is only required if source documents are too large for the maximum input size imposed by models.

-> [!NOTE]

-> If you're using the generally available version of [vector search](vector-search-overview.md), data chunking and embedding requires external code, such as library or a custom skill. A new feature called [integrated vectorization](vector-search-integrated-vectorization.md), currently in preview, offers internal data chunking and embedding. Integrated vectorization takes a dependency on indexers, skillsets, the Text Split skill, and the AzureOpenAiEmbedding skill (or a custom skill). If you can't use the preview features, the examples in this article provide an alternative path forward.

-If you have large documents, you must insert a chunking step into indexing and query workflows that breaks up large text. When using [integrated vectorization (preview)](vector-search-integrated-vectorization.md), a default chunking strategy using the [text split skill](./cognitive-search-skill-textsplit.md) is applied. You can also apply a custom chunking strategy using a [custom skill](cognitive-search-custom-skill-web-api.md). Some libraries that provide chunking include:

-+ [Text Split skill (preview](#text-split-skill-example)

-Integrated data chunking through [Text Split skill](cognitive-search-skill-textsplit.md) is in public preview. Use a preview REST API or an Azure SDK beta package for this scenario.

-This section describes the built-in data chunking using a skills-driven approach and [Text Split skill parameters](cognitive-search-skill-textsplit.md#skill-parameters).

-description: Configure vector compression options and vector storage using narrow data types, built-in scalar quantization, and storage options.

-# Configure vector quantization and reduced storage for smaller vectors in Azure AI Search

-> [!IMPORTANT]

-> These features are in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). The [2024-03-01-preview REST API](/rest/api/searchservice/operation-groups?view=rest-searchservice-2024-03-01-preview&preserve-view=true) and later preview APIs provide the new data types, vector compression properties, and the `stored` property. We recommend using the latest preview APIs.

-This article describes vector quantization and other techniques for compressing vector indexes in Azure AI Search.

-As a first step, review the three options for reducing the amount of storage used by vector fields. These options aren't mutually exclusive.

-We recommend scalar quantization because it compresses vector size in memory and on disk with minimal effort, and that tends to provide the most benefit in most scenarios. In contrast, narrow types (except for `Float16`) require a special effort into making them, and `stored` saves on disk storage, which isn't as expensive as memory.

-| [Add scalar quantization](#option-1-configure-scalar-quantization) | Use built-in scalar quantization to compress native `Float32` embeddings to `Int8`. This option reduces storage in memory and on disk with no degradation of query performance. Smaller data types like `Int8` produce vector indexes that are less content-rich than those with `Float32` embeddings. To offset information loss, built-in compression includes options for post-query processing using uncompressed embeddings and oversampling to return more relevant results. Reranking and oversampling are specific features of built-in scalar quantization of `Float32` or `Float16` fields and can't be used on embeddings that undergo custom quantization. |

-| [Assign smaller primitive data types to vector fields](#option-2-assign-narrow-data-types-to-vector-fields) | Narrow data types, such as `Float16`, `Int16`, `Int8`, and `byte` (binary) consume less space in memory and on disk, but you must have an embedding model that outputs vectors in a narrow data format. Or, you must have custom quantization logic that outputs small data. A third use case that requires less effort is recasting native `Float32` embeddings produced by most models to `Float16`. See [Index binary vectors](vector-search-how-to-index-binary-data.md) for details about binary vectors. |

-All of these options are defined on an empty index. To implement any of them, use the Azure portal, [2024-03-01-preview REST APIs](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-03-01-preview&preserve-view=true), or a beta Azure SDK package.

-## Option 1: Configure scalar quantization

-Built-in scalar quantization is recommended because it reduces memory and disk storage requirements, and it adds reranking and oversampling to offset the effects of a smaller index. Built-in scalar quantization can be applied to vector fields containing `Float32` or `Float16` data.

-To use built-in vector compression:

-+ Add `vectorSearch.compressions` to a search index. The compression algorithm supported in this preview is *scalar quantization*.

-+ Set optional properties to mitigate the effects of lossy indexing. Both `rerankWithOriginalVectors` and `defaultOversampling` provide optimizations during query execution.

-+ Add `vectorSearch.profiles.compression` to a new vector profile.

-+ Assign the new vector profile to a new vector field.

-### Add compression settings and set optional properties

-In an index definition created using [2024-03-01-preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-03-01-preview&preserve-view=true), add a `compressions` section. Use the following JSON as a template.

-```json

-"compressions": [

-ΓÇ» ΓÇ» ΓÇ» {ΓÇ»

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "name": "my-scalar-quantization",ΓÇ»

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "kind": "scalarQuantization",ΓÇ»

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "rerankWithOriginalVectors": true,ΓÇ» (optional)

-ΓÇ» ΓÇ» ΓÇ» ΓÇ» "defaultOversampling": 10.0,ΓÇ» (optional)

-        "scalarQuantizationParameters": {  (optional)

-          "quantizedDataType": "int8",  (optional)

-       }

-ΓÇ» ΓÇ» ΓÇ» }ΓÇ»

-ΓÇ» ΓÇ»]

-+ `kind` must be set to `scalarQuantization`. This is the only quantization method supported at this time.

-+ `rerankWithOriginalVectors` uses the original, uncompressed vectors to recalculate similarity and rerank the top results returned by the initial search query. The uncompressed vectors exist in the search index even if `stored` is false. This property is optional. Default is true.

-+ `defaultOversampling` considers a broader set of potential results to offset the reduction in information from quantization. The formula for potential results consists of the `k` in the query, with an oversampling multiplier. For example, if the query specifies a `k` of 5, and oversampling is 20, then the query effectively requests 100 documents for use in reranking, using the original uncompressed vector for that purpose. Only the top `k` reranked results are returned. This property is optional. Default is 4.

-+ `quantizedDataType` must be set to `int8`. This is the only primitive data type supported at this time. This property is optional. Default is `int8`.

-### Add a compression setting to a vector profile

-Scalar quantization is specified as a property in a *new* vector profile. Creation of a new vector profile is necessary for building compressed indexes in memory.

-Within the profile, you must use the Hierarchical Navigable Small Worlds (HNSW) algorithm. Built-in quantization isn't supported with exhaustive KNN.

-1. Create a new vector profile and add a compression property.

- "profiles": [

- {

- "name": "my-vector-profile",

- "compression": "my-scalar-quantization",ΓÇ»

- "algorithm": "my-hnsw-vector-config-1",

- "vectorizer": null

- }

- ]

-1. Assign a vector profile to a *new* vector field. Scalar quantization reduces content to `Int8`, so make sure your content is either `Float32` or `Float16`.

- In Azure AI Search, the Entity Data Model (EDM) equivalents of `Float32` and `Float16` types are `Collection(Edm.Single)` and `Collection(Edm.Half)`, respectively.

- "name": "DescriptionVector",

- "vectorSearchProfile": "my-vector-profile"

-Scalar quantization reduces the resolution of each number within each vector embedding. Instead of describing each number as a 32-bit floating point number, it uses an 8-bit integer. It identifies a range of numbers (typically 99th percentile minimum and maximum) and divides them into a finite number of levels or bin, assigning each bin an identifier. In 8-bit scalar quantization, there are 2^8, or 256, possible bins.

-## Option 2: Assign narrow data types to vector fields

-Vector fields store vector embeddings, which are represented as an array of numbers. When you specify a field type, you specify the underlying primitive data type used to hold each number within these arrays. The data type affects how much space each number takes up.

-Using preview APIs, you can assign narrow primitive data types to reduce the storage requirements of vector fields.

-1. Review the [data types for vector fields](/rest/api/searchservice/supported-data-types#edm-data-types-for-vector-fields):

- + `Collection(Edm.Single)` 32-bit floating point (default)

- + `Collection(Edm.Half)` 16-bit floating point

- + `Collection(Edm.Int16)` 16-bit signed integer

- + `Collection(Edm.SByte)` 8-bit signed integer

- > [!NOTE]

- > Binary data types aren't currently supported.

-1. Choose a data type that's valid for your embedding model's output, or for vectors that undergo custom quantization.

- Most embedding models output 32-bit floating point numbers, but if you apply custom quantization, your output might be `Int16` or `Int8`. You can now define vector fields that accept the smaller format.

- Text embedding models have a native output format of `Float32`, which maps to `Collection(Edm.Single)` in Azure AI Search. You can't map that output to `Int8` because casting from `float` to `int` is prohibited. However, you can cast from `Float32` to `Float16` (or `Collection(Edm.Half)`), and this is an easy way to use narrow data types without extra work.

- The following table provides links to several embedding models that use the narrow data types.

- | Embedding model | Native output | Valid types in Azure AI Search |

-1. [Define and build the index](vector-search-how-to-create-index.md). You can use the Azure portal, [2024-03-01-preview](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-03-01-preview&preserve-view=true), or a beta Azure SDK package for this step.

-1. Check the results. Assuming the vector field is marked as retrievable, use [Search explorer](search-explorer.md) or [REST API](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2024-03-01-preview&preserve-view=true) to verify the field content matches the data type. Be sure to use the correct `2024-03-01-preview` API version for the query, otherwise the new properties aren't shown.

- To check vector index size, use the Azure portal or the [2024-03-01-preview](/rest/api/searchservice/indexes/get-statistics?view=rest-searchservice-2024-03-01-preview&preserve-view=true).

-The `stored` property is a new boolean on a vector field definition that determines whether storage is allocated for retrievable vector field content. The `stored` property is set to true by default. If you don't need vector content in a query response, you can save up to 50 percent storage per field by setting `stored` to false.

-When evaluating whether to set this property, consider whether you need vectors in the response. Because vectors aren't human readable, they're typically omitted in a query response that's rendered on a search page. However, if you're using vectors in downstream processing, such as passing query results to a model or process that consumes vector content, you should keep `stored` set to true and choose a different technique for minimizing vector size.

- PUT https://[service-name].search.windows.net/indexes/[index-name]?api-version=2024-05-01-previewΓÇ»

- ΓÇ» "name": "myindex",

- ΓÇ» ΓÇ» ΓÇ» "name": "myvector",

-+ Applies to [vector fields](/rest/api/searchservice/supported-data-types#edm-data-types-for-vector-fields) only.

-+ Affects storage on disk, not memory, and it has no effect on queries. Query execution uses a separate vector index that's unaffected by the `stored` property.

-+ The `stored` property is set during index creation on vector fields and is irreversible. If you want retrievable content later, you must drop and rebuild the index, or create and load a new field that has the new attribution.

-+ Defaults are `stored` set to true and `retrievable` set to false. In a default configuration, a retrievable copy is stored, but it's not automatically returned in results. When `stored` is true, you can toggle `retrievable` between true and false at any time without having to rebuild an index. When `stored` is false, `retrievable` must be false and can't be changed.

-## Example index with vectorCompression, data types, and stored property

-Here's a composite example of a search index that specifies narrow data types, reduced storage, and vector compression.

-+ "HotelNameVector" provides a narrow data type example, recasting the original `Float32` values to `Float16`, expressed as `Collection(Edm.Half)` in the search index.

-+ "HotelNameVector" also has `stored` set to false. Extra embeddings used in a query response are not stored. When `stored` is false, `retrievable` must also be false.

-+ "DescriptionVector" provides an example of vector compression. Vector compression is defined in the index, referenced in a profile, and then assigned to a vector field. "DescriptionVector" also has `stored` set to false.

-```json

-### Create a new index

-POST {{baseUrl}}/indexes?api-version=2024-03-01-preview HTTP/1.1

- Content-Type: application/json

- api-key: {{apiKey}}

-{

- "name": "hotels-vector-quickstart",

- "fields": [

- {

- "name": "HotelId",

- "type": "Edm.String",

- "searchable": false,

- "filterable": true,

- "retrievable": true,

- "sortable": false,

- "facetable": false,

- "key": true

- },

- {

- "name": "HotelName",

- "type": "Edm.String",

- "searchable": true,

- "filterable": false,

- "retrievable": true,

- "sortable": true,

- "facetable": false

- },

- {

- "name": "HotelNameVector",

- "type": "Collection(Edm.Half)",

- "searchable": true,

- "retrievable": false,

- "dimensions": 1536,

- "stored": false,

- "vectorSearchProfile": "my-vector-profile-no-compression"

- },

- {

- "name": "Description",

- "type": "Edm.String",

- "searchable": true,

- "filterable": false,

- "retrievable": false,

- "sortable": false,

- "facetable": false

- },

- {

- "name": "DescriptionVector",

- "type": "Collection(Edm.Single)",

- "searchable": true,

- "retrievable": false,

- "dimensions": 1536,

- "stored": false,

- "vectorSearchProfile": "my-vector-profile-with-compression"

- },

- {

- "name": "Category",

- "type": "Edm.String",

- "searchable": true,

- "filterable": true,

- "retrievable": true,

- "sortable": true,

- "facetable": true

- },

- {

- "name": "Tags",

- "type": "Collection(Edm.String)",

- "searchable": true,

- "filterable": true,

- "retrievable": true,

- "sortable": false,

- "facetable": true

- },

- {

- "name": "Address",

- "type": "Edm.ComplexType",

- "fields": [

- {

- "name": "City", "type": "Edm.String",

- "searchable": true, "filterable": true, "retrievable": true, "sortable": true, "facetable": true

- },

- {

- "name": "StateProvince", "type": "Edm.String",

- "searchable": true, "filterable": true, "retrievable": true, "sortable": true, "facetable": true

- }

- ]

- },

- {

- "name": "Location",

- "type": "Edm.GeographyPoint",

- "searchable": false,

- "filterable": true,

- "retrievable": true,

- "sortable": true,

- "facetable": false

- }

- ],

-"vectorSearch": {

- "compressions": [

- {

- "name": "my-scalar-quantization",

- "kind": "scalarQuantization",

- "rerankWithOriginalVectors": true,

- "defaultOversampling": 10.0,

- "scalarQuantizationParameters": {

- "quantizedDataType": "int8"

- }

- }

- ],

- "algorithms": [

- {

- "name": "my-hnsw-vector-config-1",

- "kind": "hnsw",

- "hnswParameters":

- {

- "m": 4,

- "efConstruction": 400,

- "efSearch": 500,

- "metric": "cosine"

- }

- },

- {

- "name": "my-hnsw-vector-config-2",

- "kind": "hnsw",

- "hnswParameters":

- {

- "m": 4,

- "metric": "euclidean"

- }

- },

- {

- "name": "my-eknn-vector-config",

- "kind": "exhaustiveKnn",

- "exhaustiveKnnParameters":

- {

- "metric": "cosine"

- }

- }

- ],

- "profiles": [

- {

- "name": "my-vector-profile-with-compression",

- "compression": "my-scalar-quantization",

- "algorithm": "my-hnsw-vector-config-1",

- "vectorizer": null

- },

- {

- "name": "my-vector-profile-no-compression",

- "compression": null,

- "algorithm": "my-eknn-vector-config",

- "vectorizer": null

- }

- ]

-},

- "semantic": {

- "configurations": [

- {

- "name": "my-semantic-config",

- "prioritizedFields": {

- "titleField": {

- "fieldName": "HotelName"

- },

- "prioritizedContentFields": [

- { "fieldName": "Description" }

- ],

- "prioritizedKeywordsFields": [

- { "fieldName": "Tags" }

- ]

- }

- }

- ]

- }

-}

-The query syntax in this example applies to vector fields using built-in scalar quantization. By default, vector fields that use scalar quantization also use `rerankWithOriginalVectors` and `defaultOversampling` to mitigate the effects of a smaller vector index. Those settings are [specified in the search index](#add-compression-settings-and-set-optional-properties).

-On the query, you can override the oversampling default value. For example, if `defaultOversampling` is 10.0, you can change it to something else in the query request.

-POST https://[service-name].search.windows.net/indexes/[index-name]/docs/search?api-version=2024-05-01-Preview  

-+ Applies to vector fields that undergo vector compression, per the vector profile assignment.

-+ Overrides the `defaultOversampling` value or introduces oversampling at query time, even if the index's compression configuration didn't specify oversampling or reranking options.

-+ [Get started with REST](search-get-started-rest.md)

-+ [Supported data types](/rest/api/searchservice/supported-data-types)

-+ [Search REST APIs](/rest/api/searchservice/)

-> [!IMPORTANT]

-> This feature is in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). The [2023-10-01-preview REST API](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2023-10-01-preview&preserve-view=true) and [2024-03-01-preview REST API](/rest/api/searchservice/operation-groups?view=rest-searchservice-2024-03-01-preview&preserve-view=true) support the [AzureOpenAIEmbedding vectorizer](vector-search-vectorizer-azure-open-ai.md) and [custom vectorizer](vector-search-vectorizer-custom-web-api.md). [2024-05-01-preview REST API](/rest/api/searchservice/operation-groups?view=rest-searchservice-2024-05-01-preview&preserve-view=true) adds support for the [Azure AI Vision vectorizer](vector-search-vectorizer-ai-services-vision.md) and the [Azure AI Studio model catalog vectorizer](vector-search-vectorizer-azure-machine-learning-ai-studio-catalog.md).

-To add a vectorizer to search index, you can use the index designer in Azure portal, or call the [Create or Update Index 2024-05-01-preview](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) REST API, or use any Azure beta SDK package that's updated to provide this feature.

-+ A deployed embedding model, such as **text-embedding-ada-002** on Azure OpenAI. It's used to vectorize a query. It must be identical to the model used to generate the embeddings in your index. You can also use [models deployed from the Azure AI Studio model catalog](vector-search-integrated-vectorization-ai-studio.md) or an [Azure AI Vision model](/azure/ai-services/computer-vision/concept-image-retrieval).

-1. Use [Create or Update Index (preview)](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2023-10-01-preview&preserve-view=true) to add `vectorizers` to a search index.

-1. Paste in a [vector query request](vector-search-how-to-query.md). Be sure to use a preview REST API version.

- POST {{baseUrl}}/indexes/vector-nasa-ebook-txt/docs/search?api-version=2023-10-01-preview HTTP/1.1

-If you are setting up an Azure OpenAI vectorizer, consider the same [best practices](cognitive-search-skill-azure-openai-embedding.md#best-practices) that we recommend for the Azure OpenAI embedding skill.

-In Azure AI Search, a *vector store* has an index schema that defines vector and nonvector fields, a vector configuration for algorithms that create the embedding space, and settings on vector field definitions that are used in query requests. The [Create or Update Index](/rest/api/searchservice/indexes/create-or-update) API creates the vector store.

-> + Define a schema with vector algorithms for indexing and search

-> + Add vector fields

-> + Load prevectorized data [as a separate step](#load-vector-data-for-indexing), or use [integrated vectorization (preview)](vector-search-integrated-vectorization.md) for data chunking and encoding during indexing

-This article explains the workflow and uses REST to illustrate each step. Each recent version of the REST API adds new functionality. Once you understand the basic workflow and what each API version provides, continue with the Azure SDK code samples in the [azure-search-vector-samples](https://github.com/Azure/azure-search-vector-samples) repository for guidance on using these features in test and production code.

-+ Azure AI Search, in any region and on any tier. Most existing services support vector search. For services created before January 2019, there's a small subset that can't create a vector index. In this situation, a new service must be created.

-+ Pre-existing vector embeddings in your source documents if you're using the generally available version of the Azure SDKs and REST APIs. For more information, see [Generate embeddings](vector-search-how-to-generate-embeddings.md). An alternative is [integrated vectorization (preview)](vector-search-integrated-vectorization.md).

-+ You should know the dimensions limit of the model used to create the embeddings and how similarity is computed. In Azure OpenAI, for **text-embedding-ada-002**, the length of the numerical vector is 1536. Similarity is computed using `cosine`. Valid values are 2 through 3072 dimensions.

- Vector fields contain numeric data generated by embedding models, one embedding per field. We recommend the embedding models in [Azure OpenAI](https://aka.ms/oai/access), such as **text-embedding-ada-002** for text documents or the [Image Retrieval REST API](/rest/api/computervision/2023-02-01-preview/image-retrieval/vectorize-image) for images. Only index top-level vector fields are supported: Vector subfields aren't currently supported.

-A vector configuration specifies the [vector search algorithm](vector-search-ranking.md) and parameters used during indexing to create "nearest neighbor" information among the vector nodes:

-Looking for preview-to-stable version migration guidance? See [Upgrade REST APIs](search-api-migration.md) for steps.

-### [**2023-11-01**](#tab/config-2023-11-01)

-[**2023-11-01**](/rest/api/searchservice/search-service-api-versions#2023-11-01) is generally available. It supports a vector configuration having:

-+ `vectorSearch.algorithms` with parameters for indexing and scoring.

-+ `vectorSearch.profiles` for multiple combinations of algorithm configurations.

-Be sure to have a strategy for [vectorizing your content](vector-search-how-to-generate-embeddings.md). The stable version doesn't provide [skills](cognitive-search-predefined-skills.md) or [vectorizers](vector-search-how-to-configure-vectorizer.md) for built-in encoding.

- "name": "my-hnsw-config-1",

- "name": "my-hnsw-config-2",

- "metric": "cosine"

- "name": "my-eknn-config",

- "metric": "cosine"

- "name": "my-default-vector-profile",

- "algorithm": "my-hnsw-config-2"

- + Name of the configuration. The name must be unique within the index.

- + `profiles` add a layer of abstraction for accommodating richer definitions. A profile is defined in `vectorSearch`, and then referenced by name on each vector field.

- + `"hnsw"` and `"exhaustiveKnn"` are the Approximate Nearest Neighbors (ANN) algorithms used to organize vector content during indexing.

- + `"m"` (bi-directional link count) default is 4. The range is 4 to 10. Lower values should return less noise in the results.

- + `"efConstruction"` default is 400. The range is 100 to 1,000. It's the number of nearest neighbors used during indexing.

- + `"efSearch"` default is 500. The range is 100 to 1,000. It's the number of nearest neighbors used during search.

- + `"metric"` should be "cosine" if you're using Azure OpenAI, otherwise use the similarity metric associated with the embedding model you're using. Supported values are `cosine`, `dotProduct`, `euclidean`.

-### [**2024-05-01-Preview**](#tab/config-2024-05-01-Preview)

-[**2024-05-01-Preview**](/rest/api/searchservice/search-service-api-versions#2024-05-01-preview) is the newest version. It adds more encoding options, but the vector search configuration (`vectorSearch` structure) is mostly identical to 2024-03-01-preview.

-+ Adds `hamming` distance as a metric for nearest neighbor search over binary data. For more information, see [Index binary data for vector search](vector-search-how-to-index-binary-data.md).

-+ Expands integrated vectorization with more embedding model choices. To benefit from this capability, you must take a dependency on an indexer and skillset. See [Load vector data](#load-vector-data-for-indexing) and the **Pull APIs** section for a list of the new embedding skills.

-+ Use the [Create or Update Index Preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) to create the index.

-For more information about new preview features, see [What's new in Azure AI Search](whats-new.md).

-### [**2024-03-01-Preview**](#tab/config-2024-03-01-Preview)

-[**2024-03-01-Preview**](/rest/api/searchservice/search-service-api-versions#2024-03-01-preview) adds [scalar quantization](vector-search-how-to-configure-vectorizer.md) and storage-saving options.

-+ Adds `vectorSearch.compressions` with properties for scalar quantization and oversampling.

-+ Adds narrow data types: `Float16`, `Int16`, and `Int8` for more efficient vector storage.

-+ Inclusive of 2023-10-01-preview [integrated vectorization](vector-search-integrated-vectorization.md).

-1. Use the [Create or Update Index Preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-03-01-preview&preserve-view=true) to create the index.

- "name": "my-hnsw-config-1",

- "name": "my-hnsw-config-2",

- "metric": "cosine"

- "name": "my-eknn-config",

- "metric": "cosine"

- "name": "my-default-vector-profile",

- "algorithm": "my-hnsw-config-2"

- + `rerankWithOriginalVectors` uses the original, uncompressed vectors to recalculate similarity and rerank the top results returned by the initial search query. The uncompressed vectors exist in the search index even if `stored` is false. This property is optional. Default is true.

- + `defaultOversampling` considers a broader set of potential results to offset the reduction in information from quantization. The formula for potential results consists of the `k` in the query, with an oversampling multiplier. For example, if the query specifies a `k` of 5, and oversampling is 20, then the query effectively requests 100 documents for use in reranking, using the original uncompressed vector for that purpose. Only the top `k` reranked results are returned. This property is optional. Default is 4.

- + `quantizedDataType` must be set to `int8`. This is the only primitive data type supported at this time. This property is optional. Default is `int8`.

-### [**2023-10-01-Preview**](#tab/config-2023-10-01-Preview)

-[**2023-10-01-Preview**](/rest/api/searchservice/search-service-api-versions#2023-10-01-Preview) adds [internal vectorization](vector-search-how-to-configure-vectorizer.md), but the vector search configuration (`vectorSearch` structure) is mostly identical to 2023-11-01 version.

-1. Use the [Create or Update Index Preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2023-10-01-preview&preserve-view=true) to create the index.

-1. Add a `vectorSearch` section in the index that specifies the search algorithms used to create the embedding space.

- ```json

- "vectorSearch": {

- "algorithms": [

- {

- "name": "my-hnsw-config-1",

- "kind": "hnsw",

- "hnswParameters": {

- "m": 4,

- "efConstruction": 400,

- "efSearch": 500,

- "metric": "cosine"

- }

- },

- {

- "name": "my-hnsw-config-2",

- "kind": "hnsw",

- "hnswParameters": {

- "m": 8,

- "efConstruction": 800,

- "efSearch": 800,

- "metric": "cosine"

- }

- },

- {

- "name": "my-eknn-config",

- "kind": "exhaustiveKnn",

- "exhaustiveKnnParameters": {

- "metric": "cosine"

- }

- }

- ],

- "profiles": [

- {

- "name": "my-default-vector-profile",

- "algorithm": "my-hnsw-config-2"

- }

- ]

- }

- ```

- **Key points**:

- + Name of the configuration. The name must be unique within the index.

- + `profiles` are new in this preview. They add a layer of abstraction for accommodating richer definitions. A profile is defined in `vectorSearch`, and then as a property on each vector field.

- + `hnsw` and `"exhaustiveKnn"` are the Approximate Nearest Neighbors (ANN) algorithms used to organize vector content during indexing.

- + `m` (bi-directional link count) default is 4. The range is 4 to 10. Lower values should return less noise in the results.

- + `efConstruction` default is 400. The range is 100 to 1,000. It's the number of nearest neighbors used during indexing.

- + `efSearch` default is 500. The range is 100 to 1,000. It's the number of nearest neighbors used during search.

- + `metric` should be "cosine" if you're using Azure OpenAI, otherwise use the similarity metric associated with the embedding model you're using. Supported values are `cosine`, `dotProduct`, `euclidean`.

-### [**2023-07-01-Preview**](#tab/rest-add-config)

-> [!IMPORTANT]

-> 2023-07-01-Preview was the first REST API version to support vectors. It uses outmoded structures that have been replaced in newer previews. We recommend [migrating to a newer REST API](search-api-migration.md).

-This preview added:

-+ `vectorSearch.algorithmConfigurations` for specifying the HNSW algorithm.

-+ `hnsw` nearest neighbor algorithm for indexing vector content.

-1. Use the [Create or Update Index REST API](/rest/api/searchservice/preview-api/create-or-update-index) to create the index.

-1. Add a `vectorSearch` section in the index that specifies the search algorithm used to create the embedding space.

- ```json

- "vectorSearch": {

- "algorithmConfigurations": [

- {

- "name": "vectorConfig",

- "kind": "hnsw",

- "hnswParameters": {

- "m": 4,

- "efConstruction": 400,

- "efSearch": 500,

- "metric": "cosine"

- }

- }

- ]

- }

- ```

- **Key points**:

- + Name of the configuration. The name must be unique within the index.

- + `hnsw` is the Approximate Nearest Neighbors (ANN) algorithm used to create the proximity graph during indexing. Only Hierarchical Navigable Small World (HNSW) is supported in this API version.

- + `m` (bi-directional link count) default is 4. The range is 4 to 10. Lower values should return less noise in the results.

- + `efConstruction` default is 400. The range is 100 to 1,000. It's the number of nearest neighbors used during indexing.

- + `efSearch` default is 500. The range is 100 to 1,000. It's the number of nearest neighbors used during search.

- + `metric` should be "cosine" if you're using Azure OpenAI, otherwise use the similarity metric associated with the embedding model you're using. Supported values are `cosine`, `dotProduct`, `euclidean`.

-### [**2023-11-01**](#tab/rest-2023-11-01)

-Use this version if you want generally available features only.

- + `type` must be `Collection(Edm.Single)` in this API version.

- PUT https://my-search-service.search.windows.net/indexes/my-index?api-version=2023-11-01&allowIndexDowntime=true

- "vectorSearchProfile": "my-default-vector-profile"

- "retrievable": true,

- "vectorSearchProfile": "my-default-vector-profile"

- "name": "my-hnsw-config-1",

- "name": "my-default-vector-profile",

- "algorithm": "my-hnsw-config-1"

-### [**2024-05-01-Preview**](#tab/rest-2024-05-01-Preview)

-Vector field definitions are the same as 2024-03-01-preview, with the exception of a new binary data type. For more information, see [Index binary data for vector search](vector-search-how-to-index-binary-data.md).

-Use the [Create or Update Index Preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) to define the fields collection of an index.

-### [**2024-03-01-Preview**](#tab/rest-2024-03-01-Preview)

-This API version builds on 2023-10-01-preview by adding support for [narrow data types](/rest/api/searchservice/supported-data-types#edm-data-types-for-vector-fields) and [scalar quantization](vector-search-how-to-configure-compression-storage.md).

-1. Use the [Create or Update Index Preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-03-01-preview&preserve-view=true) to define the fields collection of an index.

- PUT https://my-search-service.search.windows.net/indexes/my-index?api-version=2024-03-01-Preview&allowIndexDowntime=true

- "vectorSearchProfile": "my-default-vector-profile"

- "vectorSearchProfile": "my-default-vector-profile"

- "vectorSearchProfile": "my-default-vector-profile"

- },

- ],

- "vectorSearch": {

- "algorithms": [

- {

- "name": "my-hnsw-config-1",

- "kind": "hnsw",

- "hnswParameters": {

- "m": 4,

- "efConstruction": 400,

- "efSearch": 500,

- "metric": "cosine"

- }

- }

- ],

- "profiles": [

- {

- "name": "my-default-vector-profile",

- "algorithm": "my-hnsw-config-1"

- }

- ]

- }

- }

- ```

-### [**2023-10-01-Preview**](#tab/rest-2023-10-01-Preview)

-In the following REST API example, "title" and "content" contain textual content used in full text search and semantic ranking, while "titleVector" and "contentVector" contain vector data. In this API version, you can use indexers and a skillset to populate vector field using [integrated vectorization](vector-search-integrated-vectorization.md). The index definition doesn't change, but you can add indexers and skills to your solution to populate the fields.

-1. Use the [Create or Update Index Preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2023-10-01-preview&preserve-view=true) to define the fields collection of an index.

-1. Add vector fields to the fields collection. You can store one generated embedding per document field. For each vector field:

- + `type` must be `Collection(Edm.Single)`.

- + `dimensions` is the number of dimensions generated by the embedding model. For text-embedding-ada-002, it's 1536.

- + `vectorSearchProfile` is the name of a profile defined elsewhere in the index.

- + `searchable` must be true.

- + `retrievable` can be true or false. True returns the raw vectors (1536 of them) as plain text and consumes storage space. Set to true if you're passing a vector result to a downstream app.

- + `filterable`, `facetable`, `sortable` must be false.

-1. Add filterable nonvector fields to the collection, such as "title" with `filterable` set to true, if you want to invoke [prefiltering or postfiltering](vector-search-filters.md) on the [vector query](vector-search-how-to-query.md

-1. Add other fields that define the substance and structure of the textual content you're indexing. At a minimum, you need a document key.

- You should also add fields that are useful in the query or in its response. The following example shows vector fields for title and content ("titleVector", "contentVector") that are equivalent to vectors. It also provides fields for equivalent textual content ("title", "content") useful for sorting, filtering, and reading in a search result.

- The following example shows the fields collection:

- ```http

- PUT https://my-search-service.search.windows.net/indexes/my-index?api-version=2023-10-01-Preview&allowIndexDowntime=true

- Content-Type: application/json

- api-key: {{admin-api-key}}

- {

- "name": "{{index-name}}",

- "fields": [

- {

- "name": "id",

- "type": "Edm.String",

- "key": true,

- "filterable": true

- },

- {

- "name": "title",

- "type": "Edm.String",

- "searchable": true,

- "filterable": true,

- "sortable": true,

- "retrievable": true

- },

- {

- "name": "titleVector",

- "type": "Collection(Edm.Single)",

- "searchable": true,

- "retrievable": true,

- "vectorSearchProfile": "my-default-vector-profile"

- "name": "content",

- "type": "Edm.String",

- "retrievable": true

- },

- {

- "name": "contentVector",

- "type": "Collection(Edm.Single)",

- "searchable": true,

- "retrievable": true,

- "vectorSearchProfile": "my-default-vector-profile"

- "name": "my-hnsw-config-1",

- "name": "my-default-vector-profile",

- "algorithm": "my-hnsw-config-1"

- ```

-### [**2023-07-01-Preview**](#tab/rest-add-field)

-> [!IMPORTANT]

-> The vector field definitions for this version are obsolete in later versions. We recommend [migrating to a newer REST API](search-api-migration.md).

-[**2023-07-01-Preview**](/rest/api/searchservice/index-preview) was the first REST API version to support vector scenarios.

-In the following REST API example, "title" and "content" contain textual content used in full text search and semantic ranking, while "titleVector" and "contentVector" contain vector data that was generated externally.

-1. Use the [Create or Update Index Preview REST API](/rest/api/searchservice/preview-api/create-or-update-index) to define the fields collection of an index.

-1. Add vector fields to the fields collection. You can store one generated embedding per document field. For each vector field:

- + Assign the `Collection(Edm.Single)` data type.

- + Provide the name of the vector search algorithm configuration.

- + Provide the number of dimensions generated by the embedding model.

- + Set attributes:

- + "searchable" must be "true".

- + "retrievable" set to "true" allows you to display the raw vectors (for example, as a verification step), but doing so increases storage. Set to "false" if you don't need to return raw vectors. You don't need to return vectors for a query, but if you're passing a vector result to a downstream app then set "retrievable" to "true".

- + "filterable", "facetable", "sortable" attributes must be "false". Don't set them to "true" because those behaviors don't apply within the context of vector fields and the request will fail.

-1. Add other fields that define the substance and structure of the textual content you're indexing. At a minimum, you need a document key.

- You should also add fields that are useful in the query or in its response. The following example shows vector fields for title and content ("titleVector", "contentVector") that are equivalent to vectors. It also provides fields for equivalent textual content ("title", "content") useful for sorting, filtering, and reading in a search result.

- An index definition with the described elements looks like this:

- ```http

- PUT https://my-search-service.search.windows.net/indexes/my-index?api-version=2023-07-01-Preview&allowIndexDowntime=true

- Content-Type: application/json

- api-key: {{admin-api-key}}

- {

- "name": "{{index-name}}",

- "fields": [

- {

- "name": "id",

- "type": "Edm.String",

- "key": true,

- "filterable": true

- },

- {

- "name": "title",

- "type": "Edm.String",

- "searchable": true,

- "filterable": true,

- "sortable": true,

- "retrievable": true

- },

- {

- "name": "titleVector",

- "type": "Collection(Edm.Single)",

- "searchable": true,

- "retrievable": true,

- "dimensions": 1536,

- "vectorSearchConfiguration": "vectorConfig"

- },

- {

- "name": "content",

- "type": "Edm.String",

- "searchable": true,

- "retrievable": true

- },

- {

- "name": "contentVector",

- "type": "Collection(Edm.Single)",

- "searchable": true,

- "retrievable": true,

- "dimensions": 1536,

- "vectorSearchConfiguration": "vectorConfig"

- }

- ],

- "vectorSearch": {

- "algorithmConfigurations": [

- {

- "name": "vectorConfig",

- "kind": "hnsw",

- "hnswParameters": {

- "m": 4,

- "efConstruction": 400,

- "efSearch": 500,

- "metric": "cosine"

- }

- }

- ]

- }

- }

- ```

-+ [2023-11-01](/rest/api/searchservice/documents/?view=rest-searchservice-2023-11-01&preserve-view=true)

-+ [2024-03-01-preview](/rest/api/searchservice/documents/?view=rest-searchservice-2024-03-01-preview&preserve-view=true)

-+ [2023-10-01-preview](/rest/api/searchservice/documents/?view=rest-searchservice-2023-10-01-preview&preserve-view=true)

-+ [2023-07-01-Preview](/rest/api/searchservice/preview-api/add-update-delete-documents)

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/index?api-version=2023-11-01

-Content-Type: application/json

-api-key: {{admin-api-key}}

-All of the newer preview releases use pull APIs (indexers and skillsets) for integrated vectorization during indexing and query time.

-Indexers can retrieve and index vector fields in source documents, assuming an index schema that meets vector field requirements and the preview REST API. Data sources provide the vectors in whatever format the data source supports (such as strings in JSON). The indexer assumes that fields typed as `Collection(Edm.Single)` contain vectors and will index that content as vector indexes.

-+ No changes to field mapping behavior or change detection for vectors. The behaviors for text indexing also apply to vectors.

-+ If vector data is sourced in files, we recommend a nondefault `parsingMode` such as `json`, `jsonLines`, or `csv` based on the shape of the data.

-+ For data sources, [Azure blob indexers](search-howto-indexing-azure-blob-storage.md) and [Azure Cosmos DB for NoSQL indexers](search-howto-index-cosmosdb.md) with one of the aforementioned parsingModes have been tested and confirmed to work.

- Azure SQL doesn't provide a way to store a collection natively as a single SQL column. A workaround hasn't been identified at this time.

-+ The dimensions of all vectors from the data source must be the same and match their index definition for the field they're mapping to. The indexer throws an error on any documents that donΓÇÖt match.

-Skills and vectorizers are used to generate embeddings. For vectorization during indexing, choose from the following skills:

-+ [AzureOpenAIEmbedding skill](cognitive-search-skill-azure-openai-embedding.md)

-+ [Azure AI Vision multimodal embeddings skill](cognitive-search-skill-vision-vectorize.md)

-+ [AML skill](cognitive-search-aml-skill.md) to generate embeddings for models hosted in the Azure AI Studio model catalog. See [How to implement integrated vectorization using models from Azure AI Studio](vector-search-integrated-vectorization-ai-studio.md) for details.

-For validation purposes, you can query the index using Search Explorer in Azure portal or a REST API call. Because Azure AI Search can't convert a vector to human-readable text, try to return fields from the same document that provide evidence of the match. For example, if the vector query targets the "titleVector" field, you could select "title" for the search results.

-You can use [Search Explorer](search-explorer.md) to query an index. Search explorer has two views: Query view (default) and JSON view.

-+ [Use the JSON view for vector queries](vector-search-how-to-query.md), pasting in a JSON definition of the vector query you want to execute.

-+ Use the default Query view for a quick confirmation that the index contains vectors. The query view is for full text search. Although you can't use it for vector queries, you can send an empty search (`search=*`) to check for content. The content of all fields, including vector fields, is returned as plain text.

-POST https://my-search-service.search.windows.net/indexes/my-index/docs/search?api-version=2023-11-01

-The standard guidance for updating an index is covered in [Drop and rebuild an index](search-howto-reindex.md).

-As a next step, we recommend [Query vector data in a search index](vector-search-how-to-query.md).

-Azure AI Search doesn't host vectorization models, so one of your challenges is creating embeddings for query inputs and outputs. You can use any embedding model, but this article assumes Azure OpenAI embeddings models. Demos in the [sample repository](https://github.com/Azure/azure-search-vector-samples/tree/main) tap the [similarity embedding models](/azure/ai-services/openai/concepts/models#embeddings-models) of Azure OpenAI.

-Dimension attributes have a minimum of 2 and a maximum of 4096 dimensions per vector field.

-> [!NOTE]

-> This article applies to the generally available version of [vector search](vector-search-overview.md), which assumes your application code calls an external resource such as Azure OpenAI for vectorization. A new feature called [integrated vectorization](vector-search-integrated-vectorization.md), currently in preview, offers embedded vectorization. Integrated vectorization takes a dependency on indexers, skillsets, and either the AzureOpenAIEmbedding skill or a custom skill that points to a model that executes externally from Azure AI Search.

-## How models are used

-+ Query inputs require that you submit user-provided input to an embedding model that quickly converts human readable text into a vector.

- + For example, you can use **text-embedding-ada-002** to generate text embeddings and [Image Retrieval REST API](/rest/api/computervision/2023-02-01-preview/image-retrieval/vectorize-image) for image embeddings.

- + To avoid [rate limiting](/azure/ai-services/openai/quotas-limits), you can implement retry logic in your workload. For the Python demo, we used [tenacity](https://pypi.org/project/tenacity/).

-+ Query outputs are any matching documents found in a search index. Your search index must have been previously loaded with documents having one or more vector fields with embeddings. Whatever model you used for indexing, use the same model for queries.

-1. [A region for the similarity embedding model](/azure/ai-services/openai/concepts/models#embeddings-models-1), currently in Europe and the United States.

-1. [A region for Azure AI Search](search-region-support.md).

-1. To support hybrid queries that include [semantic ranking](semantic-how-to-query-request.md), or if you want to try machine learning model integration using a [custom skill](cognitive-search-custom-skill-interface.md) in an [AI enrichment pipeline](cognitive-search-concept-intro.md), note the regions that provide those features.

-!pip install openai==0.28.1

-+ **Identify use cases:** Evaluate the specific use cases where embedding model integration for vector search features can add value to your search solution. This can include matching image content with text content, cross-lingual searches, or finding similar documents.

-+ **Optimize cost and performance**: Vector search can be resource-intensive and is subject to maximum limits, so consider only vectorizing the fields that contain semantic meaning.

-> [!IMPORTANT]

-> Binary data types are in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). The [2024-05-01-preview REST API](/rest/api/searchservice/operation-groups?view=rest-searchservice-2024-03-01-preview&preserve-view=true) and later preview APIs provide the new data type.

-Beginning with the 2024-05-01-preview REST API, Azure AI Search supports a packed binary type of `Collection(Edm.Byte)` for further reducing the storage and memory footprint of vector data. You can use this data type for output from models such as [Cohere's Embed v3 binary embedding models](https://cohere.com/blog/introducing-embed-v3).

-+ No scalar compression or integrated vectorization support.

-1. To add a binary field to an index, set up a [`Create or Update Index`](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true) request using the **2024-05-01-preview REST API** or the Azure portal.

-1. Add one or more vector profiles that specify the algorithm. You can't specify scalar compression or a vectorizer in this preview.

-1. Add a field to the fields collection and give it name.

-1. Set data type to `Collection(Edm.Byte)`.

-1. Set `vectorEncoding` to `packedBit` for binary encoding.

-1. Set `dimensions` to `1024`. Specify the original (unpacked) vector dimension.

-1. Set `vectorSearchProfile` to a profile you defined in the previous step.

-The minimum definition of a fields collection should look similar to the following example:

-ΓÇ» ΓÇ» {

-ΓÇ» ΓÇ» ΓÇ» "name": "Id",

-ΓÇ» ΓÇ» ΓÇ» "type": "Edm.String",

-ΓÇ» ΓÇ» ΓÇ» "key": true,

-ΓÇ» ΓÇ» ΓÇ» "searchable": true

-ΓÇ» ΓÇ» },

-ΓÇ» ΓÇ» ΓÇ» "vectorSearchProfile": "myHnswProfile"

-ΓÇ» ΓÇ» }

-> + [Query with integrated vectorization (preview)](#query-with-integrated-vectorization-preview)

-> + [Set vector weights (preview)](#vector-weighting-preview)

-You can also use [Search explorer](search-explorer.md) in the Azure portal if you [configure a vectorizer](vector-search-how-to-configure-vectorizer.md) that converts strings into embeddings.

-A second approach is [using integrated vectorization](#query-with-integrated-vectorization-preview), currently in public preview, to have Azure AI Search handle your query vectorization inputs and outputs.

-### [**2023-11-01**](#tab/query-2023-11-01)

-[**2023-11-01**](/rest/api/searchservice/search-service-api-versions#2023-11-01) is the stable REST API version for [Search POST](/rest/api/searchservice/documents/search-post). This version supports:

-+ `kind` set to `vector` specifies that the query is a vector array.

-+ `vector` is query (a vector representation of text or an image).

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-11-01

-### [**2023-10-01-Preview**](#tab/query-2023-10-01-Preview)

-[**2023-10-01-Preview**](/rest/api/searchservice/search-service-api-versions#2023-10-01-Preview) is the preview API version for [Search POST](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2023-10-01-preview&tabs=HTTP&preserve-view=true). It supports the same vector query syntax as 2023-11-01 (shown in this tab), but also adds [integrated vectorization of text queries](#query-with-integrated-vectorization-preview).

-+ `vectorQueries` is the construct for vector search.

-+ `kind` set to `vector` specifies that the query is a vector array.

-+ `vector` is query (a vector representation of text or an image).

-+ `exhaustive` (optional) invokes exhaustive KNN at query time, even if the field is indexed for HNSW.

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-10-01-Preview

- "exhaustive": true,

- "k": 5

-}

-```

-### [**2023-07-01-Preview**](#tab/query-vector-query)

-> [!IMPORTANT]

-> The vector query syntax for this version is obsolete in later versions. We recommend [upgrading to the latest REST API](search-api-migration.md).

-[**2023-07-01-Preview**](/rest/api/searchservice/index-preview) first introduced vector query support to [Search Documents](/rest/api/searchservice/preview-api/search-documents). This version added:

-+ `vectors` for specifying a vector to search for, vector fields to search in, and the k-number of nearest neighbors to return.

-In the following example, the vector is a representation of this query string: "what Azure services support full text search". The query targets the `contentVector` field. The query returns `k` results. The actual vector has 1536 embeddings. It's trimmed in this example for readability.

-```http

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-10-01-Preview

-Content-Type: application/json

-api-key: {{admin-api-key}}

-{

- "vectors": [{

- "value": [

- -0.009154141,

- 0.018708462,

- . . .

- -0.02178128,

- -0.00086512347

- ],

- "fields": "contentVector",

- "k": 5

- }],

- "select": "title, content, category"

-}

-The response includes five matches, and each result provides a search score, title, content, and category. In a similarity search, the response always includes `k` matches, even if the similarity is weak. For indexes that have fewer than `k` documents, only those number of documents are returned.

-Notice that `select` returns textual fields from the index. Although the vector field is `retrievable` in this example, its content isn't meaningful as a search result, so it's often excluded in the results.

-Use Search explorer to formulate a vector query. Search explorer has a **Query view** and **JSON View**. You must use **JSON view**, and you must formulate the vector query in JSON. The search bar in **Query view** is for full text search and treats any vector input as plain text (it doesn't execute a vector search).

-1. Sign in to Azure portal and find your search service.

-1. By default, the search API is **2023-10-01-Preview**. You can choose another API version.

-### [**2023-11-01**](#tab/filter-2023-11-01)

-[**2023-11-01**](/rest/api/searchservice/search-service-api-versions#2023-11-01) is the stable version for this API. It has:

-The filter criteria are applied to a filterable text field (`category `in this example) before the search engine executes the vector query.

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-11-01

-### [**2023-10-01-Preview**](#tab/filter-2023-10-01-Preview)

-[**2023-10-01-Preview**](/rest/api/searchservice/search-service-api-versions#2023-10-01-Preview) introduces filter options. This version adds:

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-10-01-Preview

-### [**2023-07-01-Preview**](#tab/filter-2023-07-01-Preview)

-[**2023-07-01-Preview**](/rest/api/searchservice/index-preview) supports post-filtering only.

-In the following example, the vector is a representation of this query string: "what Azure services support full text search". The query targets the `contentVector` field. The actual vector has 1536 embeddings, so it's trimmed in this example for readability.

-In this API version, there's no prefilter support or `vectorFilterMode` parameter. The filter criteria are applied after the search engine executes the vector query. The set of `"k"` nearest neighbors is retrieved, and then combined with the set of filtered results. As such, the value of `"k"` predetermines the surface over which the filter is applied. For `"k": 10`, the filter is applied to 10 most similar documents. For `"k": 100`, the filter iterates over 100 documents (assuming the index contains 100 documents that are sufficiently similar to the query).

-```http

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-07-01-Preview

-Content-Type: application/json

-api-key: {{admin-api-key}}

-{

- "vectors": [

- {

- "value": [

- -0.009154141,

- 0.018708462,

- . . .

- -0.02178128,

- -0.00086512347

- ],

- "fields": "contentVector",

- "k": 10

- },

- ],

- "select": "title, content, category",

- "filter": "category eq 'Databases'"

-}

-```

-POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2023-11-01

-## Query with integrated vectorization (preview)

-This section shows a vector query that invokes the new [integrated vectorization](vector-search-integrated-vectorization.md) preview feature that converts a text query into a vector. Use [**2023-10-01-Preview** REST API](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2023-10-01-preview&preserve-view=true) and newer preview REST APIs or an updated beta Azure SDK package.

-A prerequisite is a search index having a [vectorizer configured and assigned](vector-search-how-to-configure-vectorizer.md) to a vector field. The vectorizer provides connection information to an embedding model used at query time. Check the index definition for a vectorizers specification.

-Queries provide text strings instead of vectors:

-+ `kind` must be set to `text` .

-+ `text` must have a text string. It's passed to the vectorizer assigned to the vector field.

-+ `fields` is the vector field to search.

-POST https://{{search-service}}.search.windows.net/indexes/{{index}}/docs/search?api-version=2024-05-01-preview

-POST https://{{search-service}}.search.windows.net/indexes/{{index}}/docs/search?api-version=2024-05-01-preview

-> Vectorizers are used during indexing and querying. If you don't need data chunking and vectorization in the index, you can skip steps like creating an indexer, skillset, and data source. In this scenario, the vectorizer is used only at query time to convert a text string to an embedding.

-Because nearest neighbor search always returns the requested `k` neighbors, it's possible to get multiple low scoring matches as part of meeting the `k` number requirement on search results.

-Using the 2024-05-01-preview REST APIs, you can now add a `threshold` query parameter to exclude low-scoring search results based on a minimum score. Filtering occurs before [fusing results](hybrid-search-ranking.md) from different recall sets.

-POST https://[service-name].search.windows.net/indexes/[index-name]/docs/search?api-version=2024-05-01-Preview

-For more information, see [Set maxTextRecallSize - Create a hybrid query](hybrid-search-how-to-query.md#set-maxtextrecallsize-and-countandfacetmode-preview).

-## Vector weighting (preview)

-Add a `weight` query parameter to specify the relative weight of each vector query included in search operations. This value is used when combining the results of multiple ranking lists produced by two or more vector queries in the same request, or from the vector portion of a hybrid query.

-The default is 1.0 and the value must be a positive number larger than zero.

-Weights are used when calculating the [reciprocal rank fusion](hybrid-search-ranking.md#weighted-scores) scores of each document. The calculation is multiplier of the `weight` value against the rank score of the document within its respective result set.

-The following example is a hybrid query with two vector query strings and one text string. Weights are assigned to the vector queries. The first query is 0.5 or half the weight, reducing its importance in the request. The second vectory query is twice as important.

-Text queries have no weight parameters, but you can increase or decrease their importance by setting [maxTextRecallSize](hybrid-search-how-to-query.md#set-maxtextrecallsize-and-countandfacetmode-preview).

-```http

-POST https://[service-name].search.windows.net/indexes/[index-name]/docs/search?api-version=2024-05-01-Preview

- {

- ΓÇ» "vectorQueries": [

- ΓÇ» ΓÇ» {

- ΓÇ» ΓÇ» ΓÇ» "kind": "vector",

- ΓÇ» ΓÇ» ΓÇ» "vector": [1.0, 2.0, 3.0],

- ΓÇ» ΓÇ» ΓÇ» "fields": "my_first_vector_field",

- ΓÇ» ΓÇ» ΓÇ» "k": 10,

- ΓÇ» ΓÇ» ΓÇ» "weight": 0.5

- ΓÇ» ΓÇ» },

- ΓÇ» ΓÇ» {

- ΓÇ» ΓÇ» ΓÇ» "kind": "vector",

- ΓÇ» ΓÇ» ΓÇ» "vector": [4.0, 5.0, 6.0],

- ΓÇ» ΓÇ» ΓÇ» "fields": "my_second_vector_field",

- ΓÇ» ΓÇ» ΓÇ» "k": 10,

- ΓÇ» ΓÇ» ΓÇ» "weight": 2.0

- ΓÇ» ΓÇ» }

- ΓÇ» ],

- ΓÇ» "search": "hello world"

- }

-```

-> [Vector quantization and storage configuration](vector-search-how-to-configure-compression-storage.md) is now in preview. Use capabilities like narrow data types, scalar quantization, and elimination of redundant storage to stay under vector quota and storage quota.

-Data plane REST APIs (version 2023-10-01-preview, 2023-11-01, and all newer APIs) provide vector usage statistics:

-GET {{baseUrl}}/servicestats?api-version=2023-11-01 HTTP/1.1

-GET {{baseUrl}}/indexes/vector-healthplan-idx/stats?api-version=2023-11-01 HTTP/1.1

-> [!IMPORTANT]

-> Integrated data chunking and vectorization is in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). The [2023-10-01-Preview REST API](/rest/api/searchservice/skillsets/create-or-update?view=rest-searchservice-2023-10-01-preview&preserve-view=true) and all newer preview REST APIs provide this feature.

-+ [An indexer](search-indexer-overview.md), which retrieves raw data from a supported data source and serves as the pipeline engine.

- + [AzureOpenAIEmbedding skill](cognitive-search-skill-azure-openai-embedding.md), attached to text-embedding-ada-002 on Azure OpenAI.

- + Alternatively, you can use a [custom skill](cognitive-search-custom-skill-web-api.md) in place of AzureOpenAIEmbdding that points to another embedding model on Azure or on another side.

-+ [A vector index](search-what-is-an-index.md) to receive the chunked and vectorized content.

-+ [A vectorizer](vector-search-how-to-configure-vectorizer.md), defined in the index schema, assigned to a vector field, and used automatically at query time to convert a text query to a vector.

-1. [Create a skillset](cognitive-search-defining-skillset.md) that calls [Text Split skill](cognitive-search-skill-textsplit.md) for chunking and [AzureOpenAIEmbeddingModel](cognitive-search-skill-azure-openai-embedding.md) or a custom skill to vectorize the chunks.

-1. [Create an indexer](search-howto-create-indexers.md) to drive everything, from data retrieval, to skillset execution, through indexing.

-Finally, the following features aren't currently supported:

-+ [Customer-managed encryption keys](search-security-manage-encryption-keys.md) are not supported for vectorizer configuration.

-+ Currently, there's no batching for integrated data chunking and vectorization

-> + [Provide embeddings](vector-search-how-to-generate-embeddings.md) for your index or [generate embeddings (preview)](vector-search-integrated-vectorization.md) in an indexer pipeline

-How you get embeddings from your source content into Azure AI Search depends on your approach and whether you can use preview features. You can vectorize or generate embeddings as a preliminary step using models from OpenAI, Azure OpenAI, and any number of providers, over a wide range of source content including text, images, and other content types supported by the models. You can then push prevectorized content to [vector fields](vector-search-how-to-create-index.md) in a vector store. That's the generally available approach. If you can use preview features, Azure AI Search offers [integrated data chunking and vectorization](vector-search-integrated-vectorization.md) in an indexer pipeline. You still provide the resources (endpoints and connection information to Azure OpenAI), but Azure AI Search makes all of the calls and handles the transitions.

-On the query side, in your client application, you collect the query input from a user, usually through a prompt workflow. You can then add an encoding step that converts the input into a vector, and then send the vector query to your index on Azure AI Search for a similarity search. As with indexing, you can deploy the [integrated vectorization (preview)](vector-search-integrated-vectorization.md) to convert the question into a vector. For either approach, Azure AI Search returns documents with the requested `k` nearest neighbors (kNN) in the results.

-+ Azure portal using the [Import and vectorize data wizard](search-get-started-portal-import-vectors.md)

-+ Azure REST APIs, [version 2023-11-01](/rest/api/searchservice/operation-groups)

-+ Azure SDKs for [.NET](https://www.nuget.org/packages/Azure.Search.Documents), [Python](https://pypi.org/project/azure-search-documents), and [JavaScript](https://www.npmjs.com/package/@azure/search-documents/v/12.0.0-beta.2)

-| Azure data platforms: Azure Blob Storage, Azure Cosmos DB | You can use [indexers](search-indexer-overview.md) to automate data ingestion, and then use [integrated vectorization (preview)](vector-search-integrated-vectorization.md) to generate embeddings. Azure AI Search can automatically index vector data from two data sources: [Azure blob indexers](search-howto-indexing-azure-blob-storage.md) and [Azure Cosmos DB for NoSQL indexers](search-howto-index-cosmosdb.md). For more information, see [Add vector fields to a search index.](vector-search-how-to-create-index.md). |

-Within an index definition, you can specify one or more algorithms, and then for each vector field specify which algorithm to use:

-+ [Create a vector store](vector-search-how-to-create-index.md) to specify an algorithm in the index and on fields.

-+ For exhaustive KNN, use [2023-11-01](/rest/api/searchservice/indexes/create-or-update), [2023-10-01-Preview](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2023-10-01-preview&preserve-view=true) and all newer preview APIs, or Azure SDK beta libraries that target either REST API version.

-Exhaustive KNN support is available through [2023-11-01 REST API](/rest/api/searchservice/search-service-api-versions#2023-11-01), [2023-10-01-Preview REST API](/rest/api/searchservice/search-service-api-versions#2023-10-01-Preview), and in Azure SDK client libraries that target either REST API version.

-> This feature is in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). The [2024-05-01-Preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-Preview&preserve-view=true) supports this feature.

-> This feature is in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). The [2024-05-01-Preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2024-05-01-Preview&preserve-view=true) supports this feature.

-> [!IMPORTANT]

-> This feature is in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). The [2023-10-01-Preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2023-10-01-preview&preserve-view=true) and all newer preview REST APIs support this feature.

-> [!IMPORTANT]

-> This feature is in public preview under [Supplemental Terms of Use](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). The [2023-10-01-Preview REST API](/rest/api/searchservice/indexes/create-or-update?view=rest-searchservice-2023-10-01-preview&preserve-view=true) and all newer preview REST APIs support this feature.

-Semantic search was renamed to [semantic ranking](semantic-search-overview.md) in November 2023 to better describe the feature, which provides L2 ranking of an existing result set.

-To get health data from the *SentinelHealth* data table, or to get auditing information from the *SentinelAudit* data table, you must first turn on the Microsoft Sentinel auditing and health monitoring feature for your workspace.

-This article instructs you how to turn on these features.

-To implement the health and audit feature using API (Bicep/ARM/REST), review the [Diagnostic Settings operations](/rest/api/monitor/diagnostic-settings).

-To configure the retention time for your audit and health events, see [Manage data retention in a Log Analytics workspace](../azure-monitor/logs/data-retention-configure.md).

-## Data tables and resource types

-When the feature is turned on, the *SentinelHealth* and *SentinelAudit* data tables are created at the first event generated for the selected resources.

-The following resource types are currently supported for health monitoring:

- > [!NOTE]

- > When monitoring playbook health, you'll also need to collect Azure Logic Apps diagnostic events from your playbooks in order to get the full picture of your playbook activity. See [**Monitor the health of your automation rules and playbooks**](monitor-automation-health.md) for more information.

-Only the analytics rule resource type is currently supported for auditing.

-1. Scroll down to the **Auditing and health monitoring** section that appears below, and select it to expand.

- Or, select the **Configure diagnostic settings** link to enable health monitoring only for the data collector and/or automation resources, or to configure advanced options, like additional places to send the data.

- If you require, you may select other destinations to which to send your data, in addition to the Log Analytics workspace.

-You want to verify that the service's many moving parts are always functioning as intended, and it isn't being manipulated by unauthorized actions, whether by internal users or otherwise. You may also like to configure notifications of health drifts or unauthorized actions to be sent to relevant stakeholders who can respond or approve a response. For example, you can set conditions to trigger the sending of emails or Microsoft Teams messages to operations teams, managers, or officers, launch new tickets in your ticketing system, and so on.

-This article describes how Microsoft SentinelΓÇÖs health monitoring and auditing features let you monitor the activity of some of the serviceΓÇÖs key resources and inspect logs of user actions within the service.

-The most common way you'll use this data is by querying these tables. For best results, build your queries on the **pre-built functions** on these tables, ***_SentinelHealth()*** and ***_SentinelAudit()***, instead of querying the tables directly. These functions ensure the maintenance of your queries' backward compatibility in the event of changes being made to the schema of the tables themselves.

- - [Data connectors](monitor-data-connector-health.md#run-queries-to-detect-health-drifts)

- - [Automation rules and playbooks](monitor-automation-health.md#get-the-complete-automation-picture) (join query with Azure Logic Apps diagnostics)

- - [Analytics rules](monitor-analytics-rule-integrity.md#run-queries-to-detect-health-and-integrity-issues)

- - [Data connectors](monitor-data-connector-health.md#use-the-health-monitoring-workbook)

- - [Automation rules and playbooks](monitor-automation-health.md#use-the-health-monitoring-workbook)

- - [Analytics rules](monitor-analytics-rule-integrity.md#use-the-auditing-and-health-monitoring-workbook)

-## Next steps

-See also: