Bing 视觉搜索以获取图像洞察

引言

Visual Search SDK 支持 Bing 图像搜索功能，并简化了发送 POST 请求以获取图像洞察的细节。

背景

图像洞察包括相关图像、包含该图像的网页或显示图像中产品的商户列表。 有关更多信息，请参阅 Microsoft 文档：获取关于图像的洞察 和 Visual Search API。

有几种方法可以获取图像洞察。 这些方法需要发送图像或来自先前图像请求的洞察令牌给 Bing，以获取更多被称为图像洞察的信息。 Bing Search SDK 经济地管理 POST 请求的细节，因此您不必像 REST API 所要求的那样构造 POST 正文和元数据。

Using the Code

Visual Search Bing SDK 提供四种语言版本

• C#
• 节点
• Java
• Python

以下代码示例使用 C# SDK，可以使用 Nuget 包为 Visual Studio 开发安装它。

应用程序依赖项

要使用 Bing Visual Search SDK 设置控制台应用程序，请从 Visual Studio 的解决方案资源管理器浏览到“管理 NuGet 程序包”选项。 添加 Microsoft.Azure.CognitiveServices.Search.VisualSearch 包。

安装 NuGet Web Search SDK 包也会安装依赖项，包括

• Microsoft.Rest.ClientRuntime
• Microsoft.Rest.ClientRuntime.Azure
• Newtonsoft.Json

对于此快速入门，您可以使用免费试用订阅密钥或付费订阅密钥。

基本代码场景

以下代码显示了使用二进制图像上传获取图像洞察的基本场景。 该示例实例化 VisualSearchAPI 客户端，从本地 *TestImages* 文件夹 POST 一个图像文件，然后列出搜索结果中的标签和操作。

标签帮助用户探索图像中找到的主题。 例如，如果输入图像是可识别的名人，则标签之一可能是该人的姓名，另一个标签是该图像中该人的专业领域。

操作标识关于图像的各种信息，例如购物来源、相关搜索或包含该图像的网页。

using Microsoft.Azure.CognitiveServices.Search.VisualSearch;
using Microsoft.Azure.CognitiveServices.Search.VisualSearch.Models;
using System;
using System.IO;
using System.Linq;

namespace CodeProjectVisualSrch
{
    class Program
    {
        static void Main(string[] args)
        {
            String subscriptionKey = "YOUR-VISUAL-SEARCH-KEY";
            try
            {
                var client = new VisualSearchAPI(new ApiKeyServiceClientCredentials(subscriptionKey));

                using (System.IO.FileStream stream = 
                   new FileStream(Path.Combine("TestImages", "image.jpg"), FileMode.Open))
                {
                    // The knowledgeRequest parameter is not required 
                    // if an image binary is passed in the request body
                    var visualSearchResults = client.Images.VisualSearchMethodAsync
                                          (image: stream, knowledgeRequest: (string)null).Result;
                    Console.WriteLine("\r\nVisual search request with binary of image");

                    if (visualSearchResults == null)
                    {
                        Console.WriteLine("No visual search result data.");
                    }
                    else
                    {
                        // List of tags
                        if (visualSearchResults.Tags.Count > 0)
                        {
                            var firstTagResult = visualSearchResults.Tags.First();
                            Console.WriteLine($"Visual search tag count: 
                                                {visualSearchResults.Tags.Count}");

                            // List of actions in first tag
                            if (firstTagResult.Actions.Count > 0)
                            {
                                var firstActionResult = firstTagResult.Actions.First();
                                Console.WriteLine($"First tag action count: 
                                                {firstTagResult.Actions.Count}");
                                Console.WriteLine($"First tag action type: 
                                                {firstActionResult.ActionType}");
                            }
                            else
                            {
                                Console.WriteLine("Couldn't find tag actions!");
                            }
                        }
                        else
                        {
                            Console.WriteLine("Couldn't find image tags!");
                        }
                    }
                }
            }

            catch (Exception ex)
            {
                Console.WriteLine("Encountered exception. " + ex.Message);
            }

            // These functions are described in subsequent headings of the article.
            VisualSearchImageBinaryWithCropArea(subscriptionKey);
            VisualSearchUrlWithFilters(subscriptionKey);
            VisualSearchInsightsTokenWithCropArea(subscriptionKey);

            Console.WriteLine("Any key to quit...");
            Console.ReadKey();
        }
    }
}

图像的裁剪区域

您还可以通过设置一个由坐标（左上角和右下角）定义的裁剪区域，来发送图像某个部分的查询。 坐标以十进制格式的百分比指定：0.0 – 1.0。

以下函数将图像二进制文件与裁剪区域对象一起发送到 post 请求的正文中。 然后，它打印出标签的数量、操作的数量和第一个操作类型。

public static void VisualSearchImageBinaryWithCropArea(string subscriptionKey)
{
    var client = new VisualSearchAPI(new ApiKeyServiceClientCredentials(subscriptionKey));

    try
    {
        using (FileStream stream = new FileStream
                 (Path.Combine("TestImages", "image.jpg"), FileMode.Open))
        {
            // The ImageInfo struct contains a crop area specifying a region 
            // to crop in the uploaded image
            CropArea CropArea = new CropArea(top: (float)0.1, 
                     bottom: (float)0.5, left: (float)0.1, right: (float)0.9);
            ImageInfo ImageInfo = new ImageInfo(cropArea: CropArea);
            VisualSearchRequest VisualSearchRequest = new VisualSearchRequest(imageInfo: ImageInfo);

            // The knowledgeRequest here holds additional information about the image, 
            // which is passed in binary form
            var visualSearchResults = client.Images.VisualSearchMethodAsync
                             (image: stream, knowledgeRequest: VisualSearchRequest).Result;
            Console.WriteLine("\r\nVisual search request with binary of image 
                             and knowledgeRequest of crop area");

            if (visualSearchResults == null)
            {
                Console.WriteLine("No visual search result data.");
            }
            else
            {
                // List of tags
                if (visualSearchResults.Tags.Count > 0)
                {
                    var firstTagResult = visualSearchResults.Tags.First();
                    Console.WriteLine($"Visual search tag count: {visualSearchResults.Tags.Count}");

                    // List of actions in first tag
                    if (firstTagResult.Actions.Count > 0)
                    {
                        var firstActionResult = firstTagResult.Actions.First();
                        Console.WriteLine($"First tag action count: {firstTagResult.Actions.Count}");
                        Console.WriteLine($"First tag action type: {firstActionResult.ActionType}");
                    }
                    else
                    {
                        Console.WriteLine("Couldn't find tag actions!");
                    }
                }
                else
                {
                    Console.WriteLine("Couldn't find image tags!");
                }
            }
        }
    }

    catch (Exception ex)
    {
        Console.WriteLine("Encountered exception. " + ex.Message);
    }
}

图像的 URL

您可以发送图像信息的 URL，而不是图像二进制文件。 知识请求对象中的可选过滤器可以指定要搜索的域。

以下函数在 knowledgeRequest 参数中发送图像 URL，以及过滤器：site:www.bing.com。 该代码打印结果，包括标签的数量、操作的数量和第一个 actionType。

此示例还打印出随结果一起上传的 imageInsightsToken。

public static void VisualSearchUrlWithFilters(string subscriptionKey)
{
    var client = new VisualSearchAPI(new ApiKeyServiceClientCredentials(subscriptionKey));

    try
    {
        // The image can be specified via URL, in the ImageInfo object
        var ImageUrl = "https://images.unsplash.com/photo-1512546148165-e50d714a565a?w=600&q=80";
        ImageInfo ImageInfo = new ImageInfo(url: ImageUrl);

        // Optional filters inside the knowledgeRequest will restrict similar products 
        // and images to certain domains
        Filters Filters = new Filters(site: "www.bing.com");
        KnowledgeRequest KnowledgeRequest = new KnowledgeRequest(filters: Filters);

        // An image binary is not necessary here, as the image is specified via URL
        VisualSearchRequest VisualSearchRequest = new VisualSearchRequest
                           (imageInfo: ImageInfo, knowledgeRequest: KnowledgeRequest);
        var visualSearchResults = client.Images.VisualSearchMethodAsync
                           (knowledgeRequest: VisualSearchRequest).Result;
        Console.WriteLine("\r\nVisual search request with url of image and 
                            knowledgeRequest based on filters");

        if (visualSearchResults == null)
        {
            Console.WriteLine("No visual search result data.");
        }
        else
        {
            // Visual Search results
            if (visualSearchResults.Image?.ImageInsightsToken != null)
            {
                Console.WriteLine($"Uploaded image insights token: 
                              {visualSearchResults.Image.ImageInsightsToken}");
            }
            else
            {
                Console.WriteLine("Couldn't find image insights token!");
            }

            // List of tags
            if (visualSearchResults.Tags.Count > 0)
            {
                var firstTagResult = visualSearchResults.Tags.First();
                Console.WriteLine($"Visual search tag count: {visualSearchResults.Tags.Count}");

                // List of actions in first tag
                if (firstTagResult.Actions.Count > 0)
                {
                    var firstActionResult = firstTagResult.Actions.First();
                    Console.WriteLine($"First tag action count: {firstTagResult.Actions.Count}");
                    Console.WriteLine($"First tag action type: {firstActionResult.ActionType}");
                }
                else
                {
                    Console.WriteLine("Couldn't find tag actions!");
                }
            }
            else
            {
                Console.WriteLine("Couldn't find image tags!");
            }
        }
    }

    catch (Exception ex)
    {
        Console.WriteLine("Encountered exception. " + ex.Message);
    }
}

具有裁剪区域的洞察令牌

如果您拥有来自先前使用 Bing 图像搜索 API 的请求的图像洞察令牌，则可以在 knowledgeRequest 参数中发送该图像洞察令牌，而不是图像二进制文件或 URL。

以下代码使用图像洞察令牌以及裁剪区域对象来获取结果。 然后，它打印出 imageInsights 令牌、标签的数量、操作的数量和第一个操作类型。

public static void VisualSearchInsightsTokenWithCropArea(string subscriptionKey)
{
    var client = new VisualSearchAPI(new ApiKeyServiceClientCredentials(subscriptionKey));

    try
    {
        // The image can be specified via an insights token, in the ImageInfo object
        var ImageInsightsToken = "bcid_113F29C079F18F385732D8046EC80145*ccid_oV/
        QcH95*mid_687689FAFA449B35BC11A1AE6CEAB6F9A9B53708*thid_R.113F29C079F18F385732D8046EC80145";

        // An optional crop area can be passed in to define a region of interest in the image
        CropArea CropArea = new CropArea(top: (float)0.1, bottom: (float)0.5, 
                            left: (float)0.1, right: (float)0.9);
        ImageInfo ImageInfo = new ImageInfo(imageInsightsToken: 
                              ImageInsightsToken, cropArea: CropArea);

        // An image binary is not necessary here, as the image is specified via insights token
        VisualSearchRequest VisualSearchRequest = new VisualSearchRequest(imageInfo: ImageInfo);
        var visualSearchResults = client.Images.VisualSearchMethodAsync
                                  (knowledgeRequest: VisualSearchRequest).Result;

        Console.WriteLine("\r\nVisual search request with 
                          imageInsightsToken and knowledgeRequest based on imageInfo");

        if (visualSearchResults == null)
        {
            Console.WriteLine("No visual search result data.");
        }
        else
        {
            // Visual Search results
            if (visualSearchResults.Image?.ImageInsightsToken != null)
            {
                Console.WriteLine($"Uploaded image insights token: 
                                  {visualSearchResults.Image.ImageInsightsToken}");
            }
            else
            {
                Console.WriteLine("Couldn't find image insights token!");
            }

            // List of tags
            if (visualSearchResults.Tags.Count > 0)
            {
                var firstTagResult = visualSearchResults.Tags.First();
                Console.WriteLine($"Visual search tag count: {visualSearchResults.Tags.Count}");

                // List of actions in first tag
                if (firstTagResult.Actions.Count > 0)
                {
                    var firstActionResult = firstTagResult.Actions.First();
                    Console.WriteLine($"First tag action count: {firstTagResult.Actions.Count}");
                    Console.WriteLine($"First tag action type: {firstActionResult.ActionType}");
                }
                else
                {
                    Console.WriteLine("Couldn't find tag actions!");
                }
            }
            else
            {
                Console.WriteLine("Couldn't find image tags!");
            }
        }
    }

    catch (Exception ex)
    {
        Console.WriteLine("Encountered exception. " + ex.Message);
    }
}

历史

• 版本 1.0