WO2018095142A1 - Livestream interaction method and apparatus - Google Patents

Livestream interaction method and apparatus Download PDF

Info

Publication number
WO2018095142A1
WO2018095142A1 PCT/CN2017/104468 CN2017104468W WO2018095142A1 WO 2018095142 A1 WO2018095142 A1 WO 2018095142A1 CN 2017104468 W CN2017104468 W CN 2017104468W WO 2018095142 A1 WO2018095142 A1 WO 2018095142A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
feature information
user
feature
information
Prior art date
Application number
PCT/CN2017/104468
Other languages
French (fr)
Chinese (zh)
Inventor
蔡毅
Original Assignee
广州华多网络科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州华多网络科技有限公司 filed Critical 广州华多网络科技有限公司
Publication of WO2018095142A1 publication Critical patent/WO2018095142A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40—Network security protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5862—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/254—Management at additional data server, e.g. shopping server, rights management server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application

Definitions

  • the present invention relates to the field of image processing, and in particular, to a method and apparatus for live interaction.
  • the anchor is the host or performer who leaves the Internet during the live broadcast.
  • the anchor can be considered a new career.
  • Some anchors gain additional financial benefits by embedding ads in live content.
  • the anchor is usually broadcast live using the live application, and the viewer is also using the live application to watch the live broadcast.
  • the anchor can set the advertisement content to be displayed in the live interface of the live application, that is, display the advertisement content to the viewer through text or image on the live interface; or directly arrange the advertisement content to be displayed in the real scene. That is, displaying photos or texts in a real-life scene to achieve the purpose of displaying advertisements.
  • the anchor can display the advertisement content to be displayed in an area of the live interface, and the advertisement area of the interface is provided by the live application developer, and the user of the viewer or application clicks. After the advertising content of the live interface advertising area, it will jump to the display interface of an item to achieve the effect of displaying the product to the user.
  • the present invention proposes a live interaction method and device.
  • the object of the present invention is to provide a method for live interaction, which aims to solve the problem that the user's user experience is low when the article is displayed in a live broadcast to the user in the prior art.
  • the problem of the present invention is to provide a live interactive device that can display items naturally, so that the user's experience is better.
  • the present invention provides a method for live interaction, which includes:
  • the screen of the live video is taken as the image to be recognized
  • the product information of the item corresponding to the feature information is displayed to the user by using a preset display manner.
  • the step of establishing the feature information base includes:
  • the extracting the feature information of the image to be identified includes:
  • using the picture of the live video as the image to be identified includes:
  • displaying the product information of the item corresponding to the feature information to the user by using a preset display manner includes:
  • the plurality of the product information are displayed in the same interface or the plurality of the product information are displayed in different interfaces.
  • the present invention also provides a live interactive device, including:
  • An image recognition module is configured to: when a preset viewing instruction is received, use a screen of the live video as the image to be identified;
  • An extraction module configured to extract feature information of the image to be identified
  • a searching module configured to find whether the feature information exists in a pre-established feature information database
  • the display module is configured to display the product information of the item corresponding to the feature information to the user by using a preset display manner when the feature information exists in the feature information database.
  • it also includes:
  • An image acquisition module configured to acquire an image of the item by using a graphic acquisition device
  • a feature extraction module configured to extract feature information of the image
  • the extraction module includes:
  • a dividing unit configured to divide the grayscale image to be identified into a local small area of a preset pixel size
  • a feature data calculation unit configured to obtain feature data corresponding to each of the local small regions according to the gray value of the pixel points in the local small region
  • a statistical analysis unit configured to perform statistical analysis on the feature data to obtain the feature information of the image to be identified.
  • the identification image acquisition module includes:
  • An intercepting unit configured to use the click instruction from the user when The point clicked by the user is a center point, and a video image of a preset area of the live video is intercepted, and the video image is used as the image to be recognized.
  • the display module includes:
  • An interface display unit configured to display, according to the plurality of the corresponding relationships, product information of a plurality of corresponding items to the user in an interface form
  • the plurality of the product information are displayed in the same interface or the plurality of the product information are displayed in different interfaces.
  • the method and device for real-time interaction provided by the present invention, when receiving a preset viewing instruction, using the picture of the live video as the image to be recognized; extracting the feature information of the image to be recognized; and searching for the pre-established feature information database Whether the feature information exists in the feature information; when the feature information exists in the feature information database, the product information of the item corresponding to the feature information is displayed to the user by using a preset display manner.
  • the feature information of the image is matched with the feature information in the feature information library, and then it is determined whether the area has an item to be displayed, and when the user wants to view the video area, there is an item.
  • the information of the item is displayed to the user in a certain display manner. It can be seen that the image recognition technology is used to identify the product information of the item that the user wants to view, and then the product information of the item is displayed to the user, and the product information of the item can be naturally displayed in the live broadcast, thereby improving the user experience.
  • FIG. 1 is a schematic flowchart of a specific implementation manner of a live broadcast interaction method according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of another specific implementation manner of a live broadcast interaction method according to an embodiment of the present invention.
  • Figure 3-1 is a schematic diagram of a sample of an image to be processed
  • Figure 3-2 is a schematic diagram of feature data obtained after processing
  • FIG. 5 is a structural block diagram of an apparatus for performing live interaction according to an embodiment of the present invention.
  • FIG. 1 is a schematic flowchart diagram of a specific implementation manner of a live interaction method according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of an image deduplication operation.
  • the feature data of the sample image of 0.5 m is compared with the feature data of the sample image of 2 m, and the interference features are eliminated, and the corresponding feature data corresponding to the de-duplication is proposed.
  • the anchor may display a variety of goods at the same time, such as clothes, hats, and pants.
  • the anchor can wear clothes, hats, and trousers on the body, and walk around for advertisement display.
  • the user can click on the clothes area in the video,
  • the hat area and the trouser area are respectively used to obtain purchase information of the corresponding item. That is, the same image to be identified may include multiple feature regions. In this case, the user may click on different feature regions to learn the product information corresponding to the different feature regions.
  • a feature extraction module 506, configured to extract feature information of the image
  • An intercepting unit configured to: when receiving a click instruction from the user, intercept a video image of a preset area of the live video with the point clicked by the user as a center point, and use the video image as the The image to be identified.
  • the live interactive device provided by the embodiment of the present invention, the device obtains the image of the live video area that the user wants to view, uses the feature information of the image to match the feature information in the feature information database, and then determines whether the area needs
  • the displayed item when there is an item in the video area that the user wants to view, the information of the item is displayed to the user in a certain display manner.
  • the image recognition technology is used to identify the product information of the item that the user wants to view, and then the product information of the item is displayed to the user, and the product information of the item can be naturally displayed in the live broadcast, thereby improving the user experience.
  • the steps of a method or algorithm described in connection with the embodiments disclosed herein can be implemented directly in hardware, a software module executed by a processor, or a combination of both.
  • the software module can be placed in random access memory (RAM), memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or technical field. Any other form of storage medium known.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Accounting & Taxation (AREA)
  • Library & Information Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Strategic Management (AREA)
  • Finance (AREA)
  • Development Economics (AREA)
  • Databases & Information Systems (AREA)
  • Marketing (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method and apparatus for interaction in live streaming. When a preset viewing instruction is received, a live video image is taken as a to-be-recognized image; feature information of the to-be-recognized image is extracted; whether the feature information exists in a pre-established feature information library is searched; when the feature information exists in the feature information library, commodity information of an item corresponding to the feature information is displayed to the user in a preset display manner. An image that the user wants to view in a live streaming video area is acquired, and the feature information of the image is matched with the feature information in the feature information library, so that it can be determined whether there is an to-be-displayed item in the area, and when there is a to-be-displayed item in the video area that the user wants to view, the commodity information of the item is displayed to the user. It can be seen that, using image recognition technology to identify the commodity information of the item that the user wants to view and displaying the commodity information of the item to the user, the commodity information of the item can be displayed in the live streaming in a natural manner, thereby improving the user experience.

Description

一种直播互动的方法及装置Method and device for live interaction 技术领域Technical field
本发明涉及图像处理领域，特别是涉及一种直播互动的方法及装置。The present invention relates to the field of image processing, and in particular, to a method and apparatus for live interaction.
背景技术Background technique
随着互联网技术的发展以及社会的现实需求，催生了网络直播行业。网络直播一般是指主播通过网络技术将主播想要直播的内容展示给观众，而主播则可以利用文字、视频等手段与观众进行互动。With the development of Internet technology and the real needs of society, the webcast industry has been born. Webcasting generally means that the anchor broadcasts the content that the anchor wants to broadcast to the viewer through the network technology, and the anchor can interact with the viewer by means of text or video.
主播是指在网络直播中出境的主持人或者表演者，主播可以认为是一种新生职业。一些主播通过在直播内容中植入广告，以获取额外的经济利益。主播一般是利用直播应用程序来进行直播，而观众也是利用直播应用程序来观看直播的。主播可以在直播应用程序的直播界面内设置所要展示的广告内容，即在直播界面上通过文字或者是图片等手段，将广告内容展示给观众；也可以直接在现实场景内布置所要展示的广告内容，即在现实场景中展示照片或者是文字从而达到展示广告的目的。例如，在手机视频直播应用程序中，主播可以在直播界面的一个区域设置展示所要展示的广告内容，其界面的广告区域是由直播应用程序开发者所提供的，观众或者应用程序的用户通过点击直播界面广告区域的广告内容后，会跳转到一个商品的展示界面，以达到向用户展示商品的效果。The anchor is the host or performer who leaves the Internet during the live broadcast. The anchor can be considered a new career. Some anchors gain additional financial benefits by embedding ads in live content. The anchor is usually broadcast live using the live application, and the viewer is also using the live application to watch the live broadcast. The anchor can set the advertisement content to be displayed in the live interface of the live application, that is, display the advertisement content to the viewer through text or image on the live interface; or directly arrange the advertisement content to be displayed in the real scene. That is, displaying photos or texts in a real-life scene to achieve the purpose of displaying advertisements. For example, in a mobile video live application, the anchor can display the advertisement content to be displayed in an area of the live interface, and the advertisement area of the interface is provided by the live application developer, and the user of the viewer or application clicks. After the advertising content of the live interface advertising area, it will jump to the display interface of an item to achieve the effect of displaying the product to the user.
但是，利用直播应用程序的直播界面的广告区域展示商品信息，会影响到主播表演内容的展示，也会让用户比较生硬地接收广告内容，进而影响用户的观看体验。如何在直播中自然地展示所要展示的物品，进而提高用户的使用体验是本领域亟待解决的问题，基于此，本发明提出了一种直播互动的方法及装置。However, displaying the product information in the advertising area of the live interface of the live application may affect the display of the anchor performance content, and also allow the user to receive the advertisement content bluntly, thereby affecting the user's viewing experience. How to naturally display the items to be displayed in the live broadcast, thereby improving the user experience is an urgent problem to be solved in the field. Based on this, the present invention proposes a live interaction method and device.
发明内容Summary of the invention
本发明的目的是提供一种直播互动的方法，目标在于解决现有技术中将物品生硬地在直播中展示给用户从而使得用户的使用体验较低 的问题；本发明的另一目的是提供一种直播互动的装置，其可以将物品自然地展示，使得用户的使用体验较好。The object of the present invention is to provide a method for live interaction, which aims to solve the problem that the user's user experience is low when the article is displayed in a live broadcast to the user in the prior art. The problem of the present invention is to provide a live interactive device that can display items naturally, so that the user's experience is better.
为解决上述技术问题，本发明提供一种直播互动的方法，该方法包括：To solve the above technical problem, the present invention provides a method for live interaction, which includes:
当接收到预设查看指令时，将直播视频的画面作为待识别图像；When receiving the preset viewing instruction, the screen of the live video is taken as the image to be recognized;
提取所述待识别图像的特征信息；Extracting feature information of the image to be identified;
查找预先建立的特征信息库中是否存在所述特征信息；Finding whether the feature information exists in a pre-established feature information database;
当所述特征信息库中存在所述特征信息时，利用预设展示方式向用户展示与所述特征信息对应的物品的商品信息。When the feature information exists in the feature information database, the product information of the item corresponding to the feature information is displayed to the user by using a preset display manner.
可选地，所述特征信息库的建立步骤包括：Optionally, the step of establishing the feature information base includes:
利用图形采集设备获取所述物品的图像；Acquiring an image of the item using a graphics acquisition device;
提取所述图像的特征信息；Extracting feature information of the image;
建立所述特征信息与商品信息的对应关系，存储至所述特征信息库。Corresponding relationship between the feature information and the product information is established and stored in the feature information database.
可选地，所述提取所述待识别图像的特征信息包括：Optionally, the extracting the feature information of the image to be identified includes:
对所述待识别图像做预处理操作，得出待识别灰度图像；Performing a pre-processing operation on the image to be identified to obtain a grayscale image to be recognized;
将所述待识别灰度图像划分为预设像素大小的局部小区域；Dividing the grayscale image to be identified into a local small area of a preset pixel size;
根据所述局部小区域内的像素点灰度值，得出每个所述局部小区域对应的特征数据；Determining, according to the gray value of the pixel in the local small area, the feature data corresponding to each of the local small areas;
对所述特征数据进行统计分析，得出所述待识别图像的纹理特征向量，将所述纹理特征向量作为所述特征信息。Performing statistical analysis on the feature data to obtain a texture feature vector of the image to be identified, and using the texture feature vector as the feature information.
可选地，所述当接收到预设查看指令时，将所述直播视频的画面作为待识别图像包括：Optionally, when the preset viewing instruction is received, using the picture of the live video as the image to be identified includes:
当接收到用户的点击指令时，以所述用户所点击的点为中心点，截取所述直播视频的预设面积的视频图像，将所述视频图像作为所述待识别图像。When receiving a click command from the user, the video image of the preset area of the live video is intercepted with the point clicked by the user as the center point, and the video image is used as the image to be recognized.
可选地，所述当所述特征信息库中存在所述特征信息时，利用预设展示方式向用户展示与所述特征信息对应的物品的商品信息包括：Optionally, when the feature information exists in the feature information database, displaying the product information of the item corresponding to the feature information to the user by using a preset display manner includes:
当所述特征信息库中存在与所述待识别图像的特征信息相匹配 的多个特征信息时，获取多个所述对应关系；When the feature information database matches the feature information of the image to be identified Acquiring a plurality of the corresponding relationships when the plurality of feature information is obtained;
根据多个所述对应关系，以界面形式向所述用户展示多个对应的物品的商品信息；Displaying, according to the plurality of the corresponding relationships, product information of the plurality of corresponding items to the user in an interface form;
其中，多个所述商品信息在同一界面中进行展示或者多个所述商品信息在不同界面中进行展示。The plurality of the product information are displayed in the same interface or the plurality of the product information are displayed in different interfaces.
此外，本发明还提供了一种直播互动的装置，包括：In addition, the present invention also provides a live interactive device, including:
识别图像获取模块，用于当接收到预设查看指令时，将直播视频的画面作为待识别图像；An image recognition module is configured to: when a preset viewing instruction is received, use a screen of the live video as the image to be identified;
提取模块，用于提取所述待识别图像的特征信息；An extraction module, configured to extract feature information of the image to be identified;
查找模块，用于查找预先建立的特征信息库中是否存在所述特征信息；a searching module, configured to find whether the feature information exists in a pre-established feature information database;
展示模块，用于当所述特征信息库中存在所述特征信息时，利用预设展示方式向用户展示与所述特征信息对应的物品的商品信息。The display module is configured to display the product information of the item corresponding to the feature information to the user by using a preset display manner when the feature information exists in the feature information database.
可选地，还包括：Optionally, it also includes:
图像采集模块，用于利用图形采集设备获取所述物品的图像；An image acquisition module, configured to acquire an image of the item by using a graphic acquisition device;
特征提取模块，用于提取所述图像的特征信息；a feature extraction module, configured to extract feature information of the image;
建立模块，用于建立所述特征信息与商品信息的对应关系，存储至所述特征信息库。And a establishing module, configured to establish a correspondence between the feature information and the product information, and store the information in the feature information database.
可选地，所述提取模块包括：Optionally, the extraction module includes:
预处理单元，用于对所述待识别图像做预处理操作，得出待识别灰度图像；a preprocessing unit, configured to perform a preprocessing operation on the image to be identified, and obtain a grayscale image to be identified;
划分单元，用于将所述待识别灰度图像划分为预设像素大小的局部小区域；a dividing unit, configured to divide the grayscale image to be identified into a local small area of a preset pixel size;
特征数据计算单元，用于根据所述局部小区域内的像素点灰度值，得出每个所述局部小区域对应的特征数据；a feature data calculation unit, configured to obtain feature data corresponding to each of the local small regions according to the gray value of the pixel points in the local small region;
统计分析单元，用于对所述特征数据进行统计分析，得出所述待识别图像的所述特征信息。And a statistical analysis unit, configured to perform statistical analysis on the feature data to obtain the feature information of the image to be identified.
可选地，所述识别图像获取模块包括：Optionally, the identification image acquisition module includes:
截取单元，用于当接收到来自所述用户的点击指令时，以所述用 户所点击的点为中心点，截取所述直播视频的预设面积的视频图像，将所述视频图像作为所述待识别图像。An intercepting unit, configured to use the click instruction from the user when The point clicked by the user is a center point, and a video image of a preset area of the live video is intercepted, and the video image is used as the image to be recognized.
可选地，所述展示模块包括：Optionally, the display module includes:
获取单元，用当所述特征信息库中存在与所述待识别图像的特征信息相匹配的多个特征信息时，获取多个所述对应关系；And acquiring, by the acquiring unit, a plurality of the corresponding relationships when a plurality of feature information matching the feature information of the image to be identified exists in the feature information database;
界面展示单元，用于根据多个所述对应关系，以界面形式向所述用户展示多个对应的物品的商品信息；An interface display unit, configured to display, according to the plurality of the corresponding relationships, product information of a plurality of corresponding items to the user in an interface form;
其中，多个所述商品信息在同一界面中进行展示或者多个所述商品信息在不同界面中进行展示。The plurality of the product information are displayed in the same interface or the plurality of the product information are displayed in different interfaces.
本发明所提供的一种直播互动的方法及装置，当接收到预设查看指令时，将直播视频的画面作为待识别图像；提取所述待识别图像的特征信息；查找预先建立的特征信息库中是否存在所述特征信息；当所述特征信息库中存在所述特征信息时，利用预设展示方式向用户展示与所述特征信息对应的物品的商品信息。通过获取用户想要查看的直播视频区域的图像，利用图像的特征信息与特征信息库内的特征信息相匹配，继而判断该区域是否有需要展示的物品，当用户想要查看的视频区域有物品，则将该物品的信息以一定的展示方式展示给用户。可见，利用图像识别技术来识别用户想要查看的物品的商品信息，进而将该物品的商品信息展示给用户，可以将物品的商品信息在直播中自然地展示出来，进而提高用户的使用体验。The method and device for real-time interaction provided by the present invention, when receiving a preset viewing instruction, using the picture of the live video as the image to be recognized; extracting the feature information of the image to be recognized; and searching for the pre-established feature information database Whether the feature information exists in the feature information; when the feature information exists in the feature information database, the product information of the item corresponding to the feature information is displayed to the user by using a preset display manner. By acquiring the image of the live video area that the user wants to view, the feature information of the image is matched with the feature information in the feature information library, and then it is determined whether the area has an item to be displayed, and when the user wants to view the video area, there is an item. , the information of the item is displayed to the user in a certain display manner. It can be seen that the image recognition technology is used to identify the product information of the item that the user wants to view, and then the product information of the item is displayed to the user, and the product information of the item can be naturally displayed in the live broadcast, thereby improving the user experience.
附图说明DRAWINGS
为了更清楚的说明本发明实施例或现有技术的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单的介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are merely Some embodiments of the present invention may also be used to obtain other drawings based on these drawings without departing from the art.
图1为本发明实施例所提供的直播互动方法的一种具体实施方式的流程示意图； 1 is a schematic flowchart of a specific implementation manner of a live broadcast interaction method according to an embodiment of the present invention;
图2为本发明实施例所提供的直播互动方法的另一种具体实施方式的流程示意图；2 is a schematic flowchart of another specific implementation manner of a live broadcast interaction method according to an embodiment of the present invention;
图3-1为待处理的图像的样本示意图；Figure 3-1 is a schematic diagram of a sample of an image to be processed;
图3-2为处理后得到的特征数据的示意图；Figure 3-2 is a schematic diagram of feature data obtained after processing;
图4为图像去重操作的示意图；4 is a schematic diagram of an image deduplication operation;
图5为本发明实施例提供的直播互动的装置的结构框图。FIG. 5 is a structural block diagram of an apparatus for performing live interaction according to an embodiment of the present invention.
具体实施方式detailed description
为了使本技术领域的人员更好地理解本发明方案，下面结合附图和具体实施方式对本发明作进一步的详细说明。显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The present invention will be further described in detail below in conjunction with the drawings and embodiments. It is apparent that the described embodiments are only a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
请参见图1，图1为本发明实施例所提供的直播互动方法的一种具体实施方式的流程示意图。Referring to FIG. 1, FIG. 1 is a schematic flowchart diagram of a specific implementation manner of a live interaction method according to an embodiment of the present invention.
步骤101：当接收到预设查看指令时，将直播视频的画面作为待识别图像；Step 101: When receiving the preset viewing instruction, the screen of the live video is taken as the image to be identified;
当正在观看直播视频的用户发出预设查看指令后，根据该查看指令来获取当前播放的直播视频的视频画面，将获取得到的画面作为待识别图像。可以理解的是，待识别图像可以是直播视频中的一帧图像，此时可以将直播视频的整个画面作为待识别图像，也可以是一帧图像的局部小区域，此时可以将直播视频的画面的一部分区域作为待识别图像。After the user who is watching the live video issues a preset viewing instruction, the video image of the currently played live video is obtained according to the viewing instruction, and the obtained image is taken as the image to be recognized. It can be understood that the image to be identified may be a frame image in the live video. In this case, the entire picture of the live video may be used as the image to be recognized, or may be a small area of a frame image. A part of the screen is used as the image to be recognized.
需要说明的是，预设查看指令是预先设定的，当用户想要了解直播视频的某一个区域内的物品信息时，可以通过输入相应的查看指令，进而达到查看的目的。其预设查看指令可以为点击指令、双击指令以及长按指令中的任意一种。用户在使用直播应用程序观看直播视频时，其输入预设查看指令的方式有很多。例如，当用户所使用的直播应用程序为移动端直播应用程序时，用户可以利用手指单击或者是双击直播视频画面，从而输入点击指令或者是双击指令；而当用户所使用的 直播应用程序为电脑端直播应用程序时，此时用户可以使用鼠标来单击或者是双击直播视频画面，从而输入点击指令或者双击指令，当然，如果电脑的显示屏为触摸屏时，也可以利用手指输入查看指令。显而易见地，上述所提到的点击指令、双击指令以及长按指令只是预设查看指令优选的实施方式，预设查看指令还可以为其它类型的指令，在此不再赘述。It should be noted that the preset viewing instruction is preset. When the user wants to know the item information in a certain area of the live video, the user can input the corresponding viewing instruction to achieve the purpose of viewing. The preset viewing instruction may be any one of a click command, a double click command, and a long press command. When a user views a live video using a live application, there are many ways to enter a preset view command. For example, when the live application used by the user is a mobile live application, the user can click or double-click the live video screen to input a click command or a double click command; and when the user uses When the live application is a live application on the computer, the user can click or double-click the live video screen to input a click command or a double click command. Of course, if the display of the computer is a touch screen, the finger can also be used. Enter the view command. Obviously, the click command, the double-click command, and the long press command mentioned above are only preferred implementation manners of the preset view command, and the preset view command may also be other types of commands, and details are not described herein again.
由于视频播放的是一帧帧静态的图像，故可以输入预设查看指令，进而获取视频图像中的一帧静态图像，然后在对该图像进行处理。可以通过输入点击指令来进行截图操作，继而获取到想要获取的视频图像，在本发明的一些实施例中，其具体过程可以为：当接收到来自所述用户的点击指令时，以所述用户所点击的点为中心点，截取所述直播视频的预设面积的视频图像，将所述视频图像作为所述待识别图像。用户可以利用手指单击触摸屏来输入点击指令，或者利用鼠标单击视频画面来输入点击指令，接收到点击指令后，对直播视频进行截图操作，得到预设面积的视频图像。Since the video plays a static image of one frame, a preset viewing command can be input, and then a frame of the still image in the video image is acquired, and then the image is processed. The screenshot operation may be performed by inputting a click instruction, and then the video image to be acquired is obtained. In some embodiments of the present invention, the specific process may be: when receiving a click instruction from the user, The point clicked by the user is a center point, and a video image of a preset area of the live video is intercepted, and the video image is used as the image to be recognized. The user can click a touch screen to input a click command, or click a video screen to input a click command, and after receiving the click command, perform a screenshot operation on the live video to obtain a video image of a preset area.
预设面积的视频图像可以是指所截取的视频图像的面积是人为设定的，即可以截取整个直播视频画面的视频图像作为待识别图像，也可以截取直播视频画面的部分区域作为待识别图像。可以理解的是，可以人为地调整想要截取的视频图像的大小，即当用户输入查看指令时，会用户显示具有一定面积大小的截图框，用户可以调整截图框的大小，以调整所要截取的视频图像的面积大小。将截取局部小区域作为待识别图像，识别效率更高，响应更快。比如，用户利用手机进入到直播界面中观看直播视频，而此时主播正在展示商品，该商品的可以是衣服、鞋子以及帽子等物品，而主播可以将商品穿在身上并可以随意走动以进行全方位地展示，也可以放置直播摄像头的摄像范围内。用户可以点击商品的任意位置，也可以点击当前直播视频的任意区域，然后以用户点击的点为中心点，截取屏幕分辨率的1/4的视频画面，作为待识别图像。将预设面积为屏幕分辨率的1/4的视频画面进行识别分析，可以使得手机端的性能开销较低，响应将快，可以大幅度提 升了用户的使用体验。The video image of the preset area may be that the area of the captured video image is manually set, that is, the video image of the entire live video picture may be intercepted as the image to be recognized, or part of the live video picture may be intercepted as the image to be recognized. . It can be understood that the size of the video image that is to be intercepted can be artificially adjusted, that is, when the user inputs a viewing instruction, the user displays a screenshot frame having a certain size, and the user can adjust the size of the screenshot frame to adjust the intercepted image. The size of the video image. Taking a small local area as the image to be identified, the recognition efficiency is higher and the response is faster. For example, the user uses the mobile phone to enter the live broadcast interface to watch the live video, while the anchor is displaying the product, the item can be clothes, shoes and hats, and the anchor can wear the product on the body and can walk around for the whole Displayed in a position, it can also be placed within the camera range of the live camera. The user can click anywhere in the product, or click on any area of the current live video, and then take the point where the user clicks as the center point, and intercept the video screen of 1/4 of the screen resolution as the image to be recognized. The recognition and analysis of the video screen with the preset area of 1/4 of the screen resolution can make the performance overhead of the mobile phone terminal lower, and the response will be fast, which can be greatly improved. Increased user experience.
接收用户的查看指令，继而获取到待识别图像，该待识别图像包含着用户想要查看物品的商品信息。为了使用户能详细地了解到想要查看的物品的商品信息，需要对获取的待识别图像的图像信息进行分析，对图像信息进行分析之前需要提取待识别图像的特征信息。Receiving a viewing instruction of the user, and then acquiring an image to be recognized, the image to be recognized containing product information that the user wants to view the item. In order to enable the user to know the product information of the item to be viewed in detail, it is necessary to analyze the acquired image information of the image to be recognized, and the feature information of the image to be recognized needs to be extracted before analyzing the image information.
步骤102：提取所述待识别图像的特征信息；Step 102: Extract feature information of the image to be identified.
需要说明的是，特征信息指的是表征数字图像的图像信息，图像特征的类别有颜色特征、空间关系特征、形状特征以及纹理特征等，特征类别的不同，其提取的过程也会相应的不同。It should be noted that the feature information refers to image information representing a digital image. The types of image features include color features, spatial relationship features, shape features, and texture features. The feature categories are different, and the extraction process is also different. .
纹理特征表征的是图像像素点的灰度值的差异，可以提取待识别图像的纹理特征信息，进而对提取的纹理特征信息进行分析。在本发明的一些实例中，其纹理特征信息的提取过程可以具体为：对所述待识别图像做预处理操作，得出待识别灰度图像；将所述待识别灰度图像划分为预设像素大小的局部小区域；根据所述局部小区域内的像素点灰度值，得出每个所述局部小区域对应的特征数据；对所述特征数据进行统计分析，得出所述待识别图像的纹理特征向量，将所述纹理特征向量作为所述特征信息。先将待识别图像灰度化，转为待识别灰度图像，为了使提取的纹理特征信息的准确性，可以对图像进行去噪操作，而去噪操作可以利用中值滤波器以及锐化算法来完成。再将去噪后的待识别灰度图像划分成多个局部小区域，其每个局部小区域的面积是相同的，局部小区域的面积是预设的。可以利用马赛克算法来将待识别区域划分成多个预设面积的局部小区域，例如，划分为多个16*16的局部小区域。再对每一个局部小区域内的像素点的灰度值进行比较，得出特征数据，最后利用特征数据求出待识别图像的纹理特征向量，纹理特征向量则为整幅待识别图像的特征信息。显而易见地，纹理特征信息只是特征信息的一种优选的实施方式，特征信息还可以为其它类别的图像信息，相应的提取过程在此不再进一步介绍说明。The texture feature represents the difference of the gray value of the image pixel, and the texture feature information of the image to be identified can be extracted, and then the extracted texture feature information is analyzed. In some examples of the present invention, the extraction process of the texture feature information may be specifically: performing a pre-processing operation on the image to be identified, and obtaining a grayscale image to be recognized; and dividing the grayscale image to be recognized into a preset a local small area of a pixel size; obtaining feature data corresponding to each of the local small areas according to the gray value of the pixel in the local small area; performing statistical analysis on the feature data to obtain the image to be identified a texture feature vector, the texture feature vector being used as the feature information. First, the image to be recognized is grayed out and converted into a gray image to be recognized. In order to make the extracted texture feature information accurate, the image may be denoised, and the denoising operation may utilize a median filter and a sharpening algorithm. To be done. Then, the de-noised grayscale image to be recognized is divided into a plurality of local small regions, and the area of each local small region is the same, and the area of the local small region is preset. The mosaic algorithm may be utilized to divide the area to be identified into a plurality of local small areas of a predetermined area, for example, into a plurality of local small areas of 16*16. Then, the gray values of the pixels in each local small area are compared to obtain the feature data. Finally, the texture feature vector of the image to be identified is obtained by using the feature data, and the texture feature vector is the feature information of the entire image to be recognized. Obviously, the texture feature information is only a preferred implementation manner of the feature information, and the feature information may also be other types of image information, and the corresponding extraction process is not further described herein.
由于所获得的待识别图像可能没有包含物品信息，也可能包含的物品的商品信息不是想要展示的商品信息，故在提取待识别图像的特 征信息之后，还需要利用特征信息去判断该待识别图像中是否有想要展示的商品信息。Since the obtained image to be identified may not contain the item information, and the item information of the item to be included may not be the item information to be displayed, the special image to be recognized is extracted. After the information is collected, it is also necessary to use the feature information to determine whether there is any product information to be displayed in the image to be identified.
步骤103：查找预先建立的特征信息库中是否存在所述特征信息；Step 103: Find whether the feature information exists in a pre-established feature information database;
利用提取的特征信息与预先建立的特征信息库中的特征信息进行比对，通过判断比对是否一致来确定特征信息库中是否有待识别图像的特征信息。当特征信息库中有待识别图像的特征信息时，表明待识别图像中有需要展示的物品，当特征信息库中没有待识别图像的特征信息时，表明待识别图像中没有需要展示的物品。The extracted feature information is compared with the feature information in the pre-established feature information database, and whether the feature information is to be identified in the feature information database is determined by determining whether the comparison is consistent. When there is feature information of the image to be identified in the feature information database, it indicates that there is an item to be displayed in the image to be identified. When there is no feature information of the image to be identified in the feature information database, it indicates that there is no item to be displayed in the image to be identified.
特征信息库中包含着多种特征信息，多种特征信息对应着多种所需展示的物品，而每一种特征信息可以有多个特征信息，即每一种物品对应着多个特征信息。每一种物品对应着多个特征信息，多个不同的特征信息可以很好地包括物品的不同视角以及不同距离下的特征信息，可以使物品识别的准确率更高。例如，可以将物品的8个不同特征信息存储在特征信息库，这8个特征信息是通过提取物品的不同视角以及不同距离的图像样本获得的，可以很好的表征物品各个角度以及不同距离的特征信息。The feature information database contains a plurality of feature information, and the plurality of feature information corresponds to a plurality of items to be displayed, and each type of feature information may have a plurality of feature information, that is, each item corresponds to a plurality of feature information. Each item corresponds to a plurality of feature information, and the plurality of different feature information can well include different viewing angles of the articles and feature information at different distances, so that the accuracy of the item identification can be higher. For example, eight different feature information of the item may be stored in the feature information library, and the eight pieces of feature information are obtained by extracting different perspectives of the article and image samples of different distances, which can well represent various angles of the article and different distances. Feature information.
在特征信息库中，除了可以存储特征信息之外，还可以存储特征信息与商品信息的对应关系。在知道特征信息的前提下，通过该对应关系可以很容易地找到对应的物品的商品信息。故在每一种物品对应多个特征信息的情况下，该物品的商品信息对应着多个特征信息。而商品信息可以是物品的图像信息、价格信息以及类别信息等。可以预先选择将特征信息与预设的商品信息进行绑定。对应关系可以是物品一一对应的ID，即特征信息与商品信息之间有一个ID，通过特征信息可以找到对应的唯一ID，然后根据唯一ID可以找到对应的物品的商品信息。当然，对应关系也可以是商品信息直接与特征信息两者之间的关系，通过特征信息可以找到对应的物品的商品信息。显而易见地，上述的对应关系只是优选的实施方式，并不完全囊括对应关系的所有实施方式。In the feature information base, in addition to the feature information, the correspondence between the feature information and the product information may be stored. Under the premise of knowing the feature information, the commodity information of the corresponding article can be easily found through the correspondence. Therefore, in the case where each item corresponds to a plurality of pieces of feature information, the item information of the item corresponds to a plurality of pieces of feature information. The product information may be image information, price information, and category information of the article. The feature information can be pre-bound with the preset product information. The correspondence relationship may be an ID corresponding to the item one by one, that is, there is an ID between the feature information and the item information, and the corresponding unique ID may be found through the feature information, and then the item information of the corresponding item may be found according to the unique ID. Of course, the correspondence relationship may also be a relationship between the product information and the feature information, and the product information of the corresponding item may be found through the feature information. Obviously, the above-described correspondences are only preferred embodiments and do not fully encompass all embodiments of the corresponding relationships.
将待识别图像的特征信息与特征信息库中的图像样本信息进行 比较匹配，以寻找出该待识别图像中包含物品的商品信息。其特征信息库的预先建立是通过预先采集各种想要展示的物品的样本图像，提取该样本图像的样本特征信息，根据样本特征信息可以建立特征信息库。在本发明的一些实施例中，其特征信息库的建立过程可以具体为：利用图形采集设备获取所述物品的图像；提取所述图像的特征信息；建立所述特征信息与商品信息的对应关系，存储至所述特征信息库。利用图像采集设备采集图片的样本图像，该样本图像的数量可以是一幅，但是为了使识别的准确率尽可能地高，可以采集该物品上下左右等各个视角的图像，以及图像采集设备与该物品的不同距离情况下的图像。对应一个物品来说，采集的样本图像越多，提取的特征信息的个数也越多，识别的准确率也会相应地提高，其识别的效率则会相应地降低。综合衡量识别效率以及识别准确率，可以在该物品的正面、背面、左面以及背面4个视角，且在这4个不同的视角下分别采集两个不同距离的8个样本图像。例如，在图像采集设备距离物品0.5m以及4m的位置，分别采集物品的正面视角、背面视角、左面视角以及背面视角的样本图像，则有0.5m的4幅样本图像以及4m的4幅样本图像。Performing feature information of the image to be identified and image sample information in the feature information library The matching is matched to find the product information of the item included in the image to be identified. The feature information database is pre-established by extracting sample images of various items to be displayed in advance, extracting sample feature information of the sample image, and establishing a feature information library according to the sample feature information. In some embodiments of the present invention, the process of establishing the feature information base may be specifically: acquiring an image of the item by using a graphic collection device; extracting feature information of the image; establishing a correspondence between the feature information and the product information. And stored to the feature information library. The image capturing device is used to collect a sample image of the image, and the number of the sample images may be one, but in order to make the recognition accuracy as high as possible, an image of each view such as up, down, left, and right of the article may be collected, and the image capturing device and the image capturing device An image of the item at different distances. For an item, the more sample images are collected, the more the number of feature information is extracted, and the accuracy of recognition is correspondingly increased, and the efficiency of recognition is correspondingly reduced. Comprehensive measurement of recognition efficiency and recognition accuracy can be obtained from four perspectives of the front, back, left and back of the article, and eight sample images of two different distances are collected under these four different perspectives. For example, when the image capturing device collects the front view, the back view, the left view, and the back view sample image of the article at a position of 0.5 m and 4 m from the article, respectively, there are 4 sample images of 0.5 m and 4 sample images of 4 m. .
当特征信息库中存在有待识别图像的特征信息，则表明用户所查看的区域内有需要展示的物品的商品信息。When the feature information of the image to be identified exists in the feature information database, it indicates that the product information of the item to be displayed is displayed in the area viewed by the user.
步骤104：当所述特征信息库中存在所述特征信息时，利用预设展示方式向用户展示与所述特征信息对应的物品的商品信息。Step 104: When the feature information exists in the feature information database, the product information of the item corresponding to the feature information is displayed to the user by using a preset display manner.
当特征信息库中有待识别图像的特征信息时，将该特征信息对应的物品展示给用户，用户看到的是关于该物品的商品信息。可以理解的是，该商品信息可以是指物品的广告信息。When the feature information of the image to be recognized is included in the feature information database, the item corresponding to the feature information is displayed to the user, and the user sees the product information about the item. It can be understood that the item information can refer to the advertisement information of the item.
预设展示方式可以为界面形式，即跳出一个界面给用户，该界面可以是直播应用程序内的界面，也可以是外联到某个指定的界面。该界面上可以包括关于物品的商品信息的链接，点击该链接可以查看到相应的商品信息，也可以包括物品对应的商品信息。将商品的商品信息展示给用户，比如，用户在点击直播视频的任意区域后，利用图像 分析算法识别到该区域包含有主播预先录入的物品识别信息，则根据得到的特征信息来确定相应的商品，例如为衣服，则跳转到一个购买界面上，该购买界面上有该衣服的商品信息，用户也可以在该界面购买该衣服。当然，预设展示方式不限于上述提到的界面形式，在此不作限定。The preset display mode may be an interface form, that is, an interface is displayed to the user, and the interface may be an interface in a live application or an external interface to a specified interface. The interface may include a link about the item information of the item, and clicking the link may view the corresponding item information, and may also include the item information corresponding to the item. Display the product information of the product to the user, for example, the user uses the image after clicking on any area of the live video The analysis algorithm recognizes that the area contains the item identification information pre-recorded by the anchor, and determines the corresponding item according to the obtained feature information, for example, clothing, and then jumps to a purchase interface, where the purchase interface has the goods of the clothes. Information, the user can also purchase the clothes on the interface. Of course, the preset display manner is not limited to the interface form mentioned above, and is not limited herein.
待识别图像中可以包含多个商品信息，对应地也会有多个不同的特征信息。在进行物品展示时，可以利用不同的展示方式对多个物品进行展示。在本发明的一些实施例中，其展示多个商品信息的过程可以具体为：当所述特征信息库中存在与所述待识别图像的特征信息相匹配的多个特征信息时，获取多个所述对应关系；根据多个所述对应关系，以界面形式向所述用户展示多个对应的物品的商品信息；其中，多个所述商品信息在同一界面中进行展示或者多个所述商品信息在不同界面中进行展示。多个特征信息对应着多种对应关系，根据返回的多个对应关系得到相应的物品的商品信息，进而进行商品信息的展示。The plurality of product information may be included in the image to be identified, and correspondingly there may be a plurality of different feature information. When displaying items, multiple items can be displayed using different presentation methods. In some embodiments of the present invention, the process of displaying a plurality of item information may be specifically: when a plurality of pieces of feature information matching the feature information of the image to be identified exist in the feature information library, acquiring multiple Corresponding relationship; displaying, according to the plurality of corresponding relationships, product information of a plurality of corresponding items to the user in an interface form; wherein, the plurality of the product information are displayed in the same interface or a plurality of the goods Information is displayed in different interfaces. The plurality of feature information corresponds to a plurality of correspondences, and the product information of the corresponding article is obtained according to the plurality of corresponding correspondences, and the product information is displayed.
本发明实施例所提供的直播互动的方法，当接收到预设查看指令时，将直播视频的画面作为待识别图像；提取待识别图像的特征信息；查找预先建立的特征信息库中是否存在特征信息；当特征信息库中存在特征信息时，利用预设展示方式向用户展示与特征信息对应的物品的商品信息。通过获取用户想要查看的直播视频区域的图像，利用图像的特征信息与特征信息库内的特征信息相匹配，继而判断该区域是否有需要展示的物品，当用户想要查看的视频区域有物品，则将该物品的信息以一定的展示方式展示给用户。可见，利用图像识别技术来识别用户想要查看的物品的商品信息，进而将该物品的商品信息展示给用户，可以将物品的商品信息在直播中自然地展示出来，进而提高用户的使用体验。The live interaction method provided by the embodiment of the present invention, when receiving the preset viewing instruction, uses the picture of the live video as the image to be recognized; extracts the feature information of the image to be identified; and finds whether the feature exists in the pre-established feature information database. Information; when the feature information exists in the feature information database, the product information of the item corresponding to the feature information is displayed to the user by using a preset display manner. By acquiring the image of the live video area that the user wants to view, the feature information of the image is matched with the feature information in the feature information library, and then it is determined whether the area has an item to be displayed, and when the user wants to view the video area, there is an item. , the information of the item is displayed to the user in a certain display manner. It can be seen that the image recognition technology is used to identify the product information of the item that the user wants to view, and then the product information of the item is displayed to the user, and the product information of the item can be naturally displayed in the live broadcast, thereby improving the user experience.
现在的主播一般是利用直播应用程序来进行直播的，使用的直播工具可以为手机或电脑等终端。主播在直播时，可以在直播中打广告，即通过直播将商品展示给用户以达到广告的效应。当主播利用手机来进行直播且广告商品信息时，用户可以进入该主播的直播界面观看直 播，然后在主播进行商品展示时，用户通过点击商品区域来获得对应商品的购买信息，完成直播互动购物。下面将上述直播互动购物过程进行介绍。The current anchor is usually broadcast live using the live application, and the live broadcast tool can be a terminal such as a mobile phone or a computer. When the anchor is broadcasted, the advertisement can be advertised in the live broadcast, that is, the product is displayed to the user through the live broadcast to achieve the effect of the advertisement. When the anchor uses the mobile phone to broadcast and advertise the product information, the user can enter the live broadcast interface of the anchor to watch the direct Broadcast, and then when the anchor displays the product, the user clicks on the product area to obtain the purchase information of the corresponding product, and completes the live interactive shopping. The above live interactive shopping process will be introduced below.
请参见图2，图2为本发明实施例所提供的直播互动方法的另一种具体实施方式的流程示意图。Referring to FIG. 2, FIG. 2 is a schematic flowchart diagram of another specific implementation manner of a live broadcast interaction method according to an embodiment of the present invention.
步骤201：获取商品的样本图像，并提取样本图像的纹理特征信息；Step 201: Acquire a sample image of the commodity, and extract texture feature information of the sample image;
利用摄像头来拍摄商品的样本图像，将拍摄得到的样本图像发送给处理器。首先打开直播应用程序的商品页面，输入物品的商品信息，输入完成后，再录入商品的识别信息；再则将摄像头正对商品的正面，且摄像头与商品的距离为预设距离，点击识别按钮进行识别，如果识别成功，则表示商品的正面的样本图像已成功录入；继而将摄像头正对商品的左侧面，且摄像头与商品的距离为预设距离，点击识别按钮进行识别，如果识别成功，则表示商品的左侧面的样本图像已成功录入。以上述录入方法对商品的背面以及右侧面的样本图像进行采集录入，完成录入商品的识别信息即样本图像。显而易见地，预设距离可以为0.5m、2或其它数值。对同一商品，可以采集不同预设距离的样本图像，以增加识别的准确率。例如，可以分别采集预设距离为0.5m和2m正面样本图像、背面样本图像、左侧样本图像以及右侧样本图像。当然，预设距离以及采集的角度不限于上述提到的。A sample image of the product is captured by the camera, and the captured sample image is sent to the processor. First open the product page of the live application, enter the product information of the item, enter the identification information of the product after the input is completed; then, the camera is facing the front of the product, and the distance between the camera and the product is the preset distance, click the identification button Identifying, if the recognition is successful, it means that the sample image of the front of the product has been successfully entered; then the camera is facing the left side of the product, and the distance between the camera and the product is a preset distance, click the identification button to identify, if the recognition is successful , indicating that the sample image on the left side of the item has been successfully entered. The sample image on the back side and the right side of the product is collected and recorded by the above-described entry method, and the sample image of the identification information of the input product is completed. Obviously, the preset distance can be 0.5m, 2 or other values. For the same product, sample images of different preset distances can be collected to increase the accuracy of recognition. For example, a front sample image having a preset distance of 0.5 m and 2 m, a back sample image, a left sample image, and a right sample image may be separately collected. Of course, the preset distance and the angle of acquisition are not limited to those mentioned above.
采集得到商品的样本图像之后，需要提取获得的样本图像的纹理特征信息。首先将样本图像转化为HSV的灰阶图像，得到样本灰度图像，之后再利用中值滤波器以及锐化算法去除样本灰度图像中的噪点，再则利用马赛克算法将样本灰度图像划分为预设像素大小的小格。将每一个小格中的中心像素点与相邻的8个像素点的灰度值进行比较，如果周围像素点的灰度值大于中心像素点的灰度值，则该像素点的位置被标为1，反之则标为0，以得到样本图像的全部特征数据，比如，请参见图3-1，图3-2。图3-1为待处理的图像的样本示意图；图3-2为处理后得到的特征数据的示意图。将图3-1经过去噪操作、划分操 作后，在根据像素点的灰度值差异得出图3-2的特征数据示意图。显而易见地，样本图像的去噪以及划分的方法不限于上述所提到的。After the sample image of the commodity is acquired, the texture feature information of the obtained sample image needs to be extracted. Firstly, the sample image is transformed into the grayscale image of HSV, and the sample gray image is obtained. Then the median filter and sharpening algorithm are used to remove the noise in the sample gray image, and then the mosaic image is used to divide the sample gray image into A small grid of preset pixel sizes. Comparing the central pixel point in each small cell with the gray value of the adjacent 8 pixel points, if the gray value of the surrounding pixel point is greater than the gray value of the central pixel point, the position of the pixel point is marked If it is 1, otherwise it is marked as 0 to get all the feature data of the sample image. For example, please refer to Figure 3-1 and Figure 3-2. Figure 3-1 is a schematic diagram of samples of an image to be processed; Figure 3-2 is a schematic diagram of feature data obtained after processing. Figure 3-1 is subjected to denoising operation and division operation. After that, the feature data of FIG. 3-2 is obtained based on the difference in gray value of the pixel. Obviously, the method of denoising and dividing the sample image is not limited to the above mentioned.
在获取到所有样本图像的特征数据之后，对特征数据进行去重操作，剔除无用的干扰特征，得到纯净特征数据。例如，请参见图4，图4为图像去重操作的示意图。将0.5m的样本图像的特征数据与2m的样本图像的特征数据进行对比，剔除干扰特征，提出得到去重后对应的纯净的特征数据。After acquiring the feature data of all the sample images, the feature data is subjected to deduplication operation, and the useless interference features are eliminated to obtain the pure feature data. For example, please refer to FIG. 4, which is a schematic diagram of an image deduplication operation. The feature data of the sample image of 0.5 m is compared with the feature data of the sample image of 2 m, and the interference features are eliminated, and the corresponding feature data corresponding to the de-duplication is proposed.
得到纯净特征数据之后，可以计算每组数据中的每个小格特征点的直方图，即统计每个像素点的灰度值的概率分布情况，再对直方图进行归一化处理，最后将归一化后的直方图连接为特征向量，其特征向量为纹理特征向量，即为纹理特征信息。当有8组纯净特征数据时，相应地有8组纹理特征向量。After obtaining the pure feature data, the histogram of each small lattice feature point in each set of data can be calculated, that is, the probability distribution of the gray value of each pixel point is counted, and then the histogram is normalized, and finally The normalized histogram is connected as a feature vector, and the feature vector is a texture feature vector, which is texture feature information. When there are 8 sets of pure feature data, there are 8 sets of texture feature vectors accordingly.
步骤202：将纹理特征信息与对应商品进行绑定，生成唯一ID，建立特征信息库；Step 202: Bind the texture feature information to the corresponding commodity, generate a unique ID, and establish a feature information database.
将得出的纹理特征向量上传到服务器上，与对应的商品进行一一绑定，并生成唯一对应的ID，根据纹理特征向量以及对应的ID建立特征库。例如当得出的纹理特征向量为8个时，可以将8个纹理特征信息做格式序列化，打包成一个二进制的特征识别文件并同步到服务器。至此录入了商品的识别信息，建立了特征信息库。当然，在必要的时候，可以重新录入商品的识别信息。The obtained texture feature vector is uploaded to the server, and the corresponding products are bound one by one, and a unique corresponding ID is generated, and the feature library is established according to the texture feature vector and the corresponding ID. For example, when the obtained texture feature vector is 8, the 8 texture feature information can be serialized, packaged into a binary feature recognition file and synchronized to the server. At this point, the identification information of the product has been entered, and a feature information base has been established. Of course, the identification information of the product can be re-entered when necessary.
步骤203：获取待识别图像；Step 203: Acquire an image to be identified.
直播应用程序的用户进入到直播界面时，直播应用程序会将主播特征信息库下载到用户所使用的直播应用程序中，以便后续识别步骤的调用，直播应用程序可以是手机端的直播应用程序，也可以为电脑端的直播应用程序，例如为虎牙直播。当用户点击直播视频时，则以所点击的点为中心点，截取一定面积的视频图像作为待识别图像，该视频图像是整个视频界面的局部小区域。比如，用户进入到直播界面中观看直播视频，此时主播正在展示商品，该商品的可以是衣服、鞋子等物品，而主播可以将衣服穿着身上并随意走动以进行全方位地展 示，也可以放置直播摄像头所包括的视觉范围。用户可以点击该商品的任意位置，也可以点击当前直播视频的任意区域，然后以用户点击的点为中心点，截取屏幕分辨率的1/4的视频图像作为待识别图像。可以理解的是，上述获取待识别图像的方式只是一种优选的实施方式，还可以利用其它的方式来获取待识别图像。When the user of the live application enters the live interface, the live application downloads the anchor feature database to the live application used by the user, so that the subsequent identification step can be invoked. The live application can be a live application on the mobile phone. It can be a live application on the computer side, such as live broadcast of Tiger Tooth. When the user clicks on the live video, the video image of a certain area is taken as the image to be recognized with the clicked point as the center point, and the video image is a local small area of the entire video interface. For example, the user enters the live broadcast interface to watch the live video. At this time, the anchor is displaying the product. The product can be clothes, shoes, etc., and the anchor can wear the clothes and walk around for a full-scale exhibition. It is also possible to place the visual range included in the live camera. The user can click anywhere in the product, or click on any area of the current live video, and then take a video image of 1/4 of the screen resolution as the image to be recognized with the point clicked by the user as the center point. It can be understood that the manner of obtaining the image to be identified is only a preferred implementation manner, and other methods may be used to obtain the image to be identified.
步骤204：提取待识别图像的纹理特征信息；Step 204: Extract texture feature information of the image to be identified.
将获取的待识别图像转化为灰阶图片，并将待识别灰度图像中的噪点去掉，去噪的方法可以利用中值滤波器以及锐化算法，再则利用马赛克算法划分待识别灰度图像，将待识别灰度图像划分为一定面积的小格，例如为16*16像素面积大小。利用景深算法去除离屏幕较远的小格。提取每个小格的特征数据，提取的过程与步骤201的纹理特征信息提取过程类似，在此不再赘述。统计得出每组数据中每个小格特征点的直方图，对直方图进行归一化处理，在根据归一化后的直方图得出待识别图像的纹理特征向量。Converting the acquired image to be recognized into a grayscale image, and removing the noise in the grayscale image to be recognized, the method of denoising can utilize the median filter and the sharpening algorithm, and then the mosaic algorithm is used to divide the grayscale image to be recognized. The grayscale image to be identified is divided into small cells of a certain area, for example, a 16*16 pixel area size. Use the depth of field algorithm to remove small cells that are farther from the screen. The feature data of each cell is extracted, and the process of the extraction is similar to the process of extracting the texture feature information of step 201, and details are not described herein again. A histogram of each small lattice feature point in each set of data is obtained, and the histogram is normalized, and the texture feature vector of the image to be recognized is obtained according to the normalized histogram.
步骤205：查找特征信息库中是否有待识别图像的纹理特征信息；Step 205: Find whether there is texture feature information of the image to be identified in the feature information database;
利用之前获得的纹理特征向量与特征库中的纹理特征向量做一一比对，当发现有匹配的特征向量时，则表明用户所点击的区域内有需要展示的商品。根据纹理特征向量返回对应商品的IDThe previously obtained texture feature vector is compared with the texture feature vector in the feature library. When a matching feature vector is found, it indicates that there is a product to be displayed in the area that the user clicks. Returns the ID of the corresponding item based on the texture feature vector
步骤206：当特征信息库中有待识别图像的纹理特征信息时，则获取对应的ID，根据ID将对应的商品展示给用户。Step 206: When there is texture feature information of the image to be identified in the feature information database, the corresponding ID is obtained, and the corresponding product is displayed to the user according to the ID.
根据返回的ID，获取相应的商品信息，并将商品信息通过界面展示给用户。例如，可以根据ID跳转到相应的购买界面，在该购买界面可以了解到商品信息以及购买商品。According to the returned ID, the corresponding product information is obtained, and the product information is displayed to the user through the interface. For example, you can jump to the corresponding purchase interface according to the ID, where you can know the product information and purchase the product.
可以理解的是，主播可能同时展示多种商品，例如衣服、帽子以及裤子等，主播可以将衣服、帽子以及裤子穿在身上，随意走动进行广告展示，此时用户可以分别点击视频中衣服区域、帽子区域以及裤子区域，来分别获取到相应商品的购买信息。即同一待识别图像中可以包含多个特征区域的，此时用户可以分别点击不同特征区域，以了解到不同特征区域对应的商品信息。 Understandably, the anchor may display a variety of goods at the same time, such as clothes, hats, and pants. The anchor can wear clothes, hats, and trousers on the body, and walk around for advertisement display. At this time, the user can click on the clothes area in the video, The hat area and the trouser area are respectively used to obtain purchase information of the corresponding item. That is, the same image to be identified may include multiple feature regions. In this case, the user may click on different feature regions to learn the product information corresponding to the different feature regions.
本发明实施例所提供的直播互动的方法，提前录入商品的识别信息，为后续识别过程提供原始数据。当用户点击直播视频的任意区域时，则利用图像识别技术来确定用户所点击的区域是否有之前录入的商品特征信息。如果有，则将对应的商品信息展示给用户，以完成直播互动购物。通过对用户点击的局部小区域进行智能识别，相较于最整幅图片进行识别，识别效率更高，对性能的开销更低，也不用每次都做检测分析。整个过程自然快捷，大幅度提高了用户的购物体验。The live interaction method provided by the embodiment of the present invention inputs the identification information of the commodity in advance, and provides the original data for the subsequent identification process. When the user clicks on any area of the live video, image recognition technology is used to determine whether the area clicked by the user has previously recorded product feature information. If so, the corresponding product information is displayed to the user to complete the live interactive shopping. By intelligently identifying the small local area that the user clicks, the recognition is more efficient than the most complete picture, and the performance overhead is lower, and it is not necessary to perform detection analysis every time. The whole process is natural and fast, which greatly improves the user's shopping experience.
下面对本发明实施例提供的直播互动装置进行介绍，下文描述的直播互动装置与上文描述的直播互动方法可相互对应参照。The live interactive device provided by the embodiment of the present invention is introduced below, and the live interactive device described below and the live interactive method described above can refer to each other.
图5为本发明实施例提供的直播互动的装置的结构框图，参照图5直播互动的装置可以包括：FIG. 5 is a structural block diagram of an apparatus for performing live interaction according to an embodiment of the present invention. The apparatus for performing live interaction with reference to FIG. 5 may include:
识别图像获取模块501，用于当接收到预设查看指令时，将直播视频的画面作为待识别图像；The recognition image acquisition module 501 is configured to: when the preset viewing instruction is received, use the screen of the live video as the image to be recognized;
提取模块502，用于提取所述待识别图像的特征信息；An extraction module 502, configured to extract feature information of the image to be identified;
查找模块503，用于查找预先建立的特征信息库中是否存在所述特征信息；The searching module 503 is configured to search whether the feature information exists in the pre-established feature information database;
展示模块504，用于当所述特征信息库中存在所述特征信息时，利用预设展示方式向用户展示与所述特征信息对应的物品的商品信息。The display module 504 is configured to display the product information of the item corresponding to the feature information to the user by using a preset display manner when the feature information exists in the feature information database.
可选地，还包括：Optionally, it also includes:
图像采集模块505，用于利用图形采集设备获取所述物品的图像；An image acquisition module 505, configured to acquire an image of the item by using a graphic acquisition device;
特征提取模块506，用于提取所述图像的特征信息；a feature extraction module 506, configured to extract feature information of the image;
建立模块507，用于建立所述特征信息与商品信息的对应关系，存储至所述特征信息库。The establishing module 507 is configured to establish a correspondence between the feature information and the product information, and store the information in the feature information database.
可选地，所述提取模块502包括：Optionally, the extraction module 502 includes:
预处理单元，用于对所述待识别图像做预处理操作，得出待识别灰度图像；a preprocessing unit, configured to perform a preprocessing operation on the image to be identified, and obtain a grayscale image to be identified;
划分单元，用于将所述待识别灰度图像划分为预设像素大小的局部小区域； a dividing unit, configured to divide the grayscale image to be identified into a local small area of a preset pixel size;
特征数据计算单元，用于根据所述局部小区域内的像素点灰度值，得出每个所述局部小区域对应的特征数据；a feature data calculation unit, configured to obtain feature data corresponding to each of the local small regions according to the gray value of the pixel points in the local small region;
统计分析单元，用于对所述特征数据进行统计分析，得出所述待识别图像的所述特征信息。And a statistical analysis unit, configured to perform statistical analysis on the feature data to obtain the feature information of the image to be identified.
可选地，所述识别图像获取模块501包括：Optionally, the identification image obtaining module 501 includes:
截取单元，用于当接收到来自所述用户的点击指令时，以所述用户所点击的点为中心点，截取所述直播视频的预设面积的视频图像，将所述视频图像作为所述待识别图像。An intercepting unit, configured to: when receiving a click instruction from the user, intercept a video image of a preset area of the live video with the point clicked by the user as a center point, and use the video image as the The image to be identified.
可选地，所述展示模块504包括：Optionally, the display module 504 includes:
获取单元，用当所述特征信息库中存在与所述待识别图像的特征信息相匹配的多个特征信息时，获取多个所述对应关系；And acquiring, by the acquiring unit, a plurality of the corresponding relationships when a plurality of feature information matching the feature information of the image to be identified exists in the feature information database;
界面展示单元，用于根据多个所述对应关系，以界面形式向所述用户展示多个对应的物品的商品信息；An interface display unit, configured to display, according to the plurality of the corresponding relationships, product information of a plurality of corresponding items to the user in an interface form;
其中，多个所述商品信息在同一界面中进行展示或者多个所述商品信息在不同界面中进行展示。The plurality of the product information are displayed in the same interface or the plurality of the product information are displayed in different interfaces.
本发明实施例所提供的直播互动的装置，其装置通过获取用户想要查看的直播视频区域的图像，利用图像的特征信息与特征信息库内的特征信息相匹配，继而判断该区域是否有需要展示的物品，当用户想要查看的视频区域有物品，则将该物品的信息以一定的展示方式展示给用户。利用图像识别技术来识别用户想要查看的物品的商品信息，进而将该物品的商品信息展示给用户，可以将物品的商品信息在直播中自然地展示出来，进而提高用户的使用体验。The live interactive device provided by the embodiment of the present invention, the device obtains the image of the live video area that the user wants to view, uses the feature information of the image to match the feature information in the feature information database, and then determines whether the area needs The displayed item, when there is an item in the video area that the user wants to view, the information of the item is displayed to the user in a certain display manner. The image recognition technology is used to identify the product information of the item that the user wants to view, and then the product information of the item is displayed to the user, and the product information of the item can be naturally displayed in the live broadcast, thereby improving the user experience.
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似部分互相参见即可。对于实施例公开的装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same or similar parts of the respective embodiments may be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
专业人员还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明 中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。A person skilled in the art will further appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of both, in order to clearly illustrate the hardware and software. Interchangeability, in the above description The composition and steps of the various examples have been generally described in terms of function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。The steps of a method or algorithm described in connection with the embodiments disclosed herein can be implemented directly in hardware, a software module executed by a processor, or a combination of both. The software module can be placed in random access memory (RAM), memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or technical field. Any other form of storage medium known.
以上对本发明所提供的直播互动的方法以及装置进行了详细介绍。本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想。应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以对本发明进行若干改进和修饰，这些改进和修饰也落入本发明权利要求的保护范围内。 The method and apparatus for live interaction provided by the present invention are described in detail above. The principles and embodiments of the present invention have been described herein with reference to specific examples, and the description of the above embodiments is only to assist in understanding the method of the present invention and its core idea. It should be noted that those skilled in the art can make various modifications and changes to the present invention without departing from the spirit and scope of the invention.

Claims (10)

  1. 一种直播互动的方法，其特征在于，包括：A method for live interaction, which is characterized by comprising:
    当终端接收到预设查看指令时，将直播视频的画面作为待识别图像；When the terminal receives the preset viewing instruction, the screen of the live video is used as the image to be identified;
    提取所述待识别图像的特征信息；Extracting feature information of the image to be identified;
    查找预先建立的特征信息库中是否存在所述特征信息；Finding whether the feature information exists in a pre-established feature information database;
    当所述特征信息库中存在所述特征信息时，利用预设展示方式向用户展示与所述特征信息对应的物品的商品信息。When the feature information exists in the feature information database, the product information of the item corresponding to the feature information is displayed to the user by using a preset display manner.
  2. 如权利要求1所述的方法，其特征在于，所述特征信息库的建立步骤包括：The method of claim 1, wherein the step of establishing the feature information base comprises:
    利用图形采集设备获取所述物品的图像；Acquiring an image of the item using a graphics acquisition device;
    提取所述图像的特征信息；Extracting feature information of the image;
    建立所述特征信息与商品信息的对应关系，存储至所述特征信息库。Corresponding relationship between the feature information and the product information is established and stored in the feature information database.
  3. 如权利要求2所述的方法，其特征在于，所述提取所述待识别图像的特征信息包括：The method of claim 2, wherein the extracting the feature information of the image to be identified comprises:
    对所述待识别图像做预处理操作，得出待识别灰度图像；Performing a pre-processing operation on the image to be identified to obtain a grayscale image to be recognized;
    将所述待识别灰度图像划分为预设像素大小的局部小区域；Dividing the grayscale image to be identified into a local small area of a preset pixel size;
    根据所述局部小区域内的像素点灰度值，得出每个所述局部小区域对应的特征数据；Determining, according to the gray value of the pixel in the local small area, the feature data corresponding to each of the local small areas;
    对所述特征数据进行统计分析，得出所述待识别图像的纹理特征向量，将所述纹理特征向量作为所述特征信息。Performing statistical analysis on the feature data to obtain a texture feature vector of the image to be identified, and using the texture feature vector as the feature information.
  4. 如权利要求1至3任一项所述的方法，其特征在于，所述当接收到预设查看指令时，将所述直播视频的画面作为待识别图像包括：The method according to any one of claims 1 to 3, wherein, when the preset viewing instruction is received, the screen of the live video as the image to be recognized includes:
    当接收到用户的点击指令时，以所述用户所点击的点为中心点，截取所述直播视频的预设面积的视频图像，将所述视频图像作为所述待识别图像。When receiving a click command from the user, the video image of the preset area of the live video is intercepted with the point clicked by the user as the center point, and the video image is used as the image to be recognized.
  5. 如权利要求2所述的方法，其特征在于，所述当所述特征信息库中存在所述特征信息时，利用预设展示方式向用户展示与所述特征信息对应的物品的商品信息包括： The method according to claim 2, wherein when the feature information is stored in the feature information database, displaying the product information of the item corresponding to the feature information to the user by using a preset display manner comprises:
    当所述特征信息库中存在与所述待识别图像的特征信息相匹配的多个特征信息时，获取多个所述对应关系；Acquiring a plurality of the corresponding relationships when a plurality of feature information matching the feature information of the image to be identified exists in the feature information database;
    根据多个所述对应关系，以界面形式向所述用户展示多个对应的物品的商品信息；Displaying, according to the plurality of the corresponding relationships, product information of the plurality of corresponding items to the user in an interface form;
    其中，多个所述商品信息在同一界面中进行展示或者多个所述商品信息在不同界面中进行展示。The plurality of the product information are displayed in the same interface or the plurality of the product information are displayed in different interfaces.
  6. 一种直播互动的装置，其特征在于，包括：A live interactive device, comprising:
    识别图像获取模块，用于当接收到预设查看指令时，将直播视频的画面作为待识别图像；An image recognition module is configured to: when a preset viewing instruction is received, use a screen of the live video as the image to be identified;
    提取模块，用于提取所述待识别图像的特征信息；An extraction module, configured to extract feature information of the image to be identified;
    查找模块，用于查找预先建立的特征信息库中是否存在所述特征信息；a searching module, configured to find whether the feature information exists in a pre-established feature information database;
    展示模块，用于当所述特征信息库中存在所述特征信息时，利用预设展示方式向用户展示与所述特征信息对应的物品的商品信息。The display module is configured to display the product information of the item corresponding to the feature information to the user by using a preset display manner when the feature information exists in the feature information database.
  7. 如权利要求6所述的装置，其特征在于，还包括：The device of claim 6 further comprising:
    图像采集模块，用于利用图形采集设备获取所述物品的图像；An image acquisition module, configured to acquire an image of the item by using a graphic acquisition device;
    特征提取模块，用于提取所述图像的特征信息；a feature extraction module, configured to extract feature information of the image;
    建立模块，用于建立所述特征信息与商品信息的对应关系，存储至所述特征信息库。And a establishing module, configured to establish a correspondence between the feature information and the product information, and store the information in the feature information database.
  8. 如权利要求7所述的装置，其特征在于，所述提取模块包括：The apparatus of claim 7, wherein the extraction module comprises:
    预处理单元，用于对所述待识别图像做预处理操作，得出待识别灰度图像；a preprocessing unit, configured to perform a preprocessing operation on the image to be identified, and obtain a grayscale image to be identified;
    划分单元，用于将所述待识别灰度图像划分为预设像素大小的局部小区域；a dividing unit, configured to divide the grayscale image to be identified into a local small area of a preset pixel size;
    特征数据计算单元，用于根据所述局部小区域内的像素点灰度值，得出每个所述局部小区域对应的特征数据；a feature data calculation unit, configured to obtain feature data corresponding to each of the local small regions according to the gray value of the pixel points in the local small region;
    统计分析单元，用于对所述特征数据进行统计分析，得出所述待识别图像的所述特征信息。And a statistical analysis unit, configured to perform statistical analysis on the feature data to obtain the feature information of the image to be identified.
  9. 如权利要求6至8任一项所述的装置，其特征在于，所述识 别图像获取模块包括：Apparatus according to any one of claims 6 to 8 wherein said knowledge The image acquisition module includes:
    截取单元，用于当接收到来自所述用户的点击指令时，以所述用户所点击的点为中心点，截取所述直播视频的预设面积的视频图像，将所述视频图像作为所述待识别图像。An intercepting unit, configured to: when receiving a click instruction from the user, intercept a video image of a preset area of the live video with the point clicked by the user as a center point, and use the video image as the The image to be identified.
  10. 如权利要求7所述的装置，其特征在于，所述展示模块包括：The device of claim 7, wherein the display module comprises:
    获取单元，用当所述特征信息库中存在与所述待识别图像的特征信息相匹配的多个特征信息时，获取多个所述对应关系；And acquiring, by the acquiring unit, a plurality of the corresponding relationships when a plurality of feature information matching the feature information of the image to be identified exists in the feature information database;
    界面展示单元，用于根据多个所述对应关系，以界面形式向所述用户展示多个对应的物品的商品信息；An interface display unit, configured to display, according to the plurality of the corresponding relationships, product information of a plurality of corresponding items to the user in an interface form;
    其中，多个所述商品信息在同一界面中进行展示或者多个所述商品信息在不同界面中进行展示。 The plurality of the product information are displayed in the same interface or the plurality of the product information are displayed in different interfaces.
PCT/CN2017/104468 2016-11-28 2017-09-29 Livestream interaction method and apparatus WO2018095142A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611064091.0 2016-11-28
CN201611064091.0A CN108124184A (en) 2016-11-28 2016-11-28 A kind of method and device of living broadcast interactive

Publications (1)

Publication Number Publication Date
WO2018095142A1 true WO2018095142A1 (en) 2018-05-31

Family

ID=62194714

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/104468 WO2018095142A1 (en) 2016-11-28 2017-09-29 Livestream interaction method and apparatus

Country Status (2)

Country Link
CN (1) CN108124184A (en)
WO (1) WO2018095142A1 (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110505497A (en) * 2019-07-31 2019-11-26 湖南微算互联信息技术有限公司 A kind of cloud mobile phone operational monitoring method, system, device and storage medium
CN111144974A (en) * 2019-12-04 2020-05-12 北京三快在线科技有限公司 Information display method and device
CN111541907A (en) * 2020-04-23 2020-08-14 腾讯科技（深圳）有限公司 Article display method, apparatus, device and storage medium
CN111754302A (en) * 2020-06-24 2020-10-09 詹晨 Video live broadcast interface commodity display intelligent management system based on big data
CN113014934A (en) * 2021-01-29 2021-06-22 腾讯科技（深圳）有限公司 Product display method, product display device, computer equipment and storage medium
CN113055724A (en) * 2021-03-12 2021-06-29 北京达佳互联信息技术有限公司 Live broadcast data processing method, device, server, terminal, medium and product
CN113298587A (en) * 2020-05-25 2021-08-24 阿里巴巴集团控股有限公司 Shop commodity information display method, electronic equipment and client
CN113507626A (en) * 2021-04-19 2021-10-15 支付宝(杭州)信息技术有限公司 Commodity association method, device and equipment in video interaction
CN113810744A (en) * 2021-09-17 2021-12-17 北京快来文化传播集团有限公司 Commercial live broadcast wheat connecting system and method
CN113965771A (en) * 2021-10-22 2022-01-21 成都天翼空间科技有限公司 VR live broadcast user interactive experience system
CN114554241A (en) * 2022-02-25 2022-05-27 广州博冠信息科技有限公司 Information processing method and device in live broadcast, electronic equipment and storage medium
CN114727138A (en) * 2022-03-31 2022-07-08 大众问问(北京)信息科技有限公司 Commodity information processing method and device and computer equipment
CN114928768A (en) * 2022-06-10 2022-08-19 北京百度网讯科技有限公司 Live broadcast information pushing method, device, system, electronic equipment and computer medium
CN115225921A (en) * 2022-06-08 2022-10-21 深圳软银思创科技有限公司 Live broadcast interaction control method, device, equipment and storage medium based on big data

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109118279A (en) * 2018-08-06 2019-01-01 北京绿善心星球网络科技开发有限公司 Online live streaming method of commerce, system, electronic equipment and readable storage medium storing program for executing
CN109598541A (en) * 2018-11-19 2019-04-09 中信国安广视网络有限公司 Large-size screen monitors advertisement recommended method based on video identification
CN109766466A (en) * 2018-12-29 2019-05-17 广东益萃网络科技有限公司 Querying method, device, computer equipment and the storage medium of product information
CN111539779A (en) * 2019-01-21 2020-08-14 阿里巴巴集团控股有限公司 Commodity page generation method, electronic equipment and computer storage medium
CN111626816B (en) * 2020-05-10 2021-02-02 上海星地通讯工程研究所 Image interaction information processing method based on e-commerce live broadcast and cloud computing platform
CN111612588B (en) * 2020-05-27 2023-12-26 腾讯科技（深圳）有限公司 Commodity presenting method and device, computing equipment and computer readable storage medium
CN111652678B (en) * 2020-05-27 2023-11-14 腾讯科技（深圳）有限公司 Method, device, terminal, server and readable storage medium for displaying article information
CN112291576B (en) * 2020-10-14 2022-06-17 珠海格力电器股份有限公司 Virtual live broadcast system and method
CN113269785A (en) * 2021-05-13 2021-08-17 北京字节跳动网络技术有限公司 Image processing method, apparatus, storage medium, and program product
CN113286204A (en) * 2021-05-13 2021-08-20 北京达佳互联信息技术有限公司 Information display method and device, computer equipment and computer readable storage medium
CN113240488B (en) * 2021-05-14 2024-04-05 深圳迅销科技股份有限公司 System and method for displaying commodity information by utilizing electronic commerce
CN113382308B (en) * 2021-05-31 2022-11-22 北京达佳互联信息技术有限公司 Information display method and device, electronic equipment and computer readable storage medium
CN113596496A (en) * 2021-07-28 2021-11-02 广州博冠信息科技有限公司 Interaction control method, device, medium and electronic equipment for virtual live broadcast room
CN114449304A (en) * 2022-01-27 2022-05-06 北京达佳互联信息技术有限公司 Information display method, device, equipment, medium and product
CN114554294A (en) * 2022-03-04 2022-05-27 天比高零售管理(深圳)有限公司 Live broadcast content filtering and prompting method
CN114866853A (en) * 2022-04-12 2022-08-05 咪咕文化科技有限公司 Live broadcast interaction method, device, equipment and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060075426A1 (en) * 2004-10-05 2006-04-06 Robert Koch Methods and computer program products for taking a secondary action responsive to receipt of an advertisement
CN102802055A (en) * 2011-05-25 2012-11-28 阿里巴巴集团控股有限公司 Data interaction method and device based on on-line video
CN104991906A (en) * 2015-06-17 2015-10-21 百度在线网络技术（北京）有限公司 Information acquisition method, server, terminal and method and device for database construction
US20150350729A1 (en) * 2014-05-28 2015-12-03 United Video Properties, Inc. Systems and methods for providing recommendations based on pause point in the media asset
CN105681918A (en) * 2015-09-16 2016-06-15 乐视致新电子科技（天津）有限公司 Method and system for presenting article relevant information in video stream
CN105828123A (en) * 2016-03-30 2016-08-03 乐视控股（北京）有限公司 Method and apparatus for interaction in live broadcast
CN105916050A (en) * 2016-05-03 2016-08-31 乐视控股（北京）有限公司 TV shopping information processing method and device
CN106162213A (en) * 2016-07-11 2016-11-23 福建方维信息科技有限公司 A kind of merchandise display method and system based on net cast shopping

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105373938A (en) * 2014-08-27 2016-03-02 阿里巴巴集团控股有限公司 Method for identifying commodity in video image and displaying information, device and system

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060075426A1 (en) * 2004-10-05 2006-04-06 Robert Koch Methods and computer program products for taking a secondary action responsive to receipt of an advertisement
CN102802055A (en) * 2011-05-25 2012-11-28 阿里巴巴集团控股有限公司 Data interaction method and device based on on-line video
US20150350729A1 (en) * 2014-05-28 2015-12-03 United Video Properties, Inc. Systems and methods for providing recommendations based on pause point in the media asset
CN104991906A (en) * 2015-06-17 2015-10-21 百度在线网络技术（北京）有限公司 Information acquisition method, server, terminal and method and device for database construction
CN105681918A (en) * 2015-09-16 2016-06-15 乐视致新电子科技（天津）有限公司 Method and system for presenting article relevant information in video stream
CN105828123A (en) * 2016-03-30 2016-08-03 乐视控股（北京）有限公司 Method and apparatus for interaction in live broadcast
CN105916050A (en) * 2016-05-03 2016-08-31 乐视控股（北京）有限公司 TV shopping information processing method and device
CN106162213A (en) * 2016-07-11 2016-11-23 福建方维信息科技有限公司 A kind of merchandise display method and system based on net cast shopping

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110505497A (en) * 2019-07-31 2019-11-26 湖南微算互联信息技术有限公司 A kind of cloud mobile phone operational monitoring method, system, device and storage medium
CN111144974A (en) * 2019-12-04 2020-05-12 北京三快在线科技有限公司 Information display method and device
CN111541907A (en) * 2020-04-23 2020-08-14 腾讯科技（深圳）有限公司 Article display method, apparatus, device and storage medium
CN111541907B (en) * 2020-04-23 2023-09-22 腾讯科技（深圳）有限公司 Article display method, apparatus, device and storage medium
CN113298587A (en) * 2020-05-25 2021-08-24 阿里巴巴集团控股有限公司 Shop commodity information display method, electronic equipment and client
CN111754302A (en) * 2020-06-24 2020-10-09 詹晨 Video live broadcast interface commodity display intelligent management system based on big data
CN111754302B (en) * 2020-06-24 2021-04-27 广西壹农乐信息技术有限公司 Video live broadcast interface commodity display intelligent management system based on big data
CN113014934A (en) * 2021-01-29 2021-06-22 腾讯科技（深圳）有限公司 Product display method, product display device, computer equipment and storage medium
CN113055724B (en) * 2021-03-12 2023-01-20 北京达佳互联信息技术有限公司 Live broadcast data processing method, device, server, terminal, medium and product
US11606401B2 (en) 2021-03-12 2023-03-14 Beijing Dajia Internet Information Technology Co., Ltd. Method for processing live streaming data and server
CN113055724A (en) * 2021-03-12 2021-06-29 北京达佳互联信息技术有限公司 Live broadcast data processing method, device, server, terminal, medium and product
CN113507626A (en) * 2021-04-19 2021-10-15 支付宝(杭州)信息技术有限公司 Commodity association method, device and equipment in video interaction
CN113810744A (en) * 2021-09-17 2021-12-17 北京快来文化传播集团有限公司 Commercial live broadcast wheat connecting system and method
CN113965771A (en) * 2021-10-22 2022-01-21 成都天翼空间科技有限公司 VR live broadcast user interactive experience system
CN114554241A (en) * 2022-02-25 2022-05-27 广州博冠信息科技有限公司 Information processing method and device in live broadcast, electronic equipment and storage medium
CN114554241B (en) * 2022-02-25 2024-06-04 广州博冠信息科技有限公司 Information processing method and device in live broadcast, electronic equipment and storage medium
CN114727138A (en) * 2022-03-31 2022-07-08 大众问问(北京)信息科技有限公司 Commodity information processing method and device and computer equipment
CN114727138B (en) * 2022-03-31 2023-12-19 大众问问(北京)信息科技有限公司 Commodity information processing method, commodity information processing device and computer equipment
CN115225921A (en) * 2022-06-08 2022-10-21 深圳软银思创科技有限公司 Live broadcast interaction control method, device, equipment and storage medium based on big data
CN115225921B (en) * 2022-06-08 2024-02-06 深圳软银思创科技有限公司 Live broadcast interactive control method, device, equipment and storage medium based on big data
CN114928768A (en) * 2022-06-10 2022-08-19 北京百度网讯科技有限公司 Live broadcast information pushing method, device, system, electronic equipment and computer medium

Also Published As

Publication number Publication date
CN108124184A (en) 2018-06-05

Similar Documents

Publication Publication Date Title
WO2018095142A1 (en) Livestream interaction method and apparatus
US10776970B2 (en) Method and apparatus for processing video image and computer readable medium
US10395120B2 (en) Method, apparatus, and system for identifying objects in video images and displaying information of same
US10210423B2 (en) Image match for featureless objects
CN108322788B (en) Advertisement display method and device in live video
WO2018157746A1 (en) Recommendation method and apparatus for video data
CN108734185B (en) Image verification method and device
US10325372B2 (en) Intelligent auto-cropping of images
US11704357B2 (en) Shape-based graphics search
US8805123B2 (en) System and method for video recognition based on visual image matching
CN104991906B (en) Information acquisition method, server, terminal, database construction method and device
CN103617420A (en) Commodity fast recognition method and system based on image feature matching
CN110881134B (en) Data processing method and device, electronic equipment and storage medium
Kalia et al. An analysis of the effect of different image preprocessing techniques on the performance of SURF: Speeded Up Robust Features
CN105657514A (en) Method and apparatus for playing video key information on mobile device browser
US20210182566A1 (en) Image pre-processing method, apparatus, and computer program
CN113596496A (en) Interaction control method, device, medium and electronic equipment for virtual live broadcast room
CN113784171A (en) Video data processing method, device, computer system and readable storage medium
CN110751004B (en) Two-dimensional code detection method, device, equipment and storage medium
KR20120100124A (en) System and method for providing video related service based on image
CN108833964B (en) Real-time continuous frame information implantation identification system
CN110659923A (en) Information display method and device for user terminal
CN110636322B (en) Multimedia data processing method and device, intelligent terminal and storage medium
CN112015936B (en) Method, device, electronic equipment and medium for generating article display diagram
CN110969454A (en) Cosmetics propaganda and promotion system based on intelligent product album

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17873661

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17873661

Country of ref document: EP

Kind code of ref document: A1