WO2021043092A1 - Procédé et dispositif d'appariement sémantique d'images, terminal et support de stockage lisible par ordinateur - Google Patents

Procédé et dispositif d'appariement sémantique d'images, terminal et support de stockage lisible par ordinateur Download PDF

Info

Publication number
WO2021043092A1
WO2021043092A1 PCT/CN2020/112352 CN2020112352W WO2021043092A1 WO 2021043092 A1 WO2021043092 A1 WO 2021043092A1 CN 2020112352 W CN2020112352 W CN 2020112352W WO 2021043092 A1 WO2021043092 A1 WO 2021043092A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
image
tag set
feature tag
subject
Prior art date
Application number
PCT/CN2020/112352
Other languages
English (en)
Chinese (zh)
Inventor
王健宗
彭俊清
瞿晓阳
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021043092A1 publication Critical patent/WO2021043092A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures

Definitions

  • This application relates to the field of electronic technology, and more specifically, to an image semantic matching method, device, terminal, and computer-readable storage medium.
  • Semantic matching refers to the establishment of semantic correspondence between different object instances or scenes. Most of the semantic matching research focuses on images that have a certain relationship and have been paired.
  • This application provides an image semantic matching method, which includes:
  • the first feature tag set includes the feature tags of each subject in the first image
  • the second feature tag includes the feature tags of each subject in the second image.
  • the semantic matching relationship between the first image and the second image is determined based on the feature tags shared by the first feature tag set and the second feature tag set.
  • this application also provides a device, wherein the device includes:
  • the parsing module is used to analyze the first image and the second image to obtain a first feature tag set and a second feature tag set, the first feature tag set includes feature tags of each subject in the first image, and the first feature tag set
  • the second feature label includes the feature label of each subject in the second image
  • the matching module is configured to match each feature tag in the first feature tag set with each feature tag in the second feature tag set, and determine the features shared by the first feature tag set and the second feature tag set label;
  • the determining module is configured to determine the semantic matching relationship between the first image and the second image based on the feature tags shared by the first feature tag set and the second feature tag set.
  • this application also provides a terminal, including a processor, a memory, and a communication bus;
  • the communication bus is used to realize the connection and communication between the processor and the memory
  • the processor is used to execute one or more programs stored in the memory to implement the following steps:
  • the first feature tag set includes feature tags of each subject in the first image
  • the second feature tag includes all Describe the feature labels of each subject in the second image
  • the semantic matching relationship between the first image and the second image is determined based on the feature tags shared by the first feature tag set and the second feature tag set.
  • the present application also provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the following steps:
  • the first feature tag set includes feature tags of each subject in the first image
  • the second feature tag includes all Describe the feature labels of each subject in the second image
  • the semantic matching relationship between the first image and the second image is determined based on the feature tags shared by the first feature tag set and the second feature tag set.
  • FIG. 1 is a basic flowchart of an image semantic matching method provided by an embodiment of the application
  • FIG. 2 is a detailed flowchart of an image semantic matching method provided by an embodiment of the application
  • FIG. 3 is a schematic structural diagram of a terminal provided by an embodiment of the application.
  • Figure 1 is a basic flowchart of the image semantic matching method provided by this embodiment, which includes:
  • S101 Analyze the first image and the second image to obtain a first feature tag set and a second feature tag set, respectively.
  • step S101 the first feature tag set includes feature tags of each subject in the first image, and the second feature tag includes feature tags of each subject in the second image.
  • the subject in the image is conspicuous in the image and has contrast with the background.
  • the picture shown in an image shows a dog running on the grass.
  • the subject in this image is a running dog, and the background is the grass.
  • the feature label may include one or more of the position label, the pixel label, and the area label of each subject in the image.
  • the location tag is the position of the subject in the image;
  • the pixel tag includes the average brightness, grayscale, hue, hue, or color temperature of the subject in the image;
  • the area tag includes the subject’s Area, or the ratio of the subject to the total area of the image.
  • the first image and the second image can be images that have a certain relationship and have been paired, or they can be unrelated and unpaired images.
  • images that have a certain relationship and have been paired they can use existing images.
  • the semantic matching method realizes to determine whether two images have a semantic relationship; for unrelated and unpaired images, the existing semantic matching method will not be able to determine whether they have a semantic relationship. Based on this, this application proposes this image semantic matching method.
  • S102 Match each feature tag in the first feature tag set with each feature tag in the second feature tag set, and determine the feature tag shared by the first feature tag set and the second feature tag set.
  • the feature tags in the feature tag set are not chaotically recorded, but are recorded according to different subjects.
  • subject 1 and subject 2 are included in the first image
  • subject 1 is included in the feature label set corresponding to the first image.
  • step S102 the process of matching and determining the common feature label between the first feature tag set and the second feature tag set in step S102 may be:
  • the first image and the subject in the second image are used as the unit to match first, determine the subject shared by the first image and the second image, and then match the feature tags of the shared subject to determine the first feature The feature tag shared by the tag set and the second feature tag set.
  • the feature tag sets in the first feature tag set and the second feature tag set can be classified according to the types of feature tags, such as location tag set, pixel tag Then compare the position label set in the first feature label set and the location label set in the second feature label set respectively to determine the first feature label set and the second feature label set. Common feature tags.
  • S103 Determine a semantic matching relationship between the first image and the second image based on the feature tags shared by the first feature tag set and the second feature tag set.
  • the relationship between the number of shared feature tags and the number of preset tags can be determined to further determine whether the first image and the second image have a semantic relationship; it can also be determined by determining the first feature tag set and the second feature tag.
  • the feature tags shared by the collection include important feature tags of the first image, it is determined that the first image and the second image have a semantic relationship.
  • the present application provides an image semantic matching method, which can obtain a first feature tag set and a second feature tag set by analyzing unrelated and unpaired first images and second images, respectively, based on the first feature tag set and the first feature tag set.
  • the feature tags shared by the two feature tag sets can determine whether the first image and the second image have a semantic relationship.
  • FIG. 2 is a detailed flowchart of the image semantic matching method provided by the second embodiment of this application. The method includes:
  • the subject in the image is conspicuous in the image and has contrast with the background. At the same time, there can be multiple subjects in the image.
  • S202 Determine a feature label of each subject in the first image and the second image based on the subject in the first image and the second image.
  • the feature label may include one or more of the position label, the pixel label, and the area label of each subject in the image.
  • the location tag is the position of the subject in the image; the pixel tag includes the average brightness, grayscale, hue, hue, or color temperature of the subject in the image; the area tag includes the area of the subject, or the ratio of the subject to the total area of the image.
  • S203 Determine the feature label of each subject in the first image as the feature label set of the first image, and determine the feature label of each subject in the second image as the feature label set of the second image.
  • first feature tag set includes feature tags of each subject in the first image
  • second feature tag includes feature tags of each subject in the second image
  • S204 Match each feature tag in the first feature tag set with each feature tag in the second feature tag set, and determine a feature tag shared by the first feature tag set and the second feature tag set.
  • the feature tags in the feature tag set are not chaotically recorded, but are recorded according to different subjects.
  • the first image includes subject 1, subject 2, and the corresponding first image feature label set includes subject 1’s position label and pixel Label, area label, and position label, pixel label, and area label of body 2.
  • the method for determining the feature tags shared by the first feature tag set and the second feature tag set includes the following two methods.
  • the first method is used as an example to continue to introduce the present application.
  • S20412 Compare the feature labels of the subjects shared by the first image and the second image to obtain the feature labels of the shared subject, and use the feature labels as the feature labels shared by the first feature tag set and the second feature tag set.
  • the second type is the first type:
  • S20421 Classify each feature tag in the first feature tag set and each feature tag in the second feature tag set.
  • the feature labels include three types of location labels, pixel labels, and area labels.
  • Corresponding classification can obtain a location label set, a pixel label set, and an area label set.
  • S20422 Compare the feature tags of the same category in the first feature tag set and the second feature tag set respectively, and determine the feature tags shared by the first feature tag set and the second feature tag set.
  • the location tag set in the first feature tag set and each location tag in the location tag set in the second feature tag set are respectively compared to determine the feature tags shared by the first feature tag set and the second feature tag set.
  • S205 Determine a semantic matching relationship between the first image and the second image based on the feature tags shared by the first feature tag set and the second feature tag set.
  • the number of shared feature tags is greater than or equal to the preset number of tags, it is determined that the first image and the second image have a semantic relationship.
  • step S205 can also be implemented through the following two steps:
  • the important feature label is the set of feature labels corresponding to the important subjects in the image.
  • Important subjects can be the subject with the largest area in the image, the subject with the highest brightness, or the subjects in the fifth grid in the nine-square grid. .
  • determining the important feature label in the first feature label set of the first image can be determining the feature label of the subject with the largest image area in the first image as the important feature label; it can also be determining the subject with the highest brightness in the first image.
  • Each feature tag is an important feature tag; it can also be used to determine the feature tag of each subject in the fifth grid of the first image as an important feature tag.
  • the present application provides an image semantic matching method, which can obtain a first feature tag set and a second feature tag set by analyzing unrelated and unpaired first images and second images, respectively, based on the first feature tag set and the first feature tag set.
  • the feature tags shared by the two feature tag sets can determine whether the first image and the second image have a semantic relationship.
  • This embodiment also provides a device, wherein the device includes:
  • the parsing module is used to analyze the first image and the second image to obtain a first feature tag set and a second feature tag set, the first feature tag set includes feature tags of each subject in the first image, and the first feature tag set
  • the second feature label includes the feature label of each subject in the second image
  • the matching module is configured to match each feature tag in the first feature tag set with each feature tag in the second feature tag set, and determine the features shared by the first feature tag set and the second feature tag set label;
  • the determining module is configured to determine the semantic matching relationship between the first image and the second image based on the feature tags shared by the first feature tag set and the second feature tag set.
  • This embodiment also provides a terminal, as shown in FIG. 3, which includes a processor 31, a memory 32, and a communication bus 33, wherein:
  • the communication bus 33 is used to implement connection and communication between the processor 31 and the memory 32;
  • the processor 31 is configured to execute the image semantic matching program stored in the memory 32 to implement the steps of the image semantic matching method in each of the foregoing embodiments.
  • the memory 32 includes at least one type of readable storage medium, the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, magnetic disk, CD etc.
  • the memory 32 may be an internal storage unit of the terminal in some embodiments, such as a mobile hard disk of the terminal. In other embodiments, the memory 32 may also be an external storage device of the terminal, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), and a Secure Digital (SD) card equipped on the terminal. , Flash Card, etc. Further, the memory 32 may also include both an internal storage unit of the terminal and an external storage device.
  • the memory 32 can be used not only to store application software and various data installed in the terminal, such as codes for an intelligent sleep staging program, etc., but also to temporarily store data that has been output or will be output.
  • the processor 31 may be composed of integrated circuits in some embodiments, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same function or different functions, including one or more Combinations of central processing unit (CPU), microprocessor, digital processing chip, graphics processor, and various control chips, etc.
  • the processor 31 is the control core (Control Unit) of the terminal, which uses various interfaces and lines to connect the various components of the entire terminal, and runs or executes programs or modules stored in the memory 32 (for example, execute intelligent Sleep staging program, etc.), and call the data stored in the memory 32 to execute various functions of the terminal and process data.
  • Control Unit Control Unit
  • the communication bus 33 may be a peripheral component interconnect (PCI) bus or an extended industry standard structure (extended industry standard structure). industry standard architecture, EISA for short) bus, etc.
  • the communication bus 33 can be divided into an address bus, a data bus, a control bus, and the like.
  • the communication bus 33 is configured to implement connection and communication between the memory 32 and at least one processor 31 and the like.
  • This embodiment also provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above various implementations.
  • the computer-readable storage medium may be volatile or non-volatile, and the computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, Mobile hard disk, magnetic disk, CD, computer memory, read-only memory (ROM, Read-Only Memory).
  • An image semantic matching program is stored on the computer-readable storage medium, and the image semantic matching program can be executed by one or more processors to implement the above-mentioned image semantic matching method.
  • the image semantic matching program can be executed by one or more processors to implement the above-mentioned image semantic matching method.
  • the computer-readable storage medium For the specific implementation of the computer-readable storage medium, reference may be made to the description of the relevant steps in the embodiment corresponding to FIG. 1, which is not repeated here.
  • the disclosed terminal, device, and method may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
  • modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional modules in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé et un dispositif d'appariement sémantique d'images, un terminal et un support de stockage lisible par ordinateur, le procédé comprenant : l'analyse d'une première image et d'une seconde image pour obtenir respectivement un premier ensemble d'étiquettes de caractéristiques et un second ensemble d'étiquettes de caractéristiques ; l'appariement de chaque étiquette de caractéristiques du premier ensemble d'étiquettes de caractéristiques avec chaque étiquette de caractéristiques du second ensemble d'étiquettes de caractéristiques pour déterminer une étiquette de caractéristiques commune au premier ensemble d'étiquettes de caractéristiques et au second ensemble d'étiquettes de caractéristiques ; et la détermination d'une relation d'appariement sémantique entre la première image et la seconde image sur la base de l'étiquette de caractéristiques commune au premier ensemble d'étiquettes de caractéristiques et au second ensemble d'étiquettes de caractéristiques.
PCT/CN2020/112352 2019-09-02 2020-08-31 Procédé et dispositif d'appariement sémantique d'images, terminal et support de stockage lisible par ordinateur WO2021043092A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910824888.3 2019-09-02
CN201910824888.3A CN110633740B (zh) 2019-09-02 2019-09-02 一种图像语义匹配方法、终端及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2021043092A1 true WO2021043092A1 (fr) 2021-03-11

Family

ID=68969962

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/112352 WO2021043092A1 (fr) 2019-09-02 2020-08-31 Procédé et dispositif d'appariement sémantique d'images, terminal et support de stockage lisible par ordinateur

Country Status (2)

Country Link
CN (1) CN110633740B (fr)
WO (1) WO2021043092A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110633740B (zh) * 2019-09-02 2024-04-09 平安科技（深圳）有限公司 一种图像语义匹配方法、终端及计算机可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102598113A (zh) * 2009-06-30 2012-07-18 安芯美特控股有限公司 匹配出现在两个或多个图像内的对象或人的方法、电路和***
US9002831B1 (en) * 2011-06-10 2015-04-07 Google Inc. Query image search
CN107067030A (zh) * 2017-03-29 2017-08-18 北京小米移动软件有限公司 相似图片检测的方法和装置
CN109166261A (zh) * 2018-10-11 2019-01-08 平安科技（深圳）有限公司 基于图像识别的图像处理方法、装置、设备及存储介质
CN110633740A (zh) * 2019-09-02 2019-12-31 平安科技（深圳）有限公司 一种图像语义匹配方法、终端及计算机可读存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108304435B (zh) * 2017-09-08 2020-08-25 腾讯科技（深圳）有限公司 信息推荐方法、装置、计算机设备及存储介质
CN110059212A (zh) * 2019-03-16 2019-07-26 平安科技（深圳）有限公司 图像检索方法、装置、设备及计算机可读存储介质
CN109977253B (zh) * 2019-03-29 2022-10-28 哈尔滨工业大学 一种基于语义和内容的快速图像检索方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102598113A (zh) * 2009-06-30 2012-07-18 安芯美特控股有限公司 匹配出现在两个或多个图像内的对象或人的方法、电路和***
US9002831B1 (en) * 2011-06-10 2015-04-07 Google Inc. Query image search
CN107067030A (zh) * 2017-03-29 2017-08-18 北京小米移动软件有限公司 相似图片检测的方法和装置
CN109166261A (zh) * 2018-10-11 2019-01-08 平安科技（深圳）有限公司 基于图像识别的图像处理方法、装置、设备及存储介质
CN110633740A (zh) * 2019-09-02 2019-12-31 平安科技（深圳）有限公司 一种图像语义匹配方法、终端及计算机可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
XIONG CHANG-ZHEN , SHAN YAN-MEI , GUO FEN-HONG: "Image Retrieval Method based on Image Principal Part Detection", OPTICS AND PRECISION ENGINEERING, vol. 25, no. 3, 15 March 2017 (2017-03-15), pages 792 - 798, XP055787610, ISSN: 1004-924X *

Also Published As

Publication number Publication date
CN110633740B (zh) 2024-04-09
CN110633740A (zh) 2019-12-31

Similar Documents

Publication Publication Date Title
US11886492B2 (en) Method of matching image and apparatus thereof, device, medium and program product
WO2019019595A1 (fr) Procédé de mise en correspondance d'image, procédé de dispositif électronique, appareil, dispositif électronique et support
CN108961267B (zh) 图片处理方法、图片处理装置及终端设备
US20220100576A1 (en) Video processing method and device, electronic equipment and storage medium
WO2019096177A1 (fr) Procédé et système de reconnaissance d'image, et dispositif électronique
CN109271611B (zh) 一种数据校验方法、装置及电子设备
WO2020083019A1 (fr) Procédé de décodage basé sur un processeur multicœur, dispositif terminal et support d'informations
CN111273891A (zh) 基于规则引擎的业务决策方法、装置及终端设备
WO2019000746A1 (fr) Procédé permettant de modifier la couleur du texte d'une commande, système, dispositif électronique et support d'informations
WO2021043092A1 (fr) Procédé et dispositif d'appariement sémantique d'images, terminal et support de stockage lisible par ordinateur
WO2021042895A1 (fr) Procédé et système d'identification de code de vérification sur la base d'un réseau neuronal, et dispositif informatique
CN115357155A (zh) 窗口识别方法、装置、设备及计算机可读存储介质
CN113762303B (zh) 图像分类方法、装置、电子设备及存储介质
CN112528610B (zh) 一种数据标注方法、装置、电子设备及存储介质
US11341076B2 (en) Hot-plugged PCIe device configuration system
US7069206B2 (en) Method and apparatus for abstraction of physical hardware implementation to logical software drivers
TWI401613B (zh) 防止顯示儲存在訊框緩衝器中多餘內容之系統、方法與電腦程式產品
CN113283351A (zh) 一种使用cnn优化相似度矩阵的视频抄袭检测方法
CN111813485A (zh) 页面布局的处理方法、装置、设备及计算机可读存储介质
CN115687783A (zh) 业务应用的推送方法及装置、存储介质、计算机设备
CN113391779A (zh) 类纸屏幕的参数调节方法、装置及设备
WO2020082354A1 (fr) Procédé de détection d'état de système, appareil de détection d'état de système et dispositif terminal
CN111091488B (zh) 基于OpenCV的内存管理方法、装置及终端
CN110647946B (zh) 一种画中画的检测方法、装置和计算机存储介质
CN111881073B (zh) 一种fpga兼容多种硬件的方法、装置及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20861221

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20861221

Country of ref document: EP

Kind code of ref document: A1