KR101376596B1

KR101376596B1 - System and method for searching images

Info

Publication number: KR101376596B1
Application number: KR1020120021856A
Authority: KR
Inventors: 박시몽; 이주헌
Original assignee: 주식회사 내비오닉스코리아
Priority date: 2012-03-02
Filing date: 2012-03-02
Publication date: 2014-03-27
Also published as: KR20130100550A

Abstract

The present invention relates to an image retrieval apparatus and method, comprising: a memory for storing a name and a standard image of a search object; 1 include a standard image larger than the reference value of the search condition in the correct answer set of the first search condition, compare the search image and the standard image with respect to the second search condition, and calculate the similarity, and the similarity is greater than the reference value of the second search condition. And a processing unit for including the standard image in the correct answer set of the second search condition.

Description

Image retrieval system and method {SYSTEM AND METHOD FOR SEARCHING IMAGES}

The present invention relates to an image retrieval system and method, and more particularly, to an image retrieval system and method for searching for and providing an image similar to the image input by the user to the user.

According to the image retrieval method using a condition, when a user inputs a search image, the image retrieval apparatus simply substitutes various conditions to narrow down the search range and finally shows an image similar to the search image or the name of a similar image. To the user. In other words, this condition search method is a method of searching for an answer that satisfies all conditions by comparing the search images one by one with the images stored in the database. This method is very efficient, but if you do not find a result that meets all the search conditions, you may not get the results you want in the form of 'no results found.' In this case, as an alternative, the image retrieval apparatus may search for and present the result closest to the condition desired by the user. However, even then, depending on the arrangement of the search conditions, the result may be different or incorrect from the desired result.

For example, if a user wants to search for the exact name of a plant with the leaves he or she found, he or she would like to know the leaf shape (search condition 1), leaf color (search condition 2), leaf vein (search condition 3) and The same search condition can be used. Suppose a given leaf is actually a ginkgo leaf, but its edges are broken, so the palms are open. In the conventional image search method, only plants of the shape of Guanyin bamboo are found based on the shape of the leaves, and then the color and leaf veins of the leaves are matched in order. In this case, it is impossible to determine the exact name of the search image due to the shape of the damaged leaves. Moreover, even if the most similar plants are found, the ginkgo tree is already excluded from the correct set by the first search condition. Even if you change the order of the search conditions, the result is not much different.

Accordingly, an object of the present invention is to provide an image retrieval apparatus and method that can compensate for the limitations of the existing condition search and increase the ratio of the user's desired result, that is, the reliability of the search.

An image retrieval apparatus according to an embodiment of the present invention for solving this problem, the storage unit for storing the name and standard image of the search object, and the search image and the standard image input from the outside for the first search condition Compare and calculate a first similarity, and include a standard image having the first similarity greater than the first reference value of the first search condition in the first correctable set of the first search condition, and searching for the second search condition. Comparing the image with the standard image to calculate a second similarity, and including the standard image having the second similarity greater than the first reference value of the second search condition in the first correctable set of the second search condition; A third similarity with respect to the standard image included in the first correctable set of one search condition compared with the search image with the second search condition; Calculated, and with respect to the standard image in the first set of possible answers and the second criteria comprises a processor for calculating a fourth degree of similarity in comparison with the image search in the first search.

The processor includes the standard image having the third similarity greater than the second reference value of the second search condition in the second correctable set of the first search condition, and wherein the fourth similarity is the second of the first search condition. A standard image larger than a reference value may be included in the second correctable set of the second search conditions.

The processor may be configured to multiply a first weight value of the first search condition by the first similarity to a standard image included in the second correctable set of the first search condition, and to generate a second value for the second search condition. A final similarity level for the standard image may be calculated by adding a weight multiplied by the third similarity level.

The first and second weights and the first and second reference values of the first and second search conditions may be set.

According to another embodiment of the present invention, an image retrieval system connected to a user terminal through a communication network includes a database storing a name of a search target and a standard image, and a search image input from the user terminal with respect to a first search condition. Comparing the standard images to calculate a first similarity, including a standard image having the first similarity greater than a first reference value of the first search condition in the first correctable set of the first search condition, and a second search condition Compares the search image and the standard image with respect to a second similarity, and includes a standard image having the second similarity greater than a first reference value of the second search condition in the first correctable set of the second search condition. And the second search condition with respect to the standard image included in the first correct answer set of the first search condition. A third similarity is calculated by comparing with the searched image, and a fourth similarity is calculated by comparing the searched image with the first search condition with respect to the standard image included in the first correct answer set of the second search condition. Include a server.

The server includes the standard image having the third similarity greater than the second reference value of the second search condition in the second correctable set of the first search condition, and wherein the fourth similarity is the second of the first search condition. A standard image larger than a reference value may be included in the second correctable set of the second search conditions.

The server multiplies the first similarity by the first weight for the first search condition with respect to the standard image included in the second correct answer set of the first search condition and the second value for the second search condition. A final similarity level for the standard image may be calculated by adding a weight multiplied by the third similarity level.

The image search method according to another embodiment of the present invention may include: storing a name of a search target and a standard image, comparing a search image input from the outside with respect to a first search condition, and the standard image to calculate a first similarity; The standard image having the first similarity greater than the first reference value of the first search condition is included in the first correctable set of the first search condition, and the search image is compared with the standard image for a second search condition. Calculating a second similarity and including a standard image having a second similarity greater than a first reference value of the second search condition in the first correctable set of the second search condition, and a first of the first search condition Comparing the standard image included in the answer possible set with the search image by the second search condition, a third similarity is calculated, and the second search is performed. With respect to the standard image in the first set of possible answer condition and the step of calculating a fourth degree of similarity in comparison with the image search in the first search.

The standard image having the third similarity greater than the second reference value of the second search condition is included in the second correctable set of the first search condition, and the fourth similarity is greater than the second reference value of the first search condition. The method may further include including a standard image in the second correct answer set of the second search condition.

The first weight for the first search condition is multiplied by the first similarity and the second weight for the second search condition for the standard image included in the second correct answer set of the first search condition. The method may further include calculating a final similarity degree for the standard image by adding a value multiplied by a third similarity degree.

A computer-readable medium according to another embodiment of the present invention records a program for executing any one of the above methods.

According to the image retrieval apparatus and method according to an embodiment of the present invention, even if no result is completely matched with the search image, it is possible to provide various possible results closest to the search image. As a result, the reliability of the search can be increased, and thus the user can provide a desired result.

1 is a block diagram of an image retrieval apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating an image retrieval method according to another embodiment of the present invention.
3 is a schematic diagram illustrating an image retrieval method according to an embodiment of the present invention.
4 is a block diagram illustrating an image search system according to another exemplary embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

1 is a block diagram of an image retrieval apparatus according to an embodiment of the present invention. The image retrieval apparatus 100 may be, for example, a desktop computer, a laptop computer, a workstation, a laptop computer, a palmtop computer, an ultra mobile personal computer, a tablet PC, a personal digital assistant, a PDA. ), A web pad, a mobile phone, a smart phone, an electronic notebook, an MP3 player, a portable multimedia player (PMP), or the like, may correspond to a computer or a portable information device, but is not limited thereto.

Referring to FIG. 1, the image retrieval apparatus 100 includes an input unit 110, a processing unit 120, an output unit 130, and a storage unit 130.

The input unit 110 transmits data into the image retrieval apparatus 100 according to a user input from the outside. For example, the input unit 110 may be used to input a text or a search image, and may include a touch pad, a touch screen, a keyboard, or a mouse.

The processing unit 120 performs an operation of executing an instruction and generating or using data. For example, the processor 120 may process input and output data between the components of the image retrieval apparatus 100, and may interpret input information such as a search image received from the input unit 110, thereby analyzing an image or a search accordingly. Information corresponding to the image may be displayed on the output unit 130. The processing unit 120 may be implemented on a single chip, a plurality of chips or a plurality of electrical components, for example, a dedicated or embedded processor, a single purpose processor, a controller, or an application specific integrated circuit (ASIC).

The storage unit 140 provides a place for storing executable code and data used by the image retrieval apparatus 100. The storage unit 140 stores data in response to a request from the processing unit 120, and provides the processing unit 120 with the command and / or data. For example, the storage unit 140 stores the name of the object to be searched, the standard image and information about the object. The storage unit 140 may include a read-only memory (ROM), a random access memory (RAM), a hard disk drive, a memory card, a PC card, a CD-ROM, a floppy disk,

The output unit 130 outputs a variety of information and provides the information to a user, for example, a liquid crystal display, a cathode-ray tube, an organic light emitting diode, a plasma display monitor, or the like. Can be made.

The image retrieval apparatus 100 may further include a photographing unit (not shown) as necessary. The photographing unit includes a camera that acquires an external real world as an image, and transmits data obtained by performing appropriate processing on the acquired image to the processing unit 120.

Next, an image search method according to another embodiment of the present invention will be described in detail with reference to FIGS. 2 and 3. This image retrieval method will be described as being performed in the image retrieval apparatus 100 of FIG. However, the present invention is not limited thereto, and the image retrieval method may be applied to any device or system requiring image retrieval as well as the image retrieval apparatus 100 of FIG.

2 is a flowchart illustrating an image retrieval method according to another embodiment of the present invention, and FIG. 3 is a schematic diagram illustrating an image retrieval method according to an embodiment of the present invention.

First, when the operation is started, a search image is input from the user (S210). The object of the search image may be natural objects such as leaves, flowers, petals, trees, herbs, jewelry, or artificial objects such as buildings, cars, and airplanes. The search target may be predetermined in the image search apparatus 100, but the user may input the search target in the image search apparatus 100 while the user inputs the search image.

The image search apparatus 100 searches for the input search image according to each search condition (S220). Here, the search conditions may be plural (n), and the contents of the search conditions may be characteristics of the object, for example, color, shape, shape, size, and the like.

First, at the first level, the search image is individually searched for all the standard images related to the object for each of the first to nth search conditions. That is, for each search condition, the search image is compared with the standard images related to the search object stored in the storage unit 140 and the similarity is calculated. Then, the calculated similarity is compared with the reference value of the corresponding search condition and a standard image having a similarity greater than the reference value is included in the correct answer set of the corresponding search condition. This generates n correctable sets, each corresponding to n search conditions. Here, the reference value of each search condition may be set individually.

Then, in the second level, the searched image is searched using the extracted image according to the search result of the upper level (S230). That is, in this case, the image included in the correct answer set of each search condition extracted at the first level is compared with the search image and the similarity is calculated. At this time, the search condition at the first level need not be used again at the second level. For example, an image included in the first correctable set extracted according to the first search condition of the first level may be searched for the second to n th search conditions. The similarity calculated at the second level is compared with the reference value of the corresponding search condition, and an image having a similarity greater than the reference value is included in the correct answer set of the corresponding search condition at the second level. The reference value at the second level may be set differently from the reference value at the first level.

In this manner, step S230 is repeated to a predetermined level in this manner (S240), and the standard image included in the finally obtained correctable set and information related thereto are output (S250). In this case, since the standard images included in the final correctable set have similarities calculated for each search condition, multiplying the weights assigned to each search condition by the corresponding similarities and adding these values may calculate the final similarity for this standard image. . Based on this final similarity, the standard image and related information can be output to the user.

As an example, the search object may be a leaf, and the search conditions may be three leaf shapes (first search condition), leaf color (second search condition), and leaf vein (third search condition). 2.

When the user enters a leaf image, the entire standard image associated with the leaf is compared in terms of the shape of the leaf and the similarity is calculated. If the similarity calculated for one standard image is greater than the leaf-likeness similarity value, this standard image is included in the leaf-shaped correctable set. Similarly, for the leaf color and leaf vein, the similarity is calculated for the entire standard image and the correct set is extracted. This reference value may be set differently for each search condition and may be set differently for each level.

By applying and searching different search conditions on the three correct sets, the second correct set is created. That is, when the leaf vein search condition is applied to the first correct set of leaf colors, the second correct set is possible. When the leaf shape search condition is applied to the first correct set of leaf colors, another second correct set is formed.

For example, if the set weights are 30% leaf shape, 25% leaf color, and 45% leaf vein, respectively, for the images included in the second correctable set, the weights are multiplied by the similarity calculated in each search condition. The sum yields a final similarity, which is the similarity for the standard image and the search image.

As described above, if each search condition is searched in parallel, even if the search term has a very low similarity in at least one search condition, if the search term has a high similarity in other search conditions, it is unlikely to be excluded from the correct answer set. In other words, even if there is no result matching 100% of the search results, at least various possible answers can be provided. For example, if a ginkgo leaf whose leaf shape is damaged is entered as a search image, the leaf shape similarity is very low compared to the normal undamaged ginkgo leaf, but since the leaf color and leaf vein have high similarity, the normal ginkgo leaf may be included in the final correct set. have. On the other hand, according to the conventional serial image search method that simply substitutes several search conditions in succession to narrow down the search range, since the leaf similarity is very low, the normal ginkgo leaf image may be excluded from the search results.

Using such a parallel search, an operator or user who sets a search condition may arbitrarily determine an indexing formula using a heuristic. That is, it is possible to set the similarity reference value, the weight for each search condition, the content of the search condition, the number of search conditions, the search level, etc., whether or not they are included in the correct answer set. Can be found.

Next, an image search system according to another embodiment of the present invention will be described in detail with reference to FIG. 4.

4 is a block diagram illustrating an image search system according to another exemplary embodiment of the present invention.

The image retrieval system according to another embodiment of the present invention includes a server 420 and a database 440 connected thereto, and is connected to the user terminal 410 through a communication network 430.

The user terminal 410 may be a portable information device such as a computer or a mobile phone, a smartphone, a tablet, and the like. The user terminal 410 may use an image input from the outside as a search image, and use an image photographed by a camera included in the user terminal 410 as a search image.

The communication network 430 may include a telephone network as well as a data communication network including a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), the Internet, and the like. It can be used, and it does not matter wired and wireless, and it does not matter which communication method is used.

The database 440 stores a search target, a search target image, and information on the search target image.

The server 420 receives a search image at the request of the user terminal 410 and performs an image search on the search image by the image search method illustrated in FIG. 2. Since details are substantially the same as those described above, a detailed description thereof will be omitted.

Other embodiments of the invention include a computer readable medium having program instructions for performing various computer implemented operations. This medium records a program for executing the image retrieval method described so far. The medium may include program instructions, data files, data structures, etc., alone or in combination. Examples of such media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD and DVD, programmed instructions such as Floptical Disk and magneto-optical media, ROM, And a hardware device configured to store and execute the program. Or such medium may be a transmission medium, such as optical or metal lines, waveguides, etc., including a carrier wave that transmits a signal specifying a program command, data structure, or the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, And falls within the scope of the invention.

100: image search device, 110: input unit,
120: processing unit, 130: output unit,
140: a storage unit, 410: a user terminal,
420: server, 430: network,
440: database

Claims

A memory for storing the name and standard image of the search object, and
Comparing the search image and the standard image for each of the first to nth search conditions related to the features of the search image input from the outside at the first level, the similarity is calculated and the similarity is calculated as the reference value of the corresponding search condition. And extract a standard image having a similarity greater than the reference value to generate a correct answer set of the corresponding search condition, and extract each correct answer set of the first to nth search conditions extracted at the first level at the second level. Similarity is obtained by comparing the search image with the image included in the correct answer set of the first to nth search conditions extracted at the first level for each of the search conditions except for the search condition performed at the first level. Calculate the correctable set of each search condition at the second level after calculating The similarity is obtained by comparing the search image with the image included in the correct answer set of the higher level for each search condition except for the search condition performed at the higher level for each of the higher level correctable sets at each level. A processor for generating a correctable set at each level after calculating
The image search apparatus comprising:

In claim 1,
And the processor compares the similarity calculated at each level with reference values of the first to nth search conditions, and generates an image having a similarity greater than the reference value as a correctable set of each level.

In claim 1,
The processor calculates a final similarity by adding values obtained by multiplying the similarity calculated according to each search condition and the weights assigned to the respective search conditions with respect to the image included in the final correctable set calculated at the preset level. Search device.

4. The method of claim 3,
And the reference value and the weight can be set for each search condition.

An image retrieval system connected with a user terminal through a communication network,
A database that stores the name and standard image of the search object, and
Comparing the search image and the standard image for each of the first to nth search conditions related to the features of the search image input from the outside at the first level, the similarity is calculated and the similarity is calculated as the reference value of the corresponding search condition. And extract a standard image having a similarity greater than the reference value to generate a correct answer set of the corresponding search condition, and extract each correct answer set of the first to nth search conditions extracted at the first level at the second level. Similarity is obtained by comparing the search image with the image included in the correct answer set of the first to nth search conditions extracted at the first level for each of the search conditions except for the search condition performed at the first level. Calculate the correctable set of each search condition at the second level after calculating The similarity is obtained by comparing the search image with the image included in the correct answer set of the higher level for each search condition except for the search condition performed at the higher level for each of the higher level correctable sets at each level. A server for generating a correctable set at each level after calculating
Image search system comprising a.

The method of claim 5,
And the server compares the similarity calculated at each level with reference values of the first to nth search conditions and generates an image having a similarity greater than the reference value as a correctable set of each level.

The method of claim 5,
The server calculates a final similarity by adding values obtained by multiplying the similarity calculated according to each search condition and the weights assigned to the respective search conditions with respect to the image included in the final correctable set calculated at the preset level. Search system.

In claim 7,
And the reference value and the weight can be set for each search condition.

An image search method of an image search device,
Storing, by the image retrieval device, a name and a standard image of a search target;
The image retrieval apparatus compares the search image and the standard image with respect to each of the first to nth search conditions related to the features of the search image input from the outside at the first level, calculates a similarity, and then calculates the similarity. Comparing the reference value of the search condition and extracting a standard image having a similarity greater than the reference value to generate a correctable set of the search condition;
The image retrieval apparatus may be configured for each of the search conditions except for the search condition performed at the first level for each of the correctable sets of the first to nth search conditions extracted at the first level at the second level. Comparing the searched image with the image included in the correctable set of the first to nth search conditions extracted at one level, calculating similarity, and generating a correctable set of each search condition at the second level. , And
An image included in the higher level correctable set for each of the search conditions except for the search condition performed at the higher level for each of the higher level correctable sets at each level up to a preset level. Comparing the searched images with the searched images to calculate similarity, and generating a correctable set at each level.
/ RTI >

The method of claim 9,
The generating of the correctable set at each level may include comparing the similarity calculated at each level with a reference value of the first to nth search conditions and displaying an image having a similarity greater than the reference value. Image search method comprising the step of generating with.

The method of claim 9,
The image retrieval device calculates a final similarity by adding values obtained by multiplying the similarity calculated according to the respective search conditions and the weights assigned to the respective search conditions to the images included in the final correctable set calculated at the preset level. The image search method further comprising the step.

12. The method of claim 11,
The reference value and the weight can be set for each of the search conditions.

A computer readable medium having recorded thereon a program for executing the method of claim 9.