CN109101992A - Image matching method, device and computer readable storage medium - Google Patents

Image matching method, device and computer readable storage medium Download PDF

Info

Publication number
CN109101992A
CN109101992A CN201810728913.3A CN201810728913A CN109101992A CN 109101992 A CN109101992 A CN 109101992A CN 201810728913 A CN201810728913 A CN 201810728913A CN 109101992 A CN109101992 A CN 109101992A
Authority
CN
China
Prior art keywords
image
characteristic
network
training
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810728913.3A
Other languages
Chinese (zh)
Other versions
CN109101992B (en
Inventor
杨磊
张行程
林达华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sensetime Technology Development Co Ltd
Original Assignee
Beijing Sensetime Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sensetime Technology Development Co Ltd filed Critical Beijing Sensetime Technology Development Co Ltd
Priority to CN201810728913.3A priority Critical patent/CN109101992B/en
Publication of CN109101992A publication Critical patent/CN109101992A/en
Application granted granted Critical
Publication of CN109101992B publication Critical patent/CN109101992B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

This application discloses a kind of image matching method, device and computer readable storage mediums.This method comprises: obtaining the characteristic of the first image；Characteristic based on the first image, obtains M group target signature data, and the M is the integer more than or equal to 2；Based on the M group target signature data, the determining and matched target image of the first image from least two second images.Correspondingly, additionally providing corresponding device.Using the application, the precision of images match can be improved.

Description

Image matching method, device and computer readable storage medium
Technical field
This application involves field of computer technology more particularly to a kind of image matching method, device and computer-readable deposit Storage media.
Background technique
It refers to going to match the data in another mode with the data in a mode across mode vectors correlation.Such as: with element Draw the photo of matching camera shooting or the photo of infrared camera shooting.Wherein, sketch images are just equivalent to weak mode, photo just phase When in strong mode.How to realize be across mode vectors correlation this field research hotspot.
Summary of the invention
The application provides a kind of image matching method, device and computer readable storage medium, and images match can be improved Accuracy.
In a first aspect, the embodiment of the present application provides a kind of image matching method, comprising:
Obtain the characteristic of the first image；
Characteristic based on the first image obtains M group target signature data, wherein the M be more than or equal to 2 integer；
Based on the M group target signature data, determination is matched with the first image from least two second images Target image.
In the embodiment of the present application, by obtaining the characteristic of the first image, thus the characteristic based on first image According to, obtain M group target signature data, and then be based on the M group target signature data, determined from least two second images and The target image of first images match.Implement the embodiment of the present application, is determined based on M group target signature data and the first image The accuracy and accuracy of determining target image can be improved in the target image matched.
It in one possible implementation, can be based on the M group target signature data and described at least two the The characteristic of each second image in two images, determination is matched with the first image from least two second image Target image.
In one possible implementation, the dimension of the characteristic of second image is different from the first image Characteristic dimension, the dimensions of the target signature data is equal to the dimension of the characteristic of second image.
In one possible implementation, the dimension of the characteristic of second image is greater than the first image The dimension of characteristic.
In one possible implementation, the dimension of the target signature data is greater than the characteristic of the first image According to dimension.
In one possible implementation, the method also includes: obtain each at least two second image The characteristic of second image.
In some instances, feature extraction processing can be carried out to the second image each at least two second images, obtained To the characteristic of each second image.
In other examples, the characteristic of each second image at least two second images can be obtained from memory According to.
In one possible implementation, described to be based on the M group target signature data, from least two second images Middle determination and the matched target image of the first image, comprising:
Based on the M group target signature data, determine each in the first image and at least two second image The similarity of second image；
Based on the similarity of each second image in the first image and at least two second image, from it is described to The determining and matched target image of the first image in few two images.
In the embodiment of the present application, after obtaining M group target signature data, by the M group target signature data, first is determined The similarity of each second image, is conducive to effectively improve the accurate of determining similarity in image and at least two second images Degree improves the efficiency for determining target image.
In one possible implementation, it is described be based on the M group target signature data, determine the first image and The similarity of each second image at least two second image, comprising:
Obtain the characteristic of each second image at least two second image；
Characteristic based on each second image in the M group target signature data and at least two second image According to obtaining the corresponding M similarity of each second image；
According to the corresponding M similarity of the second image each at least two second image, first figure is determined As the similarity with each second image.
In the embodiment of the present application, a kind of method of determining similarity is provided, M group target signature data can be based on, obtained The first figure is determined to the corresponding M similarity of each second image, and then according to the corresponding M similarity of each second image As the similarity with each second image.
In one possible implementation, described corresponding according to the second image each at least two second image M similarity, determine the similarity of the first image Yu each second image, comprising:
The average treatment result of the corresponding M similarity of second image is determined as the first image and described the The similarity of two images.
In the embodiment of the present application, the average treatment result of the corresponding M similarity of the second image can include: second image The arithmetic mean result of the result of weighted average of corresponding M similarity or the corresponding M similarity of second image.
In one possible implementation, the information content that the first image includes is less than second image Information content.
Image matching method provided by the embodiment of the present application can be applied to the matching between different modalities, such as not only may be used Applied to comprising between comparable two mode of information content；The information content for applying also for a mode is considerably less than another mould The information content of state.Thus implement the embodiment of the present application, can effectively make up since the information content of the first image is very few, and lead to first Huge estrangement when mode vectors correlation is carried out between image and the second image, it is effective to improve the accuracy for determining target image.
In one possible implementation, the characteristic for obtaining the first image, comprising:
The characteristic of the first image is obtained from memory.
In the embodiment of the present application, memory includes the memory of image matching apparatus.That is, the spy of first image Sign data have obtained, and are stored in memory.The time for obtaining the characteristic of the first image can be saved as a result, and raising obtains Take efficiency.
In one possible implementation, the characteristic based on the first image obtains M group target signature Data, comprising:
Characteristic and M random vector based on the first image obtain the M group target signature data.
In the embodiment of the present application, different random vectors is inputted, different target signature data can be obtained.That is, It can be using the first image as condition, using random vector as variable, so that a series of possible target signature data are obtained, thus to the greatest extent More target signature data are likely to be obtained, the accuracy for calculating similarity is improved.
In one possible implementation, the characteristic based on the first image obtains M group target signature Data, comprising:
The characteristic of the first image is input to feature generation network to handle, it is special to obtain the M group target Levy data.
It, can also be defeated by different random vector difference using the characteristic of the first image as condition in the embodiment of the present application Enter to feature and generate network, to obtain M group target signature data.Such as by the characteristic of weak modality images and different random Vector is separately input into the feature after training and generates network, and the characteristic of the strong modality images of different puppets can be obtained.
In one possible implementation, by the characteristic of the first image be input to feature generate network into Row processing, before obtaining the M group target signature data, the method also includes:
The characteristic of first training sample is input to the feature and generates network, obtains target training characteristics data, Wherein, the dimension of the target training characteristics data is greater than the dimension of the characteristic of first training sample；
The target training characteristics data are inputted and differentiate that network is handled, obtain the first differentiation result；
Differentiated based on described first as a result, determining first-loss；
Based on the first-loss, the training feature generates network.
In the embodiment of the present application, image matching apparatus generates network by feature and exports target training characteristics data, thus The target training characteristics data are input to differentiation network, makes to differentiate that network is differentiated, such as differentiates the target training characteristics number According to belonging to strong mould probability of state or score etc.；And then based on differentiate network output first differentiate result determine first-loss come Training characteristics generate network, generate network by confrontation mode come training characteristics, can effectively improve trained efficiency, step up The accuracy of target training characteristics data.
In one possible implementation, the characteristic by the first training sample is input to the feature and generates Network obtains target training characteristics data, comprising:
The characteristic of first training sample and training random vector are input to the feature and generate network, is obtained The target training characteristics data；
Differentiated based on described first as a result, determining first-loss, comprising:
Target training characteristics data input sorter network is handled, the first classification results are obtained；
Target training characteristics data input random vector Recurrent networks are handled, training is obtained and returns feature, Wherein, the dimension that the training returns feature is equal to the dimension for inputting the trained random vector；
Differentiate that result, first classification results and the training return feature based on described first, determines described first Loss.
In the embodiment of the present application, which is inputted into sorter network, by the sorter network to the mesh Mark training characteristics data exercise supervision, and can further improve the accuracy of target training characteristics number；By the way that target training is special It levies data and inputs random vector Recurrent networks, obtain training and return feature, to not only can effectively supervise the target training characteristics The case where data can also further efficiently use random vector, avoid having input random vector, can not efficiently use.
In one possible implementation, the characteristic for obtaining the first image, comprising:
Network is extracted using fisrt feature, feature extraction is carried out to the first image, obtain the feature of the first image Data；
The method also includes:
According to the first-loss, the training fisrt feature extracts network.
In the embodiment of the present application, which, which extracts network, can also be used in the characteristic for extracting the first training sample.
In one possible implementation, the method also includes:
The characteristic of second training sample is input to the differentiation network to handle, obtain the second differentiation as a result, Wherein, the information content for including in first training sample that contains much information for including in second training sample；
Differentiate that result and the training return feature based on described second, determines the second loss；
Based on second loss, the training differentiation network, the sorter network and the random vector Recurrent networks.
In one possible implementation, described to differentiate that result and the training return feature based on described second, really Fixed second loss, comprising:
The characteristic of second training sample is inputted the sorter network to handle, obtains the second classification knot Fruit；
Differentiate that result, second classification results and the training return feature based on described second, determines described second Loss.
In one possible implementation, the characteristic of each second image at least two second image is obtained According to, comprising:
Network is extracted using second feature, and feature extraction is carried out to the second image each at least two second image, Obtain the characteristic of each second image at least two second image；
The method also includes:
Based on second loss, the training second feature extracts network.
Second aspect, the embodiment of the present application provide a kind of image matching apparatus, comprising:
Acquiring unit, for obtaining the characteristic of the first image；
First data processing unit obtains M group target signature data for the characteristic based on the first image； Wherein, the M is the integer more than or equal to 2；
First determination unit, for being based on the M group target signature data, the determining and institute from least two second images State the target image of the first images match.
In one possible implementation, first determination unit can specifically be based on the M group target signature number The characteristic of each second image accordingly and at least two second image, from least two second image really The fixed and matched target image of the first image.
In one possible implementation, the dimension of the characteristic of second image is different from the first image Characteristic dimension, the dimensions of the target signature data is equal to the dimension of the characteristic of second image.
In one possible implementation, the dimension of the characteristic of second image is greater than the first image The dimension of characteristic.
In one possible implementation, the dimension of the target signature data is greater than the characteristic of the first image According to dimension.
In one possible implementation, the acquiring unit is also used to obtain at least two second image The characteristic of each second image.
In some instances, which can propose the second image each at least two second images progress feature Processing is taken, the characteristic of each second image is obtained.
In other examples, acquiring unit can obtain each second image at least two second images from memory Characteristic.
In one possible implementation, first determination unit includes:
First determine subelement, for be based on the M group target signature data, determine the first image with it is described at least The similarity of each second image in two the second images；
Second determines subelement, for based on each second figure in the first image and at least two second image The similarity of picture, the determining and matched target image of the first image from least two image.
In one possible implementation, it described first determines subelement, is specifically used for obtaining described at least two the The characteristic of each second image in two images；And it is based on the M group target signature data and described at least two second The characteristic of each second image in image obtains the corresponding M similarity of each second image；And according to described The corresponding M similarity of each second image, determines the first image and described each second at least two second images The similarity of image.
In one possible implementation, it described first determines subelement, is specifically used for second image is corresponding The average treatment result of M similarity be determined as the similarity of the first image Yu second image.
In one possible implementation, the information content that the first image includes is less than second image Information content.
In one possible implementation, the acquiring unit, specifically for obtaining first figure from memory The characteristic of picture.
In one possible implementation, first data processing unit is specifically used for being based on the first image Characteristic and M random vector, obtain the M group target signature data.
In one possible implementation, first data processing unit, specifically for by the first image Characteristic is input to feature generation network and is handled, and obtains the M group target signature data.
In one possible implementation, described device further include:
Second data processing unit generates network for the characteristic of the first training sample to be input to the feature, Obtain target training characteristics data；
First judgement unit differentiates that network is handled for inputting the target training characteristics data, obtains first Differentiate result；
Second determination unit, for differentiating based on described first as a result, determining first-loss；
First training unit, for being based on the first-loss, the training feature generates network.
In one possible implementation, second data processing unit is specifically used for the first training sample This characteristic and training random vector is input to the feature and generates network, obtains the target training characteristics data；
Second determination unit includes:
Classification processing subelement obtains for handling target training characteristics data input sorter network One classification results；
Processing subelement is returned, for inputting the target training characteristics data at random vector Recurrent networks Reason obtains training and returns feature, wherein the dimension that the training returns feature is equal to the dimension for inputting the trained random vector Degree；
Third determines subelement, for differentiating that result, first classification results and the training are returned based on described first Return feature, determines the first-loss.
In one possible implementation, the acquiring unit is specifically used for extracting network to institute using fisrt feature It states the first image and carries out feature extraction, obtain the characteristic of the first image；
First training unit is also used to according to the first-loss, and the training fisrt feature extracts network.
In one possible implementation, described device further include:
Second judgement unit is handled for the characteristic of the second training sample to be input to the differentiation network, Obtain the second differentiation result, wherein containing much information of including in second training sample include in first training sample Information content；
Third determination unit determines the second loss for differentiating that result and the training return feature based on described second；
Second training unit, for based on second loss, the training differentiation network, the sorter network and described Random vector Recurrent networks.
In one possible implementation, the third determination unit, specifically for by second training sample Characteristic inputs the sorter network and is handled, and obtains the second classification results；And result, institute are differentiated based on described second It states the second classification results and the training returns feature, determine second loss.
In one possible implementation, the acquiring unit is specifically used for extracting network to institute using second feature It states each second image at least two second images and carries out feature extraction, obtain each at least two second image The characteristic of two images；
Second training unit is also used to based on second loss, and the training second feature extracts network.
The third aspect, the embodiment of the present application provide a kind of image matching apparatus, comprising: processor and memory；It is described to deposit Reservoir is used to couple with the processor, and program instruction and data needed for preservation described image coalignment；The place Reason device is configured as the corresponding function in the method for supporting described image coalignment to execute first aspect.
In one possible implementation, described image coalignment can also include input/output interface, described defeated Enter output interface, for supporting the communication between described device and other devices.
Fourth aspect, the embodiment of the present application provide a kind of computer readable storage medium, the computer-readable storage Readable instruction is stored in medium, when run on a computer, so that computer executes method described in the various aspects.
5th aspect, the embodiment of the present application provides a kind of computer program product comprising instruction, when it is in computer When upper operation, so that computer executes method described in the various aspects.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application or in background technique below will be implemented the application Attached drawing needed in example or background technique is illustrated.
Fig. 1 is a kind of flow diagram of image matching method provided by the embodiments of the present application；
Fig. 2 is a kind of concrete scene schematic diagram of images match provided by the embodiments of the present application；
Fig. 3 is a kind of flow diagram of training method provided by the embodiments of the present application；
Fig. 4 is the flow diagram of another training method provided by the embodiments of the present application；
Fig. 5 is the flow diagram of another training method provided by the embodiments of the present application；
Fig. 6 is a kind of concrete scene schematic diagram of training method provided by the embodiments of the present application；
Fig. 7 is a kind of structural schematic diagram of image matching apparatus provided by the embodiments of the present application；
Fig. 8 is a kind of structural schematic diagram of first determination unit provided by the embodiments of the present application；
Fig. 9 is the structural schematic diagram of another image matching apparatus provided by the embodiments of the present application；
Figure 10 is a kind of structural schematic diagram of second determination unit provided by the embodiments of the present application；
Figure 11 is the structural schematic diagram of another image matching apparatus provided by the embodiments of the present application；
Figure 12 is the structural schematic diagram of another image matching apparatus provided by the embodiments of the present application.
Specific embodiment
Term " first ", " second " in the description and claims of this application and attached drawing etc. are for distinguishing difference Object, be not use to describe a particular order.In addition, term " includes " and " having " and their any deformations, it is intended that Non-exclusive include in covering.Such as the process, method, system, product or equipment for containing a series of steps or units do not have It is defined in listed step or unit, but optionally further comprising the step of not listing or unit, or optionally further comprising Other step or units intrinsic for these process, methods or equipment.
The application is described in further detail below in conjunction with attached drawing.
It is a kind of flow diagram of image matching method provided by the embodiments of the present application referring to Fig. 1, Fig. 1, this method can Applied to image matching apparatus, which can be server or terminal device etc., and the embodiment of the present application is for this Image matching apparatus is which kind of equipment does not make uniqueness restriction.
As shown in Figure 1, the image matching method includes:
S101, the characteristic for obtaining the first image.
In the embodiment of the present application, the first image can be arbitrary image, such as still image or video frame images etc., sheet Application embodiment is not construed as limiting first image.
In some possible embodiments, network is extracted using fisrt feature and feature extraction is carried out to the first image, obtain To the characteristic of the first image.
Wherein, optionally, which, which extracts network, can be used deep learning network, such as can pass through ResNet network To obtain the characteristic of the first image.Further, which may include N number of convolutional layer, and first image is successively By the processing of N number of convolutional layer, the characteristic of first image is obtained.
It is understood that deep learning network illustrated above is only a kind of example, should not be construed as to the embodiment of the present application It limits.
Optionally, the specific implementation of the characteristic of the first image is obtained can include: the image matching apparatus is direct The characteristic of first image is obtained by the processor of the image matching apparatus, that is to say, that the image matching apparatus exists It needs directly to pass through processor reality from least two second images in the case where the determining target image with the first images match When obtain the characteristic of first image.
Alternatively, obtain the first image characteristic specific implementation may also include that the image matching apparatus from this The characteristic of first image is obtained in the memory of image matching apparatus.That is, the characteristic of first image It has been obtained that, and be stored in memory, needed determining and the first image from least two second images in image matching apparatus In the case where matched target image, which can directly obtain the characteristic of first image from memory According to.To can also effectively save the time, the speed and efficiency of matching target image are improved.
Alternatively, the image matching apparatus can also obtain the characteristic of first image from other devices, i.e., by being somebody's turn to do Other devices obtain the characteristic of the first image, then when image matching apparatus needs to implement the embodiment of the present application, from it He obtains the characteristic of first image in device, such as server receives the characteristic of the first image from terminal device According to.It is understood that the characteristic how the embodiment of the present application obtains first image for image matching apparatus does not make uniqueness It limits.It is understood that in the case where the image matching apparatus obtains the characteristic of first image from other devices, this Shen Please embodiment how the first image is obtained for other devices characteristic be not construed as limiting.And the embodiment of the present application pair It is specially which kind of device is not construed as limiting in other devices, can is also server etc. if other devices can be terminal device.
S102, the characteristic based on the first image, obtain M group target signature data, and M is the integer more than or equal to 2.
In the embodiment of the present application, the spy of all corresponding first image of each group of target signature data in M group target signature data Levy data.
Optionally, the dimension of target signature data is greater than the dimension of the characteristic of the first image.
In some possible implementations, based on the characteristic of the first image, M group target signature data are obtained, are wrapped It includes:
Characteristic and M random vector based on the first image obtain M group target signature data.
In the embodiment of the present application, different target signature data can be obtained based on different random vectors.That is, At least two target signature data can be obtained using the characteristic of the first image as condition, using random vector as variable.Specifically , which may include Gaussian random vector, that is, include the random vector sampled from Gaussian Profile, or can also Think other kinds of random vector, the embodiment of the present application is not construed as limiting the specific implementation of random vector.
In the embodiment of the present application, by random vector, it can be obtained a variety of possible based on the characteristic of first image Target signature data obtain the efficiency of target signature data to improve.
Specifically, the characteristic based on the first image, obtains M group target signature data, comprising:
The characteristic of first image is input to feature generation network to handle, obtains M group target signature data.
In the embodiment of the present application, feature, which generates network, can be used for generating M group target signature data.It further, can be with The characteristic of first image and M random vector are input to this feature generation network to handle, it is special to obtain M group target Levy data.
For example, in the case where the information content for including less than the second image in the information content that the first image includes, i.e., the One image is weak modality images, and the second image is strong modality images, then implements the embodiment of the present application, can be by weak modality images Characteristic and M random vector (or being M different random vectors) are input to feature and generate network, to obtain a system The strong modal characteristics data of the puppet of column.Wherein, pseudo- strong modal characteristics data can be regarded as feature and generate dimension that network obtains and strong The identical characteristic of the characteristic of modality images.
In the embodiment of the present application, this feature, which generates network, can generate network for the feature of image matching apparatus training, After can also be for other devices such as training device training, the feature for being sent to the image matching apparatus generates network.That is, this Apply in embodiment, which can receive the feature from training device training and generate network, alternatively, the image It can also oneself training this feature generation network with device.It is understood that how the embodiment of the present application instructs the training device Practice feature generation network to be not construed as limiting.
S103, M group target signature data are based on, the determining target with the first images match from least two second images Image.
In the embodiment of the present application, target image is the image with the first images match.
In the embodiment of the present application, the information content that the first image includes can be suitable with the information content that the second image includes, or Person, the information content that the first image includes is fewer than the information content that the second image includes.For example, the first image may include sketch (sketches), the second image may include photo (photos)；For another example the first image may include gray level image (grayscale Images), the second image may include color image (color images)；For another example the first image may include low-resolution image (low resolution images), the second image may include high-definition picture (high resolution image).Or Person, the information content that the first image includes are more than the information content that the second image includes.
In the case where the information content for including less than the second image in the information content that the first image includes, i.e. the first image packet The information content contained, can be by first image less than the information content that any one second image at least two second images includes Second image, is known as the image of strong mode by the image of referred to as weak mode.Since the information content for including in weak modality images is far small Information content in strong modality images, weak modality images possibly can not provide enough information to determine matched strong mode Image, at this point, example as shown in Figure 2, might have multiple matched strong modality images, the embodiment of the present application passes through base Multiple groups target signature data are obtained in the characteristic of weak modality images, and are determined based on the multiple groups target signature data multiple strong In modality images with the matched image of weak modality images, can obtain at least one matched target image of weak modality images, And be conducive to improve the reliability of images match.
In some possible implementations, be based on M group target signature data, from least two second images determine and The target image of first images match, comprising:
Based on M group target signature data, the phase of the first image with each second image at least two second images is determined Like degree；
Based on the similarity of each second image in the first image and at least two second images, from least two images The determining target image with the first images match.
In the embodiment of the present application, can be determined according to M group target signature data the first image and each second image it Between similarity, to carry out the similarity based on each second image in the first image and at least two second images, from least Target image is determined in two the second images.For example, can by least two second images with the first image similarity highest Image be determined as target image, or can also will be higher than default threshold at least two second images with the first image similarity The image of value is determined as target image, or can also will be higher than at least two second images with the first image similarity default Preceding several images in one or more candidate images of threshold value are determined as target image, etc., the embodiment of the present application to this not It limits.
In some possible implementations, M group target signature data are based on, determine the first image and at least two second The similarity of each second image in image, comprising:
Obtain the characteristic of each second image at least two second images；
Based on the characteristic of each second image in M group target signature data and at least two second images, obtain every The corresponding M similarity of a second image；
According to the corresponding M similarity of the second image each at least two second images, the first image and each is determined The similarity of second image.
In the embodiment of the present application, the M group target signature data and the second figure can be calculated separately based on M group target signature data Similarity between the characteristic of picture obtains the corresponding M similarity of the second image, and M corresponding according to the second image Similarity determines the similarity of the first image and the second image.
Wherein, the embodiment of the present application also provides a kind of methods of characteristic for obtaining the second image, comprising:
Network is extracted using second feature, feature extraction is carried out to the second image each at least two second images, obtain The characteristic of each second image.
Wherein, which, which extracts network, can be used deep learning network, can such as be obtained by ResNet network every The characteristic of a second image.Further, which may include i convolutional layer, which can successively lead to The process of convolution for crossing the i convolutional layer obtains the characteristic of second image.
It is understood that the weight parameter that the second feature extracts network can extract the weight parameter phase of network with fisrt feature Together, i.e., the fisrt feature extracts network network parameter same with second feature extraction network share.For example, image The same deep learning network can be used with device to extract the characteristic of the first image and each second image.Alternatively, should The weight parameter and the fisrt feature extraction weight parameter of network of second feature extraction network are different, such as the image matching apparatus The deep learning network for extracting the first image is different from the parameter of deep learning network of the second image is extracted.Specifically, using Different deep learning networks extracts the characteristic of the first image and the second image, and the accurate of feature extraction also can be improved Property.
Optionally, the specific implementation for obtaining the characteristic of each second image at least two second images can wrap Include: the image matching apparatus obtains each second figure at least two second image by the processor of the image matching apparatus The characteristic of picture, that is to say, that when image matching apparatus needs to implement the embodiment of the present application, can be obtained in real time by processor Take the characteristic of each second image.
Alternatively, obtaining the specific implementation of the characteristic of each second image at least two second images can also wrap Include: the image matching apparatus obtains the characteristic of each second image from the memory of the image matching apparatus.Also To say, the characteristic of the second image is pre-stored in memory, image matching apparatus need from this at least two second In the case where determining target image in image, which can obtain the characteristic of the second image from memory According to.
Alternatively, the image matching apparatus can also obtain the characteristic of each second image from other devices.It can manage Solution, the method for how obtaining the characteristic of each second image for the image matching apparatus can also be corresponded to reference to the image The method that the characteristic of the first image how is obtained with device, is no longer described in detail one by one here.
In some possible implementations, according to the corresponding M phase of the second image each at least two second images Like degree, the similarity of the first image Yu each second image is determined, comprising:
It is similar to the second image that the average treatment result of the corresponding M similarity of second image is determined as the first image Degree.
In the embodiment of the present application, the average treatment result of the corresponding M similarity of the second image can include: second image The arithmetic mean result, etc. of the result of weighted average of corresponding M similarity or the corresponding M similarity of second image, The embodiment of the present application is not construed as limiting specific implementation.
In one embodiment, based on each second image in M group target signature data and at least two second images Characteristic obtains the formula of the corresponding M similarity of each second image, can be as follows:
Wherein, f indicates the feature vector of the second image,AndFs indicates that second feature mentions Take network, XgIndicate that the second image, i indicate any one second image at least two second images.It indicates by the The feature vector of i-th of second images at least two second images that two feature extraction networks extract.
Wherein, fqIndicate the feature vector of the first image, fq=Fw (Xq).Fw indicates that fisrt feature extracts network, Fw (Xq) It indicates to extract the first image X that network extracts by fisrt featureqFeature vector.gjIndicate j-th of target signature data, and gj=G (fq,zj), { gj}J=1:m.Wherein, zjIndicate from multivariate Gaussian distribution N (0, I) j-th obtained through stochastical sampling it is random Vector.Wherein, the I in N (0, I) indicates that special diagonal line (identity matrix) is 1 matrix.G(fq,zj) indicate first Image XqAnd j-th of random vector is input to after feature generates network, obtained j-th of target signature data.
It is understood that the method that can determine average treatment according to the value of σ is such as corresponding for different j for formula (2) σ it is identical, then it is similar to the second image to be determined as the first image for the arithmetic mean result of the corresponding M similarity of the second image Degree.Such as corresponding for different j σ value is different, then the result of weighted average of the corresponding M similarity of the second image is determined as the The similarity of one image and the second image.
In the embodiment of the present application, network is generated by the way that the characteristic of the first image is input to feature, obtains M group target Characteristic, thus according to the characteristic of each second image in the M group target signature data and at least two second images According to determining target image.Implement the embodiment of the present application, can effectively make up since the information content of the first image is very few, and leads to the Huge estrangement when mode vectors correlation is carried out between one image and the second image, it is effective to improve the accuracy for determining target image.
Further, the embodiment of the present application passes through the characteristic for obtaining the first image, and obtains at least two second The characteristic of each image in image, i.e., determine similarity by the level in characteristic, rather than directly in image Level determines similarity.On the one hand, high-level feature (i.e. the level of characteristic) usually dimension is lower, therefore and low level Feature (i.e. image hierarchy) is compared, and learns feature generator or training characteristics generator in high-level feature, it is more easy and Cost is smaller；On the other hand, high-dimensional feature is closer to semantic space, at this level, the difference between different modalities It is smaller, therefore be easier to be connected.
For image matching method provided by vivider understanding the embodiment of the present application, the letter for including with the first image below Illustrate for the information content that breath amount includes less than the second image, at this point, image matching method is true from multiple strong modality images The fixed and matched image of weak modality images, i.e. matching of the progress across modality images.
Referring to fig. 2, Fig. 2 is a kind of concrete scene schematic diagram of images match provided by the embodiments of the present application, such as Fig. 2 institute Show, leftmost three images (sketch image) indicate the first image in figure, remaining image (photo) indicates the second figure Picture.Therefrom it can also be seen that the information content of the first image is considerably less than the information content of the second image.By extracting the first image Characteristic, to be input to, feature generates network, obtains M group target signature data, and then be based on the M group target After characteristic determines the similarity of the first image and each second image, target image can be obtained.It is understood that the target Image may also indicate that in the second image with the most matched image of the similarity of the first image.
As shown in Fig. 2, the target image can be followed successively by the first row from the right number second, the second row is from the right number third A and the third line is from the right number the 4th.That is, by implementing the embodiment of the present application, it can be quick, effectively from a large amount of (being best suitable for) photo corresponding with sketch image is obtained in photo.It is understood that each sketch shown in Fig. 2 or photo are only A kind of example should not be construed as having restriction to the embodiment of the present application.
Fig. 1 is described in detail the embodiment of the present application is how to determine from least two second images and the first image The target image matched.Wherein, feature, which generates network, can be the received feature generation network from other devices, be also possible to The image matching apparatus oneself training, therefore, by taking oneself training characteristics of the image matching apparatus generate network as an example, below will Specifically introduce training method.
It is understood that in Fig. 3 and training method shown in Fig. 4, any one image includes in the first training sample information Amount can be suitable with the information content that any one image in the second training sample includes；Alternatively, any one in the first training sample The information content that a image includes is fewer than the information content that any one image in the second training sample includes；Alternatively, the first instruction Practice the information that the information content that any one image includes in sample can also include more than any one image in the second training sample Amount.
But it is specifically chosen which kind of mode also corresponds to image matching method shown in Fig. 2.For example, shown in Fig. 2 Image matching method in, information content that the first image includes is less than the information content that the second image includes, then in training, this The information content that any one image includes in one training sample just needs Information content.
Therefore, Fig. 3 and training method shown in Fig. 4 should not be interpreted as to the restriction to the embodiment of the present application.
It is a kind of flow diagram of training method provided by the embodiments of the present application referring to Fig. 3, Fig. 3, as shown in figure 3, should Method includes:
S301, the characteristic of the first training sample is input to feature generation network, obtains target training characteristics data.
In the embodiment of the present application, the image in the first training sample can be arbitrary image, such as still image or view Frequency frame image etc., the embodiment of the present application are not construed as limiting image included in first training sample.It is understood that target The dimension of training characteristics data is greater than the dimension of the characteristic of the first training sample.
Specifically, image matching apparatus can extract network by fisrt feature and extract the first training sample before S301 This characteristic.
It is understood that the weight parameter that fisrt feature extracts network can extract the weight parameter phase of network with second feature Together, alternatively, the weight parameter that fisrt feature extracts network can also be different from the second feature extraction weight parameter of network.Also To say, extract the characteristic of the first training sample feature extraction network (i.e. fisrt feature extraction network) can with extract the The feature extraction network (i.e. second feature extraction network) of the characteristic of two training samples shares weight parameter, alternatively, can also Not share.Network is extracted for the fisrt feature and the second feature extracts the specific implementation of network, reference may also be made to Fig. 1 Shown in specific implementation, I will not elaborate.
Specifically, the characteristic of the first training sample, which is input to feature, generates network, target training characteristics number is obtained According to, comprising:
The characteristic of first training sample and training random vector are input to feature and generate network, obtains target training Characteristic；Wherein, the target training characteristics data that different training random vectors obtains are different.
In the embodiment of the present application, different training random vectors is inputted, different target training characteristics data can be obtained.It should Train random vector concretely Gaussian random vector.For example, the number of image is ten in the first training sample, then may be used It is condition by one of training sample, using different Gaussian random vectors as variable, is separately input into feature and generates network, from And at least two target training characteristics data can be obtained according to a training sample (i.e. an image).As a result, due to can basis Ten the first training samples obtain at least 20 target training characteristics data.It is understood that the Gaussian random vector can be from height Therefore the random vector sampled in this distribution is not construed as limiting the specific vector of the Gaussian random vector.Alternatively, should Training random vector can also be other kinds of random vector etc..
S302, the input of target training characteristics data is differentiated that network is handled, obtains the first differentiation result.
In the embodiment of the present application, differentiate that network can be used for distinguishing the feature of target training characteristics data and the second training sample Data.Such as differentiate that the first differentiation result of network output can be exported in the form of percentage (i.e. probability), that is, differentiates the spy of input Sign data belong to the percentage of strong mode；Alternatively, the first of differentiation network output differentiates that result can be in the form of score system Output, that is, differentiate that the characteristic of input belongs to the score of strong mode.Such as the first differentiation result can be used for indicating that the target is instructed Practice the probability that characteristic belongs to strong modal characteristics.Or the first differentiation result can be used for indicating the target training characteristics data category In the score etc. of strong modal characteristics.
S303, differentiate based on first as a result, determining first-loss.
Specifically, being differentiated based on first as a result, determining first-loss, comprising:
Target training characteristics data input sorter network is handled, the first classification results are obtained；
Target training characteristics data input random vector Recurrent networks are handled, training is obtained and returns feature, wherein The dimension that training returns feature is equal to the dimension for inputting training random vector；
Differentiate that result, the first classification results and training return feature based on first, determines first-loss.
In the embodiment of the present application, sorter network can be used for supervising target training characteristics data.Random vector Recurrent networks can For random vector can be efficiently used when training characteristics generate network.It is understood that in the embodiment of the present application, by target It, such as can be by target training characteristics data and the target training characteristics when training characteristics data input sorter network is handled The corresponding class label of data is input to the sorter network, thus output probability corresponding with such distinguishing label (i.e. output target The classification results of training characteristics data i.e. the first classification results).Wherein, class label may particularly include identity, alternatively, should Class label may also comprise classification logotype.For example, animal cat and dog are two class labels, if further distinguishing, Then two Kazakhstan and Ke Ji may belong to two class labels.Therefore, class label is specifically distinguished to what in the embodiment of the present application Kind degree is not construed as limiting.
In the embodiment of the present application, first-loss can be in the form of function etc., and the embodiment of the present application is not construed as limiting.
S304, it is based on first-loss, training characteristics generate network.
It is understood that training method described in the embodiment of the present application specifically may also be understood to be the weight of trained corresponding network Parameter updates the weight parameter of corresponding network.It is such as based on first-loss, training characteristics generate network, also are understood as being based on The first-loss updates the weight parameter that this feature generates network.
Specifically, when training characteristics generate network, can be instructed by the way of backpropagation utilizing first-loss Practice.It, can deconditioning and when the first differentiation result for differentiating result output meets objective result.Since feature generates net Network and differentiation network are trained in confrontation, therefore, can not identify that the characteristic of input belongs to strong mould in differentiation network When the characteristic of the characteristic of state or pseudo- strong mode, then illustrates that feature generation network has been trained and finish, indicate this feature The characteristic of the strong mode of puppet of network output is generated to the greatest extent close to the characteristic of strong mode.Therefore, target knot The setting of fruit can be related to the differentiation result of result output is differentiated, as differentiated, result is the characteristic of input belongs to strong mode Percentage, then the objective result can be 1, alternatively, the objective result may be greater than 0.8 etc., the embodiment of the present application is for the mesh How mark result, which limits, is not construed as limiting.
In the embodiment of the present application, image matching apparatus generates network by feature and exports target training characteristics data, thus The target training characteristics data are input to differentiation network, make to differentiate that network is differentiated (i.e. differentiation target training characteristics data Belong to strong mould probability of state or score etc.)；Network is generated come training characteristics by confrontation mode and differentiates network, can be effectively improved Trained efficiency steps up the accuracy of target training characteristics data output.
For vivider description training method shown in Fig. 3, described below with specific formula.
Differentiate that network is handled for example, target training characteristics data are inputted, the formula for obtaining the first differentiation result can It is as follows:
Wherein, D indicates to differentiate that network, G can indicate that feature generates network, and Fw indicates that fisrt feature extracts network.G(Fw (X), z) indicate that the first training sample X is input to fisrt feature extracts network, obtains the training characteristics of the first training sample Data Fw (X)；Then the characteristic of first training sample and training random vector are input to feature and generate network, obtained To target training characteristics data G (Fw (X), z).D (G (Fw (X), z)) indicates output, that is, target instruction that feature is generated to network Practice characteristic and be input to differentiation network, thus to obtain the first differentiation result.
For example, target training characteristics data input sorter network is handled, the formula for obtaining the first classification results can It is as follows:
Wherein, Wc presentation class network.
For example, target training characteristics data input random vector Recurrent networks are handled, obtains training and return feature Formula can be as follows:
Wherein, E indicates random vector Recurrent networks.For formula (5), target training characteristics data and instruction are inputted Practice random vector, then the exportable target training characteristics data with class label are input to random vector Recurrent networks again In obtain output result.Wherein, for example, Fw (X) is the vector of 1*10, and z is the vector of 1*12, then G (Fw (X), z) is 1* 10 vector, E (G (Fw (X), z)) are then the vector of 1*12.
For example, differentiating that result, the first classification results and training return feature based on first, determine that the formula of first-loss can It is as follows:
It is understood that being to illustrate how to determine first-loss using loss function as example above.
Specifically, method shown in Fig. 3 further include: according to first-loss, training fisrt feature extracts network.
It, can be respectively to G and Fw derivation, so that carrying out more new feature generates network and fisrt feature extraction such as formula (6) Network.
That is, the embodiment of the present application, can also update the power that fisrt feature extracts network according to the first differentiation result Weight parameter.
Be described in detail in training method shown in Fig. 3 how training characteristics generate network, it is described in detail below how Training differentiates network.Referring to fig. 4, Fig. 4 is the flow diagram of another training method provided by the embodiments of the present application, such as Fig. 4 Shown, which includes:
S401, by the characteristic of the second training sample be input to differentiate network handle, obtain the second differentiation as a result, Wherein, the information content for including in the first training sample that contains much information for including in the second training sample.
In the embodiment of the present application, for the second training sample, which can be the image of weak mode, Second training sample can be the image of strong mode.It is understood that since first training sample and second training sample are used for Training characteristics generate network, therefore, the quantity of first training sample and second training sample can be related to training degree etc. Deng the embodiment of the present application is not construed as limiting the quantity of first training sample He second training sample.
In the embodiment of the present application, the characteristic of target training characteristics data and the second training sample can phase in dimension Together, that is to say, that in order to further increase the accuracy for differentiating that network differentiates, respectively by the characteristic of the second training sample When being input to differentiation network with target training characteristics data, it is ensured that the target training characteristics data and second training sample The dimension of characteristic is identical.For example, the characteristic of second training sample is b1*M, the target training characteristics data For b2*M, i.e. the quantity b1 of the characteristic of second training sample can be different from the quantity b2 of the target training characteristics data, Or can also be identical, the application is not construed as limiting；But dimension is identical.
In the embodiment of the present application, the dimension of the characteristic of the second training sample is different from the characteristic of the first training sample According to dimension.Optionally, the dimension of the characteristic of second training sample is greater than the characteristic of first training sample Dimension.
It, can also be by the specifically, before the characteristic of the second training sample is input to differentiating that network handled Two feature extraction networks extract the characteristic of the second training sample.It is understood that extracting the specific of network for the second feature Implementation can refer to Fig. 1 and specific implementation shown in Fig. 3, no longer be described in detail one by one here.
S402, differentiate that result and training return feature based on second, determine the second loss.
Specifically, differentiating that result and training return feature based on second, the second loss is determined, comprising:
The characteristic input sorter network of second training sample is handled, the second classification results are obtained；
Differentiate that result, the second classification results and training return feature based on second, determines the second loss.
It is understood that the specific implementation for returning feature for sorter network and training can join in the embodiment of the present application Implementation shown in Fig. 3 is examined, which is not described herein again.
In the embodiment of the present application, the second loss can be in the form of function etc., and the embodiment of the present application is not construed as limiting.
S403, it is lost based on second, training differentiates network, sorter network and random vector Recurrent networks.
For vivider description training method shown in Fig. 4, described below with specific formula.
Differentiate that network is handled for example, the characteristic of the second training sample is input to, obtains the second differentiation result Formula it is as follows:
Wherein, Fs indicates that second feature extracts network.It is understood that before reference may also be made to for the parameters in formula (7) State formula described in embodiment.
It is handled for example, the characteristic of the second training sample is input to sorter network, obtains the second classification results Formula it is as follows:
In the embodiment of the present application, for formula (4) and formula (8), it will be appreciated that can be by for the weight parameter of the sorter network Second training sample is trained, and when using the sorter network, it can guarantee that target training is special by the sorter network The class label of data is levied, to avoid the characteristic for generating classification notable difference.
For example, based on the second loss, training differentiates the following institute of formula of network, sorter network and random vector Recurrent networks Show:
It is understood that can differentiate network, classification net to update respectively respectively to D, Wc, Fs and E derivation for formula (9) Network, second feature extract network and random vector Recurrent networks.
It is the flow diagram of another training method provided by the embodiments of the present application referring to Fig. 5, Fig. 5, as shown in figure 5, This method comprises:
S501, the characteristic of the first training sample is input to feature generation network, obtains target training characteristics data.
S502, the input of target training characteristics data is differentiated that network is handled, obtains the first differentiation as a result, target is instructed Practice characteristic input sorter network to be handled, obtains the first classification results.
S503, by the characteristic of the second training sample be input to differentiate network handle, obtain the second differentiation as a result, The characteristic input sorter network of second training sample is handled, the second classification results are obtained.
S504, target training characteristics data input random vector Recurrent networks are handled, obtains training and returns feature.
S505, differentiate that result, the first classification results and training return feature based on first, determine first-loss.
S506, differentiate that result, the second classification results and training return feature based on second, determine the second loss.
S507, it is based on first-loss, training characteristics generate network and fisrt feature extracts network.
S508, it is lost based on second, training differentiates that network, sorter network, random vector Recurrent networks and second feature mention Take network.
It is understood that generating network in the embodiment of the present application in training characteristics and fisrt feature extracts network and training is sentenced When other network, sorter network, random vector Recurrent networks and second feature extract network, two stages can be divided into train, For example first training differentiates that network, sorter network, random vector Recurrent networks and second feature extract network, then retraining training It generates network and fisrt feature extracts network.Alternatively, first training characteristics generate network and fisrt feature extracts network, then instruct again Practice and differentiates that network, sorter network, random vector Recurrent networks and second feature extract network.And institute in the embodiment of the present application The training method of offer can also continuous training stage by stage, until achieving the desired results.
Fig. 3 and method shown in Fig. 4 can refer to for training method shown in fig. 5, are no longer described in detail one by one here.
It is a kind of training provided by the embodiments of the present application referring to Fig. 6, Fig. 6 for the understanding training method shown in fig. 5 of image The concrete scene schematic diagram of method.As shown in fig. 6, dotted portion indicates the parameter of the network without update, achievement unit in Fig. 6 Dividing indicates that the parameter of the network is updated.
As shown in fig. 6, the output that fisrt feature extracts network can be input to, feature generates network and this feature generates net The output of network, which can input, is separately input into random vector Recurrent networks, sorter network and differentiation network.And second feature mentions It takes the output of network that can be input to and differentiates network and sorter network.
It is understood that being propagated in the figure shown in the left side Fig. 6 according to direction, feature generates network and can receive from differentiation net Network and the common supervisory signals of sorter network, so that target training characteristics data can either guarantee original identity information, It is consistent again in the upper feature space distribution with the second training sample of the feature space distribution of generation.Shown on the right of Fig. 6 In figure, differentiate that network can receive the target training characteristics data and second feature extraction that network output is generated from feature respectively The characteristic of second training sample of network output, thus the difference being distributed between two training samples of study.
According to implementation above mode it is found that the one kind that inputs that feature shown in fig. 6 generates network can be to mention from fisrt feature The characteristic for the first training sample for taking network to extract；A kind of differentiation result to be exported from differentiation network.Namely It says, on the one hand feature, which generates network, constantly can generate target training characteristics data according to the characteristic of the first training sample, separately On the one hand, also network can be generated to update this feature according to the differentiation result of network output is differentiated.
Specifically, differentiate network output differentiation result not only can training characteristics generate network, can also update differentiation network. Vivider says, wherein this feature generate network purpose be in order to enable the characteristic of the strong modality images of puppet arrived more Really, the purpose for differentiating network is to determine the characteristic of input as far as possible and belong to the characteristic of strong mode also It is the characteristic of pseudo- strong mode.Differentiate that network and feature generate network and form confrontation as a result, it is raw that training characteristics are carried out in confrontation At network and the differentiation network.
It is understood that the confusion of data is avoided in order to further increase training effectiveness, therefore, the first training sample is defeated Enter to feature and generate network, during obtaining target training characteristics data, the weight parameter for differentiating network can be fixed；And By the characteristic of the second training sample be input to differentiate network when, can fixed character generate network weight parameter；And it will sentence When other result is input to feature generation network, the weight parameter of the differentiation network is fixed.Thus, it is possible to effectively avoid heterogeneous networks Between the confusion that changes, and characteristic is caused to input, or training effectiveness is caused to reduce.
In the embodiment of the present application, network is generated come training characteristics using two stages training method and differentiates network, on the one hand, Network can be generated come training characteristics according to the differentiation result of network output is differentiated；It on the other hand, can also be according to sorter network Training this feature generates network, guarantees the characteristic of target training characteristics data Yu the first training sample by sorter network Difference, trained efficiency is further improved, to improve the accuracy of target image.
It is understood that Fig. 1, Fig. 3 emphasize particularly on different fields into method shown in fig. 6, therefore not detailed description in one embodiment Implementation, can also correspond to reference to other embodiments.
The method of the embodiment of the present application is illustrated, the device of the embodiment of the present application is provided below.
It is a kind of structural schematic diagram of image matching apparatus provided by the embodiments of the present application referring to Fig. 7, Fig. 7, which can For executing Fig. 1, Fig. 3 to method shown in fig. 5, as shown in fig. 7, the image matching apparatus may include:
Acquiring unit 701, for obtaining the characteristic of the first image；
First data processing unit 702 obtains M group target signature data for the characteristic based on the first image；Its In, M is the integer more than or equal to 2；
First determination unit 703 determines and for being based on M group target signature data from least two second images The target image of one images match.
Implement the embodiment of the present application, determine the target image with the first images match based on M group target signature data, also The accuracy and accuracy that can further improve determining target image avoid being based only upon one group of target signature data to determine mesh Logo image and the situation for causing accuracy not high.
In one possible implementation, the first determination unit 703, specifically can based on M group target signature data with And at least two second each second image in image characteristic, determined from least two second image with this first The target image of images match.
In one possible implementation, the dimension of the characteristic of the second image is different from the characteristic of the first image According to dimension, the dimensions of the target signature data is greater than the dimension of the characteristic of the second image.
In one possible implementation, the dimension of the characteristic of second image is greater than the feature of first image The dimension of data.
In one possible implementation, acquiring unit 701 are also used to obtain each at least two second images The characteristic of two images.
In some instances, which can propose the second image each at least two second images progress feature Processing is taken, the characteristic of each second image is obtained.
In other examples, acquiring unit can obtain each second image at least two second images from memory Characteristic.
Optionally, as shown in figure 8, the first determination unit 703 includes:
First determines subelement 7031, for being based on M group target signature data, determines the first image and at least two second The similarity of each second image in image；
Second determines subelement 7032, for based on each second image in the first image and at least two second images Similarity, the determining target image with the first images match from least two images.
Specifically, first determines subelement 7031, it is specifically used for obtaining each second image at least two second images Characteristic；And the characteristic based on each second image in M group target signature data and at least two second images, Obtain the corresponding M similarity of each second image；And according to the corresponding M of the second image each at least two second images A similarity determines the similarity of the first image Yu each second image.
Specifically, first determines subelement 7031, specifically for by the average treatment of the corresponding M similarity of the second image As a result it is determined as the similarity of the first image and the second image.
Specifically, the information content that the first image includes is less than the information content that the second image includes.
Implement the embodiment of the present application, can effectively make up since the information content of the first image is very few, and cause the first image with Huge estrangement when mode vectors correlation is carried out between second image, it is effective to improve the accuracy for determining target image.
Further, the first data processing unit 702, specifically for based on the first image characteristic and M with Machine vector obtains M group target signature data.
Specifically, the first data processing unit 702, generates specifically for the characteristic of the first image is input to feature Network is handled, and M group target signature data are obtained.
Further, as shown in figure 9, image matching apparatus further include:
Second data processing unit 704 generates network for the characteristic of the first training sample to be input to feature, obtains To target training characteristics data；
Wherein, the dimension of target training characteristics data is greater than the dimension of the characteristic of the first training sample.
First judgement unit 705 differentiates that network is handled for inputting target training characteristics data, obtains first and sentence Other result；
Second determination unit 706, for differentiating based on first as a result, determining first-loss；
First training unit 707, for being based on first-loss, training characteristics generate network.
Optionally, as shown in Figure 10, the second determination unit 706 includes:
Classification processing subelement 7061 obtains for handling target training characteristics data input sorter network One classification results；
Processing subelement 7062 is returned, for inputting target training characteristics data at random vector Recurrent networks Reason obtains training and returns feature, wherein the dimension that training returns feature is equal to the dimension for inputting training random vector；
Third determines subelement 7063, for differentiating that result, the first classification results and training return feature based on first, really Determine first-loss.
Specifically, acquiring unit 701, proposes the first image progress feature specifically for extracting network using fisrt feature It takes, obtains the characteristic of the first image；
First training unit 707 is also used to according to first-loss, and training fisrt feature extracts network.
Further, as shown in figure 9, image matching apparatus further include:
Second judgement unit 708 differentiates that network is handled for the characteristic of the second training sample to be input to, obtains Result is differentiated to second, wherein the information content for including in the first training sample that contains much information for including in the second training sample；
Third determination unit 709 determines the second loss for differentiating that result and training return feature based on second；
Second training unit 710, for based on the second loss, training to differentiate that network, sorter network and random vector return Network.
In one possible implementation, third determination unit 709, specifically for by the characteristic of the second training sample It is handled according to input sorter network, obtains the second classification results；And result, the second classification results and instruction are differentiated based on second Practice and return feature, determines the second loss.
In one possible implementation, the acquiring unit 701 is specifically used for extracting network pair using second feature Each second image carries out feature extraction at least two second images, obtains each second image at least two second images Characteristic；
Second training unit 710 is also used to based on the second loss, and training second feature extracts network.
It is understood that the realization of each unit can also correspond to referring to Fig.1, Fig. 3, Fig. 4 and method shown in fig. 5 it is real Apply the corresponding description of example.
It is the structural schematic diagram of another image matching apparatus provided by the embodiments of the present application, the figure referring to Figure 11, Figure 11 As coalignment can be used for executing Fig. 1, Fig. 3 to method shown in fig. 5, as shown in figure 11, which includes:
Fisrt feature extraction module 1101, this module can extract network based on fisrt feature to extract the feature of the first image Data, and also network can be extracted based on fisrt feature to extract the characteristic of the first training sample.That is the input of the module It can be the first image, the input for exporting characteristic (namely feature vector) and the module for the first image can be first Training sample exports the characteristic (namely feature vector) for the first training sample.It is understood that the module is alternatively referred to as weak mould State characteristic extracting module.
Second feature extraction module 1102, the module can extract network based on second feature to extract the feature of the second image Data, and also network can be extracted based on second feature to extract the characteristic of the second training sample, the i.e. input of the module For the second image, the input for exporting the characteristic (feature vector) and the module for the second image can also be the second training Sample exports as the characteristic of the second training sample.It is understood that the module is alternatively referred to as strong modal characteristics extraction module.
The input of feature generation module 1103, the module can be the first image extracted from fisrt feature extraction module Characteristic and the random vector sampled from Gaussian Profile export as target signature data.And the module is defeated Entering can also sample for the characteristic of the first training sample extracted from fisrt feature extraction module and from Gaussian Profile Obtained random vector exports as target training characteristics data, specifically, the module multi-layer perception (MLP) can be used as a result, i.e. with The feature vector that weak mode provides is condition, using the different random vector sampled in Gaussian Profile as variable, in strong mode Feature space generate a series of possible strong modal characteristics of puppet.Wherein, the module can based on feature generate network come execute with Upper operation.
Discrimination module 1104, the input of the module are the characteristic of the second image and the characteristic of target image, Output is the score for belonging to strong modal characteristics.That is, the input of the module can be to extract from second feature extraction module Characteristic out and the target signature data obtained from feature generation module.Alternatively, the input of the module is the second training The characteristic and target training characteristics data of sample, export to belong to the score of strong modal characteristics.
Categorization module 1105, the input of the module are class label, and extracted from fisrt feature extraction module Characteristic exports as the corresponding probability of class label.Feature vector can be converted to classification response by the module, then via returning One changes exponential function, is changed into the corresponding probability of class label after sampling.
Random vector regression block 1106, the input of the module can be characterized the spy of the strong mode of puppet obtained in generation module Data are levied, are exported as the feature vector with the dimensions such as random vector.The module can be used recurrence loss function and exercise supervision, and promote Fisrt feature extraction module can efficiently use the feature vector sampled in Gaussian Profile.It is understood that the random vector is It is characterized the random vector inputted in generation module.
It is understood that image matching apparatus shown in Figure 11 is that the information content for including includes in the first image less than the second image Information content in the case where show, optionally, which applies also for other situations, no longer go to live in the household of one's in-laws on getting married one by one here It states.
It is understood that the specific implementation of image matching apparatus shown in Figure 11, can refer to foregoing individual embodiments, here No longer it is described in detail one by one.
It is the structural schematic diagram of another image matching apparatus provided by the embodiments of the present application referring to Figure 12, Figure 12.The figure It can also include input interface 1202, output interface 1203 and memory 1204 as coalignment includes processor 1201.This is defeated It is connected with each other between incoming interface 1202, output interface 1203, memory 1204 and processor 1201 by bus.
Memory include but is not limited to be random access memory (random access memory, RAM), read-only storage Device (read-only memory, ROM), Erasable Programmable Read Only Memory EPROM (erasable programmable read Only memory, EPROM) or portable read-only memory (compact disc read-only memory, CD-ROM), The memory is used for dependent instruction and data.
Input interface is used for output data and/or signal for input data and/or signal and output interface.Output Interface and input interface can be independent device, be also possible to the device of an entirety.
Processor may include be one or more processors, for example including one or more central processing unit (central Processing unit, CPU), in the case where processor is a CPU, which can be monokaryon CPU, be also possible to more Core CPU.
Memory is used to store the program code and data of image matching apparatus.
Processor is used to call the program code and data in the memory, executes the step in embodiment of the method.
As in one embodiment, processor can be used for executing implementation shown in S101 to S103；For another example processor It can also be used to execute implementation shown in S301 to S303 etc..
For another example in one embodiment, processor can also be used to execute acquiring unit 701, the first data processing unit 702 With method shown in determination unit 703 etc..
Description in embodiment of the method can be found in for the specific implementation of processor, details are not described herein.
It is designed it is understood that Figure 12 illustrate only simplifying for image matching apparatus.In practical applications, image Necessary other elements can also be separately included with device, including but not limited to any number of input/output interface, processing Device, controller, memory etc., and all image matching apparatus that the embodiment of the present application may be implemented are all in the protection model of the application Within enclosing.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of description, device It with the specific work process of unit, can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with It realizes by another way.For example, the division of the unit, only a kind of logical function partition, can have in actual implementation Other division mode, for example, multiple units or components can be combined or can be integrated into another system or some features It can ignore, or not execute.Shown or discussed mutual coupling or direct-coupling or communication connection can be logical Some interfaces are crossed, the indirect coupling or communication connection of device or unit can be electrical property, mechanical or other forms.
Unit may or may not be physically separated as illustrated by the separation member, shown as a unit Component may or may not be physical unit, it can and it is in one place, or may be distributed over multiple networks On unit.It can some or all of the units may be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
In embodiment, it can be realized wholly or partly by software, hardware, firmware or any combination thereof.When When using software realization, can entirely or partly it realize in the form of a computer program product.The computer program product packet Include one or more computer instructions.When loading and execute on computers the computer program instructions, all or part of real estate Raw process or function according to the embodiment of the present application.The computer can be general purpose computer, special purpose computer, computer network Network or other programmable devices.The computer instruction may be stored in a computer readable storage medium, or pass through the meter Calculation machine readable storage medium storing program for executing is transmitted.The computer instruction can be from web-site, computer, server or a data The heart passes through wired (such as coaxial cable, optical fiber, Digital Subscriber Line (digital subscriber line, DSL)) or wireless (such as infrared, wireless, microwave etc.) mode is transmitted to another web-site, computer, server or data center.It should Computer readable storage medium can be any usable medium that computer can access or include one or more available The data storage devices such as medium integrated server, data center.The usable medium can be ROM or RAM or magnetic medium, For example, floppy disk, hard disk, tape, magnetic disk or optical medium, for example, digital versatile disc (digital versatile disc, DVD) or semiconductor medium, for example, solid state hard disk (solid state disk, SSD) etc..

Claims (10)

1. a kind of image matching method characterized by comprising
Obtain the characteristic of the first image；
Characteristic based on the first image, obtains M group target signature data, and the M is the integer more than or equal to 2；
Based on the M group target signature data, the determining and matched target of the first image from least two second images Image.
2. the method according to claim 1, wherein described be based on the M group target signature data, from least two The determining and matched target image of the first image in a second image, comprising:
Based on the M group target signature data, determine each second in the first image and at least two second image The similarity of image；
Based on the similarity of each second image in the first image and at least two second image, from described at least two The determining and matched target image of the first image in a image.
3. according to the method described in claim 2, it is characterized in that, it is described be based on the M group target signature data, determination described in The similarity of each second image in first image and at least two second image, comprising:
Obtain the characteristic of each second image at least two second image；
Based on the characteristic of each second image in the M group target signature data and at least two second image, obtain To the corresponding M similarity of each second image；
According to the corresponding M similarity of the second image each at least two second image, determine the first image with The similarity of each second image.
4. according to the method described in claim 3, it is characterized in that, described according at least two second image each The corresponding M similarity of two images determines the similarity of the first image Yu each second image, comprising:
The average treatment result of the corresponding M similarity of second image is determined as the first image and second figure The similarity of picture.
5. the method according to claim 1, which is characterized in that the information content that the first image includes Less than the information content that second image includes, the dimension of the target signature data is greater than the characteristic of the first image Dimension.
6. according to claim 1 to method described in 5 any one, which is characterized in that the spy based on the first image Data are levied, M group target signature data are obtained, comprising:
Characteristic and M random vector based on the first image obtain the M group target signature data.
7. according to claim 1 to method described in 6 any one, which is characterized in that the spy based on the first image Data are levied, M group target signature data are obtained, comprising:
The characteristic of the first image is input to feature generation network to handle, obtains the M group target signature number According to.
8. a kind of image matching apparatus characterized by comprising
Acquiring unit, for obtaining the characteristic of the first image；
First data processing unit obtains M group target signature data, the M for the characteristic based on the first image For the integer more than or equal to 2；
First determination unit, it is determining from least two second images and described for being based on the M group target signature data The target image of one images match.
9. a kind of image matching apparatus, which is characterized in that including processor and memory, the memory is stored with computer can Reading instruction when the computer-readable instruction is executed by the processor, makes the processor execute such as claim 1 to 7 times Method described in meaning one.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium Readable instruction, the computer-readable instruction when being executed by a processor, keep the processor perform claim requirement 1 to 7 any Method described in one.
CN201810728913.3A 2018-07-04 2018-07-04 Image matching method, device and computer readable storage medium Active CN109101992B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810728913.3A CN109101992B (en) 2018-07-04 2018-07-04 Image matching method, device and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810728913.3A CN109101992B (en) 2018-07-04 2018-07-04 Image matching method, device and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN109101992A true CN109101992A (en) 2018-12-28
CN109101992B CN109101992B (en) 2022-02-22

Family

ID=64845718

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810728913.3A Active CN109101992B (en) 2018-07-04 2018-07-04 Image matching method, device and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN109101992B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111143601A (en) * 2019-12-31 2020-05-12 深圳市芭田生态工程股份有限公司 Image processing method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107122826A (en) * 2017-05-08 2017-09-01 京东方科技集团股份有限公司 Processing method and system and storage medium for convolutional neural networks
CN107451619A (en) * 2017-08-11 2017-12-08 深圳市唯特视科技有限公司 A kind of small target detecting method that confrontation network is generated based on perception
CN107563509A (en) * 2017-07-17 2018-01-09 华南理工大学 A kind of dynamic adjustment algorithm for the condition DCGAN models that feature based returns
CN107609637A (en) * 2017-09-27 2018-01-19 北京师范大学 A kind of combination data represent the method with the raising pattern-recognition precision of pseudo- reversal learning self-encoding encoder
US20180165554A1 (en) * 2016-12-09 2018-06-14 The Research Foundation For The State University Of New York Semisupervised autoencoder for sentiment analysis

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180165554A1 (en) * 2016-12-09 2018-06-14 The Research Foundation For The State University Of New York Semisupervised autoencoder for sentiment analysis
CN107122826A (en) * 2017-05-08 2017-09-01 京东方科技集团股份有限公司 Processing method and system and storage medium for convolutional neural networks
CN107563509A (en) * 2017-07-17 2018-01-09 华南理工大学 A kind of dynamic adjustment algorithm for the condition DCGAN models that feature based returns
CN107451619A (en) * 2017-08-11 2017-12-08 深圳市唯特视科技有限公司 A kind of small target detecting method that confrontation network is generated based on perception
CN107609637A (en) * 2017-09-27 2018-01-19 北京师范大学 A kind of combination data represent the method with the raising pattern-recognition precision of pseudo- reversal learning self-encoding encoder

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JIANAN LI等: "Perceptual Generative Adversarial Networks for Small Object Detection", 《2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 *
RUIHUA WANG等: "An Effective Image Denoising Method for UAV Images via Improved Generative Adversarial Networks", 《SENSORS》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111143601A (en) * 2019-12-31 2020-05-12 深圳市芭田生态工程股份有限公司 Image processing method

Also Published As

Publication number Publication date
CN109101992B (en) 2022-02-22

Similar Documents

Publication Publication Date Title
Mou et al. Vehicle instance segmentation from aerial image and video using a multitask learning residual fully convolutional network
CN111797893B (en) Neural network training method, image classification system and related equipment
WO2022012407A1 (en) Neural network training method and related device
CN111401406B (en) Neural network training method, video frame processing method and related equipment
WO2021022521A1 (en) Method for processing data, and method and device for training neural network model
CN112418392A (en) Neural network construction method and device
CN109902546A (en) Face identification method, device and computer-readable medium
CN111507378A (en) Method and apparatus for training image processing model
CN110084281A (en) Image generating method, the compression method of neural network and relevant apparatus, equipment
EP4002161A1 (en) Image retrieval method and apparatus, storage medium, and device
CN111666919B (en) Object identification method and device, computer equipment and storage medium
WO2018196718A1 (en) Image disambiguation method and device, storage medium, and electronic device
CN113807399A (en) Neural network training method, neural network detection method and neural network detection device
WO2021227787A1 (en) Neural network predictor training method and apparatus, and image processing method and apparatus
CN113408570A (en) Image category identification method and device based on model distillation, storage medium and terminal
WO2021190433A1 (en) Method and device for updating object recognition model
CN109034218B (en) Model training method, device, equipment and storage medium
CN112529149A (en) Data processing method and related device
CN110147724A (en) For detecting text filed method, apparatus, equipment and medium in video
CN113627421A (en) Image processing method, model training method and related equipment
CN111429414B (en) Artificial intelligence-based focus image sample determination method and related device
CN109101992A (en) Image matching method, device and computer readable storage medium
CN112257840A (en) Neural network processing method and related equipment
CN115222047A (en) Model training method, device, equipment and storage medium
CN115937993A (en) Living body detection model training method, living body detection device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant