CN109101992A - Image matching method, device and computer readable storage medium - Google Patents
Image matching method, device and computer readable storage medium Download PDFInfo
- Publication number
- CN109101992A CN109101992A CN201810728913.3A CN201810728913A CN109101992A CN 109101992 A CN109101992 A CN 109101992A CN 201810728913 A CN201810728913 A CN 201810728913A CN 109101992 A CN109101992 A CN 109101992A
- Authority
- CN
- China
- Prior art keywords
- image
- characteristic
- network
- training
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
This application discloses a kind of image matching method, device and computer readable storage mediums.This method comprises: obtaining the characteristic of the first image;Characteristic based on the first image, obtains M group target signature data, and the M is the integer more than or equal to 2;Based on the M group target signature data, the determining and matched target image of the first image from least two second images.Correspondingly, additionally providing corresponding device.Using the application, the precision of images match can be improved.
Description
Technical field
This application involves field of computer technology more particularly to a kind of image matching method, device and computer-readable deposit
Storage media.
Background technique
It refers to going to match the data in another mode with the data in a mode across mode vectors correlation.Such as: with element
Draw the photo of matching camera shooting or the photo of infrared camera shooting.Wherein, sketch images are just equivalent to weak mode, photo just phase
When in strong mode.How to realize be across mode vectors correlation this field research hotspot.
Summary of the invention
The application provides a kind of image matching method, device and computer readable storage medium, and images match can be improved
Accuracy.
In a first aspect, the embodiment of the present application provides a kind of image matching method, comprising:
Obtain the characteristic of the first image;
Characteristic based on the first image obtains M group target signature data, wherein the M be more than or equal to
2 integer;
Based on the M group target signature data, determination is matched with the first image from least two second images
Target image.
In the embodiment of the present application, by obtaining the characteristic of the first image, thus the characteristic based on first image
According to, obtain M group target signature data, and then be based on the M group target signature data, determined from least two second images and
The target image of first images match.Implement the embodiment of the present application, is determined based on M group target signature data and the first image
The accuracy and accuracy of determining target image can be improved in the target image matched.
It in one possible implementation, can be based on the M group target signature data and described at least two the
The characteristic of each second image in two images, determination is matched with the first image from least two second image
Target image.
In one possible implementation, the dimension of the characteristic of second image is different from the first image
Characteristic dimension, the dimensions of the target signature data is equal to the dimension of the characteristic of second image.
In one possible implementation, the dimension of the characteristic of second image is greater than the first image
The dimension of characteristic.
In one possible implementation, the dimension of the target signature data is greater than the characteristic of the first image
According to dimension.
In one possible implementation, the method also includes: obtain each at least two second image
The characteristic of second image.
In some instances, feature extraction processing can be carried out to the second image each at least two second images, obtained
To the characteristic of each second image.
In other examples, the characteristic of each second image at least two second images can be obtained from memory
According to.
In one possible implementation, described to be based on the M group target signature data, from least two second images
Middle determination and the matched target image of the first image, comprising:
Based on the M group target signature data, determine each in the first image and at least two second image
The similarity of second image;
Based on the similarity of each second image in the first image and at least two second image, from it is described to
The determining and matched target image of the first image in few two images.
In the embodiment of the present application, after obtaining M group target signature data, by the M group target signature data, first is determined
The similarity of each second image, is conducive to effectively improve the accurate of determining similarity in image and at least two second images
Degree improves the efficiency for determining target image.
In one possible implementation, it is described be based on the M group target signature data, determine the first image and
The similarity of each second image at least two second image, comprising:
Obtain the characteristic of each second image at least two second image;
Characteristic based on each second image in the M group target signature data and at least two second image
According to obtaining the corresponding M similarity of each second image;
According to the corresponding M similarity of the second image each at least two second image, first figure is determined
As the similarity with each second image.
In the embodiment of the present application, a kind of method of determining similarity is provided, M group target signature data can be based on, obtained
The first figure is determined to the corresponding M similarity of each second image, and then according to the corresponding M similarity of each second image
As the similarity with each second image.
In one possible implementation, described corresponding according to the second image each at least two second image
M similarity, determine the similarity of the first image Yu each second image, comprising:
The average treatment result of the corresponding M similarity of second image is determined as the first image and described the
The similarity of two images.
In the embodiment of the present application, the average treatment result of the corresponding M similarity of the second image can include: second image
The arithmetic mean result of the result of weighted average of corresponding M similarity or the corresponding M similarity of second image.
In one possible implementation, the information content that the first image includes is less than second image
Information content.
Image matching method provided by the embodiment of the present application can be applied to the matching between different modalities, such as not only may be used
Applied to comprising between comparable two mode of information content;The information content for applying also for a mode is considerably less than another mould
The information content of state.Thus implement the embodiment of the present application, can effectively make up since the information content of the first image is very few, and lead to first
Huge estrangement when mode vectors correlation is carried out between image and the second image, it is effective to improve the accuracy for determining target image.
In one possible implementation, the characteristic for obtaining the first image, comprising:
The characteristic of the first image is obtained from memory.
In the embodiment of the present application, memory includes the memory of image matching apparatus.That is, the spy of first image
Sign data have obtained, and are stored in memory.The time for obtaining the characteristic of the first image can be saved as a result, and raising obtains
Take efficiency.
In one possible implementation, the characteristic based on the first image obtains M group target signature
Data, comprising:
Characteristic and M random vector based on the first image obtain the M group target signature data.
In the embodiment of the present application, different random vectors is inputted, different target signature data can be obtained.That is,
It can be using the first image as condition, using random vector as variable, so that a series of possible target signature data are obtained, thus to the greatest extent
More target signature data are likely to be obtained, the accuracy for calculating similarity is improved.
In one possible implementation, the characteristic based on the first image obtains M group target signature
Data, comprising:
The characteristic of the first image is input to feature generation network to handle, it is special to obtain the M group target
Levy data.
It, can also be defeated by different random vector difference using the characteristic of the first image as condition in the embodiment of the present application
Enter to feature and generate network, to obtain M group target signature data.Such as by the characteristic of weak modality images and different random
Vector is separately input into the feature after training and generates network, and the characteristic of the strong modality images of different puppets can be obtained.
In one possible implementation, by the characteristic of the first image be input to feature generate network into
Row processing, before obtaining the M group target signature data, the method also includes:
The characteristic of first training sample is input to the feature and generates network, obtains target training characteristics data,
Wherein, the dimension of the target training characteristics data is greater than the dimension of the characteristic of first training sample;
The target training characteristics data are inputted and differentiate that network is handled, obtain the first differentiation result;
Differentiated based on described first as a result, determining first-loss;
Based on the first-loss, the training feature generates network.
In the embodiment of the present application, image matching apparatus generates network by feature and exports target training characteristics data, thus
The target training characteristics data are input to differentiation network, makes to differentiate that network is differentiated, such as differentiates the target training characteristics number
According to belonging to strong mould probability of state or score etc.;And then based on differentiate network output first differentiate result determine first-loss come
Training characteristics generate network, generate network by confrontation mode come training characteristics, can effectively improve trained efficiency, step up
The accuracy of target training characteristics data.
In one possible implementation, the characteristic by the first training sample is input to the feature and generates
Network obtains target training characteristics data, comprising:
The characteristic of first training sample and training random vector are input to the feature and generate network, is obtained
The target training characteristics data;
Differentiated based on described first as a result, determining first-loss, comprising:
Target training characteristics data input sorter network is handled, the first classification results are obtained;
Target training characteristics data input random vector Recurrent networks are handled, training is obtained and returns feature,
Wherein, the dimension that the training returns feature is equal to the dimension for inputting the trained random vector;
Differentiate that result, first classification results and the training return feature based on described first, determines described first
Loss.
In the embodiment of the present application, which is inputted into sorter network, by the sorter network to the mesh
Mark training characteristics data exercise supervision, and can further improve the accuracy of target training characteristics number;By the way that target training is special
It levies data and inputs random vector Recurrent networks, obtain training and return feature, to not only can effectively supervise the target training characteristics
The case where data can also further efficiently use random vector, avoid having input random vector, can not efficiently use.
In one possible implementation, the characteristic for obtaining the first image, comprising:
Network is extracted using fisrt feature, feature extraction is carried out to the first image, obtain the feature of the first image
Data;
The method also includes:
According to the first-loss, the training fisrt feature extracts network.
In the embodiment of the present application, which, which extracts network, can also be used in the characteristic for extracting the first training sample.
In one possible implementation, the method also includes:
The characteristic of second training sample is input to the differentiation network to handle, obtain the second differentiation as a result,
Wherein, the information content for including in first training sample that contains much information for including in second training sample;
Differentiate that result and the training return feature based on described second, determines the second loss;
Based on second loss, the training differentiation network, the sorter network and the random vector Recurrent networks.
In one possible implementation, described to differentiate that result and the training return feature based on described second, really
Fixed second loss, comprising:
The characteristic of second training sample is inputted the sorter network to handle, obtains the second classification knot
Fruit;
Differentiate that result, second classification results and the training return feature based on described second, determines described second
Loss.
In one possible implementation, the characteristic of each second image at least two second image is obtained
According to, comprising:
Network is extracted using second feature, and feature extraction is carried out to the second image each at least two second image,
Obtain the characteristic of each second image at least two second image;
The method also includes:
Based on second loss, the training second feature extracts network.
Second aspect, the embodiment of the present application provide a kind of image matching apparatus, comprising:
Acquiring unit, for obtaining the characteristic of the first image;
First data processing unit obtains M group target signature data for the characteristic based on the first image;
Wherein, the M is the integer more than or equal to 2;
First determination unit, for being based on the M group target signature data, the determining and institute from least two second images
State the target image of the first images match.
In one possible implementation, first determination unit can specifically be based on the M group target signature number
The characteristic of each second image accordingly and at least two second image, from least two second image really
The fixed and matched target image of the first image.
In one possible implementation, the dimension of the characteristic of second image is different from the first image
Characteristic dimension, the dimensions of the target signature data is equal to the dimension of the characteristic of second image.
In one possible implementation, the dimension of the characteristic of second image is greater than the first image
The dimension of characteristic.
In one possible implementation, the dimension of the target signature data is greater than the characteristic of the first image
According to dimension.
In one possible implementation, the acquiring unit is also used to obtain at least two second image
The characteristic of each second image.
In some instances, which can propose the second image each at least two second images progress feature
Processing is taken, the characteristic of each second image is obtained.
In other examples, acquiring unit can obtain each second image at least two second images from memory
Characteristic.
In one possible implementation, first determination unit includes:
First determine subelement, for be based on the M group target signature data, determine the first image with it is described at least
The similarity of each second image in two the second images;
Second determines subelement, for based on each second figure in the first image and at least two second image
The similarity of picture, the determining and matched target image of the first image from least two image.
In one possible implementation, it described first determines subelement, is specifically used for obtaining described at least two the
The characteristic of each second image in two images;And it is based on the M group target signature data and described at least two second
The characteristic of each second image in image obtains the corresponding M similarity of each second image;And according to described
The corresponding M similarity of each second image, determines the first image and described each second at least two second images
The similarity of image.
In one possible implementation, it described first determines subelement, is specifically used for second image is corresponding
The average treatment result of M similarity be determined as the similarity of the first image Yu second image.
In one possible implementation, the information content that the first image includes is less than second image
Information content.
In one possible implementation, the acquiring unit, specifically for obtaining first figure from memory
The characteristic of picture.
In one possible implementation, first data processing unit is specifically used for being based on the first image
Characteristic and M random vector, obtain the M group target signature data.
In one possible implementation, first data processing unit, specifically for by the first image
Characteristic is input to feature generation network and is handled, and obtains the M group target signature data.
In one possible implementation, described device further include:
Second data processing unit generates network for the characteristic of the first training sample to be input to the feature,
Obtain target training characteristics data;
First judgement unit differentiates that network is handled for inputting the target training characteristics data, obtains first
Differentiate result;
Second determination unit, for differentiating based on described first as a result, determining first-loss;
First training unit, for being based on the first-loss, the training feature generates network.
In one possible implementation, second data processing unit is specifically used for the first training sample
This characteristic and training random vector is input to the feature and generates network, obtains the target training characteristics data;
Second determination unit includes:
Classification processing subelement obtains for handling target training characteristics data input sorter network
One classification results;
Processing subelement is returned, for inputting the target training characteristics data at random vector Recurrent networks
Reason obtains training and returns feature, wherein the dimension that the training returns feature is equal to the dimension for inputting the trained random vector
Degree;
Third determines subelement, for differentiating that result, first classification results and the training are returned based on described first
Return feature, determines the first-loss.
In one possible implementation, the acquiring unit is specifically used for extracting network to institute using fisrt feature
It states the first image and carries out feature extraction, obtain the characteristic of the first image;
First training unit is also used to according to the first-loss, and the training fisrt feature extracts network.
In one possible implementation, described device further include:
Second judgement unit is handled for the characteristic of the second training sample to be input to the differentiation network,
Obtain the second differentiation result, wherein containing much information of including in second training sample include in first training sample
Information content;
Third determination unit determines the second loss for differentiating that result and the training return feature based on described second;
Second training unit, for based on second loss, the training differentiation network, the sorter network and described
Random vector Recurrent networks.
In one possible implementation, the third determination unit, specifically for by second training sample
Characteristic inputs the sorter network and is handled, and obtains the second classification results;And result, institute are differentiated based on described second
It states the second classification results and the training returns feature, determine second loss.
In one possible implementation, the acquiring unit is specifically used for extracting network to institute using second feature
It states each second image at least two second images and carries out feature extraction, obtain each at least two second image
The characteristic of two images;
Second training unit is also used to based on second loss, and the training second feature extracts network.
The third aspect, the embodiment of the present application provide a kind of image matching apparatus, comprising: processor and memory;It is described to deposit
Reservoir is used to couple with the processor, and program instruction and data needed for preservation described image coalignment;The place
Reason device is configured as the corresponding function in the method for supporting described image coalignment to execute first aspect.
In one possible implementation, described image coalignment can also include input/output interface, described defeated
Enter output interface, for supporting the communication between described device and other devices.
Fourth aspect, the embodiment of the present application provide a kind of computer readable storage medium, the computer-readable storage
Readable instruction is stored in medium, when run on a computer, so that computer executes method described in the various aspects.
5th aspect, the embodiment of the present application provides a kind of computer program product comprising instruction, when it is in computer
When upper operation, so that computer executes method described in the various aspects.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application or in background technique below will be implemented the application
Attached drawing needed in example or background technique is illustrated.
Fig. 1 is a kind of flow diagram of image matching method provided by the embodiments of the present application;
Fig. 2 is a kind of concrete scene schematic diagram of images match provided by the embodiments of the present application;
Fig. 3 is a kind of flow diagram of training method provided by the embodiments of the present application;
Fig. 4 is the flow diagram of another training method provided by the embodiments of the present application;
Fig. 5 is the flow diagram of another training method provided by the embodiments of the present application;
Fig. 6 is a kind of concrete scene schematic diagram of training method provided by the embodiments of the present application;
Fig. 7 is a kind of structural schematic diagram of image matching apparatus provided by the embodiments of the present application;
Fig. 8 is a kind of structural schematic diagram of first determination unit provided by the embodiments of the present application;
Fig. 9 is the structural schematic diagram of another image matching apparatus provided by the embodiments of the present application;
Figure 10 is a kind of structural schematic diagram of second determination unit provided by the embodiments of the present application;
Figure 11 is the structural schematic diagram of another image matching apparatus provided by the embodiments of the present application;
Figure 12 is the structural schematic diagram of another image matching apparatus provided by the embodiments of the present application.
Specific embodiment
Term " first ", " second " in the description and claims of this application and attached drawing etc. are for distinguishing difference
Object, be not use to describe a particular order.In addition, term " includes " and " having " and their any deformations, it is intended that
Non-exclusive include in covering.Such as the process, method, system, product or equipment for containing a series of steps or units do not have
It is defined in listed step or unit, but optionally further comprising the step of not listing or unit, or optionally further comprising
Other step or units intrinsic for these process, methods or equipment.
The application is described in further detail below in conjunction with attached drawing.
It is a kind of flow diagram of image matching method provided by the embodiments of the present application referring to Fig. 1, Fig. 1, this method can
Applied to image matching apparatus, which can be server or terminal device etc., and the embodiment of the present application is for this
Image matching apparatus is which kind of equipment does not make uniqueness restriction.
As shown in Figure 1, the image matching method includes:
S101, the characteristic for obtaining the first image.
In the embodiment of the present application, the first image can be arbitrary image, such as still image or video frame images etc., sheet
Application embodiment is not construed as limiting first image.
In some possible embodiments, network is extracted using fisrt feature and feature extraction is carried out to the first image, obtain
To the characteristic of the first image.
Wherein, optionally, which, which extracts network, can be used deep learning network, such as can pass through ResNet network
To obtain the characteristic of the first image.Further, which may include N number of convolutional layer, and first image is successively
By the processing of N number of convolutional layer, the characteristic of first image is obtained.
It is understood that deep learning network illustrated above is only a kind of example, should not be construed as to the embodiment of the present application
It limits.
Optionally, the specific implementation of the characteristic of the first image is obtained can include: the image matching apparatus is direct
The characteristic of first image is obtained by the processor of the image matching apparatus, that is to say, that the image matching apparatus exists
It needs directly to pass through processor reality from least two second images in the case where the determining target image with the first images match
When obtain the characteristic of first image.
Alternatively, obtain the first image characteristic specific implementation may also include that the image matching apparatus from this
The characteristic of first image is obtained in the memory of image matching apparatus.That is, the characteristic of first image
It has been obtained that, and be stored in memory, needed determining and the first image from least two second images in image matching apparatus
In the case where matched target image, which can directly obtain the characteristic of first image from memory
According to.To can also effectively save the time, the speed and efficiency of matching target image are improved.
Alternatively, the image matching apparatus can also obtain the characteristic of first image from other devices, i.e., by being somebody's turn to do
Other devices obtain the characteristic of the first image, then when image matching apparatus needs to implement the embodiment of the present application, from it
He obtains the characteristic of first image in device, such as server receives the characteristic of the first image from terminal device
According to.It is understood that the characteristic how the embodiment of the present application obtains first image for image matching apparatus does not make uniqueness
It limits.It is understood that in the case where the image matching apparatus obtains the characteristic of first image from other devices, this Shen
Please embodiment how the first image is obtained for other devices characteristic be not construed as limiting.And the embodiment of the present application pair
It is specially which kind of device is not construed as limiting in other devices, can is also server etc. if other devices can be terminal device.
S102, the characteristic based on the first image, obtain M group target signature data, and M is the integer more than or equal to 2.
In the embodiment of the present application, the spy of all corresponding first image of each group of target signature data in M group target signature data
Levy data.
Optionally, the dimension of target signature data is greater than the dimension of the characteristic of the first image.
In some possible implementations, based on the characteristic of the first image, M group target signature data are obtained, are wrapped
It includes:
Characteristic and M random vector based on the first image obtain M group target signature data.
In the embodiment of the present application, different target signature data can be obtained based on different random vectors.That is,
At least two target signature data can be obtained using the characteristic of the first image as condition, using random vector as variable.Specifically
, which may include Gaussian random vector, that is, include the random vector sampled from Gaussian Profile, or can also
Think other kinds of random vector, the embodiment of the present application is not construed as limiting the specific implementation of random vector.
In the embodiment of the present application, by random vector, it can be obtained a variety of possible based on the characteristic of first image
Target signature data obtain the efficiency of target signature data to improve.
Specifically, the characteristic based on the first image, obtains M group target signature data, comprising:
The characteristic of first image is input to feature generation network to handle, obtains M group target signature data.
In the embodiment of the present application, feature, which generates network, can be used for generating M group target signature data.It further, can be with
The characteristic of first image and M random vector are input to this feature generation network to handle, it is special to obtain M group target
Levy data.
For example, in the case where the information content for including less than the second image in the information content that the first image includes, i.e., the
One image is weak modality images, and the second image is strong modality images, then implements the embodiment of the present application, can be by weak modality images
Characteristic and M random vector (or being M different random vectors) are input to feature and generate network, to obtain a system
The strong modal characteristics data of the puppet of column.Wherein, pseudo- strong modal characteristics data can be regarded as feature and generate dimension that network obtains and strong
The identical characteristic of the characteristic of modality images.
In the embodiment of the present application, this feature, which generates network, can generate network for the feature of image matching apparatus training,
After can also be for other devices such as training device training, the feature for being sent to the image matching apparatus generates network.That is, this
Apply in embodiment, which can receive the feature from training device training and generate network, alternatively, the image
It can also oneself training this feature generation network with device.It is understood that how the embodiment of the present application instructs the training device
Practice feature generation network to be not construed as limiting.
S103, M group target signature data are based on, the determining target with the first images match from least two second images
Image.
In the embodiment of the present application, target image is the image with the first images match.
In the embodiment of the present application, the information content that the first image includes can be suitable with the information content that the second image includes, or
Person, the information content that the first image includes is fewer than the information content that the second image includes.For example, the first image may include sketch
(sketches), the second image may include photo (photos);For another example the first image may include gray level image (grayscale
Images), the second image may include color image (color images);For another example the first image may include low-resolution image
(low resolution images), the second image may include high-definition picture (high resolution image).Or
Person, the information content that the first image includes are more than the information content that the second image includes.
In the case where the information content for including less than the second image in the information content that the first image includes, i.e. the first image packet
The information content contained, can be by first image less than the information content that any one second image at least two second images includes
Second image, is known as the image of strong mode by the image of referred to as weak mode.Since the information content for including in weak modality images is far small
Information content in strong modality images, weak modality images possibly can not provide enough information to determine matched strong mode
Image, at this point, example as shown in Figure 2, might have multiple matched strong modality images, the embodiment of the present application passes through base
Multiple groups target signature data are obtained in the characteristic of weak modality images, and are determined based on the multiple groups target signature data multiple strong
In modality images with the matched image of weak modality images, can obtain at least one matched target image of weak modality images,
And be conducive to improve the reliability of images match.
In some possible implementations, be based on M group target signature data, from least two second images determine and
The target image of first images match, comprising:
Based on M group target signature data, the phase of the first image with each second image at least two second images is determined
Like degree;
Based on the similarity of each second image in the first image and at least two second images, from least two images
The determining target image with the first images match.
In the embodiment of the present application, can be determined according to M group target signature data the first image and each second image it
Between similarity, to carry out the similarity based on each second image in the first image and at least two second images, from least
Target image is determined in two the second images.For example, can by least two second images with the first image similarity highest
Image be determined as target image, or can also will be higher than default threshold at least two second images with the first image similarity
The image of value is determined as target image, or can also will be higher than at least two second images with the first image similarity default
Preceding several images in one or more candidate images of threshold value are determined as target image, etc., the embodiment of the present application to this not
It limits.
In some possible implementations, M group target signature data are based on, determine the first image and at least two second
The similarity of each second image in image, comprising:
Obtain the characteristic of each second image at least two second images;
Based on the characteristic of each second image in M group target signature data and at least two second images, obtain every
The corresponding M similarity of a second image;
According to the corresponding M similarity of the second image each at least two second images, the first image and each is determined
The similarity of second image.
In the embodiment of the present application, the M group target signature data and the second figure can be calculated separately based on M group target signature data
Similarity between the characteristic of picture obtains the corresponding M similarity of the second image, and M corresponding according to the second image
Similarity determines the similarity of the first image and the second image.
Wherein, the embodiment of the present application also provides a kind of methods of characteristic for obtaining the second image, comprising:
Network is extracted using second feature, feature extraction is carried out to the second image each at least two second images, obtain
The characteristic of each second image.
Wherein, which, which extracts network, can be used deep learning network, can such as be obtained by ResNet network every
The characteristic of a second image.Further, which may include i convolutional layer, which can successively lead to
The process of convolution for crossing the i convolutional layer obtains the characteristic of second image.
It is understood that the weight parameter that the second feature extracts network can extract the weight parameter phase of network with fisrt feature
Together, i.e., the fisrt feature extracts network network parameter same with second feature extraction network share.For example, image
The same deep learning network can be used with device to extract the characteristic of the first image and each second image.Alternatively, should
The weight parameter and the fisrt feature extraction weight parameter of network of second feature extraction network are different, such as the image matching apparatus
The deep learning network for extracting the first image is different from the parameter of deep learning network of the second image is extracted.Specifically, using
Different deep learning networks extracts the characteristic of the first image and the second image, and the accurate of feature extraction also can be improved
Property.
Optionally, the specific implementation for obtaining the characteristic of each second image at least two second images can wrap
Include: the image matching apparatus obtains each second figure at least two second image by the processor of the image matching apparatus
The characteristic of picture, that is to say, that when image matching apparatus needs to implement the embodiment of the present application, can be obtained in real time by processor
Take the characteristic of each second image.
Alternatively, obtaining the specific implementation of the characteristic of each second image at least two second images can also wrap
Include: the image matching apparatus obtains the characteristic of each second image from the memory of the image matching apparatus.Also
To say, the characteristic of the second image is pre-stored in memory, image matching apparatus need from this at least two second
In the case where determining target image in image, which can obtain the characteristic of the second image from memory
According to.
Alternatively, the image matching apparatus can also obtain the characteristic of each second image from other devices.It can manage
Solution, the method for how obtaining the characteristic of each second image for the image matching apparatus can also be corresponded to reference to the image
The method that the characteristic of the first image how is obtained with device, is no longer described in detail one by one here.
In some possible implementations, according to the corresponding M phase of the second image each at least two second images
Like degree, the similarity of the first image Yu each second image is determined, comprising:
It is similar to the second image that the average treatment result of the corresponding M similarity of second image is determined as the first image
Degree.
In the embodiment of the present application, the average treatment result of the corresponding M similarity of the second image can include: second image
The arithmetic mean result, etc. of the result of weighted average of corresponding M similarity or the corresponding M similarity of second image,
The embodiment of the present application is not construed as limiting specific implementation.
In one embodiment, based on each second image in M group target signature data and at least two second images
Characteristic obtains the formula of the corresponding M similarity of each second image, can be as follows:
Wherein, f indicates the feature vector of the second image,AndFs indicates that second feature mentions
Take network, XgIndicate that the second image, i indicate any one second image at least two second images.It indicates by the
The feature vector of i-th of second images at least two second images that two feature extraction networks extract.
Wherein, fqIndicate the feature vector of the first image, fq=Fw (Xq).Fw indicates that fisrt feature extracts network, Fw (Xq)
It indicates to extract the first image X that network extracts by fisrt featureqFeature vector.gjIndicate j-th of target signature data, and
gj=G (fq,zj), { gj}J=1:m.Wherein, zjIndicate from multivariate Gaussian distribution N (0, I) j-th obtained through stochastical sampling it is random
Vector.Wherein, the I in N (0, I) indicates that special diagonal line (identity matrix) is 1 matrix.G(fq,zj) indicate first
Image XqAnd j-th of random vector is input to after feature generates network, obtained j-th of target signature data.
It is understood that the method that can determine average treatment according to the value of σ is such as corresponding for different j for formula (2)
σ it is identical, then it is similar to the second image to be determined as the first image for the arithmetic mean result of the corresponding M similarity of the second image
Degree.Such as corresponding for different j σ value is different, then the result of weighted average of the corresponding M similarity of the second image is determined as the
The similarity of one image and the second image.
In the embodiment of the present application, network is generated by the way that the characteristic of the first image is input to feature, obtains M group target
Characteristic, thus according to the characteristic of each second image in the M group target signature data and at least two second images
According to determining target image.Implement the embodiment of the present application, can effectively make up since the information content of the first image is very few, and leads to the
Huge estrangement when mode vectors correlation is carried out between one image and the second image, it is effective to improve the accuracy for determining target image.
Further, the embodiment of the present application passes through the characteristic for obtaining the first image, and obtains at least two second
The characteristic of each image in image, i.e., determine similarity by the level in characteristic, rather than directly in image
Level determines similarity.On the one hand, high-level feature (i.e. the level of characteristic) usually dimension is lower, therefore and low level
Feature (i.e. image hierarchy) is compared, and learns feature generator or training characteristics generator in high-level feature, it is more easy and
Cost is smaller;On the other hand, high-dimensional feature is closer to semantic space, at this level, the difference between different modalities
It is smaller, therefore be easier to be connected.
For image matching method provided by vivider understanding the embodiment of the present application, the letter for including with the first image below
Illustrate for the information content that breath amount includes less than the second image, at this point, image matching method is true from multiple strong modality images
The fixed and matched image of weak modality images, i.e. matching of the progress across modality images.
Referring to fig. 2, Fig. 2 is a kind of concrete scene schematic diagram of images match provided by the embodiments of the present application, such as Fig. 2 institute
Show, leftmost three images (sketch image) indicate the first image in figure, remaining image (photo) indicates the second figure
Picture.Therefrom it can also be seen that the information content of the first image is considerably less than the information content of the second image.By extracting the first image
Characteristic, to be input to, feature generates network, obtains M group target signature data, and then be based on the M group target
After characteristic determines the similarity of the first image and each second image, target image can be obtained.It is understood that the target
Image may also indicate that in the second image with the most matched image of the similarity of the first image.
As shown in Fig. 2, the target image can be followed successively by the first row from the right number second, the second row is from the right number third
A and the third line is from the right number the 4th.That is, by implementing the embodiment of the present application, it can be quick, effectively from a large amount of
(being best suitable for) photo corresponding with sketch image is obtained in photo.It is understood that each sketch shown in Fig. 2 or photo are only
A kind of example should not be construed as having restriction to the embodiment of the present application.
Fig. 1 is described in detail the embodiment of the present application is how to determine from least two second images and the first image
The target image matched.Wherein, feature, which generates network, can be the received feature generation network from other devices, be also possible to
The image matching apparatus oneself training, therefore, by taking oneself training characteristics of the image matching apparatus generate network as an example, below will
Specifically introduce training method.
It is understood that in Fig. 3 and training method shown in Fig. 4, any one image includes in the first training sample information
Amount can be suitable with the information content that any one image in the second training sample includes;Alternatively, any one in the first training sample
The information content that a image includes is fewer than the information content that any one image in the second training sample includes;Alternatively, the first instruction
Practice the information that the information content that any one image includes in sample can also include more than any one image in the second training sample
Amount.
But it is specifically chosen which kind of mode also corresponds to image matching method shown in Fig. 2.For example, shown in Fig. 2
Image matching method in, information content that the first image includes is less than the information content that the second image includes, then in training, this
The information content that any one image includes in one training sample just needs
Information content.
Therefore, Fig. 3 and training method shown in Fig. 4 should not be interpreted as to the restriction to the embodiment of the present application.
It is a kind of flow diagram of training method provided by the embodiments of the present application referring to Fig. 3, Fig. 3, as shown in figure 3, should
Method includes:
S301, the characteristic of the first training sample is input to feature generation network, obtains target training characteristics data.
In the embodiment of the present application, the image in the first training sample can be arbitrary image, such as still image or view
Frequency frame image etc., the embodiment of the present application are not construed as limiting image included in first training sample.It is understood that target
The dimension of training characteristics data is greater than the dimension of the characteristic of the first training sample.
Specifically, image matching apparatus can extract network by fisrt feature and extract the first training sample before S301
This characteristic.
It is understood that the weight parameter that fisrt feature extracts network can extract the weight parameter phase of network with second feature
Together, alternatively, the weight parameter that fisrt feature extracts network can also be different from the second feature extraction weight parameter of network.Also
To say, extract the characteristic of the first training sample feature extraction network (i.e. fisrt feature extraction network) can with extract the
The feature extraction network (i.e. second feature extraction network) of the characteristic of two training samples shares weight parameter, alternatively, can also
Not share.Network is extracted for the fisrt feature and the second feature extracts the specific implementation of network, reference may also be made to Fig. 1
Shown in specific implementation, I will not elaborate.
Specifically, the characteristic of the first training sample, which is input to feature, generates network, target training characteristics number is obtained
According to, comprising:
The characteristic of first training sample and training random vector are input to feature and generate network, obtains target training
Characteristic;Wherein, the target training characteristics data that different training random vectors obtains are different.
In the embodiment of the present application, different training random vectors is inputted, different target training characteristics data can be obtained.It should
Train random vector concretely Gaussian random vector.For example, the number of image is ten in the first training sample, then may be used
It is condition by one of training sample, using different Gaussian random vectors as variable, is separately input into feature and generates network, from
And at least two target training characteristics data can be obtained according to a training sample (i.e. an image).As a result, due to can basis
Ten the first training samples obtain at least 20 target training characteristics data.It is understood that the Gaussian random vector can be from height
Therefore the random vector sampled in this distribution is not construed as limiting the specific vector of the Gaussian random vector.Alternatively, should
Training random vector can also be other kinds of random vector etc..
S302, the input of target training characteristics data is differentiated that network is handled, obtains the first differentiation result.
In the embodiment of the present application, differentiate that network can be used for distinguishing the feature of target training characteristics data and the second training sample
Data.Such as differentiate that the first differentiation result of network output can be exported in the form of percentage (i.e. probability), that is, differentiates the spy of input
Sign data belong to the percentage of strong mode;Alternatively, the first of differentiation network output differentiates that result can be in the form of score system
Output, that is, differentiate that the characteristic of input belongs to the score of strong mode.Such as the first differentiation result can be used for indicating that the target is instructed
Practice the probability that characteristic belongs to strong modal characteristics.Or the first differentiation result can be used for indicating the target training characteristics data category
In the score etc. of strong modal characteristics.
S303, differentiate based on first as a result, determining first-loss.
Specifically, being differentiated based on first as a result, determining first-loss, comprising:
Target training characteristics data input sorter network is handled, the first classification results are obtained;
Target training characteristics data input random vector Recurrent networks are handled, training is obtained and returns feature, wherein
The dimension that training returns feature is equal to the dimension for inputting training random vector;
Differentiate that result, the first classification results and training return feature based on first, determines first-loss.
In the embodiment of the present application, sorter network can be used for supervising target training characteristics data.Random vector Recurrent networks can
For random vector can be efficiently used when training characteristics generate network.It is understood that in the embodiment of the present application, by target
It, such as can be by target training characteristics data and the target training characteristics when training characteristics data input sorter network is handled
The corresponding class label of data is input to the sorter network, thus output probability corresponding with such distinguishing label (i.e. output target
The classification results of training characteristics data i.e. the first classification results).Wherein, class label may particularly include identity, alternatively, should
Class label may also comprise classification logotype.For example, animal cat and dog are two class labels, if further distinguishing,
Then two Kazakhstan and Ke Ji may belong to two class labels.Therefore, class label is specifically distinguished to what in the embodiment of the present application
Kind degree is not construed as limiting.
In the embodiment of the present application, first-loss can be in the form of function etc., and the embodiment of the present application is not construed as limiting.
S304, it is based on first-loss, training characteristics generate network.
It is understood that training method described in the embodiment of the present application specifically may also be understood to be the weight of trained corresponding network
Parameter updates the weight parameter of corresponding network.It is such as based on first-loss, training characteristics generate network, also are understood as being based on
The first-loss updates the weight parameter that this feature generates network.
Specifically, when training characteristics generate network, can be instructed by the way of backpropagation utilizing first-loss
Practice.It, can deconditioning and when the first differentiation result for differentiating result output meets objective result.Since feature generates net
Network and differentiation network are trained in confrontation, therefore, can not identify that the characteristic of input belongs to strong mould in differentiation network
When the characteristic of the characteristic of state or pseudo- strong mode, then illustrates that feature generation network has been trained and finish, indicate this feature
The characteristic of the strong mode of puppet of network output is generated to the greatest extent close to the characteristic of strong mode.Therefore, target knot
The setting of fruit can be related to the differentiation result of result output is differentiated, as differentiated, result is the characteristic of input belongs to strong mode
Percentage, then the objective result can be 1, alternatively, the objective result may be greater than 0.8 etc., the embodiment of the present application is for the mesh
How mark result, which limits, is not construed as limiting.
In the embodiment of the present application, image matching apparatus generates network by feature and exports target training characteristics data, thus
The target training characteristics data are input to differentiation network, make to differentiate that network is differentiated (i.e. differentiation target training characteristics data
Belong to strong mould probability of state or score etc.);Network is generated come training characteristics by confrontation mode and differentiates network, can be effectively improved
Trained efficiency steps up the accuracy of target training characteristics data output.
For vivider description training method shown in Fig. 3, described below with specific formula.
Differentiate that network is handled for example, target training characteristics data are inputted, the formula for obtaining the first differentiation result can
It is as follows:
Wherein, D indicates to differentiate that network, G can indicate that feature generates network, and Fw indicates that fisrt feature extracts network.G(Fw
(X), z) indicate that the first training sample X is input to fisrt feature extracts network, obtains the training characteristics of the first training sample
Data Fw (X);Then the characteristic of first training sample and training random vector are input to feature and generate network, obtained
To target training characteristics data G (Fw (X), z).D (G (Fw (X), z)) indicates output, that is, target instruction that feature is generated to network
Practice characteristic and be input to differentiation network, thus to obtain the first differentiation result.
For example, target training characteristics data input sorter network is handled, the formula for obtaining the first classification results can
It is as follows:
Wherein, Wc presentation class network.
For example, target training characteristics data input random vector Recurrent networks are handled, obtains training and return feature
Formula can be as follows:
Wherein, E indicates random vector Recurrent networks.For formula (5), target training characteristics data and instruction are inputted
Practice random vector, then the exportable target training characteristics data with class label are input to random vector Recurrent networks again
In obtain output result.Wherein, for example, Fw (X) is the vector of 1*10, and z is the vector of 1*12, then G (Fw (X), z) is 1*
10 vector, E (G (Fw (X), z)) are then the vector of 1*12.
For example, differentiating that result, the first classification results and training return feature based on first, determine that the formula of first-loss can
It is as follows:
It is understood that being to illustrate how to determine first-loss using loss function as example above.
Specifically, method shown in Fig. 3 further include: according to first-loss, training fisrt feature extracts network.
It, can be respectively to G and Fw derivation, so that carrying out more new feature generates network and fisrt feature extraction such as formula (6)
Network.
That is, the embodiment of the present application, can also update the power that fisrt feature extracts network according to the first differentiation result
Weight parameter.
Be described in detail in training method shown in Fig. 3 how training characteristics generate network, it is described in detail below how
Training differentiates network.Referring to fig. 4, Fig. 4 is the flow diagram of another training method provided by the embodiments of the present application, such as Fig. 4
Shown, which includes:
S401, by the characteristic of the second training sample be input to differentiate network handle, obtain the second differentiation as a result,
Wherein, the information content for including in the first training sample that contains much information for including in the second training sample.
In the embodiment of the present application, for the second training sample, which can be the image of weak mode,
Second training sample can be the image of strong mode.It is understood that since first training sample and second training sample are used for
Training characteristics generate network, therefore, the quantity of first training sample and second training sample can be related to training degree etc.
Deng the embodiment of the present application is not construed as limiting the quantity of first training sample He second training sample.
In the embodiment of the present application, the characteristic of target training characteristics data and the second training sample can phase in dimension
Together, that is to say, that in order to further increase the accuracy for differentiating that network differentiates, respectively by the characteristic of the second training sample
When being input to differentiation network with target training characteristics data, it is ensured that the target training characteristics data and second training sample
The dimension of characteristic is identical.For example, the characteristic of second training sample is b1*M, the target training characteristics data
For b2*M, i.e. the quantity b1 of the characteristic of second training sample can be different from the quantity b2 of the target training characteristics data,
Or can also be identical, the application is not construed as limiting;But dimension is identical.
In the embodiment of the present application, the dimension of the characteristic of the second training sample is different from the characteristic of the first training sample
According to dimension.Optionally, the dimension of the characteristic of second training sample is greater than the characteristic of first training sample
Dimension.
It, can also be by the specifically, before the characteristic of the second training sample is input to differentiating that network handled
Two feature extraction networks extract the characteristic of the second training sample.It is understood that extracting the specific of network for the second feature
Implementation can refer to Fig. 1 and specific implementation shown in Fig. 3, no longer be described in detail one by one here.
S402, differentiate that result and training return feature based on second, determine the second loss.
Specifically, differentiating that result and training return feature based on second, the second loss is determined, comprising:
The characteristic input sorter network of second training sample is handled, the second classification results are obtained;
Differentiate that result, the second classification results and training return feature based on second, determines the second loss.
It is understood that the specific implementation for returning feature for sorter network and training can join in the embodiment of the present application
Implementation shown in Fig. 3 is examined, which is not described herein again.
In the embodiment of the present application, the second loss can be in the form of function etc., and the embodiment of the present application is not construed as limiting.
S403, it is lost based on second, training differentiates network, sorter network and random vector Recurrent networks.
For vivider description training method shown in Fig. 4, described below with specific formula.
Differentiate that network is handled for example, the characteristic of the second training sample is input to, obtains the second differentiation result
Formula it is as follows:
Wherein, Fs indicates that second feature extracts network.It is understood that before reference may also be made to for the parameters in formula (7)
State formula described in embodiment.
It is handled for example, the characteristic of the second training sample is input to sorter network, obtains the second classification results
Formula it is as follows:
In the embodiment of the present application, for formula (4) and formula (8), it will be appreciated that can be by for the weight parameter of the sorter network
Second training sample is trained, and when using the sorter network, it can guarantee that target training is special by the sorter network
The class label of data is levied, to avoid the characteristic for generating classification notable difference.
For example, based on the second loss, training differentiates the following institute of formula of network, sorter network and random vector Recurrent networks
Show:
It is understood that can differentiate network, classification net to update respectively respectively to D, Wc, Fs and E derivation for formula (9)
Network, second feature extract network and random vector Recurrent networks.
It is the flow diagram of another training method provided by the embodiments of the present application referring to Fig. 5, Fig. 5, as shown in figure 5,
This method comprises:
S501, the characteristic of the first training sample is input to feature generation network, obtains target training characteristics data.
S502, the input of target training characteristics data is differentiated that network is handled, obtains the first differentiation as a result, target is instructed
Practice characteristic input sorter network to be handled, obtains the first classification results.
S503, by the characteristic of the second training sample be input to differentiate network handle, obtain the second differentiation as a result,
The characteristic input sorter network of second training sample is handled, the second classification results are obtained.
S504, target training characteristics data input random vector Recurrent networks are handled, obtains training and returns feature.
S505, differentiate that result, the first classification results and training return feature based on first, determine first-loss.
S506, differentiate that result, the second classification results and training return feature based on second, determine the second loss.
S507, it is based on first-loss, training characteristics generate network and fisrt feature extracts network.
S508, it is lost based on second, training differentiates that network, sorter network, random vector Recurrent networks and second feature mention
Take network.
It is understood that generating network in the embodiment of the present application in training characteristics and fisrt feature extracts network and training is sentenced
When other network, sorter network, random vector Recurrent networks and second feature extract network, two stages can be divided into train,
For example first training differentiates that network, sorter network, random vector Recurrent networks and second feature extract network, then retraining training
It generates network and fisrt feature extracts network.Alternatively, first training characteristics generate network and fisrt feature extracts network, then instruct again
Practice and differentiates that network, sorter network, random vector Recurrent networks and second feature extract network.And institute in the embodiment of the present application
The training method of offer can also continuous training stage by stage, until achieving the desired results.
Fig. 3 and method shown in Fig. 4 can refer to for training method shown in fig. 5, are no longer described in detail one by one here.
It is a kind of training provided by the embodiments of the present application referring to Fig. 6, Fig. 6 for the understanding training method shown in fig. 5 of image
The concrete scene schematic diagram of method.As shown in fig. 6, dotted portion indicates the parameter of the network without update, achievement unit in Fig. 6
Dividing indicates that the parameter of the network is updated.
As shown in fig. 6, the output that fisrt feature extracts network can be input to, feature generates network and this feature generates net
The output of network, which can input, is separately input into random vector Recurrent networks, sorter network and differentiation network.And second feature mentions
It takes the output of network that can be input to and differentiates network and sorter network.
It is understood that being propagated in the figure shown in the left side Fig. 6 according to direction, feature generates network and can receive from differentiation net
Network and the common supervisory signals of sorter network, so that target training characteristics data can either guarantee original identity information,
It is consistent again in the upper feature space distribution with the second training sample of the feature space distribution of generation.Shown on the right of Fig. 6
In figure, differentiate that network can receive the target training characteristics data and second feature extraction that network output is generated from feature respectively
The characteristic of second training sample of network output, thus the difference being distributed between two training samples of study.
According to implementation above mode it is found that the one kind that inputs that feature shown in fig. 6 generates network can be to mention from fisrt feature
The characteristic for the first training sample for taking network to extract;A kind of differentiation result to be exported from differentiation network.Namely
It says, on the one hand feature, which generates network, constantly can generate target training characteristics data according to the characteristic of the first training sample, separately
On the one hand, also network can be generated to update this feature according to the differentiation result of network output is differentiated.
Specifically, differentiate network output differentiation result not only can training characteristics generate network, can also update differentiation network.
Vivider says, wherein this feature generate network purpose be in order to enable the characteristic of the strong modality images of puppet arrived more
Really, the purpose for differentiating network is to determine the characteristic of input as far as possible and belong to the characteristic of strong mode also
It is the characteristic of pseudo- strong mode.Differentiate that network and feature generate network and form confrontation as a result, it is raw that training characteristics are carried out in confrontation
At network and the differentiation network.
It is understood that the confusion of data is avoided in order to further increase training effectiveness, therefore, the first training sample is defeated
Enter to feature and generate network, during obtaining target training characteristics data, the weight parameter for differentiating network can be fixed;And
By the characteristic of the second training sample be input to differentiate network when, can fixed character generate network weight parameter;And it will sentence
When other result is input to feature generation network, the weight parameter of the differentiation network is fixed.Thus, it is possible to effectively avoid heterogeneous networks
Between the confusion that changes, and characteristic is caused to input, or training effectiveness is caused to reduce.
In the embodiment of the present application, network is generated come training characteristics using two stages training method and differentiates network, on the one hand,
Network can be generated come training characteristics according to the differentiation result of network output is differentiated;It on the other hand, can also be according to sorter network
Training this feature generates network, guarantees the characteristic of target training characteristics data Yu the first training sample by sorter network
Difference, trained efficiency is further improved, to improve the accuracy of target image.
It is understood that Fig. 1, Fig. 3 emphasize particularly on different fields into method shown in fig. 6, therefore not detailed description in one embodiment
Implementation, can also correspond to reference to other embodiments.
The method of the embodiment of the present application is illustrated, the device of the embodiment of the present application is provided below.
It is a kind of structural schematic diagram of image matching apparatus provided by the embodiments of the present application referring to Fig. 7, Fig. 7, which can
For executing Fig. 1, Fig. 3 to method shown in fig. 5, as shown in fig. 7, the image matching apparatus may include:
Acquiring unit 701, for obtaining the characteristic of the first image;
First data processing unit 702 obtains M group target signature data for the characteristic based on the first image;Its
In, M is the integer more than or equal to 2;
First determination unit 703 determines and for being based on M group target signature data from least two second images
The target image of one images match.
Implement the embodiment of the present application, determine the target image with the first images match based on M group target signature data, also
The accuracy and accuracy that can further improve determining target image avoid being based only upon one group of target signature data to determine mesh
Logo image and the situation for causing accuracy not high.
In one possible implementation, the first determination unit 703, specifically can based on M group target signature data with
And at least two second each second image in image characteristic, determined from least two second image with this first
The target image of images match.
In one possible implementation, the dimension of the characteristic of the second image is different from the characteristic of the first image
According to dimension, the dimensions of the target signature data is greater than the dimension of the characteristic of the second image.
In one possible implementation, the dimension of the characteristic of second image is greater than the feature of first image
The dimension of data.
In one possible implementation, acquiring unit 701 are also used to obtain each at least two second images
The characteristic of two images.
In some instances, which can propose the second image each at least two second images progress feature
Processing is taken, the characteristic of each second image is obtained.
In other examples, acquiring unit can obtain each second image at least two second images from memory
Characteristic.
Optionally, as shown in figure 8, the first determination unit 703 includes:
First determines subelement 7031, for being based on M group target signature data, determines the first image and at least two second
The similarity of each second image in image;
Second determines subelement 7032, for based on each second image in the first image and at least two second images
Similarity, the determining target image with the first images match from least two images.
Specifically, first determines subelement 7031, it is specifically used for obtaining each second image at least two second images
Characteristic;And the characteristic based on each second image in M group target signature data and at least two second images,
Obtain the corresponding M similarity of each second image;And according to the corresponding M of the second image each at least two second images
A similarity determines the similarity of the first image Yu each second image.
Specifically, first determines subelement 7031, specifically for by the average treatment of the corresponding M similarity of the second image
As a result it is determined as the similarity of the first image and the second image.
Specifically, the information content that the first image includes is less than the information content that the second image includes.
Implement the embodiment of the present application, can effectively make up since the information content of the first image is very few, and cause the first image with
Huge estrangement when mode vectors correlation is carried out between second image, it is effective to improve the accuracy for determining target image.
Further, the first data processing unit 702, specifically for based on the first image characteristic and M with
Machine vector obtains M group target signature data.
Specifically, the first data processing unit 702, generates specifically for the characteristic of the first image is input to feature
Network is handled, and M group target signature data are obtained.
Further, as shown in figure 9, image matching apparatus further include:
Second data processing unit 704 generates network for the characteristic of the first training sample to be input to feature, obtains
To target training characteristics data;
Wherein, the dimension of target training characteristics data is greater than the dimension of the characteristic of the first training sample.
First judgement unit 705 differentiates that network is handled for inputting target training characteristics data, obtains first and sentence
Other result;
Second determination unit 706, for differentiating based on first as a result, determining first-loss;
First training unit 707, for being based on first-loss, training characteristics generate network.
Optionally, as shown in Figure 10, the second determination unit 706 includes:
Classification processing subelement 7061 obtains for handling target training characteristics data input sorter network
One classification results;
Processing subelement 7062 is returned, for inputting target training characteristics data at random vector Recurrent networks
Reason obtains training and returns feature, wherein the dimension that training returns feature is equal to the dimension for inputting training random vector;
Third determines subelement 7063, for differentiating that result, the first classification results and training return feature based on first, really
Determine first-loss.
Specifically, acquiring unit 701, proposes the first image progress feature specifically for extracting network using fisrt feature
It takes, obtains the characteristic of the first image;
First training unit 707 is also used to according to first-loss, and training fisrt feature extracts network.
Further, as shown in figure 9, image matching apparatus further include:
Second judgement unit 708 differentiates that network is handled for the characteristic of the second training sample to be input to, obtains
Result is differentiated to second, wherein the information content for including in the first training sample that contains much information for including in the second training sample;
Third determination unit 709 determines the second loss for differentiating that result and training return feature based on second;
Second training unit 710, for based on the second loss, training to differentiate that network, sorter network and random vector return
Network.
In one possible implementation, third determination unit 709, specifically for by the characteristic of the second training sample
It is handled according to input sorter network, obtains the second classification results;And result, the second classification results and instruction are differentiated based on second
Practice and return feature, determines the second loss.
In one possible implementation, the acquiring unit 701 is specifically used for extracting network pair using second feature
Each second image carries out feature extraction at least two second images, obtains each second image at least two second images
Characteristic;
Second training unit 710 is also used to based on the second loss, and training second feature extracts network.
It is understood that the realization of each unit can also correspond to referring to Fig.1, Fig. 3, Fig. 4 and method shown in fig. 5 it is real
Apply the corresponding description of example.
It is the structural schematic diagram of another image matching apparatus provided by the embodiments of the present application, the figure referring to Figure 11, Figure 11
As coalignment can be used for executing Fig. 1, Fig. 3 to method shown in fig. 5, as shown in figure 11, which includes:
Fisrt feature extraction module 1101, this module can extract network based on fisrt feature to extract the feature of the first image
Data, and also network can be extracted based on fisrt feature to extract the characteristic of the first training sample.That is the input of the module
It can be the first image, the input for exporting characteristic (namely feature vector) and the module for the first image can be first
Training sample exports the characteristic (namely feature vector) for the first training sample.It is understood that the module is alternatively referred to as weak mould
State characteristic extracting module.
Second feature extraction module 1102, the module can extract network based on second feature to extract the feature of the second image
Data, and also network can be extracted based on second feature to extract the characteristic of the second training sample, the i.e. input of the module
For the second image, the input for exporting the characteristic (feature vector) and the module for the second image can also be the second training
Sample exports as the characteristic of the second training sample.It is understood that the module is alternatively referred to as strong modal characteristics extraction module.
The input of feature generation module 1103, the module can be the first image extracted from fisrt feature extraction module
Characteristic and the random vector sampled from Gaussian Profile export as target signature data.And the module is defeated
Entering can also sample for the characteristic of the first training sample extracted from fisrt feature extraction module and from Gaussian Profile
Obtained random vector exports as target training characteristics data, specifically, the module multi-layer perception (MLP) can be used as a result, i.e. with
The feature vector that weak mode provides is condition, using the different random vector sampled in Gaussian Profile as variable, in strong mode
Feature space generate a series of possible strong modal characteristics of puppet.Wherein, the module can based on feature generate network come execute with
Upper operation.
Discrimination module 1104, the input of the module are the characteristic of the second image and the characteristic of target image,
Output is the score for belonging to strong modal characteristics.That is, the input of the module can be to extract from second feature extraction module
Characteristic out and the target signature data obtained from feature generation module.Alternatively, the input of the module is the second training
The characteristic and target training characteristics data of sample, export to belong to the score of strong modal characteristics.
Categorization module 1105, the input of the module are class label, and extracted from fisrt feature extraction module
Characteristic exports as the corresponding probability of class label.Feature vector can be converted to classification response by the module, then via returning
One changes exponential function, is changed into the corresponding probability of class label after sampling.
Random vector regression block 1106, the input of the module can be characterized the spy of the strong mode of puppet obtained in generation module
Data are levied, are exported as the feature vector with the dimensions such as random vector.The module can be used recurrence loss function and exercise supervision, and promote
Fisrt feature extraction module can efficiently use the feature vector sampled in Gaussian Profile.It is understood that the random vector is
It is characterized the random vector inputted in generation module.
It is understood that image matching apparatus shown in Figure 11 is that the information content for including includes in the first image less than the second image
Information content in the case where show, optionally, which applies also for other situations, no longer go to live in the household of one's in-laws on getting married one by one here
It states.
It is understood that the specific implementation of image matching apparatus shown in Figure 11, can refer to foregoing individual embodiments, here
No longer it is described in detail one by one.
It is the structural schematic diagram of another image matching apparatus provided by the embodiments of the present application referring to Figure 12, Figure 12.The figure
It can also include input interface 1202, output interface 1203 and memory 1204 as coalignment includes processor 1201.This is defeated
It is connected with each other between incoming interface 1202, output interface 1203, memory 1204 and processor 1201 by bus.
Memory include but is not limited to be random access memory (random access memory, RAM), read-only storage
Device (read-only memory, ROM), Erasable Programmable Read Only Memory EPROM (erasable programmable read
Only memory, EPROM) or portable read-only memory (compact disc read-only memory, CD-ROM),
The memory is used for dependent instruction and data.
Input interface is used for output data and/or signal for input data and/or signal and output interface.Output
Interface and input interface can be independent device, be also possible to the device of an entirety.
Processor may include be one or more processors, for example including one or more central processing unit (central
Processing unit, CPU), in the case where processor is a CPU, which can be monokaryon CPU, be also possible to more
Core CPU.
Memory is used to store the program code and data of image matching apparatus.
Processor is used to call the program code and data in the memory, executes the step in embodiment of the method.
As in one embodiment, processor can be used for executing implementation shown in S101 to S103;For another example processor
It can also be used to execute implementation shown in S301 to S303 etc..
For another example in one embodiment, processor can also be used to execute acquiring unit 701, the first data processing unit 702
With method shown in determination unit 703 etc..
Description in embodiment of the method can be found in for the specific implementation of processor, details are not described herein.
It is designed it is understood that Figure 12 illustrate only simplifying for image matching apparatus.In practical applications, image
Necessary other elements can also be separately included with device, including but not limited to any number of input/output interface, processing
Device, controller, memory etc., and all image matching apparatus that the embodiment of the present application may be implemented are all in the protection model of the application
Within enclosing.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of description, device
It with the specific work process of unit, can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with
It realizes by another way.For example, the division of the unit, only a kind of logical function partition, can have in actual implementation
Other division mode, for example, multiple units or components can be combined or can be integrated into another system or some features
It can ignore, or not execute.Shown or discussed mutual coupling or direct-coupling or communication connection can be logical
Some interfaces are crossed, the indirect coupling or communication connection of device or unit can be electrical property, mechanical or other forms.
Unit may or may not be physically separated as illustrated by the separation member, shown as a unit
Component may or may not be physical unit, it can and it is in one place, or may be distributed over multiple networks
On unit.It can some or all of the units may be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
In embodiment, it can be realized wholly or partly by software, hardware, firmware or any combination thereof.When
When using software realization, can entirely or partly it realize in the form of a computer program product.The computer program product packet
Include one or more computer instructions.When loading and execute on computers the computer program instructions, all or part of real estate
Raw process or function according to the embodiment of the present application.The computer can be general purpose computer, special purpose computer, computer network
Network or other programmable devices.The computer instruction may be stored in a computer readable storage medium, or pass through the meter
Calculation machine readable storage medium storing program for executing is transmitted.The computer instruction can be from web-site, computer, server or a data
The heart passes through wired (such as coaxial cable, optical fiber, Digital Subscriber Line (digital subscriber line, DSL)) or wireless
(such as infrared, wireless, microwave etc.) mode is transmitted to another web-site, computer, server or data center.It should
Computer readable storage medium can be any usable medium that computer can access or include one or more available
The data storage devices such as medium integrated server, data center.The usable medium can be ROM or RAM or magnetic medium,
For example, floppy disk, hard disk, tape, magnetic disk or optical medium, for example, digital versatile disc (digital versatile disc,
DVD) or semiconductor medium, for example, solid state hard disk (solid state disk, SSD) etc..
Claims (10)
1. a kind of image matching method characterized by comprising
Obtain the characteristic of the first image;
Characteristic based on the first image, obtains M group target signature data, and the M is the integer more than or equal to 2;
Based on the M group target signature data, the determining and matched target of the first image from least two second images
Image.
2. the method according to claim 1, wherein described be based on the M group target signature data, from least two
The determining and matched target image of the first image in a second image, comprising:
Based on the M group target signature data, determine each second in the first image and at least two second image
The similarity of image;
Based on the similarity of each second image in the first image and at least two second image, from described at least two
The determining and matched target image of the first image in a image.
3. according to the method described in claim 2, it is characterized in that, it is described be based on the M group target signature data, determination described in
The similarity of each second image in first image and at least two second image, comprising:
Obtain the characteristic of each second image at least two second image;
Based on the characteristic of each second image in the M group target signature data and at least two second image, obtain
To the corresponding M similarity of each second image;
According to the corresponding M similarity of the second image each at least two second image, determine the first image with
The similarity of each second image.
4. according to the method described in claim 3, it is characterized in that, described according at least two second image each
The corresponding M similarity of two images determines the similarity of the first image Yu each second image, comprising:
The average treatment result of the corresponding M similarity of second image is determined as the first image and second figure
The similarity of picture.
5. the method according to claim 1, which is characterized in that the information content that the first image includes
Less than the information content that second image includes, the dimension of the target signature data is greater than the characteristic of the first image
Dimension.
6. according to claim 1 to method described in 5 any one, which is characterized in that the spy based on the first image
Data are levied, M group target signature data are obtained, comprising:
Characteristic and M random vector based on the first image obtain the M group target signature data.
7. according to claim 1 to method described in 6 any one, which is characterized in that the spy based on the first image
Data are levied, M group target signature data are obtained, comprising:
The characteristic of the first image is input to feature generation network to handle, obtains the M group target signature number
According to.
8. a kind of image matching apparatus characterized by comprising
Acquiring unit, for obtaining the characteristic of the first image;
First data processing unit obtains M group target signature data, the M for the characteristic based on the first image
For the integer more than or equal to 2;
First determination unit, it is determining from least two second images and described for being based on the M group target signature data
The target image of one images match.
9. a kind of image matching apparatus, which is characterized in that including processor and memory, the memory is stored with computer can
Reading instruction when the computer-readable instruction is executed by the processor, makes the processor execute such as claim 1 to 7 times
Method described in meaning one.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium
Readable instruction, the computer-readable instruction when being executed by a processor, keep the processor perform claim requirement 1 to 7 any
Method described in one.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810728913.3A CN109101992B (en) | 2018-07-04 | 2018-07-04 | Image matching method, device and computer readable storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810728913.3A CN109101992B (en) | 2018-07-04 | 2018-07-04 | Image matching method, device and computer readable storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109101992A true CN109101992A (en) | 2018-12-28 |
CN109101992B CN109101992B (en) | 2022-02-22 |
Family
ID=64845718
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201810728913.3A Active CN109101992B (en) | 2018-07-04 | 2018-07-04 | Image matching method, device and computer readable storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109101992B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111143601A (en) * | 2019-12-31 | 2020-05-12 | 深圳市芭田生态工程股份有限公司 | Image processing method |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107122826A (en) * | 2017-05-08 | 2017-09-01 | 京东方科技集团股份有限公司 | Processing method and system and storage medium for convolutional neural networks |
CN107451619A (en) * | 2017-08-11 | 2017-12-08 | 深圳市唯特视科技有限公司 | A kind of small target detecting method that confrontation network is generated based on perception |
CN107563509A (en) * | 2017-07-17 | 2018-01-09 | 华南理工大学 | A kind of dynamic adjustment algorithm for the condition DCGAN models that feature based returns |
CN107609637A (en) * | 2017-09-27 | 2018-01-19 | 北京师范大学 | A kind of combination data represent the method with the raising pattern-recognition precision of pseudo- reversal learning self-encoding encoder |
US20180165554A1 (en) * | 2016-12-09 | 2018-06-14 | The Research Foundation For The State University Of New York | Semisupervised autoencoder for sentiment analysis |
-
2018
- 2018-07-04 CN CN201810728913.3A patent/CN109101992B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180165554A1 (en) * | 2016-12-09 | 2018-06-14 | The Research Foundation For The State University Of New York | Semisupervised autoencoder for sentiment analysis |
CN107122826A (en) * | 2017-05-08 | 2017-09-01 | 京东方科技集团股份有限公司 | Processing method and system and storage medium for convolutional neural networks |
CN107563509A (en) * | 2017-07-17 | 2018-01-09 | 华南理工大学 | A kind of dynamic adjustment algorithm for the condition DCGAN models that feature based returns |
CN107451619A (en) * | 2017-08-11 | 2017-12-08 | 深圳市唯特视科技有限公司 | A kind of small target detecting method that confrontation network is generated based on perception |
CN107609637A (en) * | 2017-09-27 | 2018-01-19 | 北京师范大学 | A kind of combination data represent the method with the raising pattern-recognition precision of pseudo- reversal learning self-encoding encoder |
Non-Patent Citations (2)
Title |
---|
JIANAN LI等: "Perceptual Generative Adversarial Networks for Small Object Detection", 《2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 * |
RUIHUA WANG等: "An Effective Image Denoising Method for UAV Images via Improved Generative Adversarial Networks", 《SENSORS》 * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111143601A (en) * | 2019-12-31 | 2020-05-12 | 深圳市芭田生态工程股份有限公司 | Image processing method |
Also Published As
Publication number | Publication date |
---|---|
CN109101992B (en) | 2022-02-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Mou et al. | Vehicle instance segmentation from aerial image and video using a multitask learning residual fully convolutional network | |
CN111797893B (en) | Neural network training method, image classification system and related equipment | |
WO2022012407A1 (en) | Neural network training method and related device | |
CN111401406B (en) | Neural network training method, video frame processing method and related equipment | |
WO2021022521A1 (en) | Method for processing data, and method and device for training neural network model | |
CN112418392A (en) | Neural network construction method and device | |
CN109902546A (en) | Face identification method, device and computer-readable medium | |
CN111507378A (en) | Method and apparatus for training image processing model | |
CN110084281A (en) | Image generating method, the compression method of neural network and relevant apparatus, equipment | |
EP4002161A1 (en) | Image retrieval method and apparatus, storage medium, and device | |
CN111666919B (en) | Object identification method and device, computer equipment and storage medium | |
WO2018196718A1 (en) | Image disambiguation method and device, storage medium, and electronic device | |
CN113807399A (en) | Neural network training method, neural network detection method and neural network detection device | |
WO2021227787A1 (en) | Neural network predictor training method and apparatus, and image processing method and apparatus | |
CN113408570A (en) | Image category identification method and device based on model distillation, storage medium and terminal | |
WO2021190433A1 (en) | Method and device for updating object recognition model | |
CN109034218B (en) | Model training method, device, equipment and storage medium | |
CN112529149A (en) | Data processing method and related device | |
CN110147724A (en) | For detecting text filed method, apparatus, equipment and medium in video | |
CN113627421A (en) | Image processing method, model training method and related equipment | |
CN111429414B (en) | Artificial intelligence-based focus image sample determination method and related device | |
CN109101992A (en) | Image matching method, device and computer readable storage medium | |
CN112257840A (en) | Neural network processing method and related equipment | |
CN115222047A (en) | Model training method, device, equipment and storage medium | |
CN115937993A (en) | Living body detection model training method, living body detection device and electronic equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |