CN110019652B - Cross-modal Hash retrieval method based on deep learning - Google Patents

Cross-modal Hash retrieval method based on deep learning Download PDF

Info

Publication number
CN110019652B
CN110019652B CN201910196009.7A CN201910196009A CN110019652B CN 110019652 B CN110019652 B CN 110019652B CN 201910196009 A CN201910196009 A CN 201910196009A CN 110019652 B CN110019652 B CN 110019652B
Authority
CN
China
Prior art keywords
theta
retrieval
text
modality
modal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910196009.7A
Other languages
Chinese (zh)
Other versions
CN110019652A (en
Inventor
董西伟
邓安远
周军
杨茂保
孙丽
胡芳
贾海英
王海霞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiujiang University
Original Assignee
Jiujiang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiujiang University filed Critical Jiujiang University
Priority to CN201910196009.7A priority Critical patent/CN110019652B/en
Publication of CN110019652A publication Critical patent/CN110019652A/en
Application granted granted Critical
Publication of CN110019652B publication Critical patent/CN110019652B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/31—Indexing; Data structures therefor; Storage structures
    • G06F16/316—Indexing structures
    • G06F16/325—Hash tables
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/51—Indexing; Data structures therefor; Storage structures

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Cross-modal Hash retrieval method based on deep learning, and hypothesis
Figure 910610DEST_PATH_IMAGE002
A set of pixel feature vectors of an image modality of an individual object of
Figure 624488DEST_PATH_IMAGE004
The method is characterized by comprising the following steps: (1) binary hash coding for sharing image mode and text mode by using target function designed based on deep learning technology
Figure 138646DEST_PATH_IMAGE006
Deep neural network parameters for image modalities and text modalities
Figure 330593DEST_PATH_IMAGE008
And
Figure 4020DEST_PATH_IMAGE010
and projection matrix of image modality and text modality
Figure 623220DEST_PATH_IMAGE012
And
Figure 624674DEST_PATH_IMAGE014
(ii) a (2) Solving for unknown variables in an objective function using an alternate update approach
Figure DEST_PATH_IMAGE015
、
Figure 685559DEST_PATH_IMAGE008
、
Figure 416754DEST_PATH_IMAGE010
、
Figure DEST_PATH_IMAGE016
And
Figure DEST_PATH_IMAGE017
(ii) a (3) Depth neural network parameters based on image modality and text modality obtained by solving
Figure 800331DEST_PATH_IMAGE008
And
Figure 351398DEST_PATH_IMAGE010
and a projection matrix
Figure 947465DEST_PATH_IMAGE016
And
Figure DEST_PATH_IMAGE018
(ii) a (4) Calculating the Hamming distance from the query sample to each sample in the retrieval sample set based on the generated binary Hash codes; (5) the retrieval of the query sample is accomplished using a cross-modal retriever based on approximate nearest neighbor searching. The method effectively improves the performance of cross-modal hash retrieval.

Description

Cross-modal Hash retrieval method based on deep learning
Technical Field
The invention relates to a cross-modal Hash retrieval method based on deep learning.
Background
With the rapid development of scientific and technical and social productivity, the big data era turns to something else. By big data is meant a collection of data that cannot be captured, managed and processed within a certain time frame using conventional software tools. IBM proposes that big data have 5V characteristics, namely: volume (large data Volume), Variety (diversified types and sources), Value (relatively low data Value density, but sometimes rare), Velocity (rapid data growth rate), and Veracity (quality of data). Big data can also be considered an information asset that requires new processing modes to have greater decision-making, insight discovery, and process optimization capabilities.
Information retrieval is an important aspect of data processing, and in the face of big data, how to effectively perform information retrieval becomes an urgent and very challenging problem in the big data era. For large-scale data retrieval, the hash retrieval method plays an important role. The Hash retrieval method maps the high-dimensional characteristics of the object to the Hamming space to generate a low-dimensional Hash code to represent the object, reduces the requirement of a retrieval system on the memory space of a computer, improves the retrieval speed and can better adapt to the requirement of mass retrieval. The main idea of the hash search is to project data represented by high-dimensional vectors into a Hamming space and search K neighbors (K is more than or equal to 1) in the Hamming space. In order to keep K neighbors in hamming space consistent with the original space, the hash learning algorithm needs to satisfy local retention characteristics, i.e., to maintain the similarity before and after data projection. The Local Sensitive Hashing (LSH) method can make two points with a very close distance in a high dimensional space, and after the two points are hash-coded by a hash function, the hash codes of the two points have a very high probability of being the same, otherwise, if the distance between the two points is far, the probability of the same hash code is very small.
The cross-modality hash retrieval is mainly used for solving the mutual retrieval problem among different modality data, for example, retrieving texts by using images, retrieving images by using texts, and the like. The cross-modal hash retrieval method needs to perform hash encoding on data in different modalities to generate compact binary hash codes, and then completes mutual retrieval between the data in different modalities based on the generated hash codes. Ding et al propose a Collective Matrix Factorisation Hashing (CMFH) method. The CMFH method may learn a uniform hash encoding from the different modalities of each instance using collective matrix decomposition. In order to effectively use category information and maintain a local geometric structure in a cross-modal Hashing method based on Matrix decomposition and further achieve the purpose of effectively improving the identification capability of potential semantic features obtained by Matrix decomposition, Tang et al propose a Supervised Matrix Factorization Hashing (SMFH) method. When the SMFH method is used for Hash coding learning, consistency of marking information among the modes is considered, and consistency of local geometric structures inside the modes is also considered. Aiming at the problem that the training time complexity of a supervised cross-modal hashing method is high, Zhang et al propose a supervised cross-modal hashing method called Semantic Correlation Maximization (SCM). The SCM approach can seamlessly integrate semantic labeling information into the hash learning process.
The manual features used by the cross-modal hashing algorithm based on the shallow learning structure may not achieve optimal compatibility with hash code learning. To address this problem, Jiang et al propose a Deep Cross-modal Hashing (DCMH) method. The DCMH method is an end-to-end cross-modal Hash method based on a deep learning framework, and can effectively integrate feature learning and Hash coding learning into a learning framework. To improve the Quantization capability (Quantization) of depth feature representation in an end-to-end learning architecture so that the depth feature representation can be quantized more efficiently, Cao et al propose a Collective Depth Quantization (CDQ) method by introducing Quantization into the end-to-end depth learning architecture for cross-modal retrieval. The CDQ method jointly learns the depth feature representation and quantizer for both modalities through a well-designed hybrid network and loss function. The hybrid network of the CDQ method comprises: an image network composed of a plurality of Convolution-Pooling (fusion-Pooling) layers for extracting image feature representations, a text network composed of a plurality of Fully-Connected (fused-Connected) layers for extracting text feature representations, two Fully-Connected Bottleneck (fused-Connected bottle) layers for generating optimal low-dimensional feature representations, an adaptive cross-entropy loss for capturing cross-modal correlations, and a collective quantization loss for controlling hash quality and quantization capability. In addition, the CDQ method can also learn a quantizer codebook common to the modalities, by which the association between the two modalities can be substantially enhanced. In order to effectively capture the essential relationship between different modalities in an end-to-end Deep learning architecture for cross-modality retrieval, Yang et al propose a pair relationship oriented Deep Hashing (PRDH) method. The PRDH method learns the Hash codes which can reflect the intrinsic relationship between the modes better by integrating different types of pair-wise constraints from the view point in the modes and the view point between the modes. In addition, the PRDH method enhances the discrimination ability of hash-coding each bit by introducing decorrelation constraints in the deep learning architecture.
Cross-modal Hash retrieval requires mapping high-dimensional feature data of an object in different modes to a low-dimensional Hamming space so as to realize that binary Hash coding based on the Hamming space can quickly and accurately complete a cross-modal information retrieval task. Most of the existing cross-modal hash retrieval methods are methods based on a shallow learning structure, although the methods can quickly complete retrieval tasks based on a hash retrieval technology, the shallow learning structure enables identification information in original features not to be mined well. The deep learning technology has excellent characteristic learning capability in tasks such as classification and object detection, and the existing cross-modal hash retrieval method based on the deep learning technology also shows that the deep learning technology is beneficial to improving the performance of the cross-modal retrieval task. Therefore, the cross-modal Hash retrieval method based on the deep learning technology is designed, and has important significance and value for completing the cross-modal retrieval task under the big data situation.
Disclosure of Invention
The invention aims to provide a cross-modal hash retrieval method based on deep learning, and solves the problem that the existing cross-modal hash retrieval method based on a shallow learning structure cannot well mine identification information in original features.
In order to achieve the purpose, the technical scheme is that a cross-modal Hash retrieval method based on deep learning is adopted, and a pixel feature vector set of image modalities of n objects is assumed as
Figure BDA0001994609040000041
Wherein v isiA pixel feature vector representing the ith object in the image modality; order to
Figure BDA0001994609040000042
A feature vector representing the n objects in a text modality, wherein tiA feature vector representing the ith object in a text modality; representing a class label vector of n objects as
Figure BDA0001994609040000043
Wherein c represents the number of object categories; for vector yiIn other words, if the ith object belongs to the kth class, let vector yiIs 1, otherwise, the vector yiThe kth element of (1) is 0; the method comprises the following steps:
(1) obtaining a binary Hash code B shared by an image mode and a text mode by using an objective function designed based on a deep learning technology, and obtaining a deep neural network parameter theta of the image mode and the text modevAnd thetatAnd projection matrix P for image modality and text modalityvAnd Pt；
(2) Solving unknown variables B and theta in objective function by using alternative updating modev、θt、PvAnd PtI.e. solving the following three sub-problems alternately: fixed B, PvAnd PtSolving for thetavAnd thetat(ii) a Fixed B, thetavAnd thetatSolving for PvAnd Pt(ii) a Fixed thetav、θt、PvAnd PtSolving for B;
(3) depth neural network parameter theta based on image mode and text mode obtained by solvingvAnd thetatAnd a projection matrix PvAnd PtGenerating binary hash codes for the query samples and the samples in the retrieval sample set;
(4) calculating the Hamming distance from the query sample to each sample in the retrieval sample set based on the generated binary Hash codes;
(5) the retrieval of the query sample is accomplished using a cross-modal retriever based on approximate nearest neighbor searching.
Wherein, the objective function designed based on the deep learning technique in the step (1) has the following form:
Figure BDA0001994609040000051
wherein, γ1And gamma2Is a non-negative balance factor, B ═ B1,b2,…,bn]T∈{-1,+1}n×k，
Figure BDA0001994609040000052
And
Figure BDA0001994609040000053
for projection of the matrix, θvAnd thetatIn order to be the parameters of the deep neural network,
Figure BDA0001994609040000054
and
Figure BDA0001994609040000055
depth features of n objects in image modality and text modality, respectively, and matrixThe vectors in the ith column of the matrix G and F are respectively F (v)i；θv) And g (t)i；θt)，
Figure BDA0001994609040000056
Is a Laplace matrix for keeping intra-modal consistency and inter-modal consistency, 1 is a column vector with all elements being 1, | · | | | survivalFFrobenius norm representing the matrix, tr (·) represents the trace of the matrix, (·)TRepresenting the transpose of the matrix.
Wherein the step (2) solves the unknown variables B and theta in the objective function by using an alternate updating modev、θt、PvAnd PtSpecifically, the following three sub-problems are solved alternately:
(1) fixed B, PvAnd PtSolving for thetavAnd thetat(ii) a When the binary hash code B is fixed, and the projection matrix PvAnd PtIn time, the objective function shown in equation (1) reduces to a parameter θ with respect to the deep neural networkvAnd thetatThe sub-problems of (a):
Figure BDA0001994609040000061
(2) fixed B, thetavAnd thetatSolving for PvAnd Pt(ii) a When the binary hash code B is fixed, and the deep neural network parameter thetavAnd thetatThe objective function shown in equation (1) is reduced to that about the projection matrix PvAnd PtThe sub-problems of (a) are:
Figure BDA0001994609040000062
(3) fixed thetav、θt、PvAnd PtSolving for B; when the depth of the neural network parameter theta is fixedvAnd thetatAnd a projection matrix PvAnd PtWhen, the objective function shown in equation (1) is reduced to that ofThe sub-problem of binary hash coding B, namely:
Figure BDA0001994609040000063
in solving the unknown variable B in equation (4), a discrete hash algorithm based on singular value decomposition is used for the solution.
Wherein, the depth neural network parameter theta based on the image modality and the text modality obtained by the solution in the step (3)vAnd thetatAnd a projection matrix PvAnd PtGenerating a binary hash code for the query sample and the samples in the search sample set, in particular, assuming that a feature vector of a query sample of the image modality is
Figure BDA0001994609040000064
The feature vector of a query sample of the text modality is
Figure BDA0001994609040000065
The image mode searches the characteristics of the samples in the sample set as
Figure BDA0001994609040000066
The text modal search sample set is characterized by
Figure BDA0001994609040000071
Wherein the content of the first and second substances,
Figure BDA0001994609040000072
representing the number of samples in the search sample set; the binary hash codes of the image mode query sample and the text mode query sample and the retrieval sample set are respectively as follows:
Figure BDA0001994609040000073
Figure BDA0001994609040000074
and
Figure BDA0001994609040000075
wherein the content of the first and second substances,
Figure BDA0001994609040000076
sign (·) is a sign function.
Wherein, in the step (4), the hamming distance from the query sample to each sample in the retrieval sample set is calculated based on the generated binary hash code, specifically, a formula is used
Figure BDA0001994609040000077
Computing a query sample of an image modality to a first in a set of text modality search samples
Figure BDA00019946090400000713
Figure BDA00019946090400000714
Hamming distance of individual samples; using the formula
Figure BDA0001994609040000079
Computing a query sample of a text modality to a first in a set of image modality search samples
Figure BDA00019946090400000710
Hamming distance of individual samples.
Wherein, the step (5) uses a cross-modal searcher based on approximate nearest neighbor search to complete the search of the query sample, specifically, the Hamming distance obtained by calculation
Figure BDA00019946090400000711
(or
Figure BDA00019946090400000715
) And sorting according to the sequence from small to large, and then taking the samples corresponding to the first K minimum distances in a text mode (or an image mode) retrieval sample set as retrieval results.
Advantageous effects
Compared with the prior art, the invention has the following advantages.
1. The method can utilize the deep learning structure to dig out more identification information under the condition of keeping the retrieval speed, thereby more accurately completing the cross-modal retrieval;
2. the method of the invention fully keeps the beneficial information of the original characteristic space to the Hamming space by implementing the intra-modal consistency and inter-modal consistency keeping strategies, thereby promoting the excavation and searching performance of the identification information;
3. the discrete hash algorithm based on singular value decomposition provided by the method can enable the obtained binary hash code to have more beneficial characteristics, and further effectively improves the performance of cross-modal hash retrieval.
Drawings
Fig. 1 is a flowchart of a cross-modal hash retrieval method based on deep learning according to the present invention.
Detailed Description
The technical solution of the present invention is further described in detail below with reference to the accompanying drawings.
The invention discloses a cross-modal Hash retrieval method based on deep learning, which mainly comprises the following steps in the specific implementation process as shown in figure 1: assume a set of pixel feature vectors for an image modality of n objects as
Figure BDA0001994609040000081
Wherein v isiA pixel feature vector representing the ith object in the image modality; order to
Figure BDA0001994609040000082
A feature vector representing the n objects in a text modality, wherein tiA feature vector representing the ith object in a text modality; representing a class label vector of n objects as
Figure BDA0001994609040000083
Wherein c represents the number of object categories; for vector yiIn other words, if the ith object belongs to the kth class, let vector yiIs 1, is the k-th element of (1),otherwise, vector yiThe kth element of (1) is 0;
(1) cross-modal Hash retrieval target function construction based on deep learning
The method aims to learn the hash functions of the image modality and the text modality by utilizing the characteristic data V and T of the image modality and the text modality and the class mark information of the object, and generate the binary hash code for completing the cross-modal hash retrieval task by utilizing the learned hash function. The characteristic data V and T of the image mode and the text mode are directly used for cross-mode hash learning, so that the method is not beneficial to mining identification information from the original characteristics to generate binary hash codes with excellent performance. In order to better mine identification information from original feature data of an image modality and a text modality, the method of the invention constructs a Deep Neural Network (DNN) for the image modality and the text modality data respectively to carry out Deep feature learning.
For an image mode, the method adopts a Convolutional Neural Network (CNN) which is obtained by AlexNet improvement and is composed of seven layers to carry out image mode depth feature learning. This CNN model is described in detail below.
This CNN model for image modality depth feature learning contains five convolutional layers (Convolution Layer) and two Fully Connected layers (full Connected Layer), denoted "Conv 1-Conv 5" and "Fc 6-Fc 7", respectively. The network takes as input the pixel characteristics of the image modality. In this CNN, the first convolution layer Conv1 is filtered in 4-pixel steps using 96 input images of 11 × 11 × 3 core size and 227 × 227 × 3 core size. After activation, MAX-pooling and Local Response Normalization (LRN) by a Linear correction Unit (Rectified Linear Unit, ReLU), an output signature of 27 × 27 × 96 is obtained. The second convolutional layer Conv2 takes as input the output of the first convolutional layer Conv1, and Conv2 filters with 256 collation inputs of size 5 × 5 × 96. Similarly, after passage through ReLU, MAX-firing and LRN, an output characteristic of size 13 × 13 × 256 is obtained. Third, fourth and fifth convolutional layers Conv3, Conv4 and Conv5 384, 384 and 256 convolution kernels of size 3 x 256, 3 x 384 and 3 x 384, respectively, are used and each layer is activated with ReLU. Output characteristics of 6X 256 were obtained when Conv5 was subjected to MAX-firing. The number of neurons of the fully connected layer Fc6 was 4096, and the neurons were temporarily discarded using a discard ratio of 0.5 to prevent overfitting. Fc7 layer containing d(v)The fully-connected layer of individual neurons, and the Hyperbolic Tangent (TanH) function is used as the activation function of the Fc7 layer. Finally, a size d was obtained at Fc7 layer(v)Output characteristic of x 1.
For text mode, the method uses a multi-layer Perceptron (MLP) composed of three fully-connected layers to construct an MLP deep neural network to map the features of the text mode from the original feature space to the semantic space. The three fully-connected layers in the constructed MLP deep neural network are denoted herein as Fc1, Fc2, and Fc3, respectively. Similar to the practice of constructing the MLP deep neural network in text modal feature learning of the related literature, the Fc1 layer and the Fc2 layer of the MLP deep neural network constructed by the invention use ReLU as a nonlinear activation function. The hyperbolic tangent (TanH) function is used as the activation function for the Fc3 layer. The number of neurons in Fc3 layer is d(t)Namely: the dimension of the output feature of the MLP deep neural network for learning the depth features of the text mode is d(t)。
For the ith object, order
Figure BDA0001994609040000101
Output characteristics of CNN representing image modality, where θvA parameter of CNN that is an image modality; order to
Figure BDA0001994609040000102
An output characteristic of the MLP deep neural network representing a text modality, whereintParameters of the MLP deep neural network that are text modalities.
Suppose that the ith object has a deep learning feature f (v) in the image modality and the text modalityi；θv) And g (t)i；θt) By linear projectionShadow matrix
Figure BDA0001994609040000103
And
Figure BDA0001994609040000104
the characteristics after projection are respectively
Figure BDA0001994609040000105
And Pt Tg(ti；θt) Wherein, whereinTRepresenting the transpose of the matrix. Further suppose that
Figure BDA0001994609040000106
And Pt Tg(ti；θt) Binary hash codes in hamming space can be generated separately
Figure BDA0001994609040000111
And
Figure BDA0001994609040000112
then, cross-modality hash learning can be performed by the following minimization problem:
Figure BDA0001994609040000113
wherein, γ1And gamma2For non-negative balance factors, the third term to the right of the equal sign is a regular term to prevent overfitting, and the fourth term to the right of the equal sign is to expect that the probability of each bit of the hash code being +1 and-1 is equal and to maximize the information provided by each bit of the hash code.
Intra-modality similarity reflects a close-proximity relationship between data points composed of feature vectors within each modality. Two data points v of an image modalityiAnd vjThe intra-modal similarity between can be defined as:
Figure BDA0001994609040000114
wherein the content of the first and second substances,
Figure BDA0001994609040000115
representing a data point viK of (a)1Neighbor set (k)1-nearest neighbors)，
Figure BDA0001994609040000116
Denotes viAnd vjThe euclidean distance between, i.e.:
Figure BDA0001994609040000117
representing vector l2And (4) norm. Sigma for control
Figure BDA0001994609040000118
The decay rate of (c). Similarly, two data points t composed of two feature vectors in the text modalityiAnd tjIntra-modal similarity of
Figure BDA0001994609040000119
Is defined as:
Figure BDA00019946090400001110
wherein the content of the first and second substances,
Figure BDA00019946090400001111
for each modality, in order to keep the local neighbor structure of the data points consistent in hamming space and original feature space, namely: each data point and its neighbor relation in the original feature space are maintained in hamming space, and the following objective function can be designed:
Figure BDA0001994609040000121
based on the class label information of the object, a data point v of the image modality can be definedi(i＝1,2,…N) data point t of text modej(j ═ 1,2, …, n) of the semantic correlation matrix shown below:
Figure BDA0001994609040000122
it should be noted that: as long as viAnd tjThey are considered to have the same semantics if they belong to at least one of the same category. In order to maintain inter-modal consistency between image and text modalities in hamming space, the following objective function can be designed:
Figure BDA0001994609040000123
combining the above analysis on image modal depth feature learning, text modal depth feature learning, intra-modal consistency and inter-modal consistency preservation, the objective function of the method of the invention can be designed as:
Figure BDA0001994609040000124
according to prior work, if data in different modality spaces have the same semantics, the data in these different modalities often correspond to a common underlying space. Thus, the present invention assumes that features with the same semantics in both the image modality and the text modality can ultimately be represented as the same binary hash encoding in a common hamming space. That is to say that there are
Figure BDA0001994609040000125
This is true. Based on this assumption, the optimization problem in equation (7) can be expressed as:
Figure BDA0001994609040000131
wherein the content of the first and second substances,
Figure BDA0001994609040000132
Figure BDA0001994609040000133
Figure BDA0001994609040000134
by means of a simple derivation,
Figure BDA0001994609040000135
can be rewritten as follows:
Figure BDA0001994609040000136
wherein, B ═ B1,b2,…,bn]T∈{-1,+1}n×k，
Figure BDA0001994609040000137
And the vectors of the ith columns of the matrix F and the matrix G are respectively F (v)i；θv) And g (t)i；θt)，||·||FThe Frobenius norm of the matrix is represented. To pair
Figure BDA0001994609040000138
Its equivalent can be derived by following the derivation:
Figure BDA0001994609040000139
wherein the content of the first and second substances,
Figure BDA00019946090400001310
L-D-W is a laplacian matrix,
Figure BDA00019946090400001311
Figure BDA00019946090400001312
denotes the ith diagonal element, w, of the diagonal matrix DijFor an element in row ith and column jth of matrix W, tr (-) represents a trace of the matrix. From equations (12) and (13), equation (8) can be rewritten as:
Figure BDA00019946090400001313
(2) solving of an objective function
The objective function shown in equation (14) includes five unknown variables to be solved, namely: binary Hash code matrix B, Linear projection matrix PvAnd PtDeep neural network parameter θvAnd thetat. The objective function shown in equation (14) is non-convex for these five joint unknown variables, and therefore, an analytical solution for these five unknown variables cannot be obtained simultaneously. The unknown variables in equation (14) can be solved by solving three sub-problems alternately, namely: fixed B, PvAnd PtSolving for thetavAnd thetat(ii) a Fixed B, thetavAnd thetatSolving for PvAnd Pt(ii) a Fixed thetav、θt、PvAnd PtAnd B is solved.
(a) Fixed B, PvAnd PtSolving for thetavAnd thetat
When the binary hash code B is fixed, and the projection matrix PvAnd PtIn time, the objective function shown in equation (14) reduces to θ with respect to the deep neural network parametervAnd thetatThe sub-problems of (a):
Figure BDA0001994609040000141
the invention uses a Back Propagation (BP) algorithm to learn and update DNN network parameters thetav. Similar to most existing deep learning methods, a random gradient descent algorithm based on back propagation is used here to learn θv. Learning thetavThe specific method comprises the following steps: selecting a small batch of training samples from the training samples at each iteration, and then learning theta by using a random gradient descent algorithm based on backward propagation by using the selected samplesv. Each feature vector v of the image modality for a selected training sampleiFirst, the gradient is calculated using the following formula:
Figure BDA0001994609040000142
then, using the chain rule and the obtained
Figure BDA0001994609040000143
Calculating out
Figure BDA0001994609040000144
Finally, using the calculation
Figure BDA0001994609040000145
Updating DNN network parameter theta of image modality by BP algorithmv。
Algorithm 1 shows the solution of the network parameter theta of the image modality DNNvThe algorithm of (1).
Figure BDA0001994609040000151
Similarly, learning the deep neural network parameter θ of the updated text modality using a back-propagation-based stochastic gradient descent algorithmt. Each feature vector t of the text modality for a selected training sampleiFirst, the following gradient is calculated:
Figure BDA0001994609040000152
then, using a chain gaugeThen the obtained gradient is summed
Figure BDA0001994609040000153
Computing
Figure BDA0001994609040000154
Finally, using the calculated
Figure BDA0001994609040000155
Updating DNN network parameter theta of text mode by BP algorithmt. Using an algorithm similar to algorithm 1, the DNN network parameter θ of the text mode can be learnedt。
(b) Fixed B, thetavAnd thetatSolving for PvAnd Pt
When the binary hash code B is fixed, and the deep neural network parameter thetavAnd thetatThe objective function shown in equation (14) reduces to that about the projection matrix PvAnd PtThe sub-problems of (a):
Figure BDA0001994609040000161
for in equation (18)
Figure BDA0001994609040000162
Respectively about PvAnd PtTaking the partial derivative and making the partial derivative equal to 0, one can obtain:
Figure BDA0001994609040000163
Figure BDA0001994609040000164
by simple derivation:
Pv＝(FFT+I+F11TFT)-1FB， (21)
Pt＝(GGT+I+G11TGT)-1GB， (22)
wherein, I is a unit matrix, (·)-1Representing the inverse of the matrix.
(c) Fixed thetav、θt、PvAnd PtSolving for B
When the neural network parameter theta is fixed in depthvAnd thetatAnd a projection matrix PvAnd PtIn time, the objective function shown in equation (14) reduces to a sub-problem with binary hash encoding B, namely:
Figure BDA0001994609040000165
a simple derivation of equation (23) can yield:
Figure BDA0001994609040000166
because of Pv、Pt、θvAnd thetatIs fixed, and therefore,
Figure BDA0001994609040000167
and
Figure BDA0001994609040000168
are all constants. Further, ignoring these two terms in equation (24) does not affect the solution of B. Furthermore, since B ∈ { -1, +1}n×kCan obtain
Figure BDA0001994609040000169
That is to say, the number of the first and second,
Figure BDA00019946090400001610
is a constant. After discarding the constant terms in equation (24), equation (24) is transformed into:
Figure BDA0001994609040000171
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0001994609040000172
since the unknown variable in the formula (25) is a discrete variable, it is generally difficult to directly solve the discrete variable to obtain an analytical solution. The invention proposes a discrete hash algorithm based on singular value decomposition to solve the optimization problem on the discrete variable B shown in the formula (25). The discrete hash algorithm based on singular value decomposition is described in detail below.
Singular value decomposition of the matrix L can be obtained
Figure BDA0001994609040000173
Wherein the content of the first and second substances,
Figure BDA0001994609040000174
Figure BDA0001994609040000175
is a diagonal matrix. Will be provided with
Figure BDA0001994609040000176
Substituting equation (25) yields:
Figure BDA0001994609040000177
order to
Figure BDA0001994609040000178
And
Figure BDA0001994609040000179
respectively represent a matrix B,
Figure BDA00019946090400001710
And
Figure BDA00019946090400001711
row i of (1); order to
Figure BDA00019946090400001712
And
Figure BDA00019946090400001713
respectively represent a matrix B,
Figure BDA00019946090400001714
And
Figure BDA00019946090400001715
is at removing
Figure BDA00019946090400001716
And
Figure BDA00019946090400001717
the remaining rows then form the matrix. At this time, it is possible to obtain:
Figure BDA00019946090400001718
similarly, one can obtain:
Figure BDA00019946090400001719
wherein the content of the first and second substances,
Figure BDA00019946090400001720
the ith column of the matrix Q is represented,
Figure BDA00019946090400001721
the representation matrix Q is removing QiThe remaining columns form a matrix.
According to equation (27) and equation (28), the unknown binary hash-coding matrix B in equation (26) can be obtained by solving the following for BiThe optimization problem of (i ═ 1,2, …, n) results, namely:
Figure BDA00019946090400001722
by simple derivation, equation (29) can be converted to:
Figure BDA0001994609040000181
the optimization problem in equation (30) has the following analytical solution:
Figure BDA0001994609040000182
where sign (·) represents a sign function.
Algorithm 2 shows a discrete hash algorithm based on singular value decomposition.
Figure BDA0001994609040000183
(3) Generating a binary hash of a sample in a set of query and search samples
Assume that a query sample of an image modality has a feature vector of
Figure BDA0001994609040000184
The feature vector of a query sample of the text modality is
Figure BDA0001994609040000185
The image mode searches the characteristics of the samples in the sample set as
Figure BDA0001994609040000186
The text modal search sample set is characterized by
Figure BDA0001994609040000191
Wherein the content of the first and second substances,
Figure BDA0001994609040000192
representing the number of samples in the search sample set. Projection matrix P of image mode and text mode obtained by solvingvAnd PtAnd the deep neural network parameter theta for image modality and text modalityvAnd thetatThe binary hash codes of the query sample and the retrieval sample set in the image mode and the text mode can be obtained as follows:
Figure BDA0001994609040000193
Figure BDA0001994609040000194
and
Figure BDA0001994609040000195
wherein the content of the first and second substances,
Figure BDA0001994609040000196
sign (·) is a sign function.
(4) Calculating the Hamming distance from the query sample to each sample in the search sample set
Query sample for image modalities
Figure BDA0001994609040000197
Using the formula
Figure BDA0001994609040000198
Computing query samples for image modalities
Figure BDA0001994609040000199
Retrieval of sample set samples to text modality
Figure BDA00019946090400001910
Figure BDA00019946090400001911
Hamming distance of. Query sample for text modalities
Figure BDA00019946090400001912
Using the formula
Figure BDA00019946090400001913
Computing query samples for text modalities
Figure BDA00019946090400001914
Retrieval of samples in a sample set to an image modality
Figure BDA00019946090400001915
Hamming distance of.
(5) Completing retrieval of query samples using a cross-modality retriever
For the search task of searching text for image, firstly, the calculated search task is carried out
Figure BDA00019946090400001916
Individual hamming distance
Figure BDA00019946090400001917
And sequencing according to the sequence from small to large, and then taking samples corresponding to the first K minimum distances in the text retrieval sample set as retrieval results. Similarly, for a search task of searching for an image for a text, first, the calculated results are compared
Figure BDA00019946090400001918
Individual hamming distance
Figure BDA00019946090400001919
And sorting according to the sequence from small to large, and then taking samples corresponding to the first K minimum distances in the image retrieval sample set as retrieval results.
The following describes the advantageous effects of the present invention with reference to specific experiments.
The relevant experiments carried out for the method of the invention were mainly carried out on the Pascal VOC 2007 dataset, which is first briefly described. The Pascal VOC 2007 dataset contains 9963 images belonging to 20 categories (e.g., airplane, bottle, horse, sofa, etc.), and each image is labeled. In the experiment, the inventive method partitioned the data set into a training set containing 5011 image-tag pairs and a test set containing 4952 image-tag pairs. For the depth cross-modality hash method, the image modality uses the original pixel features as input features. For the method with manual features as input, 512-dimensional GIST features are used as input features. For the text modality, 399-dimensional word frequency features are used as input features. Two main cross-modal retrieval tasks are performed in the experiment, namely: the text retrieval with the image and the image retrieval with the text are represented by Img2Txt and Txt2Img, respectively.
The performance of the cross-modal hash retrieval method is measured by using an Average Precision Average (MAP). To obtain the MAP, it is necessary to first calculate an Average Precision (AP) for each query sample. And after the average precision APs of all the query samples are obtained, averaging all the average precision APs to obtain an average precision mean MAP.
The method of the present invention uses a small Batch gradient descent algorithm with Momentum (Momentum) and Weight Decay (Weight Decay) of 0.9 and 0.0001, respectively, and the size of the Batch (Batch) is set to 128. The first five layers of the image modality deep neural network in the method of the present invention were initialized using AlexNet pre-trained on the ImageNet dataset. Other parameters of the deep neural network in the method are initialized in a random initialization mode. The output feature dimensions of the deep neural networks of the image modality and the text modality are set to 1024. In the experiment, 5-fold cross validation is adopted to determine the parameter gamma in the method of the invention1And gamma2The optimum value of (2). And for parameters in other methods, parameter setting is carried out according to a parameter setting principle recommended by each method, and the result reported by the experiment is the average value of the results of 10 random experiments.
The method for comparing with the method of the invention respectively comprises the following steps: semantic Correlation Maximization (SCM), Supervised Matrix Factorization Hashing (SMFH) method, Deep Cross-Modal Hashing (DCMH) method, and Pairwise Relationship-oriented Deep Hashing (PRDH) method. Table 1 lists the average precision mean MAP for the cross-modal hash search on the Pascal VOC 2007 data set by the present method and comparative method. As can be seen from table 1, for two search tasks, under three hash coding lengths, the deep cross-modal hash search methods DCMH, PRDH, and the method of the present invention all can achieve better search performance than the shallow cross-modal hash search methods SCM and SMFH. This illustrates that it is beneficial to use a deep learning technique to learn the depth features used to generate the binary hash code. From table 1, it can also be seen that for the Img2Txt and Txt2Img search tasks, the cross-modal search performance of the method of the present invention is superior to that of the DCMH and PRDH methods under three hash coding lengths. This shows that the method of the present invention is an effective cross-modal hash retrieval method.
TABLE 1 MAP of methods on Pascal VOC 2007 dataset
Figure BDA0001994609040000211
Figure BDA0001994609040000221

Claims (5)

1. A cross-modal Hash retrieval method based on deep learning assumes a pixel feature vector set of image modalities of n objects as
Figure FDA0003574209630000011
Wherein v isiA pixel feature vector representing the ith object in the image modality; order to
Figure FDA0003574209630000012
A feature vector representing the n objects in a text modality, wherein tiA feature vector representing the ith object in a text modality; representing a class label vector of n objects as
Figure FDA0003574209630000013
Wherein c represents the number of object categories; for vector yiIn other words, if the ith object belongs to the kth class, let vector yiIs 1, otherwise, the vector yiThe kth element of (1) is 0; the method is characterized by comprising the following steps:
(1) obtaining a binary hash code B shared by an image mode and a text mode by using a target function designed on the basis of a deep learning technology, and obtaining a deep neural network parameter theta of the image mode and the text modevAnd thetatAnd projection matrix P for image modality and text modalityvAnd Pt；
(2) Solving the unknown variables B and theta in the objective function by using an alternative solving modev、θt、PvAnd PtI.e. solving the following three sub-problems alternately: fixed B, PvAnd PtSolving for thetavAnd thetat(ii) a Fixed B, thetavAnd thetatSolving for PvAnd Pt(ii) a Fixed thetav、θt、PvAnd PtSolving B;
(3) depth neural network parameter theta based on image mode and text mode obtained by solvingvAnd thetatAnd a projection matrix PvAnd PtGenerating binary hash codes for the query samples and the samples in the retrieval sample set;
(4) calculating the Hamming distance from the query sample to each sample in the retrieval sample set based on the generated binary Hash codes;
(5) using a cross-modal retriever based on approximate nearest neighbor search to complete the retrieval of the query sample;
the target function designed based on the deep learning technology in the step (1) is in the form as follows:
Figure FDA0003574209630000021
wherein, γ1And gamma2Is a non-negative balance factor, B ═ B1,b2,…,bn]T∈{-1,+1}n×k，
Figure FDA0003574209630000022
And
Figure FDA0003574209630000023
for projection of the matrix, θvAnd thetatIn order to be the parameters of the deep neural network,
Figure FDA0003574209630000024
and
Figure FDA0003574209630000025
depth features of n objects in an image modality and a text modality, respectively, and vectors of ith columns of the matrix F and the matrix G are F (v) respectivelyi；θv) And g (t)i；θt)，
Figure FDA0003574209630000026
Is a Laplace matrix for keeping intra-modal consistency and inter-modal consistency, 1 is a column vector with all elements being 1, | · | | | survivalFFrobenius norm representing the matrix, tr (·) represents the trace of the matrix, (·)TRepresenting the transpose of the matrix.
2. The deep learning-based cross-modal hash retrieval method according to claim 1, wherein the step (2) uses an alternative solution to solve the unknown variables B and θ in the objective functionv、θt、PvAnd PtSpecifically, the following three sub-problems are solved alternately:
(1) fixed B, PvAnd PtSolving for thetavAnd thetat(ii) a When the binary hash code B is fixed, and the projection matrix PvAnd PtIn time, the objective function shown in equation (1) reduces to a parameter θ with respect to the deep neural networkvAnd thetatThe sub-problems of (a):
Figure FDA0003574209630000027
(2) fixed B, thetavAnd thetatSolving for PvAnd Pt(ii) a When the binary hash code B is fixed, and the deep neural network parameter thetavAnd thetatThe objective function shown in equation (1) is reduced to that about the projection matrix PvAnd PtThe sub-problems of (a):
Figure FDA0003574209630000028
(3) fixed thetav、θt、PvAnd PtSolving B; when the depth of the neural network parameter theta is fixedvAnd thetatAnd a projection matrix PvAnd PtIn time, the objective function shown in equation (1) reduces to a sub-problem with binary hash coding B, namely:
Figure FDA0003574209630000031
when solving the unknown variable B in the formula (4), a discrete hash algorithm based on singular value decomposition is used for the solution.
3. The deep learning-based cross-modal hash retrieval method according to claim 1, wherein the depth neural network parameter θ based on the solved image modality and text modality in the step (3)vAnd thetatAnd a projection matrix PvAnd PtGenerating a binary hash code for the query sample and the samples in the search sample set, in particular, assuming that a feature vector of a query sample of the image modality is
Figure FDA0003574209630000032
Text modelA query sample of states has a feature vector of
Figure FDA0003574209630000033
The image mode searches the characteristics of the samples in the sample set as
Figure FDA0003574209630000034
The text modal search sample set is characterized by
Figure FDA0003574209630000035
Wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0003574209630000036
representing the number of samples in the search sample set; the binary hash codes of the image mode query sample and the text mode query sample and the retrieval sample set are respectively as follows:
Figure FDA0003574209630000037
Figure FDA0003574209630000038
and
Figure FDA0003574209630000039
wherein the content of the first and second substances,
Figure FDA00035742096300000310
sign (·) is a sign function.
4. The deep learning-based cross-modal hash retrieval method of claim 1, wherein the step (4) of calculating the hamming distance from the query sample to each sample in the retrieved sample set based on the generated binary hash code, specifically using a formula
Figure FDA00035742096300000311
Computing ith from image mode query sample to text mode retrieval sample set
Figure FDA00035742096300000312
Hamming distance of individual samples; using the formula
Figure FDA00035742096300000313
Computing the ith from the query sample of text mode to the image mode retrieval sample set
Figure FDA0003574209630000041
Hamming distance of individual samples.
5. The deep learning-based cross-modal hash retrieval method according to claim 1, wherein the step (5) uses a cross-modal retriever based on approximate nearest neighbor search to complete the retrieval of the query sample, specifically, the computed hamming distance
Figure FDA0003574209630000042
Or alternatively
Figure FDA0003574209630000043
And sequencing according to the sequence from small to large, and then taking samples corresponding to the first K minimum distances in a text mode or image mode retrieval sample set as retrieval results.
CN201910196009.7A 2019-03-14 2019-03-14 Cross-modal Hash retrieval method based on deep learning Active CN110019652B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910196009.7A CN110019652B (en) 2019-03-14 2019-03-14 Cross-modal Hash retrieval method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910196009.7A CN110019652B (en) 2019-03-14 2019-03-14 Cross-modal Hash retrieval method based on deep learning

Publications (2)

Publication Number Publication Date
CN110019652A CN110019652A (en) 2019-07-16
CN110019652B true CN110019652B (en) 2022-06-03

Family

ID=67189652

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910196009.7A Active CN110019652B (en) 2019-03-14 2019-03-14 Cross-modal Hash retrieval method based on deep learning

Country Status (1)

Country Link
CN (1) CN110019652B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110674323B (en) * 2019-09-02 2020-06-30 山东师范大学 Unsupervised cross-modal Hash retrieval method and system based on virtual label regression
US11651037B2 (en) * 2019-12-20 2023-05-16 Rakuten Group, Inc. Efficient cross-modal retrieval via deep binary hashing and quantization
US11899765B2 (en) 2019-12-23 2024-02-13 Dts Inc. Dual-factor identification system and method with adaptive enrollment
CN111628866B (en) * 2020-05-22 2021-08-31 深圳前海微众银行股份有限公司 Neural network verification method, device and equipment and readable storage medium
CN111639197B (en) * 2020-05-28 2021-03-12 山东大学 Cross-modal multimedia data retrieval method and system with label embedded online hash
CN112199375B (en) * 2020-09-30 2024-03-01 三维通信股份有限公司 Cross-modal data processing method and device, storage medium and electronic device
CN113407661B (en) * 2021-08-18 2021-11-26 鲁东大学 Discrete hash retrieval method based on robust matrix decomposition

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105184303A (en) * 2015-04-23 2015-12-23 南京邮电大学 Image marking method based on multi-mode deep learning
CN107885764A (en) * 2017-09-21 2018-04-06 银江股份有限公司 Based on the quick Hash vehicle retrieval method of multitask deep learning
CN108170755A (en) * 2017-12-22 2018-06-15 西安电子科技大学 Cross-module state Hash search method based on triple depth network
CN108334574A (en) * 2018-01-23 2018-07-27 南京邮电大学 A kind of cross-module state search method decomposed based on Harmonious Matrix
CN109271486A (en) * 2018-09-19 2019-01-25 九江学院 A kind of similitude reservation cross-module state Hash search method
CN109299342A (en) * 2018-11-30 2019-02-01 武汉大学 A kind of cross-module state search method based on circulation production confrontation network
CN109446347A (en) * 2018-10-29 2019-03-08 山东师范大学 A kind of multi-modal Hash search method of fast discrete and system having supervision

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6188777B1 (en) * 1997-08-01 2001-02-13 Interval Research Corporation Method and apparatus for personnel detection and tracking
US10146991B2 (en) * 2015-06-11 2018-12-04 Duke University Systems and methods for large scale face identification and verification
CN107402993B (en) * 2017-07-17 2018-09-11 山东师范大学 The cross-module state search method for maximizing Hash is associated with based on identification
CN108108657B (en) * 2017-11-16 2020-10-30 浙江工业大学 Method for correcting locality sensitive Hash vehicle retrieval based on multitask deep learning

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105184303A (en) * 2015-04-23 2015-12-23 南京邮电大学 Image marking method based on multi-mode deep learning
CN107885764A (en) * 2017-09-21 2018-04-06 银江股份有限公司 Based on the quick Hash vehicle retrieval method of multitask deep learning
CN108170755A (en) * 2017-12-22 2018-06-15 西安电子科技大学 Cross-module state Hash search method based on triple depth network
CN108334574A (en) * 2018-01-23 2018-07-27 南京邮电大学 A kind of cross-module state search method decomposed based on Harmonious Matrix
CN109271486A (en) * 2018-09-19 2019-01-25 九江学院 A kind of similitude reservation cross-module state Hash search method
CN109446347A (en) * 2018-10-29 2019-03-08 山东师范大学 A kind of multi-modal Hash search method of fast discrete and system having supervision
CN109299342A (en) * 2018-11-30 2019-02-01 武汉大学 A kind of cross-module state search method based on circulation production confrontation network

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Continuum regression for cross-modal multimedia retrieval;Yongming Chen 等;《2012 19th IEEE International Conference on Image Processing》;20130221;1949-1952 *
Cross-Modal Subspace Learning via Pairwise Constraints;Ran He 等;《IEEE Transactions on Image Processing》;20150807;第24卷;5543-5556 *
基于局部流形重构的半监督多视图图像分类;董西伟;《计算机工程与应用》;20160930;第52卷(第18期);24-30 *
基于深度哈希算法的图像—文本跨模态检索研究;姚伟娜;《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》;20190115(第01期);I138-5069 *
跨模态检索研究综述;欧卫华 等;《贵州师范大学学报(自然科学版)》;20180331;第36卷(第02期);114-120 *

Also Published As

Publication number Publication date
CN110019652A (en) 2019-07-16

Similar Documents

Publication Publication Date Title
CN110019652B (en) Cross-modal Hash retrieval method based on deep learning
Jiang et al. Asymmetric deep supervised hashing
Chen et al. Deep image retrieval: A survey
Shabbir et al. Satellite and scene image classification based on transfer learning and fine tuning of ResNet50
Xia et al. Supervised hashing for image retrieval via image representation learning
CN111783831B (en) Complex image accurate classification method based on multi-source multi-label shared subspace learning
An et al. Fast and incremental loop closure detection with deep features and proximity graphs
Yan et al. Semi-Supervised Deep Hashing with a Bipartite Graph.
CN106055573B (en) Shoe print image retrieval method and system under multi-instance learning framework
CN109271486B (en) Similarity-preserving cross-modal Hash retrieval method
CN110929080B (en) Optical remote sensing image retrieval method based on attention and generation countermeasure network
CN110647907B (en) Multi-label image classification algorithm using multi-layer classification and dictionary learning
CN111080551B (en) Multi-label image complement method based on depth convolution feature and semantic neighbor
CN112132186A (en) Multi-label classification method with partial deletion and unknown class labels
CN109960732B (en) Deep discrete hash cross-modal retrieval method and system based on robust supervision
CN112597324A (en) Image hash index construction method, system and equipment based on correlation filtering
CN111008224A (en) Time sequence classification and retrieval method based on deep multitask representation learning
CN112163114B (en) Image retrieval method based on feature fusion
Menaga et al. Deep learning: a recent computing platform for multimedia information retrieval
CN115795065A (en) Multimedia data cross-modal retrieval method and system based on weighted hash code
CN114780767A (en) Large-scale image retrieval method and system based on deep convolutional neural network
CN114329031A (en) Fine-grained bird image retrieval method based on graph neural network and deep hash
CN116383422B (en) Non-supervision cross-modal hash retrieval method based on anchor points
Bibi et al. Deep features optimization based on a transfer learning, genetic algorithm, and extreme learning machine for robust content-based image retrieval
CN116108217A (en) Fee evasion vehicle similar picture retrieval method based on depth hash coding and multitask prediction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant