CN110378366B - Cross-domain image classification method based on coupling knowledge migration - Google Patents
Cross-domain image classification method based on coupling knowledge migration Download PDFInfo
- Publication number
- CN110378366B CN110378366B CN201910482559.5A CN201910482559A CN110378366B CN 110378366 B CN110378366 B CN 110378366B CN 201910482559 A CN201910482559 A CN 201910482559A CN 110378366 B CN110378366 B CN 110378366B
- Authority
- CN
- China
- Prior art keywords
- data
- target domain
- domain data
- domain
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a cross-domain image classification method based on coupling knowledge migration, which is characterized in that a common low-dimensional subspace of a source domain and a target domain is searched based on a maximum mean difference criterion, and the difference between the edge distribution and the class condition distribution of data of the source domain and the target domain is eliminated; constructing respective adjacent maps according to label information of source domain data and pseudo label information of target domain data, keeping structural consistency of the data from an original space to a low-dimensional subspace, and simultaneously dynamically adjusting the structure of the adjacent maps to promote the forward migration of domain knowledge; training a nearest neighbor classifier by using source domain data with label information in a low-dimensional subspace, and continuously iterating and optimizing pseudo label information of target domain data to obtain final label information of the target domain data, namely finishing cross-domain image classification; in addition, the method of the invention endows different confidence degrees of the pseudo label of the target domain image by designing a sample reweighting strategy, effectively reduces the negative transfer of the knowledge in the domain, and improves the precision of cross-domain image classification.
Description
Technical Field
The invention relates to the technical field of computer vision image classification, in particular to a cross-domain image classification method based on coupling knowledge migration.
Background
Conventional machine learning algorithms typically require a large number of labeled data samples and require that the training samples and the test samples follow separate and identical distributions. A straightforward approach is to migrate an existing tagged data set onto a new unknown data set, i.e. from the source domain to the target domain. However, the distribution must be different between different data sets, and the assumption that "independent equal distribution is obeyed" is often not true. Therefore, the conventional image classification method has difficulty in obtaining good classification performance.
In order to overcome the difference of sample distribution between a source domain and a target domain and improve the generalization capability of a classification algorithm, currently, domain adaptation is mainly used as a means for processing the problem of knowledge migration in cross-domain image classification, and the method can be generally divided into a method based on feature transformation, a method based on an example and a method based on neural network migration. However, there are many problems to be solved and improved in the current cross-domain learning, for example, in the case of a large difference between the source domain and the target domain, the classification performance of the domain adaptation method based on feature transformation is far from reaching the original expectation of people; the method based on the example needs to calculate additional sample weight, has high calculation complexity, and cannot carry out rapid cross-field learning on a large data set; the neural network-based approach requires a large number of labeled source domain samples for long training and has a large limitation in practical applications.
Disclosure of Invention
The invention provides a cross-domain image classification method based on coupling knowledge migration, which aims to solve the problems of long calculation time, insufficient classification accuracy and the like of the existing cross-domain image classification method.
In order to realize the purpose of the invention, the technical means adopted is as follows:
a cross-domain image classification method based on coupling knowledge migration comprises the following steps:
s1, obtaining a source domain image and a target domain image, extracting characteristics, respectively obtaining a source domain characteristic matrix as source domain data, and obtaining a target domain characteristic matrix as target domain data;
s2, respectively constructing an adjacency graph of source domain data and an adjacency graph of target domain data according to label information of the source domain data, and respectively obtaining a weight matrix of the source domain and a weight matrix of the target domain by calculating edge weights of the adjacency graphs;
s3, performing inter-domain data distribution alignment in a common low-dimensional subspace of the source domain and the target domain, and performing knowledge migration of intra-domain data by using a weight matrix of the source domain and a weight matrix of the target domain;
s4, in the low-dimensional subspace, utilizing source domain data with label information to train a nearest neighbor classifier and predicting the category information of target domain data;
s5, reweighing the label information of the predicted target domain data;
and S6, judging whether the maximum iteration number is reached, if not, returning to the step S2, and if so, outputting the label information of the predicted target domain image.
In the scheme, the common low-dimensional subspace of the source domain and the target source is searched, the difference of sample distribution is reduced, meanwhile, the adjacent map is used for carrying out the in-domain knowledge positive transfer, the structural consistency from the original space to the common low-dimensional subspace is kept, the data distribution is aligned by eliminating the difference of the edge distribution and class condition distribution of the data of the source domain and the target domain, therefore, the source domain data with label information is used for training a nearest neighbor classifier in the low-dimensional subspace to predict the label information of the data of the target domain, and finally, the predicted label information of the data of the target domain is subjected to weighting to inhibit the knowledge negative transfer, so that the accurate classification of the cross-domain image is realized.
Preferably, the specific steps of step S1 are:
s1.1. Input n s A source domain image with label and n t A label-free target domain image; wherein n is s 、n t Is a positive integer;
s1.2, respectively extracting the features of the source domain image and the target domain image, and arranging the feature vectors obtained by extracting the source domain image in columns to obtain a source domain data feature matrixWhere m is the dimension of the feature vector, n s The number of the feature vectors;
arranging the characteristic vectors extracted from the target domain image according to columns to obtain a target domain data characteristic matrixWhere m is the dimension of the feature vector, n t The number of the feature vectors;
the feature vector of the source domain image represents source domain data of the source domain image, and the feature vector of the target domain image represents target domain data of the target domain image.
Preferably, the feature extraction in step S1 is to extract feature vectors of the source domain image and the target domain image respectively through a pre-trained neural network. In the preferred embodiment, the neural network may use VGG16, *** lenet, resNet50, or the like.
Preferably, the feature extraction in step S1 is specifically to arrange the gray values of the source domain image into feature vectors and the gray values of the target domain image into feature vectors after graying the source domain image and the target domain image respectively.
Preferably, the specific step S2 is:
s2.1, judging whether the iteration is the first iteration, if not, directly carrying out the step S2.2, if so, constructing a nearest neighbor classifier and training the nearest neighbor classifier by utilizing the source domain data for predicting pseudo label information of the target domain data
S2.2, constructing an adjacency graph of the source domain data according to the label information of the source domain data, namely establishing adjacent edges among the same type data in the source domain data, and calculating the weights of the adjacent edges to obtain a weight matrix
Wherein W s A weight matrix for the source domain, calculated by:
whereinIs the feature vector of the ith source domain image,the label information corresponding to the ith source domain image is represented by the bandwidth of a Gaussian kernel, and | · | | | represents a vector two-norm;
constructing an adjacency graph of the target domain data according to the pseudo label information of the target domain data, establishing an adjacency edge between the same type data in the target domain data, and calculating the weight of the adjacency edge to obtain a weight matrixWherein W t Is the weight matrix of the target domain and is calculated by:
whereinIs the feature vector of the ith target domain image,for the pseudo label corresponding to the ith target field image, beta i Is composed ofThe initial value of (1) is set to be 1.
In the preferred scheme, by means of the method for constructing the adjacent maps in the source domain and the target domain respectively, the structural consistency of the data from an original space to an invariant subspace can be maintained, the semantic information of the source domain data label information is utilized to construct the adjacent maps, the structure of the adjacent maps can be dynamically adjusted, so that the more essential local structure of the data is mined, and the forward migration of the knowledge in the domain is promoted.
Preferably, the specific step S3 is:
s3.1, in a common low-dimensional subspace of the source domain and the target domain, performing inter-domain data distribution alignment including edge distribution and conditional probability distribution by using a maximum mean difference criterion, specifically, minimizing the following first objective function:
wherein C represents the number of categories of label information, x s,c And x t,c Respectively indicating that the category of the source domain data label information belongs to the set of categories c and the category of the target domain data label information belongs to the set of categories c,andrespectively indicating the number of data of which the source domain data belongs to the category c of the tag information and the number of data of which the destination domain data belongs to the category c of the tag information,is a projection matrix, where d < m, for projecting data from m dimensions to d-dimensional subspace;
s3.2. Using weight matrix W s And W t Performing knowledge migration of data within the domain, specifically by minimizing the following second objective function:
s3.3, combining the first objective function and the second objective function and adding a regularizationThe following third objective function is obtained:
s.t.P T XHX T P=I
wherein λ and α are balance factors, λ > 0 and α > 0, X = [ X ] = s ,X t ]H is the centering matrix:i is a unit matrix, and the unit matrix is,is a full 1 column vector of value 1;
solving a projection matrix P through generalized eigen decomposition, and obtaining the expressions of the source domain data and the target domain data in the low-dimensional subspace as follows:and
in the preferred scheme, the difference between the edge distribution and the class condition distribution of the data of the source domain and the target domain is eliminated by searching the common low-dimensional subspace of the source domain and the target domain based on the maximum mean difference criterion.
Preferably, the specific steps of step S4 are:
obtaining the source domain data with label information in the low-dimensional subspace according to the representation of the source domain data in the low-dimensional subspaceFor training the nearest neighbor classifier and predicting target domain data in a low-dimensional spaceClass information of
In the preferred scheme, a nearest neighbor classifier is trained by using source domain data with label information in a low-dimensional subspace, and the pseudo label information of the target domain data is continuously optimized in an iterative manner, so that the final label information of the target domain data is obtained.
Preferably, the specific step of step S5 includes:
s5.1. for target domain data in a low-dimensional spaceConstructing a new adjacency graph, i.e. each target domain data point in the low-dimensional spaceRespectively establishing adjacent edges with k target domain data points with the shortest Euclidean distances;
s5.2, based on the adjacency graph in the step S5.1, for the ith target domain data point on the granted adjacency graphIf and withThe connected k target domain data points are all connected withIf they belong to one category, the target domain data pointIs marked with a labelConfidence of (beta) i =1, otherwise β i =0, wherein k is a positive integer.
In the preferred scheme, different confidences are given to the target domain samples based on a sample re-weighting strategy, so that the negative migration of the knowledge in the domain is effectively inhibited.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the method of the invention carries out inter-domain data distribution alignment comprising edge distribution and conditional probability distribution by utilizing the maximum mean difference criterion, searches a common low-dimensional subspace of a source domain and a target domain, eliminates the difference between the edge distribution and the class conditional distribution of the data of the source domain and the target domain, and leads the data distribution to be aligned;
constructing respective adjacency graphs according to the label information of the source domain data and the pseudo label information of the target domain data, keeping the structural consistency of the data from an original space to a low-dimensional subspace, and simultaneously dynamically adjusting the structure of the adjacency graphs to promote the forward migration of the knowledge in the domain; training a nearest neighbor classifier by using source domain data with label information in a low-dimensional subspace, and continuously iterating and optimizing pseudo label information of target domain data to obtain final label information of the target domain data, namely finishing cross-domain image classification;
in addition, the method of the invention endows different confidence coefficients of the pseudo labels of the target domain images by designing a sample re-weighting strategy, effectively reduces the negative transfer of the knowledge in the domain, and improves the accuracy of cross-domain image classification.
The method is based on an iterative method, and can effectively combine data distribution alignment to promote the intra-domain knowledge positive migration and inhibit the intra-domain knowledge negative migration, thereby achieving the purpose of coupling knowledge migration and greatly improving the performance of cross-domain image classification. The method solves the problems of long calculation time, insufficient classification accuracy and the like of the conventional cross-domain image classification method.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is an effect diagram after performing inter-domain data distribution alignment in embodiment 2.
Fig. 3 is a diagram illustrating the distribution effect of the label information types in the low-dimensional space for the target domain in this embodiment 2.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
for the purpose of better illustrating the present embodiments, certain elements of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;
it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical solution of the present invention is further described with reference to the drawings and the embodiments.
Example 1
A cross-domain image classification method based on coupled knowledge migration, as shown in FIG. 1, includes the following steps:
s1, obtaining a source domain image and a target domain image, extracting features, respectively obtaining a source domain feature matrix as source domain data, and obtaining a target domain feature matrix as target domain data:
s1.1. Input n s A tagged source domain image and n t Opening an image of the target domain without a label; wherein n is s 、n t Is a positive integer;
s1.2, respectively extracting the features of the source domain image and the target domain image, respectively extracting the feature vectors of the source domain image and the target domain image through any one of pre-trained neural networks of VGG16, ***Ne and ResNet50, and arranging the feature vectors extracted from the source domain image in columns to obtain a source domain data feature matrixWhere m is the dimension of the feature vector, n s The number of the feature vectors;
arranging the feature vectors extracted from the target domain image according to columns to obtain a target domain data feature matrixWhere m is the dimension of the feature vector, n t The number of the feature vectors;
the feature vector of the source domain image represents source domain data of the source domain image, and the feature vector of the target domain image represents target domain data of the target domain image.
S2, respectively constructing an adjacency graph of source domain data and an adjacency graph of target domain data according to label information of the source domain data, and respectively obtaining a weight matrix of the source domain and a weight matrix of the target domain by calculating edge weights of the adjacency graphs:
s2.1, judging whether the iteration is the first iteration or not, if not, directly carrying out the step S2.2, if so, constructing a nearest neighbor classifier and training the nearest neighbor classifier by utilizing the source domain data for predicting pseudo label information of target domain data
S2.2, constructing an adjacency graph of the source domain data according to the label information of the source domain data, namely establishing adjacent edges among the same-class data in the source domain data, and calculating the weights of the adjacent edges to obtain a weight matrix
Wherein W s A weight matrix for the source domain, calculated by:
whereinIs the feature vector of the ith source domain image,the label information corresponding to the ith source domain image is represented by the bandwidth of a Gaussian kernel, and | · | | | represents a vector two-norm;
constructing an adjacency graph of the target domain data according to the pseudo label information of the target domain data, establishing an adjacency edge between the same type data in the target domain data, and calculating the weight of the adjacency edge to obtain a weight matrix
Wherein W t Is the weight matrix of the target domain and is calculated by:
whereinIs the feature vector of the ith target domain image,for the pseudo label corresponding to the ith target field image, beta i Is composed ofThe initial value of (1) is set.
S3, performing inter-domain data distribution alignment in a common low-dimensional subspace of the source domain and the target domain, and simultaneously performing knowledge migration of intra-domain data by using a weight matrix of the source domain and a weight matrix of the target domain:
s3.1, in a common low-dimensional subspace of the source domain and the target domain, performing inter-domain data distribution alignment including edge distribution and conditional probability distribution by using a maximum mean difference criterion, specifically, minimizing the following first objective function:
wherein C represents the number of categories of label information, x s,c And x t,c Respectively indicating that the category of the source domain data label information belongs to the set of categories c and the category of the target domain data label information belongs to the set of categories c,andrespectively indicating the number of data of which the source domain data belongs to the category c of the tag information and the number of data of which the destination domain data belongs to the category c of the tag information,is a projection matrix, where d < m, for projecting data from m dimensions to d-dimensional subspace;
s3.2. Using weight matrix W s And W t Performing knowledge migration of data in the domain, specifically by minimizing the following second objective function:
s3.3, combining the first objective function and the second objective function and adding a regularizationThe following third objective function is obtained:
s.t.P T XHX T P=I
wherein λ and α are balance factors, λ > 0 and α > 0, X = [ X ] = s ,X t ]H is the centering matrix:i is a unit matrix, and the unit matrix is,is a full 1 column vector of value 1;
solving a projection matrix P through generalized eigen decomposition to obtain the expressions of the source domain data and the target domain data in the low-dimensional subspace, wherein the expressions are respectively as follows:and
s4, in the low-dimensional subspace, utilizing the source domain data with the label information to train a nearest neighbor classifier, and predicting the category information of the target domain data: obtaining the source domain data with label information in the low-dimensional subspace according to the representation of the source domain data in the low-dimensional subspaceFor training the nearest neighbor classifier and predicting target domain data in a low-dimensional spaceClass information of
S5, reweighing the label information of the predicted target domain data:
s5.1. for target domain data in a low-dimensional spaceConstructing a new adjacency graph, i.e. each target domain data point in the low-dimensional spaceRespectively establishing adjacent edges with k target domain data points with the shortest Euclidean distance;
s5.2, based on the adjacency graph in the step S5.1, for the ith target domain data point on the granted adjacency graphIf and withThe connected k target domain data points are all connected withIf they belong to one category, the target domain data pointOf (2)Confidence of (beta) i =1, otherwise β i =0, wherein k is a positive integer.
S6, judging whether the maximum iteration times is reached or not, and if not, returning to the step S2, if yes, outputting the label information of the predicted target domain image
Example 2
In this embodiment 2, 980 face images each of which is composed of face images of 10 persons randomly selected from the CMU PIE data set were selected, and each image was cut to 32 × 32 resolution. Further, 490 images in the C05 subset are used as a source domain data set, and 490 images in the C27 subset are used as a target domain data set to be classified.
S1, obtaining a source domain image in a source domain data set and a target domain image in a target domain data set, after graying the source domain image and the target domain image respectively, arranging gray values of the source domain image into a feature vector and arranging gray values of the target domain image into a feature vector, in this embodiment 2, gray values of each image are arranged into a feature vector, that is, each image is represented as a 1024-dimensional column vector, and correspondingly, source domain data feature matrix matrixes can be obtained respectively by arranging the images in columnsAnd a target domain feature matrix
S2, respectively constructing an adjacency graph of source domain data and an adjacency graph of target domain data according to label information of the source domain data, and respectively obtaining a weight matrix of the source domain and a weight matrix of the target domain by calculating edge weights of the adjacency graphs:
s2.1, judging whether the iteration is the first iteration or not, if not, directly carrying out the step S2.2, if so, constructing a nearest neighbor classifier and utilizing the source domain data with the labeled informationTraining nearest neighbor classifiers for predicting target domain dataPseudo category information of
S2.2. Tag information y according to source domain data s Constructing an adjacency graph of the source domain data, namely establishing an adjacency edge between homogeneous data in the source domain data, and calculating the weight of the adjacency edge to obtain a weight matrix
Wherein W s A weight matrix for the source domain, calculated by:
whereinIs the feature vector of the ith source domain image,for the label information corresponding to the ith source domain image, σ is the bandwidth of the gaussian kernel, which is set to 1 in this embodiment 1, and | · | | | represents a vector two-norm;
pseudo label information according to the target domain dataConstructing an adjacent graph of target domain data, establishing adjacent edges among the same-class data in the target domain data, and calculating the weights of the adjacent edges to obtain a weight matrixWherein W t A weight matrix for the target domain and calculated by:
whereinIs the feature vector of the ith target domain image,for the pseudo label corresponding to the ith target field image, beta i Is composed ofThe initial value of (1) is set to be 1.
S3, performing inter-domain data distribution alignment in a common low-dimensional subspace of the source domain and the target domain, and simultaneously performing knowledge migration of intra-domain data by using a weight matrix of the source domain and a weight matrix of the target domain;
by minimizing the following objective function:
s.t.P T XHX T P=I
where λ =0.1 and α =0.01 are balance factors. X = [ X = s ,X t ]H is the centering matrix:I 980 is a unit matrix, 1 980 Is a full 1 column vector. Solving the above formula through generalized eigen decomposition to obtain a projection matrixThe dimension of the low-dimensional subspace is set to d =100 in this embodiment 2. And finally, the representation of the source domain data and the target domain data in the low-dimensional subspace is obtained:and
s4, according to the representation of the source domain data in the low-dimensional subspace, obtaining the source domain data with label information in the low-dimensional subspaceFor training the nearest neighbor classifier and predicting target domain data in a low-dimensional spaceClass information of
S5, reweighing the label information of the predicted target domain data;
s5.1. For target domain data in a low-dimensional spaceConstructing a new adjacency graph, i.e. each target domain data point in the low-dimensional spaceRespectively establishing adjacent edges with k =5 target domain data points with the shortest Euclidean distance;
s5.2, based on the adjacency graph in the step S5.1, for the ith target domain data point on the granted adjacency graphIf and withThe 5 connected target domain data points are all connected withIf they belong to one category, the target domain data pointIs markedLabel (Bao)Confidence of (beta) i =1, otherwise beta i =0。
S6, judging whether the maximum iteration number is reached to 10, if not, returning to the step S2, and if so, outputting the label information of the predicted target domain imageThereby obtaining a classification result.
In this embodiment 2, the experimental platform is MATLAB R2017a software on WIN10 system, and the CPU model is Intel i7-6700k @4.00ghz. The experimental result diagram of this embodiment 2 is shown in fig. 2 and 3, where in fig. 2, a circle represents source domain data, a pentagon represents target domain data, and a left diagram of fig. 2 is an original distribution of the source domain data and the target domain data, and it can be obviously seen that the distribution of the source domain data and the target domain data has a great difference; the right diagram of fig. 2 is data distribution after inter-domain data distribution alignment is performed on source domain data and data target domain data in a low-dimensional subspace in the method of embodiment 2, and it can be obviously seen that the difference of the source domain data and the target domain data distribution is greatly reduced. In fig. 3, labels of different shapes represent different label information categories, for a total of ten label information categories. The left sub-graph in fig. 3 is the distribution of each label information of the original target domain data, and the category distribution is relatively disordered; fig. 3 is a right sub-graph of the distribution of the target domain data in the low-dimensional subspace obtained by the method of this embodiment 2, which shows that the category distribution of the target domain data is clear, and each data with the same tag information forms a cluster. This demonstrates the high efficiency of the process of the invention.
The terms describing positional relationships in the drawings are for illustrative purposes only and are not to be construed as limiting the patent;
it should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (7)
1. A cross-domain image classification method based on coupling knowledge migration is characterized by comprising the following steps:
s1, obtaining a source domain image and a target domain image, extracting characteristics, respectively obtaining a source domain characteristic matrix as source domain data, and obtaining a target domain characteristic matrix as target domain data;
s2, respectively constructing an adjacency graph of source domain data and an adjacency graph of target domain data according to label information of the source domain data, and respectively obtaining a weight matrix of the source domain and a weight matrix of the target domain by calculating edge weights of the adjacency graphs;
s3, performing inter-domain data distribution alignment in a common low-dimensional subspace of the source domain and the target domain, and performing knowledge migration of intra-domain data by using a weight matrix of the source domain and a weight matrix of the target domain;
s4, in the low-dimensional subspace, training a nearest neighbor classifier by using source domain data with label information, and predicting the label information of target domain data;
s5, reweighing the label information of the predicted target domain data;
s6, judging whether the maximum iteration number is reached, if not, returning to the step S2, and if so, outputting label information of the predicted target domain image so as to obtain a classification result;
the S3 comprises the following specific steps:
s3.1, in a common low-dimensional subspace of the source domain and the target domain, performing inter-domain data distribution alignment including edge distribution and conditional probability distribution by using a maximum mean difference criterion, specifically, minimizing the following first objective function:
wherein C represents the number of categories of label information, x s,c And x t,c Respectively indicating that the category of the source domain data label information belongs to the set of categories c and the category of the target domain data label information belongs to the set of categories c,andrespectively indicating the number of data of which the source domain data belongs to the category c of the tag information and the number of data of which the destination domain data belongs to the category c of the tag information,is a projection matrix, where d < m, for projecting data from m dimensions into a d-dimensional subspace, m representing the dimensions of the eigenvectors;the feature vector of the ith source domain image is obtained; n is s Number of source domain images for which a label is applied; n is t Representing the number of unlabeled target domain images;the feature vector of the ith target domain image is obtained;
s3.2. Using weight matrix W s And W t Performing knowledge migration of data within the domain, specifically by minimizing the following second objective function:
s3.3, combining the first objective function and the second objective function and adding a regularizationThe following third objective function is obtained:
s.t.P T XHX T P=I
wherein λ and α are balance factors, λ > 0 and α > 0, X = [ X ] = s ,X t ]H is the centering matrix:i is an identity matrix and is a matrix of the identity,is a full 1-column vector of value 1; x s Representing a source domain data feature matrix, X t Representing a target domain data feature matrix;
2. the cross-domain image classification method based on coupling knowledge migration according to claim 1, wherein the specific steps of the step S1 are:
s1.1. Input n s A source domain image with label and n t Opening an image of the target domain without a label; wherein n is s 、n t Is a positive integer;
s1.2, respectively extracting the features of the source domain image and the target domain image, and arranging the feature vectors obtained by extracting the source domain image in columns to obtain a source domain data feature matrixWherein m is the dimension of the feature vector;
arranging the characteristic vectors extracted from the target domain image according to columns to obtain a target domain data characteristic matrixWherein m is the dimension of the feature vector;
the feature vector of the source domain image represents source domain data of the source domain image, and the feature vector of the target domain image represents target domain data of the target domain image.
3. The method for cross-domain image classification based on coupling knowledge migration according to claim 2, wherein the feature extraction in step S1 is to extract feature vectors of the source domain image and the target domain image respectively through a pre-trained neural network.
4. The cross-domain image classification method based on coupling knowledge migration according to claim 2, wherein the feature extraction in step S1 specifically is to arrange gray values of the source domain image into feature vectors and gray values of the target domain image into feature vectors after graying the source domain image and the target domain image respectively.
5. The cross-domain image classification method based on coupling knowledge migration according to claim 3 or 4, wherein the specific step S2 is:
s2.1, judging whether the iteration is the first iteration or not, if not, directly carrying out the step S2.2, if so, constructing a nearest neighbor classifier and training the nearest neighbor classifier by utilizing the source domain data for predicting pseudo label information of target domain data
S2.2. According to the source domain data markLabeling information, constructing an adjacency graph of source domain data, establishing an adjacency edge between the same type data in the source domain data, and calculating the weight of the adjacency edge to obtain a weight matrix
Wherein W s A weight matrix for the source domain, calculated by:
whereinIs the feature vector of the ith source domain image,the label information corresponding to the ith source domain image is represented by the bandwidth of a Gaussian kernel, and | · | | | represents a vector two-norm;
constructing an adjacency graph of the target domain data according to the pseudo label information of the target domain data, establishing an adjacency edge between the same type data in the target domain data, and calculating the weight of the adjacency edge to obtain a weight matrix
Wherein W t Is the weight matrix of the target domain and is calculated by:
6. The cross-domain image classification method based on coupling knowledge migration according to claim 5, wherein the specific steps of the step S4 are as follows:
obtaining the source domain data with label information in the low-dimensional subspace according to the representation of the source domain data in the low-dimensional subspaceFor training the nearest neighbor classifier and predicting target domain data in a low-dimensional spacePseudo tag information of
7. The cross-domain image classification method based on coupling knowledge migration according to claim 6, wherein the specific steps of the step S5 comprise:
s5.1. For target domain data in a low-dimensional spaceConstructing a new adjacency graph, and setting each target domain data point in a low-dimensional spaceRespectively establishing adjacent edges with k target domain data points with the shortest Euclidean distance; i =1,2, \8230;, n t ;
S5.2. Based onFor the adjacency graph stated in step S5.1, for the ith target domain data point on the granted adjacency graphIf and withThe k target domain data points connected withIf the same belongs to a category, the target domain data pointIs marked with a labelWeight of beta i =1, otherwise β i =0, wherein k is a positive integer.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910482559.5A CN110378366B (en) | 2019-06-04 | 2019-06-04 | Cross-domain image classification method based on coupling knowledge migration |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910482559.5A CN110378366B (en) | 2019-06-04 | 2019-06-04 | Cross-domain image classification method based on coupling knowledge migration |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110378366A CN110378366A (en) | 2019-10-25 |
CN110378366B true CN110378366B (en) | 2023-01-17 |
Family
ID=68249801
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910482559.5A Active CN110378366B (en) | 2019-06-04 | 2019-06-04 | Cross-domain image classification method based on coupling knowledge migration |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110378366B (en) |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110781970B (en) * | 2019-10-30 | 2024-04-26 | 腾讯科技(深圳)有限公司 | Classifier generation method, device, equipment and storage medium |
CN111312403A (en) * | 2020-01-21 | 2020-06-19 | 山东师范大学 | Disease prediction system, device and medium based on instance and feature sharing cascade |
CN111814871B (en) * | 2020-06-13 | 2024-02-09 | 浙江大学 | Image classification method based on reliable weight optimal transmission |
CN111929053B (en) * | 2020-07-07 | 2022-02-01 | 中国矿业大学 | DA-RVFLN-based pneumatic control valve field adaptive fault diagnosis method |
CN112085085B (en) * | 2020-09-03 | 2024-02-23 | 哈尔滨工业大学 | Multi-source migration learning method based on graph structure |
CN112733970B (en) * | 2021-03-31 | 2021-06-18 | 腾讯科技(深圳)有限公司 | Image classification model processing method, image classification method and device |
CN113420775B (en) * | 2021-03-31 | 2024-03-29 | 中国矿业大学 | Picture classification method based on nonlinearity self-adaptive subdomain field adaptation under extremely small quantity of training samples |
CN113469405B (en) * | 2021-05-18 | 2024-04-02 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Subimage segmentation-based cross-city tail gas migration prediction method, system and medium |
CN113657651B (en) * | 2021-07-27 | 2024-07-09 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Deep migration learning-based diesel vehicle emission prediction method, medium and equipment |
CN113836996B (en) * | 2021-08-10 | 2024-02-02 | 中国地质大学(武汉) | Hyperspectral remote sensing image part migration method and hyperspectral remote sensing image part migration system |
CN114743043B (en) * | 2022-03-15 | 2024-04-26 | 北京迈格威科技有限公司 | Image classification method, electronic device, storage medium and program product |
CN115393362B (en) * | 2022-10-28 | 2023-02-03 | 中南大学 | Method, equipment and medium for selecting automatic glaucoma recognition model |
CN117216507A (en) * | 2023-10-08 | 2023-12-12 | 深圳大学 | Deep neural network model mobility measurement method based on geographic partition |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103971129A (en) * | 2014-05-27 | 2014-08-06 | 浙江大学 | Classification method and device based on learning image content recognition in cross-data field subspace |
CN107895177A (en) * | 2017-11-17 | 2018-04-10 | 南京邮电大学 | A kind of migration classification learning method for keeping image classification sparsity structure |
CN109389174A (en) * | 2018-10-23 | 2019-02-26 | 四川大学 | A kind of crowd massing Sensitive Image Detection Method |
-
2019
- 2019-06-04 CN CN201910482559.5A patent/CN110378366B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103971129A (en) * | 2014-05-27 | 2014-08-06 | 浙江大学 | Classification method and device based on learning image content recognition in cross-data field subspace |
CN107895177A (en) * | 2017-11-17 | 2018-04-10 | 南京邮电大学 | A kind of migration classification learning method for keeping image classification sparsity structure |
CN109389174A (en) * | 2018-10-23 | 2019-02-26 | 四川大学 | A kind of crowd massing Sensitive Image Detection Method |
Non-Patent Citations (1)
Title |
---|
迁移学习方法研究及其在跨领域数据分类中的应用;覃姜维;《中国博士学位论文全文数据库 信息科技辑》;20111215(第12期);I140-39 * |
Also Published As
Publication number | Publication date |
---|---|
CN110378366A (en) | 2019-10-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110378366B (en) | Cross-domain image classification method based on coupling knowledge migration | |
Liu et al. | Connectionist temporal classification with maximum entropy regularization | |
Wang et al. | Exploring domain-invariant parameters for source free domain adaptation | |
CN110532542B (en) | Invoice false invoice identification method and system based on positive case and unmarked learning | |
US20220382553A1 (en) | Fine-grained image recognition method and apparatus using graph structure represented high-order relation discovery | |
CN110224987B (en) | Method for constructing network intrusion detection model based on transfer learning and detection system | |
Xiao et al. | Mcapsnet: Capsule network for text with multi-task learning | |
US20220076074A1 (en) | Multi-source domain adaptation with mutual learning | |
CN113326731A (en) | Cross-domain pedestrian re-identification algorithm based on momentum network guidance | |
CN112906770A (en) | Cross-modal fusion-based deep clustering method and system | |
CN110751027B (en) | Pedestrian re-identification method based on deep multi-instance learning | |
Shah et al. | Max-margin contrastive learning | |
CN113554100B (en) | Web service classification method for enhancing attention network of special composition picture | |
CN112861929B (en) | Image classification method based on semi-supervised weighted migration discriminant analysis | |
Liu et al. | Semi-supervised change detection based on graphs with generative adversarial networks | |
CN114863091A (en) | Target detection training method based on pseudo label | |
CN114048295A (en) | Cross-modal retrieval method and system for data processing | |
CN115114409A (en) | Civil aviation unsafe event combined extraction method based on soft parameter sharing | |
Zhang et al. | Projected-prototype based classifier for text categorization | |
CN114792114B (en) | Unsupervised domain adaptation method based on black box multi-source domain general scene | |
Sener et al. | Unsupervised transductive domain adaptation | |
CN113392876B (en) | Small sample image classification method based on graph neural network | |
CN114676755A (en) | Non-supervision domain self-adaptive classification method based on graph convolution network | |
Yu et al. | Graph-structure based multi-label prediction and classification for unsupervised person re-identification | |
CN113344189A (en) | Neural network training method and device, computer equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |