CN113095229B - Self-adaptive pedestrian re-identification system and method for unsupervised domain - Google Patents
Self-adaptive pedestrian re-identification system and method for unsupervised domain Download PDFInfo
- Publication number
- CN113095229B CN113095229B CN202110399589.7A CN202110399589A CN113095229B CN 113095229 B CN113095229 B CN 113095229B CN 202110399589 A CN202110399589 A CN 202110399589A CN 113095229 B CN113095229 B CN 113095229B
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- domain sample
- pictures
- picture
- representing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000012549 training Methods 0.000 claims abstract description 21
- 238000005457 optimization Methods 0.000 claims abstract description 15
- 230000006870 function Effects 0.000 claims description 132
- 230000003044 adaptive effect Effects 0.000 claims description 13
- 238000010606 normalization Methods 0.000 claims description 13
- 238000005259 measurement Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 3
- 238000013508 migration Methods 0.000 abstract description 12
- 230000005012 migration Effects 0.000 abstract description 12
- 238000012163 sequencing technique Methods 0.000 abstract 1
- 238000012360 testing method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an unsupervised domain self-adaptive pedestrian re-recognition system and method, belongs to the technical field of pedestrian re-recognition, and solves the problems of high difficulty and low recognition accuracy of the existing unsupervised domain self-adaptive pedestrian re-recognition. The system comprises a data acquisition module for acquiring a plurality of source domain sample subsets and a plurality of target domain sample subsets; the network model training module is used for obtaining a classification loss function and a sample invariance loss function of the pedestrian re-recognition network model, sequencing and layering the pedestrian pictures in the target domain sample subset according to the similarity of each pedestrian picture in the source domain sample subset and each pedestrian picture in the target domain sample subset to obtain a layering loss function, and further carrying out iterative optimization on the pedestrian re-recognition network model; and the re-recognition module is used for re-recognizing the pedestrians by using the optimized pedestrian re-recognition network model, and obtaining an image which is the same as or similar to the image of the pedestrians to be recognized. The system can reduce the migration loss of the network and improve the accuracy of pedestrian re-identification.
Description
Technical Field
The invention relates to the technical field of pedestrian re-recognition, in particular to an unsupervised domain self-adaptive pedestrian re-recognition system and method.
Background
In traditional unsupervised domain adaptive learning, most methods are used in closed set scenarios where the sample categories in the source domain and the target domain are consistent. However, the conventional unsupervised domain adaptive algorithm cannot be used for unsupervised domain adaptive pedestrian re-recognition because the pedestrian categories of the source domain and the target domain are almost different due to image acquisition, and a large migration loss is generated when a network trained by using the source domain image is migrated to the target domain for recognition. The large migration loss caused by network migration can make unsupervised domain adaptive pedestrian re-recognition more challenging than most unsupervised learning.
In the prior art, the purpose of distinguishing the unlabeled dataset is achieved mainly through self-clustering of the target domain by self-adaptive pedestrian re-recognition of the unsupervised domain. In the prior art, three kinds of invariance loss based on a target domain are proposed to reduce the migration loss of a network, but marked data is not fully utilized, and deep semantic information of a label-free picture is not fully mined, so that the migration loss of the network cannot be effectively reduced, and the accuracy of pedestrian re-identification is still to be improved.
At least the following defects exist in the prior art, the source domain picture information and the target domain picture information cannot be fully utilized, and the negative samples with different similarities cannot be subjected to network distinguishing learning according to the similarities between the source domain picture and the target domain picture, so that the migration loss of the pedestrian re-recognition network is large, and the accuracy of the pedestrian re-recognition result is low.
Disclosure of Invention
In view of the above analysis, the embodiment of the invention aims to provide an unsupervised domain self-adaptive pedestrian re-recognition system and method, which are used for solving the problems of large migration loss of a pedestrian re-recognition network and low accuracy of a pedestrian re-recognition result in the prior art.
In one aspect, the present invention provides an unsupervised domain adaptive pedestrian re-recognition system, comprising:
a data acquisition module for acquiring a source domain sample set comprising a plurality of source domain sample subsets and a target domain sample set comprising a plurality of target domain sample subsets;
the network model training module is used for obtaining a classification loss function and a sample invariance loss function of the pedestrian re-identification network model, sorting and layering pedestrian pictures in the target domain sample subset according to the similarity of each pedestrian picture in the source domain sample subset and each pedestrian picture in the target domain sample subset, and obtaining a layering loss function based on the characteristics of the layered pedestrian pictures and the corresponding layering weights; performing iterative optimization on the pedestrian re-recognition network model by traversing each source domain sample subset and each target domain sample subset based on the classification loss function, the sample invariance loss function and the hierarchical loss function;
and the re-recognition module is used for recognizing the pedestrian image to be recognized by utilizing the optimized pedestrian re-recognition network model, and obtaining an image identical or similar to the pedestrian image to be recognized.
Further, the pedestrian re-recognition network model comprises a residual network structure, a full connection layer and a Softmax normalization layer which are sequentially connected and correspond to the classification loss function, a target domain memory which is sequentially connected and corresponds to the sample invariance loss function, and L 2 And the residual error network structure is respectively connected with the full-connection layer, the target domain memory, the source domain memory and the similarity measurement axis network structure.
Further, the network model training module obtains the layered loss function specifically by:
respectively inputting a source domain sample subset and a target domain sample subset into a residual network structure of a pedestrian re-identification network model to extract image features so as to obtain the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset, respectively storing the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset, and multiplying the features of each pedestrian picture in the source domain sample subset by the features of each pedestrian picture in the target domain sample subset to obtain corresponding similarity;
sorting pedestrian pictures in the target domain sample subset in a descending order based on the similarity, sequentially selecting a first preset number of pictures as a first layer of pictures according to the order, selecting a second preset number of pictures as a second layer of pictures, selecting the rest pictures as a third layer of pictures, and respectively setting layer weights of the three layers of pictures;
obtaining a layering loss function based on the characteristics of each pedestrian picture in the target domain sample subset and the layering weight corresponding to the layering to which the characteristics belong:
wherein L is SL Represents a hierarchical loss function, n t Representing the number, w, of pedestrian pictures in a target domain sample subset t,r Representing layer weights, r represents the sequence number, x of pedestrian pictures in the target domain sample subset according to similarity sequence t,i Representing a pedestrian picture with input sequence i when a target domain sample subset is input into a pedestrian re-recognition network model, f (x) t,i ) Representing pedestrian picture x t,i Is characterized by p (m|x t,i ) Representing pedestrian picture x t,i Probability of similarity to pedestrian pictures of all categories in all source domain sample subsets stored, R 1 [m]Features representing all categories of pedestrian pictures in all source domain sample subsets stored, R 1 [j]Representing the jth class in all source domain sample subsets storedCharacteristics of pedestrian pictures, N s And representing the category number of the pedestrian pictures in all the stored source domain sample subsets, wherein beta is a temperature coefficient.
Further, the set layer weights of the three layers of pictures are as follows:
wherein b represents the number of pictures of the first layer, k 1 Representing the sum of the numbers of the first layer picture and the second layer picture, and p (l|i) represents the pedestrian picture x t,i Probability of class I pedestrian picture in all stored target domain sample subsets, R 1 [l]Representing the characteristics of the class I pedestrian pictures in all the stored target domain sample subsets, N t And representing the category number of the pedestrian pictures in all the stored target domain sample subsets.
Further, the network model training module obtains the classification loss function specifically by:
inputting the source domain sample subset into a residual network structure of a pedestrian re-recognition network model to extract image features so as to obtain the features of each pedestrian picture in the source domain sample subset;
sequentially inputting the characteristics of each pedestrian picture into a full-connection layer and a softmax regression layer of the pedestrian re-recognition network model, and carrying out characteristic dimension conversion and characteristic normalization;
classifying the loss function based on the characteristics of each pedestrian picture after dimension conversion and normalization by adopting the following formula;
wherein L is src Represents a classification loss function, n s Representing the number of pedestrian pictures in a source domain sample subset, x s,m Representing the mth pedestrian picture, f (x s,m ) Representing pedestrian picture x s,m Is characterized by y s,m Representing pedestrian picture x in source domain sample subset s,m Category labels of (c), p (y) s,m |x s,m ) Representing pedestrian picture x s,m Belonging to category y s,m Probability of f (x) s,j ) Representing the features of the jth pedestrian picture in the source domain sample subset, N s Representing the number of categories of pedestrian pictures in all the stored source domain sample subsets.
Further, the network model training module obtains the sample invariance loss function by:
inputting the target domain sample subset into a residual network structure of a pedestrian re-recognition network model to extract image features so as to obtain the features of each pedestrian picture in the target domain sample subset, and carrying out L on the features of each pedestrian picture 2 Standardization treatment;
the sample invariance loss function is obtained according to the characteristics of each pedestrian picture after the normalization processing through the following formula:
wherein L is T Representing a sample invariance loss function, n t Representing the number of pedestrian pictures in a target domain sample subset, x t,i Representing a pedestrian picture with input sequence i when a target domain sample subset is input into a pedestrian re-recognition network model, f (x) t,i ) Representing pedestrian picture x t,i Is characterized by w i,l Representing pedestrian picture x t,i Weights of pedestrian pictures belonging to class I，p(l|x t,i ) Representing pedestrian picture x t,i Probability of belonging to class I pedestrian pictures, N t And representing the category number of the pedestrian pictures in all the stored target domain sample subsets, wherein beta is a temperature coefficient.
Further, the network model training module performs iterative optimization on the pedestrian re-recognition network model specifically by the following manner:
obtaining a total loss function according to the classification loss function, the sample invariance loss function and the layering loss function:
L＝λ 1 L src +λ 2 L T +λ 3 L SL ，
wherein L represents the total loss function, L src Represents a class loss function, lambda 1 Weights representing classification loss functions, L T Representing the sample invariance loss function, lambda 2 Weights representing sample invariance loss functions, L SL Represents a layered loss function, lambda 3 Weights representing hierarchical loss functions;
and traversing each source domain sample subset and each target domain sample subset, and iteratively updating the total loss function until the variation of the total loss function value is smaller than a preset value, so as to finish the optimization of the pedestrian re-identification network model.
On the other hand, the invention provides an unsupervised domain self-adaptive pedestrian re-identification method, which comprises the following steps:
obtaining a source domain sample set comprising a plurality of source domain sample subsets and a target domain sample set comprising a plurality of target domain sample subsets;
respectively obtaining a classification loss function and a sample invariance loss function of a pedestrian re-identification network model based on the source domain sample subset and the target domain sample subset, sorting and layering pedestrian pictures in the target domain sample subset according to the similarity of each pedestrian picture in the source domain sample subset and each pedestrian picture in the target domain sample subset, and obtaining a layering loss function based on the characteristics of the layered pedestrian pictures and the corresponding layering weights; performing iterative optimization on the pedestrian re-recognition network model by traversing each source domain sample subset and each target domain sample subset based on the classification loss function, the sample invariance loss function and the hierarchical loss function;
and identifying the pedestrian image to be identified by using the optimized pedestrian re-identification network model, and obtaining the image which is the same as or similar to the pedestrian image to be identified.
Further, the layered loss function is obtained specifically by:
respectively inputting a source domain sample subset and a target domain sample subset into a residual network structure of a pedestrian re-identification network model to extract image features so as to obtain the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset, respectively storing the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset, and multiplying the features of each pedestrian picture in the source domain sample subset by the features of each pedestrian picture in the target domain sample subset to obtain corresponding similarity;
sorting pedestrian pictures in the target domain sample subset in a descending order based on the similarity, sequentially selecting a first preset number of pictures as a first layer of pictures according to the order, selecting a second preset number of pictures as a second layer of pictures, selecting the rest pictures as a third layer of pictures, and respectively setting layer weights of the three layers of pictures;
obtaining a layering loss function based on the characteristics of each pedestrian picture in the target domain sample subset and the layering weight corresponding to the layering to which the characteristics belong:
wherein L is SL Represents a hierarchical loss function, n t Representing the number, w, of pedestrian pictures in a target domain sample subset t,r Representing layer weights, r represents the sequence number, x of pedestrian pictures in the target domain sample subset according to similarity sequence t,i Representing target domain sample subset input pedestrian re-recognitionInputting pedestrian pictures with the sequence of i and f (x) in other network models t,i ) Representing pedestrian picture x t,i Is characterized by p (m|x t,i ) Representing pedestrian picture x t,i Probability of similarity to pedestrian pictures of all categories in all source domain sample subsets stored, R 1 [m]Features representing all categories of pedestrian pictures in all source domain sample subsets stored, R 1 [j]Representing the features of the jth pedestrian picture in all the stored source domain sample subsets, N s And representing the category number of the pedestrian pictures in all the stored source domain sample subsets, wherein beta is a temperature coefficient.
Further, the set layer weights of the three layers of pictures are as follows:
wherein b represents the number of pictures of the first layer, k 1 Representing the sum of the numbers of the first layer picture and the second layer picture, and p (l|i) represents the pedestrian picture x t,i Probability of class I pedestrian picture in all stored target domain sample subsets, R 1 [l]Representing the characteristics of the class I pedestrian pictures in all the stored target domain sample subsets, N t And representing the category number of the pedestrian pictures in all the stored target domain sample subsets.
Compared with the prior art, the invention has at least one of the following beneficial effects:
1. according to the self-adaptive pedestrian re-recognition system and method for the unsupervised domain, the source domain sample with the label and the target domain sample without the label are combined to effectively combine the supervised pedestrian re-recognition and the unsupervised pedestrian re-recognition, and the target domain sample is layered through the similarity between the source domain sample and the target domain sample for the first time, so that the distinguishing degree of the pedestrian re-recognition network on the target domain sample is improved, the migration loss of the network is reduced, and the accuracy of the pedestrian re-recognition network on the pedestrian re-recognition result is improved.
2. According to the unsupervised domain self-adaptive pedestrian re-recognition system and method provided by the invention, the negative samples are layered and weighted by utilizing the similarity of the characteristics of the source domain samples and the target domain samples to obtain the layered loss function, the loss function enables the network to self-adaptively reduce the learning of the characteristics of the negative samples in the training of the pedestrian re-recognition network model, so that the migration loss of the network is reduced, on the basis, the classification loss function and the sample invariance loss function are combined to perform iterative optimization on the pedestrian re-recognition network, and the precision of the pedestrian re-recognition result of the pedestrian re-recognition network can be greatly improved.
In the invention, the technical schemes can be mutually combined to realize more preferable combination schemes. Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, like reference numerals being used to refer to like parts throughout the several views.
FIG. 1 is a schematic diagram of an unsupervised domain adaptive pedestrian re-identification system in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram of a pedestrian re-recognition network model according to an embodiment of the present invention;
fig. 3 is a flowchart of an unsupervised domain adaptive pedestrian re-identification method according to an embodiment of the present invention.
Reference numerals:
110-a data acquisition module; 120-a network model training module; 130-re-identification module.
Detailed Description
Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form a part hereof, and together with the description serve to explain the principles of the invention, and are not intended to limit the scope of the invention.
System embodiment
An embodiment of the invention discloses an unsupervised domain self-adaptive pedestrian re-identification system. As shown in fig. 1, the system includes:
a data acquisition module 110 for acquiring a source domain sample set comprising a plurality of source domain sample subsets and a target domain sample set comprising a plurality of target domain sample subsets. Specifically, a large number of pictures of pedestrians are randomly collected from different angles, each picture contains a pedestrian, the collected pictures are divided into a training set and a testing set, the training set is used for training the pedestrian re-recognition network model, and the testing set is used for testing the trained pedestrian re-recognition network model so as to ensure the recognition accuracy of the pedestrian re-recognition network model. Selecting part of pictures randomly in a training set, adding labels for each picture, specifically adding the same label for a plurality of pictures of the same pedestrian, wherein different labels represent different pedestrians, dividing the labeled pictures into a plurality of source domain sample subsets, taking other pictures without labels in the training set as target domain sample sets, adding numbers for each picture, and randomly dividing the numbered pictures into a plurality of target domain sample subsets.
The network model training module 120 is configured to obtain a classification loss function and a sample invariance loss function of the pedestrian re-recognition network model, sort and layer the pedestrian pictures in the target domain sample subset according to the similarity between each pedestrian picture in the source domain sample subset and each pedestrian picture in the target domain sample subset, and obtain a layering loss function based on the characteristics of the layered pedestrian pictures and the corresponding layering weights. And performing iterative optimization on the pedestrian re-recognition network model based on the classification loss function, the sample invariance loss function and the hierarchical loss function traversing each source domain sample subset and each target domain sample subset.
And the re-recognition module is used for recognizing the pedestrian image to be recognized by utilizing the optimized pedestrian re-recognition network model, and obtaining an image identical or similar to the pedestrian image to be recognized. Specifically, the pedestrian picture to be identified is input into a trained pedestrian re-identification network model, the model can output the picture most similar to the pedestrian picture to be identified, the model is set to output the numbers of the first three pictures with the highest similarity to the pedestrian picture to be identified, the first three pictures are pictures in a target domain sample set, and then the pedestrian picture to be identified and the first three pictures are manually compared to determine the category of the pedestrian picture to be identified.
Preferably, as shown in fig. 2, the pedestrian re-recognition network model comprises a residual network structure, a full connection layer and a Softmax normalization layer which are sequentially connected and correspond to the classification loss function, a target domain memory which is sequentially connected and corresponds to the sample invariance loss function, and L 2 And the residual error network structure is respectively connected with the full-connection layer, the target domain memory, the source domain memory and the similarity measurement axis network structure. Specifically, the target domain memory and the source domain memory are both key value storage structures, the key is used for storing the characteristics of the pedestrian picture, the value is used for storing the number or the label corresponding to the pedestrian picture, and the residual network structure is shown as ResNet50.
Preferably, the network model training module obtains the classification loss function, the sample invariance loss function, and the layering loss function specifically by:
step 1, respectively inputting a source domain sample subset and a target domain sample subset into a residual network structure of a pedestrian re-recognition network model to extract image features so as to obtain the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset.
The features of each pedestrian picture in the target domain sample subset are stored in the target domain memory and the features of each pedestrian picture in the source domain sample subset are stored in the Yu Yuanyu memory.
Step 2, specifically, obtaining a classification loss function by the following method:
and sequentially inputting the characteristics of each pedestrian picture in the source domain sample subset into a full-connection layer and a softmax regression layer of the pedestrian re-recognition network model, and carrying out characteristic dimension conversion and characteristic normalization to strengthen the nonlinearity of the pedestrian re-recognition network model in learning.
Classifying the loss function based on the characteristics of each pedestrian picture after dimension conversion and normalization by adopting the following formula;
wherein L is src Represents a classification loss function, n s Representing the number of pedestrian pictures in a source domain sample subset, x s,m Representing the mth pedestrian picture, f (x s,m ) Representing pedestrian picture x s,m Is characterized by y s,m Representing pedestrian picture x in source domain sample subset s,m Category labels of (c), p (y) s,m |x s,m ) Representing pedestrian picture x s,m Belonging to category y s,m Probability of f (x) s,j ) Representing the features of the jth pedestrian picture in the source domain sample subset, N s Representing the number of categories of pedestrian pictures in all the stored source domain sample subsets.
Step 3, specifically, obtaining a sample invariance loss function by the following method:
l is carried out on the characteristics of each pedestrian picture in the current target domain sample subset stored in the target domain memory 2 And (5) standardization treatment.
The sample invariance loss function is obtained according to the characteristics of each pedestrian picture after the normalization processing through the following formula:
wherein L is T Representing a sample invariance loss function, n t Representing the number of pedestrian pictures in the current target domain sample subset, x t,i Representing a pedestrian picture with the input sequence of i when the current target domain sample subset is input into the pedestrian re-recognition network model, and f (x) t,i ) Representing pedestrian picture x t,i Is characterized by w i,l Representing pedestrian picture x t,i Weights belonging to class i pedestrian pictures, p (l|x t,i ) Representing pedestrian picture x t,i Probability of belonging to class I pedestrian pictures, N t And representing the category number of the pedestrian pictures in all the stored target domain sample subsets, wherein beta is a temperature coefficient.
Step 4, obtaining a layered loss function by specifically:
multiplying the characteristics of each pedestrian picture in the current source domain sample subset with the characteristics of each pedestrian picture in the current target domain sample subset based on the similarity measurement axis network structure to obtain corresponding similarity;
and sorting pedestrian pictures in the target domain sample subset in a descending order based on the similarity, sequentially selecting a first preset number of pictures as a first layer of pictures according to the order, selecting a second preset number of pictures as a second layer of pictures, selecting the rest pictures as a third layer of pictures, and respectively setting the layer weights of the three layers of pictures. Illustratively, the number of first layer pictures is 3 and the number of second layer pictures is 147.
Obtaining a layering loss function based on the characteristics of each pedestrian picture in the target domain sample subset and the layering weight corresponding to the layering to which the characteristics belong:
wherein L is SL Represents a hierarchical loss function, n t Representing target domain samplesConcentrate the number of pedestrian pictures, w t,r Representing layer weights, r represents the sequence number, x of pedestrian pictures in the target domain sample subset according to similarity sequence t,i Representing a pedestrian picture with input sequence i when a target domain sample subset is input into a pedestrian re-recognition network model, f (x) t,i ) Representing pedestrian picture x t,i Is characterized by p (m|x t,i ) Representing pedestrian picture x t,i Probability of similarity to pedestrian pictures of all categories in all source domain sample subsets stored, R 1 [m]Features representing all categories of pedestrian pictures in all source domain sample subsets stored, R 1 [j]Representing the features of the jth pedestrian picture in all the stored source domain sample subsets, N s And representing the category number of the pedestrian pictures in all the stored source domain sample subsets, wherein beta is a temperature coefficient.
Preferably, the layer weights of the three layers of pictures are set as follows:
wherein b represents the number of pictures of the first layer, k 1 Representing the sum of the numbers of the first layer picture and the second layer picture, and p (l|i) represents the pedestrian picture x t,i Probability of class I pedestrian picture in all stored target domain sample subsets, R 1 [l]Representing the characteristics of the class I pedestrian pictures in all the stored target domain sample subsets, N t And representing the category number of the pedestrian pictures in all the stored target domain sample subsets.
Specifically, the steps 2 to 4 may be performed simultaneously without separating the steps from each other.
And 5, carrying out iterative optimization on the pedestrian re-recognition network model in the following manner:
obtaining a total loss function from the classification loss function, the sample invariance loss function and the hierarchical loss function:
L＝λ 1 L src +λ 2 L T +λ 3 L SL ，
wherein L represents the total loss function, L src Represents a class loss function, lambda 1 Weights representing classification loss functions, L T Representing the sample invariance loss function, lambda 2 Weights representing sample invariance loss functions, L SL Represents a layered loss function, lambda 3 Weights representing hierarchical loss functions, illustratively λ 1 Take a value of 0.7 lambda 2 Takes a value of 0.3 lambda 3 The value of (2) is 0.2.
And (3) traversing each source domain sample subset and each target domain sample subset, repeating the steps 1 to 5, and iteratively updating the total loss function until the variation of the total loss function value is smaller than a preset value, so as to finish the optimization of the pedestrian re-identification network model.
Specifically, in the iterative updating process, the pedestrian picture characteristics of the target domain sample subset stored in the target domain memory are updated in real time by the following modes:
x t,i representing a pedestrian picture with input sequence i when a target domain sample subset is input into a pedestrian re-recognition network model, f (x) t,i ) Representing pedestrian picture x t,i Warp L 2 Normalized features, R i Representing target domain pedestrian pictures x stored in target domain memory t,i Is characterized in that,is a hyper-parameter that controls the update rate of features.
Method embodiment
Another embodiment of the present invention discloses an unsupervised domain adaptive pedestrian re-recognition method, and since the method embodiment and the system embodiment are based on the same principle, the details are not repeated herein, and reference may be made to the system embodiment on the market for the repetition.
Specifically, as shown in fig. 3, the method includes:
s110, acquiring a source domain sample set comprising a plurality of source domain sample subsets and a target domain sample set comprising a plurality of target domain sample subsets.
S120, respectively obtaining a classification loss function and a sample invariance loss function of a pedestrian re-identification network model based on the source domain sample subset and the target domain sample subset, sorting and layering pedestrian pictures in the target domain sample subset according to the similarity of each pedestrian picture in the source domain sample subset and each pedestrian picture in the target domain sample subset, and obtaining a layering loss function based on the layered pedestrian picture characteristics and the corresponding layering weights; and performing iterative optimization on the pedestrian re-recognition network model by traversing each source domain sample subset and each target domain sample subset based on the classification loss function, the sample invariance loss function and the hierarchical loss function.
And S130, identifying the pedestrian image to be identified by using the optimized pedestrian re-identification network model, and obtaining the image identical or similar to the pedestrian image to be identified.
Preferably, the layered loss function is obtained in particular by:
respectively inputting a source domain sample subset and a target domain sample subset into a residual network structure of a pedestrian re-identification network model to extract image features so as to obtain the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset, respectively storing the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset, and multiplying the features of each pedestrian picture in the source domain sample subset by the features of each pedestrian picture in the target domain sample subset to obtain corresponding similarity;
sorting pedestrian pictures in the target domain sample subset in a descending order based on the similarity, sequentially selecting a first preset number of pictures as a first layer of pictures according to the order, selecting a second preset number of pictures as a second layer of pictures, selecting the rest pictures as a third layer of pictures, and respectively setting layer weights of the three layers of pictures;
obtaining a layering loss function based on the characteristics of each pedestrian picture in the target domain sample subset and the layering weight corresponding to the layering to which the characteristics belong:
wherein L is SL Represents a hierarchical loss function, n t Representing the number, w, of pedestrian pictures in a target domain sample subset t,r Representing layer weights, r represents the sequence number, x of pedestrian pictures in the target domain sample subset according to similarity sequence t,i Representing a pedestrian picture with input sequence i when a target domain sample subset is input into a pedestrian re-recognition network model, f (x) t,i ) Representing pedestrian picture x t,i Is characterized by p (m|x t,i ) Representing pedestrian picture x t,i Probability of similarity to pedestrian pictures of all categories in all source domain sample subsets stored, R 1 [m]Features representing all categories of pedestrian pictures in all source domain sample subsets stored, R 1 [j]Representing the features of the jth pedestrian picture in all the stored source domain sample subsets, N s And representing the category number of the pedestrian pictures in all the stored source domain sample subsets, wherein beta is a temperature coefficient.
Preferably, the set layer weights of the three layers of pictures are as follows:
wherein b represents the number of pictures of the first layer, k 1 Representing the sum of the numbers of the first layer picture and the second layer picture, and p (l|i) represents the pedestrian picture x t,i Class I pedestrian pictures in all target domain sample subsets belonging to storageProbability of R 1 [l]Representing the characteristics of the class I pedestrian pictures in all the stored target domain sample subsets, N t And representing the category number of the pedestrian pictures in all the stored target domain sample subsets.
Compared with the prior art, the unsupervised domain self-adaptive pedestrian re-recognition system and method provided by the embodiment effectively combine the supervised pedestrian re-recognition and the unsupervised pedestrian re-recognition through the combination of the labeled source domain sample and the unlabeled target domain sample, namely layering the target domain sample through the similarity between the source domain sample and the target domain sample for the first time, so that the distinguishing degree of the pedestrian re-recognition network to the target domain sample is improved, the migration loss of the network is reduced, and the accuracy of the pedestrian re-recognition result of the pedestrian re-recognition network is further improved; on the other hand, the negative samples are layered and weighted by utilizing the similarity of the characteristics of the source domain samples and the target domain samples to obtain a layered loss function, the loss function enables the network to adaptively reduce the learning of the characteristics of the negative samples in the training of the pedestrian re-recognition network model, so that the migration loss of the network is reduced, on the basis, the classification loss function and the sample invariance loss function are combined to carry out iterative optimization on the pedestrian re-recognition network, and the precision of the pedestrian re-recognition network on the pedestrian re-recognition result can be greatly improved.
Those skilled in the art will appreciate that all or part of the flow of the methods of the embodiments described above may be accomplished by way of a computer program to instruct associated hardware, where the program may be stored on a computer readable storage medium. Wherein the computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory, etc.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention.
Claims (5)
1. An unsupervised domain adaptive pedestrian re-recognition system, comprising:
a data acquisition module for acquiring a source domain sample set comprising a plurality of source domain sample subsets and a target domain sample set comprising a plurality of target domain sample subsets;
the network model training module is used for obtaining a classification loss function and a sample invariance loss function of the pedestrian re-identification network model, sorting and layering pedestrian pictures in the target domain sample subset according to the similarity of each pedestrian picture in the source domain sample subset and each pedestrian picture in the target domain sample subset, and obtaining a layering loss function based on the characteristics of the layered pedestrian pictures and the corresponding layering weights; performing iterative optimization on the pedestrian re-recognition network model by traversing each source domain sample subset and each target domain sample subset based on the classification loss function, the sample invariance loss function and the hierarchical loss function; the pedestrian re-recognition network model comprises a residual network structure, a full connection layer and a Softmax normalization layer which are sequentially connected and correspond to a classification loss function, a target domain memory which is sequentially connected and correspond to a sample invariance loss function, and L 2 The standard layer, and source domain memory and similarity measurement shaft network structure which are sequentially connected and correspond to the layered loss function, and the residual error network structure is respectively connected with the full-connection layer, the target domain memory, the source domain memory and the similarity measurement shaft network structure;
the network model training module obtains a layered loss function specifically by the following method:
respectively inputting a source domain sample subset and a target domain sample subset into a residual network structure of a pedestrian re-identification network model to extract image features so as to obtain the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset, respectively storing the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset, and multiplying the features of each pedestrian picture in the source domain sample subset by the features of each pedestrian picture in the target domain sample subset to obtain corresponding similarity;
sorting pedestrian pictures in the target domain sample subset in a descending order based on the similarity, sequentially selecting a first preset number of pictures as a first layer of pictures according to the order, selecting a second preset number of pictures as a second layer of pictures, selecting the rest pictures as a third layer of pictures, and respectively setting layer weights of the three layers of pictures;
obtaining a layering loss function based on the characteristics of each pedestrian picture in the target domain sample subset and the layering weight corresponding to the layering to which the characteristics belong:
wherein L is SL Represents a hierarchical loss function, n t Representing the number, w, of pedestrian pictures in a target domain sample subset t,r Representing layer weights, r represents the sequence number, x of pedestrian pictures in the target domain sample subset according to similarity sequence t,i Representing a pedestrian picture with input sequence i when a target domain sample subset is input into a pedestrian re-recognition network model, f (x) t,i ) Representing pedestrian picture x t,i Is characterized by p (m|x t,i ) Representing pedestrian picture x t,i Probability of similarity to pedestrian pictures of all categories in all source domain sample subsets stored, R 1 [m]Features representing all categories of pedestrian pictures in all source domain sample subsets stored, R 1 [j]Representing the features of the jth pedestrian picture in all the stored source domain sample subsets, N s Representing the category number of pedestrian pictures in all the stored source domain sample subsets, wherein beta is a temperature coefficient;
the set layer weights of the three layers of pictures are as follows:
wherein b represents the number of pictures of the first layer, k 1 Representing the sum of the numbers of the first layer picture and the second layer picture, and p (l|i) represents the pedestrian picture x t, i probability of belonging to class I pedestrian picture in all stored target domain sample subsets, R 1 [l]Representing the characteristics of the class I pedestrian pictures in all the stored target domain sample subsets, N t Representing the category number of the pedestrian pictures in all the stored target domain sample subsets;
and the re-recognition module is used for recognizing the pedestrian image to be recognized by utilizing the optimized pedestrian re-recognition network model, and obtaining an image identical or similar to the pedestrian image to be recognized.
2. The unsupervised domain adaptive pedestrian re-recognition system according to claim 1, wherein the network model training module obtains the classification loss function by:
inputting the source domain sample subset into a residual network structure of a pedestrian re-recognition network model to extract image features so as to obtain the features of each pedestrian picture in the source domain sample subset;
sequentially inputting the characteristics of each pedestrian picture into a full-connection layer and a softmax regression layer of the pedestrian re-recognition network model, and carrying out characteristic dimension conversion and characteristic normalization;
classifying the loss function based on the characteristics of each pedestrian picture after dimension conversion and normalization by adopting the following formula;
wherein L is src Represents a classification loss function, n s Representing the number of pedestrian pictures in a source domain sample subset, x s,m Representation sourceMth pedestrian picture in domain sample subset, f (x s,m ) Representing pedestrian picture x s,m Is characterized by y s,m Representing pedestrian picture x in source domain sample subset s,m Category labels of (c), p (y) s,m |x s,m ) Representing pedestrian picture x s,m Belonging to category y s,m Probability of f (x) s,j ) Representing the features of the jth pedestrian picture in the source domain sample subset, N s Representing the number of categories of pedestrian pictures in all the stored source domain sample subsets.
3. The unsupervised domain adaptive pedestrian re-recognition system according to claim 1, wherein the network model training module obtains the sample invariance loss function by:
inputting the target domain sample subset into a residual network structure of a pedestrian re-recognition network model to extract image features so as to obtain the features of each pedestrian picture in the target domain sample subset, and carrying out L on the features of each pedestrian picture 2 Standardization treatment;
the sample invariance loss function is obtained according to the characteristics of each pedestrian picture after the normalization processing through the following formula:
wherein L is T Representing a sample invariance loss function, n t Representing the number of pedestrian pictures in a target domain sample subset, x t,i Representing a pedestrian picture with input sequence i when a target domain sample subset is input into a pedestrian re-recognition network model, f (x) t,i ) Representing pedestrian picture x t,i Is characterized by w i,l Representing pedestrian picture x t,i Weights belonging to class i pedestrian pictures, p (l|x t,i ) Representing pedestrian picture x t,i Belonging to the first classProbability of pedestrian picture, N t And representing the category number of the pedestrian pictures in all the stored target domain sample subsets, wherein beta is a temperature coefficient.
4. The unsupervised domain adaptive pedestrian re-recognition system according to claim 1, wherein the network model training module performs iterative optimization on the pedestrian re-recognition network model by:
obtaining a total loss function according to the classification loss function, the sample invariance loss function and the layering loss function:
L＝λ 1 L src +λ 2 L T +λ 3 L SL ，
wherein L represents the total loss function, L src Represents a class loss function, lambda 1 Weights representing classification loss functions, L T Representing the sample invariance loss function, lambda 2 Weights representing sample invariance loss functions, L SL Represents a layered loss function, lambda 3 Weights representing hierarchical loss functions;
and traversing each source domain sample subset and each target domain sample subset, and iteratively updating the total loss function until the variation of the total loss function value is smaller than a preset value, so as to finish the optimization of the pedestrian re-identification network model.
5. An unsupervised domain adaptive pedestrian re-recognition method, comprising:
obtaining a source domain sample set comprising a plurality of source domain sample subsets and a target domain sample set comprising a plurality of target domain sample subsets;
respectively obtaining a classification loss function and a sample invariance loss function of a pedestrian re-identification network model based on the source domain sample subset and the target domain sample subset, sorting and layering pedestrian pictures in the target domain sample subset according to the similarity of each pedestrian picture in the source domain sample subset and each pedestrian picture in the target domain sample subset, and obtaining a layering loss function based on the characteristics of the layered pedestrian pictures and the corresponding layering weights; performing iterative optimization on the pedestrian re-recognition network model by traversing each source domain sample subset and each target domain sample subset based on the classification loss function, the sample invariance loss function and the hierarchical loss function;
the pedestrian re-recognition network model comprises a residual network structure, a full connection layer and a Softmax normalization layer which are sequentially connected and correspond to a classification loss function, a target domain memory which is sequentially connected and correspond to a sample invariance loss function, and L 2 The standard layer, and source domain memory and similarity measurement shaft network structure which are sequentially connected and correspond to the layered loss function, and the residual error network structure is respectively connected with the full-connection layer, the target domain memory, the source domain memory and the similarity measurement shaft network structure;
the layered loss function is obtained specifically by:
respectively inputting a source domain sample subset and a target domain sample subset into a residual network structure of a pedestrian re-identification network model to extract image features so as to obtain the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset, respectively storing the features of each pedestrian picture in the source domain sample subset and the features of each pedestrian picture in the target domain sample subset, and multiplying the features of each pedestrian picture in the source domain sample subset by the features of each pedestrian picture in the target domain sample subset to obtain corresponding similarity;
sorting pedestrian pictures in the target domain sample subset in a descending order based on the similarity, sequentially selecting a first preset number of pictures as a first layer of pictures according to the order, selecting a second preset number of pictures as a second layer of pictures, selecting the rest pictures as a third layer of pictures, and respectively setting layer weights of the three layers of pictures;
obtaining a layering loss function based on the characteristics of each pedestrian picture in the target domain sample subset and the layering weight corresponding to the layering to which the characteristics belong:
wherein L is SL Represents a hierarchical loss function, n t Representing the number, w, of pedestrian pictures in a target domain sample subset t,r Representing layer weights, r represents the sequence number, x of pedestrian pictures in the target domain sample subset according to similarity sequence t,i Representing a pedestrian picture with input sequence i when a target domain sample subset is input into a pedestrian re-recognition network model, f (x) t,i ) Representing pedestrian picture x t,i Is characterized by p (m|x t,i ) Representing pedestrian picture x t,i Probability of similarity to pedestrian pictures of all categories in all source domain sample subsets stored, R 1 [m]Features representing all categories of pedestrian pictures in all source domain sample subsets stored, R 1 [j]Representing the features of the jth pedestrian picture in all the stored source domain sample subsets, N s Representing the category number of pedestrian pictures in all the stored source domain sample subsets, wherein beta is a temperature coefficient;
the set layer weights of the three layers of pictures are as follows:
wherein b represents the number of pictures of the first layer, k 1 Representing the sum of the numbers of the first layer picture and the second layer picture, and p (l|i) represents the pedestrian picture x t, i probability of belonging to class I pedestrian picture in all stored target domain sample subsets, R 1 [l]Representing the characteristics of the class I pedestrian pictures in all the stored target domain sample subsets, N t Representing the category number of the pedestrian pictures in all the stored target domain sample subsets;
and identifying the pedestrian image to be identified by using the optimized pedestrian re-identification network model, and obtaining the image which is the same as or similar to the pedestrian image to be identified.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110399589.7A CN113095229B (en) | 2021-04-14 | 2021-04-14 | Self-adaptive pedestrian re-identification system and method for unsupervised domain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110399589.7A CN113095229B (en) | 2021-04-14 | 2021-04-14 | Self-adaptive pedestrian re-identification system and method for unsupervised domain |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113095229A CN113095229A (en) | 2021-07-09 |
| CN113095229B true CN113095229B (en) | 2024-04-12 |
Family
ID=76677387
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110399589.7A Active CN113095229B (en) | 2021-04-14 | 2021-04-14 | Self-adaptive pedestrian re-identification system and method for unsupervised domain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113095229B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115510926B (en) * | 2022-11-23 | 2023-04-18 | 武汉理工大学 | Cross-machine type diesel engine combustion chamber fault diagnosis method and system |
| CN116405651B (en) * | 2023-06-09 | 2023-08-01 | 中国矿业大学(北京) | Multi-camera cross-view pedestrian data generation method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111126360A (en) * | 2019-11-15 | 2020-05-08 | 西安电子科技大学 | Cross-domain pedestrian re-identification method based on unsupervised combined multi-loss model |
| CN111401281A (en) * | 2020-03-23 | 2020-07-10 | 山东师范大学 | Unsupervised pedestrian re-identification method and system based on deep clustering and sample learning |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10074041B2 (en) * | 2015-04-17 | 2018-09-11 | Nec Corporation | Fine-grained image classification by exploring bipartite-graph labels |
| US11138469B2 (en) * | 2019-01-15 | 2021-10-05 | Naver Corporation | Training and using a convolutional neural network for person re-identification |
-
2021
- 2021-04-14 CN CN202110399589.7A patent/CN113095229B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111126360A (en) * | 2019-11-15 | 2020-05-08 | 西安电子科技大学 | Cross-domain pedestrian re-identification method based on unsupervised combined multi-loss model |
| CN111401281A (en) * | 2020-03-23 | 2020-07-10 | 山东师范大学 | Unsupervised pedestrian re-identification method and system based on deep clustering and sample learning |
Non-Patent Citations (3)
| Title |
|---|
| Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-identification;Zhun Zhong等;2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR);598-607 * |
| Unsupervised Person Re-Identification Based on Measurement Axis;Jiahan Li等;IEEE SIGNAL PROCESSING LETTERS;第28卷;第379-383页摘要、正文第Ⅱ-Ⅲ节，图1 * |
| Unsupervised person re-identification by hierarchical cluster and domain transfer;Suncheng Xiang等;Multimedia Tools and Applications;第79卷;19769-19786 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113095229A (en) | 2021-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111967294B (en) | Unsupervised domain self-adaptive pedestrian re-identification method | |
| CN110516095B (en) | Semantic migration-based weak supervision deep hash social image retrieval method and system | |
| CN109993236B (en) | One-shot Simese convolutional neural network-based small-sample Manchu matching method | |
| CN109446332B (en) | People reconciliation case classification system and method based on feature migration and self-adaptive learning | |
| CN114241273B (en) | Multi-modal image processing method and system based on Transformer network and hypersphere space learning | |
| CN110942091B (en) | Semi-supervised few-sample image classification method for searching reliable abnormal data center | |
| CN113190699A (en) | Remote sensing image retrieval method and device based on category-level semantic hash | |
| CN110866530A (en) | Character image recognition method and device and electronic equipment | |
| CN113095229B (en) | Self-adaptive pedestrian re-identification system and method for unsupervised domain | |
| CN113887580B (en) | Contrast type open set image recognition method and device considering multi-granularity correlation | |
| CN105894050A (en) | Multi-task learning based method for recognizing race and gender through human face image | |
| CN113065516B (en) | Sample separation-based unsupervised pedestrian re-identification system and method | |
| CN113326390B (en) | Image retrieval method based on depth feature consistent Hash algorithm | |
| CN114444600A (en) | Small sample image classification method based on memory enhanced prototype network | |
| CN111581368A (en) | Intelligent expert recommendation-oriented user image drawing method based on convolutional neural network | |
| CN108052959A (en) | A kind of method for improving deep learning picture recognition algorithm robustness | |
| CN112232395B (en) | Semi-supervised image classification method for generating countermeasure network based on joint training | |
| CN114842343A (en) | ViT-based aerial image identification method | |
| CN111582506A (en) | Multi-label learning method based on global and local label relation | |
| CN110442736B (en) | Semantic enhancer spatial cross-media retrieval method based on secondary discriminant analysis | |
| CN111191033A (en) | Open set classification method based on classification utility | |
| CN114357221A (en) | Self-supervision active learning method based on image classification | |
| CN113535928A (en) | Service discovery method and system of long-term and short-term memory network based on attention mechanism | |
| CN116910571A (en) | Open-domain adaptation method and system based on prototype comparison learning | |
| CN114610924A (en) | Commodity picture similarity matching search method and system based on multi-layer classification recognition model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Cheng Deqiang Inventor after: Kou Qiqi Inventor after: Li Jiahan Inventor after: Li Yunlong Inventor after: Zhang Haoxiang Inventor after: Han Chenggong Inventor after: Xu Jinyang Inventor after: Jiang Man Inventor after: Liu Ruihang Inventor before: Li Jiahan Inventor before: Li Yunlong Inventor before: Cheng Deqiang Inventor before: Kou Qiqi Inventor before: Zhang Haoxiang Inventor before: Han Chenggong Inventor before: Xu Jinyang Inventor before: Jiang Man Inventor before: Liu Ruihang |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240315 Address after: 221116 No. 1 Tongshan University Road, Xuzhou City, Jiangsu Province Applicant after: CHINA University OF MINING AND TECHNOLOGY Country or region after: China Address before: 221116 No. 1 Tongshan University Road, Xuzhou City, Jiangsu Province Applicant before: CHINA University OF MINING AND TECHNOLOGY Country or region before: China Applicant before: Jiangsu Huatu Mining Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |