CN114399724B - Pedestrian re-recognition method and device, electronic equipment and storage medium - Google Patents
Pedestrian re-recognition method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114399724B CN114399724B CN202111464319.6A CN202111464319A CN114399724B CN 114399724 B CN114399724 B CN 114399724B CN 202111464319 A CN202111464319 A CN 202111464319A CN 114399724 B CN114399724 B CN 114399724B
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- data
- recognition model
- training
- recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000012549 training Methods 0.000 claims abstract description 84
- 238000002372 labelling Methods 0.000 claims abstract description 30
- 238000005070 sampling Methods 0.000 claims abstract description 18
- 238000001514 detection method Methods 0.000 claims abstract description 9
- 238000012163 sequencing technique Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 7
- 230000000694 effects Effects 0.000 abstract description 17
- 238000005516 engineering process Methods 0.000 abstract description 5
- 238000012545 processing Methods 0.000 abstract description 3
- 238000004891 communication Methods 0.000 description 8
- 238000000605 extraction Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000001174 ascending effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The application relates to the technical field of data processing, in particular to a pedestrian re-identification method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: pedestrian detection is carried out on the acquired data, a non-labeling pedestrian data set is generated, and a first pedestrian re-recognition model is generated after non-supervision training is carried out; when the first pedestrian re-recognition model does not meet the target performance condition, extracting pedestrian characteristics of the unmarked pedestrian data set by using the first pedestrian re-recognition model, generating a first pedestrian characteristic set, carrying out importance sampling and marking, carrying out semi-supervised training according to marked data and unmarked data in the first pedestrian characteristic set, generating a second pedestrian re-recognition model, and carrying out pedestrian recognition by using the second pedestrian re-recognition model meeting the target performance condition until the second pedestrian re-recognition model meets the target performance condition. Therefore, the problems of high data labeling cost, poor recognition effect and the like required by training the pedestrian re-recognition model in the related technology are solved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a pedestrian re-recognition method, device, electronic apparatus, and storage medium.
Background
Pedestrian re-recognition refers to matching pedestrians between pedestrian images or video clips captured by multiple cameras that do not overlap each other. Compared with the traditional tasks such as image classification and the like, only a single picture needs to be marked, the marking data needed by the pedestrian re-recognition model are paired pictures or video clips, namely whether pedestrians in the two pictures or video clips are the same or not, so that the data marking cost needed by training the pedestrian re-recognition model is greatly increased.
Disclosure of Invention
The application provides a weak supervision pedestrian re-recognition method and device based on active learning, electronic equipment and a storage medium, and aims to solve the problems that data labeling cost required by training a pedestrian re-recognition model in the related technology is high, recognition effect is poor and the like.
An embodiment of a first aspect of the present application provides a weak supervision pedestrian re-identification method based on active learning, including the following steps: pedestrian detection is carried out on the acquired data, and a non-labeling pedestrian data set is generated; performing non-supervision training according to the non-marked pedestrian data set to generate a first pedestrian re-recognition model, and performing pedestrian feature extraction on the non-marked pedestrian data set by using the first pedestrian re-recognition model when the first pedestrian re-recognition model does not meet a target performance condition to generate a first pedestrian feature set; and importance sampling is carried out on the first pedestrian feature set, the sampled data are marked, semi-supervised training is carried out according to marked data and unmarked data in the first pedestrian feature set, a second pedestrian re-recognition model is generated, and the second pedestrian re-recognition model meeting the target performance condition is utilized to carry out pedestrian recognition until the second pedestrian re-recognition model meets the target performance condition.
Further, the performing semi-supervised training according to the marked data and the unmarked data in the first pedestrian feature set, and generating a second pedestrian re-recognition model includes: clustering the first pedestrian feature set by taking the labeling data as constraint conditions to generate a first pseudo-tag pedestrian data set; and performing supervision training by using the first pseudo tag pedestrian data set to obtain the second pedestrian re-recognition model.
Further, the importance sampling of the first pedestrian feature set, and the labeling of the sampled data, includes: calculating the distance between any two data pairs formed by the data x i,xj in the first pedestrian feature set; selecting data x j which is the same as and farthest from x i pseudo tag y i, and selecting data x k which is different from and closest to x i pseudo tag y i, to form a triplet (x i,xj,xk); calculating the entropy of the triples (x i,xj,xk), sequencing the entropy of all the triples according to a preset sequencing rule, and selecting a preset number of triples from the big to the small according to the entropy; the annotation data is composed according to (x i,xj) and (x i,xk) in each triplet selected.
Further, performing unsupervised training according to the unlabeled pedestrian data set, and generating a first pedestrian re-recognition model, including: extracting pedestrian characteristics of the unmarked pedestrian data set by using a preset model to generate a second pedestrian characteristic set; clustering the second pedestrian feature set by using a density-based clustering algorithm to generate a second pseudo tag pedestrian data set; and performing supervision training by using the second pseudo tag pedestrian data set to obtain the first pedestrian re-recognition model.
Further, after generating the first pedestrian re-recognition model, further comprising: judging whether the first pedestrian re-recognition model meets a target performance condition or not; and if so, carrying out pedestrian recognition by using the first pedestrian re-recognition model.
An embodiment of a second aspect of the present application provides a weak supervision pedestrian re-identification device based on active learning, including: the detection module is used for detecting pedestrians on the acquired data and generating a non-marked pedestrian data set; the first training module is used for performing non-supervision training according to the non-labeling pedestrian data set to generate a first pedestrian re-recognition model; and the second training module is used for extracting the pedestrian characteristics of the unmarked pedestrian data set by using the first pedestrian re-recognition model when the first pedestrian re-recognition model does not meet the target performance condition, generating a first pedestrian characteristic set, sampling the importance of the first pedestrian characteristic set, marking the sampled data, performing semi-supervised training according to the marked data and the unmarked data in the first pedestrian characteristic set, generating a second pedestrian re-recognition model, and performing pedestrian recognition by using the second pedestrian re-recognition model meeting the target performance condition until the second pedestrian re-recognition model meets the target performance condition.
Further, the second training module is configured to cluster the first pedestrian feature set with the labeling data as a constraint condition, so as to generate a first pseudo-tag pedestrian data set, and perform supervised training by using the first pseudo-tag pedestrian data set, so as to obtain the second pedestrian re-recognition model.
Further, the second training module is further configured to calculate a distance between a pair of data formed by any two data x i,xj in the first pedestrian feature set; selecting data x j which is the same as and farthest from x i pseudo tag y i, and selecting data x k which is different from and closest to x i pseudo tag y i, to form a triplet (x i,xj,xk); calculating the entropy of the triples (x i,xj,xk), sequencing the entropy of all the triples according to a preset sequencing rule, and selecting a preset number of triples from the big to the small according to the entropy; the annotation data is composed according to (x i,xj) and (x i,xk) in each triplet selected.
Further, the first training module is configured to perform pedestrian feature extraction on the unlabeled pedestrian data set by using a preset model, generate a second pedestrian feature set, cluster the second pedestrian feature set by using a density-based clustering algorithm, generate a second pseudo-tag pedestrian data set, and perform supervised training by using the second pseudo-tag pedestrian data set to obtain the first pedestrian re-recognition model.
Further, the method further comprises the following steps: and the judging module is used for judging whether the first pedestrian re-recognition model meets the target performance condition after the first pedestrian re-recognition model is generated, and if so, carrying out pedestrian recognition by using the first pedestrian re-recognition model.
An embodiment of a third aspect of the present application provides an electronic device, including: the system comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to realize the weak supervision pedestrian re-identification method based on active learning as described in the embodiment.
An embodiment of a fourth aspect of the present application provides a computer-readable storage medium having stored thereon a computer program for execution by a processor for implementing the weak supervision pedestrian re-recognition method based on active learning as described in the above embodiment.
Therefore, the application has at least the following beneficial effects:
the pedestrian data which is most useful for improving the effect of the current model is marked through importance sampling, the required data marking quantity is reduced, the marking cost is reduced, meanwhile, marked and unmarked pedestrian data can be fully utilized, more accurate pseudo labels are obtained, and the recognition effect of the pedestrian recognition model is improved. Therefore, the technical problems of high data labeling cost, poor recognition effect and the like required by training the pedestrian re-recognition model in the related technology are solved.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
Fig. 1 is a schematic flow chart of a weak supervision pedestrian re-identification method based on active learning according to an embodiment of the application;
FIG. 2 is a flow chart of a weak supervision pedestrian re-identification method based on active learning according to an embodiment of the application;
FIG. 3 is an exemplary diagram of a weak supervision pedestrian re-identification device based on active learning in accordance with an embodiment of the present application;
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.
The scheme for training the pedestrian re-recognition model in the related art is as follows:
(1) Unsupervised learning scheme: the self-supervision method mainly based on the cluster-pseudo tag-model update is iterated continuously to obtain the final pedestrian re-identification model without marking data. However, since no labeling data is used, the quality of the pseudo tag formed by clustering is difficult to ensure, so that the model effect is poor and the distance supervision learning gap is large.
(2) Sampling scheme based on similarity measure: and selecting the pedestrian picture pair most likely to be the positive example, forming marking data after manual verification, and performing model training on the marking data by adopting a supervised learning method. However, most of the selected sample pairs to be marked are simple sample pairs, so that the effect of marking data on a model is limited, the waste of marking cost is increased, meanwhile, only marking data is adopted for training, and effective information in unmarked data cannot be fully mined, so that the model identification effect is poor.
(3) Sampling scheme based on confidence measures: selecting the least credible class cluster in class clusters formed by the unsupervised clustering, pairwise pairing all pedestrian pictures, forming a new class cluster after manual labeling, and performing model training on the pseudo-label data set by adopting a supervised learning method. However, the scheme needs to label all sample pairs in the pseudo tag, so that the data size is large and the labeling cost is high; meanwhile, sample pairs among different pseudo tags cannot be effectively processed, so that the phenomenon that different pictures of the same pedestrian are scattered into a plurality of pseudo classes is generated, the data marking quality is low, and the model training effect is limited.
Therefore, the embodiment of the application provides a weak supervision pedestrian re-recognition method, a weak supervision pedestrian re-recognition device, electronic equipment and a storage medium based on active learning, which can select a pedestrian data sample pair most likely to improve a model effect to carry out manual annotation by the active learning method, and carry out model training on part of annotated pedestrian data and a large amount of unlabeled pedestrian data by combining with the weak supervision pedestrian re-recognition model training method, so that the data annotation cost required by training a high-quality pedestrian re-recognition model is reduced.
The weak supervision pedestrian re-identification method, the weak supervision pedestrian re-identification device, the electronic equipment and the storage medium based on the active learning in the embodiment of the application are described below with reference to the accompanying drawings. Aiming at the problems of high data labeling cost, poor recognition effect and the like required by training a pedestrian re-recognition model in the related art in the background art, the application provides a weak supervision pedestrian re-recognition method based on active learning. Therefore, the technical problems of high data labeling cost, poor recognition effect and the like required by training the pedestrian re-recognition model in the related technology are solved.
Specifically, fig. 1 is a schematic flow chart of a weak supervision pedestrian re-identification method based on active learning according to an embodiment of the present application.
As shown in fig. 1, the weak supervision pedestrian re-identification method based on active learning comprises the following steps:
In step S101, pedestrian detection is performed on the acquired data, and a non-labeling pedestrian data set is generated.
The collected data may include data such as video, which is not limited in particular.
Taking video data as an example, it can be appreciated that the embodiment of the application can perform data acquisition through video data acquisition equipment such as a monitoring camera and the like, and perform pedestrian detection on the acquired video to form a non-labeling pedestrian data set.
In the present embodiment, the data in the unlabeled pedestrian data set may be pedestrian picture data or video clip data or the like, and pedestrian picture data is taken as an example in the following embodiments.
In step S102, an unsupervised training is performed according to the unlabeled pedestrian data set, and a first pedestrian re-recognition model is generated.
It will be appreciated that, as shown in fig. 2, after generating the unlabeled pedestrian dataset, the embodiment of the present application may perform an unsupervised pedestrian re-recognition model training on the unlabeled pedestrian dataset to generate the first pedestrian re-recognition model M.
In this embodiment, performing unsupervised training according to an unlabeled pedestrian data set, generating a first pedestrian re-recognition model includes: extracting pedestrian characteristics of the unlabeled pedestrian data set by using a preset model to generate a second pedestrian characteristic set; clustering the second pedestrian feature set by using a density-based clustering algorithm to generate a second pseudo tag pedestrian data set; and performing supervision training by using the second pseudo-tag pedestrian data set to obtain a first pedestrian re-recognition model.
The preset model may be ResNet models, which may be specifically selected by those skilled in the art according to the training requirements, and is not specifically limited.
Specifically, performing an unsupervised pedestrian re-recognition model training on the unlabeled pedestrian dataset includes:
(1) Pre-trained model (e.g., resNet) using ImageNet dataset for unlabeled pedestrian picture dataset Performing special extraction to obtain pedestrian feature setWherein n is the number of pedestrian pictures;
(2) Clustering pedestrian feature sets by using a density-based clustering algorithm DBSCAN to form a pseudo-tag pedestrian data set Wherein y i is a pseudo tag corresponding to the i-th pedestrian picture;
(3) And performing supervision training by using the pseudo tag data set to obtain a pedestrian re-identification model M.
In this embodiment, after generating the first pedestrian re-recognition model, further includes: judging whether the first pedestrian re-identification model meets the target performance condition or not; if so, the first pedestrian re-recognition model is utilized for pedestrian recognition.
It will be understood that, as shown in fig. 2, after the first pedestrian re-recognition model M is generated, it is determined whether the model M meets the performance requirement, and if so, the flow is ended; otherwise, the pedestrian feature extraction step is entered, i.e., step S103 is executed.
And step S103, when the first pedestrian re-recognition model does not meet the target performance condition, extracting the pedestrian characteristics of the unmarked pedestrian data set by using the first pedestrian re-recognition model, and generating a first pedestrian characteristic set.
It will be appreciated that as shown in FIG. 2, when the first pedestrian re-recognition model does not meet the target performance condition, embodiments of the present application may use the model M to pair the pedestrian datasetExtracting features to obtain pedestrian feature setThe non-supervision training belongs to one of weak supervision pedestrian re-identification model training methods.
In step S104, importance sampling is performed on the first pedestrian feature set, the sampled data is labeled, semi-supervised training is performed according to the labeled data and unlabeled data in the first pedestrian feature set, and a second pedestrian re-recognition model is generated, until the second pedestrian re-recognition model meets the target performance condition, and pedestrian recognition is performed by using the second pedestrian re-recognition model meeting the target performance condition.
It can be understood that, as shown in fig. 2, the embodiment of the present application may adopt an active learning data sampling strategy, select a pedestrian data sample pair most likely to promote a model effect to perform manual labeling, combine with a weak supervision pedestrian re-recognition model training method, perform model training on part of labeled pedestrian data and a large number of unlabeled pedestrian data to obtain a second pedestrian re-recognition model M, and execute step S102 after replacing the model M with the pedestrian re-recognition model M, so as to reduce the data labeling cost required for training the high-quality pedestrian re-recognition model. The data sampling strategy of active learning comprises importance sampling, and the weak supervision pedestrian re-recognition model training method comprises a semi-supervision pedestrian re-recognition model training method.
In this embodiment, importance sampling is performed on the first pedestrian feature set, and labeling is performed on sampled data, including: calculating the distance between any two data pairs formed by the data x i,xj in the first pedestrian feature set; selecting data x j which is the same as and farthest from x i pseudo tag y i, and selecting data x k which is different from and closest to x i pseudo tag y i, to form a triplet (x i,xj,xk); calculating the entropy of the triples (x i,xj,xk), sequencing the entropy of all the triples according to a preset sequencing rule, and selecting a preset number of triples according to the entropy from big to small; labeling data is composed according to (x i,xj) and (x i,xk) in each triplet selected.
The preset ordering rule may be ascending order or descending order, which is not limited specifically; the preset number may be specifically set according to actual requirements, which is not specifically limited.
Specifically, the importance of every two image pairs in the pedestrian data set is calculated, and the pedestrian image pair set needing manual labeling is returned according to the importance, wherein the method specifically comprises the following steps:
(1) Using pedestrian feature sets Calculating the distance d ij between the picture pairs formed by any two pedestrian pictures x i,xj;
(2) For each pedestrian picture x i, selecting a pedestrian picture x j which is the same as x i pseudo tag y i and has the farthest distance, and selecting a pedestrian picture x k which is different from x i pseudo tag y i and has the nearest distance to form a triplet (x i,xj,xk);
(3) For each triplet (x i,xj,xk), the corresponding entropy is calculated as E ijk＝-pijk*log pijk-(1-pijk)*log(1-pijk), where
(4) Ordering all triples which are not marked at present according to entropy, and selecting the maximum entropyAnd taking (x i,xj) and (x i,xk) in each selected triplet as pedestrian picture pairs to be marked, and forming K pedestrian picture pairs to be marked, wherein K is the number of the pedestrian picture pairs to be marked.
In this embodiment, semi-supervised training is performed according to the marked data and the unmarked data in the first pedestrian feature set, and a second pedestrian re-recognition model is generated when the target performance condition is satisfied, including: clustering the first pedestrian feature set by taking the labeling data as constraint conditions to generate a first pseudo-tag pedestrian data set; and performing supervision training by using the first pseudo-tag pedestrian data set to obtain a second pedestrian re-recognition model.
Specifically, the training method of the semi-supervised pedestrian re-recognition model is adopted on the partially-marked pedestrian data set, and the specific steps are as follows:
(1) Using model M for all pedestrian pictures Performing special extraction to obtain pedestrian feature set
(2) Clustering the pedestrian feature set by using a density-based constrained clustering algorithm C-DBSCAN, wherein the marked pedestrian pictures are used as constraint information input by the algorithm, so as to form a pseudo-tag pedestrian data setWherein y i is a pseudo tag corresponding to the i-th pedestrian picture;
(3) And performing supervision training by using the pseudo tag data set to obtain a pedestrian re-identification model M'.
According to the weak supervision pedestrian re-identification method based on the active learning, which is provided by the embodiment of the application, based on the data sampling strategy of the active learning, pedestrian data which is most useful for improving the effect of the current model can be selected for manual marking, and compared with the selection strategy in the related art, the required data marking quantity can be greatly reduced, and the marking cost is reduced; the weak supervision pedestrian re-identification model training method can fully utilize marked and unmarked pedestrian data at the same time, and compared with the training method in the related art, the training method can obtain more accurate pseudo labels, so that the model effect is improved; meanwhile, the data labeling cost required by training the high-quality pedestrian re-recognition model can be effectively reduced.
Next, a weak supervision pedestrian re-identification device based on active learning according to an embodiment of the present application will be described with reference to the accompanying drawings.
Fig. 3 is a block diagram of a weak supervision pedestrian re-identification device based on active learning according to an embodiment of the application.
As shown in fig. 3, the weak supervision pedestrian re-recognition device 10 based on active learning includes: the device comprises a detection module 100, a first training module 200 and a second training module 300.
The detection module 100 is used for detecting pedestrians on the collected data and generating a non-marked pedestrian data set; the first training module 200 is configured to perform unsupervised training according to the unlabeled pedestrian data set, and generate a first pedestrian re-recognition model; and the second training module is used for extracting the pedestrian characteristics of the unmarked pedestrian data set by using the first pedestrian re-recognition model when the first pedestrian re-recognition model does not meet the target performance condition, generating a first pedestrian characteristic set, sampling the importance of the first pedestrian characteristic set, marking the sampled data, performing semi-supervised training according to the marked data and the unmarked data in the first pedestrian characteristic set, generating a second pedestrian re-recognition model, and performing pedestrian recognition by using the second pedestrian re-recognition model meeting the target performance condition until the second pedestrian re-recognition model meets the target performance condition.
Further, the second training module 300 is configured to cluster the first pedestrian feature set with the labeling data as a constraint condition, so as to generate a first pseudo-tag pedestrian data set, and perform supervised training by using the first pseudo-tag pedestrian data set, so as to obtain a second pedestrian re-recognition model.
Further, the second training module 300 is further configured to calculate a distance between any two data pairs formed by the data x i,xj in the first pedestrian feature set; selecting data x j which is the same as and farthest from x i pseudo tag y i, and selecting data x k which is different from and closest to x i pseudo tag y i, to form a triplet (x i,xj,xk); calculating the entropy of the triples (x i,xj,xk), sequencing the entropy of all the triples according to a preset sequencing rule, and selecting a preset number of triples according to the entropy from big to small; labeling data is composed according to (x i,xj) and (x i,xk) in each triplet selected.
Further, the first training module 200 is configured to perform pedestrian feature extraction on the unlabeled pedestrian data set by using a preset model, generate a second pedestrian feature set, cluster the second pedestrian feature set by using a density-based clustering algorithm, generate a second pseudo-tag pedestrian data set, and perform supervised training by using the second pseudo-tag pedestrian data set to obtain a first pedestrian re-recognition model.
Further, the apparatus 10 according to the embodiment of the present application further includes: and a judging module. The judging module is used for judging whether the first pedestrian re-recognition model meets the target performance condition after the first pedestrian re-recognition model is generated, and if so, the first pedestrian re-recognition model is used for pedestrian recognition.
It should be noted that the foregoing explanation of the embodiment of the weak supervision pedestrian re-recognition method based on active learning is also applicable to the weak supervision pedestrian re-recognition device based on active learning of this embodiment, and will not be repeated here.
According to the weak supervision pedestrian re-identification device based on the active learning, which is provided by the embodiment of the application, based on the data sampling strategy of the active learning, pedestrian data which is most useful for improving the effect of the current model can be selected for manual marking, and compared with the selection strategy in the related art, the required data marking quantity can be greatly reduced, and the marking cost is reduced; the weak supervision pedestrian re-identification model training method can fully utilize marked and unmarked pedestrian data at the same time, and compared with the training method in the related art, the training method can obtain more accurate pseudo labels, so that the model effect is improved; meanwhile, the data labeling cost required by training the high-quality pedestrian re-recognition model can be effectively reduced.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device may include:
Memory 401, processor 402, and a computer program stored on memory 401 and executable on processor 402.
The processor 402 implements the weak supervised pedestrian re-recognition method based on active learning provided in the above embodiment when executing a program.
Further, the electronic device further includes:
A communication interface 403 for communication between the memory 401 and the processor 402.
A memory 401 for storing a computer program executable on the processor 402.
Memory 401 may include high-speed RAM (Random Access Memory ) memory, and may also include non-volatile memory, such as at least one disk memory.
If the memory 401, the processor 402, and the communication interface 403 are implemented independently, the communication interface 403, the memory 401, and the processor 402 may be connected to each other by a bus and perform communication with each other. The bus may be an ISA (Industry Standard Architecture ) bus, a PCI (PERIPHERAL COMPONENT, external device interconnect) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 4, but not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 401, the processor 402, and the communication interface 403 are integrated on a chip, the memory 401, the processor 402, and the communication interface 403 may perform communication with each other through internal interfaces.
The processor 402 may be a CPU (Central Processing Unit ) or an ASIC (Application SPECIFIC INTEGRATED Circuit, application specific integrated Circuit) or one or more integrated circuits configured to implement embodiments of the present application.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, implements the weak supervision pedestrian re-identification method based on active learning.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, "N" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more N executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable gate arrays, field programmable gate arrays, and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
Claims (8)
1. The weak supervision pedestrian re-identification method based on active learning is characterized by comprising the following steps of:
Pedestrian detection is carried out on the acquired data, and a non-labeling pedestrian data set is generated;
Performing unsupervised training according to the unlabeled pedestrian data set to generate a first pedestrian re-recognition model, performing unsupervised training according to the unlabeled pedestrian data set to generate the first pedestrian re-recognition model, including: extracting pedestrian characteristics of the unmarked pedestrian data set by using a preset model to generate a second pedestrian characteristic set; clustering the second pedestrian feature set by using a density-based clustering algorithm to generate a second pseudo tag pedestrian data set; performing supervision training by using the second pseudo tag pedestrian data set to obtain the first pedestrian re-recognition model;
When the first pedestrian re-recognition model does not meet the target performance condition, extracting pedestrian characteristics of the unmarked pedestrian data set by using the first pedestrian re-recognition model to generate a first pedestrian characteristic set;
Performing importance sampling on the first pedestrian feature set, marking the sampled data, performing semi-supervised training according to marked data and unmarked data in the first pedestrian feature set, and generating a second pedestrian re-recognition model, and performing pedestrian recognition by using the second pedestrian re-recognition model meeting the target performance condition until the second pedestrian re-recognition model meets the target performance condition; the importance sampling of the first pedestrian feature set, and the labeling of the sampled data, includes: calculating the distance between any two data pairs formed by the data x i,xj in the first pedestrian feature set; selecting data x j which is the same as and farthest from x i pseudo tag y i, and selecting data x k which is different from and closest to x i pseudo tag y i, to form a triplet (x i,xj,xk); calculating the entropy of the triples (x i,xj,xk), sequencing the entropy of all the triples according to a preset sequencing rule, and selecting a preset number of triples from the big to the small according to the entropy; the annotation data is composed according to (x i,xj) and (x i,xk) in each triplet selected.
2. The method of claim 1, wherein the performing semi-supervised training based on the marked data and the unmarked data in the first set of pedestrian features to generate a second pedestrian re-recognition model comprises:
Clustering the first pedestrian feature set by taking the labeling data as constraint conditions to generate a first pseudo-tag pedestrian data set;
And performing supervision training by using the first pseudo tag pedestrian data set to obtain the second pedestrian re-recognition model.
3. The method of any of claims 1-2, further comprising, after generating the first pedestrian re-recognition model:
judging whether the first pedestrian re-recognition model meets a target performance condition or not;
And if so, carrying out pedestrian recognition by using the first pedestrian re-recognition model.
4. The utility model provides a weak supervision pedestrian re-identification device based on initiative study which characterized in that includes:
The detection module is used for detecting pedestrians on the acquired data and generating a non-marked pedestrian data set;
The first training module is used for performing non-supervision training according to the non-marked pedestrian data set to generate a first pedestrian re-recognition model, the first training module is used for extracting pedestrian characteristics of the non-marked pedestrian data set by using a preset model to generate a second pedestrian characteristic set, clustering the second pedestrian characteristic set by using a density-based clustering algorithm to generate a second pseudo-tag pedestrian data set, and performing supervision training by using the second pseudo-tag pedestrian data set to obtain the first pedestrian re-recognition model;
The second training module is used for extracting pedestrian characteristics of the unmarked pedestrian data set by utilizing the first pedestrian re-recognition model when the first pedestrian re-recognition model does not meet the target performance condition, generating a first pedestrian characteristic set, sampling importance of the first pedestrian characteristic set, marking sampled data, performing semi-supervised training according to marked data and unmarked data in the first pedestrian characteristic set, generating a second pedestrian re-recognition model, and performing pedestrian recognition by utilizing the second pedestrian re-recognition model meeting the target performance condition until the second pedestrian re-recognition model meets the target performance condition, wherein the second training module is further used for calculating the distance between any two data x i,xj data pairs in the first pedestrian characteristic set; selecting data x j which is the same as and farthest from x i pseudo tag y i, and selecting data x k which is different from and closest to x i pseudo tag y i, to form a triplet (x i,xj,xk); calculating the entropy of the triples (x i,xj,xk), sequencing the entropy of all the triples according to a preset sequencing rule, and selecting a preset number of triples from the big to the small according to the entropy; the annotation data is composed according to (x i,xj) and (x i,xk) in each triplet selected.
5. The apparatus of claim 4, wherein the second training module is configured to cluster the first pedestrian feature set using the labeling data as a constraint condition to generate a first pseudo-tag pedestrian data set, and to perform supervised training using the first pseudo-tag pedestrian data set to obtain the second pedestrian re-recognition model.
6. The apparatus according to any one of claims 4-5, further comprising:
and the judging module is used for judging whether the first pedestrian re-recognition model meets the target performance condition after the first pedestrian re-recognition model is generated, and if so, utilizing the first pedestrian re-recognition model to conduct pedestrian recognition.
7. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the weak supervised pedestrian re-recognition method based on active learning of any of claims 1-3.
8. A computer-readable storage medium, on which a computer program is stored, characterized in that the program is executed by a processor for implementing the weak supervision pedestrian re-identification method based on active learning as claimed in any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111464319.6A CN114399724B (en) | 2021-12-03 | 2021-12-03 | Pedestrian re-recognition method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111464319.6A CN114399724B (en) | 2021-12-03 | 2021-12-03 | Pedestrian re-recognition method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114399724A CN114399724A (en) | 2022-04-26 |
| CN114399724B true CN114399724B (en) | 2024-06-28 |
Family
ID=81225609
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111464319.6A Active CN114399724B (en) | 2021-12-03 | 2021-12-03 | Pedestrian re-recognition method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114399724B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111967429A (en) * | 2020-08-28 | 2020-11-20 | 清华大学 | Pedestrian re-recognition model training method and device based on active learning |
| CN112069929A (en) * | 2020-08-20 | 2020-12-11 | 之江实验室 | Unsupervised pedestrian re-identification method and device, electronic equipment and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11537817B2 (en) * | 2018-10-18 | 2022-12-27 | Deepnorth Inc. | Semi-supervised person re-identification using multi-view clustering |
| CN112836675B (en) * | 2021-03-01 | 2023-06-23 | 中山大学 | Unsupervised pedestrian re-identification method and system for generating pseudo tags based on clusters |
| CN113191183A (en) * | 2021-03-02 | 2021-07-30 | 清华大学 | Unsupervised domain false label correction method and unsupervised domain false label correction device in personnel re-identification |
-
2021
- 2021-12-03 CN CN202111464319.6A patent/CN114399724B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112069929A (en) * | 2020-08-20 | 2020-12-11 | 之江实验室 | Unsupervised pedestrian re-identification method and device, electronic equipment and storage medium |
| CN111967429A (en) * | 2020-08-28 | 2020-11-20 | 清华大学 | Pedestrian re-recognition model training method and device based on active learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114399724A (en) | 2022-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021238281A1 (en) | Neural network training method, image classification system, and related device | |
| CN107133569B (en) | Monitoring video multi-granularity labeling method based on generalized multi-label learning | |
| Yi et al. | ASSD: Attentive single shot multibox detector | |
| CN112232231B (en) | Pedestrian attribute identification method, system, computer equipment and storage medium | |
| Liu et al. | Weakly-supervised dual clustering for image semantic segmentation | |
| Yuan et al. | Congested scene classification via efficient unsupervised feature learning and density estimation | |
| CN112232293A (en) | Image processing model training method, image processing method and related equipment | |
| CN109285105A (en) | Method of detecting watermarks, device, computer equipment and storage medium | |
| Wu et al. | Vehicle re-identification in still images: Application of semi-supervised learning and re-ranking | |
| Tian et al. | Video object detection for tractability with deep learning method | |
| CN115294150A (en) | Image processing method and terminal equipment | |
| Feng et al. | Beyond tag relevance: integrating visual attention model and multi-instance learning for tag saliency ranking | |
| Qu et al. | Improved YOLOv5-based for small traffic sign detection under complex weather | |
| CN116229406B (en) | Lane line detection method, system, electronic equipment and storage medium | |
| Liao et al. | Multi-scale saliency features fusion model for person re-identification | |
| Sun et al. | SES-YOLOv8n: automatic driving object detection algorithm based on improved YOLOv8 | |
| Qi et al. | Unstructured road detection via combining the model‐based and feature‐based methods | |
| Guo et al. | UDTIRI: An online open-source intelligent road inspection benchmark suite | |
| CN117115565A (en) | Autonomous perception-based image classification method and device and intelligent terminal | |
| CN114399724B (en) | Pedestrian re-recognition method and device, electronic equipment and storage medium | |
| Mallick et al. | Video retrieval using salient foreground region of motion vector based extracted keyframes and spatial pyramid matching | |
| Gao et al. | Segmentation-free vehicle license plate recognition using CNN | |
| CN115935015A (en) | Model training method and device and graph classification method and device | |
| CN115393673A (en) | Training method of object recognition model, object recognition method and electronic equipment | |
| CN112784691B (en) | Target detection model training method, target detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |