CN114882372A - Target detection method and device - Google Patents

Target detection method and device Download PDF

Info

Publication number
CN114882372A
CN114882372A CN202210807357.5A CN202210807357A CN114882372A CN 114882372 A CN114882372 A CN 114882372A CN 202210807357 A CN202210807357 A CN 202210807357A CN 114882372 A CN114882372 A CN 114882372A
Authority
CN
China
Prior art keywords
feature map
low
training
dimensional feature
detection model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202210807357.5A
Other languages
Chinese (zh)
Inventor
沈孔怀
朱亚伦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Dahua Technology Co Ltd
Original Assignee
Zhejiang Dahua Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Dahua Technology Co Ltd filed Critical Zhejiang Dahua Technology Co Ltd
Priority to CN202210807357.5A priority Critical patent/CN114882372A/en
Publication of CN114882372A publication Critical patent/CN114882372A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method and equipment for detecting a target, which are used for improving the detection performance of a detection model in an actual scene while not increasing the labeled training data volume. The method comprises the following steps: acquiring a target image to be detected; inputting the target image into a trained detection model, and outputting the position of a target object in the target image; the detection model is obtained by training a labeled first sample set and an unlabeled second sample set, a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set are extracted during each training, the first low-dimensional feature map and the second low-dimensional feature map are fused to obtain a fused feature map, and the position of a target object labeled by the fused feature map and the first sample image is used for training.

Description

Target detection method and device
Technical Field
The invention relates to the technical field of computer graphics and image processing, in particular to a method and equipment for target detection.
Background
The current method based on deep learning is generally a data-driven method, however, the real environment is complex and diverse, and if the data distribution of the actual scene and the scene of the current training set is obviously different, the performance of the detection model may be significantly reduced. In order to ensure the performance of the algorithm, data under actual scenes needs to be covered during training, but the cost of data labeling is "expensive", and a large amount of labor and time are needed.
Therefore, under the condition that the data distribution of the actual scene and the scene where the training set is located has a large difference, how to ensure that the detection performance of the detection model in the actual scene is improved while the amount of the labeled training data is not increased becomes a technical problem to be solved urgently at present.
Disclosure of Invention
The invention provides a method and equipment for detecting a target, which are used for improving the detection performance of a detection model in an actual scene while not increasing the labeled training data volume.
In a first aspect, an embodiment of the present invention provides a method for target detection, including:
acquiring a target image to be detected;
inputting the target image into a trained detection model, and outputting the position of a target object in the target image;
the detection model is obtained by training a labeled first sample set and an unlabeled second sample set, a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set are extracted during each training, the first low-dimensional feature map and the second low-dimensional feature map are fused to obtain a fused feature map, and the positions of target objects labeled by the fused feature map and the first sample image are used for training; wherein the first sample set and the second sample set both contain target objects, the second sample set containing different scene characteristics than the first sample set.
According to the embodiment, on the basis of labeled data, a large amount of unlabeled data are used for training the detection model, and the separation and fusion of features are directly carried out on the shallow layer of the network, so that the training process is simplified, the low-dimensional features in the target domain are strengthened, the low-cost domain migration is realized, and the detection stability of the detection model in different scenes is improved.
As an optional implementation manner, the fusing the first low-dimensional feature map and the second low-dimensional feature map to obtain a fused feature map includes:
determining the fused feature map according to a first mean value and a first standard deviation of the first low-dimensional feature map on each channel of the feature extraction layer and a second mean value and a second standard deviation of the second low-dimensional feature map on each channel of the feature extraction layer;
wherein the feature extraction layer is a convolution layer for extracting low-dimensional features in the detection model.
As an optional implementation, the determining the fused feature map includes:
normalizing the first low-dimensional feature map according to the first mean value and the first standard deviation to obtain a normalized first low-dimensional feature map;
and determining the fusion feature map by taking the second mean value as the offset of the normalized first low-dimensional feature map and the second standard deviation as the scaling weight of the normalized first low-dimensional feature map.
As an optional implementation, the detection model further includes, at each training time:
determining a domain classification loss value according to the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in the training process according to the domain classification loss value so as to extract the domain-invariant features.
In the embodiment, the source domain, the target domain and the fusion feature are used in the domain classifier for distinguishing, so that the method is more beneficial for extracting the feature with unchanged domain from the backbone network of the detection model.
As an optional implementation manner, the adjusting parameters of the detection model in the training process according to the domain classification loss value to extract domain-invariant features includes:
and performing back propagation on the gradient corresponding to the domain classification loss value by using a gradient inversion layer, and adjusting parameters of the detection model in a training process to extract the feature with unchanged domain.
As an optional implementation, the detection model further includes, at each training time:
determining a similarity loss value according to the semantic similarity of the first low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in a training process according to the similarity loss value so as to constrain semantic consistency of the first low-dimensional feature map and the fusion feature map.
According to the embodiment, through separation and fusion of low-dimensional features between different domains and semantic consistency constraint, unique features of the target domain are strengthened on the basis of extracting the domain invariant features, and the performance of the detector on the target domain is improved.
As an optional implementation, the detection model further includes, at each training time:
determining a candidate box set associated with a target object in the fusion feature map;
determining a similar mutual exclusion loss value according to the candidate frame set associated with different target objects;
and adjusting parameters of the detection model in the training process according to the similar mutual exclusion loss value so as to enlarge the distance between different candidate frame sets.
In the embodiment, on the basis that the candidate frames are regressed to the target frame, the mutual exclusion loss of the same kind is introduced, and the distance between different candidate frames is enlarged, so that the candidate frames matched with different targets are far away as possible, the quality of the candidate frames of the detector is improved, and the problem of virtual detection of the target object in a dense scene is solved.
As an optional implementation manner, the determining a homogeneous mutual exclusion loss value according to a candidate box set associated with different target objects includes:
determining a plurality of candidate frame groups from a first candidate frame set associated with a first target object and a second candidate frame set associated with a second target object;
and determining the similar mutual exclusion loss value according to the distance between the first candidate frame and the second candidate frame in each candidate frame group.
In a second aspect, an embodiment of the present invention provides a training method for a detection model, where the method includes:
acquiring training samples of a detection model to be trained, wherein the training samples comprise a labeled first sample set and an unlabeled second sample set; the first sample set and the second sample set both contain target objects, the second sample set containing scene features different from the first sample set;
and extracting a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set during each training, fusing the first low-dimensional feature map and the second low-dimensional feature map to obtain a fused feature map, and training an initial detection model by using the fused feature map and the position of a target object marked by the first sample image to obtain a trained detection model.
In a third aspect, an apparatus for object detection provided by an embodiment of the present invention includes a processor and a memory, where the memory is configured to store a program executable by the processor, and the processor is configured to read the program in the memory and execute the following steps:
acquiring a target image to be detected;
inputting the target image into a trained detection model, and outputting the position of a target object in the target image;
the detection model is obtained by training a labeled first sample set and an unlabeled second sample set, a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set are extracted during each training, the first low-dimensional feature map and the second low-dimensional feature map are fused to obtain a fused feature map, and the positions of target objects labeled by the fused feature map and the first sample image are used for training; wherein the first sample set and the second sample set both contain target objects, the second sample set containing different scene characteristics than the first sample set.
As an alternative embodiment, the processor is configured to perform:
determining the fused feature map according to a first mean value and a first standard deviation of the first low-dimensional feature map on each channel of the feature extraction layer and a second mean value and a second standard deviation of the second low-dimensional feature map on each channel of the feature extraction layer;
wherein the feature extraction layer is a convolution layer for extracting low-dimensional features in the detection model.
As an alternative embodiment, the processor is configured to perform:
normalizing the first low-dimensional feature map according to the first mean value and the first standard deviation to obtain a normalized first low-dimensional feature map;
and determining the fusion feature map by taking the second mean value as the offset of the normalized first low-dimensional feature map and the second standard deviation as the scaling weight of the normalized first low-dimensional feature map.
As an optional implementation, in each training of the detection model, the processor is specifically further configured to perform:
determining a domain classification loss value according to the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in the training process according to the domain classification loss value so as to extract the domain-invariant features.
As an alternative embodiment, the processor is configured to perform:
and performing back propagation on the gradient corresponding to the domain classification loss value by using a gradient inversion layer, and adjusting parameters of the detection model in a training process to extract the feature with unchanged domain.
As an optional implementation, in each training of the detection model, the processor is specifically further configured to perform:
determining a similarity loss value according to the semantic similarity of the first low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in a training process according to the similarity loss value so as to constrain semantic consistency of the first low-dimensional feature map and the fusion feature map.
As an optional embodiment, in each training of the detection model, the processor is specifically further configured to perform:
determining a candidate box set associated with a target object in the fusion feature map;
determining a similar mutual exclusion loss value according to the candidate frame set associated with different target objects;
and adjusting parameters of the detection model in the training process according to the similar mutual exclusion loss value so as to enlarge the distance between different candidate frame sets.
As an alternative embodiment, the processor is configured to perform:
determining a plurality of candidate frame groups from a first candidate frame set associated with a first target object and a second candidate frame set associated with a second target object;
and determining the similar mutual exclusion loss value according to the distance between the first candidate frame and the second candidate frame in each candidate frame group.
In a fourth aspect, an embodiment of the present invention further provides an apparatus for target detection, where the apparatus includes:
the image acquisition unit is used for acquiring a target image to be detected;
the detection target unit is used for inputting the target image into a trained detection model and outputting the position of a target object in the target image; the detection model is obtained by training a labeled first sample set and an unlabeled second sample set, a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set are extracted during each training, the first low-dimensional feature map and the second low-dimensional feature map are fused to obtain a fused feature map, and the positions of target objects labeled by the fused feature map and the first sample image are used for training; wherein the first sample set and the second sample set both contain target objects, the second sample set containing different scene characteristics than the first sample set.
As an optional implementation manner, the detection target unit is specifically configured to:
determining the fused feature map according to a first mean value and a first standard deviation of the first low-dimensional feature map on each channel of the feature extraction layer and a second mean value and a second standard deviation of the second low-dimensional feature map on each channel of the feature extraction layer;
wherein the feature extraction layer is a convolution layer for extracting low-dimensional features in the detection model.
As an optional implementation manner, the detection target unit is specifically configured to:
normalizing the first low-dimensional feature map according to the first mean value and the first standard deviation to obtain a normalized first low-dimensional feature map;
and determining the fusion feature map by taking the second mean value as the offset of the normalized first low-dimensional feature map and the second standard deviation as the scaling weight of the normalized first low-dimensional feature map.
As an optional implementation manner, in each training of the detection model, the detection target unit is further specifically configured to:
determining a domain classification loss value according to the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in the training process according to the domain classification loss value so as to extract the features with invariable domains.
As an optional implementation manner, the detection target unit is specifically configured to:
and performing back propagation on the gradient corresponding to the domain classification loss value by using a gradient inversion layer, and adjusting parameters of the detection model in a training process to extract the feature with unchanged domain.
As an optional implementation manner, in each training of the detection model, the detection target unit is further specifically configured to:
determining a similarity loss value according to the semantic similarity of the first low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in a training process according to the similarity loss value so as to constrain semantic consistency of the first low-dimensional feature map and the fusion feature map.
As an optional implementation manner, in each training of the detection model, the detection target unit is further specifically configured to:
determining a candidate box set associated with a target object in the fusion feature map;
determining a similar mutual exclusion loss value according to the candidate frame set associated with different target objects;
and adjusting parameters of the detection model in the training process according to the similar mutual exclusion loss value so as to enlarge the distance between different candidate frame sets.
As an optional implementation manner, the detection target unit is specifically configured to:
determining a plurality of candidate frame groups from a first candidate frame set associated with a first target object and a second candidate frame set associated with a second target object;
and determining the similar mutual exclusion loss value according to the distance between the first candidate frame and the second candidate frame in each candidate frame group.
In a fifth aspect, embodiments of the present invention further provide a computer storage medium, on which a computer program is stored, where the computer program is used to implement the steps of the method in the first or second aspect when executed by a processor.
These and other aspects of the present application will be more readily apparent from the following description of the embodiments.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1A is a schematic diagram of an image of a target to be detected according to an embodiment of the present invention;
fig. 1B is a schematic diagram of an image of a target to be detected according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a false detection of a target object according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating an embodiment of a method for target detection according to the present invention;
FIG. 4 is a schematic diagram of a detection model according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a target detection model in a training phase according to an embodiment of the present invention;
fig. 6 is a flowchart of an implementation of a method for detecting a target object according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating an implementation of a model training method according to an embodiment of the present invention;
fig. 8 is a schematic diagram of an apparatus for target detection according to an embodiment of the present invention;
fig. 9 is a schematic diagram of an apparatus for object detection according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The term "and/or" in the embodiments of the present invention describes an association relationship of associated objects, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
The application scenario described in the embodiment of the present invention is for more clearly illustrating the technical solution of the embodiment of the present invention, and does not form a limitation on the technical solution provided in the embodiment of the present invention, and it can be known by a person skilled in the art that with the occurrence of a new application scenario, the technical solution provided in the embodiment of the present invention is also applicable to similar technical problems. In the description of the present invention, the term "plurality" means two or more unless otherwise specified.
Embodiment 1, with the improvement of the computing power of modern computers, massive data can be truly utilized. In the field of computer vision, the indexes of related tasks are greatly improved by a deep learning-based method. For example, in the scenes of video monitoring, driving assistance and the like, the abstract features of the target are automatically extracted through training and analyzing a large amount of data, and the effects of common downstream tasks such as classification, detection, segmentation and the like are well improved. The current method based on deep learning is generally a data-driven method, however, the real environment is complex and diverse, and if the distribution of data of an actual scene and a scene where the current training set is located is obviously different, the performance of the model may be significantly reduced. For example, when a lightweight non-motor vehicle detector is trained using only the data in the scene of fig. 1A, there is a possibility that the detector has more missed detections and false detections due to differences in points, weather, illumination, etc. in the scene of fig. 1B. In order to ensure the performance of the algorithm, data under all scenes needs to be covered during training, but the cost of data labeling is "expensive", and a large amount of labor and time are needed. In addition, in the high peak period of traffic flow in the morning and evening, the number of non-motor vehicles is obviously increased, and false detection is easier to occur. As shown in fig. 2, in the training phase, the upper left labeled box is labeled GT1 and the lower right labeled box is labeled GT 2; the top left candidate box is denoted as PR1 and the bottom right candidate box is denoted as PR 2. Assuming that both PR1 and PR2 are matched to GT1, while PR1 and PR2 have the same IoU as GT1 (Intersection Over Union), PR2 is more susceptible to GT2, thereby causing PR2 to move away from the intended target, resulting in false detection between GT1 and GT 2.
Due to the fact that the real environment is complex and diverse, if the data distribution of the actual scene and the scene where the current training set is located is obviously different, for example, the actual scene is an image in a night scene, and the scene of the training set is an image in a day scene, the performance of the detection model may be significantly reduced. Therefore, in order to solve the problem that the performance of the detection model is reduced when the data distribution of the actual scene and the scene where the training set is located are greatly different, the present embodiment provides a method for detecting a target, which can improve the detection performance of the detection model in the actual scene while ensuring that the amount of labeled training data is not increased.
The core idea of the method for target detection provided by this embodiment is that a model is trained by using an unlabeled second sample set and an labeled first sample set, wherein a second low-dimensional feature of a second sample image in the second sample set and a first low-dimensional feature of a first sample image in the first sample set are fused, and the model is trained by using the fusion feature.
The target object of target detection in the present embodiment may be a person, a vehicle, an animal, or the like, and the present embodiment is not limited to this. The vehicle comprises at least one of an automobile and a non-automobile.
As shown in fig. 3, a specific implementation flow of the method for detecting a target provided in the embodiment of the present invention is as follows:
step 300, obtaining a target image to be detected;
in the embodiment, the target image to be detected comprises an image shot in any scene and/or an image shot by any camera device; optionally, the scene features in this embodiment are used to represent environmental features, such as lighting conditions, weather conditions, and the like, in which the image is captured, and/or style attributes used in capturing the image, which is not limited in this embodiment.
Step 301, inputting the target image into a trained detection model, and outputting the position of a target object in the target image; the detection model is obtained by training a labeled first sample set and an unlabeled second sample set, a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set are extracted during each training, the first low-dimensional feature map and the second low-dimensional feature map are fused to obtain a fused feature map, and the positions of target objects labeled by the fused feature map and the first sample image are used for training; wherein the first sample set and the second sample set both contain target objects, the second sample set containing different scene characteristics than the first sample set.
In some embodiments, the detection model in this embodiment is constructed by using a feature extractor, and the feature extractor in this embodiment includes, but is not limited to, at least one of CNN (Convolutional Neural Networks) and transform.
In some embodiments, as shown in fig. 4, the detection model in this embodiment includes a feature extraction module 400 and a classification regression module 401, where the feature extraction module includes a first feature extraction module and a second feature extraction module, the first feature extraction module is configured to extract low-dimensional features of an image, the second feature extraction module is configured to extract high-dimensional features of the image, an input target image is subjected to feature extraction by the first feature extraction module to obtain a low-dimensional feature map, and then the low-dimensional feature map is input to the second feature extraction module to perform feature extraction to obtain a high-dimensional feature map; and then inputting the high-dimensional feature map into a classification regression module for classification and regression, determining a candidate frame set of the target object, screening a detection frame of the target object from the candidate frame set, outputting the detection frame, and determining the position of the target object in the target image through the detection frame.
For example, in the present embodiment, the VGG16 is used as the backbone network G (i.e., the feature extraction module) of the detection model, and a total of 5 blocks are connected in series. Wherein G is 1 Representing the first 2 blocks for extracting low-dimensional features, G 2 And representing the last 3 blocks for extracting high-dimensional features. The classification regression module can take two-stage fast RCNN as an example, including RPN (Region Proposal network) and RCNN (Region-CNN). Where RPN is used to extract candidate boxes and RCNN is used for quadratic regression to correct the position of the candidate boxes.
In some embodiments, the detection model in this embodiment is initialized using a pre-trained model D _ pretrain based on ImageNet, that is, the pre-trained model is pre-trained by using labeled data in ImageNet, so as to obtain an initial detection model. Wherein, the structures of the detection model and the pre-training model D _ pretrain are the same. It should be noted that, in general, a target detection network uses a large classification data set, such as ImageNet, to pre-train a backbone network, and a large number of data sets labeled with training data in the ImageNet are used, so that the feature expression learned on the large-scale data set can help a detection algorithm to converge more quickly.
In some embodiments, the first low-dimensional feature map in this embodiment is obtained by feature extraction of the first sample image according to a feature extraction layer; the second low-dimensional feature map in this embodiment is obtained by performing feature extraction on the second sample image according to the feature extraction layer; in this embodiment, the first low-dimensional feature map and the second low-dimensional feature map are obtained by performing feature extraction by using the same feature extraction method, and in a specific implementation, the features of the first sample image and the second sample image may be extracted by using the same feature extraction layer, so as to obtain the first low-dimensional feature map and the second low-dimensional feature map. Wherein, the feature extraction layer is a convolution layer used for extracting low-dimensional features in the detection model. The detection model further includes a convolution layer for extracting high-dimensional features.
In some embodiments, the first low-dimensional feature map and the second low-dimensional feature map in this embodiment are used for characterizing at least one of color, texture, and style of the image. That is, the low-dimensional feature in the present embodiment is used to represent at least one of color, texture, and style of the image. The color, texture and style of the image are related to the environmental characteristics, such as the illumination condition and the weather condition, of the captured image and the style attribute used when the image is captured, so that the present embodiment can effectively fuse the characteristics of different scenes by extracting the low-dimensional characteristics in the first sample image and the second sample image, and since the unlabeled second sample set is generally related to the actual detection scene, the second sample set can be directly collected from the actual environment (without labeling) in the implementation, thereby improving the detection performance of the detection model in the actual scene (actual scene).
In implementation, in this embodiment, the labeled first sample set and the unlabeled second sample set include scene features that are not completely the same or different, and the scene features in the second sample set are used to perform fusion of the low-dimensional features, and the position of the target object in the fused feature map after fusion does not change relative to the position of the target object in the first sample image before fusion, so that the position of the labeled target object in the fused feature map and the corresponding first sample image is used to perform training, which can improve the detection performance of the detection model in the scene in the second sample set.
The detection model in the embodiment is trained by using the sample sets with different scene characteristics, so that the detection can be performed on target images with different scene characteristics, and the detection performance can be ensured without increasing the cost of the labeling data.
In some embodiments, the present embodiment obtains the fused feature map by:
determining the fused feature map according to a first mean value and a first standard deviation of the first low-dimensional feature map on each channel of the feature extraction layer and a second mean value and a second standard deviation of the second low-dimensional feature map on each channel of the feature extraction layer;
wherein the feature extraction layer is a convolution layer for extracting low-dimensional features in the detection model.
In implementation, the mean and standard deviation of each channel of the low-dimensional feature extraction layer are used to realize fusion of different feature maps, and the fusion is performed to fuse the scene features of the second sample set and the scene features of the first sample set, so that the detection model improves the detection performance in a new scene.
Optionally, the number of channels in this embodiment is related to the number of convolution kernels in the detection model.
It should be noted that the first low-dimensional feature map is a three-dimensional vector, and when calculating the mean value and the standard deviation, the mean value and the standard deviation of each two-dimensional image of the three-dimensional vector in a certain dimension are calculated, so as to obtain a first mean value and a first standard deviation; similarly, the second low-dimensional feature map is also a three-dimensional vector, and when the mean value and the standard deviation are calculated, the mean value and the standard deviation of each two-dimensional image of the three-dimensional vector in a certain dimension are calculated, and finally, a second mean value and a second standard deviation are obtained.
In some embodiments, the fused feature map is calculated using the mean and variance;
step 1) normalizing the first low-dimensional feature map according to the first mean value and the first standard deviation to obtain a normalized first low-dimensional feature map;
in practice, the normalization process is performed by the following formula:
Figure 379500DEST_PATH_IMAGE001
formula (1);
wherein, F s Representing a normalized first low-dimensional feature map, f s A first low-dimensional feature map is represented,
Figure 482454DEST_PATH_IMAGE002
which represents the first mean value of the first mean value,
Figure 78651DEST_PATH_IMAGE003
the first standard deviation is indicated.
And 2) determining the fusion feature map by taking the second mean value as the offset of the normalized first low-dimensional feature map and the second standard deviation as the scaling weight of the normalized first low-dimensional feature map.
In implementation, the fused feature map is calculated by the following formula:
Figure 368687DEST_PATH_IMAGE004
formula (2);
wherein the content of the first and second substances,
Figure 856169DEST_PATH_IMAGE005
representing a fused feature map, f s A first low-dimensional feature map is represented,
Figure 813630DEST_PATH_IMAGE002
which represents the first mean value of the first mean value,
Figure 846308DEST_PATH_IMAGE003
represents a first standard deviation;
Figure 154798DEST_PATH_IMAGE006
a second low-dimensional feature map is represented,
Figure 445971DEST_PATH_IMAGE007
which represents the second mean value of the first mean value,
Figure 477512DEST_PATH_IMAGE008
the second standard deviation is indicated.
It should be noted that, usually, during training of the target detection model, the loss value is used to adjust parameters of the detection model during the training process, and when the loss value satisfies the preset condition, the training is completed. In this embodiment, when training the target detection model, the loss value of the fusion feature graph during classification and regression is calculated, that is, the loss value calculated after the fusion features are input to the classification regression module, in addition, because the feature fusion of different domains (the first sample set and the second sample set) is performed in this embodiment, in order to constrain the fusion features, it is ensured that the domain-invariant features are extracted, this embodiment further provides the following steps:
determining a domain classification loss value according to the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map when the detection model is trained each time; and adjusting parameters of the detection model in the training process according to the domain classification loss value so as to extract the domain-invariant features.
In some embodiments, a gradient inversion layer is used to perform back propagation on a gradient corresponding to the domain classification loss value, and parameters of the detection model in a training process are adjusted to extract domain-invariant features. And performing confusion of different task domains on the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map by using a gradient inversion layer.
In implementation, in a training phase, a typical domain classifier can be used, the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map are input to the domain classifier, and a countermeasure training is implemented by using a gradient inversion layer GRL in the domain classifier, so as to achieve confusion of source domain (a first sample set) and target domain data (a second sample set), so that a backbone network in a detection model extracts domain-invariant features. Optionally, when the class confidence of the domain classification output in this embodiment approaches to uniform distribution (for example, when the two domain labels are provided, the confidence corresponding to the two nodes of the classifier are both 0.5), it is determined that the trunk network in the detection model can better extract the feature with unchanged domain at this time. Wherein the domain classification penalty value may be determined by
Figure 461518DEST_PATH_IMAGE009
And (4) showing.
It should be noted that the GRL layer does not change any parameter during the forward propagation process, and only acts on the reverse propagation processUpdating the parameters to be propagated. In the forward propagation process, the features extracted by the domain classifier (assumed to be D) are used for calculating a domain classification loss value, and in the backward propagation process, the gradient corresponding to the domain classification loss value calculated in D is propagated backward, that is, the loss is minimized; but for a backbone network G (e.g. G) 1 、G 2 ) In other words, the gradient corresponding to the domain classification loss value is reversely propagated after GRL inversion, that is, the loss is maximized, thereby realizing a countermeasure training and enabling the backbone network to extract the feature with unchanged domain.
In some embodiments, the detection model further determines a similarity loss value according to semantic similarity between the first low-dimensional feature map and the fused feature map during each training; and adjusting parameters of the detection model in a training process according to the similarity loss value so as to constrain semantic consistency of the first low-dimensional feature map and the fusion feature map.
In implementation, the target objects in the first low-dimensional feature map and the fused feature map should have semantic similarity, so that the first low-dimensional feature map and the fused feature map can be used as a similar network
Figure 726146DEST_PATH_IMAGE010
The semantic similarity of the two is calculated, the similarity loss value is determined, and the problem that the network is not easy to converge is solved by adding constraint conditions. Alternative, similar networks
Figure 571742DEST_PATH_IMAGE010
Similar network can be composed of the last 3 blocks of VGG16
Figure 503795DEST_PATH_IMAGE010
Can also be derived from a pre-trained model
Figure 393122DEST_PATH_IMAGE011
Initialization is performed and the parameters of the similar network are fixed and not updated during the training of the detection model. Alternatively, the similarity Loss value may be a typical Perceptual Loss (Perceptual Loss), which is denoted as
Figure 895779DEST_PATH_IMAGE012
。
In some embodiments, the detection model may further determine a set of candidate boxes associated with a target object in the fused feature map each time it is trained; in an implementation, a set of candidate boxes is determined, for example, by using the RPN, and a fusion feature is input to the RPN to output the set of candidate boxes associated with a target object, wherein the set of candidate boxes includes one or more candidate boxes; when a plurality of target objects exist in the fusion characteristic diagram, determining a similar mutual exclusion loss value according to a candidate frame set associated with different target objects; and adjusting parameters of the detection model in the training process according to the similar mutual exclusion loss value so as to enlarge the distance between different candidate frame sets.
In some embodiments, homogeneous mutual exclusion penalty values are determined by:
determining a plurality of candidate frame groups from a first candidate frame set associated with a first target object and a second candidate frame set associated with a second target object; and determining the similar mutual exclusion loss value according to the distance between the first candidate frame and the second candidate frame in each candidate frame group.
In implementation, the mutual exclusion loss values of the same type are used for reducing the distance between different candidate frames, so that the problems of multiple detection and false detection caused by the fact that a plurality of dense target objects are contained in a target image, such as a plurality of dense vehicles, are solved. The candidate box generally needs to be associated with the target object (i.e. the target box) according to a certain matching policy (e.g. maximum IoU), and the set of candidate boxes associated with the kth target object is assumed to be
Figure 528754DEST_PATH_IMAGE013
When is coming into contact with
Figure 315314DEST_PATH_IMAGE014
When the temperature of the water is higher than the set temperature,
Figure 126275DEST_PATH_IMAGE015
. Then the same kind of mutual exclusion loss value
Figure 631074DEST_PATH_IMAGE016
The specific calculation process is shown as the following formula:
Figure 536582DEST_PATH_IMAGE017
formula (3);
wherein the content of the first and second substances,
Figure 928381DEST_PATH_IMAGE018
represent
Figure 159511DEST_PATH_IMAGE019
Each candidate frame of (1) and
Figure 847148DEST_PATH_IMAGE020
the corresponding distance is calculated pairwise for each candidate frame in (1), and N represents the logarithm of the candidate frame subjected to distance calculation. Optionally, the distance function may select the GIoU distance.
In some embodiments, the total loss used by the detection model in this embodiment during the training phase is shown as follows:
Figure 759609DEST_PATH_IMAGE021
formula (4);
wherein the content of the first and second substances,
Figure 724023DEST_PATH_IMAGE022
are weight factors respectively;
Figure 63737DEST_PATH_IMAGE023
indicating loss of detection, including, for example, loss of classification and regression in RPN and RCNN;
Figure 480812DEST_PATH_IMAGE024
representing a similarity loss value;
Figure 993702DEST_PATH_IMAGE025
representing a homogeneous mutual exclusion penalty value;
Figure 609360DEST_PATH_IMAGE026
representing a domain classification loss value.
In the testing stage and the using stage of the detection model, a target image to be detected directly obtains a feature map through a feature extraction module, and the feature map is input into a RPN module and an RCNN module, so that the predicted coordinates of the target object are output. I.e. feature fusion, semantic consistency constraints
Figure 933025DEST_PATH_IMAGE027
The domain classifier GRL is only used in the training process.
As shown in fig. 5, this embodiment provides a schematic diagram of a target detection model in a training phase, in which VGG16 is used as a main network G (i.e., a feature extraction module) of the detection model, and the main network G is formed by connecting 5 blocks in series. Wherein G is 1 Representing the first 2 blocks, and extracting low-dimensional features, wherein the low-dimensional features comprise a first low-dimensional feature graph T, a second low-dimensional feature graph G and a fusion feature graph TG; g 2 And representing the last 3 blocks for extracting high-dimensional features. The classification regression module employs two-stage fast RCNN, including RPN and RCNN. Wherein, the DADAIN module is used for carrying out feature fusion, and the GRL module is used for classifying the domain into loss values
Figure 165292DEST_PATH_IMAGE028
The corresponding gradient is propagated reversely, so that the parameters in the main network are optimized and adjusted in the direction of negative gradient, and the similar network
Figure 481873DEST_PATH_IMAGE029
For calculating similarity loss value
Figure 437190DEST_PATH_IMAGE030
RPN is also used to calculate homogeneous mutual exclusion penalty values
Figure 181024DEST_PATH_IMAGE031
。
It should be noted that, in fact, since the feature extraction module is divided into the low-dimensional part and the high-dimensional part in this embodiment, when the domain classification loss value is calculated, the two parts are divided into two parts, one part calculates the domain classification loss values of the first low-dimensional feature map, the second low-dimensional feature map and the fused feature map that are output after the low-dimensional feature extraction, and mainly aims at the domain alignment processing of the pixel level, and the other part calculates the first high-dimensional feature map, the second high-dimensional feature map and the high-dimensional fused feature map that are output after the first low-dimensional feature map, the second low-dimensional feature map and the fused feature map are respectively subjected to the high-dimensional feature extraction, and mainly aims at the domain alignment processing of the image level.
As shown in fig. 6, this embodiment further provides a method for detecting a target object, and the specific implementation flow of the method is as follows:
step 600, obtaining a first sample set which is marked and a second sample set which is not marked;
601, in the training process of the detection model, sampling one image from the first sample set and the second sample set each time for training;
that is, the number of samples selected in one training is 2, wherein the Size of the sample Size is determined according to the optimization degree and speed of the model.
Step 602, extracting a first low-dimensional feature map of a first sample image and a second low-dimensional feature map of a second sample image;
step 603, fusing the first low-dimensional feature map and the second low-dimensional feature map to obtain a fused feature map;
step 604, training by using the position of the target object marked by the fusion feature map and the first sample image;
step 605, determining a domain classification loss value according to the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map;
step 606, determining a similarity loss value according to semantic similarity of the first low-dimensional feature map and the fusion feature map;
step 607, determining a candidate frame set associated with the target object in the fusion feature map; determining a similar mutual exclusion loss value according to the candidate frame set associated with different target objects;
step 608, adjusting parameters of the detection model in the training process according to the domain classification loss value, the similarity loss value, the homogeneous mutual exclusion loss value and the detection loss value, stopping until the total loss value meets a preset condition, and determining that the training of the detection model is finished;
step 609, inputting the target image to be detected into the trained detection model, and outputting the position of the target object in the target image.
In some embodiments, as shown in fig. 7, the present embodiment further provides a model training method based on the above target detection method, and a specific implementation flow of the method is as follows:
step 700, obtaining training samples of a detection model to be trained, wherein the training samples comprise a first sample set which is labeled and a second sample set which is not labeled; the first sample set and the second sample set both contain target objects, the second sample set containing scene features different from the first sample set;
in implementation, the training samples in the embodiment are different from the commonly labeled training samples, and unlabeled training samples under different scene characteristics are added on the basis of the labeled training samples, so that the accuracy of target detection under wider and more real scenes can be ensured in the training process, the detection performance is improved, and the data labeling cost is effectively saved.
Step 701, extracting a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set during each training, fusing the first low-dimensional feature map and the second low-dimensional feature map to obtain a fused feature map, and training an initial detection model by using the fused feature map and the position of a target object marked by the first sample image to obtain a trained detection model.
In implementation, each training is performed, a first sample image and a second sample image are input into an initial detection model, a first low-dimensional feature map of the first sample image and a second low-dimensional feature map of the second sample image are extracted, then the first low-dimensional feature map and the second low-dimensional feature map are fused to obtain a fusion feature map, and the initial detection model is trained by using the positions of target objects marked by the fusion feature map and the first sample image, so that a trained detection model is obtained.
It should be noted that, since the position of the target object in the fused feature map after fusion does not change relative to the position of the target object in the first sample image before fusion, the detection performance of the detection model in the scene in the second sample set can be improved by training with the fused feature map and the position of the labeled target object in the corresponding first sample image.
In some embodiments, the detection model, upon each training, is further to:
determining a domain classification loss value according to the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map; and adjusting parameters of the detection model in the training process according to the domain classification loss value so as to extract the domain-invariant features.
In some embodiments, a gradient inversion layer is used to perform back propagation on a gradient corresponding to the domain classification loss value, and parameters of the detection model in a training process are adjusted to extract domain-invariant features.
In some embodiments, the detection model, upon each training, is further configured to:
determining a similarity loss value according to the semantic similarity of the first low-dimensional feature map and the fusion feature map; and adjusting parameters of the detection model in a training process according to the similarity loss value so as to constrain semantic consistency of the first low-dimensional feature map and the fusion feature map.
In some embodiments, the detection model, upon each training, is further configured to:
determining a candidate box set associated with a target object in the fusion feature map; determining a similar mutual exclusion loss value according to the candidate frame set associated with different target objects; and adjusting parameters of the detection model in the training process according to the similar mutual exclusion loss value so as to enlarge the distance between different candidate frame sets.
In some embodiments, homogeneous mutual exclusion penalty values are determined by:
determining a plurality of candidate frame groups from a first candidate frame set associated with a first target object and a second candidate frame set associated with a second target object; and determining the similar mutual exclusion loss value according to the distance between the first candidate frame and the second candidate frame in each candidate frame group.
In some embodiments, the loss value of the detection model during classification and regression through the fusion feature map is calculated during each training, and a domain classification loss value, a similarity loss value and a similar mutual exclusion loss value are also calculated, so that parameters of the detection model during the training process are adjusted by using the total loss value, and when the total loss value meets a preset condition, the completion of the training is determined, and the trained detection model is obtained.
The structure of the detection model in the training phase in this embodiment can refer to fig. 5 and the above description, and is not repeated here.
The method solves the problem that under the condition that the data distribution of a source domain (a first sample set) and a target domain (a second sample set) has a large difference, the unmarked data of the target domain is directly used, and the performance of a detection model under the target domain is improved; meanwhile, the problem of virtual detection of the target object in a dense scene is solved. On the basis of labeled data, a large amount of unlabeled data are used for training a detection model, compared with the method of training a transformation network independently in advance, the source domain data style is migrated to a target domain, the method directly performs separation and fusion of features on the shallow layer of the network, the training process is simplified, low-dimensional features in the target domain are enhanced, low-cost domain migration is achieved, separation and fusion of the low-dimensional features between different domains are performed, semantic consistency constraint is performed, on the basis of extracting the invariant features of the domains, unique features of the target domain are enhanced, the performance of a detector on the target domain is improved, and meanwhile, the detection stability of the detection model under different scenes is improved. In addition, in the domain classifier, the source domain, the target domain and the fusion feature are used for distinguishing, so that the domain invariant feature can be extracted by a backbone network of the detection model more favorably. In the training stage of the detection model, on the basis that the candidate frames are regressed to the target frame, the mutual exclusion loss of the same kind is introduced, and the distance between different candidate frames is enlarged, so that the candidate frames matched with different targets are far away as possible, the quality of the candidate frames of the detector is improved, and the problem of virtual detection of the target object in a dense scene is solved.
Embodiment 2 is based on the same inventive concept, and the embodiment of the present invention further provides a device for target detection, where the device is a device in the method in the embodiment of the present invention, and the principle of the device to solve the problem is similar to that of the method, so that the implementation of the device may refer to the implementation of the method, and repeated details are not repeated.
As shown in fig. 8, the apparatus includes a processor 800 and a memory 801, the memory 801 is used for storing programs executable by the processor 800, the processor 800 is used for reading the programs in the memory 801 and executing the following steps:
acquiring a target image to be detected;
inputting the target image into a trained detection model, and outputting the position of a target object in the target image;
the detection model is obtained by training a labeled first sample set and an unlabeled second sample set, a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set are extracted during each training, the first low-dimensional feature map and the second low-dimensional feature map are fused to obtain a fused feature map, and the positions of target objects labeled by the fused feature map and the first sample image are used for training; wherein the first sample set and the second sample set both contain target objects, the second sample set containing different scene characteristics than the first sample set.
As an alternative implementation, the processor 800 is specifically configured to perform:
determining the fused feature map according to a first mean value and a first standard deviation of the first low-dimensional feature map on each channel of the feature extraction layer and a second mean value and a second standard deviation of the second low-dimensional feature map on each channel of the feature extraction layer;
wherein the feature extraction layer is a convolution layer for extracting low-dimensional features in the detection model.
As an alternative implementation, the processor 800 is specifically configured to perform:
normalizing the first low-dimensional feature map according to the first mean value and the first standard deviation to obtain a normalized first low-dimensional feature map;
and determining the fusion feature map by taking the second mean value as the offset of the normalized first low-dimensional feature map and the second standard deviation as the scaling weight of the normalized first low-dimensional feature map.
As an optional implementation, in each training of the detection model, the processor 800 is specifically further configured to perform:
determining a domain classification loss value according to the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in the training process according to the domain classification loss value so as to extract the features with invariable domains.
As an alternative implementation, the processor 800 is specifically configured to perform:
and performing back propagation on the gradient corresponding to the domain classification loss value by using a gradient inversion layer, and adjusting parameters of the detection model in a training process to extract the feature with unchanged domain.
As an optional implementation, in each training of the detection model, the processor 800 is specifically further configured to perform:
determining a similarity loss value according to the semantic similarity of the first low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in a training process according to the similarity loss value so as to constrain semantic consistency of the first low-dimensional feature map and the fusion feature map.
As an optional implementation, in each training of the detection model, the processor 800 is specifically further configured to perform:
determining a candidate box set associated with a target object in the fusion feature map;
determining a similar mutual exclusion loss value according to the candidate frame set associated with different target objects;
and adjusting parameters of the detection model in the training process according to the similar mutual exclusion loss value so as to enlarge the distance between different candidate frame sets.
As an alternative implementation, the processor 800 is specifically configured to perform:
determining a plurality of candidate frame groups from a first candidate frame set associated with a first target object and a second candidate frame set associated with a second target object;
and determining the similar mutual exclusion loss value according to the distance between the first candidate frame and the second candidate frame in each candidate frame group.
Embodiment 3, based on the same inventive concept, an embodiment of the present invention further provides a device for target detection, and since the device is the device in the method in the embodiment of the present invention, and the principle of the device to solve the problem is similar to that of the method, the implementation of the device may refer to the implementation of the method, and repeated details are not repeated.
As shown in fig. 9, the apparatus includes:
an image acquiring unit 900, configured to acquire a target image to be detected;
a detection target unit 901, configured to input the target image into a trained detection model, and output a position of a target object in the target image; the detection model is obtained by training a labeled first sample set and an unlabeled second sample set, a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set are extracted during each training, the first low-dimensional feature map and the second low-dimensional feature map are fused to obtain a fused feature map, and the positions of target objects labeled by the fused feature map and the first sample image are used for training; wherein the first sample set and the second sample set both contain target objects, the second sample set containing different scene characteristics than the first sample set.
As an optional implementation manner, the detection target unit 901 is specifically configured to:
determining the fused feature map according to a first mean value and a first standard deviation of the first low-dimensional feature map on each channel of the feature extraction layer and a second mean value and a second standard deviation of the second low-dimensional feature map on each channel of the feature extraction layer;
wherein the feature extraction layer is a convolution layer for extracting low-dimensional features in the detection model.
As an optional implementation manner, the detection target unit 901 is specifically configured to:
normalizing the first low-dimensional feature map according to the first mean value and the first standard deviation to obtain a normalized first low-dimensional feature map;
and determining the fusion feature map by taking the second mean value as the offset of the normalized first low-dimensional feature map and the second standard deviation as the scaling weight of the normalized first low-dimensional feature map.
As an optional implementation manner, in each training of the detection model, the detection target unit 901 is further specifically configured to:
determining a domain classification loss value according to the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in the training process according to the domain classification loss value so as to extract the domain-invariant features.
As an optional implementation manner, the detection target unit 901 is specifically configured to:
and performing back propagation on the gradient corresponding to the domain classification loss value by using a gradient inversion layer, and adjusting parameters of the detection model in a training process to extract the feature with unchanged domain.
As an optional implementation manner, in each training of the detection model, the detection target unit 901 is further specifically configured to:
determining a similarity loss value according to the semantic similarity of the first low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in a training process according to the similarity loss value so as to constrain semantic consistency of the first low-dimensional feature map and the fusion feature map.
As an optional implementation manner, in each training of the detection model, the detection target unit 901 is further specifically configured to:
determining a candidate box set associated with a target object in the fusion feature map;
determining a similar mutual exclusion loss value according to the candidate frame set associated with different target objects;
and adjusting parameters of the detection model in the training process according to the similar mutual exclusion loss value so as to enlarge the distance between different candidate frame sets.
As an optional implementation manner, the detection target unit 901 is specifically configured to:
determining a plurality of candidate frame groups from a first candidate frame set associated with a first target object and a second candidate frame set associated with a second target object;
and determining the similar mutual exclusion loss value according to the distance between the first candidate frame and the second candidate frame in each candidate frame group.
Based on the same inventive concept, an embodiment of the present invention further provides a computer storage medium, on which a computer program is stored, which when executed by a processor implements the following steps:
acquiring a target image to be detected;
inputting the target image into a trained detection model, and outputting the position of a target object in the target image;
the detection model is obtained by training a labeled first sample set and an unlabeled second sample set, a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set are extracted during each training, the first low-dimensional feature map and the second low-dimensional feature map are fused to obtain a fused feature map, and the positions of target objects labeled by the fused feature map and the first sample image are used for training; wherein the first sample set and the second sample set both contain target objects, the second sample set containing different scene characteristics than the first sample set.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (11)

1. A method of object detection, the method comprising:
acquiring a target image to be detected;
inputting the target image into a trained detection model, and outputting the position of a target object in the target image;
the detection model is obtained by training a labeled first sample set and an unlabeled second sample set, a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set are extracted during each training, the first low-dimensional feature map and the second low-dimensional feature map are fused to obtain a fused feature map, and the positions of target objects labeled by the fused feature map and the first sample image are used for training; wherein the first sample set and the second sample set both contain target objects, the second sample set containing different scene characteristics than the first sample set.
2. The method according to claim 1, wherein the fusing the first low-dimensional feature map and the second low-dimensional feature map to obtain a fused feature map comprises:
determining the fused feature map according to a first mean value and a first standard deviation of the first low-dimensional feature map on each channel of the feature extraction layer and a second mean value and a second standard deviation of the second low-dimensional feature map on each channel of the feature extraction layer;
wherein the feature extraction layer is a convolution layer for extracting low-dimensional features in the detection model.
3. The method of claim 2, wherein determining the fused feature map comprises:
normalizing the first low-dimensional feature map according to the first mean value and the first standard deviation to obtain a normalized first low-dimensional feature map;
and determining the fusion feature map by taking the second mean value as the offset of the normalized first low-dimensional feature map and the second standard deviation as the scaling weight of the normalized first low-dimensional feature map.
4. The method of claim 1, wherein the detection model, upon each training, further comprises:
determining a domain classification loss value according to the first low-dimensional feature map, the second low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in the training process according to the domain classification loss value so as to extract the domain-invariant features.
5. The method of claim 4, wherein the adjusting parameters of the detection model in the training process according to the domain classification loss value to extract domain-invariant features comprises:
and performing back propagation on the gradient corresponding to the domain classification loss value by using a gradient inversion layer, and adjusting parameters of the detection model in a training process to extract the feature with unchanged domain.
6. The method of claim 1, wherein the detection model, upon each training, further comprises:
determining a similarity loss value according to the semantic similarity of the first low-dimensional feature map and the fusion feature map;
and adjusting parameters of the detection model in a training process according to the similarity loss value so as to constrain semantic consistency of the first low-dimensional feature map and the fusion feature map.
7. The method of claim 1, wherein the detection model, upon each training, further comprises:
determining a candidate box set associated with a target object in the fusion feature map;
determining a similar mutual exclusion loss value according to the candidate frame set associated with different target objects;
and adjusting parameters of the detection model in the training process according to the similar mutual exclusion loss value so as to enlarge the distance between different candidate frame sets.
8. The method of claim 7, wherein determining homogeneous mutual exclusion loss values according to the candidate box sets associated with different target objects comprises:
determining a plurality of candidate frame groups from a first candidate frame set associated with a first target object and a second candidate frame set associated with a second target object;
and determining the similar mutual exclusion loss value according to the distance between the first candidate frame and the second candidate frame in each candidate frame group.
9. A method for training a test model, the method comprising:
acquiring training samples of a detection model to be trained, wherein the training samples comprise a labeled first sample set and an unlabeled second sample set; the first sample set and the second sample set both contain target objects, the second sample set containing scene features different from the first sample set;
and extracting a first low-dimensional feature map of a first sample image in the first sample set and a second low-dimensional feature map of a second sample image in the second sample set during each training, fusing the first low-dimensional feature map and the second low-dimensional feature map to obtain a fused feature map, and training an initial detection model by using the fused feature map and the position of a target object marked by the first sample image to obtain a trained detection model.
10. An apparatus for object detection, comprising a processor and a memory, the memory storing a program executable by the processor, the processor being configured to read the program from the memory and to perform the steps of the method of any one of claims 1 to 8 or 9.
11. A computer storage medium having a computer program stored thereon, the program, when executed by a processor, implementing the steps of the method according to any one of claims 1 to 8 or 9.
CN202210807357.5A 2022-07-11 2022-07-11 Target detection method and device Withdrawn CN114882372A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210807357.5A CN114882372A (en) 2022-07-11 2022-07-11 Target detection method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210807357.5A CN114882372A (en) 2022-07-11 2022-07-11 Target detection method and device

Publications (1)

Publication Number Publication Date
CN114882372A true CN114882372A (en) 2022-08-09

Family

ID=82683478

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210807357.5A Withdrawn CN114882372A (en) 2022-07-11 2022-07-11 Target detection method and device

Country Status (1)

Country Link
CN (1) CN114882372A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116883765A (en) * 2023-09-07 2023-10-13 腾讯科技（深圳）有限公司 Image classification method, device, electronic equipment and storage medium

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104951800A (en) * 2015-06-15 2015-09-30 许昌学院 Resource exploitation-type area-oriented remote sensing image fusion method
CN110414631A (en) * 2019-01-29 2019-11-05 腾讯科技（深圳）有限公司 Lesion detection method, the method and device of model training based on medical image
CN112465140A (en) * 2020-12-07 2021-03-09 电子科技大学 Convolutional neural network model compression method based on packet channel fusion
CN112541928A (en) * 2020-12-18 2021-03-23 上海商汤智能科技有限公司 Network training method and device, image segmentation method and device and electronic equipment
CN112651441A (en) * 2020-12-25 2021-04-13 深圳市信义科技有限公司 Fine-grained non-motor vehicle feature detection method, storage medium and computer equipment
CN112801236A (en) * 2021-04-14 2021-05-14 腾讯科技（深圳）有限公司 Image recognition model migration method, device, equipment and storage medium
CN114187550A (en) * 2021-12-10 2022-03-15 电子科技大学长三角研究院(湖州) Bow net core part identification method based on improved YOLO V3 network
WO2022111219A1 (en) * 2020-11-30 2022-06-02 华南理工大学 Domain adaptation device operation and maintenance system and method
US20220189083A1 (en) * 2021-09-09 2022-06-16 Beijing Baidu Netcom Science Technology Co., Ltd. Training method for character generation model, character generation method, apparatus, and medium

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104951800A (en) * 2015-06-15 2015-09-30 许昌学院 Resource exploitation-type area-oriented remote sensing image fusion method
CN110414631A (en) * 2019-01-29 2019-11-05 腾讯科技（深圳）有限公司 Lesion detection method, the method and device of model training based on medical image
WO2022111219A1 (en) * 2020-11-30 2022-06-02 华南理工大学 Domain adaptation device operation and maintenance system and method
CN112465140A (en) * 2020-12-07 2021-03-09 电子科技大学 Convolutional neural network model compression method based on packet channel fusion
CN112541928A (en) * 2020-12-18 2021-03-23 上海商汤智能科技有限公司 Network training method and device, image segmentation method and device and electronic equipment
CN112651441A (en) * 2020-12-25 2021-04-13 深圳市信义科技有限公司 Fine-grained non-motor vehicle feature detection method, storage medium and computer equipment
CN112801236A (en) * 2021-04-14 2021-05-14 腾讯科技（深圳）有限公司 Image recognition model migration method, device, equipment and storage medium
US20220189083A1 (en) * 2021-09-09 2022-06-16 Beijing Baidu Netcom Science Technology Co., Ltd. Training method for character generation model, character generation method, apparatus, and medium
CN114187550A (en) * 2021-12-10 2022-03-15 电子科技大学长三角研究院(湖州) Bow net core part identification method based on improved YOLO V3 network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WANG JIAWEI等: "Decouple and Reconstruct: Mining Discriminative Features for Crossdomain Object Detection", 《ICLR 2022 CONFERENCE》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116883765A (en) * 2023-09-07 2023-10-13 腾讯科技（深圳）有限公司 Image classification method, device, electronic equipment and storage medium
CN116883765B (en) * 2023-09-07 2024-01-09 腾讯科技（深圳）有限公司 Image classification method, device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
Chen et al. Disparity-based multiscale fusion network for transportation detection
Wang et al. Embedding structured contour and location prior in siamesed fully convolutional networks for road detection
CN108961327A (en) A kind of monocular depth estimation method and its device, equipment and storage medium
CN105512683B (en) Object localization method and device based on convolutional neural networks
Li et al. Robust visual tracking based on convolutional features with illumination and occlusion handing
CN113240691A (en) Medical image segmentation method based on U-shaped network
CN112699834B (en) Traffic identification detection method, device, computer equipment and storage medium
CN110827312B (en) Learning method based on cooperative visual attention neural network
CN112036514B (en) Image classification method, device, server and computer readable storage medium
Xing et al. Traffic sign recognition using guided image filtering
CN113221770B (en) Cross-domain pedestrian re-recognition method and system based on multi-feature hybrid learning
CN110222572A (en) Tracking, device, electronic equipment and storage medium
CN110852327A (en) Image processing method, image processing device, electronic equipment and storage medium
CN114549557A (en) Portrait segmentation network training method, device, equipment and medium
CN112861776A (en) Human body posture analysis method and system based on dense key points
Wang et al. Combining semantic scene priors and haze removal for single image depth estimation
CN114882372A (en) Target detection method and device
CN114596548A (en) Target detection method, target detection device, computer equipment and computer-readable storage medium
CN116824333A (en) Nasopharyngeal carcinoma detecting system based on deep learning model
Mao et al. Power transmission line image segmentation method based on binocular vision and feature pyramid network
Krajewski et al. VeGAN: Using GANs for augmentation in latent space to improve the semantic segmentation of vehicles in images from an aerial perspective
Tabelini et al. Deep traffic sign detection and recognition without target domain real images
Lin et al. Mlf-det: Multi-level fusion for cross-modal 3d object detection
CN112634331A (en) Optical flow prediction method and device
Huang et al. An anti-occlusion and scale adaptive kernel correlation filter for visual object tracking

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication
WW01 Invention patent application withdrawn after publication

Application publication date: 20220809