WO2021051887A1 - Procédé et dispositif de criblage d'échantillons difficiles - Google Patents

Procédé et dispositif de criblage d'échantillons difficiles Download PDF

Info

Publication number
WO2021051887A1
WO2021051887A1 PCT/CN2020/094109 CN2020094109W WO2021051887A1 WO 2021051887 A1 WO2021051887 A1 WO 2021051887A1 CN 2020094109 W CN2020094109 W CN 2020094109W WO 2021051887 A1 WO2021051887 A1 WO 2021051887A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target area
missed
area image
label
Prior art date
Application number
PCT/CN2020/094109
Other languages
English (en)
Chinese (zh)
Inventor
马贤忠
董维山
江浩
胡皓瑜
范一磊
Original Assignee
初速度（苏州）科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 初速度（苏州）科技有限公司 filed Critical 初速度（苏州）科技有限公司
Publication of WO2021051887A1 publication Critical patent/WO2021051887A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features

Definitions

  • the present invention relates to the technical field of intelligent driving, and in particular to a method and device for screening difficult samples.
  • Deep learning relies on a large amount of training data or samples, but when the number of samples reaches a certain scale, the potential of different newly added sample images to improve the performance of the model is different.
  • difficult samples are samples that include missed targets and falsely detected targets, which are valuable data for improving the performance of the target detection model.
  • object detection object detection
  • difficult samples are samples that include missed targets and falsely detected targets, which are valuable data for improving the performance of the target detection model.
  • the present invention provides a method and device for screening difficult samples to realize automatic screening of difficult samples.
  • the specific technical solutions are as follows:
  • an embodiment of the present invention provides a method for screening difficult samples, including:
  • the target detection model is: Containing the area where the target is located and the confidence that the detected target is located in the area where the target exists, the first missed target area image is: an image of an area with a corresponding confidence that is lower than a preset threshold;
  • the image to be screened that contains at least one first missed target area image whose corresponding target label is a missed label is determined as a difficult sample image.
  • the step of determining the target label corresponding to each first missed target area image based on the image feature of each first missed target area image and the corresponding relationship established in advance includes:
  • the target label corresponding to each first missed target area image is determined.
  • the step of determining the candidate label corresponding to each first missed target area image based on the image feature of each first missed target area image and the corresponding relationship established in advance includes:
  • a candidate label corresponding to each first missed target region image is determined.
  • the step of determining a candidate label corresponding to each first missed target area image based on the similarity value includes:
  • the first preset number of labels in the label queue corresponding to the first missed target area image are determined as candidate labels corresponding to the suspected target area image.
  • the step of determining the target label corresponding to each first missed target area image based on the candidate label corresponding to each first missed target area image includes:
  • the meeting the preset statistical condition includes: greater than a preset number threshold, or a ratio of the total number of candidate tags corresponding to the corresponding first missed target area image Greater than the preset ratio threshold;
  • the target label corresponding to the first missed target area image is a non-missed label.
  • the step of using a pre-established target detection model to detect each obtained image to be screened, and determining the image to be screened that includes at least one first missed target region image includes:
  • the area image corresponding to the candidate target area of the rectangular area will be determined as the first missed target area image
  • the area image corresponding to the smallest rectangular area containing the candidate target area is determined as the first missed target area image to determine the image that contains at least one first missed target area The image to be filtered.
  • the method before the step of determining the candidate label corresponding to each first missed target area image based on the image feature of each first missed target area image and the corresponding relationship established in advance, the method further include:
  • the annotation information includes: the annotation location information of the area where the target contained in the corresponding establishment image is located;
  • each established image is detected, and each established image including at least one second missed target area image and detection position information corresponding to the at least one second missed target area image are determined, wherein ,
  • the at least one second missed-detected target area image is an area image with a corresponding confidence level lower than the preset threshold;
  • the The second missed detection of the label corresponding to the target area image For each second missed target area image, based on the detection location information corresponding to the second missed target area image, and the label location information in the label information corresponding to the established image where the second missed target area image is located, the The second missed detection of the label corresponding to the target area image to establish the corresponding relationship.
  • the second missed target area image is based on the detection position information corresponding to the second missed target area image, and the second missed target area image is located in the annotation information corresponding to the established image
  • the step of marking the position information and determining the label corresponding to the second missed target area image to establish the corresponding relationship includes:
  • the The intersection ratio between the label frame and the detection frame corresponding to the second missed target area image For each second missed target area image, based on the detection location information corresponding to the second missed target area image, and the label location information in the label information corresponding to the established image where the second missed target area image is located, the The intersection ratio between the label frame and the detection frame corresponding to the second missed target area image;
  • intersection ratio corresponding to the second missed target area image is not less than the preset intersection ratio threshold, determine that the label corresponding to the second missed target area image is the missed label
  • intersection ratio corresponding to the second missed target area image is less than the preset intersection ratio threshold, the label corresponding to the second missed target area image is determined to be a non-missed label to establish the corresponding relationship.
  • an embodiment of the present invention provides a difficult sample screening device, including:
  • the first determining module is configured to use a pre-established target detection model to detect each obtained image to be screened, and determine the image to be screened containing at least one first missed target area image, wherein the target detection
  • the model is: used to detect the area where the target is contained in the image and determine the confidence of the existence of the target in the area where the detected target is located, and the first missed target area image is: an image of an area with a corresponding confidence lower than a preset threshold;
  • the second determining module is configured to perform image feature extraction on each first missed target area image, and determine the image feature of each first missed target area image;
  • the third determining module is configured to determine the target label corresponding to each first missed target area image based on the image feature of each first missed target area image and the pre-established corresponding relationship, wherein the corresponding relationship includes : Correspondence between the image features of the annotated images and their corresponding labels;
  • the fourth determining module is configured to determine the image to be screened containing at least one first missed target area image corresponding to the missed label as the difficult sample image.
  • the third determining module includes:
  • the first determining unit is configured to determine the candidate label corresponding to each first missed target area image based on the image feature of each first missed target area image and the corresponding relationship established in advance;
  • the second determining unit is configured to determine the target label corresponding to each first missed target area image based on the candidate label corresponding to each first missed target area image.
  • the first determining unit includes:
  • the first determining sub-module is configured to determine the first missed target based on the image feature of the first missed target area image and the image feature of each marked image for each first missed target area image The similarity value between the regional image and each marked image;
  • the second determining sub-module is configured to determine, based on the similarity value, a candidate label corresponding to each first missed target area image.
  • the second determining sub-module is specifically configured to, for each first missed target area image, according to the similarity value between the first missed target area image and each marked image, which is larger In the smallest order, arrange the labels corresponding to each marked image to obtain the label queue corresponding to the first missed target area image;
  • the first preset number of labels in the label queue corresponding to the first missed target area image are determined as candidate labels corresponding to the suspected target area image.
  • the second determining unit is specifically configured to, for each first missed target area image, count the candidate labels corresponding to the first missed target area image, which are the candidate labels of the missed label The quantity of, as the first quantity;
  • the meeting the preset statistical condition includes: greater than a preset number threshold, or a ratio of the total number of candidate tags corresponding to the corresponding first missed target area image Greater than the preset ratio threshold;
  • the target label corresponding to the first missed target area image is a non-missed label.
  • the first determining module is specifically configured to use a pre-established target detection model to detect each obtained image to be screened, determine the image to be screened containing at least one suspected target area, and determine each A confidence level corresponding to the suspected target area;
  • the area image corresponding to the candidate target area of the rectangular area will be determined as the first missed target area image
  • the area image corresponding to the smallest rectangular area containing the candidate target area is determined as the first missed target area image to determine the image that contains at least one first missed target area The image to be filtered.
  • the device further includes:
  • the relationship establishment module is configured to establish a corresponding relationship before determining the candidate label corresponding to each first missed target area image based on the image characteristics of each first missed target area image and the pre-established corresponding relationship
  • the process of, wherein the relationship establishment module includes:
  • the obtaining unit is configured to obtain the establishment image and the annotation information corresponding to each establishment image, wherein the annotation information includes: the annotation location information of the area where the target contained in the corresponding establishment image is located;
  • the third determining unit is configured to use the target detection model to detect each established image, and determine each established image including at least one second missed target area image and the at least one second missed target area Detection position information corresponding to the image, wherein the at least one second missed-detected target area image is an area image with a corresponding confidence level lower than the preset threshold;
  • the fourth determining unit is configured to perform image feature extraction on each second missed target area image, and determine the image feature of each second missed target area image;
  • the fifth determining unit is configured to, for each second missed target area image, based on the detection position information corresponding to the second missed target area image, and the annotation information corresponding to the established image where the second missed target area image is located To determine the label corresponding to the second miss-detected target area image in order to establish and obtain the corresponding relationship.
  • the fifth determining unit is specifically configured to, for each second missed target area image, based on the detection position information corresponding to the second missed target area image, and the second missed target area image
  • the label location information in the label information corresponding to the established image is determined to determine the intersection ratio between the label frame and the detection frame corresponding to the second missed-detected target area image;
  • intersection ratio corresponding to the second missed target area image is not less than the preset intersection ratio threshold, determine that the label corresponding to the second missed target area image is the missed label
  • intersection ratio corresponding to the second missed target area image is less than the preset intersection ratio threshold, the label corresponding to the second missed target area image is determined to be a non-missed label to establish the corresponding relationship.
  • the difficult sample screening method and device can use a pre-established target detection model to detect each obtained image to be screened, and determine that it contains at least one first missed target
  • the to-be-screened image of the area image where the target detection model is: used to detect the area of the target contained in the image and determine the confidence of the existence of the target in the area of the detected target, the first missed target area image is: the corresponding confidence Area images below a preset threshold; perform image feature extraction on each first missed target area image to determine the image features of each first missed target area image; images based on each first missed target area image Feature and the pre-established correspondence relationship, determine the target label corresponding to each first missed target area image, where the correspondence relationship includes the correspondence relationship between the image features of the labeled image and the corresponding label; it will contain at least one corresponding
  • the target label is the image to be screened of the first missed target area image of the missed label, and is determined to be the difficult sample image.
  • the first missed target area image of the label contains the missed target, and the image to be screened that contains at least one first missed target area image whose corresponding target label is the missed label is determined as a difficult sample image.
  • the image of the missed target area is extracted.
  • the content of the partial image block of the screened image is used as the main body of the search to avoid the interference of irrelevant information, effectively improve the accuracy of the search and recognition, and save the memory and increase the speed, so as to realize the automatic screening of difficult samples.
  • any product or method of the present invention does not necessarily need to achieve all the advantages described above at the same time.
  • each first missed target is determined
  • the target label corresponding to the area image where the target label may include a missed detection label indicating that the first missed detection target area image contains the missed target; further, the corresponding target label can be considered as the first missed detection label of the missed detection target.
  • the detected target area image contains the missed target, and the image to be screened that contains at least one first missed target area image whose corresponding target label is the missed detection label is determined as a difficult sample image.
  • the image of the missed target area is extracted.
  • the content of the partial image block of the screened image is used as the main body of the search to avoid the interference of irrelevant information, effectively improve the accuracy of the search and recognition, and save the memory and increase the speed, so as to realize the automatic screening of difficult samples.
  • the target label corresponding to each first missed target area image you can first determine the similarity value between the first missed target area image and each marked image, and then, for each first missed target area image The detection target area image, based on its corresponding similarity value, determines the labels corresponding to the previously preset number of labeled images that are most similar to the first missed detection target area image, and the labels determined above are used as the first The candidate label corresponding to the missed target area image; based on the candidate label corresponding to the first missed target area image, the target label corresponding to the first missed target area image is determined to improve the determined target label to a certain extent accuracy.
  • FIG. 1 is a schematic flowchart of a method for screening difficult samples according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of a process for establishing a corresponding relationship according to an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of a difficult sample screening device provided by an embodiment of the present invention.
  • the present invention provides a method and device for screening difficult samples to realize automatic screening of difficult samples.
  • the embodiments of the present invention will be described in detail below.
  • FIG. 1 is a schematic flowchart of a method for screening difficult samples according to an embodiment of the present invention. The method can include the following steps:
  • S101 Use a pre-established target detection model to detect each obtained image to be screened, and determine the image to be screened that includes at least one first missed target region image.
  • the target detection model is: used to detect the area of the target contained in the image and determine the confidence that the detected target exists in the area of the target, and the first missed target area image is: the corresponding confidence is lower than the preset threshold image.
  • the target detection model is: a network model trained based on an image marked with a target to be detected.
  • the method can be applied to any type of electronic device with computing capability, and the electronic device can be a server or a terminal device.
  • the pre-established target detection model can be a neural network model, for example: it can be a convolutional neural network model, specifically it can be Faster R-CNN (Faster Region-Convolutional Neural Networks, fast region-convolution) Neural network model) and YOLO (You Only Look Once) model.
  • the pre-established target detection model can be any type of neural network model that can detect the location of the target in the image in related technologies. The specific types of pre-established target detection models are defined. For the training method of the pre-established target detection model, reference may be made to related technologies, and the embodiment of the present invention does not specifically limit it.
  • the target to be detected may be any type of target, including but not limited to lane lines, vehicles, traffic lights, signs, and/or pedestrians.
  • the electronic device after the electronic device obtains one or more frames of images to be screened, it can use a pre-established target detection model to detect each of the obtained images to be screened, and separate the regions where the target may exist in the image to be screened. Identify and determine the respective confidence level of each area where the target may exist; cut out the area image corresponding to the identified area where the target may exist, and subsequently, based on the corresponding confidence level of each area image, The region image whose corresponding confidence is lower than the preset threshold is determined from the intercepted region image as the first missed detection target region image, and it can be determined from the obtained images to be screened that at least one first missed detection is included The image to be filtered of the target area image.
  • the confidence level can represent the possibility that the corresponding regional image has the target to be detected.
  • the lower the confidence of the region image is, the less likely it is that the target detection model predicts that there is a target to be detected in the region image of the image to be screened.
  • the confidence level corresponding to the region image is low, there is a possibility that the region image contains the missed target to be detected.
  • the intercepted region image corresponding to the region where the target to be detected may exist may include the region image whose corresponding confidence is within a preset confidence threshold, and the lower limit of the preset confidence threshold is 0, The upper limit is greater than or equal to the aforementioned preset threshold.
  • the electronic device may mark and record the correspondence between the to-be-screened image and the at least one first missed-detected target area image contained in it. To be used in subsequent processes.
  • S102 Perform image feature extraction on each first missed target area image, and determine the image feature of each first missed target area image.
  • the electronic device can use any type of preset feature extraction algorithm to perform image feature extraction on each first missed target area image to determine the image feature of each first missed target area image.
  • the preset feature extraction algorithm may include, but is not limited to, SIFT (Scale-invariant feature transform) feature extraction algorithm, HOG (Histogram of Oriented Gradient, directional gradient histogram) feature extraction algorithm, Harr feature extraction Algorithm and GIST (Global Feature) extraction algorithm, etc.
  • the preset feature extraction algorithm can also be a feature extraction algorithm of convolutional neural network.
  • S103 Determine the target label corresponding to each first missed target area image based on the image feature of each first missed target area image and the pre-established corresponding relationship.
  • the correspondence relationship includes: the correspondence relationship between the image features of the annotated image and their corresponding tags.
  • the pre-established correspondence relationship may be stored locally or in a storage device connected to the electronic device, and the correspondence relationship includes: the correspondence relationship between the image features of the marked image and the corresponding label, and the marked image may include
  • the pre-established target detection model cuts out from the original image in which the corresponding confidence is lower than the preset threshold region image, the original image may refer to the later-mentioned established image; the marked image may also include: The captured image contains or does not contain the target to be detected. In this case, correspondingly, in order to ensure the accuracy of the difficult sample screening process, the marked image may include only the target to be detected, or only the target that is not to be detected. .
  • the label corresponding to each labeled image may include: a missed detection label that characterizes that the labeled image contains a target to be detected that is missed by the pre-established target detection model, or that the labeled image does not contain a target that is pre-established
  • the non-missed label of the target to be detected that is missed by the detection model for example: the label corresponding to the marked image may be a label that characterizes that the marked image includes a lane line, that is, a missed label, that is, the content of the missed label can be: "Lane line", or a label indicating that the lane line is not included in the marked image, that is, a non-missing label, and the content of the non-missing label may be "non-lane line".
  • the pre-established correspondence relationship may be stored in a preset index database, so as to compare and match the image feature of the first missed-detected target area image with the image feature of the marked image in the correspondence relationship.
  • the electronic device may match the image feature of the first missed target area image with the image feature of each labeled image in the corresponding relationship for each first missed target area, and the corresponding relationship is with The label corresponding to the image feature that most matches the image feature of the first missed target area image is determined to be the target label corresponding to the first missed target area image.
  • the above-mentioned matching process may be: based on a preset similarity algorithm, calculating the similarity value between the image feature of the first missed-detected target area image and the image feature of each labeled image in the corresponding relationship, correspondingly,
  • the image feature that best matches the image feature of the first missed target area image in the above correspondence may refer to the image feature with the largest similarity value between the image feature of the first missed target area image in the corresponding relationship.
  • the preset similarity algorithms include but are not limited to: Euclidean distance, cosine distance, Min-type distance, correlation coefficient and other algorithms.
  • the S103 may include the following steps 01-02:
  • the electronic device may, for each first missed target area image, based on the image feature of the first missed target area image and the image feature of the labeled image in the pre-established correspondence, from the corresponding relationship , It is determined that a plurality of labeled images that match the image feature of the first missed target area image is determined, and the labels corresponding to the multiple labeled images that match the image feature of the first missed target area image are determined as the The candidate label corresponding to the first missed target area image, and further, based on the candidate label corresponding to each first missed target area image, the target label corresponding to each first missed target area image is determined. In order to improve the accuracy of the target label corresponding to each first missed target area image determined to a certain extent.
  • the 01 may include the following steps 011-012:
  • 012 Determine the candidate label corresponding to each first missed target area image based on the similarity value.
  • the electronic device calculates the image based on the preset similarity algorithm, the image features of the first missed target area image, and the image features of each labeled image.
  • the similarity value between the first missed target area image and each labeled image, and then, based on the similarity value, the candidate label corresponding to each first missed target area image is determined, for example: the corresponding similarity
  • the preset number of labels corresponding to the marked image with the largest value is determined as the candidate label corresponding to the first missed target area image.
  • the 012 may include the following steps:
  • the first preset number of labels in the label queue corresponding to the first missed target area image are determined as candidate labels corresponding to the suspected target area image.
  • the electronic device may for each first missed target area image, according to the first missed target area image and The similarity value between each marked image is in descending order, and the label corresponding to each marked image is arranged to obtain the label queue corresponding to the first missed target area image; further, from the first missed detection The first preset number of tags in the tag queue corresponding to the target area image are determined as candidate tags corresponding to the suspected target area image.
  • it can be: for each first missed-detected target area image, according to the order of the similarity value between the first missed-detected target area image and each marked image, arrange the corresponding to each marked image To obtain another label queue corresponding to the first missed target area image; further, a preset number of labels from another label queue corresponding to the first missed target area image are determined to be the suspect target area
  • the candidate label corresponding to the image is also possible.
  • the preset number is a preset number, and it can also be set independently by the electronic device according to the number of image features of the marked image contained in the pre-established correspondence relationship, which is all right.
  • the 02 may include the following steps 021-224:
  • each candidate label corresponding to the image of the first missed detection target area may include a missed detection label and/or a non-missed detection label.
  • the electronic device may, for each first missed target area image, count the number of candidate labels corresponding to the first missed target area image, which is the missed label, as the first quantity ; And determine whether the first number meets the preset statistical conditions, that is, whether the first number is greater than the preset number threshold, or whether the ratio of the first number to the total number of candidate labels corresponding to the first missed target area image is determined Greater than the preset ratio threshold.
  • the first number is greater than the preset number threshold, or it is determined that the ratio of the first number to the total number of candidate labels corresponding to the first missed target area image is greater than the preset ratio threshold, then it is determined that the first number meets the preset threshold. Assuming the statistical condition, that is, among the candidate labels corresponding to the first missed target area image, the proportion of labels representing the first missed target area image containing the missed target to be detected is relatively large, and accordingly, the first missed target area image can be determined.
  • the target label corresponding to the detected target area image is a missed label; on the contrary, if it is determined that the first number is not greater than the preset number threshold, or it is determined that the first number corresponds to the total number of candidate labels corresponding to the first missed target area image If the ratio is not greater than the preset ratio threshold, it is determined that the first number does not meet the preset statistical conditions, that is, among the candidate labels corresponding to the first missed target area image, the first missed target area image contains the missed target to be detected The proportion of the labels of is small, and accordingly, it can be determined that the target label corresponding to the first missed target area image is a non-missed label.
  • the electronic device can set a weight value when determining the target label corresponding to each first missed target area image based on the candidate label corresponding to each first missed target area image, where the image of the marked image The greater the similarity value between the feature and the image feature of the first missed-detected target region image, the greater the weight value corresponding to the label corresponding to the labeled image.
  • the corresponding target label is a non-missing label. Wherein, it may be: the value corresponding to the candidate label of the missing label is 1, and the value corresponding to the candidate label of the non-missing label is 0.
  • S104 Determine an image to be screened that includes at least one first missed target area image whose corresponding target label is a missed label as a difficult sample image.
  • each image to be screened may not include the first missing-detected target area image, or may include at least one first missing-detected target area image.
  • the to-be-screened image includes at least one first missing-detected target area image, then It can be considered that the image to be screened contains the target to be detected that is missed by the pre-established target detection model, and the electronic device can determine the image to be screened as a difficult sample.
  • the difficult sample can be re-stored and annotated, and the corresponding relationship between each first missed target area image and its corresponding image to be screened can be saved. Furthermore, using the difficult sample and its labeling information to continue training the pre-established target detection model, that is, using the difficult sample and its labeling information to update the parameters of the pre-established target detection model, so as to improve the performance of the pre-established target detection model. Detection accuracy.
  • the first missed target area image of the label contains the missed target, and the image to be screened that contains at least one first missed target area image whose corresponding target label is the missed label is determined as a difficult sample image.
  • the image of the missed target area is extracted.
  • the content of the partial image block of the screened image is used as the main body of the search to avoid the interference of irrelevant information, effectively improve the accuracy of the search and recognition, and save the memory and increase the speed, so as to realize the automatic screening of difficult samples.
  • the S104 may include the following steps 11-14:
  • the candidate target area is a rectangular area
  • the area image corresponding to the candidate target area of the rectangular area will be determined as the first missed target area image
  • the candidate target area is a non-rectangular area, determine the area image corresponding to the smallest rectangular area containing the candidate target area as the first missed target area image to determine that it contains at least one first missed target The image to be filtered of the area image.
  • the electronic device may use a pre-established target detection model to detect each image to be screened, determine the image to be screened containing at least one suspected target area, and determine the confidence level corresponding to each suspected target area.
  • the at least one suspected target area is the aforementioned area that may include the target to be detected, and the image block represented by each suspected target area may be referred to as a regional image.
  • the suspected target area determined by the electronic device is an area whose corresponding confidence is within a preset confidence threshold.
  • the upper limit of the preset reliability threshold may not be less than the preset threshold, and the lower limit may be 0.
  • the electronic device determines a suspected target area whose corresponding confidence is lower than a preset threshold from the suspected target area as a candidate target area. And determine whether each candidate target area is a rectangle. If the candidate target area is a rectangular area, the area image corresponding to the candidate target area of the rectangular area will be determined as the first missed target area image; if the candidate target area is The area is a non-rectangular area, and the area image corresponding to the smallest rectangular area containing the candidate target area is determined as the first missed target area image to determine the image to be screened containing at least one first missed target area image .
  • the method may further include:
  • the process of establishing the corresponding relationship may include:
  • S201 Obtain established images and annotation information corresponding to each established image.
  • the labeling information includes: labeling location information of the area where the target contained in the corresponding established image is located.
  • S202 Use the target detection model to detect each established image, and determine each established image including at least one second missed target area image and detection position information corresponding to the at least one second missed target area image.
  • At least one second missed-detected target area image is an area image with a corresponding confidence level lower than a preset threshold
  • S203 Perform image feature extraction on each second missed target area image, and determine the image feature of each second missed target area image.
  • the electronic device may also include a process of establishing a corresponding relationship.
  • the electronic device can obtain multiple images for establishing the corresponding relationship.
  • the embodiment of the present invention is called an established image.
  • the established image containing the target to be detected can be marked with the area where the target to be detected is located, and the established image containing the target to be detected can be marked.
  • the corresponding labeling information contains the position information of the target to be detected in the corresponding established image, which can be referred to as labeling position information.
  • Input the established image and its corresponding annotation information into the pre-established target detection model use the pre-established target detection model to detect each established image, and determine each established image that contains at least one second missed target area image and Detection location information corresponding to at least one second missed-detected target area image; wherein each second missed-detected target area image is an image of an area with a corresponding confidence level lower than a preset threshold.
  • the electronic device After the electronic device obtains the at least one second missed target area image, it uses a preset feature extraction algorithm to perform image feature extraction on the at least one second missed target area image to obtain the image feature of the at least one second missed target area image; And, for each second missed target area image, based on the detection location information corresponding to the second missed target area image, and the annotation location information in the annotation information corresponding to the established image where the second missed target area image is located, Determine the label corresponding to the second missed target area image.
  • the S204 may be: determining the detection frame corresponding to the detection position information corresponding to the second missed target area image, and the label in the annotation information corresponding to the established image where the second missed target area image is located Whether the overlapping area between the label boxes corresponding to the location information exceeds the preset area ratio, if it exceeds the preset area ratio, it can be considered that there is a pending target detection model missed by the pre-established target detection model in the second missed target area image Detect the target and determine that the label corresponding to the second missed target area image is the missed label; on the contrary, if it does not exceed the preset area ratio, it can be considered that there is no pre-established target detection model in the second missed target area image For the missed target to be detected, it is determined that the label corresponding to the image of the second missed target area is a non-missed label.
  • the S204 may include the following steps 021-224:
  • intersection ratio corresponding to the second missed target area image is not less than the preset intersection ratio threshold, determine that the label corresponding to the second missed target area image is the missed label
  • intersection ratio corresponding to the second missed target area image is less than the preset intersection ratio threshold, determine that the label corresponding to the second missed target area image is a non-missed label to establish a corresponding relationship.
  • the electronic device may, for each second missed target area image, be based on the detection position information corresponding to the second missed target area image, and the annotation information corresponding to the established image where the second missed target area image is located.
  • the label position information in the image to determine the intersection of the label frame and the detection frame corresponding to the second missed target area image and the ratio between the unions, that is, the intersection and union ratio, where the label frame corresponds to the second missed target area
  • the marked location information of the image, and the detection frame corresponds to the detection location information corresponding to the second missed-detected target area image.
  • the label corresponding to the second missed target area image is determined to be a missed label; otherwise, if it is judged to be less than, it is considered that there is no pre-established target detection model in the second missed target area image
  • the label corresponding to the second missed target area image is determined to be a non-missed label, and the corresponding relationship between the image characteristics of each second missed target area image and the corresponding label is recorded, In order to establish the corresponding relationship between the image features of the marked image and the corresponding label.
  • the labeled image includes the above-mentioned second miss-detected target area image.
  • an embodiment of the present invention provides a difficult sample screening device, as shown in FIG. 3, which may include:
  • the first determining module 310 is configured to use a pre-established target detection model to detect each obtained image to be screened, and determine the image to be screened containing at least one first missed target area image, wherein the target
  • the detection model is: used to detect the area of the target contained in the image and determine the confidence of the existence of the target in the area where the detected target is located, and the first missed target area image is: an image of an area with a corresponding confidence lower than a preset threshold ;
  • the second determining module 320 is configured to perform image feature extraction on each first missed target area image, and determine the image feature of each first missed target area image;
  • the third determining module 330 is configured to determine the target label corresponding to each first missed target area image based on the image feature of each first missed target area image and the corresponding relationship established in advance, wherein the corresponding relationship Including: the correspondence between the image features of the annotated images and their corresponding labels;
  • the fourth determining module 340 is configured to determine the image to be screened containing at least one first missed target area image corresponding to the missed label as the difficult sample image.
  • the first missed target area image of the label contains the missed target, and the image to be screened that contains at least one first missed target area image whose corresponding target label is the missed label is determined as a difficult sample image.
  • the image of the missed target area is extracted.
  • the content of the partial image block of the screened image is used as the main body of the search to avoid the interference of irrelevant information, effectively improve the accuracy of the search and recognition, and save the memory and increase the speed, so as to realize the automatic screening of difficult samples.
  • the third determining module 330 includes:
  • the first determining unit is configured to determine the candidate label corresponding to each first missed target area image based on the image feature of each first missed target area image and the corresponding relationship established in advance;
  • the second determining unit is configured to determine the target label corresponding to each first missed target area image based on the candidate label corresponding to each first missed target area image.
  • the first determining unit includes:
  • the first determining sub-module is configured to determine the first missed target based on the image feature of the first missed target area image and the image feature of each marked image for each first missed target area image The similarity value between the regional image and each marked image;
  • the second determining sub-module is configured to determine, based on the similarity value, a candidate label corresponding to each first missed target area image.
  • the second determining submodule is specifically configured to, for each first missed target area image, according to the difference between the first missed target area image and each marked image Arrange the labels corresponding to each marked image in descending order of the similarity value to obtain the label queue corresponding to the first missed-detected target area image;
  • the first preset number of labels in the label queue corresponding to the first missed target area image are determined as candidate labels corresponding to the suspected target area image.
  • the second determining unit is specifically configured to, for each first missed-detected target area image, count the candidate labels corresponding to the first missed-detected target area image.
  • the number of candidate labels for the inspection label shall be regarded as the first quantity;
  • the meeting the preset statistical condition includes: greater than a preset number threshold, or a ratio of the total number of candidate tags corresponding to the corresponding first missed target area image Greater than the preset ratio threshold;
  • the target label corresponding to the first missed target area image is a non-missed label.
  • the first determining module 310 is specifically configured to use a pre-established target detection model to detect each obtained image to be screened, determine the image to be screened containing at least one suspected target area, and determine The confidence level corresponding to each suspected target area;
  • the area image corresponding to the candidate target area of the rectangular area will be determined as the first missed target area image
  • the area image corresponding to the smallest rectangular area containing the candidate target area is determined as the first missed target area image to determine the image that contains at least one first missed target area The image to be filtered.
  • the device further includes:
  • the relationship establishment module is configured to establish a corresponding relationship before determining the candidate label corresponding to each first missed target area image based on the image characteristics of each first missed target area image and the pre-established corresponding relationship
  • the process of, wherein the relationship establishment module includes:
  • the obtaining unit is configured to obtain the establishment image and the annotation information corresponding to each establishment image, wherein the annotation information includes: the annotation location information of the area where the target contained in the corresponding establishment image is located;
  • the third determining unit is configured to use the target detection model to detect each established image, and determine each established image including at least one second missed target area image and the at least one second missed target area Detection position information corresponding to the image, wherein the at least one second missed-detected target area image is an area image with a corresponding confidence level lower than the preset threshold;
  • the fourth determining unit is configured to perform image feature extraction on each second missed target area image, and determine the image feature of each second missed target area image;
  • the fifth determining unit is configured to, for each second missed target area image, based on the detection position information corresponding to the second missed target area image, and the annotation information corresponding to the established image where the second missed target area image is located To determine the label corresponding to the second miss-detected target area image in order to establish and obtain the corresponding relationship.
  • the fifth determining unit is specifically configured to, for each second missed target area image, based on the detection position information corresponding to the second missed target area image, and the second missed target area image. 2.
  • the label location information in the label information corresponding to the established image where the image of the missed target area is located, and the intersection ratio between the label frame and the detection frame corresponding to the second missed target area image is determined;
  • intersection ratio corresponding to the second missed target area image is not less than the preset intersection ratio threshold, determine that the label corresponding to the second missed target area image is the missed label
  • intersection ratio corresponding to the second missed target area image is less than the preset intersection ratio threshold, the label corresponding to the second missed target area image is determined to be a non-missed label to establish the corresponding relationship.
  • modules in the device in the embodiment may be distributed in the device in the embodiment according to the description of the embodiment, or may be located in one or more devices different from this embodiment with corresponding changes.
  • the modules of the above-mentioned embodiments can be combined into one module, or further divided into multiple sub-modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

Des modes de réalisation de la présente invention concernent un procédé et un dispositif de criblage d'échantillons difficiles, le procédé consistant : à utiliser un modèle de détection d'objet préconstruit pour effectuer une détection sur chaque image obtenue à cribler et à déterminer une image à cribler contenant au moins une image d'une première zone cible non détectée, le modèle de détection d'objet représentant la confiance utilisée pour détecter la zone dans laquelle est située une cible contenue dans une image et pour déterminer que la cible est présente dans la zone détectée dans laquelle est située la cible ; à extraire des caractéristiques d'image de chaque image de la première zone cible non détectée ; en fonction de caractéristiques d'image de chaque image de la première zone cible non détectée et d'une correspondance préétablie, à déterminer une étiquette cible correspondant à chaque image de la première zone cible non détectée, la correspondance comprenant la correspondance entre des caractéristiques d'image d'images marquées et leurs marqueurs correspondants ; et à déterminer l'image à cribler, qui contient lesdites images de la première zone cible non détectée dont l'étiquette cible correspondante est une étiquette non détectée, sous forme d'image d'un échantillon difficile, de façon à obtenir le criblage automatique d'échantillons difficiles.
PCT/CN2020/094109 2019-09-20 2020-06-03 Procédé et dispositif de criblage d'échantillons difficiles WO2021051887A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910890908.7A CN112541372B (zh) 2019-09-20 2019-09-20 一种困难样本筛选方法及装置
CN201910890908.7 2019-09-20

Publications (1)

Publication Number Publication Date
WO2021051887A1 true WO2021051887A1 (fr) 2021-03-25

Family

ID=74883929

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/094109 WO2021051887A1 (fr) 2019-09-20 2020-06-03 Procédé et dispositif de criblage d'échantillons difficiles

Country Status (2)

Country Link
CN (1) CN112541372B (fr)
WO (1) WO2021051887A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113627413A (zh) * 2021-08-12 2021-11-09 杭州海康威视数字技术股份有限公司 数据标注方法、图像比对方法及装置
CN114445811A (zh) * 2022-01-30 2022-05-06 北京百度网讯科技有限公司 一种图像处理方法、装置及电子设备
CN117710756A (zh) * 2024-02-04 2024-03-15 成都数之联科技股份有限公司 一种目标检测及模型训练方法、装置、设备、介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115033731B (zh) * 2022-07-04 2023-07-18 小米汽车科技有限公司 图像检索方法、装置、电子设备及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109034190A (zh) * 2018-06-15 2018-12-18 广州深域信息科技有限公司 一种动态选择策略的主动样本挖掘的物体检测***及方法
CN110084113A (zh) * 2019-03-20 2019-08-02 阿里巴巴集团控股有限公司 活体检测方法、装置、***、服务器及可读存储介质
US20190279091A1 (en) * 2018-03-12 2019-09-12 Carnegie Mellon University Discriminative Cosine Embedding in Machine Learning

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107122378B (zh) * 2017-01-13 2021-03-16 北京星选科技有限公司 对象处理方法、装置及移动终端
CN109697449A (zh) * 2017-10-20 2019-04-30 杭州海康威视数字技术股份有限公司 一种目标检测方法、装置及电子设备
CN109871730A (zh) * 2017-12-05 2019-06-11 杭州海康威视数字技术股份有限公司 一种目标识别方法、装置及监控设备
CN108446707B (zh) * 2018-03-06 2020-11-24 北方工业大学 基于关键点筛选及dpm确认的遥感图像飞机检测方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190279091A1 (en) * 2018-03-12 2019-09-12 Carnegie Mellon University Discriminative Cosine Embedding in Machine Learning
CN109034190A (zh) * 2018-06-15 2018-12-18 广州深域信息科技有限公司 一种动态选择策略的主动样本挖掘的物体检测***及方法
CN110084113A (zh) * 2019-03-20 2019-08-02 阿里巴巴集团控股有限公司 活体检测方法、装置、***、服务器及可读存储介质

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113627413A (zh) * 2021-08-12 2021-11-09 杭州海康威视数字技术股份有限公司 数据标注方法、图像比对方法及装置
CN113627413B (zh) * 2021-08-12 2024-06-04 杭州海康威视数字技术股份有限公司 数据标注方法、图像比对方法及装置
CN114445811A (zh) * 2022-01-30 2022-05-06 北京百度网讯科技有限公司 一种图像处理方法、装置及电子设备
CN117710756A (zh) * 2024-02-04 2024-03-15 成都数之联科技股份有限公司 一种目标检测及模型训练方法、装置、设备、介质
CN117710756B (zh) * 2024-02-04 2024-04-26 成都数之联科技股份有限公司 一种目标检测及模型训练方法、装置、设备、介质

Also Published As

Publication number Publication date
CN112541372B (zh) 2023-03-28
CN112541372A (zh) 2021-03-23

Similar Documents

Publication Publication Date Title
WO2021051887A1 (fr) Procédé et dispositif de criblage d'échantillons difficiles
US11455805B2 (en) Method and apparatus for detecting parking space usage condition, electronic device, and storage medium
US8902053B2 (en) Method and system for lane departure warning
CN109919002B (zh) 黄色禁停线识别方法、装置、计算机设备及存储介质
CN105321350A (zh) ***检测方法及装置
WO2020038138A1 (fr) Procédé et dispositif de marquage d'échantillon, et procédé et dispositif d'identification de catégorie d'endommagement
US11721088B2 (en) Image translation for image recognition to compensate for source image regional differences
CN111274926B (zh) 图像数据筛选方法、装置、计算机设备和存储介质
CN111191611A (zh) 基于深度学习的交通标志标号识别方法
CN111626275B (zh) 一种基于智能视频分析的异常停车检测方法
CN110751619A (zh) 一种绝缘子缺陷检测方法
CN113095301B (zh) 占道经营监测方法、***与服务器
CN111898491A (zh) 一种车辆逆向行驶的识别方法、装置及电子设备
WO2022048572A1 (fr) Procédé et appareil d'identification de cible, et dispositif électronique
CN115830399B (zh) 分类模型训练方法、装置、设备、存储介质和程序产品
CN109389105A (zh) 一种基于多任务的虹膜检测和视角分类方法
CN112733666A (zh) 一种难例图像的搜集、及模型训练方法、设备及存储介质
CN114926791A (zh) 一种路口车辆异常变道检测方法、装置、存储介质及电子设备
CN114429577A (zh) 一种基于高置信标注策略的旗帜检测方法及***及设备
CN114462469B (zh) 目标检测模型的训练方法、目标检测方法及相关装置
CN109684953B (zh) 基于目标检测和粒子滤波算法进行猪只跟踪的方法及装置
CN112581495A (zh) 图像处理方法、装置、设备及存储介质
CN112699711A (zh) 车道线检测方法、装置、存储介质及电子设备
CN112132892A (zh) 目标位置标注方法、装置及设备
CN114170269A (zh) 一种基于时空相关性的多目标跟踪方法、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20864611

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20864611

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20864611

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27.09.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20864611

Country of ref document: EP

Kind code of ref document: A1