CN111461056A - Sample data acquisition method and device - Google Patents
Sample data acquisition method and device Download PDFInfo
- Publication number
- CN111461056A CN111461056A CN202010293247.2A CN202010293247A CN111461056A CN 111461056 A CN111461056 A CN 111461056A CN 202010293247 A CN202010293247 A CN 202010293247A CN 111461056 A CN111461056 A CN 111461056A
- Authority
- CN
- China
- Prior art keywords
- video data
- labeling
- image
- images
- sample data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000002372 labelling Methods 0.000 claims abstract description 118
- 238000012545 processing Methods 0.000 claims abstract description 10
- 238000012937 correction Methods 0.000 claims abstract description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 12
- 238000013135 deep learning Methods 0.000 claims description 10
- 238000003062 neural network model Methods 0.000 claims description 10
- 238000005286 illumination Methods 0.000 claims description 7
- 238000013527 convolutional neural network Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a sample data acquisition method and device, relates to the technical field of data processing, and aims to improve the efficiency of acquiring sample data and reduce the cost of acquiring the sample data. The method comprises the following steps: acquiring video data acquired by a vehicle; labeling each frame of image in the video data through a preset labeling model, and acquiring a labeling result of each frame of image in the video data, wherein the labeling result comprises: marking content and marking confidence; extracting a target image set from the video data according to the labeling confidence coefficient; and receiving the correction of the labeled content of the images in the target image set, and acquiring sample data. The embodiment of the invention is used for acquiring the sample data.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for obtaining sample data.
Background
The automatic driving system is divided into six grades of L0-L5 according to the intelligentization degree from low to high, the current driving technology is developed to a L3 grade which allows the automatic driving system to replace a driver to independently drive the vehicle, and the L3 grade needs the automatic driving system to independently drive the vehicle, so the application scene of the system is more complex, and a large amount of sample data is needed to train and verify the automatic driving model.
Currently, the commonly adopted sample data acquisition mode is as follows: the method comprises the steps that a vehicle-mounted image acquisition device (such as a driving recorder) acquires images in the vehicle movement process, then manual processing is carried out on part or all of the images acquired by the vehicle-mounted image acquisition device, elements such as road signs, natural objects and traffic participants in the images acquired by the vehicle are marked, and finally the manually marked images are used as sample data for automatic driving model training and verification. When sample data is acquired in the conventional sample data acquisition mode, if all images acquired by the vehicle-mounted image acquisition device are manually marked, the efficiency of acquiring the sample data is low, the cost is high, and if only partial images acquired by the vehicle-mounted image acquisition device are manually marked, the sample data with high price value can be omitted, so that the quality of the sample data is influenced.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for obtaining sample data, so as to improve efficiency of obtaining sample data and reduce cost of obtaining sample data.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
in a first aspect, an embodiment of the present invention provides a sample data obtaining method, including:
acquiring video data acquired by a vehicle;
labeling each frame of image in the video data through a preset labeling model, and acquiring a labeling result of each frame of image in the video data, wherein the labeling result comprises: marking content and marking confidence;
extracting a target image set from the video data according to the labeling confidence coefficient;
and receiving the correction of the labeled content of the images in the target image set, and acquiring sample data.
As an optional implementation manner of the embodiment of the present invention, the acquiring video data acquired in a driving process of a vehicle includes:
acquiring a vehicle working condition and a control instruction input by a driver to the vehicle;
acquiring a driving decision according to an automatic driving algorithm and the vehicle working condition;
judging whether the driving decision is matched with the control instruction;
and if the driving decision is not matched with the control instruction, storing video data acquired by the vehicle within a preset time period, wherein the preset time period comprises the moment when the working condition of the vehicle and the control instruction are acquired.
As an optional implementation manner of the embodiment of the present invention, the extracting a target image set from the video data includes:
sorting the images in the video data in a descending order according to the labeling confidence of the images in the video data;
dividing images in the video data into a first image set and a second image set according to the sorting result, wherein the first image set comprises m% of images with the top sorting result, the second image set comprises 1-m% of images with the bottom sorting result, and m is a positive integer;
extracting n% of images from the first image set through a random extraction algorithm to obtain a third image set;
combining the second set of images and the third set of images into the set of target images.
As an alternative implementation of the embodiment of the present invention,
the m% is greater than or equal to 85% and less than or equal to 95%;
the n% is greater than or equal to 5% and less than or equal to 15%.
As an optional implementation manner of the embodiment of the present invention, the preset labeling model includes: at least one of a deep learning neural network model, a convolutional neural network model, a continuous feature correlation model, a multi-bernoulli correlation model.
As an optional implementation manner of the embodiment of the present invention, the tagging each frame of image in the video data through a preset tagging model to obtain a tagging result of each frame of image in the video data includes:
performing element identification on each frame of image of the video data through the preset labeling model;
and labeling each frame of image of the video data according to the identification result, and acquiring the labeling result of each frame of image in the video data.
As an optional implementation manner of the embodiment of the present invention, after the sample data is acquired, the method further includes:
classifying the sample data according to a preset classification mode, wherein the preset classification mode comprises the following steps: classifying according to at least one of working condition type, natural illumination intensity, weather condition and road condition;
and storing the sample data of each category into a sample data base corresponding to each category.
In a second aspect, an embodiment of the present invention provides a sample data acquiring apparatus, including:
the image acquisition module is used for acquiring video data acquired in the vehicle driving process;
the labeling module is used for labeling each frame of image in the video data through a preset labeling model, and obtaining a labeling result of each frame of image in the video data, wherein the labeling result comprises: marking content and marking confidence;
the extracting module extracts a target image set from the video data according to the annotation confidence;
and the receiving module is used for receiving the correction of the labeled content of the images in the target image set and acquiring sample data.
As an optional implementation manner of the embodiment of the present invention, the image acquisition module includes:
the acquisition unit is used for acquiring the working condition of the vehicle and the control instruction input by the driver to the vehicle;
the decision unit is used for acquiring a driving decision according to an automatic driving algorithm and the working condition of the vehicle;
and the processing unit is used for storing the video data acquired by the vehicle within a preset time period under the condition that the driving decision is not matched with the control instruction, wherein the preset time period comprises the moment when the acquisition unit acquires the working condition of the vehicle and the control instruction.
As an optional implementation manner of the embodiment of the present invention, the extraction module includes:
the sorting unit is used for sorting the images in the video data in a descending order according to the labeling confidence degrees of the images in the video data;
the dividing module is used for dividing the images in the video data into a first image set and a second image set according to the sorting result, wherein the first image set comprises m% of the images in the front of the sorting result, the second image set comprises 1-m% of the images in the back of the sorting result, and m is a positive integer;
the extraction unit is used for extracting n% of images from the first image set through a random extraction algorithm to obtain a third image set;
a combining unit for combining the second image set and the third image set into the target image set.
As an alternative implementation of the embodiment of the present invention,
the m% is greater than or equal to 85% and less than or equal to 95%;
the n% is greater than or equal to 5% and less than or equal to 15%.
As an optional implementation manner of the embodiment of the present invention, the preset labeling model includes: at least one of a deep learning neural network model, a convolutional neural network model, a continuous feature correlation model, a multi-bernoulli correlation model.
As an optional implementation manner of the embodiment of the present invention, the preset labeling model is a deep learning neural network model;
the labeling module is specifically configured to perform element identification on each frame of image of the video data through the deep learning neural network model, label each frame of image of the video data according to an identification result, and obtain a labeling result of each frame of image in the video data.
As an optional implementation manner of the embodiment of the present invention, the apparatus further includes:
the storage unit is used for classifying the sample data according to a preset classification mode and storing the sample data of each class into a sample database corresponding to each class;
wherein, the preset classification mode comprises the following steps: and classifying according to at least one of the working condition type, the natural illumination intensity, the weather condition and the road condition.
In a third aspect, an embodiment of the present invention provides a sample data acquiring apparatus, including: a memory for storing a computer program and a processor; the processor is configured to execute the sample data obtaining method according to the first aspect or any implementation manner of the first aspect when the computer program is called.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the sample data acquiring method according to the first aspect or any implementation manner of the first aspect.
According to the sample data acquisition method provided by the embodiment of the invention, after video data acquired by a vehicle is acquired, a preset labeling model is used for labeling each frame of image in the video data to acquire a labeling result of each frame of image in the video data, then a target image set is extracted from the video data according to a labeling confidence coefficient, and the received image labeling content in the extracted target image set is corrected to acquire sample data. Because the embodiment of the invention labels each frame of image in the video data through the preset labeling model to obtain the labeling result of each frame of image in the video data, the embodiment of the invention can finish most of relatively simple and repeated labeling work through automatic labeling processing, improve the labeling efficiency and release a large amount of labor cost; in addition, because the embodiment of the invention selects the target image set according to the labeling confidence and only corrects the labeling content of the image in the target image set, repeated sample data with lower value can be filtered, sample data with higher value is reserved, and the error of labeling content is avoided, so that the embodiment of the invention can also avoid the labeling error and omission of the sample data with higher value, and ensure the quality of the sample data. Namely, the embodiment of the invention can improve the efficiency of obtaining the sample data and reduce the cost of obtaining the sample data while ensuring the quality of the sample data.
Drawings
Fig. 1 is a flowchart illustrating steps of a sample data obtaining method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a second step of a sample data obtaining method according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a sample data obtaining apparatus according to an embodiment of the present invention;
fig. 4 is a second schematic structural diagram of a sample data acquiring apparatus according to an embodiment of the present invention;
fig. 5 is a third schematic structural diagram of a sample data acquiring apparatus according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a hardware structure of a sample data acquisition device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
For the convenience of clearly describing the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, the terms "first" and "second" are used to distinguish the same items or similar items with basically the same functions or actions, and those skilled in the art can understand that the terms "first" and "second" are not limited to the quantity and execution order.
In the embodiments of the present invention, words such as "exemplary" or "for example" are used to mean serving as examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "e.g.," an embodiment of the present invention is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion. In the embodiments of the present invention, the meaning of "a plurality" means two or more unless otherwise specified.
An embodiment of the present invention provides a sample data obtaining method, and specifically, referring to fig. 1, the sample data obtaining method includes the following steps S11 to S14.
And S11, acquiring video data collected by the vehicle.
Specifically, the video data acquired by the vehicle may include video data acquired by an image acquisition device such as a vehicle data recorder and a reverse image during parking, starting, driving, reversing and the like of the vehicle.
And S12, labeling each frame of image in the video data through a preset labeling model, and acquiring a labeling result of each frame of image in the video data.
Wherein, the labeling result comprises: annotation content and annotation confidence. Namely, the annotation result of each frame of image comprises the annotation content and the annotation confidence of the frame of image.
The labeling content in the embodiment of the invention refers to labeling the elements such as road signs, pedestrians, vehicles, buildings and the like in the image as data content which can be identified and analyzed by a computer.
In statistics, confidence is used to indicate the confidence level of the measured value of the measured parameter, and specifically, in the embodiment of the present invention, the annotation confidence is used to indicate the probability that the annotated content of the image is correct. That is, if the labeling confidence of an image is higher, the probability that the preset labeling model correctly labels the image is higher.
Optionally, the preset labeling model in the embodiment of the present invention may be at least one of a deep learning neural network model, a convolutional neural network model, a continuous feature correlation model, and a multi-bernoulli correlation model.
Specifically, when the preset labeling model in the embodiment of the present invention is a deep learning neural network model, the deep learning neural network model may be at least one of a classification model, a detection model, and a semantic segmentation model obtained by performing model training using a deep neural network algorithm as a model training algorithm and manually labeled sample data as model training data.
And S13, extracting a target image set from the video data according to the annotation confidence.
As an optional implementation manner of the embodiment of the present invention, the target image set includes: n% of the images with the labeling confidence degrees larger than or equal to the threshold confidence degree and all the images with the labeling confidence degrees smaller than the threshold confidence degree, wherein n is a positive integer.
Illustratively, n may be equal to 10. That is, 10% of the images with annotation confidence greater than or equal to the threshold confidence and all the images with annotation confidence less than the threshold confidence are extracted from the video data to form a target image set.
It should be noted that the threshold confidence in the embodiment of the present invention may be set as a fixed value by a person skilled in the art according to experience, or may be set dynamically according to the confidence of an image in video data acquired by a vehicle.
And S14, receiving the correction of the labeling content of the images in the target image set, and acquiring sample data.
Specifically, the annotation content of each frame of image in the target image set can be manually consulted, for each frame of image, if the annotation content obtained by annotating the image with the preset annotation model is correct, the next frame of image is directly consulted, and if the annotation content obtained by annotating the image with the preset annotation model is wrong, the next frame of image is consulted after the correct annotation content is input to replace the annotation content obtained by annotating with the preset annotation model.
According to the sample data acquisition method provided by the embodiment of the invention, after video data acquired by a vehicle is acquired, a preset labeling model is used for labeling each frame of image in the video data to acquire a labeling result of each frame of image in the video data, then a target image set is extracted from the video data according to a labeling confidence coefficient, and the received image labeling content in the extracted target image set is corrected to acquire sample data. Because the embodiment of the invention labels each frame of image in the video data through the preset labeling model to obtain the labeling result of each frame of image in the video data, the embodiment of the invention can finish most of relatively simple and repeated labeling work through automatic labeling processing, improve the labeling efficiency and release a large amount of labor cost; in addition, because the embodiment of the invention selects the target image set according to the labeling confidence and only corrects the labeling content of the image in the target image set, repeated sample data with lower value can be filtered, sample data with higher value is reserved, and the error of labeling content is avoided, so that the embodiment of the invention can also avoid the labeling error and omission of the sample data with higher value, and ensure the quality of the sample data. Namely, the embodiment of the invention can improve the efficiency of obtaining the sample data and reduce the cost of obtaining the sample data while ensuring the quality of the sample data.
Based on the above embodiments, the sample data acquisition method provided by the embodiments of the present invention is further described in more detail below. Referring to fig. 2, fig. 2 is a schematic flow chart illustrating another step of the sample data obtaining method provided by the present invention, which may specifically include the following steps:
s201, obtaining the working condition of the vehicle and a control command input by a driver to the vehicle.
Specifically, the working condition of the vehicle includes an overall environment composed of a static environment (such as a natural environment) and a dynamic environment (such as a traffic participant) where the vehicle is located, which are acquired by the vehicle-mounted radar and the vehicle-mounted image acquisition device.
The manipulation instruction input to the vehicle by the driver may include an operation instruction generated by the driver operating at least one of a steering wheel, a clutch pedal, an accelerator pedal, a brake pedal, a lighting lamp, a turn lamp, and the like. The Control command input by the driver to the vehicle can be obtained by monitoring the vehicle state through an on-board Electronic Control Unit (ECU).
And S202, obtaining a driving decision according to an automatic driving algorithm and the working condition of the vehicle.
Specifically, an automatic driving decision module can be deployed in the vehicle-mounted ECU, and the automatic driving decision module is controlled to work in a shadow mode, so that a driving decision is obtained according to the working condition of the vehicle. The shadow mode refers to an automatic driving decision module deployed in a vehicle-mounted ECU, and is a working mode for giving driving decisions such as whether a vehicle needs to be reduced/accelerated, whether the vehicle needs to be started/stopped and the like according to vehicle working conditions and automatic driving algorithm comprehensive calculation.
It should be noted that, in the shadow mode, the automatic driving decision module does not actually participate in controlling the vehicle, and only gives a driving decision based on the working condition of the vehicle.
S203, judging whether the driving decision is matched with the control command.
Specifically, in the embodiment of the present invention, the determining whether the driving decision is matched with the control command may be: and judging whether the error between the driving decision and the control command is within a certain error range. Namely, in the process of judging whether the driving decision is matched with the control command, a certain error is allowed to exist between the driving decision and the control line. For example: the driving decision is that the steering wheel turns right 90 degrees, the control command is that the steering wheel turns right 91 degrees, although the driving decision and the control command are not completely the same, the error between the driving decision and the control command is within an allowable range, and in this case, the driving decision is still judged to be matched with the control command.
In some cases, the driving decision is very different from the maneuver instruction, when it is determined that the driving decision does not match the maneuver instruction. For example: if the driving decision is that the steering wheel turns right 90 degrees, and the control command turns left 90 degrees, determining that the driving decision is not matched with the control command; for another example: and if the driving decision is to step on an accelerator pedal and the control instruction is to step on a brake pedal, determining that the driving decision is not matched with the control instruction.
In step S203, if the driving decision does not match the control command, the following step S204 is executed.
And S204, storing the video data collected by the vehicle within a preset time period.
And the preset time period comprises the moment of acquiring the working condition of the vehicle and the control command.
Specifically, the vehicle may be controlled to perform image capturing in real time, and it is determined whether to retain the captured image based on the determination result of step S203.
For example, the preset time period may be 30 seconds before the time when the vehicle condition and the manipulation command are obtained and 30 seconds after the time when the vehicle condition and the manipulation command are obtained. That is, if the time at which the vehicle operating condition and the manipulation command are obtained is T1, the preset time period is [ T1-30s, T1+30 ].
Optionally, the video data acquired by the vehicle within the preset time period may specifically be the video data that the vehicle needs to store and uploaded to the server for storage, or the video data acquired by the vehicle may be stored in the local memory first, and when a specific condition (for example, WIFI is connected) is met, the video data in the local memory is uploaded to the server, and the video data stored in the memory is emptied.
In the embodiment, the video data acquired by the vehicle is only the video data acquired by the vehicle within the preset time period under the condition that the driving decision is not matched with the control instruction, so that the embodiment can filter a large amount of repeated video data with low value during video data acquisition, reduce the transmission quantity of the video data, further reduce the efficiency of acquiring sample data and reduce the cost of acquiring the sample data.
S205, performing element identification on each frame of image of the video data through the preset annotation model.
It should be noted that the video data may include video data collected by the same vehicle in one time period, or video data collected by the same vehicle in multiple time periods, or video data collected by multiple vehicles in multiple time periods.
And S206, labeling each frame of image of the video data according to the identification result, and obtaining the labeling result of each frame of image in the video data.
Wherein, the labeling result comprises: annotation content and annotation confidence.
S207, sorting the images in the video data in a descending order according to the labeling confidence degrees of the images in the video data.
Namely, the images in the video data are sorted according to the labeling confidence degree from high to low.
And S208, dividing the images in the video data into a first image set and a second image set according to the sorting result.
The first image set comprises m% of images with a front sorting result, the second image set comprises 1-m% of images with a rear sorting result, and m is a positive integer.
As an alternative implementation of the embodiment of the present invention,
the m% is greater than or equal to 85% and less than or equal to 95%.
Illustratively, the m% equals 90%. Namely, after the images in the video data are sorted in a descending order according to the labeling confidence degrees of the images in the video data, 90% of the images in the top of the order are divided into a first image set, and 10% of the images in the back of the order are divided into a second image set.
S209, extracting n% of images from the first image set through a random extraction algorithm, and acquiring a third image set.
As an alternative implementation of the embodiment of the present invention,
the n% is greater than or equal to 5% and less than or equal to 15%.
Illustratively, the n% equals 10%. That is, 10% of the top m% of the images are randomly selected to form the third image set.
S210, combining the second image set and the third image set into the target image set.
When the n% is 10% and the m% is 90%, the proportion of the images in the target image set to all the images in the video data is: 90% + 10% + 19%, the ratio of the other images in the video data except the image in the target image set is: 100% -19% — 81%, for which 81% of the images can be directly discarded as duplicate, low value images.
S211, receiving the correction of the labeling content of the images in the target image set, and acquiring sample data.
The explanation of step S211 can refer to the explanation of step S14, and will not be described herein.
The technical effects that can be achieved by the embodiment shown in fig. 2 are the same as the technical effects that can be achieved by the embodiment shown in fig. 1, and the technical effects that can be achieved by the embodiment shown in fig. 2 are not repeatedly described here.
Further, after step S211, the method for acquiring sample data according to the embodiment of the present invention may further include:
and classifying the sample data according to a preset classification mode, and storing the sample data of each class into a sample data base corresponding to each class.
Wherein, the preset classification mode comprises the following steps: and classifying according to at least one of the working condition type, the natural illumination intensity, the weather condition and the road condition.
That is, the sample data may be classified according to one or more of the operating condition type, the natural illumination intensity, the road condition, and the like, and stored in the corresponding sample databases, so that the sample data in the sample databases of the corresponding categories may be directly called according to the requirements when the sample data is called.
According to the method example, the sample data acquisition device and the like can be divided into the functional modules. For example, the functional blocks may be divided for the respective functions, or two or more functions may be integrated into one block. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation.
In the case of using an integrated unit, fig. 3 shows a schematic diagram of a possible structure of the sample data acquiring apparatus according to the foregoing embodiment, where the sample data acquiring apparatus 300 includes:
the acquisition module 31 is used for acquiring video data acquired by a vehicle;
the labeling module 32 is configured to label each frame of image in the video data through a preset labeling model, and obtain a labeling result of each frame of image in the video data, where the labeling result includes: marking content and marking confidence;
an extracting module 33, configured to extract a target image set from the video data according to the annotation confidence;
and the receiving module 34 is configured to receive the modification of the labeled content of the images in the target image set, and acquire sample data.
As an alternative embodiment of the present invention, referring to fig. 4, the obtaining module 31 includes:
the obtaining unit 311 is configured to obtain a vehicle operating condition and a control instruction input by a driver to the vehicle;
a decision unit 312, configured to obtain a driving decision according to an automatic driving algorithm and the vehicle operating condition;
the processing unit 313 is configured to save the video data collected by the vehicle within a preset time period under the condition that the driving decision is not matched with the control instruction, where the preset time period includes a time when the obtaining unit obtains the vehicle working condition and the control instruction.
As an alternative implementation manner of the embodiment of the present invention, referring to fig. 5, the extracting module 33 includes:
the sorting unit 331 is configured to sort the images in the video data in a descending order according to the annotation confidence of the images in the video data;
a dividing module 332, configured to divide the images in the video data into a first image set and a second image set according to the sorting result, where the first image set includes m% of the images before the sorting result, the second image set includes 1-m% of the images after the sorting result, and m is a positive integer;
an extracting unit 333, configured to extract n% of images from the first image set by using a random extraction algorithm, and obtain a third image set;
a combining unit 334, configured to combine the second image set and the third image set into the target image set.
As an alternative embodiment of the present invention,
the m% is greater than or equal to 85% and less than or equal to 95%;
the n% is greater than or equal to 5% and less than or equal to 15%.
As an optional implementation manner of the embodiment of the present invention, the preset labeling model includes: at least one of a deep learning neural network model, a convolutional neural network model, a continuous feature correlation model, a multi-bernoulli correlation model.
As an optional implementation manner of the embodiment of the present invention, the labeling module 32 is specifically configured to perform element identification on each frame of image of the video data through the preset labeling model, label each frame of image of the video data according to an identification result, and obtain a labeling result of each frame of image in the video data.
As an alternative implementation manner of the embodiment of the present invention, referring to fig. 5, the sample data acquiring apparatus 300 further includes:
the storage unit 35 is configured to classify the sample data according to a preset classification manner, and store the sample data of each category into a sample data base corresponding to each category;
wherein, the preset classification mode comprises the following steps: and classifying according to at least one of the working condition type, the natural illumination intensity, the weather condition and the road condition.
The sample data acquisition device provided by the embodiment of the invention comprises: the system comprises an acquisition module, a marking module, an extraction module and a receiving module; the acquisition module is used for acquiring video data acquired by a vehicle; the labeling module is used for labeling each frame of image in the video data through a preset labeling model and acquiring a labeling result of each frame of image in the video data; the extraction module is used for extracting a target image set from the video data; the receiving module is used for receiving the correction of the labeling content of the images in the target image set and acquiring sample data; because the embodiment of the invention labels each frame of image in the video data through the preset labeling model to obtain the labeling result of each frame of image in the video data, the embodiment of the invention can finish most of relatively simple and repeated labeling work through automatic labeling processing, improve the labeling efficiency and release a large amount of labor cost; in addition, because the embodiment of the invention selects the target image set according to the labeling confidence and only corrects the labeling content of the image in the target image set, repeated sample data with lower value can be filtered, sample data with higher value is reserved, and the error of labeling content is avoided, so that the embodiment of the invention can also avoid the labeling error and omission of the sample data with higher value, and ensure the quality of the sample data. Namely, the embodiment of the invention can improve the efficiency of obtaining the sample data and reduce the cost of obtaining the sample data while ensuring the quality of the sample data.
Based on the same inventive concept, the embodiment of the invention also provides a sample data acquisition device. Fig. 6 is a schematic structural diagram of a sample data acquiring device according to an embodiment of the present invention, and as shown in fig. 6, the sample data acquiring device according to the embodiment includes: a memory 61 and a processor 62, the memory 61 being for storing computer programs; the processor 62 is configured to execute the sample data obtaining method according to the above method embodiment when the computer program is called.
The sample data obtaining apparatus provided in this embodiment may perform the sample data obtaining method provided in the above method embodiment, and the implementation principle and the technical effect are similar, which are not described herein again.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for obtaining sample data according to the above method embodiment is implemented.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied in the medium.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer readable media include both permanent and non-permanent, removable and non-removable storage media. Storage media may implement information storage by any method or technology, and the information may be computer-readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (16)
1. A sample data acquisition method is characterized by comprising the following steps:
acquiring video data acquired by a vehicle;
labeling each frame of image in the video data through a preset labeling model, and acquiring a labeling result of each frame of image in the video data, wherein the labeling result comprises: marking content and marking confidence;
extracting a target image set from the video data according to the labeling confidence coefficient;
and receiving the correction of the labeled content of the images in the target image set, and acquiring sample data.
2. The method of claim 1, wherein the obtaining video data collected during vehicle driving comprises:
acquiring a vehicle working condition and a control instruction input by a driver to the vehicle;
acquiring a driving decision according to an automatic driving algorithm and the vehicle working condition;
and if the driving decision is not matched with the control instruction, storing video data acquired by the vehicle within a preset time period, wherein the preset time period comprises the moment when the working condition of the vehicle and the control instruction are acquired.
3. The method of claim 1, wherein the extracting a set of target images from the video data comprises:
sorting the images in the video data in a descending order according to the labeling confidence of each frame of image in the video data;
dividing images in the video data into a first image set and a second image set according to the sorting result, wherein the first image set comprises m% of images with the top sorting result, the second image set comprises 1-m% of images with the bottom sorting result, and m is a positive integer;
extracting n% of images from the first image set through a random extraction algorithm to obtain a third image set;
combining the second set of images and the third set of images into the set of target images.
4. The method of claim 3,
the m% is greater than or equal to 85% and less than or equal to 95%;
the n% is greater than or equal to 5% and less than or equal to 15%.
5. The method according to any one of claims 1 to 4, wherein the preset labeling model comprises: at least one of a deep learning neural network model, a convolutional neural network model, a continuous feature correlation model, a multi-bernoulli correlation model.
6. The method according to any one of claims 1 to 4, wherein the labeling each frame of image in the video data through a preset labeling model to obtain a labeling result of each frame of image in the video data comprises:
performing element identification on each frame of image of the video data through the preset labeling model;
and labeling each frame of image of the video data according to the identification result, and acquiring the labeling result of each frame of image in the video data.
7. The method according to any of claims 1-4, wherein after acquiring said sample data, said method further comprises:
classifying the sample data according to a preset classification mode, wherein the preset classification mode comprises the following steps: classifying according to at least one of working condition type, natural illumination intensity, weather condition and road condition;
and storing the sample data of each category into a sample data base corresponding to each category.
8. A sample data acquisition apparatus, comprising:
the image acquisition module is used for acquiring video data acquired by the vehicle;
the labeling module is used for labeling each frame of image in the video data through a preset labeling model, and obtaining a labeling result of each frame of image in the video data, wherein the labeling result comprises: marking content and marking confidence;
the extracting module is used for extracting a target image set from the video data according to the annotation confidence;
and the receiving module is used for receiving the correction of the labeled content of the images in the target image set and acquiring sample data.
9. The apparatus of claim 8, wherein the image acquisition module comprises:
the acquisition unit is used for acquiring the working condition of the vehicle and the control instruction input by the driver to the vehicle;
the decision unit is used for acquiring a driving decision according to an automatic driving algorithm and the working condition of the vehicle;
and the processing unit is used for storing the video data acquired by the vehicle within a preset time period under the condition that the driving decision is not matched with the control instruction, wherein the preset time period comprises the moment when the acquisition unit acquires the working condition of the vehicle and the control instruction.
10. The apparatus of claim 8, wherein the extraction module comprises:
the sorting unit is used for sorting the images in the video data in a descending order according to the labeling confidence degrees of the images in the video data;
the dividing module is used for dividing the images in the video data into a first image set and a second image set according to the sorting result, wherein the first image set comprises m% of the images in the front of the sorting result, the second image set comprises 1-m% of the images in the back of the sorting result, and m is a positive integer;
the extraction unit is used for extracting n% of images from the first image set through a random extraction algorithm to obtain a third image set;
a combining unit for combining the second image set and the third image set into the target image set.
11. The apparatus of claim 10,
the m% is greater than or equal to 85% and less than or equal to 95%;
the n% is greater than or equal to 5% and less than or equal to 15%.
12. The apparatus according to any one of claims 8-11, wherein the preset labeling model comprises: at least one of a deep learning neural network model, a convolutional neural network model, a continuous feature correlation model, a multi-bernoulli correlation model.
13. The apparatus according to any one of claims 8 to 11,
the labeling module is specifically configured to perform element identification on each frame of image of the video data through the preset labeling model, label each frame of image of the video data according to an identification result, and acquire a labeling result of each frame of image in the video data.
14. The apparatus according to any one of claims 8-11, further comprising:
the storage unit is used for classifying the sample data according to a preset classification mode and storing the sample data of each class into a sample database corresponding to each class;
wherein, the preset classification mode comprises the following steps: and classifying according to at least one of the working condition type, the natural illumination intensity, the weather condition and the road condition.
15. A sample data acquisition apparatus, comprising: a memory for storing a computer program and a processor; the processor is used for executing the sample data acquisition method of any one of claims 1 to 7 when calling the computer program.
16. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements the sample data acquisition method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010293247.2A CN111461056A (en) | 2020-04-15 | 2020-04-15 | Sample data acquisition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010293247.2A CN111461056A (en) | 2020-04-15 | 2020-04-15 | Sample data acquisition method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111461056A true CN111461056A (en) | 2020-07-28 |
Family
ID=71681250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010293247.2A Pending CN111461056A (en) | 2020-04-15 | 2020-04-15 | Sample data acquisition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111461056A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112349150A (en) * | 2020-11-19 | 2021-02-09 | 飞友科技有限公司 | Video acquisition method and system for airport flight guarantee time node |
| CN112382165A (en) * | 2020-11-19 | 2021-02-19 | 北京罗克维尔斯科技有限公司 | Driving strategy generation method, device, medium, equipment and simulation system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103324937A (en) * | 2012-03-21 | 2013-09-25 | 日电（中国）有限公司 | Method and device for labeling targets |
| CN106529485A (en) * | 2016-11-16 | 2017-03-22 | 北京旷视科技有限公司 | Method and apparatus for obtaining training data |
| CN109886338A (en) * | 2019-02-25 | 2019-06-14 | 苏州清研精准汽车科技有限公司 | A kind of intelligent automobile test image mask method, device, system and storage medium |
-
2020
- 2020-04-15 CN CN202010293247.2A patent/CN111461056A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103324937A (en) * | 2012-03-21 | 2013-09-25 | 日电（中国）有限公司 | Method and device for labeling targets |
| CN106529485A (en) * | 2016-11-16 | 2017-03-22 | 北京旷视科技有限公司 | Method and apparatus for obtaining training data |
| CN109886338A (en) * | 2019-02-25 | 2019-06-14 | 苏州清研精准汽车科技有限公司 | A kind of intelligent automobile test image mask method, device, system and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 徐美香;孙福明;李豪杰;: "主动学习的多标签图像在线分类" * |
| 袁勋;吴秀清;洪日昌;宋彦;华先胜;: "基于主动学习SVM分类器的视频分类" * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112349150A (en) * | 2020-11-19 | 2021-02-09 | 飞友科技有限公司 | Video acquisition method and system for airport flight guarantee time node |
| CN112382165A (en) * | 2020-11-19 | 2021-02-19 | 北京罗克维尔斯科技有限公司 | Driving strategy generation method, device, medium, equipment and simulation system |
| CN112382165B (en) * | 2020-11-19 | 2022-10-04 | 北京罗克维尔斯科技有限公司 | Driving strategy generation method, device, medium, equipment and simulation system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109993031B (en) | Method and device for detecting target illegal driving behaviors of livestock-powered vehicle and camera | |
| Souani et al. | Efficient algorithm for automatic road sign recognition and its hardware implementation | |
| CN110111581A (en) | Target identification method, device, computer equipment and storage medium | |
| CN111428875A (en) | Image recognition method and device and corresponding model training method and device | |
| CN110555347B (en) | Vehicle target identification method and device with dangerous cargo-carrying behavior and electronic equipment | |
| CN113762209A (en) | Multi-scale parallel feature fusion road sign detection method based on YOLO | |
| CN113298050B (en) | Lane line recognition model training method and device and lane line recognition method and device | |
| CN111507226B (en) | Road image recognition model modeling method, image recognition method and electronic equipment | |
| CN111461056A (en) | Sample data acquisition method and device | |
| WO2020007589A1 (en) | Training a deep convolutional neural network for individual routes | |
| CN112200193B (en) | Distributed license plate recognition method, system and device based on multi-attribute fusion | |
| CN110909598A (en) | Deep learning-based method for recognizing illegal traffic driving of non-motor vehicle lane | |
| CN110599453A (en) | Panel defect detection method and device based on image fusion and equipment terminal | |
| CN111008554B (en) | Deep learning-based method for identifying pedestrians without giving away in dynamic traffic zebra stripes | |
| CN112613434A (en) | Road target detection method, device and storage medium | |
| CN115424217A (en) | AI vision-based intelligent vehicle identification method and device and electronic equipment | |
| CN114495060A (en) | Road traffic marking identification method and device | |
| Thomas et al. | Smart car parking system using convolutional neural network | |
| CN112733864B (en) | Model training method, target detection method, device, equipment and storage medium | |
| CN110555344B (en) | Lane line recognition method, lane line recognition device, electronic device, and storage medium | |
| CN113903014B (en) | Lane line prediction method and device, electronic device and computer-readable storage medium | |
| CN114037976A (en) | Road traffic sign identification method and device | |
| CN112257869A (en) | Fake-licensed car analysis method and system based on random forest and computer medium | |
| CN110321973B (en) | Combined vehicle detection method based on vision | |
| CN110659384B (en) | Video structured analysis method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |