CN117636268A - Unmanned aerial vehicle aerial natural driving data set construction method oriented to ice and snow environment - Google Patents
Unmanned aerial vehicle aerial natural driving data set construction method oriented to ice and snow environment Download PDFInfo
- Publication number
- CN117636268A CN117636268A CN202311648037.0A CN202311648037A CN117636268A CN 117636268 A CN117636268 A CN 117636268A CN 202311648037 A CN202311648037 A CN 202311648037A CN 117636268 A CN117636268 A CN 117636268A
- Authority
- CN
- China
- Prior art keywords
- image
- vehicle
- road
- snow
- ice
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000010276 construction Methods 0.000 title claims abstract description 11
- 238000000034 method Methods 0.000 claims abstract description 54
- 238000011156 evaluation Methods 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 11
- 238000001914 filtration Methods 0.000 claims description 22
- 230000001133 acceleration Effects 0.000 claims description 19
- 239000011159 matrix material Substances 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 8
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 230000005484 gravity Effects 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000012805 post-processing Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000012937 correction Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 claims description 3
- 238000003708 edge detection Methods 0.000 claims description 3
- 238000003709 image segmentation Methods 0.000 claims description 3
- 238000012417 linear regression Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000012706 support-vector machine Methods 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 230000000295 complement effect Effects 0.000 claims description 2
- 238000007781 pre-processing Methods 0.000 claims description 2
- 238000011160 research Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 201000004569 Blindness Diseases 0.000 description 1
- 102220559998 Kinesin-like protein KIF1A_M30T_mutation Human genes 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000004393 visual impairment Effects 0.000 description 1
Landscapes
- Traffic Control Systems (AREA)
Abstract
The invention belongs to the technical field of automatic driving, and particularly relates to an unmanned aerial vehicle aerial natural driving data set construction method facing an ice and snow environment. Firstly, carrying out video shooting on typical structured road traffic in an ice and snow environment through an unmanned aerial vehicle with high acquisition precision at an aerial shooting view angle, and then carrying out a rapid denoising method on rain and snow shielding of a video image, thereby improving the video processing efficiency as much as possible on the basis of obtaining the minimum requirement of constructing a data set; then, road identification is carried out, a complete road structure is deduced through basic road information in the existing image, and a coverage area is accurately identified; secondly, vehicle identification is carried out, road vehicles in the video are identified, prediction filling is carried out aiming at the lost area, and therefore information such as vehicle speed, following distance and vehicle movement track is extracted; finally, through defining multidimensional evaluation indexes, multidimensional comprehensive evaluation is carried out on traffic situation under ice and snow environment, and subsequent traffic flow related research is conveniently carried out by using the data set.
Description
Technical Field
The invention belongs to the technical field of automatic driving, and particularly relates to an unmanned aerial vehicle aerial natural driving data set construction method facing an ice and snow environment.
Background
The ice and snow environment is the most common scene in winter in northern areas, the snow, low adhesion, frost and other characteristics in the ice and snow environment greatly affect the traffic flow operation, and the traffic flow operation condition is changed, so that the traffic scene in the ice and snow environment has higher complexity and uncertainty compared with the traffic scene in normal weather, is a scene with special challenges and value, and provides higher requirements and standards for tasks such as automatic driving, traffic analysis, road assessment and the like. However, the construction of aerial photographing data sets facing ice and snow environments faces some technical problems, firstly, the aerial photographing images of ice and snow have low quality, more noise and poor contrast, and the definition of the images is reduced; secondly, the ice and snow aerial photographing data has the problem of partial short-time sight shielding, so that detected vehicles are lost, and subsequent track information extraction is affected; finally, the road is unclear and covered and shielded by ice and snow, so that the difficulty in identifying the road structure and state is increased. Meanwhile, the driving characteristics in the ice and snow scene are related to various factors such as weather conditions, road conditions, traffic flow density and the like, and complex ice and snow environment data characteristics are difficult to embody by single track extraction. Therefore, a strategy for image processing, track extraction, road analysis and objective evaluation of natural driving aerial data in ice and snow environments is needed, so that an unmanned aerial vehicle aerial natural driving dataset facing the ice and snow environments is constructed.
Disclosure of Invention
In order to solve the problems, the invention provides an unmanned aerial vehicle aerial natural driving data set construction method facing to an ice and snow environment, which comprises the steps of firstly carrying out video shooting on typical structured road traffic in the ice and snow environment through an unmanned aerial vehicle with high acquisition precision at an aerial shooting view angle, and then carrying out a rapid denoising method on rain and snow shielding of a video image, thereby improving the video processing efficiency as much as possible on the basis of obtaining the minimum requirement of the constructed data set; then, road identification is carried out, a complete road structure is deduced through basic road information in the existing image, and a coverage area is accurately identified; secondly, vehicle identification is carried out, road vehicles in the video are identified, prediction filling is carried out aiming at the lost area, and therefore information such as vehicle speed, following distance and vehicle movement track is extracted; finally, through defining multidimensional evaluation indexes, multidimensional comprehensive evaluation is carried out on traffic situation under ice and snow environment, and subsequent traffic flow related research is conveniently carried out by using the data set.
The technical scheme of the invention is as follows in combination with the accompanying drawings:
an unmanned aerial vehicle aerial natural driving data set construction method facing ice and snow environment comprises the following steps:
step one, video acquisition is carried out on urban road traffic flow by using an unmanned aerial vehicle;
step two, rapidly denoising the video image acquired by the unmanned aerial vehicle;
thirdly, road identification is carried out on roads in ice and snow environments, a complete road structure is deduced through existing basic road image information in video images, and road coverage areas are accurately identified;
step four, identifying road vehicles in the video image, extracting the speed of the vehicles, the relative distance between the vehicles and the front vehicle and the track, and predicting the positions and the tracks of the vehicles which are affected by the noise points and lack local vehicle information to complement the missing track information;
and fifthly, establishing a multi-dimensional evaluation index of the traffic data set of the ice and snow environment, and evaluating the traffic state under the ice and snow environment through three methods of ice and snow strength, vehicle flow and road state.
Further, in the first step, the included angle between the camera body and the camera holder is ensured to be 90 degrees during video acquisition, the road is parallel to the camera holder, the video recording is performed on the road by using the overlooking view angle, and the typical structured road is selected for shooting the road scene.
Further, the typical structured roads include urban expressways, crossroads, expressway-up-down ramps and ordinary urban roads.
Further, the specific method of the second step is as follows:
21 Preprocessing the collected video, and cutting, rotating and scaling;
22 The convolutional neural network is adopted to rapidly denoise and enhance each frame of image, and meanwhile, the details and the definition of the image are maintained;
23 Post-processing the denoised video, performing color correction, contrast adjustment and sharpening.
Further, the specific method of the third step is as follows:
31 Using the post-processing video of the second step, carrying out road identification based on image segmentation by using a full convolution network, carrying out pixel-level classification on each frame of image, and dividing the image into a road area and a non-road area;
32 Adopting the generation of an countermeasure network model to carry out image filling-based road completion, and processing the information loss of road lane lines in the road caused by the road ice and snow shielding factors;
33 The ice and snow covered area and the uncovered area in the full image are marked and divided by the existing ice and snow edge detection method.
Further, in the step 32), generating the countermeasure network model includes:
a generator for extracting missing road information from the inputted road image under the ice and snow environment, generating a complete road image, and inputting the complete road image as a road image I under the ice and snow environment in Output as a complete road image I out ；
A discriminator for judging whether the generated road image is true and reliable, i.e. is similar to the true road image, and inputting into a road image I and a road image I under ice and snow environment in The output is a probability value p, which indicates whether the road image is authentic or not.
Further, the specific method of the step 32) is as follows:
randomly selecting a road image I from a real road image dataset real Randomly generating an ice and snow shielding or noise interference area on the real road image to obtain a road image I under the ice and snow environment in The method comprises the steps of carrying out a first treatment on the surface of the Road image I under ice and snow environment in Input into a generator to obtain a generated road image I out ；
Real road image I real And the generated road image I out Respectively with road image I under ice and snow environment in Input deviceTo the discriminator to obtain two probability values p real And p out Respectively representing whether the real road image and the generated road image are real and credible;
according to two probability values p real And p out Calculating the loss functions of the generator and the discriminator, and updating the parameters of the generator and the discriminator by using a method of back propagation and gradient descent, so that the generator can generate a more real road image, and the discriminator can more accurately distinguish the real road image from the generated road image;
the loss functions of the generator and the arbiter are respectively:
L G ＝-logp out +λ||I real -I out || 1 (1)
L D ＝-logp real -log(1-p out ) (2)
wherein L is G A loss function of the generator; lambda is a super parameter for controlling the generator to generate a balance between the quality and the authenticity of the road image; I.I 1 Is L1 norm; l (L) D Is the loss function of the arbiter.
Further, the specific method of the fourth step is as follows:
41 For a video frame, using selective search to generate candidate areas, extracting a plurality of candidate areas containing vehicle targets in the video frame image, using a pretrained convolutional neural network VGG-16 to extract feature vectors for the candidate areas, inputting the feature vectors into a support vector machine classifier so as to judge whether the areas contain vehicles, using a linear regression model to finely adjust the frames of the vehicles for each candidate area judged to contain the vehicles, enabling the frames to be attached to the vehicles, using a kernel correlation filtering algorithm to track the vehicles for the detected vehicles, and allocating unique label IDs for the vehicles, and keeping consistency in the subsequent video frames;
42 The vehicle information extraction method based on OpenCV is designed, the pixel coordinates marking the center of gravity of the vehicle in each frame of image are obtained, so that the pixel coordinate difference on the corresponding time stamp is obtained, then the mapping relation between the pixels and the real coordinates is established, the vehicle coordinate movement condition on each time stamp is obtained, and the driving direction, the speed, the acceleration and the track of the required vehicle are obtained;
the vehicle bounding box is extracted, and in each frame of image, the pixel coordinate of the gravity center of the ith vehicle at the moment t can be expressed as [ x ] i|t ,y i|t ]After passing Δt, the vehicle barycentric pixel coordinates become [ x ] i|t+Δt ,y i|t+Δt ]The mapping relation between the pixel coordinates and the real coordinates is as follows:
[x pixel ,y pixel ]＝k[x real ,y real ] (3)
wherein k is a conversion coefficient between a pixel coordinate and a real coordinate;
calculating the instantaneous speed, the average speed and the acceleration of the vehicle by acquiring pixel coordinates of the same vehicle in different frames, and acquiring the following distances between different vehicles on the same lane in the same frame by acquiring the vehicle coordinates in a period of time to obtain the running track of the vehicle; the specific calculation formulas of the average speed, the acceleration and the following distance are as follows:
Δx i,i+1 ＝k(x i|t -x i+1|t ) (6)
in the method, in the process of the invention,is the average speed of the ith vehicle within Δt; a, a i Acceleration for the ith vehicle; deltax i,i+1 The following distance between the ith vehicle and the (i+1) th vehicle is set;
43 Designing a prediction algorithm based on Kalman filtering to estimate vehicle information in a video frame lost by a vehicle until the vehicle corresponds to the missing information in a subsequent video frame;
for each video frame, detecting whether there is a vehicle that loses recognition tracking due to occlusion, i.e., vehicle information cannot be detected by the aforementioned vehicle recognition step in the present frame, but there is a detected vehicle in the previous frame; if the missing vehicle exists, the motion information of the missing vehicle is complemented by prediction through a Kalman filtering algorithm, and the motion information is represented through a virtual frame;
for lost vehicles, the following motion state is predicted continuously by using Kalman filtering, and a prediction equation of the Kalman filtering is as follows:
in which Q k Is a 3 x 3 symmetric positive definite matrix for the process noise covariance matrix,for the state prior estimation at time k, the speed, the acceleration and the running direction of the vehicle are included, namely x k ＝[v k ,a k ,θ k ] T 。/>For covariance a priori estimation at time k, i.e
Wherein,the variance of the speed, acceleration and direction of the state space model; ρ is the covariance between the two.
F k For state transition matricesSpecifically, the method comprises the steps of,
where Δt is the inverse of the sampling frequency of the acquired video.
And stopping prediction until the identifier matching is detected again in the subsequent video frames and the vehicle with the matched motion information. When the vehicle information is re-detected, the Kalman filtering prediction result is used for replacing the vehicle information in the missing frame, and the solid line frame is re-used for representing in the video frame; in addition, a time threshold is set for the Kalman filtering, and if the time threshold is exceeded, the vehicle is considered to leave the video shooting scene, and prediction is not performed any more.
Further, the specific method of the fifth step is as follows:
51 The gray level co-occurrence matrix model based on image processing is designed to evaluate the ice and snow strength, the non-denoising image in the second step is processed, the image texture characteristics are calculated, and the ice and snow strength in the ice and snow scene is evaluated, wherein the process is as follows:
converting the image into a gray scale image; dividing the gray image into a plurality of small blocks, wherein each small block is the field of one pixel; calculating a gray level co-occurrence matrix for each small block, recording the joint distribution of gray level values between two pixels, and finally calculating texture features according to the gray level co-occurrence matrix; the texture features include contrast, uniformity, and entropy;
the contrast reflects the gray level change degree in the image, and the calculation formula is as follows:
the uniformity reflects the gray level distribution uniformity degree in the image, and the calculation formula is as follows:
the entropy reflects the gray level complexity in the image, and the calculation formula is as follows:
wherein L is the gray level of the gray image; i, j are the pixel values of the image respectively; p (i, j) is the number of times the pixel with gray value i is next to the pixel with gray value j to the right;
the three characteristics of contrast, uniformity and entropy are used for comprehensively evaluating the ice and snow strength, and images with high contrast, low uniformity and high entropy show that the ice and snow strength is high, and conversely show that the ice and snow strength is low;
defining a comprehensive evaluation function of the ice and snow intensity, and defining the ice and snow intensity S for the contrast C, the uniformity U and the entropy E of a given image:
S＝αC-βU+γE (14)
wherein alpha, beta and gamma are weight coefficients;
according to the value of S, the ice and snow intensity is divided into small snow, medium snow, large snow and snow storm:
snow: s is less than or equal to S1;
medium snow: s1 is less than or equal to S2;
snow: s2 is less than or equal to S3;
snow storm: s > S3.
Wherein S1, S2 and S3 are threshold values;
52 Analyzing the image coverage information in the third step, and evaluating the road ice and snow coverage so as to determine the road state;
and (3) calculating the area ratio of the covered area H to the total road area I by acquiring the image marked with the ice and snow covered area in the step three, so as to determine the road ice and snow coverage.
H＝∑h i (15)
Where hi is the area of the ith ice and snow covered region;
according to the ratio, the road ice and snow coverage is divided into non-shielding, partial shielding, most shielding and all shielding:
no shielding: H/I is less than or equal to 0.1
Small part shielding: H/I is 0.1< 0.5 or less
Most of the shielding: H/I is less than or equal to 0.5 and less than or equal to 0.9
All shielding: H/I >0.9;
53 Analyzing the image vehicle information in the fourth step, calculating the number of vehicles and the following distance distribution in the image, evaluating the vehicle density in the ice and snow scene, and determining the road vehicle congestion condition;
firstly, counting the number of vehicles according to the vehicle detection result, and calculating the vehicle density in the image by using the following formula, wherein N is the total number of vehicles in the image, A is the image area, and D is the image vehicle density;
and secondly, analyzing the following distance, and calculating the relative distance between adjacent vehicles in the same lane, wherein the following time interval has the following calculation formula:
in the formula, h ij A following time interval for the ith vehicle and the jth vehicle; v i Is the speed of the ith vehicle; d, d ij Is the following distance; n is the number of vehicles in the current lane; l is the number of lanes of the image road;
and finally, evaluating the road vehicle congestion condition in the ice and snow environment:
wherein c is a vehicle congestion degree coefficient; d (D) max Is the maximum density of vehicles in the image; h min Is the smallest following time interval in the image.
The beneficial effects of the invention are as follows:
1) The method and the device perform quick denoising on the acquired image, and improve the efficiency and the scale of data set construction;
2) The invention provides an information prediction method for a vehicle which is identified by the vehicle and is interfered by noise points to track and lose, so that the information such as the speed, the acceleration and the track of the vehicle can be conveniently extracted;
3) According to the invention, the traffic situation under the ice and snow environment is quantitatively evaluated by establishing the multidimensional evaluation index, so that the traffic flow research by the subsequent application of the data set is facilitated, and the applicability and the construction value of the data set are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of the present invention;
fig. 2 is a schematic diagram of unmanned aerial vehicle data acquisition;
FIG. 3 is a schematic diagram of an original image acquisition and a fast denoising image;
FIG. 4 is a schematic diagram of a vehicle identification process based on image information;
FIG. 5a is a schematic view of traffic flow vehicle following distance extracted based on vehicle information on a sunny day;
FIG. 5b is a schematic view of the following distance of a traffic flow vehicle based on vehicle information extraction during snowy days;
FIG. 6a is a schematic diagram of vehicle speed of a traffic flow extracted based on vehicle information on a sunny day;
FIG. 6b is a schematic illustration of vehicle speed of a traffic flow extracted based on vehicle information during snowy days;
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Referring to fig. 1, the invention provides a method for constructing an unmanned aerial vehicle aerial natural driving data set facing ice and snow environments, which comprises the following steps:
step one, video acquisition is carried out on urban road traffic flow by using an unmanned aerial vehicle; when the video is collected, the shooting angle of the cradle head and the included angle of the machine body are ensured to be 90 degrees, the road is parallel to the cradle head, the video recording is carried out on the road by using the overlooking view angle, and a typical road is selected by shooting a road scene. The typical roads include urban expressways, crossroads, expressway up-down ramps, common urban roads and the like.
Referring to fig. 2, in the embodiment, a large-scale M30T unmanned aerial vehicle is adopted, a 4K high-definition camera is carried, video resolution is 3840×2160, frame rate is 30fps, hover precision is +/-0.1M, scene information acquisition of a high-resolution typical structured road is realized, and requirements for constructing a data set are met. Through calibrating unmanned aerial vehicle camera distortion, when unmanned aerial vehicle flight altitude is 153m, can gather all vehicle information in the 200m road length. And the shooting road scene is used for selecting typical structured roads such as urban expressways, crossroads, expressway up-down ramps, common urban roads and the like.
Step two, fast denoising the video image collected by the unmanned aerial vehicle, and improving video processing efficiency as much as possible on meeting the minimum requirement of constructing the data set, and improving the efficiency and the scale of constructing the data set, and the method specifically comprises the following steps:
21 The collected video is preprocessed, and operations such as cutting, rotation, scaling and the like are performed, so that irrelevant information is reduced, and the subsequent processing speed is improved;
22 The existing convolutional neural network is adopted to rapidly denoise and enhance each frame of image, and meanwhile, the details and the definition of the image are maintained;
23 After denoising, performing operations such as color correction, contrast adjustment, sharpening and the like on the video, improving the visual quality and usability of the video, and fig. 3 is an original image and an image after denoising, video post-processing and the like.
Thirdly, road identification is carried out on roads in ice and snow environments, a complete road structure is deduced through existing basic road image information in video images, and road coverage areas are accurately identified, specifically as follows:
31 Using the post-processing video of the second step, carrying out road identification based on image segmentation by using a full convolution network, carrying out pixel-level classification on each frame of image, and dividing the image into a road area and a non-road area;
32 Considering that part of road lane lines are blocked by road ice and snow in an ice and snow environment, adopting the generation of an countermeasure network model to carry out road complementation based on image filling, and processing the information deletion of the road lane lines in the road caused by the road ice and snow blocking factors;
the generating an countermeasure network model includes:
a generator (G) for extracting missing road information from the road image under the input ice and snow environment and generating a complete road image;
specifically, the generator adopts an encoder-decoder structure, which can encode the inputted road image in the ice and snow environment into a low-dimensional feature vector, and then decode the feature vector into a road image containing complete lane lines. The input of the generator is a road image Iin under an ice and snow environment, the output is a complete road image Iout, and the expression is as follows:
I out ＝G(I in ) (20)
and the discriminator (D) is used for judging whether the generated road image is true and reliable, namely whether the generated road image is identical with the true road image.
Specifically, the discriminator adopts a convolutional neural network structure, which can extract the high-level features from the input road image and output a probability value to indicate whether the road image is true or not. The input of the discriminator isRoad image I and road image I under ice and snow environment in The output is a probability value p which indicates whether the road image is true or not, and the expression is as follows:
p＝D(I,I in ) (21)
the specific method of the step 32) is as follows:
randomly selecting a road image I from a real road image dataset real Randomly generating an ice and snow shielding or noise interference area on the real road image to obtain a road image I under the ice and snow environment in The method comprises the steps of carrying out a first treatment on the surface of the Road image I under ice and snow environment in Input into a generator to obtain a generated road image I out ；
Real road image I real And the generated road image I out Respectively with road image I under ice and snow environment in Input into a discriminator to obtain two probability values p real And p out Respectively representing whether the real road image and the generated road image are real and credible;
according to two probability values p real And p out Calculating the loss functions of the generator and the discriminator, and updating the parameters of the generator and the discriminator by using a method of back propagation and gradient descent, so that the generator can generate a more real road image, and the discriminator can more accurately distinguish the real road image from the generated road image;
the loss functions of the generator and the arbiter are respectively:
L G ＝-logp out +λ||I real -I out || 1 (22)
L D ＝-logp real -log(1-p out ) (23)
wherein L is G A loss function of the generator; lambda is a super parameter for controlling the generator to generate a balance between the quality and the authenticity of the road image; I.I 1 Is L1 norm; l (L) D Is the loss function of the arbiter.
Through the training process, the generation of the countermeasure network model can realize road completion based on image filling, and the road information loss caused by factors such as road ice and snow shielding, noise interference and the like in the road is processed, so that the continuity and consistency of the road are maintained.
33 The ice and snow covered area and the uncovered area in the full image are marked and divided by the existing ice and snow edge detection method, so that road state evaluation can be conveniently carried out by using the ice and snow covered area subsequently.
Fourth, referring to fig. 4, a road vehicle in a video image is identified, and the speed, the relative distance between the vehicle and a preceding vehicle and the track of the vehicle are extracted, in addition, the position and the track of the vehicle which is affected by noise points and lacks local vehicle information are predicted, and the missing track information is complemented, specifically as follows:
41 For a video frame, using selective search to generate candidate areas, extracting a plurality of candidate areas containing vehicle targets in the video frame image, using a pretrained convolutional neural network VGG-16 to extract feature vectors for the candidate areas, inputting the feature vectors into a support vector machine classifier so as to judge whether the areas contain vehicles, using a linear regression model to finely adjust the frames of the vehicles for each candidate area judged to contain the vehicles, enabling the frames to be attached to the vehicles, improving information extraction precision in the subsequent steps, using a kernel correlation filtering algorithm to track the vehicles for the detected vehicles, assigning unique label IDs for the vehicles, and keeping consistency in the subsequent video frames;
42 The vehicle information extraction method based on OpenCV is designed, the pixel coordinates of the center of gravity of the vehicle in each frame of image are obtained, so that the pixel coordinate difference on the corresponding time stamp is obtained, then the mapping relation between the pixels and the real coordinates is established, the vehicle coordinate movement condition on each time stamp is obtained, the information such as the driving direction, the speed, the acceleration, the track and the like of the required vehicle is obtained, and the data set is convenient to be used for carrying out research in the fields of intelligent vehicle decision planning, control, testing and the like.
Specifically, the vehicle boundary box is extracted through the foregoing steps, and in each frame of image, the i-th vehicle gravity center pixel coordinate at the time t can be represented as [ x ] i|t ,y i|t ]After passing Δt, the vehicle barycentric pixel coordinates become [ x ] i|t+Δt ,y i|t+Δt ]The mapping relation between the pixel coordinates and the real coordinates is as follows:
[x pixel ,y pixel ]＝k[x real ,y real ] (24)
where k is a conversion coefficient between the pixel coordinates and the true coordinates, in this example k=19.1;
calculating the instantaneous speed, the average speed and the acceleration of the vehicle by acquiring pixel coordinates of the same vehicle in different frames, and acquiring the following distances between different vehicles on the same lane in the same frame by acquiring the vehicle coordinates in a period of time to obtain the running track of the vehicle; the specific calculation formulas of the average speed, the acceleration and the following distance are as follows:
Δx i,i+1 ＝k(x i|t -x i+1|t ) (27)
in the method, in the process of the invention,is the average speed of the ith vehicle within Δt; a, a i Acceleration for the ith vehicle; deltax i,i+1 The following distance between the ith vehicle and the (i+1) th vehicle is set;
referring to fig. 5a, 5b, 6a and 6b, according to the vehicle information identification method designed by the present invention, the speed of a highway vehicle and the following distance of the vehicle in normal weather and ice and snow weather are analyzed, and the result shows that the following distance of the vehicle in ice and snow environment is larger than that in normal weather, the following distance peak value represented in the image in ice and snow environment is 40-50m, and the following distance of 50-60m and 60-70m is more common than that in normal weather; compared with the vehicle in normal weather, the vehicle speed is lower, the peak value appears on the image at 50-60km/h, the vehicle speed at 20-30km/h and 30-40km/h is more common than that in normal condition, the vehicle information extraction result in the ice and snow environment provided by the invention accords with the actual real road vehicle traffic condition, and the road vehicle information in the data set can be extracted truly and accurately.
43 Designing a prediction algorithm based on Kalman filtering to estimate vehicle information in a video frame lost by a vehicle until the vehicle corresponds to the missing information in a subsequent video frame;
in consideration of the situation that shielding occurs in an ice and snow environment, part of vehicle information is tracked and lost in certain time stamps, a prediction algorithm based on Kalman filtering is designed to estimate the vehicle information in a video frame lost by a vehicle until the vehicle corresponds to the vehicle with the information lost in a subsequent video frame, and therefore the problem of vision loss of the vehicle due to shielding is effectively solved.
For each video frame, detecting whether there is a vehicle that loses recognition tracking due to occlusion, i.e., vehicle information cannot be detected by the aforementioned vehicle recognition step in the present frame, but there is a detected vehicle in the previous frame; if the missing vehicle exists, the motion information of the missing vehicle is complemented by prediction through a Kalman filtering algorithm, and the motion information is represented through a virtual frame;
for lost vehicles, the following motion state is predicted continuously by using Kalman filtering, and a prediction equation of the Kalman filtering is as follows:
in which Q k Is a 3 x 3 symmetric positive definite matrix for the process noise covariance matrix,for the state prior estimation at time k, the speed, the acceleration and the running direction of the vehicle are included, namely x k ＝[v k ,a k ,θ k ] T ；/>For covariance a priori estimation at time k, i.e
Wherein,the variance of the speed, acceleration and direction of the state space model; ρ is the covariance between the two.
F k For a state transition matrix, in particular,
where Δt is the inverse of the sampling frequency of the acquired video.
Stopping prediction until the identifier is detected again in the subsequent video frame, and stopping prediction when the vehicle information is detected again, using a Kalman filtering prediction result to replace the vehicle information in the missing frame, and reusing a solid line frame in the video frame for representation; in addition, a time threshold is set for the Kalman filtering, and if the time threshold is exceeded, the vehicle is considered to leave the video shooting scene, and prediction is not performed any more.
Fifthly, establishing a multidimensional evaluation index of an ice and snow environment traffic data set, evaluating traffic states under the ice and snow environment by three methods of ice and snow intensity, vehicle flow and road states, and reducing traffic state classification steps of carrying out ice and snow environment traffic research by using the data set subsequently, wherein the method comprises the following concrete steps:
51 The gray level co-occurrence matrix model based on image processing is designed to evaluate the ice and snow strength, the non-denoising image in the second step is processed, the image texture characteristics are calculated, and the ice and snow strength in the ice and snow scene is evaluated, wherein the process is as follows:
converting the image into a gray image, and reducing interference of color information; dividing the gray image into a plurality of small blocks, wherein each small block is the field of one pixel, such as 3×3, 5×5, etc.; calculating a gray level co-occurrence matrix for each small block, recording the joint distribution of gray level values between two pixels, and finally calculating texture features such as contrast, uniformity, entropy and the like according to the gray level co-occurrence matrix;
the contrast reflects the gray level change degree in the image, and the calculation formula is as follows:
the uniformity reflects the gray level distribution uniformity degree in the image, and the calculation formula is as follows:
the entropy reflects the gray level complexity in the image, and the calculation formula is as follows:
wherein L is the gray level of the gray image; i, j are the pixel values of the image respectively; p (i, j) is the number of times the pixel with gray value i is next to the pixel with gray value j to the right;
the three characteristics of contrast, uniformity and entropy are used for comprehensively evaluating the ice and snow strength, and images with high contrast, low uniformity and high entropy show that the ice and snow strength is high, and conversely show that the ice and snow strength is low;
defining a comprehensive evaluation function of the ice and snow intensity, and defining the ice and snow intensity S for the contrast C, the uniformity U and the entropy E of a given image:
S＝αC-βU+γE (35)
wherein alpha, beta and gamma are weight coefficients;
according to the value of S, the ice and snow intensity is divided into small snow, medium snow, large snow and snow storm:
snow: s is less than or equal to S1;
medium snow: s1 is less than or equal to S2;
snow: s2 is less than or equal to S3;
snow storm: s > S3.
Wherein S1, S2 and S3 are threshold values;
52 Analyzing the image coverage information in the third step, and evaluating the road ice and snow coverage so as to determine the road state;
and (3) calculating the area ratio of the covered area H to the total road area I by acquiring the image marked with the ice and snow covered area in the step three, so as to determine the road ice and snow coverage.
H＝∑h i (36)
Where hi is the area of the ith ice and snow covered region;
according to the ratio, the road ice and snow coverage is divided into non-shielding, partial shielding, most shielding and all shielding:
no shielding: H/I is less than or equal to 0.1
Small part shielding: H/I is 0.1< 0.5 or less
Most of the shielding: H/I is less than or equal to 0.5 and less than or equal to 0.9
All shielding: H/I >0.9;
53 Analyzing the image vehicle information in the fourth step, calculating the number of vehicles and the following distance distribution in the image, evaluating the vehicle density in the ice and snow scene, and determining the road vehicle congestion condition;
firstly, counting the number of vehicles according to the vehicle detection result, and calculating the vehicle density in the image by using the following formula, wherein N is the total number of vehicles in the image, A is the image area, and D is the image vehicle density;
and secondly, analyzing the following distance, and calculating the relative distance between adjacent vehicles in the same lane, wherein the following time interval has the following calculation formula:
in the formula, h ij A following time interval for the ith vehicle and the jth vehicle; v i Is the speed of the ith vehicle; d, d ij Is the following distance; n is the number of vehicles in the current lane; l is the number of lanes of the image road;
and finally, evaluating the road vehicle congestion condition in the ice and snow environment:
wherein c is a vehicle congestion degree coefficient; d (D) max Is the maximum density of vehicles in the image; h min Is the smallest following time interval in the image.
And (3) completing the construction and classification evaluation of the ice and snow environment data set through the first step and the fifth step, and classifying the ice and snow environment natural driving data set by different comprehensive evaluation indexes.
In conclusion, the invention classifies the natural driving data set in the ice and snow environment according to the multidimensional evaluation index and different comprehensive indexes, thereby facilitating the analysis and application of subsequent researchers.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (9)
1. The unmanned aerial vehicle aerial natural driving data set construction method facing the ice and snow environment is characterized by comprising the following steps of:
step one, video acquisition is carried out on urban road traffic flow by using an unmanned aerial vehicle;
step two, rapidly denoising the video image acquired by the unmanned aerial vehicle;
thirdly, road identification is carried out on roads in ice and snow environments, a complete road structure is deduced through existing basic road image information in video images, and road coverage areas are accurately identified;
step four, identifying road vehicles in the video image, extracting the speed of the vehicles, the relative distance between the vehicles and the front vehicle and the track, and predicting the positions and the tracks of the vehicles which are affected by the noise points and lack local vehicle information to complement the missing track information;
and fifthly, establishing a multi-dimensional evaluation index of the traffic data set of the ice and snow environment, and evaluating the traffic state under the ice and snow environment through three methods of ice and snow strength, vehicle flow and road state.
2. The method for constructing the unmanned aerial vehicle aerial natural driving data set for the ice and snow environment according to claim 1, wherein in the first step, the camera angle of the cradle head is ensured to be 90 degrees with the body angle during video acquisition, the road is parallel to the cradle head, the video recording is carried out on the road by using the overlooking view, and the typical structured road is selected for the scene of the photographed road.
3. The method for constructing the unmanned aerial vehicle aerial natural driving data set facing ice and snow environments according to claim 2, wherein the typical structured roads comprise urban expressways, crossroads, expressway-to-expressways and common urban roads.
4. The method for constructing the unmanned aerial vehicle aerial natural driving data set facing the ice and snow environment according to claim 1, wherein the specific method of the second step is as follows:
21 Preprocessing the collected video, and cutting, rotating and scaling;
22 The convolutional neural network is adopted to rapidly denoise and enhance each frame of image, and meanwhile, the details and the definition of the image are maintained;
23 Post-processing the denoised video, performing color correction, contrast adjustment and sharpening.
5. The method for constructing the unmanned aerial vehicle aerial natural driving data set facing the ice and snow environment according to claim 1, wherein the specific method in the third step is as follows:
31 Using the post-processing video of the second step, carrying out road identification based on image segmentation by using a full convolution network, carrying out pixel-level classification on each frame of image, and dividing the image into a road area and a non-road area;
32 Adopting the generation of an countermeasure network model to carry out image filling-based road completion, and processing the information loss of road lane lines in the road caused by the road ice and snow shielding factors;
33 The ice and snow covered area and the uncovered area in the full image are marked and divided by the existing ice and snow edge detection method.
6. The method for constructing an unmanned aerial vehicle aerial natural driving dataset for ice and snow oriented environment according to claim 5, wherein in the step 32), generating the countermeasure network model comprises:
a generator for extracting missing road information from the inputted road image under the ice and snow environment, generating a complete road image, and inputting the complete road image as a road image I under the ice and snow environment in Output as a complete road image I out ；
A discriminator for judging whether the generated road image is true and reliable, i.e. is similar to the true road image, and inputting into a road image I and a road image I under ice and snow environment in The output is a probability value p, which indicates whether the road image is authentic or not.
7. The method for constructing the unmanned aerial vehicle aerial natural driving data set for the ice and snow environment according to claim 6, wherein the specific method of the step 32) is as follows:
randomly selecting a road image I from a real road image dataset real Randomly generating an ice and snow shielding or noise interference area on the real road image to obtain a road image I under the ice and snow environment in The method comprises the steps of carrying out a first treatment on the surface of the Road image I under ice and snow environment in Input into a generator to obtain a generated road image I out ；
Real road image I real And the generated road image I out Respectively with road image I under ice and snow environment in Input into a discriminator to obtain two probability values p real And p out Respectively representing whether the real road image and the generated road image are real and credible;
according to two probability values p real And p out Calculating the loss functions of the generator and the discriminator, and updating the parameters of the generator and the discriminator by using a method of back propagation and gradient descent, so that the generator can generate a more real road image, and the discriminator can more accurately distinguish the real road image from the generated road image;
the loss functions of the generator and the arbiter are respectively:
L G ＝-logp out +λ||I real -I out || 1 (1)
L D ＝-logp real -log(1-p out ) (2)
wherein L is G A loss function of the generator; lambda is a super parameter for controlling the generator to generate a balance between the quality and the authenticity of the road image; I.I 1 Is L1 norm; l (L) D Is the loss function of the arbiter.
8. The method for constructing the unmanned aerial vehicle aerial natural driving data set facing the ice and snow environment according to claim 1, wherein the specific method in the fourth step is as follows:
41 For a video frame, using selective search to generate candidate areas, extracting a plurality of candidate areas containing vehicle targets in the video frame image, using a pretrained convolutional neural network VGG-16 to extract feature vectors for the candidate areas, inputting the feature vectors into a support vector machine classifier so as to judge whether the areas contain vehicles, using a linear regression model to finely adjust the frames of the vehicles for each candidate area judged to contain the vehicles, enabling the frames to be attached to the vehicles, using a kernel correlation filtering algorithm to track the vehicles for the detected vehicles, and allocating unique label IDs for the vehicles, and keeping consistency in the subsequent video frames;
42 The vehicle information extraction method based on OpenCV is designed, the pixel coordinates marking the center of gravity of the vehicle in each frame of image are obtained, so that the pixel coordinate difference on the corresponding time stamp is obtained, then the mapping relation between the pixels and the real coordinates is established, the vehicle coordinate movement condition on each time stamp is obtained, and the driving direction, the speed, the acceleration and the track of the required vehicle are obtained;
the vehicle bounding box is extracted, and in each frame of image, the pixel coordinate of the gravity center of the ith vehicle at the moment t can be expressed as [ x ] i|t ,y i|t ]After passing Δt, the vehicle barycentric pixel coordinates become [ x ] i|t+Δt ,y i|t+Δt ]The mapping relation between the pixel coordinates and the real coordinates is as follows:
[x pixel ,y pixel ]＝k[x real ,y real ] (3)
wherein k is a conversion coefficient between a pixel coordinate and a real coordinate;
calculating the instantaneous speed, the average speed and the acceleration of the vehicle by acquiring pixel coordinates of the same vehicle in different frames, and acquiring the following distances between different vehicles on the same lane in the same frame by acquiring the vehicle coordinates in a period of time to obtain the running track of the vehicle; the specific calculation formulas of the average speed, the acceleration and the following distance are as follows:
Δx i,i+1 ＝k(x i|t -x i+1|t ) (6)
in the method, in the process of the invention,is the average speed of the ith vehicle within Δt; a, a i Acceleration for the ith vehicle; deltax i,i+1 The following distance between the ith vehicle and the (i+1) th vehicle is set;
43 Designing a prediction algorithm based on Kalman filtering to estimate vehicle information in a video frame lost by a vehicle until the vehicle corresponds to the missing information in a subsequent video frame;
for each video frame, detecting whether there is a vehicle that loses recognition tracking due to occlusion, i.e., vehicle information cannot be detected by the aforementioned vehicle recognition step in the present frame, but there is a detected vehicle in the previous frame; if the missing vehicle exists, the motion information of the missing vehicle is complemented by prediction through a Kalman filtering algorithm, and the motion information is represented through a virtual frame;
for lost vehicles, the following motion state is predicted continuously by using Kalman filtering, and a prediction equation of the Kalman filtering is as follows:
in which Q k The process noise covariance matrix is a 3 multiplied by 3 symmetric positive definite matrix;for the state prior estimation at time k, the speed, the acceleration and the running direction of the vehicle are included, namely x k ＝[v k ,a k ,θ k ] T ；/>For covariance a priori estimation at time k, i.e
Wherein,the variance of the speed, acceleration and direction of the state space model; ρ is the covariance between the two;
F k for a state transition matrix, in particular,
wherein Δt is the inverse of the sampling frequency of the acquired video;
stopping prediction until the identifier is detected again in the subsequent video frame, and stopping prediction when the vehicle information is detected again, using a Kalman filtering prediction result to replace the vehicle information in the missing frame, and reusing a solid line frame in the video frame for representation; in addition, a time threshold is set for the Kalman filtering, and if the time threshold is exceeded, the vehicle is considered to leave the video shooting scene, and prediction is not performed any more.
9. The method for constructing the unmanned aerial vehicle aerial natural driving data set facing the ice and snow environment according to claim 1, wherein the specific method in the fifth step is as follows:
51 The gray level co-occurrence matrix model based on image processing is designed to evaluate the ice and snow strength, the non-denoising image in the second step is processed, the image texture characteristics are calculated, and the ice and snow strength in the ice and snow scene is evaluated, wherein the process is as follows:
converting the image into a gray scale image; dividing the gray image into a plurality of small blocks, wherein each small block is the field of one pixel; calculating a gray level co-occurrence matrix for each small block, recording the joint distribution of gray level values between two pixels, and finally calculating texture features according to the gray level co-occurrence matrix; the texture features include contrast, uniformity, and entropy;
the contrast reflects the gray level change degree in the image, and the calculation formula is as follows:
the uniformity reflects the gray level distribution uniformity degree in the image, and the calculation formula is as follows:
the entropy reflects the gray level complexity in the image, and the calculation formula is as follows:
wherein L is the gray level of the gray image; i, j are the pixel values of the image respectively; p (i, j) is the number of times the pixel with gray value i is next to the pixel with gray value j to the right;
the three characteristics of contrast, uniformity and entropy are used for comprehensively evaluating the ice and snow strength, and images with high contrast, low uniformity and high entropy show that the ice and snow strength is high, and conversely show that the ice and snow strength is low;
defining a comprehensive evaluation function of the ice and snow intensity, and defining the ice and snow intensity S for the contrast C, the uniformity U and the entropy E of a given image:
S＝αC-βU+γE (14)
wherein alpha, beta and gamma are weight coefficients;
according to the value of S, the ice and snow intensity is divided into small snow, medium snow, large snow and snow storm:
snow: s is less than or equal to S1;
medium snow: s1 is less than or equal to S2;
snow: s2 is less than or equal to S3;
snow storm: s > S3;
wherein S1, S2 and S3 are threshold values;
52 Analyzing the image coverage information in the third step, and evaluating the road ice and snow coverage so as to determine the road state;
the area ratio of the covered area H to the total road area I is calculated by acquiring the image marked with the ice and snow covered area in the step three, so that the ice and snow coverage of the road is determined;
H＝∑h i (15)
where hi is the area of the ith ice and snow covered region;
according to the ratio, the road ice and snow coverage is divided into non-shielding, partial shielding, most shielding and all shielding:
no shielding: H/I is less than or equal to 0.1
Small part shielding: H/I is 0.1< 0.5 or less
Most of the shielding: H/I is less than or equal to 0.5 and less than or equal to 0.9
All shielding: H/I >0.9;
53 Analyzing the image vehicle information in the fourth step, calculating the number of vehicles and the following distance distribution in the image, evaluating the vehicle density in the ice and snow scene, and determining the road vehicle congestion condition;
firstly, counting the number of vehicles according to the vehicle detection result, and calculating the vehicle density in the image by using the following formula, wherein N is the total number of vehicles in the image, A is the image area, and D is the image vehicle density;
and secondly, analyzing the following distance, and calculating the relative distance between adjacent vehicles in the same lane, wherein the following time interval has the following calculation formula:
in the formula, h ij A following time interval for the ith vehicle and the jth vehicle; v i Is the speed of the ith vehicle; d, d ij Is the following distance; n is the number of vehicles in the current lane; l is the number of lanes of the image road;
and finally, evaluating the road vehicle congestion condition in the ice and snow environment:
wherein c is a vehicle congestion degree coefficient; d (D) max Is the maximum density of vehicles in the image; h min Is the smallest following time interval in the image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311648037.0A CN117636268A (en) | 2023-12-04 | 2023-12-04 | Unmanned aerial vehicle aerial natural driving data set construction method oriented to ice and snow environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311648037.0A CN117636268A (en) | 2023-12-04 | 2023-12-04 | Unmanned aerial vehicle aerial natural driving data set construction method oriented to ice and snow environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117636268A true CN117636268A (en) | 2024-03-01 |
Family
ID=90021316
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311648037.0A Pending CN117636268A (en) | 2023-12-04 | 2023-12-04 | Unmanned aerial vehicle aerial natural driving data set construction method oriented to ice and snow environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117636268A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118101970A (en) * | 2024-04-17 | 2024-05-28 | 哈尔滨师范大学 | Deep learning-based high-efficiency communication method for ice and snow project site monitoring images |
-
2023
- 2023-12-04 CN CN202311648037.0A patent/CN117636268A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118101970A (en) * | 2024-04-17 | 2024-05-28 | 哈尔滨师范大学 | Deep learning-based high-efficiency communication method for ice and snow project site monitoring images |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3614308B1 (en) | Joint deep learning for land cover and land use classification | |
| CN107818326B (en) | A kind of ship detection method and system based on scene multidimensional characteristic | |
| CN101141633B (en) | Moving object detecting and tracing method in complex scene | |
| CN102542289B (en) | Pedestrian volume statistical method based on plurality of Gaussian counting models | |
| CN106778595B (en) | Method for detecting abnormal behaviors in crowd based on Gaussian mixture model | |
| CN104200657B (en) | A kind of traffic flow parameter acquisition method based on video and sensor | |
| CN101339655B (en) | Visual sense tracking method based on target characteristic and bayesian filtering | |
| CN108710875A (en) | A kind of take photo by plane road vehicle method of counting and device based on deep learning | |
| CN111626290A (en) | Infrared ship target detection and identification method under complex sea surface environment | |
| CN110992381A (en) | Moving target background segmentation method based on improved Vibe + algorithm | |
| CN110334692B (en) | Blind road identification method based on image processing | |
| CN101976504B (en) | Multi-vehicle video tracking method based on color space information | |
| CN108921076B (en) | Pavement crack disease self-adaptive constant false alarm detection method based on image | |
| CN108804992B (en) | Crowd counting method based on deep learning | |
| CN110633678B (en) | Quick and efficient vehicle flow calculation method based on video image | |
| CN109685045A (en) | A kind of Moving Targets Based on Video Streams tracking and system | |
| CN117636268A (en) | Unmanned aerial vehicle aerial natural driving data set construction method oriented to ice and snow environment | |
| Xing et al. | Traffic sign recognition using guided image filtering | |
| CN111797738A (en) | Multi-target traffic behavior fast extraction method based on video identification | |
| CN106778540A (en) | Parking detection is accurately based on the parking event detecting method of background double layer | |
| CN115841633A (en) | Power tower and power line associated correction power tower and power line detection method | |
| CN106056078A (en) | Crowd density estimation method based on multi-feature regression ensemble learning | |
| CN116758421A (en) | Remote sensing image directed target detection method based on weak supervised learning | |
| CN104123569B (en) | Video person number information statistics method based on supervised learning | |
| CN113343819B (en) | Efficient unmanned airborne SAR image target segmentation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |