CN115457505A - Small obstacle detection method, device and equipment for camera and storage medium - Google Patents
Small obstacle detection method, device and equipment for camera and storage medium Download PDFInfo
- Publication number
- CN115457505A CN115457505A CN202211059142.6A CN202211059142A CN115457505A CN 115457505 A CN115457505 A CN 115457505A CN 202211059142 A CN202211059142 A CN 202211059142A CN 115457505 A CN115457505 A CN 115457505A
- Authority
- CN
- China
- Prior art keywords
- obstacle
- vanishing
- pixel points
- points
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims description 24
- 238000000034 method Methods 0.000 claims abstract description 38
- 238000010586 diagram Methods 0.000 claims abstract description 19
- 238000012549 training Methods 0.000 claims abstract description 17
- 238000002372 labelling Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims description 14
- 102100034112 Alkyldihydroxyacetonephosphate synthase, peroxisomal Human genes 0.000 claims description 12
- 101000799143 Homo sapiens Alkyldihydroxyacetonephosphate synthase, peroxisomal Proteins 0.000 claims description 12
- 238000000848 angular dependent Auger electron spectroscopy Methods 0.000 claims description 12
- 230000004888 barrier function Effects 0.000 claims description 10
- 230000008447 perception Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 2
- 230000007613 environmental effect Effects 0.000 abstract description 10
- 230000006870 function Effects 0.000 description 12
- 230000003287 optical effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005282 brightening Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a method, a device, equipment and a storage medium for detecting small obstacles of a camera, wherein the method comprises the following steps: dividing the acquired road environment image, and inputting the divided image into a training model to obtain a hot spot map of a vanishing point; connecting vanishing points in the heat point diagram, and determining the area of the obstacle according to statistics of the vanishing points and the position of the obstacle; gridding the determined region of the obstacle, preliminarily judging the obstacle pixel points of the gridding region, and labeling the preliminarily judged obstacle pixel points; and judging the marked obstacle pixel points to determine the small obstacle at a long distance. The obstacle avoidance method and the obstacle avoidance system can detect small obstacles in advance under various environmental conditions, provide a necessary obstacle avoidance strategy for an automatic driving function under the condition that additional configuration is not added, can avoid potential safety hazards simultaneously, and improve driving experience.
Description
Technical Field
The invention relates to the technical field of vehicle environment sensing detection, in particular to a method, a device, equipment and a storage medium for detecting a small obstacle of a camera.
Background
At present, an ADAS environment perception camera system is used for carrying out picture comparison and identification on large dynamic targets such as vehicles, pedestrians and the like, and can be used for detecting remote dynamic targets independently or in combination with a millimeter wave radar.
The existing ADAS environmental perception camera system can only identify large targets such as vehicles, pedestrians and the like or moving targets, but cannot identify tiny targets.
The existing ultrasonic radar system can only detect a short-distance obstacle of about 5-7m, but cannot detect a long-distance target and cannot be used for ADAS or other automatic driving functions. Although the existing laser radar can detect small obstacles in a certain range, due to the design characteristics of the laser radar, the detection is a certain blind area with periodic property, and the laser radar is high in cost and not beneficial to popularization of an automatic driving function.
Therefore, how to identify a small obstacle on a long-distance road to avoid potential safety hazards is a technical problem which needs to be solved urgently at present.
Disclosure of Invention
The invention mainly aims to provide a small obstacle detection method, a small obstacle detection device, small obstacle detection equipment and a storage medium of a camera, which can detect small obstacles in advance under various environmental conditions, provide a necessary obstacle avoidance strategy for an automatic driving function under the condition of not increasing additional configuration, avoid potential safety hazards and improve driving experience.
In a first aspect, the present application provides a small obstacle detection method for a camera, including:
segmenting the acquired road environment image, and inputting the segmented image into a training model to obtain a hot spot map of the vanishing point;
connecting vanishing points in the heat point diagram, and determining the area of the obstacle according to statistics of the vanishing points and the position of the obstacle;
gridding the determined region of the obstacle, preliminarily judging the obstacle pixel points of the gridded region, and labeling the preliminarily judged obstacle pixel points;
and judging the marked obstacle pixel points to determine the small obstacle at a long distance.
With reference to the first aspect, as an optional implementation manner, three times of comparison are performed on the obstacle pixel points labeled in the fixed grid, and a small obstacle that is far away is determined when the labeled obstacle pixel points are larger than a set threshold.
With reference to the first aspect, as an optional implementation manner, after a first signal period, performing a first image comparison on the marked obstacle pixel points in a 32 × 32 grid;
performing a second image comparison on the marked obstacle pixel points at a 32x96 grid after the second signal period;
after a third signal period, performing a third image comparison on the marked obstacle pixel points at a 96x96 grid;
wherein, the set threshold is 100 pixels.
With reference to the first aspect, as an optional implementation manner, vanishing points in the hot spot diagram are connected into a straight line, and an area of the obstacle is determined according to a pixel distance distribution interval between the vanishing point and the obstacle.
With reference to the first aspect, as an optional implementation manner, a relative coordinate system between the vanishing point and the obstacle is established;
the relative coordinate is defined as the coordinate of the center point of the obstacle minus the coordinate of the vanishing point and then is taken as positive, wherein the point (0,0) represents the original point position of the vanishing point;
and determining the position relation between the long-distance barrier on the road and the road vanishing point according to the position of the vanishing point.
With reference to the first aspect, as an optional implementation manner, the gridding an area of a determined obstacle, preliminarily determining obstacle pixel points of the gridding area, and labeling the preliminarily determined obstacle pixel points includes:
fixing grids of 32x32 pixel points where the far-end vanishing points of the gridding area have breakpoints or difference points and the pixel points with vanishing point truth values not 1 on the hot spot map coded by the Gaussian kernel are located;
and preliminarily judging the pixel points in the fixed grids as the pixel points of the barrier and labeling.
With reference to the first aspect, as an optional implementation manner, the step of segmenting the acquired road environment image, and inputting the segmented image into a training model to obtain a hot spot map of a vanishing point includes:
dividing a road environment image acquired by an ADAS environment perception camera into four parts according to the ratio of the sky to the ground, wherein the intersection point of the four parts is a region with the maximum distribution probability of vanishing points;
and inputting the four divided parts into a training model to obtain a hot spot diagram of the vanishing point.
In a second aspect, the present application provides a minimum obstacle detection apparatus for a camera, the apparatus including:
the acquisition module is used for segmenting the acquired road environment image and inputting the segmented image into the training model to obtain a hot spot map of the vanishing point;
the determining module is used for connecting the vanishing points in the heat point diagram and determining the area of the obstacle according to the statistics of the vanishing points and the position of the obstacle;
the processing module is used for gridding the determined region of the obstacle, preliminarily judging the obstacle pixel points of the gridding region and marking the preliminarily judged obstacle pixel points;
and the judging module is used for judging the marked obstacle pixel points so as to determine the small obstacle in a long distance.
In combination with the second aspect described above, as an alternative implementation,
in a third aspect, the present application further provides an electronic device, including: a processor; a memory having computer readable instructions stored thereon which, when executed by the processor, implement the method of any of the first aspects.
In a fourth aspect, the present application also provides a computer readable storage medium storing computer program instructions which, when executed by a computer, cause the computer to perform the method of any of the first aspects.
The application provides a small obstacle detection method, a device, equipment and a storage medium for a camera, wherein the method comprises the following steps: dividing the acquired road environment image, and inputting the divided image into a training model to obtain a hot spot map of a vanishing point; connecting vanishing points in the heat point diagram, and determining the area of the obstacle according to statistics of the vanishing points and the position of the obstacle; gridding the determined region of the obstacle, preliminarily judging the obstacle pixel points of the gridding region, and labeling the preliminarily judged obstacle pixel points; and judging the marked obstacle pixel points to determine the small obstacle in a long distance. The obstacle avoidance method and the obstacle avoidance system can detect small obstacles in advance under various environmental conditions, provide a necessary obstacle avoidance strategy for an automatic driving function under the condition that additional configuration is not added, can avoid potential safety hazards simultaneously, and improve driving experience.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
Fig. 1 is a flowchart of a small obstacle detection method for a camera provided in an embodiment of the present application;
fig. 2 is a schematic view of a small obstacle detection device of a camera provided in an embodiment of the present application;
FIG. 3 is a schematic diagram of an electronic device provided in an embodiment of the present application;
fig. 4 is a schematic diagram of a computer-readable program medium provided in an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities.
The embodiment of the application provides a small obstacle detection method, a small obstacle detection device, small obstacle detection equipment and a storage medium of a camera, which can detect small obstacles in advance under various environmental conditions, provide a necessary obstacle avoidance strategy for an automatic driving function under the condition of not increasing additional configuration, avoid potential safety hazards and improve driving experience.
In order to achieve the technical effect, the general idea of the application is as follows:
a small obstacle detection method for a camera, the method comprising the steps of:
s101: and segmenting the acquired road environment image, and inputting the segmented image into a training model to obtain a hot spot map of the vanishing point.
S102: and connecting the vanishing points in the heat point diagram, and determining the area of the obstacle according to the statistics of the vanishing points and the position of the obstacle.
S103: and gridding the determined region of the obstacle, preliminarily judging the obstacle pixel points of the gridding region, and labeling the preliminarily judged obstacle pixel points.
S104: and judging the marked obstacle pixel points to determine the small obstacle at a long distance.
Embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a flowchart illustrating a small obstacle detection method for a camera according to the present invention, and as shown in fig. 1, the method includes the steps of:
and S101, segmenting the acquired road environment image, and inputting the segmented image into a training model to obtain a hot spot map of a vanishing point.
Specifically, a road environment image is collected through an ADAS environment perception camera, the road environment image is calibrated according to the space proportion of the camera, the space proportion is the proportion of the sky to the ground, namely the road environment image collected through the ADAS environment perception camera is divided into four parts, the four parts can be understood as four quadrants, the predicted position of a road vanishing point is predicted by covering 4 quadrants of a road environment scene, and the intersection point of the 4 quadrants is a region with the maximum vanishing point distribution probability. It should be noted that, if the road is not a narrow space, the road vanishing point is generally located on the calibrated occupancy boundary, and it can be understood that the road vanishing point is generally an intersection of four quadrants, or is near the intersection of four quadrants, and if the vehicle is located on a slope or a corner, the vanishing point deviates from the intersection of four quadrants and is distributed at a slightly longer distance from the intersection of four quadrants.
The ADAS is an Advanced Driving Assistance System (Advanced Driving Assistance System) which utilizes various sensors (millimeter wave radar, laser radar, monocular/binocular camera and satellite navigation) installed on a vehicle to sense the surrounding environment at any time during the Driving process of the vehicle, collect data, identify, detect and track static and dynamic objects, and combine navigation map data to perform systematic operation and analysis, thereby enabling drivers to perceive possible dangers in advance and effectively increasing the comfort and safety of vehicle Driving. It should be noted that the ADAS environmental perception camera is used for collecting video information of a real scene environment, and for accurately determining target parameters of a target object by an original function, the original function of the ADAS environmental perception camera can visualize the target types, the transverse and longitudinal distances, the target length and width, the speed and the like of the targets such as vehicles and pedestrians, and also provide the original input of the road surface vanishing point area range image for the present invention.
The vanishing point is a visual intersection of parallel lines, which is understood to mean that when you look at two rails along a railway line and at two regularly arranged trees along a public line, the two parallel rails or two rows of trees intersect at a point that is far away, which is called the vanishing point in the perspective view.
In an embodiment, whether the vanishing point is located at the left or right of the center can be further judged by combining the left and right of the lane line. Specifically, comparison and calibration are carried out according to the judgment results of the gray processing and color boundary processing vanishing point of the image, the judgment results of lane line information and the like.
The method comprises the steps of dividing an image collected by an ADAS camera into four quadrants, inputting the divided images into an established road vanishing point thermal model, wherein the input of the model is a collected picture, the input picture is a picture of an original image after data brightening operation during training, and the input picture is 1344 multiplied by 768, and the model input is the original image during testing, and the size is 2 multiplied by 1 242. Carrying out 32 times of downsampling by using a ResNet-50 feature extraction network to obtain a deep feature map, wherein the dimension is 2 048 multiplied by 42; and then, carrying out Atrousconvolution (expansion convolution) on the obtained depth features for 3 times to obtain feature vectors with the dimensions of 256 multiplied by 42 multiplied by 24, wherein the intervals of the expansion convolution for 3 times are respectively 2, 4 and 8. After 1 convolution, a quadrant graph of 4 × 42 × 24 is obtained, and then after 1 convolution, a hotspot graph of vanishing points is obtained, wherein the hotspot graph of the vanishing points has a plurality of vanishing points.
The loss function of the whole road vanishing point detection network model RVPNet is as follows:
L vp ＝L qua +L vph ，L vp = -1/N detect vanishing points, where Lqua is cross entropy loss, used to generate quadrant graph; lvph is a variant of focal loss used to generate a hotspot map of vanishing points; m represents the number of samples; y is m Represents the true value; y represents a predicted value; n is the number of vanishing points in each picture.
In one embodiment, the output of the detection network model is defined as 4 quadrant channels. Each pixel in the output image will belong to 1 of these 4 channels, with the 4 quadrant channels representing one of the 4 quadrants on the image.
And S102, connecting the vanishing points in the heat point diagram, and determining the area of the obstacle according to the statistics of the vanishing points and the position of the obstacle.
Processing the segmented image according to the training model to obtain a hotspot graph of vanishing points, wherein the hotspot graph of the vanishing points is provided with a plurality of vanishing points, the vanishing points in the hotspot graph are connected into a straight line, and the vanishing points and the positions of the obstacles are statistically divided, so that the main calculation area, namely, the area (feasible area) of the small obstacle is judged. The statistical division of the positions of the obstacle and the obstacle is specifically that in the statistics of the pixel distances between the obstacle and the vanishing point, the number of the pixel distances between the obstacle and the vanishing point distributed in the interval of 0-30 is the largest, and more than 90% of the pixel distances between the obstacle and the vanishing point are within 180%. The pixel distance distribution interval between the barrier and the vanishing point can be understood as 0-30 pixel points and 30-180 pixel points.
Establishing a relative coordinate system between the vanishing point and the obstacle, wherein the relative coordinate is defined as the coordinate of the center point of the obstacle minus the coordinate of the vanishing point and then taking the positive, wherein the point (0,0) represents the original position of the vanishing point, and the obstacle can be seen to be distributed around the vanishing point, so that a certain position relation exists between the long-distance obstacle on the road and the vanishing point of the road. Accurate detection of road vanishing points will facilitate detection of distant obstacles. Therefore, in the present application, the pixel area in the road vanishing point 180 is taken as the reference area to perform the reference judgment.
It can be understood that vanishing points in the heat point diagram are connected into a straight line, and the obstacle distribution area can be determined to be 0-180 pixel points according to the vanishing points and the positions of the obstacles.
And step S103, gridding the determined region of the obstacle, primarily judging the obstacle pixel points of the gridded region, and labeling the primarily judged obstacle pixel points.
Specifically, the determined obstacle region is gridded in order to determine an obstacle in the obstacle region, specifically, after the obstacle region is determined, the obstacle is detected in the region, a pixel region in the road vanishing point 180 is taken as a reference region, the region is divided into basic grids of 32 × 32 (calibratable) pixel points, and an image in only one grid is processed in 1 signal cycle.
The processing mode is that a breakpoint or a difference point exists in a far-end vanishing point, and a basic grid of 32x32 pixel points where y _ cij indicates that pixel points of which vanishing point truth values on a hot spot graph coded by a Gaussian kernel are not 1 are located is fixed, and the pixel points which are preliminarily determined as the obstacles are labeled.
And step S104, judging the marked obstacle pixel points to determine a long-distance small obstacle.
Specifically, marking pixel points which are preliminarily determined as obstacles in a fixed grid, and carrying out three times of comparison on marked obstacle pixel points, wherein the three times of comparison specifically comprises carrying out first time image comparison on the marked obstacle pixel points in a 32x32 grid after a signal period, carrying out second time image comparison on the marked obstacle pixel points in a 32x96 grid after a second signal period, and carrying out third time image comparison on the marked obstacle pixel points in a 96x96 grid after a third signal period, wherein the comparison is the marked pixel points.
In one embodiment, after three times of comparison are performed on the marked obstacle pixel points in the fixed grid, the minimum obstacle in the distance is determined when the marked obstacle pixel points are larger than 100 pixel points.
This application can start this little barrier detecting system under various environmental conditions, has guaranteed that the autopilot function can survey in advance little barrier, guarantees that the function detours.
Referring to fig. 2, fig. 2 is a schematic view of a small obstacle detection device of a camera according to the present invention, as shown in fig. 2, including:
the acquisition module 201: the method is used for segmenting the acquired road environment image and inputting the segmented image into the training model to obtain the hot spot map of the vanishing point.
The determination module 202: and the device is used for connecting the vanishing points in the heat point diagram and determining the area of the obstacle according to the statistics of the vanishing points and the position of the obstacle.
The processing module 203: the method is used for gridding the determined region of the obstacle, preliminarily judging the obstacle pixel points of the gridded region, and labeling the preliminarily judged obstacle pixel points.
The judging module 204: and the method is used for judging the marked obstacle pixel points so as to determine the small obstacle at a long distance.
Further, in a possible implementation manner, the determining module 204 is further configured to compare the labeled obstacle pixel points in the fixed grid for three times, and determine that the labeled obstacle pixel points are larger than a set threshold value, and determine that the labeled obstacle pixel points are a small remote obstacle.
Further, in a possible embodiment, the determining module 204 is further configured to perform a first image comparison on the labeled obstacle pixel point in the 32 × 32 grid after the first signal period;
performing a second image comparison of the marked obstacle pixel points at a 32x96 grid after the second signal period;
after a third signal period, performing a third image comparison on the marked obstacle pixel points at a 96x96 grid;
wherein, the set threshold is 100 pixels.
Further, in a possible implementation manner, the determining module 202 is further configured to connect vanishing points in the heat point map into a straight line, and determine the area of the obstacle according to a pixel distance distribution interval between the vanishing point and the obstacle.
Further, in a possible implementation, the determining module 202 is further configured to establish a relative coordinate system between the vanishing point and the obstacle;
the relative coordinate is defined as the coordinate of the center point of the obstacle minus the coordinate of the vanishing point and then is taken as positive, wherein the point (0,0) represents the original point position of the vanishing point;
and determining the position relation between the long-distance barrier on the road and the road vanishing point according to the position of the vanishing point.
Further, in a possible implementation manner, the processing module 203 is further configured to fix a grid of 32 × 32 pixels where a pixel where a vanishing point true value on the hotspot graph coded by the gaussian kernel is not 1 exists and a breakpoint or a difference exists at a far-end vanishing point of the gridding region;
and preliminarily judging the pixel points in the fixed grids as the pixel points of the barrier and labeling.
Further, in a possible implementation manner, the acquisition module 201 is further configured to divide the road environment image acquired by the ADAS environmental perception camera into four parts according to a ratio of the sky to the ground, where an intersection point of the four parts is a region with a maximum distribution probability of the vanishing point; and inputting the four divided parts into a training model to obtain a hot spot diagram of the vanishing point.
An electronic device 300 according to this embodiment of the invention is described below with reference to fig. 3. The electronic device 300 shown in fig. 3 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 3, electronic device 300 is embodied in the form of a general purpose computing device. The components of electronic device 300 may include, but are not limited to: the at least one processing unit 310, the at least one memory unit 320, and a bus 330 that couples various system components including the memory unit 320 and the processing unit 310.
Wherein the storage unit stores program code that can be executed by the processing unit 310, such that the processing unit 310 performs the steps according to various exemplary embodiments of the present invention described in the section "example methods" above in this specification.
The storage unit 320 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM) 321 and/or a cache memory unit 322, and may further include a read only memory unit (ROM) 323.
The storage unit 320 may also include a program/utility 324 having a set (at least one) of program modules 325, such program modules 325 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may comprise an implementation of a network environment.
The electronic device 300 may also communicate with one or more external devices 400 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 300, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 300 to communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interface 350. Also, electronic device 300 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via network adapter 360. As shown, network adapter 360 communicates with the other modules of electronic device 300 via bus 330. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
According to an aspect of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above section "exemplary methods" of the present description, when said program product is run on the terminal device.
Referring to fig. 4, a program product 400 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
Furthermore, the above-described drawings are only schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily appreciated that the processes illustrated in the above figures are not intended to indicate or limit the temporal order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
To sum up, the present application provides a method, an apparatus, a device and a storage medium for detecting a small obstacle of a camera, where the method includes the steps of: dividing the acquired road environment image, and inputting the divided image into a training model to obtain a hot spot map of a vanishing point; connecting vanishing points in the heat point diagram, and determining the area of the obstacle according to statistics of the vanishing points and the position of the obstacle; gridding the determined region of the obstacle, preliminarily judging the obstacle pixel points of the gridding region, and labeling the preliminarily judged obstacle pixel points; and judging the marked obstacle pixel points to determine the small obstacle in a long distance. The obstacle avoidance method and the obstacle avoidance system can detect small obstacles in advance under various environmental conditions, provide a necessary obstacle avoidance strategy for an automatic driving function under the condition that additional configuration is not added, can avoid potential safety hazards simultaneously, and improve driving experience.
The foregoing are merely exemplary embodiments of the present application and no attempt is made to show structural details of the invention in more detail than is necessary for the fundamental understanding of the art, the description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be embodied in practice with the teachings of the invention. It should be noted that, for those skilled in the art, without departing from the structure of the present invention, several variations and modifications can be made, which should also be considered as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the utility of the patent. The scope of the claims of the present application shall be determined by the contents of the claims, and the description of the embodiments and the like in the specification shall be used to explain the contents of the claims.
Claims (10)
1. A small obstacle detection method for a camera is characterized by comprising the following steps:
dividing the acquired road environment image, and inputting the divided image into a training model to obtain a hot spot map of a vanishing point;
connecting vanishing points in the heat point diagram, and determining the area of the obstacle according to statistics of the vanishing points and the position of the obstacle;
gridding the determined region of the obstacle, preliminarily judging the obstacle pixel points of the gridding region, and labeling the preliminarily judged obstacle pixel points;
and judging the marked obstacle pixel points to determine the small obstacle in a long distance.
2. The method of claim 1, wherein said determining labeled obstacle pixel points to determine a small obstacle at a distance comprises:
and comparing the marked obstacle pixel points in the fixed grid for three times, and judging that the marked obstacle pixel points are larger than a set threshold value as a small remote obstacle.
3. The method of claim 2, wherein:
performing a first image comparison of the marked obstacle pixel points at a 32x32 grid after the first signal period;
performing a second image comparison of the marked obstacle pixel points at a 32x96 grid after the second signal period;
after a third signal period, performing a third image comparison on the marked obstacle pixel points at a 96x96 grid;
wherein, the set threshold is 100 pixels.
4. The method according to claim 1, wherein the connecting vanishing points in the heat point map and determining the area of the obstacle according to the vanishing points and the obstacle position statistics comprises:
and connecting the vanishing points in the heat point diagram into a straight line, and determining the area of the obstacle according to the pixel distance distribution interval between the vanishing points and the obstacle.
5. The method of claim 4, wherein the statistics based on vanishing points and obstacle positions further comprises:
establishing a relative coordinate system between the vanishing point and the obstacle;
the relative coordinate is defined as the coordinate of the center point of the barrier minus the coordinate of the vanishing point and then is taken as positive, wherein the point (0,0) represents the original point position of the vanishing point;
and determining the position relation between the long-distance barrier on the road and the road vanishing point according to the position of the vanishing point.
6. The method according to claim 1, wherein the gridding the determined region of the obstacle, performing a preliminary determination on the obstacle pixel points of the gridded region, and labeling the preliminarily determined obstacle pixel points comprises:
fixing grids of 32x32 pixel points where the far-end vanishing points of the gridding area have breakpoints or difference points and the pixel points with vanishing point truth values not 1 on the hot spot map coded by the Gaussian kernel are located;
and preliminarily judging the pixel points in the fixed grids as the pixel points of the barrier and labeling.
7. The method of claim 1, wherein the acquired road environment image is segmented, and the segmented image is input into a training model to obtain a hot spot map of vanishing points, and the method comprises:
dividing a road environment image acquired by an ADAS environment perception camera into four parts according to the ratio of the sky to the ground, wherein the intersection point of the four parts is a region with the maximum distribution probability of vanishing points;
and inputting the four divided parts into a training model to obtain a hot spot diagram of the vanishing point.
8. A minimum obstacle detection device of a camera, comprising:
the acquisition module is used for segmenting the acquired road environment image and inputting the segmented image into the training model to obtain a hot spot map of the vanishing point;
the determining module is used for connecting the vanishing points in the heat point diagram and determining the area of the obstacle according to the statistics of the vanishing points and the position of the obstacle;
the processing module is used for gridding the determined region of the obstacle, preliminarily judging the obstacle pixel points of the gridding region and labeling the preliminarily judged obstacle pixel points;
and the judging module is used for judging the marked obstacle pixel points so as to determine the small obstacle in a long distance.
9. An electronic device, characterized in that the electronic device comprises:
a processor;
a memory having stored thereon computer readable instructions which, when executed by the processor, implement the method of any of claims 1 to 7.
10. A computer-readable storage medium, characterized in that it stores computer program instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211059142.6A CN115457505A (en) | 2022-08-31 | 2022-08-31 | Small obstacle detection method, device and equipment for camera and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211059142.6A CN115457505A (en) | 2022-08-31 | 2022-08-31 | Small obstacle detection method, device and equipment for camera and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115457505A true CN115457505A (en) | 2022-12-09 |
Family
ID=84301847
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211059142.6A Pending CN115457505A (en) | 2022-08-31 | 2022-08-31 | Small obstacle detection method, device and equipment for camera and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115457505A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116630923A (en) * | 2023-05-22 | 2023-08-22 | 小米汽车科技有限公司 | Marking method and device for vanishing points of roads and electronic equipment |
-
2022
- 2022-08-31 CN CN202211059142.6A patent/CN115457505A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116630923A (en) * | 2023-05-22 | 2023-08-22 | 小米汽车科技有限公司 | Marking method and device for vanishing points of roads and electronic equipment |
| CN116630923B (en) * | 2023-05-22 | 2024-01-02 | 小米汽车科技有限公司 | Marking method and device for vanishing points of roads and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109271944B (en) | Obstacle detection method, obstacle detection device, electronic apparatus, vehicle, and storage medium | |
| CN115273039B (en) | Small obstacle detection method based on camera | |
| EP3581890A2 (en) | Method and device for positioning | |
| US20200217656A1 (en) | Method and apparatus for measuring distance using vehicle-mounted camera, storage medium, and electronic device | |
| CN110516621B (en) | Method and device for detecting barrier-free driving area, vehicle and storage medium | |
| CN110298311B (en) | Method and device for detecting surface water accumulation | |
| CN108399360B (en) | Continuous obstacle detection method, device and terminal | |
| CN114882316A (en) | Target detection model training method, target detection method and device | |
| KR101030317B1 (en) | Apparatus for tracking obstacle using stereo vision and method thereof | |
| Rasib et al. | Pixel level segmentation based drivable road region detection and steering angle estimation method for autonomous driving on unstructured roads | |
| CN116071729A (en) | Method and device for detecting drivable area and road edge and related equipment | |
| CN115755097A (en) | Weather condition detection method, device, equipment and storage medium | |
| CN115457505A (en) | Small obstacle detection method, device and equipment for camera and storage medium | |
| CN115147809A (en) | Obstacle detection method, device, equipment and storage medium | |
| CN113421217A (en) | Method and device for detecting travelable area | |
| CN113192057A (en) | Target detection method, system, device and storage medium | |
| KR20190134303A (en) | Apparatus and method for image recognition | |
| Oniga et al. | A fast ransac based approach for computing the orientation of obstacles in traffic scenes | |
| Ahmed et al. | Lane marking detection using LiDAR sensor | |
| CN114565906A (en) | Obstacle detection method, obstacle detection device, electronic device, and storage medium | |
| CN114998863A (en) | Target road identification method, target road identification device, electronic equipment and storage medium | |
| CN114842431A (en) | Method, device and equipment for identifying road guardrail and storage medium | |
| Linfeng et al. | One estimation method of road slope and vehicle distance | |
| CN108416305B (en) | Pose estimation method and device for continuous road segmentation object and terminal | |
| CN118172423B (en) | Sequential point cloud data pavement element labeling method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |