CN113781543A - Binocular camera-based height limiting device detection method and system and intelligent terminal - Google Patents
Binocular camera-based height limiting device detection method and system and intelligent terminal Download PDFInfo
- Publication number
- CN113781543A CN113781543A CN202111324101.0A CN202111324101A CN113781543A CN 113781543 A CN113781543 A CN 113781543A CN 202111324101 A CN202111324101 A CN 202111324101A CN 113781543 A CN113781543 A CN 113781543A
- Authority
- CN
- China
- Prior art keywords
- target
- limiting device
- height
- disparity map
- height limiting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/02—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
- G01B11/06—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness for measuring thickness ; e.g. of sheet material
- G01B11/0608—Height gauges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Electromagnetism (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a binocular camera-based height limiting device detection method, a binocular camera-based height limiting device detection system and an intelligent terminal, wherein the method comprises the following steps: constructing a target detection model, and training the target detection model by using a target detection algorithm to obtain a deep learning target detection model; acquiring the position of a target height limiting device based on the deep learning target detection model, and marking the target height limiting device on an original image of a binocular camera; acquiring an original disparity map of the binocular camera, and cleaning the original disparity map to obtain a target disparity map; and calculating the height of the target height limiting device and the distance between the target height limiting device and the current vehicle based on the target disparity map. The technical problem of among the prior art limit for height device's height, distance detection accuracy relatively poor is solved.
Description
Technical Field
The invention relates to the technical field of auxiliary driving, in particular to a binocular camera-based height limiting device detection method and system and an intelligent terminal.
Background
With the continuous development of automobile technology, the number of vehicles with higher vehicle body heights, such as large buses, large trucks, motor homes and the like, continuously increases. However, accidents such as casualties and property loss due to collisions between vehicles and height-limiting devices occur frequently due to negligence of drivers of large vehicles or failure to accurately estimate the height of the height-limiting devices. The main reasons for the occurrence of an accident are as follows: 1) the characteristics of color, texture and the like of part of height limiting devices are not obvious, and the characteristics are difficult to find by a driver; 2) the actual height of the height limiting device is marked by the part of the height limiting device without the signboard or the height marked by the height limiting signboard is not consistent with the height of the actual height limiting device.
Based on the reasons, the binocular camera-based height limiting device detection method, system and intelligent terminal are provided, so that the position accuracy, height accuracy and distance accuracy of the height limiting device are improved, a driver of a large vehicle with a collision risk in the height direction can obtain accurate early warning in advance, collision accidents are avoided, and driving safety is guaranteed.
Disclosure of Invention
Therefore, the embodiment of the invention provides a binocular camera-based height limiting device detection method, a binocular camera-based height limiting device detection system and an intelligent terminal, and aims to solve the technical problem that in the prior art, the height and distance detection accuracy of a height limiting device is poor.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
a binocular camera-based height limiting device detection method comprises the following steps:
constructing a target detection model, and training the target detection model by using a target detection algorithm to obtain a deep learning target detection model;
acquiring the position of a target height limiting device based on the deep learning target detection model, and marking the target height limiting device on an original image of a binocular camera;
acquiring an original disparity map of the binocular camera, and cleaning the original disparity map to obtain a target disparity map;
and calculating the height of the target height limiting device and the distance between the target height limiting device and the current vehicle based on the target disparity map.
Further, the constructing a target detection model, and training the target detection model by using a target detection algorithm to obtain a deep learning target detection model specifically includes:
and after constructing a height limiting device data set and finishing accurate marking, selecting the SSD as a deep learning target detection algorithm to train a target detection model.
Further, the obtaining the position of the target height limiting device based on the deep learning target detection model specifically includes:
and inputting the test set data in the height limiting device data set into the model to test the model.
Further, the cleaning of the disparity map is performed to obtain a target disparity map, and the method specifically includes:
carrying out coarse-grained cleaning on the original disparity map by using mean filtering to obtain a disparity map after coarse cleaning;
and carrying out fine-grained cleaning on the parallax image after the coarse cleaning to obtain the target parallax image.
Further, the performing coarse-grained cleaning on the original disparity map by using mean filtering specifically includes:
setting a parallax maximum threshold value and a parallax minimum threshold value;
calculating a non-zero average value of the whole image of the original disparity map;
calculating the average value and the non-zero average value of each line in the original disparity map according to the line, and calculating the difference value between the line average value and the non-zero average value;
if the result of the comparison calculation exceeds the maximum threshold value, setting the row to be 0;
if the calculation is below the min threshold, the row is considered to be more empty and the row is replaced with a full-map non-zero average.
Further, the fine-grained cleaning of the disparity map after the coarse cleaning specifically includes:
setting fine-grained filtering step number =15 and fine-grained filtering step length =3, and repeatedly cleaning;
and the cleaning times are the product of the filtering step number and the filtering step length, wherein the filtering step number is reduced by 1 and the filtering step number is increased by 1 when cleaning is finished for 1 time until the cleaning times are finished.
Further, the calculating the height of the target height limiting device based on the target disparity map specifically includes:
the height of the target height-limiting device is calculated according to the following formula:
wherein the content of the first and second substances,heightwhich represents the height of the target object,Vrepresenting the coordinates of the target phase element,V 0 representing the coordinates of the optical center of the camera,disparityrepresenting the magnitude of the disparity value.
Further, calculating the distance between the target height limiting device and the current vehicle based on the target disparity map specifically includes:
calculating a disparity value of the target height limiting device on the target disparity map based on the target disparity map;
calculating the distance between the target height limiting device and the current vehicle according to the following formula:
the distance is the distance between the target height limiting device and the current vehicle, b is a base line of the binocular camera, f is the focal length of the binocular camera, and disparity is the parallax value of the target height limiting device on the target parallax map.
The invention also provides a binocular camera-based height limiting device detection system for implementing the method, the system comprising:
the model construction unit is used for constructing a target detection model and training the target detection model by using a target detection algorithm to obtain a deep learning target detection model;
the position detection unit is used for acquiring the position of a target height limiting device based on the deep learning target detection model and marking the target height limiting device on an original image of a binocular camera;
the disparity map cleaning unit is used for acquiring an original disparity map of the binocular camera and cleaning the original disparity map to obtain a target disparity map;
and the height and distance calculation unit is used for calculating the height of the target height limiting device and the distance between the target height limiting device and the current vehicle on the basis of the target parallax map.
The present invention also provides an intelligent terminal, including: the device comprises a data acquisition device, a processor and a memory;
the data acquisition device is used for acquiring data; the memory is to store one or more program instructions; the processor is configured to execute one or more program instructions to perform the method as described above.
The binocular camera-based height limiting device detection method provided by the invention comprises the following steps: constructing a target detection model, and training the target detection model by using a target detection algorithm to obtain a deep learning target detection model; acquiring the position of a target height limiting device based on the deep learning target detection model, and marking the target height limiting device on an original image of a binocular camera; acquiring an original disparity map of the binocular camera, and cleaning the original disparity map to obtain a target disparity map; and calculating the height of the target height limiting device and the distance between the target height limiting device and the current vehicle based on the target disparity map. According to the method, the accuracy of the model is improved by training the target detection model, so that the accuracy of the position of the target height limiting device obtained based on the model is improved; meanwhile, the accuracy of the disparity map is improved by cleaning the disparity map, so that the accuracy of the height and distance of the target height limiting device calculated based on the disparity map is improved. Therefore, the technical problem that the height and distance detection accuracy of the height limiting device in the prior art is poor is solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.
The structures, ratios, sizes, and the like shown in the present specification are only used for matching with the contents disclosed in the specification, so as to be understood and read by those skilled in the art, and are not used to limit the conditions that the present invention can be implemented, so that the present invention has no technical significance, and any structural modifications, changes in the ratio relationship, or adjustments of the sizes, without affecting the effects and the achievable by the present invention, should still fall within the range that the technical contents disclosed in the present invention can cover.
Fig. 1 is a flowchart of a binocular camera-based height limiting device detection method according to an embodiment of the present invention;
FIG. 2 is a network architecture diagram of an SSD;
fig. 3 is a schematic diagram of a result of a visualization process performed on a disparity map;
FIG. 4 is a schematic diagram of the original disparity map after coarse-grained cleaning;
FIG. 5 is a schematic diagram of the fine-grained cleaned parallax map;
fig. 6 is a block diagram of an embodiment of a binocular camera-based height limiter detection system according to the present invention.
Detailed Description
The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In a specific embodiment, as shown in fig. 1, the method for detecting a height limiting device based on a binocular camera provided by the invention comprises the following steps:
s1: and constructing a target detection model, and training the target detection model by using a target detection algorithm to obtain a deep learning target detection model.
Specifically, after a height limiting device data set is constructed and accurate labeling is completed, a proper deep learning target detection algorithm is selected for training a target detection model. In the process of model training, the hyper-parameters of the algorithm are continuously adjusted until the hyper-parameters of the algorithm are completely matched with the height limiting device data set and the target detection algorithm, so that a deep learning target detection model with high detection precision is trained.
In order to improve the accuracy of model training, before deep learning target detection model training, a target detection network suitable for the detection problem of the height limiting device is selected. The SSD network uses a design of dense anchor points with multi-scale and multi-aspect ratio and a feature pyramid, and has a high degree of adaptation to the geometric features of the height limiting device, so that the SSD network is selected as a target detection network for training, wherein the network structure of the SSD is as shown in fig. 2.
S2: and acquiring the position of a target height limiting device based on the deep learning target detection model, and marking the target height limiting device on an original image of a binocular camera. After a deep learning target detection model with high detection precision is trained, test set data in a data set of the height limiting device is input into the model to test the model, the position of the height limiting device can be accurately detected based on a test result of the target detection model, and the region of the height limiting device is wrapped in a rectangular frame mode, so that the target height limiting device is marked.
S3: and acquiring an original parallax image of the binocular camera, and cleaning the original parallax image to obtain a target parallax image.
In order to improve the cleaning effect, two steps of rough cleaning and fine cleaning can be carried out. That is to say, the cleaning of the disparity map to obtain the target disparity map specifically includes:
carrying out coarse-grained cleaning on the original disparity map by using mean filtering to obtain a disparity map after coarse cleaning;
and carrying out fine-grained cleaning on the parallax image after the coarse cleaning to obtain the target parallax image.
The coarse-grained cleaning of the original disparity map by using the mean filtering specifically comprises the following steps:
setting a parallax maximum threshold value and a parallax minimum threshold value;
calculating a non-zero average value of the whole image of the original disparity map;
calculating the average value and the non-zero average value of each line in the original disparity map according to the line, and calculating the difference value between the line average value and the non-zero average value;
if the result of the comparison calculation exceeds the maximum threshold value, setting the row to be 0;
if the calculation is below the min threshold, the row is considered to be more empty and the row is replaced with a full-map non-zero average.
The fine-grained cleaning of the parallax image after the coarse cleaning specifically comprises the following steps:
setting fine-grained filtering step number =15 and fine-grained filtering step length = 3;
repeatedly cleaning according to the steps;
and the cleaning times are the product of the filtering step number and the filtering step length, wherein the filtering step number is reduced by 1 and the filtering step number is increased by 1 when cleaning is finished for 1 time until the cleaning times are finished.
In principle, the original image of the binocular camera is used to calculate the parallax of the overlapped and effective area of the left camera and the right camera point by point, and a parallax map corresponding to the original image is obtained. The size of the disparity value in the disparity map represents the size of the distance of the object from the camera. However, the calculated disparity map has a large noise interference at a long distance, and the result of the visualization processing by taking out the disparity map at the corresponding position according to the position of the rectangular frame of the parcel height limiting device detected by the target detection model is shown in fig. 3.
As can be seen from the visualization result, the disparity distribution of the original disparity map is very uneven, so that the original disparity map needs to be cleaned. The cleaning of the disparity map is divided into coarse grain cleaning and fine grain cleaning, and the coarse grain cleaning is firstly performed on the original disparity map by using mean filtering, and the result is shown in fig. 4.
Based on the principle, the strategy of coarse grain cleaning is as follows:
setting a parallax maximum threshold value and a parallax minimum threshold value;
calculating a non-zero average value of the whole image;
calculating the average value and the non-zero average value of the line according to the line, and calculating the difference value between the average value and the non-zero average value of the line;
if the calculation exceeds the maximum threshold value, the behavior is considered to be abnormal, and the row is set to be 0;
if the calculation is below the min threshold, the row is considered to be more empty and the row is replaced with a full-map non-zero average.
As can be seen from the visualization result, the numerical distribution of the disparity map after coarse-grained cleaning is much more uniform than that of the initial disparity map, but still is not enough to support the algorithm to complete accurate calculation. Therefore, the disparity map needs to be further cleaned in a fine-grained manner, and the visualization result of the disparity map after being cleaned in the fine-grained manner is shown in fig. 5.
Based on the principle, the fine-grained cleaning mode is as follows:
setting fine-grained filtering step number =15 and fine-grained filtering step length = 3;
fine-grained cleaning each time, subtracting 1 from the number of filtering steps, adding 1 to the number of filtering steps, and totally cleaning the sub-parallax image of the filtering steps multiplied by the filtering step size;
each step of cleaning is strictly performed according to the coarse grain cleaning method.
The parallax images after fine-grained cleaning are uniform in numerical distribution, and the distance and the height of the height limiting device in the next step can be calculated.
S4: and calculating the height of the target height limiting device and the distance between the target height limiting device and the current vehicle based on the target disparity map.
Wherein the calculating the height of the target height limiting device based on the target disparity map specifically includes:
the height of the target height-limiting device is calculated according to the following formula:
wherein the content of the first and second substances,heightwhich represents the height of the target object,Vrepresenting the coordinates of the target phase element,V 0 representing the coordinates of the optical center of the camera,disparityrepresenting the magnitude of the disparity value.
Calculating the distance between the target height limiting device and the current vehicle based on the target disparity map, specifically comprising:
calculating a disparity value of the target height limiting device on the target disparity map based on the target disparity map;
calculating the distance between the target height limiting device and the current vehicle according to the following formula:
the distance is the distance between the target height limiting device and the current vehicle, b is a base line of the binocular camera, f is the focal length of the binocular camera, and disparity is the parallax value of the target height limiting device on the target parallax map.
In the above specific embodiment, the method improves the accuracy of the model by training the target detection model, thereby improving the accuracy of the position of the target height limiting device obtained based on the model; meanwhile, the accuracy of the disparity map is improved by cleaning the disparity map, so that the accuracy of the height and distance of the target height limiting device calculated based on the disparity map is improved. Therefore, the technical problem that the height and distance detection accuracy of the height limiting device in the prior art is poor is solved.
Further, the method uses a binocular camera to construct a height limiting device database, and uses a deep learning target detection method to accurately detect the height limiting device in front of the vehicle. And calculating parallax values point by point according to left and right views of the binocular camera and extracting the detected parallax map of the position corresponding to the height limiting device, wherein after coarse-grained and fine-grained cleaning, the numerical distribution of the parallax map is more uniform and reliable, and the distance and the height of the height limiting device can be measured according to the calculated parallax values. And then can provide effective information for the driver in advance, avoid taking place the ascending collision in direction of height, effectively reduce the incidence of limit for height accident.
In addition to the above method, the present invention also provides a binocular camera based height limiting device detection system for implementing the method as described above, and in one embodiment, as shown in fig. 6, the system includes:
the model construction unit 100 is configured to construct a target detection model, and train the target detection model by using a target detection algorithm to obtain a deep learning target detection model;
and the position detection unit 200 is used for acquiring the position of the target height limiting device based on the deep learning target detection model and marking the target height limiting device on an original image of the binocular camera. Specifically, the position detection unit 200 is used to test the model using the test set data in the height-limiting device dataset.
The disparity map cleaning unit 300 is configured to acquire an original disparity map of the binocular camera, and clean the original disparity map to obtain a target disparity map.
The disparity map cleaning unit 300 is specifically configured to:
carrying out coarse-grained cleaning on the original disparity map by using mean filtering to obtain a disparity map after coarse cleaning;
and carrying out fine-grained cleaning on the parallax image after the coarse cleaning to obtain the target parallax image.
A height and distance calculation unit 400 for calculating the height of the target height limiting device and the distance between the target height limiting device and the current vehicle based on the target disparity map.
The height and distance calculation unit 400 is specifically configured to calculate the height of the target height limiting device according to the following formula:
wherein the content of the first and second substances,heightwhich represents the height of the target object,Vrepresenting the coordinates of the target phase element,V 0 representing the coordinates of the optical center of the camera,disparityrepresenting the magnitude of the disparity value.
The height and distance calculating unit 400 is specifically configured to calculate, based on the target disparity map, disparity values of the target height limiting device on the target disparity map;
calculating the distance between the target height limiting device and the current vehicle according to the following formula:
the distance is the distance between the target height limiting device and the current vehicle, b is a base line of the binocular camera, f is the focal length of the binocular camera, and disparity is the parallax value of the target height limiting device on the target parallax map.
In the above specific embodiment, the system improves the accuracy of the model by training the target detection model, thereby improving the accuracy of the position of the target height limiting device obtained based on the model; meanwhile, the accuracy of the disparity map is improved by cleaning the disparity map, so that the accuracy of the height and distance of the target height limiting device calculated based on the disparity map is improved. Therefore, the technical problem that the height and distance detection accuracy of the height limiting device in the prior art is poor is solved.
The present invention also provides an intelligent terminal, including: the device comprises a data acquisition device, a processor and a memory;
the data acquisition device is used for acquiring data; the memory is to store one or more program instructions; the processor is configured to execute one or more program instructions to perform the method as described above.
In correspondence with the above embodiments, embodiments of the present invention also provide a computer storage medium containing one or more program instructions therein. Wherein the one or more program instructions are for executing the method as described above by a binocular camera depth calibration system.
In an embodiment of the invention, the processor may be an integrated circuit chip having signal processing capability. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The processor reads the information in the storage medium and completes the steps of the method in combination with the hardware.
The storage medium may be a memory, for example, which may be volatile memory or nonvolatile memory, or which may include both volatile and nonvolatile memory.
The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory.
The volatile Memory may be a Random Access Memory (RAM) which serves as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), SLDRAM (SLDRAM), and Direct Rambus RAM (DRRAM).
The storage media described in connection with the embodiments of the invention are intended to comprise, without being limited to, these and any other suitable types of memory.
Those skilled in the art will appreciate that the functionality described in the present invention may be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above embodiments are only for illustrating the embodiments of the present invention and are not to be construed as limiting the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made on the basis of the embodiments of the present invention shall be included in the scope of the present invention.
Claims (10)
1. A binocular camera-based height limiting device detection method is characterized by comprising the following steps:
constructing a target detection model, and training the target detection model by using a target detection algorithm to obtain a deep learning target detection model;
acquiring the position of a target height limiting device based on the deep learning target detection model, and marking the target height limiting device on an original image of a binocular camera;
acquiring an original disparity map of the binocular camera, and cleaning the original disparity map to obtain a target disparity map;
and calculating the height of the target height limiting device and the distance between the target height limiting device and the current vehicle based on the target disparity map.
2. The method for detecting a height limiting device according to claim 1, wherein the constructing a target detection model and training the target detection model by using a target detection algorithm to obtain a deep learning target detection model specifically comprises:
and after constructing a height limiting device data set and finishing accurate marking, selecting the SSD as a deep learning target detection algorithm to train a target detection model.
3. The method for detecting a height-limiting device according to claim 2, wherein the obtaining a position of a target height-limiting device based on the deep learning target detection model specifically includes:
and inputting the test set data in the height limiting device data set into the model to test the model.
4. The method for detecting a height limiting device according to claim 3, wherein the step of cleaning the disparity map to obtain a target disparity map specifically comprises:
carrying out coarse-grained cleaning on the original disparity map by using mean filtering to obtain a disparity map after coarse cleaning;
and carrying out fine-grained cleaning on the parallax image after the coarse cleaning to obtain the target parallax image.
5. The height-limiting device detection method according to claim 4, wherein the coarse-grained cleaning of the original disparity map by using the mean filtering specifically comprises:
setting a parallax maximum threshold value and a parallax minimum threshold value;
calculating a non-zero average value of the whole image of the original disparity map;
calculating the average value and the non-zero average value of each line in the original disparity map according to the line, and calculating the difference value between the line average value and the non-zero average value;
if the result of the comparison calculation exceeds a maximum threshold value, the row is considered to cover the background, and the row is set to be 0;
if the row average is below the minimum threshold, the row is deemed to be more empty and the row is replaced with the full-graph non-zero average.
6. The method for detecting the height limiting device according to claim 5, wherein the fine-grained cleaning of the disparity map after the rough cleaning specifically comprises:
setting fine-grained filtering step number =15 and fine-grained filtering step length =3, and repeatedly cleaning;
and the cleaning times are the product of the filtering step number and the filtering step length, wherein the filtering step number is reduced by 1 and the filtering step number is increased by 1 when cleaning is finished for 1 time until the cleaning times are finished.
7. The method for detecting a height-limiting device according to claim 6, wherein the calculating the height of the target height-limiting device based on the target disparity map specifically includes:
the height of the target height-limiting device is calculated according to the following formula:
wherein height represents the height of the target, V represents the coordinates of the target pixel, and V represents the height of the target pixel 0 Representative camera lightThe center coordinate, disparity, represents the magnitude of the disparity value.
8. The method for detecting a height limiting device according to claim 7, wherein calculating the distance between the target height limiting device and the current vehicle based on the target disparity map specifically comprises:
calculating a disparity value of the target height limiting device on the target disparity map based on the target disparity map;
calculating the distance between the target height limiting device and the current vehicle according to the following formula:
the distance is the distance between the target height limiting device and the current vehicle, b is a base line of the binocular camera, f is the focal length of the binocular camera, and disparity is the parallax value of the target height limiting device on the target parallax map.
9. A binocular camera based height limiting device detection system for implementing the method according to any one of claims 1 to 8, the system comprising:
the model construction unit is used for constructing a target detection model and training the target detection model by using a target detection algorithm to obtain a deep learning target detection model;
the position detection unit is used for acquiring the position of a target height limiting device based on the deep learning target detection model and marking the target height limiting device on an original image of a binocular camera;
the disparity map cleaning unit is used for acquiring an original disparity map of the binocular camera and cleaning the original disparity map to obtain a target disparity map;
and the height and distance calculation unit is used for calculating the height of the target height limiting device and the distance between the target height limiting device and the current vehicle on the basis of the target parallax map.
10. An intelligent terminal, characterized in that, intelligent terminal includes: the device comprises a data acquisition device, a processor and a memory;
the data acquisition device is used for acquiring data; the memory is to store one or more program instructions; the processor, configured to execute one or more program instructions to perform the method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111324101.0A CN113781543B (en) | 2021-11-10 | 2021-11-10 | Binocular camera-based height limiting device detection method and system and intelligent terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111324101.0A CN113781543B (en) | 2021-11-10 | 2021-11-10 | Binocular camera-based height limiting device detection method and system and intelligent terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113781543A true CN113781543A (en) | 2021-12-10 |
| CN113781543B CN113781543B (en) | 2022-03-18 |
Family
ID=78873667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111324101.0A Active CN113781543B (en) | 2021-11-10 | 2021-11-10 | Binocular camera-based height limiting device detection method and system and intelligent terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113781543B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114463388A (en) * | 2022-04-13 | 2022-05-10 | 北京中科慧眼科技有限公司 | Binocular camera-based height limiting device detection method and system and intelligent terminal |
| CN117553695A (en) * | 2024-01-11 | 2024-02-13 | 摩斯智联科技有限公司 | Method, apparatus and computer storage medium for calculating vehicle height based on aspect ratio suppression noise |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103679707A (en) * | 2013-11-26 | 2014-03-26 | 西安交通大学 | Binocular camera disparity map based road obstacle detection system and method |
| CN104700414A (en) * | 2015-03-23 | 2015-06-10 | 华中科技大学 | Rapid distance-measuring method for pedestrian on road ahead on the basis of on-board binocular camera |
| EP3637307A1 (en) * | 2018-10-11 | 2020-04-15 | Tindei Network Technology (Shanghai) Co., Ltd. | Binocular pedestrian detection system having dual-stream deep learning neural network and the methods of using the same |
| CN111260720A (en) * | 2020-01-13 | 2020-06-09 | 浙江大学 | Target height measuring system based on deep learning method |
| CN111314688A (en) * | 2020-03-16 | 2020-06-19 | 北京迈格威科技有限公司 | Disparity map hole filling method and device and electronic system |
| CN111339956A (en) * | 2020-02-28 | 2020-06-26 | 北京中科慧眼科技有限公司 | Height-limiting early warning method, device and system based on binocular camera and storage medium |
| CN111862077A (en) * | 2020-07-30 | 2020-10-30 | 浙江大华技术股份有限公司 | Disparity map processing method and device, storage medium and electronic device |
| CN113240632A (en) * | 2021-04-22 | 2021-08-10 | 北京中科慧眼科技有限公司 | Road surface detection method and system based on semantic segmentation network and intelligent terminal |
| CN113378805A (en) * | 2021-08-13 | 2021-09-10 | 北京中科慧眼科技有限公司 | Height limiting device detection method and system based on deep learning and intelligent terminal |
-
2021
- 2021-11-10 CN CN202111324101.0A patent/CN113781543B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103679707A (en) * | 2013-11-26 | 2014-03-26 | 西安交通大学 | Binocular camera disparity map based road obstacle detection system and method |
| CN104700414A (en) * | 2015-03-23 | 2015-06-10 | 华中科技大学 | Rapid distance-measuring method for pedestrian on road ahead on the basis of on-board binocular camera |
| EP3637307A1 (en) * | 2018-10-11 | 2020-04-15 | Tindei Network Technology (Shanghai) Co., Ltd. | Binocular pedestrian detection system having dual-stream deep learning neural network and the methods of using the same |
| CN111260720A (en) * | 2020-01-13 | 2020-06-09 | 浙江大学 | Target height measuring system based on deep learning method |
| CN111339956A (en) * | 2020-02-28 | 2020-06-26 | 北京中科慧眼科技有限公司 | Height-limiting early warning method, device and system based on binocular camera and storage medium |
| CN111314688A (en) * | 2020-03-16 | 2020-06-19 | 北京迈格威科技有限公司 | Disparity map hole filling method and device and electronic system |
| CN111862077A (en) * | 2020-07-30 | 2020-10-30 | 浙江大华技术股份有限公司 | Disparity map processing method and device, storage medium and electronic device |
| CN113240632A (en) * | 2021-04-22 | 2021-08-10 | 北京中科慧眼科技有限公司 | Road surface detection method and system based on semantic segmentation network and intelligent terminal |
| CN113378805A (en) * | 2021-08-13 | 2021-09-10 | 北京中科慧眼科技有限公司 | Height limiting device detection method and system based on deep learning and intelligent terminal |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114463388A (en) * | 2022-04-13 | 2022-05-10 | 北京中科慧眼科技有限公司 | Binocular camera-based height limiting device detection method and system and intelligent terminal |
| CN117553695A (en) * | 2024-01-11 | 2024-02-13 | 摩斯智联科技有限公司 | Method, apparatus and computer storage medium for calculating vehicle height based on aspect ratio suppression noise |
| CN117553695B (en) * | 2024-01-11 | 2024-05-03 | 摩斯智联科技有限公司 | Method and device for calculating vehicle height and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113781543B (en) | 2022-03-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113781543B (en) | Binocular camera-based height limiting device detection method and system and intelligent terminal | |
| CN105206109A (en) | Infrared CCD based foggy day identifying early-warning system and method for vehicle | |
| CN114495043B (en) | Method and system for detecting up-and-down slope road conditions based on binocular vision system and intelligent terminal | |
| CN110991232B (en) | Vehicle position correction method and system, storage medium and terminal | |
| CN113792707A (en) | Terrain environment detection method and system based on binocular stereo camera and intelligent terminal | |
| CN113256701B (en) | Distance acquisition method, device, equipment and readable storage medium | |
| CN112465831B (en) | Bend scene sensing method, system and device based on binocular stereo camera | |
| CN114509045A (en) | Wheel area elevation detection method and system | |
| CN113173502B (en) | Anticollision method and system based on laser vision fusion and deep learning | |
| CN113140002B (en) | Road condition detection method and system based on binocular stereo camera and intelligent terminal | |
| CN115100621A (en) | Ground scene detection method and system based on deep learning network | |
| CN111191538B (en) | Obstacle tracking method, device and system based on binocular camera and storage medium | |
| EP3623994A2 (en) | Method, device, apparatus and storage medium for detecting a height of an obstacle | |
| CN114463388A (en) | Binocular camera-based height limiting device detection method and system and intelligent terminal | |
| CN113155143A (en) | Method, device and vehicle for evaluating a map for automatic driving | |
| CN113674275B (en) | Dense disparity map-based road surface unevenness detection method and system and intelligent terminal | |
| CN113689565B (en) | Road flatness grade detection method and system based on binocular stereo vision and intelligent terminal | |
| CN114049307A (en) | Road surface height detection method and system based on binocular stereoscopic vision and intelligent terminal | |
| DE102011111856B4 (en) | Method and device for detecting at least one lane in a vehicle environment | |
| CN113706622B (en) | Road surface fitting method and system based on binocular stereo vision and intelligent terminal | |
| CN112061118A (en) | Vehicle collision prediction method and device | |
| CN115205809B (en) | Method and system for detecting roughness of road surface | |
| CN116118749B (en) | Road pavement identification method and device, computer equipment and storage medium | |
| CN113657205B (en) | Obstacle detection method and system based on motion grid and intelligent terminal | |
| CN117391931A (en) | Self-detection method and device for external parameter change of all-around fish-eye camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |