CN107958222B

CN107958222B - Pavement detection method and device and terminal

Info

Publication number: CN107958222B
Application number: CN201711315782.8A
Authority: CN
Inventors: 高伟杰
Original assignee: Hisense Co Ltd
Current assignee: Hisense Co Ltd
Priority date: 2017-12-12
Filing date: 2017-12-12
Publication date: 2020-10-16
Anticipated expiration: 2037-12-12
Also published as: CN107958222A

Abstract

The invention discloses a road surface detection method, a road surface detection device and a road surface detection terminal, and belongs to the technical field of automobile auxiliary driving. The method comprises the following steps: when a V disparity map corresponding to the road condition image is obtained, carrying out binarization processing on the V disparity map to obtain a target V disparity map; dividing the target V disparity map according to the first boundary line to obtain two subareas, detecting oblique lines in the two subareas, and correcting the position of the first boundary line based on the detected oblique lines to obtain the target boundary line if the oblique lines detected in the two subareas do not meet preset conditions; the road surface in the target V parallax map is determined based on oblique lines detected in two partitions into which the target V parallax map is divided by the target boundary line. In the embodiment of the present invention, the target boundary line is obtained by correcting the position of the first boundary line, and therefore, the target V parallax map is divided by the target boundary line to obtain two divisional areas, and the road surface can be determined more accurately and stably from the oblique lines detected in the two divisional areas.

Description

Pavement detection method and device and terminal

Technical Field

The invention relates to the technical field of automobile auxiliary driving, in particular to a road surface detection method, a road surface detection device and a road surface detection terminal.

Background

With the continuous development of intellectualization, the auxiliary driving technology of vehicles has become a hot technical spot for competitive pursuit of various manufacturers. In the existing advanced driving assistance system, the road surface in front of the vehicle can be detected by processing road condition images acquired by the binocular camera, so that a data basis is provided for safe driving of the vehicle.

In the related art, after the road condition image is acquired through the binocular camera, the road condition image can be processed to obtain a corresponding parallax image. And then, calculating a corresponding V disparity map according to the disparity image. The V disparity map is divided into two fixed subareas according to a preset dividing line, oblique lines are respectively detected in the two fixed subareas, and the detected oblique lines can be used for representing the road surface.

When the road surface is detected by the method, the V disparity map is divided by the preset dividing line to obtain two fixed partitions, so that when the position of the preset dividing line is unreasonable, two accurate fixed partitions for detecting oblique lines representing an uneven road surface cannot be obtained when the V disparity map is divided by the preset dividing line, that is, the two fixed partitions divided by the preset dividing line have low accuracy, on the basis, the oblique lines are detected in the two fixed partitions, and the detected oblique lines are used as the road surface, so that a large detection error is generated.

Disclosure of Invention

In order to solve the problem that in the related art, under the condition that the position of the preset boundary is unreasonable to select, the accuracy of two fixed partitions obtained by dividing the preset boundary is low, and on the basis, a large detection error exists when oblique lines detected in the two fixed partitions are used as a road surface, the embodiment of the invention provides a road surface detection method, a road surface detection device and a road surface detection terminal. The technical scheme is as follows:

in a first aspect, a method for detecting a road surface is provided, the method comprising:

when a V disparity map corresponding to a road condition image is obtained, carrying out binarization processing on the V disparity map to obtain a target V disparity map;

dividing the target V disparity map according to a first boundary line to obtain two subareas, detecting oblique lines in the two subareas, and correcting the position of the first boundary line based on the detected oblique lines to obtain a target boundary line if the oblique lines detected in the two subareas do not meet a preset condition;

and determining the road surface in the target V parallax map based on oblique lines detected in two subareas obtained after the target boundary line divides the target V parallax map.

Optionally, the detecting the oblique lines in the two partitions, and if the oblique lines detected in the two partitions do not satisfy a preset condition, correcting the position of the first boundary line based on the detected oblique lines to obtain a target boundary line includes:

respectively detecting oblique lines in the two subareas to obtain a first oblique line and a second oblique line;

judging whether a preset condition is met between the first oblique line and the second oblique line;

and when the first oblique line and the second oblique line do not meet a preset condition, correcting the position of the first boundary line based on the first oblique line and the second oblique line, dividing the target V disparity map based on the first boundary line after the position is corrected to obtain two partitions, respectively detecting oblique lines in the two partitions obtained after the division, and determining the first boundary line after the position is corrected for the last time as the target boundary line until the detected first oblique line and the second oblique line meet the preset condition.

Optionally, the determining whether a preset condition is satisfied between the first oblique line and the second oblique line includes:

determining an intersection point of the first oblique line and the first boundary line to obtain a first intersection point, and determining an intersection point of the second oblique line and the first boundary line to obtain a second intersection point;

when the distance between the first intersection point and the second intersection point is greater than a preset distance, determining that the preset condition is not met between the first oblique line and the second oblique line;

and when the distance between the first intersection point and the second intersection point is smaller than or equal to the preset distance, determining that the first oblique line and the second oblique line meet the preset condition.

Optionally, the correcting the position of the first boundary line based on the first oblique line and the second oblique line includes:

determining an intersection point of the first oblique line and the second oblique line to obtain a third intersection point;

and determining a straight line which passes through the third intersection point and is parallel to the longitudinal axis of the target V disparity map as a first boundary line after the position is corrected.

Optionally, the determining the road surface in the target V disparity map based on oblique lines detected in two partitions obtained by dividing the target V disparity map by the target boundary line includes:

determining the slopes of two oblique lines detected in two partitions obtained after the target boundary line divides the target V disparity map to obtain a first slope and a second slope;

and when the first slope and the second slope are different, determining a broken line obtained after one of the two oblique lines is intersected with the other oblique line of the two oblique lines through an extension line as the road surface in the target V parallax image.

Optionally, after determining slopes of two oblique lines detected in two partitions obtained after the target boundary line divides the target V disparity map, and obtaining a first slope and a second slope, the method further includes:

connecting two intersection points of the two oblique lines and the target boundary line when the first slope and the second slope are the same;

and determining the connected broken lines as the road surface in the target V parallax image.

In a second aspect, there is provided a road surface detecting device, the device including:

the processing module is used for carrying out binarization processing on the V disparity map to obtain a target V disparity map when the V disparity map corresponding to the road condition image is obtained;

the first determining module is used for dividing the target V disparity map according to a first boundary line to obtain two subareas, detecting oblique lines in the two subareas, and correcting the position of the first boundary line based on the detected oblique lines to obtain a target boundary line if the oblique lines detected in the two subareas do not meet a preset condition;

and the second determining module is used for determining the road surface in the target V parallax map based on oblique lines detected in two subareas obtained after the target boundary line divides the target V parallax map.

Optionally, the first determining module includes:

the detection submodule is used for respectively detecting oblique lines in the two partitions to obtain a first oblique line and a second oblique line;

the judgment submodule is used for judging whether the first oblique line and the second oblique line meet preset conditions or not;

and the correction submodule is used for correcting the position of the first boundary line based on the first oblique line and the second oblique line when a preset condition is not met between the first oblique line and the second oblique line, dividing the target V disparity map based on the first boundary line after the position is corrected to obtain two partitions, respectively detecting oblique lines in the two partitions obtained after the division, and determining the first boundary line after the position is corrected for the last time as the target boundary line until the detected position between the first oblique line and the second oblique line meets the preset condition.

Optionally, the determining sub-module is specifically configured to:

Optionally, the modification submodule is specifically configured to:

Optionally, the second determining module includes:

a first determining submodule, configured to determine slopes of two oblique lines detected in two partitions obtained after the target V disparity map is divided by the target boundary line, so as to obtain a first slope and a second slope;

and the second determining submodule is used for determining a broken line obtained after one of the two oblique lines is intersected with the other of the two oblique lines through an extension line as the road surface in the target V parallax image when the first slope and the second slope are different.

Optionally, the second determining module further includes:

a connection submodule for connecting two intersections of the two oblique lines and the target boundary line when the first slope and the second slope are the same;

and the third determining submodule is used for determining the connected broken line as the road surface in the target V disparity map.

In a third aspect, a road surface detection terminal is provided, the terminal comprising:

a processor;

the camera assembly is used for acquiring road condition images and sending the road condition images to the processor for processing;

a memory for storing processor-executable instructions;

wherein the processor is configured to perform the steps of any one of the methods of the first aspect.

In a fourth aspect, a computer-readable storage medium is provided, having instructions stored thereon, which when executed by a processor, implement the steps of any of the methods of the first aspect described above.

In a fifth aspect, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the road surface detection method of the first aspect described above.

The technical scheme provided by the embodiment of the invention has the following beneficial effects: when the V disparity map corresponding to the road condition image is acquired, the V disparity map may be subjected to binarization processing to obtain a target V disparity map, then the target V disparity map is divided into two partitions according to a first boundary line to obtain two partitions, oblique lines are detected in the two partitions, if the oblique lines detected in the two partitions do not satisfy a preset condition, the position of the first boundary line is corrected based on the detected oblique lines to obtain a target boundary line, and the road surface in the target V disparity map is determined based on the oblique lines detected in the two partitions obtained by dividing the target V disparity map by the target boundary line. That is, in the embodiment of the present invention, when the diagonal line detected in the two partitions into which the first boundary line is divided does not satisfy the preset condition, the target boundary line may be obtained by correcting the position of the first boundary line. Therefore, the target V disparity map is divided by the target boundary line to obtain two more accurate subareas, so that the road surface determined according to the oblique lines detected in the two subareas is more accurate, and the accuracy of road surface detection is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1A is a schematic diagram illustrating a principle of calculating a parallax image according to an embodiment of the present invention;

fig. 1B is a schematic diagram illustrating a principle of calculating a V disparity map from disparity images according to an embodiment of the present invention;

fig. 1C is a system architecture diagram of a road surface detection method according to an embodiment of the present invention;

fig. 2A is a flowchart of a road surface detection method according to an embodiment of the present invention;

fig. 2B is a flowchart of determining a target boundary of a processed V-disparity map according to an embodiment of the present invention;

fig. 2C is a schematic diagram illustrating a method for determining whether a continuous condition is satisfied between a first oblique line and a second oblique line according to an embodiment of the present invention;

fig. 2D is a schematic diagram illustrating a road surface in a V-disparity map after determining processing based on two obtained oblique lines according to an embodiment of the present invention;

fig. 2E is a schematic diagram of another road surface in the V-disparity map after the processing is determined based on the two obtained oblique lines according to the embodiment of the present invention;

FIG. 2F is a schematic diagram illustrating a comparison between the detection effect of detecting an uneven road surface by using the related art and the detection effect of detecting an uneven road surface by using the method provided by the embodiment of the invention;

fig. 3A is a schematic structural diagram of a road surface detection device according to an embodiment of the present invention;

fig. 3B is a schematic structural diagram of a first determining module according to an embodiment of the present invention;

fig. 3C is a schematic structural diagram of a second determining module according to an embodiment of the present invention;

fig. 3D is a schematic structural diagram of another second determining module according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a terminal according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

For convenience of understanding, before explaining the embodiments of the present invention in detail, terms related to the embodiments of the present invention will be explained.

Parallax images: the binocular image acquisition method is obtained by calculating a left image and a right image which are shot by the binocular camera at the same time. One of the left and right images is used as a reference image, the other image is used as a comparison image, and the left and right images have the same size. The reference image and the comparison image are placed in the same coordinate system, pixel points in the comparison image are matched with a row of pixel points with the same Y coordinate in the reference image, so that pixel points matched with the pixel points in the comparison image are determined from the reference image, the difference of the abscissa between every two mutually matched pixel points in the comparison image and the reference image is calculated, and the difference of the abscissas is the parallax value between the two pixel points. And taking the parallax value as a pixel value corresponding to the pixel point in the reference image, thereby obtaining the parallax image with the same size as the reference image.

Fig. 1A is a schematic diagram illustrating a principle of calculating a parallax image according to an embodiment of the present invention. In fig. 1A, the left image is a reference image, and the right image is a comparison image. For convenience of illustration, each cell in fig. 1A may be regarded as a pixel. For the pixel point a in the comparison image, when finding the matching pixel point of the pixel point a in the reference image, first, the pixel point a may be used as the central pixel point to form a 9 × 9 pixel matrix a, as shown by the dashed line frame in the right diagram in fig. 1A. Then, a row of pixels having the same Y coordinate as the center pixel can be determined in the reference image, as shown by the solid line box in the left image of fig. 1A. When the pixel point a in the comparison image is subjected to matching calculation with the pixel point B in the reference image, a pixel matrix B of the same size 9 × 9 can be formed with the pixel point B as a center pixel point, as shown by a dashed box in the left diagram in fig. 1A. And then, calculating pixel differences between each pixel point in the pixel matrix A and the pixel point at the corresponding position in the pixel matrix B, and adding the pixel differences to obtain a pixel difference sum. For other pixel points in the reference image, which have the same Y coordinate with the pixel point A, matching calculation can be performed with the pixel point A through the method, and finally, a plurality of pixel difference sums are obtained correspondingly. And determining the pixel point corresponding to the pixel difference sum with the minimum pixel difference sum of the pixel differences as the matching point of the pixel point A. Assuming that the matching point of the pixel point a in the reference image is the pixel point B, the difference between the abscissa of the pixel point a and the abscissa of the pixel point B can be used as the disparity value between the two pixel points, and the disparity value can be used as the pixel value of the pixel point B in the disparity image with the same size as the reference image.

V disparity map: the V disparity map is calculated from the disparity image. As can be seen from the above description of the parallax image, in the parallax image, the pixel value of each pixel point is the corresponding parallax value. The vertical coordinate of the parallax image is kept unchanged, the horizontal coordinate of the parallax image is changed into a parallax value, and the pixel value of each point (x, y) in the V parallax image is the total number of the pixel points with the parallax value of x in the pixel points with the vertical coordinate of y in the parallax image.

Fig. 1B is a schematic diagram illustrating a method for calculating a V disparity map from a disparity image according to an embodiment of the present invention. As can be seen from the above description of the parallax image, the abscissa and the ordinate of the parallax image are the same as the abscissa and the ordinate of the reference image, and the pixel value of each pixel is the parallax value corresponding to the pixel. The right image of fig. 1B is a V disparity map, in which the ordinate corresponds to the ordinate of the disparity image, and the abscissa is the disparity value. For convenience of description, each cell in fig. 1B may be regarded as a pixel, and the value in the cell is the pixel value of the pixel. If a line of pixel points in the solid line frame in the left image of fig. 1B, the ordinate of the line of pixel points is 7, the pixel value of the line of pixel points is counted, 5 pixel points with pixel values of 4 can be obtained, 8 pixel points with pixel values of 6 can be obtained, the remaining pixel points with pixel values of 3 are obtained, the number of the remaining pixel points is 7, and at this time, the pixel points in the line in the corresponding V parallax image can be generated according to the counting result. Since the pixel value of the parallax image is actually the parallax value, the number of the pixel points having a pixel value of 4 is 5, and actually the number of the pixel points having a parallax value of 4 is 5, and at this time, the pixel value of the pixel point at the position where the abscissa is 4 and the ordinate is 7 in the V parallax image is 5. Accordingly, the pixel value of the pixel point at the position with the abscissa of 6 and the ordinate of 7 is 8, and the pixel value of the pixel point at the position with the abscissa of 3 and the ordinate of 7 is 7. Because the pixels with the ordinate of 7 in the parallax image do not have the remaining pixels with parallax values, the pixel values of the remaining pixels with the ordinate of 7 in the V parallax image are all 0.

For the pixel points of each line in the parallax image, statistics can be performed by the method, and the pixel difference of the pixel points of the corresponding line in the V parallax image is obtained according to the result after the statistics, so that the V parallax image is finally obtained through calculation.

Next, an application scenario related to the embodiment of the present invention will be described.

In the current advanced driving assistance system, road condition images acquired by a radar, a sensor or a camera can be processed through image processing and computer vision technologies, pedestrians and obstacles in front are predicted according to the road condition images, and a driver is warned or the vehicle is controlled to brake emergently under the condition that potential danger exists. The accurate detection of the front obstacle is the key for effective early warning, and the accurate extraction of the road surface area from the road condition image is a necessary step for detecting the front obstacle.

At present, with the development of camera technology and computer vision technology, a method for acquiring road condition images by using a binocular camera and detecting obstacles by processing the images through the computer vision technology obtains a better detection effect. The road surface detection method provided by the embodiment of the invention can be applied to detecting the road surface in the process of detecting the obstacles by adopting the road condition images shot by the binocular camera.

Next, a system architecture according to an embodiment of the present invention will be described.

Fig. 1C is a system architecture diagram of a road surface detection method according to an embodiment of the present invention. As shown in fig. 1C, the system includes an automobile 101, a binocular camera 102, and a terminal 103. The binocular camera 102 is mounted on the automobile 101, and the binocular camera 102 can communicate with the terminal 103.

The binocular camera 102 may be mounted in front of the automobile 101 and located on the vertical axis of the automobile 101. As indicated by the arrow in fig. 1C. After the binocular camera 102 is mounted on the automobile 101, the binocular camera 102 may be calibrated. During the driving of the automobile 101, the binocular camera 102 may acquire road condition images.

After acquiring the road condition image, the binocular camera 102 may send the road condition image to the terminal 103, and the terminal 103 may process the road condition image to obtain a parallax image, and then calculate a V parallax image according to the parallax image, and detect a road surface from the V parallax image by a road surface detection method shown in fig. 2A below.

It should be noted that, in the embodiment of the present invention, the terminal 103 may be an in-vehicle terminal installed on the automobile 101, in which case, the binocular camera 102 may communicate with the terminal 103 through bluetooth, a wireless network, or a wired network.

Alternatively, the terminal 103 and the binocular camera 102 may be integrated devices, that is, the terminal 103 may be a terminal integrated with a binocular camera and an image processing function chip.

Alternatively, the terminal 103 may be a terminal that communicates with a plurality of binocular cameras mounted on the automobile at the same time. That is, the terminal 103 may be a terminal specially configured to receive road condition images returned by binocular cameras of multiple automobiles and analyze the road condition images.

After the application scenario and the system architecture of the embodiment of the present invention are introduced, the road surface detection method provided by the embodiment of the present invention will be explained in detail with reference to the drawings.

Fig. 2A is a flowchart of a road surface detection method according to an embodiment of the present invention, where the method may be applied to a terminal in the system architecture shown in fig. 1C. Referring to fig. 2A, the method includes the steps of:

step 201: and when the V disparity map corresponding to the road condition image is acquired, carrying out binarization processing on the V disparity map to obtain a target V disparity map.

Based on the introduction of the system architecture, the binocular camera carried on the automobile can acquire road condition images in real time. Wherein, binocular camera includes left camera and right camera, can gather two road conditions images through left camera and right camera. Then, the binocular camera can send the acquired road condition image to the terminal, the terminal can use the road condition image acquired by the left camera as a reference image, use the image acquired by the right camera as a comparison image, and calculate and obtain the parallax image according to the reference image and the comparison image according to the method for calculating the parallax image mentioned in the introduction of the noun. After the parallax image is obtained through calculation, the terminal may calculate the corresponding V parallax map according to the parallax image by the method for calculating the V parallax map.

In the subsequent steps, a straight line needs to be detected in the V disparity map, and in a commonly used method for detecting a straight line, an input image is often required to be a binarized image, so that after the terminal acquires the V disparity map corresponding to the road condition image, the V disparity map can be subjected to binarization processing to obtain a target V disparity map.

Specifically, for each pixel point in the V disparity map, if the pixel value of the pixel point is greater than the preset pixel value, the terminal may set the pixel value of the pixel point to 255, and if the pixel value of the pixel point is less than or equal to the preset pixel value, the terminal may set the pixel value of the pixel point to 0, thereby implementing binarization processing of the V disparity map. The preset pixel value may be a numerical value preset based on an empirical value, and the preset pixel value may ensure that as many pixel points as possible in the V disparity map are retained.

For example, the preset pixel value may be 4, at this time, for the pixel points whose pixel values are greater than 4 in the V disparity map, the terminal may set the pixel values of the pixel points to 255, and for the pixel points whose pixel values are less than or equal to 4 in the V disparity map, the terminal may set the pixel values of the pixel points to 0.

It should be noted that the above is only one possible example of the preset pixel value, and in practical applications, the preset pixel value may also be other values, for example, 5, 8, or 10, and the like.

Step 202: dividing the target V parallax map according to a first boundary line to obtain two partitions, detecting a slant line in the two partitions, and correcting the position of the first boundary line based on the detected slant line to obtain a target boundary line if the slant line detected in the two partitions does not satisfy a preset condition, wherein the target boundary line is a straight line parallel to the vertical axis of the target V parallax map.

After the terminal binarizes the V disparity map to obtain the target V disparity map, the terminal may obtain the target boundary line by correcting the position of the first boundary line, so that two oblique lines detected in two target sections into which the target boundary line is divided are taken as oblique lines for determining the road surface.

Specifically, as shown in fig. 2B, this step can be implemented by the following steps:

step 2021: the terminal determines a preset demarcation point according to the designated distance and initializes the first demarcation line according to the preset demarcation point.

The specified distance is used for indicating that the road surface which is in front of the automobile and is within the specified distance value range from the automobile is a plane. Specifically, the terminal may calculate, according to the specified distance, a disparity value corresponding to a point at a specified distance from the vehicle in the disparity image, and then determine a point on the abscissa axis corresponding to the disparity value in the target V disparity map as a preset dividing point, and a straight line passing through the preset dividing point and parallel to the ordinate axis of the target V disparity map may be determined as the first dividing line.

For example, assuming that the specified distance is h meters, that is, the road surface within a range h meters away from the front of the automobile can be considered to be a plane, according to the specified distance h, the corresponding parallax value m in the parallax image of the point h meters away from the front of the automobile can be calculated, since the abscissa axis of the target V parallax map is the parallax value, at this time, the point with the coordinate (m, 0) on the abscissa axis of the target V parallax map can be determined as the preset dividing point, and the straight line passing through the preset dividing point and parallel to the ordinate axis of the target V parallax map is also the first dividing line.

Step 2022: dividing two subareas obtained by the target V parallax diagram according to the first dividing line, respectively detecting oblique lines in the two subareas to obtain a first oblique line and a second oblique line, and judging whether the first oblique line and the second oblique line meet preset conditions or not.

Since the first boundary line is a straight line parallel to the ordinate axis of the target V disparity map, the target V disparity map can be divided into two left and right partitions by the first boundary line. The terminal may detect the slope in the two partitions respectively to obtain a first slope and a second slope.

The terminal can respectively perform straight line fitting on the pixel points in the two partitions to obtain a first oblique line and a second oblique line through detection. Specifically, the terminal may detect the oblique lines in the two partitions by hough transform, or may detect the oblique lines in the two partitions by a straight line fitting method such as least square method.

It should be noted that, when straight line fitting is performed in the two partitions, straight line fitting can be performed on the pixel points in the two partitions through preset fitting parameters according to the characteristics of the road surface, so that the fitted oblique line can be accurate as much as possible.

After the first oblique line and the second oblique line are detected, in order to avoid a problem that a large deviation exists between the first oblique line and the second oblique line and an oblique line corresponding to a real road surface, the terminal may determine whether a preset condition is satisfied between the first oblique line and the second oblique line based on a principle that the road surface is continuous. The preset condition is a condition for restraining the first oblique line and the second oblique line so as to ensure that the first oblique line and the second oblique line can meet the principle that the road surface is continuous.

Specifically, the terminal may determine an intersection of the first oblique line and the first boundary line to obtain a first intersection, and determine an intersection of the second oblique line and the first boundary line to obtain a second intersection. And when the distance between the first intersection point and the second intersection point is smaller than or equal to the preset distance, the first oblique line and the second oblique line are determined to meet the preset condition.

In the embodiment of the present invention, in consideration of the fact that the road surface is continuous in the actual situation, even if the road surface is not flat, a continuous oblique line composed of oblique line segments of different slopes is used to represent the road surface in the target V disparity map. That is, if the position of the first boundary line used to divide the target V disparity map into two partitions is reasonably selected, in an ideal case, the first oblique line and the second oblique line respectively detected according to the pixel points in the two partitions will be continuous. That is, in an ideal case, a first intersection point of the first oblique line and the first boundary line and a second intersection point of the second oblique line and the first boundary line should be the same point. In consideration of the error in the actual detection, even if there is a corresponding deviation between the first intersection point and the second intersection point, the deviation should be within a certain range in the case where the first boundary line position is reasonably selected.

Based on the above consideration, the terminal may set a preset distance according to the allowable deviation, compare the actually obtained distance between the first intersection point and the second intersection point with the preset distance, and when the distance between the first intersection point and the second intersection point is greater than the preset distance, it may be determined that the current position of the first boundary line is not selected reasonably, and the condition that the road surface is continuous is not satisfied between the first oblique line and the second oblique line respectively detected in the two zones obtained by dividing according to the first boundary line, and the first oblique line and the second oblique line cannot be used to represent the road surface. If the distance between the first intersection point and the second intersection point is less than or equal to the preset distance, it can be determined that the current position of the first boundary line is reasonably selected, and the first oblique line and the second oblique line can meet the condition that the road surface is continuous within the allowable deviation, that is, the first oblique line and the second oblique line can be used for representing the road surface.

For example, fig. 2C is a schematic diagram illustrating an embodiment of the present invention for determining whether a preset condition is satisfied between a first oblique line and a second oblique line. As shown in fig. 2C, the straight line L is a first boundary line, and for convenience of description, the two partitions are referred to as a first partition and a second partition, respectively. Wherein, the left side of the straight line L is a first subarea A, the right side of the straight line L is a second subarea B, when the oblique line is detected in the first subarea A, the obtained first oblique line is L₁When the diagonal line is detected in the second partition B, the obtained second diagonal line is L₂. Wherein, the first oblique line L₁An intersection O with the first boundary line L₁A first intersection point, a second oblique line L₂An intersection O with the first boundary line L₂Is the second intersection point. Assuming that the preset distance is d_tCalculating O₁And O₂D, when d > d_tThen, it may be determined that a preset condition is not satisfied between the first oblique line and the second oblique line.

When it is determined that the preset condition is satisfied between the first oblique line and the second oblique line by the above method, the terminal may perform step 203, and when the preset condition is not satisfied between the first oblique line and the second oblique line, the terminal may perform step 2023 described below.

Step 2023: when the preset condition is not met between the first oblique line and the second oblique line, the position of the first boundary line is corrected based on the first oblique line and the second oblique line, the target V parallax image is divided based on the first boundary line after the position is corrected to obtain two partitions, and the oblique lines are detected in the two divided partitions again.

As can be seen from the description in step 2022, when the preset condition is not satisfied between the first oblique line and the second oblique line, it may be determined that the position of the first boundary line currently used for dividing the target V disparity map is not reasonably selected, and the oblique line detected in the two partitions divided by the first boundary line cannot accurately represent the road surface. Based on this, the terminal may correct the position of the first boundary line based on the first oblique line and the second oblique line when it is determined that the first oblique line and the second oblique line do not satisfy the preset condition.

Specifically, the terminal may determine an intersection point of the first oblique line and the second oblique line, obtain a third intersection point, and determine a straight line that passes through the third intersection point and is parallel to the vertical axis of the target V disparity map as the corrected first boundary line.

Still taking fig. 2C as an example, when the terminal determines that the preset condition of continuity is not satisfied between the first oblique line and the second oblique line, the second oblique line may be intersected with the first oblique line through the extension line to obtain an intersection O of the first oblique line and the second oblique line₃At this time, the intersection point O₃I.e. the third intersection point, i.e. the corrected cut-off point, passing through the third intersection point O₃And a straight line L parallel to the longitudinal axis of the target V disparity map₃The corrected first boundary line is obtained.

After the terminal determines the corrected first boundary line, the target V disparity map may be re-divided by the corrected first boundary line to obtain two partitions, and then the terminal may re-detect the first oblique line and the second oblique line in the two partitions, and determine whether a preset condition is satisfied between the first oblique line and the second oblique line. And if the preset condition between the first oblique line and the second oblique line is still not met, continuously correcting the position of the first boundary line, and repeating the process.

Step 2024: and when the first oblique line and the second oblique line meet the preset condition, determining the first boundary line after the position is corrected for the last time as the target boundary line.

If the target V disparity map is divided by the first boundary line determined in step 2021, and a preset condition is satisfied between a first oblique line and a second oblique line detected in the two divided regions, it may be determined that the position of the first boundary line is reasonably selected, and the first oblique line and the second oblique line may be used to represent a road surface, and in this case, the first boundary line may be determined as the target boundary line. Of course, if the position of the first boundary line determined for the first time in step 2021 is not reasonably selected, after the position of the first boundary line is continuously corrected by the method in

step

2022 and 2023, it is detected that the first oblique line and the second oblique line satisfy the predetermined condition, at this time, the terminal may determine the first boundary line after the position is corrected for the last time as the target boundary line.

In the embodiment of the present invention, when the preset condition is not satisfied between the first oblique line and the second oblique line, the terminal may continue to correct the position of the first boundary line through step 2023 until the preset condition is satisfied between the first oblique line and the second oblique line detected in the two partitions divided by the corrected first boundary line, that is, the first boundary line currently used for dividing the target V disparity map is determined as the target boundary line. That is, in the embodiment of the present invention, the terminal may search and select a reasonable boundary line position by continuously adjusting the first boundary line, so as to perform reasonable area division on the target V disparity map, thereby ensuring that the oblique lines detected in the two target partitions are continuous under the condition that the deviation is allowed, and thus determining the actual continuous road surface based on the two continuous oblique lines not only ensures the accuracy of road surface detection, reduces the detection error, but also can avoid the problem of excessive calculation amount caused by detecting more oblique lines to determine the road surface.

Alternatively, the terminal may divide the target V disparity map into a plurality of partitions by a plurality of first boundary lines, wherein the above method may be applied to any two adjacent partitions in the plurality of partitions to determine a reasonable boundary line position for dividing the two partitions, that is, a target boundary line, and then the terminal may determine the road surface based on detected oblique lines in the plurality of partitions obtained by dividing the target V disparity map by the plurality of target boundary lines. The embodiment of the present invention is explained by taking an example of dividing two partitions by one first dividing line as an example, but this does not constitute a specific limitation to the embodiment of the present invention.

Step 203: and acquiring oblique lines detected in two partitions obtained after the target boundary divides the target V disparity map.

After the target boundary is determined by the method in step 202, the terminal may obtain two oblique lines detected in two target partitions obtained by dividing the target boundary, where the two oblique lines are the first oblique line and the second oblique line that satisfy the preset condition in step 202, and the subsequent terminal may determine the road surface more accurately by the method in step 204 according to the two oblique lines.

Step 204: and determining the road surface in the target V parallax map based on the two obtained oblique lines.

The two obtained oblique lines are a first oblique line and a second oblique line which meet preset conditions, namely, the first oblique line and the second oblique line are continuous within a deviation allowable range. In this case, the road surface can be represented by the first oblique line and the second oblique line. In this case, the terminal may determine the road surface in the processed V disparity map based on the first oblique line and the second oblique line.

Specifically, the terminal may determine slopes of the two oblique lines to obtain a first slope and a second slope, and when the first slope is different from the second slope, the terminal may determine a broken line obtained after one of the two oblique lines intersects with the other of the two oblique lines through an extension line as the road surface in the target V parallax map. When the first slope and the second slope are the same, the terminal may connect two intersections of the two oblique lines and the target boundary line, and in this case, the connected broken line may represent the road surface in the processed V-disparity map.

It should be noted that the first slope and the second slope may be the same or different. When the first slope and the second slope are different, the terminal may make the two oblique lines intersect directly through the extension line, and determine a broken line obtained after the intersection as a road surface in the target V disparity map. Of course, in some special cases, the two oblique lines may be parallel, and in this case, since the distance between the two oblique lines and the two intersection points of the target boundary line is close enough to be completely within the allowable range of the deviation, the terminal may directly connect the two intersection points to obtain a continuous broken line, which is the road surface in the processed V-disparity map.

For example, fig. 2D is a schematic diagram illustrating a method for determining a road surface in a target V disparity map based on two obtained oblique lines according to an embodiment of the present invention. As shown in fig. 2D, two slantsLine L₁And L₂Is different, in this case, the slope L can be extended₂Up to and with the slope L₁Intersect, by the oblique line L₁And an oblique line L₂And the broken line formed after connection is the road surface in the target V parallax map, as shown by the solid line part in FIG. 2D.

FIG. 2E shows the case where the slopes of the two oblique lines are the same, in which case the oblique line L₁The intersection point with the target boundary line L is O₁Oblique line L₂The intersection point with the target boundary line L is O₂Introducing O₁And O₂And connecting to obtain a continuous broken line, as shown by a solid line part in fig. 2E, where the continuous broken line is the road surface in the processed V-disparity map.

In the embodiment of the invention, when the V disparity map corresponding to the road condition image is acquired, the terminal can perform binarization processing on the V disparity map to obtain the target V disparity map. Then, the terminal may divide the target V parallax map into two sections based on the first boundary line to obtain two divided regions, detect oblique lines in the two divided regions, correct the position of the first boundary line based on the detected oblique lines to obtain the target boundary line if the oblique lines detected in the two first divided regions do not satisfy a preset condition, and determine the road surface in the target V parallax map based on the oblique lines detected in the two divided regions obtained after the target V parallax map is divided by the target boundary line. That is, in the embodiment of the present invention, when the oblique lines detected in the two sub-areas obtained by dividing the first boundary line do not satisfy the preset condition, the terminal may continuously adjust the position of the first boundary line to re-divide the target V disparity map, and finally determine to obtain the target boundary line, so that the two oblique lines detected in the two target sub-areas obtained by dividing the target boundary line can satisfy the condition that the road surface is continuous in practice.

Fig. 2F is a comparison graph showing the detection effect when detecting an uneven road surface based on the road surface detection method provided by the embodiment of the invention and detecting an uneven road surface based on the road surface detection method in the related art. The white thick dotted line in the graph is an accurate oblique line corresponding to the uneven road surface in the target V parallax map, wherein the inflection point is actually a turning surface with high and low unevenness in the corresponding road surface, theoretically, when the target boundary line passes through the inflection point, the obtained target partition is the most accurate, and at the moment, the two oblique lines with different slopes on two sides of the inflection point can accurately reflect the uneven road surface.

The left diagram in fig. 2F is a diagram of the detection effect of detecting the road surface by the road surface detection method in the related art, in which a straight line L₁Is a predetermined boundary line L₁Dividing the target V disparity map into two partitions according to the preset dividing line at the middle point of the width of the target V disparity map, wherein the oblique line detected in the left partition is L₂The slope detected in the right partition is L₃. Due to the preset boundary L in the related art₁The position is fixed when the preset boundary L₁If the position of the line is not properly selected, as shown in the left diagram, the predetermined dividing line deviates from the inflection point too much, and the slope detected in the two sub-areas cannot accurately represent the road surface.

The right diagram in fig. 2F is a detection effect diagram for detecting an uneven road surface by using the road surface detection method provided by the embodiment of the invention. By the pavement detection method provided by the embodiment of the invention, the terminal can continuously correct the position of the first boundary line according to the continuous condition of the pavement, so that the final target boundary line L is obtained₁At this time, the target boundary line L₁Will pass through the inflection point, or even if it deviates from the inflection point, the degree of deviation will be extremely small due to the existence of the constraint condition, and at this time, the target boundary line L is drawn₁Two oblique lines L detected in the two divided regions₂And L₃Obviously, the road surface can be represented more accurately.

As can be seen from the comparison graph of the detection effects, in the embodiment of the present invention, the terminal continuously adjusts the position of the first boundary line by using the road surface continuity as the constraint condition, and the finally obtained target boundary line satisfying the condition actually passes through the boundary line of the corresponding point in the target V disparity map where the road surface is uneven, so that two oblique lines existing in the target V disparity map due to the uneven road surface can be accurately divided into two partitions by the target boundary line.

Next, a road surface detection device provided by an embodiment of the present invention will be described.

Referring to fig. 3A, an embodiment of the present invention provides a road surface detecting device 300, where the device 300 includes:

the processing module 301 is configured to, when a V disparity map corresponding to a road condition image is obtained, perform binarization processing on the V disparity map to obtain a target V disparity map;

a first determining module 302, configured to divide the target V disparity map according to a first boundary line to obtain two partitions, detect a slope in the two partitions, and correct a position of the first boundary line based on the detected slope to obtain a target boundary line if the slope detected in the two first partitions does not satisfy a preset condition;

a second determining module 303, configured to determine a road surface in the target V disparity map based on oblique lines detected in two partitions obtained after the target boundary divides the target V disparity map.

Optionally, referring to fig. 3B, the first determining module 302 includes:

the detection submodule 3021 is configured to detect oblique lines in the two partitions respectively to obtain a first oblique line and a second oblique line;

the judgment submodule 3022 is configured to judge whether the first oblique line and the second oblique line meet a preset condition;

a correction submodule 3023, configured to, when a preset condition is not satisfied between the first oblique line and the second oblique line, correct a position of the first boundary line based on the first oblique line and the second oblique line, divide the target V disparity map based on the first boundary line after the position correction to obtain two partitions, and detect oblique lines in the two partitions obtained after the division, respectively, until it is detected that the preset condition is satisfied between the first oblique line and the second oblique line, determine the first boundary line after the last correction as the target boundary line.

Optionally, the judgment sub-module 3022 is specifically configured to:

determining the intersection point of the first oblique line and the first boundary line to obtain a first intersection point, and determining the intersection point of the second oblique line and the first boundary line to obtain a second intersection point;

when the distance between the first intersection point and the second intersection point is larger than a preset distance, determining that a preset condition is not met between the first oblique line and the second oblique line;

Optionally, the modification submodule 3023 is specifically configured to:

determining the intersection point of the first oblique line and the second oblique line to obtain a third intersection point;

and determining a straight line which passes through the third intersection point and is parallel to the longitudinal axis of the target V parallax map as the corrected first boundary line.

Optionally, referring to fig. 3C, the second determining module 303 includes:

a first determining submodule 3031, configured to determine slopes of two oblique lines detected in two partitions obtained after the target boundary line divides the target V disparity map, so as to obtain a first slope and a second slope;

and a second determining submodule 3032, configured to determine, when the first slope is different from the second slope, a broken line obtained by intersecting one of the two oblique lines with the other of the two oblique lines through an extension line as the road surface in the target V disparity map.

Optionally, referring to fig. 3D, the second determining module 303 further includes:

a connection submodule 3033, configured to connect two intersections of the two oblique lines and the target boundary line when the first slope and the second slope are the same;

and a third determining submodule 3034, configured to determine the connected polygonal line as the road surface in the target V disparity map.

In summary, in the embodiment of the present invention, when the V disparity map corresponding to the road condition image is obtained, the terminal may perform binarization processing on the V disparity map to obtain the target V disparity map. Then, the terminal may divide the target V parallax map into two sections based on the first boundary line to obtain two divided regions, detect oblique lines in the two divided regions, correct the position of the first boundary line based on the detected oblique lines to obtain the target boundary line if the oblique lines detected in the two first divided regions do not satisfy a preset condition, and determine the road surface in the target V parallax map based on the oblique lines detected in the two divided regions obtained after the target V parallax map is divided by the target boundary line. That is, in the embodiment of the present invention, when the oblique lines detected in the two sub-areas obtained by dividing the first boundary line do not satisfy the preset condition, the terminal may continuously adjust the position of the first boundary line to re-divide the target V disparity map, and finally determine to obtain the target boundary line, so that the two oblique lines detected in the two target sub-areas obtained by dividing the target boundary line can satisfy the condition that the road surface is continuous in practice.

It should be noted that: in the road surface detection device provided in the above embodiment, when performing road surface detection, only the division of the functional modules is illustrated, and in practical application, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the road surface detection device provided by the above embodiment and the road surface detection method embodiment belong to the same concept, and the specific implementation process thereof is described in the method embodiment in detail and is not described herein again.

Fig. 4 is a schematic structural diagram of a terminal 400 according to an embodiment of the present invention, and the terminal in the system architecture shown in fig. 1C may be the terminal shown in fig. 4. Specifically, the terminal 400 may be a vehicle-mounted terminal, a mobile terminal such as a smart phone and a tablet computer, or a terminal such as a desktop computer and a laptop computer.

Referring to fig. 4, the terminal 400 includes: a processor 401 and a memory 402.

The processor 401 is a control center of the terminal 400, connects various parts of the entire terminal using various interfaces and lines, performs various functions of the terminal 400 and processes data by operating or executing software programs and/or modules stored in the memory 402 and calling data stored in the memory 402, thereby monitoring the terminal 400 as a whole. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by operating the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to the use of the terminal 400 (such as a captured image, a calculated parallax image or a V-parallax map, etc.), and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 access to the memory 402.

In some embodiments, the terminal 400 may further optionally include: a peripheral interface 403 and at least one peripheral. The processor 401, memory 402 and peripheral interface 403 may be connected by bus or signal lines. Each peripheral may be connected to the peripheral interface 403 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 404, touch screen display 405, camera 406, audio circuitry 407, positioning components 408, and power supply 409.

The camera assembly 406 is used to capture images or video, among other things. Alternatively, camera assembly 406 may include at least two cameras. In some embodiments, the at least two cameras may be left and right cameras of a binocular camera, respectively. In some embodiments, camera assembly 406 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

Although not shown, the terminal 400 may further include various sensors, a display screen, and the like, which will not be described in detail herein. In this embodiment, the display unit of the terminal 400 is a touch screen display, and the terminal 400 further includes a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for executing the road surface detection method provided in the above-described embodiments.

In an exemplary embodiment, a non-transitory computer readable storage medium comprising instructions, such as the memory 402 comprising instructions, executable by the processor 401 of the terminal 400 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

A non-transitory computer readable storage medium, instructions in which, when executed by a processor of the terminal 400, enable the terminal to perform the road surface detection method provided in the above-described embodiments.

It should be noted that the terminal 400 provided in the foregoing embodiment may include the road surface detection device in the foregoing embodiment, the terminal 400 provided in the foregoing embodiment and the road surface detection device and the road surface detection method in the foregoing embodiment belong to the same concept, and the detailed implementation process thereof is referred to as the method embodiment, and is not described herein again.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims

1. A method of detecting a road surface, the method comprising:

dividing the target V disparity map according to a first boundary line to obtain two subareas, detecting oblique lines in the two subareas, determining an intersection point between the detected oblique lines if the oblique lines detected in the two subareas do not meet a preset condition, and correcting the position of the first boundary line based on the position of the intersection point to obtain a target boundary line, wherein the preset condition is a condition for ensuring that the detected oblique lines meet the principle that a road surface is continuous;

2. The method according to claim 1, wherein the detecting oblique lines in the two partitions, determining an intersection point between the detected oblique lines if the oblique lines detected in the two partitions do not satisfy a preset condition, and correcting the position of the first boundary line based on the position of the intersection point to obtain a target boundary line comprises:

when the preset condition is not met between the first oblique line and the second oblique line, determining an intersection point of the first oblique line and the second oblique line to obtain a third intersection point, correcting the position of the first boundary line based on the position of the third intersection point, dividing the target V parallax map based on the first boundary line after the position is corrected to obtain two partitions, respectively detecting oblique lines in the two partitions obtained after the division, and determining the first boundary line after the position is corrected for the last time as the target boundary line until the detected first oblique line and the second oblique line meet the preset condition.

3. The method according to claim 2, wherein the determining whether a preset condition is satisfied between the first oblique line and the second oblique line comprises:

4. The method of claim 2, wherein said correcting the position of the first boundary line based on the position of the third intersection comprises:

5. The method according to any one of claims 1 to 4, wherein the determining the road surface in the target V disparity map based on oblique lines detected in two regions obtained by dividing the target V disparity map by the target dividing line comprises:

6. The method according to claim 5, wherein the determining the slopes of two oblique lines detected in two regions obtained by dividing the target V disparity map by the target boundary line, and obtaining a first slope and a second slope further comprises:

7. A road surface detecting device, characterized in that the device comprises:

a first determining module, configured to divide the target V disparity map according to a first boundary line to obtain two sub-regions, detect oblique lines in the two sub-regions, determine an intersection point between the detected oblique lines if the oblique lines detected in the two sub-regions do not satisfy a preset condition, and correct the position of the first boundary line based on the position of the intersection point to obtain a target boundary line, where the preset condition is a condition for ensuring that the detected oblique lines satisfy a principle that a road surface is continuous;

8. The apparatus of claim 7, wherein the first determining module comprises:

and the correction submodule is used for determining an intersection point of the first oblique line and the second oblique line to obtain a third intersection point when a preset condition is not met between the first oblique line and the second oblique line, correcting the position of the first boundary line based on the position of the third intersection point, dividing the target V disparity map based on the first boundary line after the position is corrected to obtain two partitions, respectively detecting oblique lines in the two partitions obtained after the division, and determining the first boundary line after the position is corrected for the last time as the target boundary line when the preset condition is met between the detected first oblique line and the second oblique line.

9. The apparatus according to claim 8, wherein the determining sub-module is specifically configured to:

10. A road surface detection terminal, characterized in that the terminal comprises:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to the steps of any of the methods of claims 1-6.