CN112017170B - Road pavement pit identification method, device and equipment based on three-dimensional shadow model - Google Patents
Road pavement pit identification method, device and equipment based on three-dimensional shadow model Download PDFInfo
- Publication number
- CN112017170B CN112017170B CN202010872629.0A CN202010872629A CN112017170B CN 112017170 B CN112017170 B CN 112017170B CN 202010872629 A CN202010872629 A CN 202010872629A CN 112017170 B CN112017170 B CN 112017170B
- Authority
- CN
- China
- Prior art keywords
- projection
- shadow
- area
- road surface
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 239000002131 composite material Substances 0.000 claims description 12
- 201000010099 disease Diseases 0.000 claims description 6
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 150000001875 compounds Chemical class 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000007547 defect Effects 0.000 abstract description 4
- 238000001514 detection method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0006—Industrial image inspection using a design-rule based approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30132—Masonry; Concrete
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Geometry (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a method, a device and equipment for identifying pits on a road surface based on a three-dimensional light and shadow model, wherein the method comprises the following steps: acquiring three-dimensional image data of a road surface; setting a plurality of groups of bi-directional projection group light beams with different projection angles by using a three-dimensional light and shadow model, respectively projecting the bi-directional projection group light beams on a road surface, and then determining a road surface shadow area formed by the road surface under the projection of each bi-directional projection group light beam; according to the road surface shadow areas formed by each two-way projection group, determining a common road surface shadow area which is a shadow area under the projection of the light beams of each two-way projection group as a final road surface projection area; carrying out image connected domain analysis on the final pavement shadow area, and determining that the independent and connected shadow area is a possible pavement pit area; determining the physical area and the average depth of each possible pavement pit; and determining pavement pits forming pavement defects according to the physical area and the average depth. The invention can accurately identify the pit slot on the road surface.
Description
Technical Field
The invention relates to the technical field of pavement detection, in particular to a method, a device and equipment for identifying pits in a pavement of a road based on a three-dimensional light and shadow model.
Background
The road surface pit is a serious road surface disease, which not only affects the comfort and stability of the driving, but also affects the safety of the driving; therefore, the method can find and repair the pits on the road surface in time and as early as possible, and has great significance for road traffic safety.
In recent years, machine learning, particularly deep learning technology has made a major breakthrough in the fields of image processing, object recognition and the like, and is also widely applied to detection of pits in road surfaces. However, machine learning techniques generally rely on learning samples and have a "black box" like mechanism of operation, which may produce abnormal, erroneous, difficult-to-interpret recognition results for unknown recognition samples, resulting in an inability to accurately identify roadway pits.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a method, a device and equipment for identifying the pits on the road surface based on a three-dimensional light and shadow model, which can avoid the phenomenon that the pits on the road surface cannot be accurately identified due to insufficient learning samples in the prior art.
According to one aspect of the invention, a method for identifying pits on a road surface based on a three-dimensional light shadow model is provided, which comprises the following steps:
Step 1, obtaining three-dimensional image data of a road surface;
Step 2, according to the three-dimensional image data of the road surface, utilizing a three-dimensional light shadow model to set a plurality of groups of two-way projection group light beams with different projection angles to be respectively projected on the road surface, and then determining a road surface shadow area formed by the road surface under the projection of each two-way projection group light beam; the two-way projection group light beams refer to two projection light beams with the same projection angle and different horizontal rotation angles, and the components of the two projection light beams on the horizontal plane are opposite to each other;
Step 3, determining a common pavement shadow area which is a shadow area under the projection of the light beams of each bi-directional projection group as a final pavement shadow area according to the pavement shadow area formed by each bi-directional projection group;
Step 4, carrying out image connected domain analysis on the final pavement shadow area, and determining that the independent and connected shadow area is a possible pavement pit area;
step 5, performing geometric measurement on the possible pavement pit areas, and determining the physical area and average depth of each possible pavement pit;
And 6, determining pavement pits forming pavement faults according to whether the physical area and the average depth of each possible pavement pit are within the preset range values.
Further, let L andFor a set of bi-directional projection set beams, the step of determining a shadow area of the road surface formed by the road surface under the projection of each bi-directional projection set beam comprises:
Step 201, determining an elevation value P z and a highest beam height delta max (P) of each three-dimensional image pixel point (P x,py) in the three-dimensional image data under the L projection beam, and determining a pixel value B (P x,py) of the three-dimensional image pixel point according to the following formula:
Wherein, the pixel value of 0 represents that the three-dimensional image pixel point is a shadow pixel, and the pixel value of 1 represents that the three-dimensional image pixel point is a non-shadow pixel, so as to determine a shadow image B formed by the road surface under the projection of the L projection light beam;
step 202, determining the position of the sensor in the sensor according to the above formula Pixel value/>, of each three-dimensional image pixel under projection of projection beamThereby determining the road surface position/>Shadow image formed under projection of projection beam/>
Step 203, determining a composite shadow image B c under the beam projection of the bi-directional projection group using the following formula:
wherein B c(px,py) refers to the pixel value of the compound shadow image B c at pixel point (p x,py), thereby determining the projection direction L and The common shadow area below is the road shadow area.
Further, in step 3, the final road surface projection area is determined according to the following formula:
Where, B f(px,py) is the pixel value of the final road shadow image B f at pixel point (p x,py), For composite shadow image/>, under the ith bi-directional projectionThe pixel value at pixel point (p x,py) to determine the final road projection area from the pixel value of B f(px,py).
Further, the projection angles of the plurality of groups of bi-directional projection group beams are the same, and the horizontal rotation angles are different; the projection angle range is (0 °,30 ° ].
Further, step 4 specifically includes:
Carrying out image connected domain analysis on all three-dimensional image pixels confirmed as a final shadow area of the pavement, and marking all connectable shadow area pixels as an independent pixel group in an 8-adjacent mode; wherein a single independent pixel group is a possible pavement pit area.
Further, step 5 comprises the following sub-steps:
Step 501, determining pixel groups forming each shadow area, counting the number of pixels in each pixel group, and calculating the physical area of each possible pavement pit area according to the physical area occupied by each pixel;
Step 502, calculating an average elevation of each shadow area pixel group according to elevation information of pixels in each shadow area pixel group, and selecting a non-shadow area with a preset range in the shadow area pixel group neighborhood to determine the average elevation of the non-shadow area, wherein a difference value between the average elevation of the non-shadow area and the average elevation of the shadow area is an average depth of the shadow area.
According to another aspect of the present invention, there is provided a road pavement pit recognition apparatus based on a three-dimensional light shadow model, comprising:
the road surface data acquisition module is used for acquiring three-dimensional image data of a road surface;
The road surface shadow area determining module is used for setting a plurality of groups of bi-directional projection group light beams with different projection angles to be respectively projected on the road surface by utilizing a three-dimensional light shadow model according to the three-dimensional image data of the road surface, and then determining a road surface shadow area formed by the road surface under the projection of each bi-directional projection group light beam; the two-way projection group light beams refer to two projection light beams with the same projection angle and different horizontal rotation angles, and the components of the two projection light beams on the horizontal plane are opposite to each other;
The final road surface projection area module is used for determining a common road surface shadow area which is a shadow area under the projection of the light beams of each two-way projection group as a final road surface projection area according to the road surface shadow area formed by each two-way projection group;
The possible pavement pit area determining module is used for carrying out image connected domain analysis on the final pavement shadow area and determining that the independent and connected shadow area is the possible pavement pit area;
The geometric information determining module is used for performing geometric measurement on the possible pavement pit slot areas and determining the physical area and average depth of each possible pavement pit slot;
The real pavement pit determining module is used for determining pavement pits forming pavement diseases according to whether the physical area and the average depth of each possible pavement pit are within the preset range value.
According to still another aspect of the present invention, there is provided a road pavement pit recognition apparatus based on a three-dimensional light shadow model, including a processor and a memory; wherein the memory is used for storing a computer program; the processor is used for executing the road pavement pit identification method based on the three-dimensional light shadow model.
Compared with the prior art, the invention has the following advantages: the invention is based on the three-dimensional geometric characteristics of the pavement pit, firstly sets a plurality of groups of bi-directional projection group beams with different projection angles by utilizing a three-dimensional light and shadow model to find the pit-type low-lying area on the pavement, and then discriminates the found low-lying area according to the physical area and the average depth, thereby identifying the pavement pit really conforming to the engineering practice definition.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.
FIG. 1 is a flowchart of an embodiment of a method for identifying a pit on a road surface based on a three-dimensional light and shadow model according to the present invention;
FIG. 2 is a block diagram of an embodiment of a three-dimensional shadow model-based road surface pit recognition device according to the present invention;
FIG. 3 is a schematic view of a single light source projection principle;
FIG. 4 is an exploded view of the projection direction of the point light source;
FIG. 5 is a schematic diagram of a multi-set bi-directional projection arrangement;
FIG. 6 is a schematic diagram of a binary shadow image connected domain analysis;
FIG. 7 is a schematic view of an elevation reference area;
FIG. 8 is a flowchart of a specific implementation of combining three-dimensional image data of a real road surface;
FIG. 9 is a three-dimensional image of an example of a typical pothole;
FIG. 10 is a graph of the pit identification end result of FIG. 9;
fig. 11 is a block diagram illustrating a construction of an embodiment of a road surface pit recognition apparatus based on a three-dimensional light and shadow model according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Firstly, the road pavement pit has the following obvious geometric characteristics compared with the normal pavement area: (1) The height of the pit is lower than that of a normal pavement area around the pit, and the pit is a pit-type low-lying area on the pavement; (2) The pit grooves have a certain area, and the pit grooves with too small area can only fall off the pavement aggregate; (3) The pit should have a certain depth, and too shallow a pit may be a local uneven area only and should not be counted as a road surface defect. The embodiment of the invention provides an automatic recognition method for the pits on the road surface based on the pure three-dimensional geometric characteristics based on the geometric characteristics.
Referring specifically to fig. 1, the method for identifying the pit on the road surface based on the three-dimensional light and shadow model disclosed by the embodiment of the invention comprises the following steps:
Step 1, obtaining three-dimensional image data of a road surface;
step 2, according to the three-dimensional image data of the road surface, utilizing a three-dimensional light shadow model to set a plurality of groups of two-way projection group light beams with different projection angles to be respectively projected on the road surface, and then determining a road surface shadow area formed by the road surface under the projection of each two-way projection group light beam;
Step 3, determining a common pavement shadow area which is shadow under the projection of each bi-directional projection group beam as a final pavement shadow area according to the pavement shadow area formed by each bi-directional projection group;
Step 4, carrying out image connected domain analysis on the final pavement shadow area, and determining that the independent and connected shadow area is a possible pavement pit area;
Step 5, carrying out geometric measurement on the possible pavement pit areas, and determining geometric information of each possible pavement pit, wherein the geometric information comprises the physical area and average depth of the possible pavement pit areas;
And 6, determining a real pavement pit according to the geometric information.
Referring to fig. 2, the embodiment of the invention further discloses a road pavement pit identification device based on a three-dimensional light and shadow model, which is characterized by comprising:
the road surface data acquisition module is used for acquiring three-dimensional image data of a road surface;
The road surface shadow area determining module is used for setting a plurality of groups of bi-directional projection group light beams with different projection angles to be respectively projected on the road surface by utilizing a three-dimensional light shadow model according to the three-dimensional image data of the road surface, and then determining a road surface shadow area formed by the road surface under the projection of each bi-directional projection group light beam; the two-way projection group light beams refer to two projection light beams with the same projection angle and different horizontal rotation angles, and the components of the two projection light beams on the horizontal plane are opposite to each other;
The final road surface projection area module is used for determining a common road surface shadow area which is a shadow area under the projection of the light beams of each two-way projection group as a final road surface projection area according to the road surface shadow area formed by each two-way projection group;
The possible pavement pit area determining module is used for carrying out image connected domain analysis on the final pavement shadow area and determining that the independent and connected shadow area is the possible pavement pit area;
The geometric information determining module is used for performing geometric measurement on the possible pavement pit slot areas and determining the physical area and average depth of each possible pavement pit slot;
The real pavement pit determining module is used for determining pavement pits forming pavement diseases according to whether the physical area and the average depth of each possible pavement pit are within the preset range value.
The road surface pit recognition method based on the three-dimensional light shadow model takes a road surface pit recognition device based on the three-dimensional light shadow model as an execution object of the step, or takes a module in the road surface pit recognition device based on the three-dimensional light shadow model as an execution object of the step. Specifically, step 1 uses a road surface data acquisition module as an execution object of the step, step 2 uses a road surface shadow area determination module as an execution object of the step, step 3 uses a final road surface projection area module as an execution object of the step, step 4 uses a possible road surface pit area determination module as an execution object of the step, step 5 uses a geometric information determination module as an execution object of the step, and step 6 uses a real road surface pit determination module as an execution object of the step.
Specifically, the three-dimensional image data of the road surface in the step 1 can be collected by using a three-dimensional road detection vehicle. Specifically, the three-dimensional road detection vehicle collects three-dimensional image data of a road surface by using a laser line scanning imaging system, and the scanning width is set to be the width of a single traffic lane. The three-dimensional laser line scanning imaging system can intercept and save three-dimensional pavement images along the travelling direction of the detection vehicle according to fixed intervals. Compared with the two-dimensional road detection vehicle, the three-dimensional road detection vehicle has the advantages of more abundant acquired information, less interference from light conditions and the like, and has been widely applied to road detection practice in recent years.
In the step 2, a three-dimensional shadow model is utilized to simulate the projection behavior of the point light source on the road surface, and the shadow area of the road surface under the projection of the simulated point light source is traced. As shown in fig. 3, any one point light source is assumed to originate from infinity, i.e., each projection direction corresponds to one point light source at infinity, and the projection light beams thereof are parallel light beams and are uniformly distributed on the road surface when reaching the road surface. Referring to fig. 4, the projection direction of the point light source is determined by a projection angle θ and a horizontal rotation angle ρ, wherein the projection angle θ refers to an angle between the projection beam and an xy plane (i.e., a horizontal plane), and the horizontal rotation angle ρ refers to an angle between a projection component of the projection beam on the xy plane and an x axis.
Based on the input three-dimensional pavement image data, when the three-dimensional light shadow model simulates a shadow area of a single point light source on a pavement, each three-dimensional image pixel is considered to be a starting tracing point of one light beam, and then the height of the light beam when the light beam reaches other image pixels is traced back and calculated along the projection direction until a specific condition for stopping tracing is reached. The three-dimensional light and shadow model repeats the tracing process for all three-dimensional image pixels until the corresponding stopping tracing condition is reached. Assuming that the three-dimensional image pixel point p= { P x,py,pz } is the start trace point of the light beam S, a series of light beam sampling points along the projection direction L can be defined as:
Where S (k) is the kth beam sampling point and Δs is the pitch used in the xy plane. Due to the discreteness of the image data, the minimum pitch between pixels is 1, so Δs=1 is desirable.
According to formula (1), each time a beam spot S (k) is sampled, the three-dimensional shadow model will compare it with a three-dimensional image pixel point adjacent to the xy planeBinding:
In the method, in the process of the invention, And/>X and y components of S (k),/>, respectivelyTo round up operators.
Based on the binding principle of formula (2), the light beam S is arranged at the pixel point of the three-dimensional imageThe beam height delta (S, P (k)) of (C) can be expressed as:
In the method, in the process of the invention, Is the z component of S (k).
Since a single three-dimensional image pixel may be associated in the trace back of multiple beams, i.e., a single three-dimensional image pixel may have multiple associated beam heights. The three-dimensional light shadow model only keeps track of the highest beam height at a single three-dimensional image pixel:
δmax(P)＝max{δ(S1,P),δ(S2,P),…,δ(Sn,P)} (4)
Where δ max (P) refers to the highest beam height at the three-dimensional image pixel point P, and δ (S i, P) refers to the ith beam height associated with the three-dimensional image pixel point P.
All beams starting from all three-dimensional image pixels will be traced back according to the principles described above and the highest beam height corresponding to the update is recorded at the associated three-dimensional image pixels. The judging conditions for ending tracing a certain light beam are as follows: 1) The beam has exceeded the image boundary; 2) The beam height of the beam at the currently associated three-dimensional image pixel is lower than the height of the currently associated three-dimensional image pixel, i.e. the beam cannot exceed the three-dimensional image pixel and reach the next pixel.
The highest beam height δ max (P) is a key indicator for determining whether the three-dimensional image pixel point P is a shadow pixel point. Specifically, if the elevation value of the three-dimensional image pixel point P is lower than the highest beam height, the three-dimensional image pixel point P is a shadow pixel point; otherwise, the three-dimensional image pixel point P is a non-shadow pixel point. The three-dimensional shadow model uses a shadow image to record a shadow region, the shadow image being in fact a binary image, in the present embodiment, a pixel value of "0" indicates that the pixel is a shadow pixel, and a pixel value of "1" indicates that the pixel is a non-shadow pixel. Let B be the shadow image in the projection direction L, the value of the shadow image B can be simply expressed as:
Where B (P x,py) refers to the pixel value of the shadow image B at pixel point (P x,py), and (P x,py) also refers to the pixel coordinates of the three-dimensional image pixel P, and P z refers to the elevation value of the three-dimensional image pixel P.
The above formula illustrates the basic principle of shadow simulation under single point source projection. In order to avoid shadows in the slope area, the three-dimensional shadow model in the embodiment of the invention adopts a two-way projection mode to simulate and calculate shadow areas in two projection directions of which the horizontal planes are opposite to each other, so as to find a low-lying area instead of the slope area, namely, consider that two conjugate point light sources are used for illuminating the pavement. Let L andTwo projection directions for bi-directional projection, i.e. L and/>Is a group of two-way projection group beams, and according to the decomposition principle shown in FIG. 4, L and/>The method has the following characteristics:
It can be seen that L and The two components on the horizontal plane are opposite to each other. Due to L and/>The projection direction of the bi-directional projection may be expressed by only (θ, ρ). Likewise, the projection direction/>, can be modeled using equations (1) through (5)Shadow image/>Then composite shadow image B c under bi-directional projection can be expressed as:
Where B c(px,py) refers to the pixel value of the compound shadow image B c at pixel point (p x,py). It can be seen that the shadow areas of the road surface under one set of two-way projection beams are the projection directions L and The common shadow region below.
Thus, in combination with the above, in step 2, the step of determining the shadow area of the road surface formed by the projection of each bi-directional projection group beam includes:
Step 201, determining an elevation value P z and a highest beam height delta max (P) of each three-dimensional image pixel point (P x,py) in the three-dimensional image data under the L projection beam, and determining a pixel value B (P x,py) of the three-dimensional image pixel point according to formula (5), thereby determining a shadow image B formed by the road surface under the projection of the L projection beam;
step 202, determining that the current is in the process according to the formula (5) Pixel value/>, of each three-dimensional image pixel under projection of projection beamThereby determining the road surface position/>Shadow image formed under projection of projection beam/>
Step 203, determining the composite shadow image B c under the projection of the bi-directional projection group beam by using the formula (7) to determine the projection direction L andThe common shadow area below is the road shadow area.
The above is a determination of the area of road surface shadow created by the projection of one of the sets of bi-directional projection beams. In order to better utilize the geometric characteristics of the pavement pit, the invention sets a plurality of groups of bi-directional projection group beams with different projection angles, and the three-dimensional light and shadow model can trace back shadow areas (or low-lying areas) in a plurality of bi-directional projection modes.
In one embodiment of the present invention, four sets of bi-directional projection beams are used for projection, and the specific arrangement is shown in fig. 5 and the following table.
In addition, in the step 2, the composite shadow image under the projection of each group of the two-way projection group beams, namely the pavement shadow area formed under the projection of each two-way projection group beam, can be calculated respectively according to the formula.
Further, in step 3, based on the composite shadow map under the beam projection of each set of bi-directional projection sets obtained in step 2, the embodiment of the present invention calculates the final shadow image B f of the road surface by using the rule of logical and, that is, according to formula (8), selects the common shadow area that is the shadow area under each set of bi-directional projection as the final shadow area of the road surface:
Where, B f(px,py) refers to the pixel value of the final road shadow image B f at pixel point (p x,py), Refers to the composite shadow image/>, under the ith two-way projectionThe final road surface projection area is determined from the pixel values at pixel points (p x,py) according to the pixel values of B f(px,py).
In the embodiment of the invention, as the final pavement shadow image B f simultaneously considers the composite projection behaviors of four bidirectional projection groups, the threshold condition for forming the shadow area is greatly improved, so that the invention not only can effectively identify the pit-type low-lying area such as the pavement pit, but also can well avoid the false identification at the non-pit-type low-lying area. In addition, the simulation of the projection behavior of the three-dimensional shadow model to the point light source is completely based on the three-dimensional geometric characteristics of the identified object, so that the identification method of the pit slot of the road pavement has good objectivity and stability, and the problems of over fitting, under fitting and the like which are required by a machine learning method do not exist.
In addition, because the road surface pit has very clear three-dimensional characteristics compared with the normal road surface area, the road surface pit automatic identification method based on the pure geometric characteristics is more direct, simple and efficient than a machine learning method, and can avoid various abnormal machine misjudgment or missed judgment cases caused by insufficient learning samples.
Further, step 4 specifically includes: carrying out image connected domain analysis on all three-dimensional image pixels confirmed as a final shadow area of the pavement, and marking all connectable shadow area pixels as an independent pixel group in an 8-adjacent mode; wherein a single independent pixel group is a possible pavement pit area.
Performing binary image connected domain analysis on the pavement final shadow image B f in an 8-adjacent mode by adopting a general image processing technology, and sequentially classifying each pixel group connected into pixel groups with independent marks, wherein the specific reference is shown in fig. 6; the single individual pixel group is an individual pavement depression area, and is also regarded as a possible pavement pit area.
Further, step 5 comprises the following sub-steps:
Step 501, determining pixel groups forming each shadow area, counting the number of pixels in each pixel group, and calculating the physical area of each possible pavement pit area according to the physical area occupied by each pixel;
Step 502, calculating an average elevation of each shadow area pixel group according to elevation information of pixels in each shadow area pixel group, and selecting a non-shadow area with a preset range in the shadow area pixel group neighborhood to determine the average elevation of the non-shadow area, wherein a difference value between the average elevation of the non-shadow area and the average elevation of the shadow area is an average depth of the shadow area.
In the embodiment of the present invention, based on the individual shadow pixel groups marked in step 4, each individual shadow pixel group is regarded as a possible pavement pit area, and the physical area and the average depth thereof are calculated.
Specifically, in step 501, the physical area of the i-th independent shadow pixel group T i can be calculated as follows:
A(Ti)＝Δα·ni (9)
where a (T i) is the physical area of the independent shadow pixel group T i, Δα is the data precision of the input three-dimensional road surface image (i.e., the physical area occupied by a single pixel, for example, 1mm or 2 mm), and n i is the number of pixels of the independent shadow pixel group T i.
Specifically, in step 502, for calculating the average depth, the average elevation of the entire shadow pixel group is calculated according to the elevation information of each individual shadow pixel group; and selecting a non-shadow area with a certain range in the neighborhood of the shadow pixel group as a reference, and calculating the average elevation of the non-shadow area. The difference between the average elevation of the non-shadow area and the average elevation of the shadow area is the average depth of the shadow area. As shown in fig. 7, assuming that the dashed frame is the smallest rectangular frame surrounding the independent shadow pixel group T i, an extended rectangular area can be obtained along the smallest rectangular frame extension distance r, and the extended rectangular area is the elevation reference area when calculating the average depth of the independent shadow pixel group T i. In the embodiment of the invention, the recommended reference range of the epitaxial distance r is as follows: r is more than or equal to 10cm and less than or equal to 25cm. Let Ω i be the set of all non-shadow pixels in the elevation reference region, then the average depth of the independent shadow pixel group T i can be calculated as follows:
where D (T i) refers to the average depth of the individual shadow pixel group T i, Mean elevation of independent shadow pixel group T i,/>Refers to the average elevation of all non-shadow pixel groups Ω i within the corresponding elevation reference region.
Further, in step 6, for the shadow area where the physical area a (T i) is too small or the average depth D (T i) is too light, the present invention will be excluded, so the present invention determines the real pit of the road surface according to whether the physical area and the average depth of each of the individual shadow pixel groups are within the preset range values. In practical engineering practice, pits with too small physical areas may be only pavement aggregates to fall off, while pits with too shallow areas may be only local uneven areas and should not be counted as pavement diseases, so that the actual pavement pits are finally determined. The final discrimination conditions for the pavement pits constituting the pavement defect identified by the present invention can be expressed as:
Where a min is the minimum defined area of the road surface pit, D min is the minimum defined depth of the road surface pit, T i =0 indicates that the independent shadow pixel group T i is the road surface pit, and T i =1 indicates that the independent shadow pixel group T i is not the road surface pit. Wherein, the values of A min and D min are considered by referring to industry specifications and combining engineering practice experience, and the recommended values are as follows: the minimum defined area a min＝25cm2 of the pit, the minimum defined depth D min = 15mm of the pit.
As shown in fig. 8, the concrete implementation flow of the present invention is explained in detail in conjunction with the real road surface three-dimensional image data. The present invention will be further described with reference to fig. 9 and 10, but embodiments of the present invention are not limited thereto.
Fig. 9 shows three-dimensional image data of 4 real roads, which contains a plurality of road pits of different sizes and different depths. The following identifies the road pits in fig. 9 with the same model parameters, respectively, where the light and shadow model parameters are shown in the following table:
the 4 example patterns in fig. 9 are respectively identified according to the unified model parameters, so that the final pit identification result shown in fig. 10 can be obtained. In fig. 10, the dark shaded area on the road surface is the pit area that is finally identified. It can be seen that some small pits are not identified as pit areas due to too small an area or too shallow an average depth, and that other pit areas as long as the area or average depth satisfies equation (11) can be effectively identified by the method of the present invention.
In addition, as shown in fig. 11, the embodiment of the invention further provides a road surface pit identifying device based on a three-dimensional light shadow model, which comprises: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, so that the at least one processor can execute the road pavement pit identification method based on the three-dimensional light and shadow model.
Where the memory and the processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting the various circuits of the one or more processors and the memory together. The bus may also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or may be a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over the wireless medium via the antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory may be used to store data used by the processor in performing operations.
According to the invention, based on the three-dimensional geometric characteristics of the pavement pit, firstly, a plurality of groups of bidirectional projection groups of light beams with different projection angles are set by utilizing a three-dimensional shadow model to find the pit-type low-lying area on the pavement, and then, the found low-lying area is screened according to the physical area and the average depth, so that the pavement pit really conforming to the engineering practice definition is identified.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.
Claims (4)
1. The method for identifying the pits on the road surface based on the three-dimensional shadow model is characterized by comprising the following steps of:
Step 1, obtaining three-dimensional image data of a road surface;
Step 2, according to the three-dimensional image data of the road surface, utilizing a three-dimensional light shadow model to set a plurality of groups of two-way projection group light beams with different projection angles to be respectively projected on the road surface, and then determining a road surface shadow area formed by the road surface under the projection of each two-way projection group light beam; the two-way projection group light beams refer to two projection light beams with the same projection angle and different horizontal rotation angles, and the components of the two projection light beams on the horizontal plane are opposite to each other;
Step 3, determining a common pavement shadow area which is a shadow area under the projection of the light beams of each bi-directional projection group as a final pavement shadow area according to the pavement shadow area formed by each bi-directional projection group;
Step 4, carrying out image connected domain analysis on the final pavement shadow area, and determining that the independent and connected shadow area is a possible pavement pit area;
step 5, performing geometric measurement on the possible pavement pit areas, and determining the physical area and average depth of each possible pavement pit;
Step 6, determining pavement pits forming pavement diseases according to whether the physical area and the average depth of each possible pavement pit are within a preset range value;
Let L and For a set of bi-directional projection set beams, the step of determining a shadow area of the road surface formed by the road surface under the projection of each bi-directional projection set beam comprises:
Step 201, determining an elevation value P z and a highest beam height delta max (P) of each three-dimensional image pixel point (P x,py) in the three-dimensional image data under the L projection beam, and determining a pixel value B (P x,py) of the three-dimensional image pixel point according to the following formula:
Wherein, the pixel value of 0 represents that the three-dimensional image pixel point is a shadow pixel, and the pixel value of 1 represents that the three-dimensional image pixel point is a non-shadow pixel, so as to determine a shadow image B formed by the road surface under the projection of the L projection light beam;
step 202, determining the position of the sensor in the sensor according to the above formula Pixel value of each three-dimensional image pixel under projection of projection beamThereby determining the road surface position/>Shadow image formed under projection of projection beam/>
Step 203, determining a composite shadow image B c under the beam projection of the bi-directional projection group using the following formula:
wherein B c(px,py) refers to the pixel value of the compound shadow image B c at pixel point (p x,py), thereby determining the projection direction L and The lower common shadow area is the road shadow area;
In step 3, the final road surface projection area is determined according to the following formula:
Where, B f(px,py) is the pixel value of the final road shadow image B f at pixel point (p x,py), For composite shadow image/>, under the ith bi-directional projection-A pixel value at a pixel point (p x,py), whereby the final road surface projection area is determined from the pixel value of B f(px,py);
Step 4 comprises:
carrying out image connected domain analysis on all three-dimensional image pixels confirmed as a final shadow area of the pavement, and marking all connectable shadow area pixels as an independent pixel group in an 8-adjacent mode; wherein a single independent pixel group is a possible pavement pit area;
Step 5 comprises the following sub-steps:
Step 501, determining pixel groups forming each shadow area, counting the number of pixels in each pixel group, and calculating the physical area of each possible pavement pit area according to the physical area occupied by each pixel;
Step 502, calculating an average elevation of each shadow area pixel group according to elevation information of pixels in each shadow area pixel group, and selecting a non-shadow area with a preset range in the shadow area pixel group neighborhood to determine the average elevation of the non-shadow area, wherein a difference value between the average elevation of the non-shadow area and the average elevation of the shadow area is an average depth of the shadow area.
2. The method for identifying the pits on the road surface based on the three-dimensional light and shadow model according to claim 1, wherein the projection angles of the light beams of the plurality of groups of two-way projection groups are the same, and the horizontal rotation angles are different; the projection angle range is (0 °,30 ° ].
3. Road pavement pit recognition device based on three-dimensional shadow model, characterized by comprising:
the road surface data acquisition module is used for acquiring three-dimensional image data of a road surface;
the road surface shadow area determining module is used for setting a plurality of groups of bi-directional projection group light beams with different projection angles to be respectively projected on the road surface by utilizing a three-dimensional light shadow model according to the three-dimensional image data of the road surface, and then determining a road surface shadow area formed by the road surface under the projection of each bi-directional projection group light beam; the two-way projection group light beams refer to two projection light beams with the same projection angle and different horizontal rotation angles, and the components of the two projection light beams on the horizontal plane are opposite to each other; let L and For a set of bi-directional projection set beams, the step of determining a shadow area of the road surface formed by the road surface under the projection of each bi-directional projection set beam comprises:
Determining an elevation value P z and a maximum beam height delta max (P) for each three-dimensional image pixel (P x,py) in three-dimensional image data under an L projection beam, determining a pixel value B (P x,py) for the three-dimensional image pixel according to the following formula:
Wherein, the pixel value of 0 represents that the three-dimensional image pixel point is a shadow pixel, and the pixel value of 1 represents that the three-dimensional image pixel point is a non-shadow pixel, so as to determine a shadow image B formed by the road surface under the projection of the L projection light beam;
also according to the above formula Pixel value of each three-dimensional image pixel under projection of projection beamThereby determining the road surface position/>Shadow image formed under projection of projection beam/>
The composite shadow image B c under bi-projection group beam projection is determined using the following formula:
wherein B c(px,py) refers to the pixel value of the compound shadow image B c at pixel point (p x,py), thereby determining the projection direction L and The lower common shadow area is the road shadow area;
The final road surface projection area module is used for determining a common road surface shadow area which is a shadow area under the projection of the light beams of each two-way projection group as a final road surface projection area according to the road surface shadow area formed by each two-way projection group; determining the final road surface projected area according to the following formula:
Where, B f(px,py) is the pixel value of the final road shadow image B f at pixel point (p x,py), For composite shadow image/>, under the ith bi-directional projection-A pixel value at a pixel point (p x,py), whereby the final road surface projection area is determined from the pixel value of B f(px,py);
The possible pavement pit area determining module is used for carrying out image connected domain analysis on the final pavement shadow area, determining that the independent and connected shadow area is the possible pavement pit area, and marking all the three-dimensional image pixel points confirmed as the final pavement shadow area as an independent pixel group according to an 8-adjacent mode; wherein a single independent pixel group is a possible pavement pit area;
The geometric information determining module is used for performing geometric measurement on the possible pavement pit slot areas and determining the physical area and average depth of each possible pavement pit slot; the process comprises the following steps:
Determining pixel groups forming each shadow area, counting the number of pixels in each pixel group, and calculating the physical area of each possible pavement pit area according to the physical area occupied by each pixel;
Calculating the average elevation of each shadow area pixel group according to the elevation information of the pixels in each shadow area pixel group, and selecting a non-shadow area with a preset range in the shadow area pixel group neighborhood to determine the average elevation of the non-shadow area, wherein the difference value between the average elevation of the non-shadow area and the average elevation of the shadow area is the average depth of the shadow area;
The real pavement pit determining module is used for determining pavement pits forming pavement diseases according to whether the physical area and the average depth of each possible pavement pit are within the preset range value.
4. The road pavement pit identification equipment based on the three-dimensional shadow model is characterized by comprising a processor and a memory; wherein the memory is used for storing a computer program; the processor is configured to execute the computer program to implement the road surface pit identification method based on the three-dimensional light shadow model according to claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010872629.0A CN112017170B (en) | 2020-08-26 | 2020-08-26 | Road pavement pit identification method, device and equipment based on three-dimensional shadow model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010872629.0A CN112017170B (en) | 2020-08-26 | 2020-08-26 | Road pavement pit identification method, device and equipment based on three-dimensional shadow model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112017170A CN112017170A (en) | 2020-12-01 |
| CN112017170B true CN112017170B (en) | 2024-05-14 |
Family
ID=73503297
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010872629.0A Active CN112017170B (en) | 2020-08-26 | 2020-08-26 | Road pavement pit identification method, device and equipment based on three-dimensional shadow model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112017170B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112444490B (en) * | 2021-02-01 | 2021-04-16 | 深圳宜美智科技股份有限公司 | Hole plugging defect detection method based on image detection and hole plugging defect detection equipment |
| CN115424232B (en) * | 2022-11-04 | 2023-02-14 | 深圳市城市交通规划设计研究中心股份有限公司 | Method for identifying and evaluating pavement pit, electronic equipment and storage medium |
| CN115830032B (en) * | 2023-02-13 | 2023-05-26 | 杭州闪马智擎科技有限公司 | Road expansion joint lesion recognition method and device based on old facilities |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005033682A1 (en) * | 2003-10-02 | 2005-04-14 | Daimlerchrysler Ag | Three-dimensional reconstruction of surface profiles |
| CN106023226A (en) * | 2016-05-31 | 2016-10-12 | 彭博 | Crack automatic detection method based on three-dimensional virtual pavement |
| KR101785205B1 (en) * | 2016-05-17 | 2017-10-18 | 한국과학기술원 | Method and Apparatus for Multi-object Segmentation based on Shadow using Depth Sensor |
| CN110414418A (en) * | 2019-07-25 | 2019-11-05 | 电子科技大学 | A kind of Approach for road detection of image-lidar image data Multiscale Fusion |
-
2020
- 2020-08-26 CN CN202010872629.0A patent/CN112017170B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005033682A1 (en) * | 2003-10-02 | 2005-04-14 | Daimlerchrysler Ag | Three-dimensional reconstruction of surface profiles |
| KR101785205B1 (en) * | 2016-05-17 | 2017-10-18 | 한국과학기술원 | Method and Apparatus for Multi-object Segmentation based on Shadow using Depth Sensor |
| CN106023226A (en) * | 2016-05-31 | 2016-10-12 | 彭博 | Crack automatic detection method based on three-dimensional virtual pavement |
| CN110414418A (en) * | 2019-07-25 | 2019-11-05 | 电子科技大学 | A kind of Approach for road detection of image-lidar image data Multiscale Fusion |
Non-Patent Citations (2)
| Title |
|---|
| 一种基于数字伴潮海岸线的潮滩淹没区仿真算法;张立华;朱庆;暴景阳;徐胜华;;武汉大学学报(信息科学版)(07);全文 * |
| 基于对比度受限自适应直方图多种路面裂缝检测与识别;周慧媛;邱书波;刘海英;马宏伟;;齐鲁工业大学学报(05);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112017170A (en) | 2020-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112017170B (en) | Road pavement pit identification method, device and equipment based on three-dimensional shadow model | |
| CN111855664B (en) | Adjustable three-dimensional tunnel defect detection system | |
| CN109165549B (en) | Road identification obtaining method based on three-dimensional point cloud data, terminal equipment and device | |
| CN109522804B (en) | Road edge identification method and system | |
| CN111666805B (en) | Class marking system for autopilot | |
| CN109583280A (en) | Lane detection method, apparatus, equipment and storage medium | |
| CN111179300A (en) | Method, apparatus, system, device and storage medium for obstacle detection | |
| EP1890264B1 (en) | Vehicle, image processing system, image processing method, image processing program, image processing system configuration method, and server | |
| CN103499337B (en) | Vehicle-mounted monocular camera distance and height measuring device based on vertical target | |
| CN102842031A (en) | Environment recognition device and environment recognition method | |
| CN107909012B (en) | Real-time vehicle tracking detection method and device based on disparity map | |
| CN103714343A (en) | Method for splicing and homogenizing road face images collected by double-linear-array cameras under linear laser illumination condition | |
| TWI611961B (en) | Device, method, and computer-readable medium for analyzing lane line image | |
| CN110136186B (en) | Detection target matching method for mobile robot target ranging | |
| CN108154146A (en) | A kind of car tracing method based on image identification | |
| CN106845482A (en) | A kind of license plate locating method | |
| CN105354816A (en) | Electronic component positioning method and device | |
| JP7485011B2 (en) | Road deterioration diagnosis device, road deterioration diagnosis system, road deterioration diagnosis method, and program | |
| CN112434583B (en) | Lane transverse deceleration marking line detection method and system, electronic equipment and storage medium | |
| US20230314169A1 (en) | Method and apparatus for generating map data, and non-transitory computer-readable storage medium | |
| CN117198057A (en) | Experimental method and system for road side perception track data quality inspection | |
| CN113032272A (en) | Automatic parking system test evaluation method, device, equipment and storage medium | |
| DE102016212774B4 (en) | Method and device for generating a map of the surroundings and for locating a vehicle | |
| CN109117866B (en) | Lane recognition algorithm evaluation method, computer device, and storage medium | |
| Yao et al. | Automated measurements of road cracks using line-scan imaging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |