CN111768437B - Image stereo matching method and device for mine inspection robot - Google Patents
Image stereo matching method and device for mine inspection robot Download PDFInfo
- Publication number
- CN111768437B CN111768437B CN202010614124.4A CN202010614124A CN111768437B CN 111768437 B CN111768437 B CN 111768437B CN 202010614124 A CN202010614124 A CN 202010614124A CN 111768437 B CN111768437 B CN 111768437B
- Authority
- CN
- China
- Prior art keywords
- image
- neighborhood
- pixel
- gray scale
- gray level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000007689 inspection Methods 0.000 title claims abstract description 21
- 238000004364 calculation method Methods 0.000 claims abstract description 26
- 230000002776 aggregation Effects 0.000 claims abstract description 13
- 238000004220 aggregation Methods 0.000 claims abstract description 13
- 238000010586 diagram Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 2
- 230000002159 abnormal effect Effects 0.000 description 9
- 230000007547 defect Effects 0.000 description 4
- 238000006467 substitution reaction Methods 0.000 description 3
- 239000003245 coal Substances 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20228—Disparity calculation for image-based rendering
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application relates to an image stereo matching method and device for a mine inspection robot, belongs to the technical field of stereo matching, and solves the problem that the existing image stereo matching method is poor in matching precision of repeated texture areas and weak texture areas. Acquiring a left view and a right view of an object as a reference image and a target image respectively, and performing pixel-by-pixel neighborhood replacement on the reference image and the target image respectively to obtain a gray level image corresponding to each pixel point in the reference image and the target image; binary codes corresponding to the gray level images of the reference image and the target image are respectively obtained, and cost quantity is obtained based on binary code calculation; respectively carrying out matching cost aggregation on the reference image and the target image based on the cost quantity to obtain the matching cost for removing noise; based on the matching cost of removing noise, the parallax image of the object is obtained, and the precision and quality of stereo matching are improved.
Description
Technical Field
The application relates to the technical field of stereo matching, in particular to an image stereo matching method and device for a mine inspection robot.
Background
Mine inspection robots are one of many types of robots used as coal mine robots and are often used for inspecting related equipment, environments, personnel and the like or identifying abnormal conditions in underground roadways. The physical space in the underground tunnel of the coal mine is limited, and a plurality of instruments, equipment and the like exist in the limited space, so that the autonomous navigation of the inspection robot is a great challenge. Therefore, in order to flexibly and autonomously avoid the obstacle and navigate and walk, the inspection robot needs to carry out three-dimensional matching on binocular vision images in the tunnel, so that the scene reconstruction of the whole tunnel environment is realized.
Binocular stereoscopic vision is one of important fields of computer vision research, and is widely applied to the fields of self-protection navigation of robots, unmanned vehicles, three-dimensional reconstruction, three-dimensional scanning, target tracking and the like. The stereo matching technology is the core of the whole binocular stereo vision system, and the precision and the speed of stereo matching directly influence the precision and the speed of the whole binocular stereo vision system. Along with the development of binocular stereoscopic vision systems, the real-time requirement of the high-precision binocular stereoscopic vision systems is urgent, the speed can be increased only to a certain extent by means of hardware acceleration, and the real-time requirement can not be met after a plurality of high-precision algorithms are accelerated, so that the real-time requirement has very important significance for the research of the rapid high-precision stereoscopic matching algorithms.
The traditional image stereo matching method has poor matching precision of the repeated texture region and the weak texture region, and meanwhile, the 8 neighborhood pixel points of each point to be matched in the range of the parallax are compared and encoded, and the encoding process is excessively dependent on the magnitude of a central pixel value and is easily interfered by abnormal points such as noise.
Disclosure of Invention
In view of the above analysis, the embodiment of the application aims to provide an image stereo matching method and device for a mine inspection robot, which are used for solving the problem that the matching precision of a repeated texture area and a weak texture area is poor in the existing image stereo matching method.
In one aspect, the embodiment of the application provides an image stereo matching method for a mine inspection robot, which comprises the following steps:
acquiring a left view and a right view of an object as a reference image and a target image respectively, and performing pixel-by-pixel neighborhood replacement on the reference image and the target image respectively to obtain a gray level image corresponding to each pixel point in the reference image and the target image;
binary codes corresponding to the gray level images of the reference image and the target image are respectively obtained, and a cost amount is calculated based on the binary codes;
respectively carrying out matching cost aggregation on the reference image and the target image based on the cost quantity to obtain noise-removed matching cost;
and obtaining a parallax image of the object based on the noise-removed matching cost.
Further, pixel-by-pixel neighborhood replacement is performed on the reference image and the target image respectively to obtain gray maps corresponding to each pixel point in the reference image and the target image, and the method comprises the following steps:
obtaining an 8 neighborhood gray scale map of each pixel point based on each pixel point in the reference image and the target image;
and acquiring a neighborhood value from four sides of the 8 neighborhood gray level image, acquiring gray level values of the four neighborhood values at the positions extending along different directions by a distance D, and replacing the corresponding neighborhood values with the gray level values to obtain the gray level image corresponding to each pixel point in the reference image and the target image.
Further, the calculation formula of the extension distance D is as follows:
wherein D is the extension distance, W is the width of the reference image or the target image, and H is the height of the reference image or the target image.
Further, binary codes corresponding to the gray level images of the reference image and the target image are respectively obtained, and the method comprises the following steps:
obtaining a first gray level image and a second gray level image of each pixel point based on the gray level image corresponding to each pixel point, wherein the first gray level image consists of four neighborhood pixels which are sequentially connected in the gray level image of each pixel point, and the rest part of the gray level image of each pixel point after the first gray level image is removed is the second gray level image;
respectively calculating the contrast value corresponding to the first gray scale image and the second gray scale image of each pixel point;
binary codes corresponding to the first gray level image and the second gray level image are respectively obtained based on the comparison values, and two groups of binary codes are spliced to obtain binary codes corresponding to the gray level images of the reference image and the target image respectively.
Further, the calculation formula of the comparison value is as follows:
wherein ,
wherein α is the contrast value of the first gray scale or the second gray scale, λ= Σλ pq ，λ pq Is the neighborhood value weight of the pixel point p, I q Is the neighborhood value of the neighborhood pixel q, I p For the pixel value of the pixel point p, q is a certain neighborhood pixel of the pixel point p, ω is a set of neighborhood pixels in the first gray scale map or the second gray scale map, σ 2 E is a balance constant, which is the variance of the neighborhood pixels.
Further, the formula for calculating the cost amount based on the binary code is:
in the formula ,Ccen (p, l) as cost quantity, i as parallax, S (p) as binary code corresponding to gray scale map of reference image, S' (p-l) as binary code corresponding to gray scale map of target image,is exclusive or.
On the other hand, the embodiment of the application provides an image stereo matching device for a mine inspection robot, which comprises the following components:
the pixel-by-pixel neighborhood replacing module is used for acquiring a left view and a right view of an object as a reference image and a target image respectively, and respectively carrying out pixel-by-pixel neighborhood replacement on the reference image and the target image to obtain a gray level image corresponding to each pixel point in the reference image and the target image;
the cost quantity calculation module is used for respectively acquiring binary codes corresponding to the gray level images of the reference image and the target image, and calculating the cost quantity based on the binary codes;
the matching cost aggregation module is used for respectively carrying out matching cost aggregation on the reference image and the target image according to the cost quantity to obtain the noise-removed matching cost;
and the parallax calculation and optimization module is used for obtaining a parallax image of the object according to the noise-removed matching cost.
Further, the pixel-by-pixel neighborhood replacement module performs the following procedure:
obtaining an 8 neighborhood gray scale map of each pixel point based on each pixel point in the reference image and the target image;
and acquiring a neighborhood value from four sides of the 8 neighborhood gray level image, acquiring gray level values of the four neighborhood values at the positions extending along different directions by a distance D, and replacing the corresponding neighborhood values with the gray level values to obtain the gray level image corresponding to each pixel point in the reference image and the target image.
Further, the calculation formula of the extension distance D is as follows:
wherein D is the extension distance, W is the width of the reference image or the target image, and H is the height of the reference image or the target image.
Further, the cost amount calculation module executes the following flow:
obtaining a first gray level image and a second gray level image of each pixel point based on the gray level image corresponding to each pixel point, wherein the first gray level image consists of four neighborhood pixels which are sequentially connected in the gray level image of each pixel point, and the rest part of the gray level image of each pixel point after the first gray level image is removed is the second gray level image;
respectively calculating the contrast value corresponding to the first gray scale image and the second gray scale image of each pixel point;
and respectively obtaining binary codes corresponding to the first gray level image and the second gray level image based on the comparison values, and splicing the two groups of binary codes to obtain binary codes respectively corresponding to the gray level images of the reference image and the target image.
Compared with the prior art, the application has at least one of the following beneficial effects:
1. the image stereo matching method for the mine inspection robot solves the defect that the existing stereo matching method is low in matching precision of weak textures and repeated areas by carrying out pixel-by-pixel neighborhood replacement on a reference image and a target image, and improves the stereo matching precision. Meanwhile, the gray level image after pixel-by-pixel replacement is divided into two parts, binary codes corresponding to the gray level images are obtained by splicing binary codes of the two gray level images, dependence on the central pixel value is reduced, and matching quality is improved.
2. The pixel-by-pixel neighborhood replacement is respectively carried out on the reference image and the target image, so that the gray level image corresponding to each pixel point in the reference image and the target image is obtained, the problem of mismatching caused by the existence of a repeated area or a weak texture area in the view in the existing image stereo matching method is solved, and the matching precision is improved.
3. The gray level images of the reference image and the target image are grouped to obtain a first gray level image and a second gray level image, binary codes obtained by the first gray level image and the second gray level image are spliced to obtain binary codes corresponding to the gray level images, the problem that the central pixel point is easily interfered by abnormal points such as noise due to the fact that the central pixel value is excessively depended in the encoding process of the existing image stereo matching method is solved, the dependence on the central pixel is reduced, and the stereo matching quality is improved.
In the application, the technical schemes can be mutually combined to realize more preferable combination schemes. Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the application, like reference numerals being used to refer to like parts throughout the several views.
FIG. 1 is a flow chart of a method for stereo image matching for a mine inspection robot in one embodiment;
FIG. 2 is a gray scale map obtained after pixel-by-pixel neighborhood replacement of a reference image and a target image, respectively, in one embodiment;
FIG. 3 is an 8-neighborhood gray scale map of pixel 128 in one embodiment;
FIG. 4 is a block diagram of an image stereo matching device for a mine inspection robot in another embodiment;
reference numerals:
100-pixel-by-pixel neighborhood replacement module; 200-a cost calculation module; 300-matching cost aggregation module; 400-a parallax computation and optimization module.
Detailed Description
The following detailed description of preferred embodiments of the application is made in connection with the accompanying drawings, which form a part hereof, and together with the description of the embodiments of the application, are used to explain the principles of the application and are not intended to limit the scope of the application.
The traditional image stereo matching method has poor matching precision of the repeated texture region and the weak texture region, and meanwhile, the 8 neighborhood pixel points of each point to be matched in the range of the parallax are compared and encoded, and the encoding process is excessively dependent on the magnitude of a central pixel value and is easily interfered by abnormal points such as noise. Therefore, the application provides the image stereo matching method and the device for the mine inspection robot, which are used for obtaining the gray level image corresponding to each pixel point in the reference image and the target image by carrying out pixel-by-pixel neighborhood replacement on the reference image and the target image, so that the problem that the matching precision of a repeated texture area and a weak texture area is poor in the existing image stereo matching method is solved, and the matching precision is improved. Meanwhile, the gray level images corresponding to each pixel point after pixel-by-pixel neighborhood replacement are divided into two groups, the contrast values corresponding to the two groups of gray level images are calculated respectively, and binary codes corresponding to the two groups of gray level images are spliced to obtain the binary code corresponding to each pixel point, so that the problem that the central pixel point is easily interfered by abnormal points such as noise in the encoding process by the existing image stereo matching method is solved, the dependence of the encoding on the central pixel point is reduced, and the problem of low matching precision under noise interference is effectively solved.
In one embodiment of the application, a stereo image matching method for a mine inspection robot is disclosed, as shown in fig. 1. The method comprises the following steps:
step S1, obtaining a left view and a right view of an object as a reference image and a target image respectively, and performing pixel-by-pixel neighborhood replacement on the reference image and the target image respectively to obtain gray maps corresponding to each pixel point in the reference image and the target image.
Specifically, a binocular camera mounted on the mine inspection robot obtains left and right views of an object by photographing, wherein one of the left and right views is used as a reference image, and the other view is used as a target image. In the application, four neighborhood values of each pixel point in the reference image and the target image are respectively replaced, and finally, the gray level image corresponding to each pixel point in the reference image and the target image is obtained. By respectively carrying out pixel-by-pixel neighborhood replacement on the reference image and the target image, the problem of mismatching caused by the existence of a repeated area and a weak texture area in the view in the existing stereo matching method can be solved, and the matching precision is improved.
Preferably, pixel-by-pixel neighborhood replacement is performed on the reference image and the target image to obtain gray maps corresponding to each pixel point in the reference image and the target image, including the following steps:
and step S101, obtaining an 8-neighborhood gray level map of the pixel points based on each pixel point in the reference image and the target image. Specifically, pixel-by-pixel neighborhood substitution is a corresponding substitution of a neighborhood value of each pixel, and first, an 8-neighborhood gray scale map of the pixel is obtained, wherein the 8-neighborhood gray scale map is of a 3*3 structure. For example, as shown in fig. 2, for a pixel with a pixel value of 18 in the reference image, the original 3*3 region has 8 neighborhood values as shown in fig. 2 (2), the first row includes three neighborhood values of 15, 16 and 17, the second row includes two neighborhood values of 17 and 19, and the third row includes three neighborhood values of 18, 20 and 21.
Step S102, acquiring a neighborhood value from four sides of the 8 neighborhood gray level map, acquiring gray level values of the positions of the four neighborhood values extending along different directions by a distance D, and replacing the corresponding neighborhood values with the gray level values to obtain the gray level map corresponding to each pixel point in the reference image and the target image.
Preferably, the calculation formula of the extension distance D is:
wherein D is the extension distance, W is the width of the reference image or the target image, and H is the height of the reference image or the target image.
Specifically, a neighborhood value is obtained from four sides of the 8-neighborhood gray level map, gray level values of the four neighborhood values extending along different directions for a distance D are collected, and the corresponding neighborhood values are replaced by the gray level values, so that the gray level map corresponding to each pixel point in the reference image and the target image is obtained. Exemplary, as shown in FIG. 2 (2), the neighborhood values at the four corners of the 8-neighborhood gray scale are 15, 17, 18 and 21, respectively, and are extended from the positions of the four neighborhood values by a certain distance D in different directions, wherein the neighborhood value 15 at the upper left corner can be extended in the vertical upward direction, the neighborhood value 17 at the upper left corner can be extended in the horizontal rightward direction, the neighborhood value 18 at the lower left corner can be extended in the horizontal leftward direction, the neighborhood value 21 at the lower right corner can be extended in the vertical downward direction, and the calculation formula of the extension distance D is thatThe extension distance may be adjusted based on the size of the view. Then, gray values extending a certain distance are collected, and the gray values are used for replacing corresponding neighborhood values, so that a gray image corresponding to the pixel 18 can be obtained, and as shown in fig. 2 (3), four neighborhood values after pixel-by-pixel neighborhood replacement are 87, 17, 163 and 55 respectively.
The pixel-by-pixel neighborhood replacement is respectively carried out on the reference image and the target image, so that the gray level image corresponding to each pixel point in the reference image and the target image is obtained, the problem of mismatching caused by the existence of a repeated area or a weak texture area in the view in the existing image stereo matching method is solved, and the matching precision is improved.
And S2, respectively acquiring binary codes corresponding to the gray level images of the reference image and the target image, and calculating based on the binary codes to obtain cost.
In consideration of the problem that the central pixel point is easily interfered by noise and other abnormal points caused by excessive dependence on the central pixel value in the coding process of the existing image stereo matching method, the gray level images corresponding to each pixel point are grouped to obtain two gray level images, binary codes obtained by the two gray level images are spliced to obtain binary codes respectively corresponding to the gray level images of the reference image and the target image, dependence of coding on the central pixel point is reduced, and the problem of low matching precision under noise interference is effectively solved.
Preferably, binary codes corresponding to gray levels of the reference image and the target image are respectively acquired, and the method comprises the following steps:
step 201, a first gray scale map and a second gray scale map of each pixel point are obtained based on the gray scale map corresponding to each pixel point, wherein the first gray scale map is composed of four adjacent pixels which are sequentially connected in the gray scale map of each pixel point, and the rest part of the gray scale map of each pixel point after the first gray scale map is removed is the second gray scale map.
For example, as shown in fig. 3, the 8-neighborhood gray scale map obtained by performing 8-neighborhood replacement on the pixel 128 groups four sequentially connected neighborhood pixels in the gray scale map to obtain a first gray scale map, and the gray scale map formed by the four remaining connected neighborhood values in the gray scale map of the pixel after the first gray scale map is removed is a second gray scale map, where sequentially connected neighborhood values 135, 116, 147 and 201 form the first gray scale map, and 121, 120, 107 and 117 form the second gray scale map.
Step S202, calculating contrast values corresponding to the first gray scale map and the second gray scale map of each pixel point respectively. In consideration of the problem that in the existing method, the central pixel point is easily interfered by noise and other abnormal points due to excessive dependence on the central pixel value in the coding process, the binary codes of the corresponding gray level images are obtained by calculating the contrast value, and then the binary codes of the two gray level images are spliced to obtain the binary codes corresponding to the gray level images of the reference image and the target image respectively.
Preferably, the calculation formula of the comparison value is:
wherein ,
wherein α is the contrast value of the first gray scale or the second gray scale, λ= Σλ pq ，λ pq Is the neighborhood value weight of the pixel point p, I q Is the neighborhood value of the neighborhood pixel q, I p For the pixel value of the pixel point p, q is a certain neighborhood pixel of the pixel point p, ω is a set of neighborhood pixels in the first gray scale map or the second gray scale map, σ 2 E is a balance constant, which is the variance of the neighborhood pixels.
Step 203, binary codes corresponding to the first gray level image and the second gray level image are respectively obtained based on the comparison value, and the two sets of binary codes are spliced to obtain binary codes corresponding to the gray level images of the reference image and the target image respectively.
For example, the contrast values 149 and 116 corresponding to the first gray scale map and the second gray scale map of the pixel point 128 in fig. 3 can be obtained through calculation in step S202, then the contrast value in the first gray scale map is compared with four neighborhood values according to Census transformation, if the contrast value is greater than the neighborhood value, the corresponding code is 0, otherwise, 1. The comparison value 149 of the first gray map is compared with the four neighborhood values 135, 116, 147 and 201, respectively, to obtain a binary code 0001. Similarly, the comparison value 116 of the second gray map is compared with its four neighborhood values 121, 120, 107 and 117, respectively, to obtain the binary code 1101. And finally, splicing the binary codes corresponding to the first gray level diagram and the second gray level diagram to obtain the binary code 00011101 corresponding to the pixel point 128. After binary codes respectively corresponding to the gray level images of the reference image and the target image are respectively calculated, the cost quantity can be calculated through the following formula.
Preferably, the formula for calculating the cost amount based on the binary code is:
in the formula ,Ccen (p, l) as a cost quantity, S (p) is a binary code corresponding to a gray scale map of the reference image, S' (p-l) is a binary code corresponding to a gray scale map of the target image, l is parallax,is exclusive or.
The gray level images of the reference image and the target image are grouped to obtain a first gray level image and a second gray level image, binary codes obtained by the first gray level image and the second gray level image are spliced to obtain binary codes corresponding to the gray level images, the problem that the central pixel point is easily interfered by abnormal points such as noise due to the fact that the central pixel value is excessively depended in the encoding process of the existing image stereo matching method is solved, the dependence on the central pixel is reduced, and the stereo matching quality is improved.
And step S3, respectively carrying out matching cost aggregation on the reference image and the target image based on the cost quantity to obtain the noise-removed matching cost. Specifically, cost aggregation is an essential step of a stereo matching algorithm, specifically, a balanced combination of an absolute value of gray level difference of three channels of an image and a gradient value in an X direction is adopted as a matching cost, and the formula is as follows:
C(p,l)＝C 1 (p,l)+C 2 (p,l)+C cen (p,l)
in the formula ,C1 (p, l) represents the matching cost value when the pixel point p is based on the absolute value of the image gray difference at the parallax of l, C 2 (p, l) represents the matching cost value when the pixel point p is based on the absolute value of the gradient difference of the image at the parallax of l, C (p, l) represents the matching cost, C cen (p, l) represents the cost amount obtained in step S2, I (p) represents the color component at pixel p in the reference view,representing the gradient value of pixel p in the x-direction in the reference view, p l Representing the corresponding point of p where the disparity is equal to l. />And->For balancing the influence of gray difference absolute value and gradient on total cost quantity, tau c 、τ g And the threshold value is cut off and used for weakening the influence of the abnormal cost quantity on the result. After the matching cost is obtained through the calculation formula, a least square method can be adoptedObtaining a noise-free matching cost-> wherein ,to normalize constant, N p Representing the neighborhood set of the pixel point p, and C (j, l) represents the initial cost value of a certain point j in the neighborhood of the pixel point p when the disparity value is l. K (p, j) represents the similarity kernel of the minimum spanning tree, and the calculation formula isD (p, j) is the minimum distance between pixel points p and j, and σ' is a constant.
The matching cost aggregation is carried out on the reference image and the target image respectively, so that the matching cost for removing noise is obtained, support and basis are provided for obtaining the parallax image in the later stage, and the method has important significance.
And S4, obtaining a parallax image of the object based on the matching cost of removing the noise. Specifically, after obtaining the matching cost of removing noise, searching a point with the optimal accumulated cost in the parallax range by using a WTA strategy as a corresponding matching point, and taking the parallax corresponding to the point as an initial parallax value, wherein a calculation formula corresponding to the WTA strategy is as followsWherein L is a parallax search range to obtain an initial parallax map corresponding to the left view and the right view. Then, the occlusion region and the mismatching error are detected by adopting a consistency principle, and the occlusion region is filled by a background filling method. And finally, enhancing the parallax refinement degree by using a weighted median filtering method to obtain a parallax image of the object.
Compared with the prior art, the image stereo matching method for the mine inspection robot solves the defect that the existing stereo matching method is low in matching precision in weak textures and repeated areas by replacing the reference image and the target image pixel by pixel neighborhood, and improves the stereo matching precision. Meanwhile, the gray level image after pixel-by-pixel replacement is divided into two parts, binary codes corresponding to the gray level images are obtained by splicing binary codes of the two gray level images, dependence on the central pixel value is reduced, and matching quality is improved.
In another embodiment of the present application, an image stereo matching device for a mine inspection robot is disclosed, as shown in fig. 4, including a pixel-by-pixel neighborhood replacing module 100, configured to obtain a left view and a right view of an object as a reference image and a target image, and perform pixel-by-pixel neighborhood replacement on the reference image and the target image to obtain a gray scale map corresponding to each pixel point in the reference image and the target image; the cost calculation module 200 is configured to obtain binary codes corresponding to the gray level images of the reference image and the target image respectively, and calculate a cost based on the binary codes; the matching cost aggregation module 300 is configured to perform matching cost aggregation on the reference image and the target image according to the cost amount, so as to obtain a matching cost for removing noise; the parallax calculation and optimization module 400 is configured to obtain a parallax map of the object according to the matching cost of removing noise.
The image stereo matching device for the mine inspection robot solves the defect that the existing stereo matching method is low in matching precision of weak textures and repeated areas by replacing the reference image and the target image pixel by pixel neighborhood, and improves the stereo matching precision. Meanwhile, the gray level image after pixel-by-pixel replacement is divided into two parts, binary codes corresponding to the gray level images are obtained by splicing binary codes of the two gray level images, dependence on the central pixel value is reduced, and matching quality is improved.
Preferably, the pixel-by-pixel neighborhood replacement module performs the following procedure:
obtaining an 8 neighborhood gray level map of the pixel points based on each pixel point in the reference image and the target image;
and acquiring a neighborhood value from four sides of the 8-neighborhood gray level map, acquiring gray level values of the positions of the four neighborhood values extending by a distance D along different directions, and replacing the corresponding neighborhood values with the gray level values to obtain the gray level map corresponding to each pixel point in the reference image and the target image.
Preferably, the calculation formula of the extension distance D is:
wherein D is the extension distance, W is the width of the reference image or the target image, and H is the height of the reference image or the target image.
The pixel-by-pixel neighborhood replacement module is used for carrying out pixel-by-pixel neighborhood replacement on the reference image and the target image, so that the defect of low matching precision of the existing stereo matching method in weak textures and repeated areas is overcome, and the stereo matching precision is improved.
Preferably, the cost amount calculation module performs the following procedure:
obtaining a first gray level image and a second gray level image of each pixel point based on the gray level image corresponding to each pixel point, wherein the first gray level image consists of four neighborhood pixels which are sequentially connected in the gray level image of each pixel point, and the rest part of the gray level image of each pixel point after the first gray level image is removed is the second gray level image;
respectively calculating the contrast value corresponding to the first gray scale image and the second gray scale image of each pixel point;
binary codes corresponding to the first gray level image and the second gray level image are obtained based on the comparison values respectively, and the two groups of binary codes are spliced to obtain binary codes corresponding to the gray level images of the reference image and the target image respectively.
The gray level image after pixel-by-pixel replacement is divided into two parts by the cost calculation module, binary codes corresponding to the gray level images are obtained by splicing binary codes of the two gray level images, dependence on a central pixel value is reduced, and matching quality is improved.
The present application is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present application are intended to be included in the scope of the present application.
Claims (5)
1. The image stereo matching method for the mine inspection robot is characterized by comprising the following steps of:
step S1, obtaining a left view and a right view of an object as a reference image and a target image respectively, and performing pixel-by-pixel neighborhood replacement on the reference image and the target image respectively to obtain a gray level image corresponding to each pixel point in the reference image and the target image; comprising the steps of (a) a step of,
obtaining an 8 neighborhood gray scale map of each pixel point based on each pixel point in the reference image and the target image;
acquiring a neighborhood value from four sides of the 8 neighborhood gray level image, acquiring gray level values of the four neighborhood values at the positions extending along different directions by a distance D, and replacing the corresponding neighborhood values with the gray level values to obtain gray level images corresponding to each pixel point in the reference image and the target image;
s2, binary codes corresponding to gray level images of the reference image and the target image are respectively obtained, and cost quantity is calculated based on the binary codes; wherein,
binary codes corresponding to gray level images of a reference image and a target image are respectively obtained, and the method comprises the following steps:
step S201, a first gray scale image and a second gray scale image of each pixel point are obtained based on the gray scale image corresponding to each pixel point, wherein the first gray scale image is composed of four neighborhood pixels which are sequentially connected in the gray scale image of each pixel point, and the rest part of the gray scale image of each pixel point after the first gray scale image is removed is the second gray scale image;
step S202, respectively calculating the contrast value corresponding to the first gray scale image and the second gray scale image of each pixel point;
the calculation formula of the comparison value is as follows:
wherein ,
wherein α is the contrast value of the first gray scale or the second gray scale, λ= Σλ pq ，λ pq Is the neighborhood value weight of the pixel point p, I q Is the neighborhood value of the neighborhood pixel q, I p For the pixel value of the pixel point p, q is a certain neighborhood pixel of the pixel point p, ω is a set of neighborhood pixels in the first gray scale map or the second gray scale map, σ 2 As the variance of the neighborhood pixels, ε is the equilibrium constant;
step S203, binary codes corresponding to the first gray level image and the second gray level image are respectively obtained based on the comparison value, and the two groups of binary codes are spliced to obtain binary codes corresponding to the gray level images of the reference image and the target image respectively; comparing the contrast value in the first gray level diagram with four field values according to Census transformation, if the contrast value is larger than the neighborhood value, the corresponding code is 0, otherwise, the code is 1;
step S3, respectively carrying out matching cost aggregation on the reference image and the target image based on the cost quantity to obtain the matching cost for removing noise;
step S4, obtaining a parallax map of the object based on the noise-removed matching cost, wherein the parallax map specifically comprises the following steps:
after obtaining the matching cost for removing noise, searching a point with the optimal accumulated cost in the parallax range by utilizing a WTA strategy to serve as a corresponding matching point, and taking the parallax corresponding to the point as an initial parallax value;
then, detecting a shielding region and a mismatching error by adopting a consistency principle, and filling the shielding region by using a background filling method;
and finally, enhancing the parallax refinement degree by using a weighted median filtering method to obtain a parallax image of the object.
2. The image stereo matching method according to claim 1, wherein the calculation formula of the extension distance D is:
wherein D is the extension distance, W is the width of the reference image or the target image, and H is the height of the reference image or the target image.
3. The image stereo matching method according to claim 1, wherein the formula for calculating the cost amount based on the binary code is:
in the formula ,Ccen (p, l) as a cost quantity, S (p) is a binary code corresponding to a gray scale map of the reference image, S' (p-l) is a binary code corresponding to a gray scale map of the target image, l is parallax,is exclusive or.
4. An image stereo matching device for a mine inspection robot, comprising:
the pixel-by-pixel neighborhood replacing module is used for acquiring a left view and a right view of an object as a reference image and a target image respectively, and respectively carrying out pixel-by-pixel neighborhood replacement on the reference image and the target image to obtain a gray level image corresponding to each pixel point in the reference image and the target image; wherein,
respectively carrying out pixel-by-pixel neighborhood replacement on the reference image and the target image to obtain gray maps corresponding to each pixel point in the reference image and the target image, wherein the method comprises the following steps of:
obtaining an 8 neighborhood gray scale map of each pixel point based on each pixel point in the reference image and the target image;
acquiring a neighborhood value from four sides of the 8 neighborhood gray level image, acquiring gray level values of the four neighborhood values at the positions extending along different directions by a distance D, and replacing the corresponding neighborhood values with the gray level values to obtain gray level images corresponding to each pixel point in the reference image and the target image;
the cost quantity calculation module is used for respectively acquiring binary codes corresponding to the gray level images of the reference image and the target image, and calculating the cost quantity based on the binary codes; wherein,
binary codes corresponding to gray level images of a reference image and a target image are respectively obtained, and the method comprises the following steps:
step S201, a first gray scale image and a second gray scale image of each pixel point are obtained based on the gray scale image corresponding to each pixel point, wherein the first gray scale image is composed of four neighborhood pixels which are sequentially connected in the gray scale image of each pixel point, and the rest part of the gray scale image of each pixel point after the first gray scale image is removed is the second gray scale image;
step S202, respectively calculating the contrast value corresponding to the first gray scale image and the second gray scale image of each pixel point;
the calculation formula of the comparison value is as follows:
wherein ,
wherein α is the contrast value of the first gray scale or the second gray scale, λ= Σλ pq ，λ pq Is the neighborhood value weight of the pixel point p, I q Is the neighborhood value of the neighborhood pixel q, I p For the pixel value of the pixel point p, q is a certain neighborhood pixel of the pixel point p, ω is a set of neighborhood pixels in the first gray scale map or the second gray scale map, σ 2 As the variance of the neighborhood pixels, ε is the equilibrium constant;
step S203, binary codes corresponding to the first gray level image and the second gray level image are respectively obtained based on the comparison value, and the two groups of binary codes are spliced to obtain binary codes corresponding to the gray level images of the reference image and the target image respectively;
the matching cost aggregation module is used for respectively carrying out matching cost aggregation on the reference image and the target image according to the cost quantity to obtain the noise-removed matching cost;
the parallax calculation and optimization module is used for obtaining a parallax image of an object according to the noise-removed matching cost, and the specific flow is as follows:
after obtaining the matching cost for removing noise, searching a point with the optimal accumulated cost in the parallax range by utilizing a WTA strategy to serve as a corresponding matching point, and taking the parallax corresponding to the point as an initial parallax value;
then, detecting a shielding region and a mismatching error by adopting a consistency principle, and filling the shielding region by using a background filling method;
and finally, enhancing the parallax refinement degree by using a weighted median filtering method to obtain a parallax image of the object.
5. The image stereo matching device according to claim 4, wherein the calculation formula of the extension distance D is:
wherein D is the extension distance, W is the width of the reference image or the target image, and H is the height of the reference image or the target image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010614124.4A CN111768437B (en) | 2020-06-30 | 2020-06-30 | Image stereo matching method and device for mine inspection robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010614124.4A CN111768437B (en) | 2020-06-30 | 2020-06-30 | Image stereo matching method and device for mine inspection robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111768437A CN111768437A (en) | 2020-10-13 |
| CN111768437B true CN111768437B (en) | 2023-09-05 |
Family
ID=72724199
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010614124.4A Active CN111768437B (en) | 2020-06-30 | 2020-06-30 | Image stereo matching method and device for mine inspection robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111768437B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116258759B (en) * | 2023-05-15 | 2023-09-22 | 北京爱芯科技有限公司 | Stereo matching method, device and equipment |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016001920A1 (en) * | 2014-07-03 | 2016-01-07 | Amiad Gurman | A method of perceiving 3d structure from a pair of images |
| CN106355608A (en) * | 2016-09-09 | 2017-01-25 | 南京信息工程大学 | Stereoscopic matching method on basis of variable-weight cost computation and S-census transformation |
| EP3182369A1 (en) * | 2015-12-15 | 2017-06-21 | Ricoh Company, Ltd. | Stereo matching method, controller and system |
| CN106931962A (en) * | 2017-03-29 | 2017-07-07 | 武汉大学 | A kind of real-time binocular visual positioning method based on GPU SIFT |
| CN107220997A (en) * | 2017-05-22 | 2017-09-29 | 成都通甲优博科技有限责任公司 | A kind of solid matching method and system |
| CN108010081A (en) * | 2017-12-01 | 2018-05-08 | 中山大学 | A kind of RGB-D visual odometry methods based on Census conversion and Local map optimization |
| CN108510529A (en) * | 2018-03-14 | 2018-09-07 | 昆明理工大学 | A kind of figure based on adaptive weight cuts solid matching method |
| CN108898575A (en) * | 2018-05-15 | 2018-11-27 | 华南理工大学 | A kind of NEW ADAPTIVE weight solid matching method |
| CN110473217A (en) * | 2019-07-25 | 2019-11-19 | 沈阳工业大学 | A kind of binocular solid matching process based on Census transformation |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10621446B2 (en) * | 2016-12-22 | 2020-04-14 | Texas Instruments Incorporated | Handling perspective magnification in optical flow processing |
-
2020
- 2020-06-30 CN CN202010614124.4A patent/CN111768437B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016001920A1 (en) * | 2014-07-03 | 2016-01-07 | Amiad Gurman | A method of perceiving 3d structure from a pair of images |
| EP3182369A1 (en) * | 2015-12-15 | 2017-06-21 | Ricoh Company, Ltd. | Stereo matching method, controller and system |
| CN106355608A (en) * | 2016-09-09 | 2017-01-25 | 南京信息工程大学 | Stereoscopic matching method on basis of variable-weight cost computation and S-census transformation |
| CN106931962A (en) * | 2017-03-29 | 2017-07-07 | 武汉大学 | A kind of real-time binocular visual positioning method based on GPU SIFT |
| CN107220997A (en) * | 2017-05-22 | 2017-09-29 | 成都通甲优博科技有限责任公司 | A kind of solid matching method and system |
| CN108010081A (en) * | 2017-12-01 | 2018-05-08 | 中山大学 | A kind of RGB-D visual odometry methods based on Census conversion and Local map optimization |
| CN108510529A (en) * | 2018-03-14 | 2018-09-07 | 昆明理工大学 | A kind of figure based on adaptive weight cuts solid matching method |
| CN108898575A (en) * | 2018-05-15 | 2018-11-27 | 华南理工大学 | A kind of NEW ADAPTIVE weight solid matching method |
| CN110473217A (en) * | 2019-07-25 | 2019-11-19 | 沈阳工业大学 | A kind of binocular solid matching process based on Census transformation |
Non-Patent Citations (1)
| Title |
|---|
| 基于视差与灰度双层支持窗的立体匹配算法;李小林等;《电子科技》;第32卷(第11期);12-17+27 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111768437A (en) | 2020-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110569704B (en) | Multi-strategy self-adaptive lane line detection method based on stereoscopic vision | |
| CN108520536B (en) | Disparity map generation method and device and terminal | |
| CN109902702B (en) | Method and device for detecting target | |
| CN111047620A (en) | Unmanned aerial vehicle visual odometer method based on depth point-line characteristics | |
| CN111429527B (en) | Automatic external parameter calibration method and system for vehicle-mounted camera | |
| CN109579825B (en) | Robot positioning system and method based on binocular vision and convolutional neural network | |
| CN111340922A (en) | Positioning and mapping method and electronic equipment | |
| CN112097732A (en) | Binocular camera-based three-dimensional distance measurement method, system, equipment and readable storage medium | |
| CN112801047B (en) | Defect detection method and device, electronic equipment and readable storage medium | |
| CN112700486B (en) | Method and device for estimating depth of road surface lane line in image | |
| CN111028281A (en) | Depth information calculation method and device based on light field binocular system | |
| CN112257668A (en) | Main and auxiliary road judging method and device, electronic equipment and storage medium | |
| CN109115232B (en) | Navigation method and device | |
| CN111768437B (en) | Image stereo matching method and device for mine inspection robot | |
| CN111415305A (en) | Method for recovering three-dimensional scene, computer-readable storage medium and unmanned aerial vehicle | |
| Tao et al. | Automated processing of mobile mapping image sequences | |
| CN114842340A (en) | Robot binocular stereoscopic vision obstacle sensing method and system | |
| CN113592015A (en) | Method and device for positioning and training feature matching network | |
| CN113706599B (en) | Binocular depth estimation method based on pseudo label fusion | |
| Li et al. | An efficient dense stereo matching method for planetary rover | |
| CN111862146A (en) | Target object positioning method and device | |
| CN117011481A (en) | Method and device for constructing three-dimensional map, electronic equipment and storage medium | |
| Diskin et al. | Dense 3D point-cloud model using optical flow for a monocular reconstruction system | |
| CN116152321A (en) | Model training method and device, image processing method and device | |
| CN112258582B (en) | Camera attitude calibration method and device based on road scene recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |