CN108564604A

CN108564604A - Binocular vision solid matching method and device based on plane restriction and triangulation

Info

Publication number: CN108564604A
Application number: CN201810252302.6A
Authority: CN
Inventors: 张雪松; 白肖艳; 康学净; 明安龙; 苏圣
Original assignee: Beijing University of Posts and Telecommunications
Current assignee: Beijing University of Posts and Telecommunications
Priority date: 2018-03-26
Filing date: 2018-03-26
Publication date: 2018-09-21
Anticipated expiration: 2038-03-26
Also published as: CN108564604B

Abstract

An embodiment of the present invention provides a kind of binocular vision solid matching method and device based on plane restriction and triangulation.This method includes：The left images for obtaining two camera acquisitions, using any width as with reference to image；It determines the support point of reference picture and calculates parallax；Divide reference picture by predetermined manner；Segmentation area is divided into the first kind and/or the second class cut zone；By each first kind cut zone secondary splitting；According to the support point and parallax of every sub- cut zone, the parallax of non-supporting point in the region is determined；By each second class cut zone triangulation；The disparity search range of non-supporting point in the region is determined according to the vertex of each delta-shaped region and parallax；The corresponding point to be matched of the range is matched pixel-by-pixel with non-supporting point, determines the parallax of non-supporting point.The present invention can more accurately match occlusion area and large area texture-free region.And the disparity search range of small area cut zone is reduced, improve matching efficiency.

Description

Binocular vision solid matching method and device based on plane restriction and triangulation

Technical field

The present invention relates to technical field of computer vision, more particularly to a kind of pair based on plane restriction and triangulation Visually feel solid matching method and device.

Background technology

Binocular stereo vision includes mainly four steps：Binocular camera calibration, obtain two dimensional image to, image rectification, Binocular vision Stereo matching.Wherein, binocular vision Stereo matching includes：It is taken the photograph first with what binocular camera or two were placed in parallel Camera shoots same physics scene simultaneously, obtains left images；Then throwing of the same target in left images in scene is found Shadow point, is called corresponding points；Finally according to the parallax of corresponding points, i.e. corresponding points are given birth in the offset of pixel coordinate system u axis directions At disparity map.Institute in captured physics scene can be further calculated out according to the disparity map and similar triangle theory that are generated The actual range of target and video camera is stated, which is also referred to as stereoscopic vision depth calculation.Stereoscopic vision depth calculation is extensive It applies in the practical applications such as 3 D scene rebuilding, mobile robot autonomous navigation, while in necks such as medical imaging, industrial detections Application in domain also more comes also extensively.

Currently used binocular vision Stereo Matching Algorithm has：Local algorithm based on support window is cut based on image Or the Global Algorithm of Dynamic Programming, the non local matching algorithm etc. based on minimum spanning tree.

Since the left images are obtained from from different shooting angles, occlusion area or large area are unavoidably had The presence in texture-free region.Wherein, occlusion area refers to only in piece image as it can be seen that in another piece image without corresponding picture The region of prime information, be often as shooting angle it is different and caused by.Large area texture-free region, which typically refers to those, not to be had The region of obvious characteristic point, such as the white wall etc. in room area.And above-mentioned Stereo Matching Algorithm is all to carry out pixel-by-pixel It is matched, thus the ambiguity of missing and large area texture-free region in matching for the Pixel Information of occlusion area, Above-mentioned Stereo Matching Algorithm not can be well solved the matching in occlusion area or large area texture-free region, and matching can be caused wrong Accidentally rate is higher and matching efficiency is relatively low.

Invention content

Being designed to provide for the embodiment of the present invention is a kind of based on the binocular vision of plane restriction and triangulation solid Method of completing the square and device to reduce the matching error rate of binocular vision Stereo matching, and improve matching efficiency.Specific technical solution is such as Under：

In order to achieve the above objectives, in a first aspect, the present invention implement to provide it is a kind of based on plane restriction and triangulation Binocular vision solid matching method, this method include：

Obtain left image and the right side of the same photographed scene that two video cameras being placed in parallel were acquired respectively in the same time Image, and using any of which width as image is referred to, another width is as image to be matched；

It determines the support point in the reference picture, and calculates each parallax for supporting point；Wherein, the support point is Pixel in the reference picture, having in the image to be matched unique and correct match point；

According to preset image segmentation mode, the reference picture is split, obtains multiple cut zone；By each institute It states cut zone and is divided into first kind cut zone and/or divided area of the divided area more than first threshold no more than described Second class cut zone of first threshold；

For each first kind cut zone, execute：To the first kind cut zone second is carried out to divide, by this first Class cut zone is divided into the sub- cut zone of multiple first kind；According in the sub- cut zone of each first kind support point and support The parallax of point, is fitted the first plane equation of the sub- cut zone of the first kind；According to first plane equation, determine this first The parallax of non-supporting point in the sub- cut zone of class；Wherein, the sub- cut zone of the multiple first kind corresponds respectively to the shooting Different physical plane in scene；

For each second class cut zone, execute：Triangulation is carried out to the second class cut zone, by second class Cut zone is divided into multiple delta-shaped regions；For non-supporting point each of in each delta-shaped region, according to the triangle Each vertex in region and the parallax on each vertex determine the corresponding first disparity search range of the non-supporting point；By first parallax The corresponding point to be matched of each parallax is matched pixel-by-pixel with the non-supporting point in search range, determines regarding for the non-supporting point Difference；Wherein, each vertex is support point.

Optionally, the support point in the determination reference picture, including：

Using the top left corner pixel point of the reference picture as starting point, along the corresponding pixel coordinate system of the reference picture U axis and v axis build the grid that step-length is default step-length in the reference picture；It will be in the grid, described in addition to being located at The intersection point of reference picture edge supports point as candidate；Wherein, u is axial right, under v axial directions；

For each candidate support point, calculates the pre-set candidate and support within the scope of corresponding second disparity search of point The vector of the corresponding point to be matched of each parallax and candidate support to put it is vectorial at a distance from；If what is be calculated is described apart from medium and small In each distance of second threshold, the number of minimum distance is 1, then supports point is used as to support point the candidate；Wherein, described Within the scope of second disparity search the vector of the corresponding point to be matched of each parallax and the described candidate support the vector of point all in accordance with The sobel operators of pre-set dimension are calculated.

Optionally, described that the reference picture is split according to preset image segmentation mode, obtain multiple segmentations Region, including：

According to mean shift Meanshift image segmentation modes, the reference picture is split, is obtained at least one Cut zone.

Optionally, the described pair of first kind cut zone carries out second of segmentation, which is divided into Multiple sub- cut zone of the first kind, including：

According to preset plane fitting rule, the corresponding object of multiple support idea collection being fitted in the first kind cut zone Pat the plane equation in face；Wherein, the multiple support point subset corresponds respectively to physical different in the photographed scene Face；

In each physical plane that fitting obtains, if plurality of physical plane, which meets, presets merging condition, root According to the parallax of support point and support point in the multiple physical plane, the plane equation of physical plane after being merged； According to the plane equation of the plane equation of physical plane after merging and the physical plane not being merged, physical after each merging is calculated Intersection between face, the physical plane not being merged determines the sub- cut zone of the multiple first kind；Wherein, the default conjunction And condition is：In the multiple physical plane, any physical plane is less than third at a distance from least one other physical plane Threshold value；

In each physical plane that fitting obtains, if any two of which physical plane is unsatisfactory for the default merging Condition calculates the intersection between each physical plane, determines the multiple then according to the plane equation of each physical plane The sub- cut zone of the first kind.

Optionally, described according to preset plane fitting rule, the multiple support points being fitted in the first kind cut zone The plane equation of the corresponding physical plane of subset, including：

S1, random sampling consistency RANSAC algorithms are based on, from the current support point set Y in the first kind cut zone It chooses target and supports point subset O；Wherein, the O is in the Y, supports the largest number of support point subsets of point；

S2, the parallax put according to the support point and support of the O, are fitted the plane equation of the corresponding physical planes of the O；

S3, judge not to be fitted the first kind for supporting whether the support point number of point subset Y-O is less than preset ratio currently Support point number in cut zone；

If the support point number of the Y-O is no less than the support point number in the first kind cut zone of preset ratio, Then using the Y-O as the current support point set in the first kind cut zone, S1 is returned to step；

If the support point number of the Y-O is less than the support point number in the first kind cut zone of preset ratio, Using the plane equation for the physical plane being currently fitted as final fitting result.

Optionally, the support point in the sub- cut zone of each first kind of the basis and support the parallax of point, be fitted this First plane equation of a kind of sub- cut zone, including：

Set the first plane equation of the sub- cut zone of the first kind as：

D (u, v)=au+bv+c

Wherein, d (u, v) indicates that the parallax of pixel (u, v) in the sub- cut zone of the first kind, a, b, c are described first flat The parameter to be solved of face equation solves a, b, c by following formula：

Wherein, u_iIt indicates to support point in the corresponding pixel coordinate of the reference picture i-th in the sub- cut zone of the first kind The coordinate of the u axis of system, v_iIt indicates to support point in the corresponding pixel coordinate of the reference picture i-th in the sub- cut zone of the first kind The coordinate of the v axis of system, u is axial right, and under v axial directions, i=1 ..., m, m indicate the support point number in the sub- cut zone of the first kind, d_iIndicate i-th of parallax for supporting point in the sub- cut zone of the first kind.

Optionally, described for non-supporting point each of in each delta-shaped region, according to each top of the delta-shaped region Point and the parallax on each vertex determine the corresponding first disparity search range of the non-supporting point, including：

For each delta-shaped region, according to the parallax on each vertex and each vertex in the delta-shaped region, determine this three Second plane equation of plane where angular domain；

The disparity estimation value of the non-supporting point is determined according to second plane equation；

The first disparity search range includes：The parallax on each vertex, the parallax on each vertex adduction is subtracted it is pre- If the target in the default neighborhood described in parallax, the disparity estimation value and the non-supporting point after value supports the parallax of point； Wherein, the target supports the parallax of point to support the disparity estimation value of point with the target determined according to second plane equation Absolute value of the difference be no more than the 4th threshold value.

Optionally, described by the corresponding point to be matched of each parallax within the scope of first disparity search and the non-supporting click-through Row matches pixel-by-pixel, determines the parallax of the non-supporting point, including：

By following formula, the non-supporting point is determinedParallax：

Wherein,For the non-supporting pointParallax, d_nIndicate the disparity estimation value of the non-supporting point, Indicate that each parallax within the scope of first disparity search, N indicate that the parallax numbers within the scope of first disparity search, S indicate The support point set of the reference picture；It indicatesPosterior probability, be calculated by the following formula：

Wherein,Indicate d_nCorresponding point to be matched is the non-supporting pointCorrect match point it is general Rate,Indicate that the corresponding match point of each parallax within the scope of first disparity search is the non-branch It holds a littleCorrect match point probability.

Second aspect, an embodiment of the present invention provides a kind of based on the binocular vision of plane restriction and triangulation solid With device, which includes：

Acquisition module, the same photographed scene acquired respectively in the same time for obtaining the video camera that two are placed in parallel Left image and right image, and using any of which width as refer to image, another width is as image to be matched；

Determining module for determining the support point in the reference picture, and calculates each parallax for supporting point；Its In, the support point is in the reference picture, has the pixel of unique and correct match point in the image to be matched；

Divide module, for according to preset image segmentation mode, being split to the reference picture, obtaining multiple points Cut region；Each cut zone is divided into the first kind cut zone and/or divisional plane that divided area is more than first threshold Second class cut zone of the product no more than the first threshold；

First execution module is executed for being directed to each first kind cut zone：The is carried out to the first kind cut zone The first kind cut zone is divided into the sub- cut zone of multiple first kind by secondary splitting；According to the sub- cut section of each first kind The parallax of support point and support point in domain, is fitted the first plane equation of the sub- cut zone of the first kind；According to described first Plane equation determines the parallax of non-supporting point in the sub- cut zone of the first kind；Wherein, the sub- cut zone of the multiple first kind Correspond respectively to physical plane different in the photographed scene；

Second execution module is executed for being directed to each second class cut zone：Three are carried out to the second class cut zone The second class cut zone is divided into multiple delta-shaped regions by angle subdivision；For non-branch each of in each delta-shaped region It holds a little, the corresponding first disparity search model of the non-supporting point is determined according to the parallax on each vertex of the delta-shaped region and each vertex It encloses；The corresponding point to be matched of each parallax within the scope of first disparity search is matched pixel-by-pixel with the non-supporting point, really The parallax of the fixed non-supporting point；Wherein, each vertex is support point.

The third aspect, an embodiment of the present invention provides a kind of electronic equipment, including processor, communication interface, memory and Communication bus, wherein processor, communication interface, memory complete mutual communication by communication bus；

Memory, for storing computer program；

Processor, when for executing the program stored on memory, realize described in first aspect as above based on plane The method and step of the binocular vision Stereo matching of constraint and triangulation.

Fourth aspect, an embodiment of the present invention provides a kind of computer readable storage medium, the computer-readable storage Instruction is stored in medium, when run on a computer so that computer execute described in first aspect as above based on flat Face constrains and the method and step of the binocular vision Stereo matching of triangulation.

5th aspect, an embodiment of the present invention provides a kind of computer program products including instruction, when it is in computer When upper operation so that computer executes three-dimensional based on the binocular vision of plane restriction and triangulation described in first aspect as above Matched method and step.

Binocular vision solid matching method and device provided in an embodiment of the present invention based on plane restriction and triangulation, After reference picture is divided, the cut zone larger to area carries out second and divides, to determine to correspond to different physicals The sub- cut zone in face；The plane of sub- cut zone is further fitted according to the parallax of the support of sub- cut zone point and support point Equation, and determine according to the plane equation parallax of non-supporting point in sub- cut zone.Meanwhile the cut zone smaller to area Triangulation is carried out, multiple triangles are divided into；Further determined in triangle according to the parallax on vertex of a triangle and vertex First disparity search range of non-supporting point, and by the corresponding point to be matched of each parallax within the scope of this and the non-supporting point carry out by Pixel matching, so that it is determined that going out the parallax of the non-supporting point.

As it can be seen that the embodiment of the present invention, for each sub- cut zone of the larger cut zone of area, based on plane restriction Thought is determined the parallax of all non-supporting points in each sub- cut zone by the plane equation of each sub- cut zone, can realized Occlusion area and presence to missing pixel information match ambiguous large area texture-free region more precisely and robust Property preferably matches, to reduce matching error rate.And the embodiment of the present invention is carried out for the smaller cut zone of area Triangulation determines the disparity search range of non-supporting point in triangle according to vertex of a triangle and vertex parallax, greatly contracts The small disparity search range of non-supporting point, improves matching efficiency.

Certainly, it implements any of the products of the present invention or method must be not necessarily required to reach all the above excellent simultaneously Point.

Description of the drawings

In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technology description to be briefly described.

Fig. 1 is the binocular vision solid matching method provided in an embodiment of the present invention based on plane restriction and triangulation Flow chart；

Fig. 2 carries out second of segmentation to be realized in the step S104 of embodiment illustrated in fig. 1 to the first kind cut zone, by The first kind cut zone is divided into a kind of particular flow sheet of the sub- cut zone of multiple first kind；

Fig. 3 is a kind of particular flow sheet of step S201 in embodiment illustrated in fig. 2；

Fig. 4 is the simulation experiment result figure provided in an embodiment of the present invention；

Fig. 5 is the binocular vision Stereo matching device provided in an embodiment of the present invention based on plane restriction and triangulation Structure chart；

Fig. 6 is the structure chart of a kind of electronic equipment provided in an embodiment of the present invention.

Specific implementation mode

Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall within the protection scope of the present invention.

In binocular vision Stereo matching, in order to reduce the matching error rate of occlusion area and large area texture-free region, And matching efficiency is improved, as shown in Figure 1, the present invention provides a kind of based on the binocular vision of plane restriction and triangulation solid Matching process, this method include：

S101 obtains the left image for the same photographed scene that two video cameras being placed in parallel were acquired respectively in the same time And right image, and using any of which width as image is referred to, another width is as image to be matched.

In order to which the three-dimensional stereoscopic visual that apish eyes are generated when watching scene can obtain in computer vision The left image and right image for taking the same photographed scene that two video cameras being placed in parallel acquire respectively in the same time, with into one Step finds subpoint of the target in left images in the photographed scene, also referred to as corresponding points, to generate disparity map.So Afterwards, disparity map can be converted to the depth comprising each target in video camera and photographed scene by similar triangle theory to believe The depth map of breath, i.e. video camera and the actual range of each target in photographed scene, further to carry out three-dimensional reconstruction.

In the present embodiment, a photographed scene can also be acquired simultaneously by two camera units of a binocular camera Left images.

The left image or right image that video camera is acquired can be used as and refer to image, if being used as a wherein width with reference to figure Picture, then another width is exactly image to be matched.For example, using left image as image is referred to, then the purpose of the present invention is just to determine The parallax for going out each pixel in left image, generates the disparity map of left image.In the present embodiment, illustratively, by left image As with reference to image, each step is described in detail.

S102 determines the support point in the reference picture, and calculates each parallax for supporting point；Wherein, the branch It is a little pixel in the reference picture, to have in the image to be matched unique and correct match point to hold.

The present invention is based on such observations：Some point in actual scene is not usually isolated, but and some other Point coexists in the same physical plane, and accordingly, some pixel in reference picture is not usually isolated, but with it is other Some pixels correspond to the same physical plane in photographed scene jointly.So, the thought based on plane restriction, if energy Enough accurately determine the physical plane corresponding to each pixel in reference picture, so that it may using using plane as minimum calculation unit into Row parallax value is filled, you can to determine the parallax of the pixel according to the plane equation of the corresponding physical plane of pixel.Cause And having robustness is determined for the parallax of the pixel in occlusion area and large area texture-free region.

Based on above-mentioned observation, from the point of view of another angle, reference picture has necessarily corresponded at least one of photographed scene object Pat face.In order to determine the plane equation of the corresponding physical plane of reference picture, in the present embodiment, it may be predetermined that go out reference Some in image support point, these support that point is the pixel for having unique and correct match point in image to be matched, depending on Difference has preferable robustness, can be used for fit Plane equation in follow-up calculate.

It is understood that certainly existed in reference picture much has unique and correct match point in image to be matched Pixel, but in the present embodiment, do not need to all determine all this pixels, but can be selectively by this A part in kind pixel is determined as supporting point.

In one implementation, it determines the support point in the reference picture, may comprise steps of：

S11, using the top left corner pixel point of the reference picture as starting point, along the corresponding pixel coordinate of the reference picture The u axis and v axis of system build the grid that step-length is default step-length in the reference picture；By it is in the grid, in addition to being located at The intersection point of the reference picture edge supports point as candidate；Wherein, u is axial right, under v axial directions.

It is appreciated that by building grid in a reference image, by it is in grid, in addition to positioned at reference picture edge Intersection point supports point as candidate, and further supports to determine to support the mode of point that can make support point equably in point from candidate Distribution is in a reference image.When by supporting point and supporting the parallax fit Plane equation of point, the support point in each plane Distribution it is relatively uniform, and the density of the support point of Different Plane is more close, so as to so that the plane equation that fitting obtains It is more accurate.

For example, the size of reference picture is 256 × 256, top left corner pixel point is (0,0), and default step-length is 5 pixels, Along the u axis and v axis of the corresponding pixel coordinate system of reference picture, grid is built, then the pixel of point of intersection includes in grid：Grid The first row：(0,0), (0,5), (0,10) ..., (0,255), the second row of grid：(5,0), (5,5), (5,10) ..., (5, 255) ..., grid last column：(255,0), (255,5), (255,10) ..., (255,255).In above-mentioned intersection point, removal After the intersection point of reference picture edge, the pixel of remaining point of intersection is candidate support point：The second row of grid：(5,5), (5, 10) ..., (5,255), grid the third line：(10,5), (10,10) ..., (10,255) ..., grid last column：(255,5), (255,10) ..., (255,255).

Default step-length can be according to actual needs or empirically determined.

S12 calculates the pre-set candidate and supports the corresponding second disparity search model of point for each candidate support point The vector of the corresponding point to be matched of interior each parallax is enclosed at a distance from the vector that the candidate supports point；If the distance being calculated In be less than in each distance of second threshold, the number of minimum distance is 1, then supports point to be used as support point candidate；Wherein, The vector of the corresponding point to be matched of each parallax and the described candidate support the vector of point within the scope of second disparity search It is calculated according to the sobel operators of pre-set dimension.

Parallax is the difference of the abscissa of two subpoint of the target on left images in photographed scene, for example, mesh It is marked on the subpoint x of left image_lFor (5,10), in the subpoint x of right image_rFor (5,8), then x_lParallax d (x_l)=2.

Second disparity search range includes multiple parallax values, for a candidate support point, corresponding second Each parallax value within the scope of disparity search and a pixel (referred to as point to be matched) to be matched in image to be matched It is corresponding.For example, supporting point (5,255) for candidate, a parallax within the scope of corresponding second disparity search is 2, then should The corresponding point to be matched of parallax 2 is (5,253).

In the present embodiment, the minimum value of the second disparity search range can be 0, and maximum value can be set as needed, such as It is set as the half of the width of reference picture.For example, the width of reference picture is 256, then the second disparity search model can be set Enclose is 0~128.So candidate is supported for point (5,255), each parallax corresponds within the scope of corresponding second disparity search Point to be matched be：Pixel (5,137) ... in image to be matched, (5,255).It is, of course, understood that different Candidate supports that the maximum value of the corresponding second disparity search range of point can be different.

The pre-set dimension of sobel operators can be set according to actual needs, such as be set as 3 × 3.Second threshold can also It rule of thumb sets, is such as set as 2.In step s 12, the vector sum that candidate support point is calculated by sobel operators is each After the vector of point to be matched, at a distance from vector that can be by calculating the candidate vector for supporting point and each point to be matched, determine Go out the distance between candidate support point and each point to be matched, such as the distance can be 0,1,2.Apart from smaller, the candidate branch of expression It holds a little more similar to point to be matched.Further, if the number of minimum distance is 1 in calculated each distance, illustrate to wait Choosing support point has unique correct match point, then can support point is used as to support point the candidate.If conversely, calculated each The number of minimum distance is more than 1 in distance, then explanation has multiple points to be matched to support point similar to candidate, i.e., supports candidate There is ambiguity in the matching of point, then the candidate supports point will not be by as support point.

Such as：Candidate supports point L1, corresponding point to be matched to have 5, respectively R1-R5, second threshold 2.If Candidate supports that point L1 is respectively at a distance from point R1-R5 to be matched：0,1,3,2,1, then in each distance less than second threshold In (be followed successively by 0,1,1), the number of minimum distance 0 is 1, can then candidate support point L1 to have unique correctly match point R1 To support point as support point candidate.If candidate supports that point L1 is respectively at a distance from point R1-R5 to be matched：1,1,2, 2,2, then (it is followed successively by 1,1) in each distance less than second threshold, the number of minimum distance 1 is 2, then candidate support point There is ambiguity in the matching of L1, which supports point will not be by as support point.

If supporting point as support point one candidate, the parallax of the support point is exactly the abscissa of candidate support point The difference of the abscissa of matched point.

S103 is split the reference picture according to preset image segmentation mode, obtains multiple cut zone； It is little more than the first kind cut zone of first threshold and/or divided area that each cut zone is divided into divided area In the second class cut zone of the first threshold.

The purpose of reference picture segmentation is, is partitioned into as precisely as possible in reference picture and corresponds to different physical planes Cut zone.It in one implementation, can be according to mean shift Meanshift image segmentation modes, to the reference Image is split, and obtains multiple cut zone.Specifically, in cutting procedure, it can be directed to the size of reference picture, and/ Or other attributes of image, the color radius and space radius in Meanshift segmentations are rule of thumb adjusted, so that segmentation knot Fruit is as accurate as possible.Under normal conditions, when for this Attribute tuning color radius of image size and space radius, if figure As relatively large, then color radius is relatively small, and space radius is relatively large.For example, the image for being 450 × 375 for size, Color radius can be set as 30, space radius is set as 30；The image for being 128 × 128 for size, can be by color Radius is set as 40, and space radius is set as 20.

But in actual segmentation, it is also possible to which the case where will appear segmentation errors does not correspond in photographed scene Conplane different pixels point may be divided in same cut zone.The area of cut zone is bigger, this mistake occurs Possibility accidentally is bigger.Thus, in the present embodiment, a first threshold can be rule of thumb set, area is more than first The first kind cut zone of threshold value carries out secondary splitting, and thinks area no more than in the second class cut zone of first threshold Pixel both corresponds to same plane.Specifically, using Meanshift divide reference picture when, the setting of first threshold with Color radius in Meanshift segmentations is related with space radius.By taking above-mentioned example as an example：It is 450 × 375 in image size, When color radius is 30 and space radius is set as 30, first threshold can be set as to 800 pixels, i.e. a cut section If the pixel number in domain, more than 800, which is first kind cut zone.

Under normal conditions, when space radius is constant, if color radius is relatively large, first threshold is relatively small；Color When radius is constant, if space radius is relatively large, first threshold is relatively large.

In the present embodiment, reference picture can also be split using other image partition methods, the present invention to this simultaneously It does not limit.

S104 is executed for each first kind cut zone：Second is carried out to the first kind cut zone to divide, it will The first kind cut zone is divided into the sub- cut zone of multiple first kind；According to the support point in the sub- cut zone of each first kind Parallax with point is supported, is fitted the first plane equation of the sub- cut zone of the first kind；According to first plane equation, determine The parallax of non-supporting point in the sub- cut zone of the first kind；Wherein, the sub- cut zone of the multiple first kind corresponds respectively to institute State physical plane different in photographed scene.

Due to setting first kind cut zone there are segmentation errors, i.e., by corresponding to the different cut zone of multiple planes all It has been divided into the same first kind cut zone.Second thus can be carried out to first kind cut zone to divide, obtained multiple The sub- cut zone of the first kind makes the pixel in the sub- cut zone of each first kind both correspond to same plane.

In one implementation, the support point in the sub- cut zone of each first kind of basis in step S104 and support The parallax of point, is fitted the first plane equation of the sub- cut zone of the first kind, may include：

Set the first plane equation of the sub- cut zone of the first kind as：

D (u, v)=au+bv+c

The first plane equation based on the sub- cut zone of each first kind, it may be determined that go out in the sub- cut zone of the first kind The parallax of non-supporting point.Here non-supporting point is the pixel with the concept, in particular to parallax to be determined of supporting point opposite.

S105 is executed for each second class cut zone：Triangulation is carried out to the second class cut zone, by this Second class cut zone is divided into multiple delta-shaped regions；For non-supporting point each of in each delta-shaped region, according to this Each vertex of delta-shaped region and the parallax on each vertex determine the corresponding first disparity search range of the non-supporting point；By described The corresponding point to be matched of each parallax is matched pixel-by-pixel with the non-supporting point within the scope of one disparity search, determines the non-supporting point Parallax；Wherein, each vertex is support point.

In the present embodiment, although it is believed that area is all corresponding no more than the pixel in the second class cut zone of first threshold In same plane.But since the area of the second class cut zone is smaller, it includes support point number it is less, it is even possible that Not enough support points carry out plane fitting.Thus, for the second class cut zone, present invention employs the sides of triangulation Method carries out disparity computation.

Triangulation builds Delaunay triangles, is the basic research method in algebraic topology.To an image district After domain carries out triangulation, any two triangle is either non-intersecting or intersects at a line just.In the present embodiment, by In each vertex of triangle be support point, so the parallax on the parallax of the pixel in triangle and each vertex of triangle is more It is similar.And the parallax of the pixel in triangle and the parallax of a certain range of support point of surrounding are also more similar.It will be with one Point to be matched corresponding to the more similar parallax of the parallax of a pixel is matched pixel-by-pixel with the pixel, can be very big Ground reduces the disparity search range of the pixel, on the basis of ensureing to match accuracy, matching efficiency is improved, specific During computer is realized, matching primitives more quickly are realized.

In a kind of realization method of embodiment shown in Fig. 1, as shown in Fig. 2, dividing to the first kind in step S104 Region carries out second and divides, which is divided into the sub- cut zone of multiple first kind, may include：

S201, according to preset plane fitting rule, the multiple support idea set pairs being fitted in the first kind cut zone The plane equation for the physical plane answered；Wherein, the multiple support point subset corresponds respectively to different in the photographed scene Physical plane.

S202, in each physical plane that fitting obtains, if plurality of physical plane, which meets, presets merging condition, Then according to the parallax of support point and support point in the multiple physical plane, the plane side of physical plane after being merged Journey；According to the plane equation of the plane equation of physical plane after merging and the physical plane not being merged, object after each merging is calculated It pats face, the intersection between the physical plane that is not merged, determines the sub- cut zone of the multiple first kind；Wherein, described pre- If the condition of merging is：In the multiple physical plane, any physical plane is less than at a distance from least one other physical plane Third threshold value；

In the present embodiment, for multiple planes of the obtained first kind cut zone of step S201, if two therein flat The distance in face is smaller, then can merge the two planes, regards a plane as.It is appreciated that plane after merging Point number is supported to be more than the support point number for the single plane not merged, then being fitted by the support point of plane after merging Plane equation, the plane equation than single plane are more accurate.Above-mentioned third threshold value can rule of thumb be set, and such as be set as 2.

Specifically, calculating the distance of two interplanars, may be accomplished by：

Set the plane equation of plane 1 as：A₁*x+B₁*y+C₁*d+D₁=0, central point is (x₁, y₁, d₁), plane 2 Plane equation is：A₂*x+B₂*y+C₂*d+D₂=0, central point is (x₂, y₂, d₂), then the distance D=between plane 1 and plane 2 dis₁+dis₂, wherein

Specifically, " according to the parallax of support point and support point in the multiple physical plane, being fitted in step S202 The realization process of the plane equation of physical plane after being merged " can refer to " sub according to each first kind in step S104 The parallax of support point and support point in cut zone, is fitted the first plane equation of the sub- cut zone of the first kind " this step Rapid realization process.

In plane, each plane not merged after each merging, the pixel on the intersection of two neighboring interplanar meets The two neighboring respective plane equation of plane.After the intersection between determining all adjacent planes, first has also been determined that out The sub- cut zone of multiple first kind of class cut zone.

S203, in the obtained each physical plane of fitting, if any two of which physical plane be unsatisfactory for it is described pre- If the condition of merging, then according to the plane equation of each physical plane, the intersection between each physical plane is calculated, determines institute State the sub- cut zone of multiple first kind.

It is realized by above-mentioned steps S201-S203 and secondary splitting is carried out to first kind cut zone, it is obtained each The sub- cut zone of the first kind has corresponded to physical plane different in photographed scene.Thus in the present embodiment, it can be based on determining The sub- cut zone of the first kind in support point be fitted the plane equation of the sub- cut zone of the first kind.

In a kind of specific implementation of realization method shown in Fig. 2, as shown in figure 3, in step S201 according to default Plane fitting rule, be fitted the plane side of the corresponding physical plane of multiple support idea collection in the first kind cut zone Journey may include：

S301 is based on random sampling consistency RANSAC algorithms, the current support point set Y out of this first kind cut zone Middle selection target supports point subset O；Wherein, the O is in the Y, supports the largest number of support point subsets of point；

RANSAC algorithms relatively accurately will can select in image corresponding to conplane pixel.Due to this Invention is fitted the equation of the plane particular by the parallax of support point and support point corresponding to a plane.So this implementation In example, it is only necessary to click the support corresponding to each plane in first kind cut zone by RANSAC algorithms and take out i.e. It can.Specifically, the sequence of point number from more to less can be supported according to supporting that idea is concentrated, fitting is each one by one supports point subset The plane equation of corresponding plane.

S302 is fitted the plane side of the corresponding physical planes of the O according to the parallax of the support of O point and support point Journey；

Specifically, the realization process of step S302 can refer to step S104 in " according to the sub- cut section of each first kind Support point in domain and the parallax for supporting point, are fitted the first plane equation of the sub- cut zone of the first kind " the step for reality Existing process.

S303, judge not to be fitted currently the support point number for supporting point subset Y-O whether less than preset ratio this first Support point number in class cut zone；

If it is appreciated that current be not fitted supports the support point number of point subset Y-O very little, or even not currently being fitted support Support point not enough point subset Y-O carries out plane fitting, then in this case, cannot not be fitted according to currently The support point of point subset Y-O is supported to be fitted its plane equation.Specifically, above-mentioned preset ratio can rule of thumb be set, such as It is set as 10%.

S304, if the support point number of the Y-O is no less than the support point in the first kind cut zone of preset ratio Number returns to step S301 then using the Y-O as the current support point set in the first kind cut zone；

S305, if the support point number of the Y-O is less than the support point in the first kind cut zone of preset ratio Number, then using the plane equation for the physical plane being currently fitted as final fitting result.

For the support point of the remaining support point subset Y-O not being fitted, the step of in the exemplary embodiment illustrated in fig. 2 During determining plane intersection line in S202 or step S203, it is comprised in the sub- cut zone of some first kind.Specifically, this A little support points not being fitted may be comprised in the sub- cut zone of the different first kind, or may be comprised in identical The sub- cut zone of the first kind.

By the iteration of above-mentioned steps S301-S305, the flat of multiple and different planes in first kind cut zone can be obtained Face equation.And during specific computer is realized, can above-mentioned steps be carried out to multiple first kind cut zone simultaneously The iterative processing of S301-S305, therefore improve matching efficiency.

In a kind of realization method of embodiment shown in Fig. 1, being directed in step S105 is every in each delta-shaped region A non-supporting point determines corresponding first parallax of the non-supporting point according to the parallax on each vertex of the delta-shaped region and each vertex Search range may include：

The first disparity search range includes：The parallax on each vertex, the parallax on each vertex adduction is subtracted it is pre- If the target in the default neighborhood belonging to parallax, the disparity estimation value and the non-supporting point after value supports the parallax of point； Wherein, the target supports the parallax of point to support the disparity estimation value of point with the target determined according to second plane equation Absolute value of the difference be no more than the 4th threshold value.

In above-mentioned realization method, " be directed to each delta-shaped region, according in the delta-shaped region each vertex and each top The realization process of the parallax of point, the second plane equation of plane where determining the delta-shaped region " can refer in step S104 " according in the sub- cut zone of each first kind support point and support point parallax, be fitted the sub- cut zone of the first kind The realization process of the step for first plane equation ".

Illustratively, it is assumed that identified second plane equation is：A*x+B*y+C*d+D=0, then a non-branch in triangle Hold point (x₀, y₀) disparity estimation value be

Above-mentioned preset value can be a value or multiple values, be worth if it is one, can be 1, if it is multiple values, Can be 1,2 etc..Such as：The parallax value on one vertex of triangle is 2, preset value 1, then parallax adduction in vertex is subtracted preset value Parallax afterwards is：1 and 3.

Default neighborhood belonging to the above-mentioned non-supporting point can be：Reference picture is divided into the rectangle of multiple default sizes Behind region, the rectangular area belonging to it is determined according to the coordinate of the non-supporting point.It more specifically, can be to each rectangle region Domain carries out position mark, and the position mark of the rectangular area belonging to it is determined according to the coordinate of the non-supporting point, is also determined that out Which rectangular area rectangular area described in the non-supporting point is.Such as：The image that reference picture is 40 × 40, is divided into This 4 rectangular areas are respectively labeled as (0,0), (0,1), (1,0), (1,1) by 4 20 × 20 zonules.One non-supporting The coordinate of point is (3,5), then 3/20=0,5/20=0, so it belongs to (0,0) this region；The seat of another non-supporting point (35,2) are designated as, similarly 35/20=1,2/20=0, then it belongs to (1,0) this rectangular area.Above-mentioned default size can basis Experience is set, and is such as set as 20 × 20.Equally, above-mentioned 4th threshold value can also rule of thumb be set, and such as be set as 2.

It determines in triangle after the first disparity search range of a non-supporting point, it can be by the first disparity search model The corresponding each point to be matched of each parallax enclosed is matched one by one with the non-supporting point, to determine the branch from each point to be matched Hold match point a little.

In a kind of realization method of embodiment shown in Fig. 1, in step S105 will be within the scope of first disparity search The corresponding point to be matched of each parallax is matched pixel-by-pixel with the non-supporting point, is determined the parallax of the non-supporting point, be may include：

By following formula, the non-supporting point is determinedParallax：

It is above-mentionedAndIt is a kind of expression of randomization.Specifically, this implementation In example, non-supporting point can be calculatedWith the similarity of each point to be matched, which can use non-supporting pointPixel value It is indicated with the absolute value of the difference of the pixel value of point to be matched.If the pixel value of a point to be matched and non-supporting pointPixel The absolute value of the difference of value is smaller, indicates this point to be matched and non-supporting pointMore similar, then this point to be matched is non-supporting PointCorrect match point probability it is bigger.

Binocular vision solid matching method provided in an embodiment of the present invention based on plane restriction and triangulation, for face Each sub- cut zone of the larger cut zone of product, the thought based on plane restriction pass through the plane equation of each sub- cut zone The parallax for determining all non-supporting points in each sub- cut zone, can realize the occlusion area to missing pixel information and presence It matches ambiguous large area texture-free region more precisely and robustness preferably matches, to reduce matching error Rate.And the embodiment of the present invention carries out triangulation, according to vertex of a triangle and vertex for the smaller cut zone of area Parallax determines the disparity search range of non-supporting point in triangle, greatly reduces the disparity search range of non-supporting point, improves Matching efficiency.

In order to verify the binocular vision Stereo matching side provided in an embodiment of the present invention based on plane restriction and triangulation The advantageous effect of method, the present invention have carried out specific verification by emulation experiment, as shown in Figure 4.Wherein, (a) is Cones tests The left figure of concentration；(b) it is right figure in Cones test sets；(c) be Middlebury platforms provide with (a) be reference picture True disparity map；(d) it is that the result figure after Meanshift segmentations is carried out to (a)；(e) black box indicates Meanshift points in Cut the first kind cut zone of segmentation errors in result figure；(f) it is a certain the first of the pending secondary splitting extracted from (e) The profile of class cut zone；(g) it is secondary point will obtained after the first kind cut zone progress RANSAC plane fittings in (f) Cut result figure；(h) it is that the triangle that triangulation obtains is carried out to (a)；(i) it is to be obtained based on matching process proposed by the present invention Disparity map；(j) it is the disparity map obtained with the non local method based on minimum spanning tree；(k) it is to be based on LIBELAS The disparity map that (Library for Efficient Large-scale Stereo Matching) algorithm obtains；(1) it is to be based on The disparity map that the partial approach of support window obtains.

In emulation experiment, in order to improve matching efficiency, two are carried out at the same time to reference picture using existing computing resource Road operation carries out Meanshift segmentations to reference picture all the way, and another way directly carries out triangulation to whole picture reference picture, As shown in (h).

Pair based on plane restriction and triangulation based on proposed by the present invention is can be seen that by the simulation result of Fig. 4 Visually feel that the effect for the disparity map that solid matching method obtains will be substantially better than the disparity map obtained based on other three kinds of methods.

In order to further verify inventive embodiments offer based on three-dimensional of the binocular vision of plane restriction and triangulation The advantageous effect of method of completing the square, as shown in table 1, the present invention are directed to the four groups of standardized test charts provided from Middlebury platforms, Specially：Venus, Cones, Tsukuba and Teddy are compared provided by the present application based on plane restriction and triangulation Matching error of the binocular vision solid matching method with other three kinds of Stereo Matching Algorithms when matching each group of standardized test chart Rate.Specifically, for each group of standardized test chart, the matching error rate under three kinds of concrete conditions is compared.Wherein, other three kinds Stereo Matching Algorithm includes：Steerable filter (Guided filter, abbreviation GF) algorithm；Non local average (Non-Local, abbreviation NL) algorithm；LIBELAS algorithms.Three kinds of concrete conditions include：Nonocc：The case where there are occlusion areas is not considered；All：Including The case where full figure all areas；Disc：The case where only considering depth discontinuity zone.

The matching error rate of 1 the method for the present invention of table and other three kinds of Stereo Matching Algorithms

From table 1 it follows that in most cases, pair proposed by the present invention based on plane restriction and triangulation Visually feel that the matching error rate of solid matching method will be less than the matching error rate of other three kinds of Stereo Matching Algorithms.Therefore, originally The Stereo matching effect of inventive method is better than other three kinds of Stereo Matching Algorithms.

Fig. 5 is provided in an embodiment of the present invention a kind of based on the binocular vision Stereo matching of plane restriction and triangulation dress The structure chart set, the device include：

Acquisition module 501, the same shooting acquired respectively in the same time for obtaining the video camera that two are placed in parallel The left image and right image of scene, and using any of which width as image is referred to, another width is as image to be matched；

Determining module 502 for determining the support point in the reference picture, and calculates each parallax for supporting point； Wherein, the pixel for supporting that it is in the reference picture to put, has unique and correct match point in the image to be matched Point；

Divide module 503, for according to preset image segmentation mode, being split, obtaining more to the reference picture A cut zone；Each cut zone is divided into the first kind cut zone that divided area is more than first threshold, and/or is divided Second class cut zone of the face product no more than the first threshold；

First execution module 504 is executed for being directed to each first kind cut zone：To the first kind cut zone into Second of segmentation of row, the sub- cut zone of multiple first kind is divided by the first kind cut zone；According to each first kind point It cuts the support point in region and supports the parallax of point, be fitted the first plane equation of the sub- cut zone of the first kind；According to described First plane equation determines the parallax of non-supporting point in the sub- cut zone of the first kind；Wherein, the multiple first kind segmentation Region corresponds respectively to physical plane different in the photographed scene；

Second execution module 505 is executed for being directed to each second class cut zone：To the second class cut zone into The second class cut zone is divided into multiple delta-shaped regions by row triangulation；For each of in each delta-shaped region It is non-supporting, determine that corresponding first parallax of the non-supporting point is searched according to the parallax on each vertex of the delta-shaped region and each vertex Rope range；The corresponding point to be matched of each parallax within the scope of first disparity search and the non-supporting point are carried out pixel-by-pixel Match, determines the parallax of the non-supporting point；Wherein, each vertex is support point.

Binocular vision Stereo matching device provided in an embodiment of the present invention based on plane restriction and triangulation it is beneficial Effect is identical as the advantageous effect of embodiment of the method.The device is directed to each sub- cut zone of the larger cut zone of area, base In the thought of plane restriction, regarding for all non-supporting points in each sub- cut zone is determined by the plane equation of each sub- cut zone Difference can be realized to the occlusion area of missing pixel information and more accurate in the presence of ambiguous large area texture-free region is matched Really and robustness preferably matches, to reduce matching error rate.And present example, for the smaller segmentation of area Region carries out triangulation, and the disparity search model of non-supporting point in triangle is determined according to vertex of a triangle and vertex parallax It encloses, greatly reduces the disparity search range of non-supporting point, improve matching efficiency.

Optionally, the determining module 502 includes：Grid builds submodule, determination sub-module.

Grid builds submodule, is used for using the top left corner pixel point of the reference picture as starting point, along the reference chart As the u axis and v axis of corresponding pixel coordinate system, the grid that step-length is default step-length is built in the reference picture；It will be described It is in grid, as candidate support point in addition to the intersection point positioned at the reference picture edge；Wherein, u is axial right, under v axial directions.

First determination sub-module, for for each candidate support point, calculating the pre-set candidate and point being supported to correspond to The second disparity search within the scope of the corresponding point to be matched of each parallax vector and candidate support to put it is vectorial at a distance from；If meter Less than in each distance of second threshold in the obtained distance, the number of minimum distance is 1, then the candidate is supported point As support point；Wherein, the vector of the corresponding point to be matched of each parallax and the described time within the scope of second disparity search Choosing supports the vector of point to be calculated all in accordance with the sobel operators of pre-set dimension.

Optionally, the segmentation module 503 is specifically used for according to mean shift Meanshift image segmentation modes, to institute It states reference picture to be split, obtains at least one cut zone.

Optionally, first execution module 504 includes：First fitting submodule, the second fitting submodule, second determine Submodule and third determination sub-module.

First fitting submodule, for according to preset plane fitting rule, being fitted more in the first kind cut zone A plane equation for supporting the corresponding physical plane of idea collection；Wherein, the multiple support point subset corresponds respectively to the bat Take the photograph physical plane different in scene；In each physical plane that fitting obtains, if plurality of physical plane meets in advance If the condition of merging, then the second fitting submodule is triggered；In each physical plane that fitting obtains, if any two of which object The face of patting is unsatisfactory for the default merging condition, then triggers third determination sub-module；Wherein, the default merging condition is：Institute It states in multiple physical planes, any physical plane is less than third threshold value at a distance from least one other physical plane.

Second fitting submodule, for the parallax according to support point and support point in the multiple physical plane, fitting The plane equation of physical plane after being merged triggers the second determination sub-module；

Second determination sub-module, for according to the plane equation of physical plane after merging and the physical plane not being merged Plane equation calculates physical plane after each merging, the intersection between the physical plane that is not merged, determines the multiple first kind Sub- cut zone.

Third determination sub-module, for according to the plane equation of each physical plane, calculating between each physical plane Intersection, determine the sub- cut zone of the multiple first kind.

Optionally, the first fitting submodule includes：Choose subelement, fitting subelement and judgment sub-unit.

Subelement is chosen, for being based on random sampling consistency RANSAC algorithms, working as out of this first kind cut zone Target is chosen in preceding support point set Y supports point subset O；Wherein, the O is in the Y, supports the largest number of support ideas of point Collection.

It is fitted subelement, be used for the support point according to the O and supports the parallax of point, is fitted the corresponding physicals of the O The plane equation in face.

Judgment sub-unit, for whether judging not being fitted currently the support point number for supporting point subset Y-O less than default ratio Support point number in the first kind cut zone of example；If the support point number of the Y-O be no less than preset ratio this Support point number in a kind of cut zone is touched then using the Y-O as the current support point set in the first kind cut zone Hair chooses subelement；If the support point number of the Y-O is less than the support point in the first kind cut zone of preset ratio Number, then using the plane equation for the physical plane being currently fitted as final fitting result.

Optionally, first execution module 504 is specifically used for the first plane equation of the setting sub- cut zone of the first kind For：

D (u, v)=au+bv+c

Optionally, the second execution module 505, including：4th determination sub-module, the 5th determination sub-module.

4th determination sub-module, for being directed to each delta-shaped region, according to each vertex in the delta-shaped region and each The parallax on vertex, the second plane equation of plane where determining the delta-shaped region.

5th determination sub-module, the disparity estimation value for determining the non-supporting point according to second plane equation.

The first disparity search range includes：The parallax on each vertex, the parallax on each vertex adduction is subtracted it is pre- If the disparity estimation value and the default neighbour centered on the non-supporting point that the parallax, the 5th determination sub-module after value determine Target in domain supports the parallax of point；Wherein, the target supports the parallax of point and according to second plane equation determination The target supports the absolute value of the difference of the disparity estimation value of point to be no more than the 4th threshold value.

Optionally, the second execution module 505 is specifically used for, by following formula, determining the non-supporting pointParallax：

Wherein,Indicate d_nPrior probability,Indicate that first parallax is searched The corresponding match point of each parallax within the scope of rope is the non-supporting pointCorrect match point probability.

The embodiment of the present invention additionally provides a kind of electronic equipment, as shown in fig. 6, including processor 601, communication interface 602, Memory 603 and communication bus 604, wherein processor 601, communication interface 602, memory 603 are complete by communication bus 604 At mutual communication,

Memory 603, for storing computer program；

Processor 601 when for executing the program stored on memory 603, realizes following steps：

Electronic equipment provided in an embodiment of the present invention, when processor is by executing the program stored on memory, for Each sub- cut zone of the larger cut zone of area, the thought based on plane restriction pass through the plane side of each sub- cut zone Journey determines the parallax of all non-supporting points in each sub- cut zone, can realize the occlusion area to missing pixel information and deposit Ambiguous large area texture-free region is being matched more precisely and robustness preferably matches, to reduce matching error Rate.And present example carries out triangulation for the smaller cut zone of area, is regarded according to vertex of a triangle and vertex Difference determines the disparity search range of non-supporting point in triangle, greatly reduces the disparity search range of non-supporting point, improves Matching efficiency.

The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral Component Interconnect, abbreviation PCI) bus or expanding the industrial standard structure (Extended Industry Standard Architecture, abbreviation EISA) bus etc..The communication bus can be divided into address bus, data/address bus, controlling bus etc.. For ease of indicating, only indicated with a thick line in figure, it is not intended that an only bus or a type of bus.

Communication interface is for the communication between above-mentioned electronic equipment and other equipment.

Memory may include random access memory (Random Access Memory, abbreviation RAM), can also include Nonvolatile memory (non-volatile memory), for example, at least a magnetic disk storage.Optionally, memory may be used also To be at least one storage device for being located remotely from aforementioned processor.

Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit, Abbreviation CPU), network processing unit (Network Processor, abbreviation NP) etc.；It can also be digital signal processor (Digital Signal Processing, abbreviation DSP), application-specific integrated circuit (Application Specific Integrated Circuit, abbreviation ASIC), field programmable gate array (Field-Programmable Gate Array, Abbreviation FPGA) either other programmable logic device, discrete gate or transistor logic, discrete hardware components.

In another embodiment provided by the invention, a kind of computer readable storage medium is additionally provided, which can It reads to be stored with instruction in storage medium, when run on a computer so that computer executes any in above-described embodiment Binocular vision solid matching method based on plane restriction and triangulation.

The instruction stored in computer readable storage medium provided in an embodiment of the present invention, when it runs on computers When, for each sub- cut zone of the larger cut zone of area, the thought based on plane restriction passes through each sub- cut zone Plane equation determines the parallax of all non-supporting points in each sub- cut zone, can realize the occlusion area to missing pixel information And exist and match ambiguous large area texture-free region more precisely and robustness preferably matches, to reduce With error rate.And present example carries out triangulation for the smaller cut zone of area, according to vertex of a triangle and Vertex parallax determines the disparity search range of non-supporting point in triangle, greatly reduces the disparity search range of non-supporting point, Improve matching efficiency.

In another embodiment provided by the invention, a kind of computer program product including instruction is additionally provided, when it When running on computers so that computer executes any binocular based on plane restriction and triangulation in above-described embodiment Visual stereoscopic matching process.

The computer program product provided in an embodiment of the present invention for including instruction, when run on a computer, for face Each sub- cut zone of the larger cut zone of product, the thought based on plane restriction pass through the plane equation of each sub- cut zone The parallax for determining all non-supporting points in each sub- cut zone, can realize the occlusion area to missing pixel information and presence It matches ambiguous large area texture-free region more precisely and robustness preferably matches, to reduce matching error Rate.And present example carries out triangulation for the smaller cut zone of area, is regarded according to vertex of a triangle and vertex Difference determines the disparity search range of non-supporting point in triangle, greatly reduces the disparity search range of non-supporting point, improves Matching efficiency.

In the above-described embodiments, can come wholly or partly by software, hardware, firmware or its arbitrary combination real It is existing.When implemented in software, it can entirely or partly realize in the form of a computer program product.Computer program product Including one or more computer instructions.When loading on computers and executing computer program instructions, all or part of real estate Raw flow or function according to the embodiment of the present invention.Computer can be all-purpose computer, special purpose computer, computer network, Or other programmable devices.Computer instruction can store in a computer-readable storage medium, or from a computer Readable storage medium storing program for executing to another computer readable storage medium transmit, for example, computer instruction can from a web-site, Computer, server or data center by wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as Infrared, wireless, microwave etc.) mode is transmitted to another web-site, computer, server or data center.Computer Readable storage medium storing program for executing can be that any usable medium that computer can access either includes one or more usable medium collection At the data storage devices such as server, data center.Usable medium can be magnetic medium, (for example, floppy disk, hard disk, magnetic Band), optical medium (for example, DVD) or semiconductor medium (such as solid state disk Solid State Disk (SSD)) etc..

It should be noted that herein, relational terms such as first and second and the like are used merely to a reality Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those Element, but also include other elements that are not explicitly listed, or further include for this process, method, article or equipment Intrinsic element.In the absence of more restrictions, the element limited by sentence " including one ... ", it is not excluded that There is also other identical elements in the process, method, article or equipment including element.

Each embodiment in this specification is all made of relevant mode and describes, identical similar portion between each embodiment Point just to refer each other, and each embodiment focuses on the differences from other embodiments.Especially for device/ For electronic equipment/storage medium/computer program product embodiments, since it is substantially similar to the method embodiment, so retouching That states is fairly simple, and the relevent part can refer to the partial explaination of embodiments of method.

The above is merely preferred embodiments of the present invention, it is not intended to limit the scope of the present invention.It is all in this hair Any modification, equivalent replacement, improvement and so within bright spirit and principle, are included within the scope of protection of the present invention.

Claims

1. a kind of binocular vision solid matching method based on plane restriction and triangulation, which is characterized in that including：

The left image and right image of the same photographed scene that two video cameras being placed in parallel were acquired respectively in the same time are obtained, And using any of which width as image is referred to, another width is as image to be matched；

It determines the support point in the reference picture, and calculates each parallax for supporting point；Wherein, the support point is described Pixel in reference picture, having in the image to be matched unique and correct match point；

According to preset image segmentation mode, the reference picture is split, obtains multiple cut zone；By each described point It is that divided area is more than the first kind cut zone of first threshold and/or divided area is not more than described first to cut region division Second class cut zone of threshold value；

For each first kind cut zone, execute：Second is carried out to the first kind cut zone to divide, by the first kind point It is the sub- cut zone of multiple first kind to cut region division；According to the support point and support point in the sub- cut zone of each first kind Parallax is fitted the first plane equation of the sub- cut zone of the first kind；According to first plane equation, first kind is determined The parallax of non-supporting point in cut zone；Wherein, the sub- cut zone of the multiple first kind corresponds respectively to the photographed scene Middle different physical plane；

For each second class cut zone, execute：Triangulation is carried out to the second class cut zone, which is divided Region division is multiple delta-shaped regions；For non-supporting point each of in each delta-shaped region, according to the delta-shaped region Each vertex and the parallax on each vertex determine the corresponding first disparity search range of the non-supporting point；By first disparity search The corresponding point to be matched of each parallax is matched pixel-by-pixel with the non-supporting point in range, determines the parallax of the non-supporting point；Its In, each vertex is support point.

2. according to the method described in claim 1, it is characterized in that, support point in the determination reference picture, including：

Using the top left corner pixel point of the reference picture as starting point, along the u axis of the corresponding pixel coordinate system of the reference picture With v axis, the grid that step-length is default step-length is built in the reference picture；By it is in the grid, in addition to being located at the ginseng It examines the intersection point at image border and supports point as candidate；Wherein, u is axial right, under v axial directions；

For each candidate support point, calculates the pre-set candidate and support respectively to regard within the scope of corresponding second disparity search of point The vector of the corresponding point to be matched of difference is at a distance from the vector that the candidate supports point；If being less than the in the distance being calculated In each distance of two threshold values, the number of minimum distance is 1, then supports point is used as to support point the candidate；Wherein, described second The vector of the corresponding point to be matched of each parallax and the described candidate support the vector of point all in accordance with default within the scope of disparity search The sobel operators of size are calculated.

3. according to the method described in claim 1, it is characterized in that, described according to preset image segmentation mode, to the ginseng It examines image to be split, obtains multiple cut zone, including：

According to mean shift Meanshift image segmentation modes, the reference picture is split, multiple cut sections are obtained Domain.

4. according to the method described in claim 1, it is characterized in that, the described pair of first kind cut zone carries out second point It cuts, which is divided into the sub- cut zone of multiple first kind, including：

According to preset plane fitting rule, the corresponding physical of multiple support idea collection being fitted in the first kind cut zone The plane equation in face；Wherein, the multiple support point subset corresponds respectively to physical plane different in the photographed scene；

In each physical plane that fitting obtains, if plurality of physical plane, which meets, presets merging condition, according to institute It states the support point in multiple physical planes and supports the parallax of point, the plane equation of physical plane after being merged；According to The plane equation of the plane equation of physical plane and the physical plane not being merged after merging, calculate physical plane after each merging, Intersection between the physical plane not being merged determines the sub- cut zone of the multiple first kind；Wherein, the default merging item Part is：In the multiple physical plane, any physical plane is less than third threshold value at a distance from least one other physical plane；

In each physical plane that fitting obtains, if any two of which physical plane is unsatisfactory for the default merging item Part calculates the intersection between each physical plane then according to the plane equation of each physical plane, determines the multiple the A kind of sub- cut zone.

5. according to the method described in claim 4, it is described according to preset plane fitting rule, be fitted the first kind cut zone The plane equation of the interior corresponding physical plane of multiple support idea collection, including：

S1, random sampling consistency RANSAC algorithms are based on, are chosen from the current support point set Y in the first kind cut zone Target supports point subset O；Wherein, the O is in the Y, supports the largest number of support point subsets of point；

S3, judge that not being fitted the first kind for supporting a support point number of subset Y-O whether to be less than preset ratio currently divides Support point number in region；

It, will if the support point number of the Y-O is no less than the support point number in the first kind cut zone of preset ratio The Y-O returns to step S1 as the current support point set in the first kind cut zone；

If the support point number of the Y-O will be worked as less than the support point number in the first kind cut zone of preset ratio The plane equation of the preceding physical plane being fitted is as final fitting result.

6. according to the method described in claim 1, it is characterized in that, support in the sub- cut zone of each first kind of the basis Point and the parallax for supporting point, are fitted the first plane equation of the sub- cut zone of the first kind, including：

Set the first plane equation of the sub- cut zone of the first kind as：

D (u, v)=au+bv+c

Wherein, d (u, v) indicates that the parallax of pixel (u, v) in the sub- cut zone of the first kind, a, b, c are the first plane side The parameter to be solved of journey solves a, b, c by following formula：

Wherein, u_iIt indicates to support point in the u of the corresponding pixel coordinate system of the reference picture i-th in the sub- cut zone of the first kind The coordinate of axis, v_iIt indicates to support point in the v of the corresponding pixel coordinate system of the reference picture i-th in the sub- cut zone of the first kind The coordinate of axis, u is axial right, and under v axial directions, i=1 ..., m, m indicate the support point number in the sub- cut zone of the first kind, d_iTable Show i-th of parallax for supporting point in the sub- cut zone of the first kind.

7. according to the method described in claim 1, it is characterized in that, described for non-supporting each of in each delta-shaped region Point determines the corresponding first disparity search model of the non-supporting point according to the parallax on each vertex of the delta-shaped region and each vertex It encloses, including：

The triangle is determined according to the parallax on each vertex and each vertex in the delta-shaped region for each delta-shaped region Second plane equation of plane where region；

The first disparity search range includes：The parallax adduction on each vertex is subtracted preset value by the parallax on each vertex The target in default neighborhood belonging to rear parallax, the disparity estimation value and the non-supporting point supports the parallax of point；Wherein, The target supports the parallax of point and supports the difference of the disparity estimation value of point according to the target that second plane equation determines Absolute value be no more than the 4th threshold value.

8. the method according to the description of claim 7 is characterized in that described by each parallax pair within the scope of first disparity search The point to be matched answered is matched pixel-by-pixel with the non-supporting point, determines the parallax of the non-supporting point, including：

By following formula, the non-supporting point is determinedParallax：

Wherein,For the non-supporting pointParallax, d_nIndicate the disparity estimation value of the non-supporting point,It indicates Each parallax within the scope of first disparity search, N indicate the parallax numbers within the scope of first disparity search, described in S is indicated The support point set of reference picture；It indicatesPosterior probability, be calculated by the following formula：

Wherein,Indicate d_nCorresponding point to be matched is the non-supporting pointCorrect match point probability,Indicate that the corresponding point to be matched of each parallax within the scope of first disparity search is the non-branch It holds a littleCorrect match point probability.

9. a kind of binocular vision Stereo matching device based on plane restriction and triangulation, which is characterized in that including：

Acquisition module, a left side for the same photographed scene acquired respectively in the same time for obtaining video camera that two are placed in parallel Image and right image, and using any of which width as image is referred to, another width is as image to be matched；

Determining module for determining the support point in the reference picture, and calculates each parallax for supporting point；Wherein, institute State the pixel for supporting that it is in the reference picture to put, has unique and correct match point in the image to be matched；

Divide module, for according to preset image segmentation mode, being split to the reference picture, obtaining multiple cut sections Domain；By each cut zone be divided into divided area more than first threshold first kind cut zone and/or divided area not More than the second class cut zone of the first threshold；

First execution module is executed for being directed to each first kind cut zone：The first kind cut zone is carried out second Segmentation, the sub- cut zone of multiple first kind is divided by the first kind cut zone；According in the sub- cut zone of each first kind Support point and support point parallax, be fitted the first plane equation of the sub- cut zone of the first kind；According to first plane Equation determines the parallax of non-supporting point in the sub- cut zone of the first kind；Wherein, the sub- cut zone difference of the multiple first kind Corresponding to physical plane different in the photographed scene；

Second execution module is executed for being directed to each second class cut zone：Triangle is carried out to the second class cut zone to cut open Point, which is divided into multiple delta-shaped regions；For non-supporting point each of in each delta-shaped region, The corresponding first disparity search range of the non-supporting point is determined according to the parallax on each vertex of the delta-shaped region and each vertex；It will The corresponding point to be matched of each parallax is matched pixel-by-pixel with the non-supporting point within the scope of first disparity search, determines that this is non- Support the parallax of point；Wherein, each vertex is support point.

10. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein processing Device, communication interface, memory complete mutual communication by communication bus；

Memory, for storing computer program；

Processor when for executing the program stored on memory, realizes any method and steps of claim 1-8.