CN116029996A

CN116029996A - Stereo matching method and device and electronic equipment

Info

Publication number: CN116029996A
Application number: CN202211683316.6A
Authority: CN
Inventors: 陈方平; 高明; 张晓琪; 汤秋嫄
Original assignee: Tianjin Yunsheng Intelligent Technology Co ltd
Current assignee: Tianjin Yunsheng Intelligent Technology Co ltd
Priority date: 2022-12-27
Filing date: 2022-12-27
Publication date: 2023-04-28

Abstract

The invention provides a stereo matching method, a device and electronic equipment, comprising the following steps: acquiring an image pair shot by a binocular camera, and extracting features of the image pair to obtain features of the image pair; performing cost matching calculation according to the characteristics of the image pair to obtain a parallax space image; performing cost aggregation on the parallax space images to obtain an initial parallax image; performing parallax calculation on each pixel according to the initial parallax map to obtain an intermediate parallax map; and performing parallax optimization on the intermediate parallax image to obtain a target parallax image, and further completing stereo matching of the image pair. According to the method, the characteristics of the image pair are automatically extracted in an automatic convolution mode, cost matching calculation, cost aggregation, parallax calculation and parallax optimization are further carried out, the target parallax image is finally obtained, the three-dimensional matching of the image pair is further completed, the processing speed is improved, the real-time performance is good, the illumination change can be better adapted, the method is applicable to a scene lacking texture monotonically, and the calculation is simple.

Description

Stereo matching method and device and electronic equipment

Technical Field

The present invention relates to the technical field of machine learning, and in particular, to a method, an apparatus, and an electronic device for stereo matching.

Background

In recent years, the unmanned aerial vehicle market is rapidly growing, and the obstacle avoidance technology is also changed along with the development of technology as a guarantee technology for increasing the safe flight of the unmanned aerial vehicle. The unmanned aerial vehicle can collect surrounding environment information (such as image data, radar data, GPS data and the like) through sensors carried by the unmanned aerial vehicle in the flight process, and then calculates the collected environment information, so that the distance between each object in the surrounding environment and the aircraft is measured, and the coordinate information of the unmanned aerial vehicle in the environment is used for making corresponding action instructions to achieve the obstacle avoidance effect. In unmanned aerial vehicle obstacle avoidance system, mainly there is: the method comprises 7 main steps of internal and external parameter calibration, data acquisition, data preprocessing, image correction, stereo matching, depth prediction and obstacle avoidance guidance.

Traditional unmanned aerial vehicle keeps away barrier system and adopts binocular vision to carry out the obstacle recognition more. Specifically, a binocular camera is arranged on the unmanned aerial vehicle, two image pairs with overlapped pictures are obtained through the binocular camera, the image pairs are preprocessed and corrected, then three-dimensional matching is carried out, a parallax image is obtained, two-dimensional pixel point coordinates in the overlapped images can be restored to three-dimensional point coordinate information in the real world according to a parallax principle, namely, coordinate information of the unmanned aerial vehicle and all surrounding objects in a world coordinate system (in the real world) is obtained, information between the unmanned aerial vehicle and all surrounding objects can be calculated according to the coordinate information, and accordingly whether obstacle avoidance is needed or not is judged according to the obtained distance between the unmanned aerial vehicle and all surrounding objects. Wherein, stereo matching is an important step that directly affects depth prediction accuracy.

The traditional unmanned aerial vehicle obstacle avoidance system has the advantages that hardware cost is low, indoor and outdoor can be used, but the defect of three-dimensional matching step is also obvious: (1) The camera is very sensitive to ambient light, and the camera exposure is required to be constant according to the binocular vision model principle, so that the capability of adapting to illumination change is weak; (2) The method is not applicable to a monotonically lacking texture scene, and because the binocular vision model needs to perform image matching when processing overlapped image parts, the matching failure is caused by the lack of texture, so that the acquisition of distance information is failed; (3) The calculation complexity is high, and because the binocular vision model is a pure vision method, a large calculation amount is required to ensure the accuracy and the real-time performance of the algorithm, so the calculation force requirement on the unmanned aerial vehicle is extremely high.

In summary, the existing stereo matching method has the technical problems of weak illumination change adaptation capability, inapplicability in a scene lacking texture monotonically and high calculation complexity.

Disclosure of Invention

In view of the above, the present invention aims to provide a stereo matching method, device and electronic equipment, so as to alleviate the technical problems of weak illumination change capability, inapplicability in a scene lacking texture monotonically, and high computation complexity of the existing stereo matching method.

In a first aspect, an embodiment of the present invention provides a stereo matching method, including:

acquiring an image pair shot by a binocular camera, and extracting features of the image pair to obtain the features of the image pair;

performing cost matching calculation according to the characteristics of the image pair to obtain a parallax space image;

performing cost aggregation on the parallax space image to obtain an initial parallax image;

performing parallax calculation on each pixel according to the initial parallax map to obtain an intermediate parallax map;

and performing parallax optimization on the intermediate parallax image to obtain a target parallax image, and further completing the stereoscopic matching of the image pairs.

Further, acquiring an image pair shot by a binocular camera, and extracting features of the image pair, including:

carrying out polar correction on the image pair to obtain an polar corrected image pair;

and extracting the characteristics of the image pair after polar line correction to obtain the characteristics of the image pair.

Further, the method further comprises:

and calculating according to the target parallax map, the baseline distance and the pre-calibrated focal length to obtain the distance between the binocular camera and the obstacle.

Further, performing cost aggregation on the parallax space image includes:

and carrying out cost aggregation on the parallax space image according to the criterion that adjacent pixels have continuous parallax values, and obtaining the initial parallax image.

Further, performing parallax calculation on each pixel according to the initial parallax map, including:

and determining the minimum cost value in cost values of all the parallaxes of the target pixels in the initial parallax map, and taking the parallax corresponding to the minimum cost value as the optimal parallax of the target pixels to further obtain the intermediate parallax map, wherein the target pixels are any one of all the pixels.

Further, performing disparity optimization on the intermediate disparity map includes:

adopting a left-right consistency check algorithm to remove error parallaxes in the intermediate parallaxes to obtain a first optimized parallaxes;

removing isolated abnormal points in the first optimized parallax map by adopting a small connected region removing algorithm to obtain a second optimized parallax map;

and carrying out smoothing processing on the second optimized parallax image by adopting a smoothing algorithm to obtain the target parallax image.

Further, the cost matching calculation method comprises any one of the following steps: and (5) summing the gray absolute value differences and normalizing the correlation coefficient.

In a second aspect, an embodiment of the present invention further provides a stereo matching apparatus, including:

the feature extraction unit is used for obtaining an image pair shot by the binocular camera, and extracting features of the image pair to obtain the features of the image pair;

the cost matching calculation unit is used for carrying out cost matching calculation according to the characteristics of the image pair to obtain a parallax space image;

the cost aggregation unit is used for carrying out cost aggregation on the parallax space images to obtain an initial parallax image;

the parallax calculation unit is used for carrying out parallax calculation on each pixel according to the initial parallax map to obtain an intermediate parallax map;

and the parallax optimization unit is used for carrying out parallax optimization on the intermediate parallax image to obtain a target parallax image, and further completing the stereo matching of the image pairs.

In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the method according to any one of the first aspects when the processor executes the computer program.

In a fourth aspect, embodiments of the present invention also provide a computer-readable storage medium storing machine-executable instructions which, when invoked and executed by a processor, cause the processor to perform the method of any one of the first aspects.

In an embodiment of the present invention, a stereo matching method is provided, including: acquiring an image pair shot by a binocular camera, and extracting features of the image pair to obtain features of the image pair; performing cost matching calculation according to the characteristics of the image pair to obtain a parallax space image; performing cost aggregation on the parallax space images to obtain an initial parallax image; performing parallax calculation on each pixel according to the initial parallax map to obtain an intermediate parallax map; and performing parallax optimization on the intermediate parallax image to obtain a target parallax image, and further completing stereo matching of the image pair. According to the three-dimensional matching method, the characteristics of the image pairs are automatically extracted in an automatic convolution mode, then cost matching calculation, cost aggregation, parallax calculation and parallax optimization are carried out, a target parallax image is finally obtained, three-dimensional matching of the image pairs is further completed, processing speed is improved, instantaneity is good, illumination change can be well adapted, the three-dimensional matching method can be also applied to a scene lacking in texture monotonically, calculation is simple, and the technical problems that the existing three-dimensional matching method is weak in illumination change adaptation capability, inapplicable to the scene lacking in texture monotonically and high in calculation complexity are solved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for stereo matching according to an embodiment of the present invention;

fig. 2 is a schematic diagram of an overall flow of a stereo matching method according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a stereo matching device according to an embodiment of the present invention;

fig. 4 is a schematic diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions of the present invention will be clearly and completely described in connection with the embodiments, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In the traditional method, feature matching points are manually extracted, the robustness is poor, the calculation is complex, the illumination change adaptation capability is weak, and the method is inapplicable in a scene lacking texture monotonically and has high calculation complexity.

Based on the method, the method automatically extracts the characteristics of the image pair by an automatic convolution mode, further performs cost matching calculation, cost aggregation, parallax calculation and parallax optimization, finally obtains the target parallax image, further completes the three-dimensional matching of the image pair, improves the processing speed, has good instantaneity, can better adapt to illumination change, is applicable to a scene lacking texture monotonically, and is simple in calculation.

For the convenience of understanding the present embodiment, a method for stereo matching disclosed in the present embodiment will be described in detail first.

Embodiment one:

in accordance with an embodiment of the present invention, there is provided an embodiment of a method of stereo matching, it being noted that the steps shown in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order other than that shown or described herein.

Fig. 1 is a flowchart of a method of stereo matching according to an embodiment of the present invention, as shown in fig. 1, the method comprising the steps of:

step S102, obtaining an image pair shot by a binocular camera, and extracting features of the image pair to obtain features of the image pair;

in the embodiment of the invention, the step of stereo matching in the unmanned aerial vehicle obstacle avoidance system refers to designating a pixel point in a left image and finding the same name point (namely, a pixel point formed by mapping a three-dimensional point in the real world on left and right camera images) in a right image in an image pair with an overlapping area, wherein the horizontal coordinate difference value of the same name point is parallax (d= |lx-Rx|).

The characteristic extraction is to automatically extract the characteristics of the image pairs by adopting a 3D convolution mode, so that high-efficiency stereo matching can be realized, accuracy and adaptability are improved, calculation speed is improved, and obstacle avoidance of the unmanned aerial vehicle is further efficiently completed.

Step S104, performing cost matching calculation according to the characteristics of the image pair to obtain a parallax space image;

specifically, the purpose of the cost matching calculation is to measure the correlation between the pixels to be matched and the candidate pixels. Whether the two pixels are homonymous points or not, the matching cost can be calculated through the matching cost function, and the smaller the cost is, the larger the correlation is, and the larger the probability of being the homonymous points is.

Before searching for the homonymy point, each pixel usually designates a parallax search range D (Dmin-Dmax), the range is limited in D during parallax search, and a three-dimensional matrix C with the size of w×h×d (W is the image width and H is the image height) is used to store the matching cost value of each pixel under each parallax in the parallax range. The matrix C is commonly referred to as DSI (Disparity Space Image, i.e., parallax spatial image).

There are many methods for calculating the cost matching, in the traditional photogrammetry, gray-scale absolute value difference (AD, absolute Differences) 1, sum of gray-scale absolute value difference (SAD, sum of Absolute Differences), normalized Correlation Coefficient (NCC), and other methods are used to calculate the matching cost of two pixels; in computer vision, mutual information (MI, mutual Information) method 2 3, census Transform (CT) method 4 5, rank Transform (RT) method 6 7, BT (Birchfield and Tomasi) method 8, and the like are often used as calculation methods of matching costs. Different cost calculation algorithms have respective characteristics, the performances of the different data are different, and the selection of a proper matching cost calculation function is a key step which cannot be ignored in three-dimensional matching.

The matching cost is generally determined by calculating the gray value difference of 3 channels of corresponding pixels of the left image and the right image, and is generally calculated based on pixel point matching cost, such as AD, SD and TAD, and is generally calculated based on region matching cost, such as SAD, SSD and STAD. The matching cost calculation generates a disparity space image (i.e., disparity space image), i.e., DSI. The DSI is a three-dimensional space, i.e. each disparity, resulting in a cost map (i.e. a disparity space image). If the parallax range is 0 to 16, 17 cost maps are obtained. The parallax search range is the stereo pair value on the middleBurry website, that is, the matching cost is searched in the parallax range (for example, 0-16), 17 matching cost graphs are obtained, and then the corresponding parallax value with the minimum matching cost is found to be the parallax corresponding to the pixel.

And traversing the left image pixel points in sequence, and generating cost feature matching on the right image through epipolar constraint to construct a cost space, namely a parallax space image.

Step S106, carrying out cost aggregation on the parallax space images to obtain an initial parallax image;

step S108, performing parallax calculation on each pixel according to the initial parallax map to obtain an intermediate parallax map;

step S110, performing parallax optimization on the intermediate parallax image to obtain a target parallax image, and further completing stereo matching of the image pair.

The process from step S106 to step S110 is described in detail below, and will not be described again here.

In an alternative embodiment of the present invention, an image pair captured by a binocular camera is acquired, and feature extraction is performed on the image pair, which specifically includes the following steps:

(1) Carrying out polar correction on the image pair to obtain an image pair after polar correction;

(2) And extracting features of the image pair after polar line correction to obtain features of the image pair.

Specifically, the left and right cameras obtain image pairs, correct (polar correction) the image pairs, extract features of the images, and perform multi-scale feature fusion to obtain features of the image pairs.

In an alternative embodiment of the invention, the method further comprises:

In an optional embodiment of the present invention, cost aggregation is performed on the difference space image, which specifically includes the following steps:

and carrying out cost aggregation on the parallax space images according to the criterion that adjacent pixels have continuous parallax values, and obtaining an initial parallax image.

Specifically, the root purpose of cost aggregation is to enable cost values to accurately reflect correlations between pixels. The cost matching calculation in the last step only considers local information, and calculates the cost value through pixel information in a window with a certain size in the neighborhood of two pixels, which is easily affected by image noise, and when the image is in a weak texture or repeated texture area, the cost value is very likely to not accurately reflect the correlation between pixels, and the direct expression is that the cost value of a real homonymy point is not minimum.

The cost aggregation is to build a link between adjacent pixels, and optimize the cost matrix by a certain criterion, such as that adjacent pixels should have continuous parallax values, where the optimization is often global, and the new cost value of each pixel under a certain parallax is recalculated according to the cost value of its adjacent pixels under the same parallax value or near-parallax value, so as to obtain a new DSI (i.e. an initial parallax map) represented by a matrix S.

In practice, the cost aggregation is similar to a parallax transmission step, the region with high signal-to-noise ratio is good in matching effect, the initial cost can well reflect the correlation, the optimal parallax value can be obtained more accurately, the cost aggregation is transmitted to the region with low signal-to-noise ratio and poor matching effect, and finally, the cost values of all images can accurately reflect the real correlation. Common cost aggregation methods include a scan line method, a dynamic programming method, a path aggregation method in an SGM algorithm, and the like.

By aggregating the costs of the same disparity in the disparity space image to some extent, the effect of erroneous costs is reduced or eliminated, and the window of the same disparity is enlarged and the corresponding costs in the disparity space image (i.e., cost cube) are aggregated together. And pixels of different parallaxes are avoided from being mixed in the polymerization process, so that an initial parallax map is obtained.

In an alternative embodiment of the present invention, the parallax calculation is performed on each pixel according to the initial parallax map, and specifically includes the following steps:

in the initial disparity map, determining the minimum cost value in cost values of all disparities of a target pixel, and taking the disparity corresponding to the minimum cost value as the optimal disparity of the target pixel to obtain an intermediate disparity map, wherein the target pixel is any pixel in all pixels.

Specifically, the parallax calculation is to determine an optimal parallax value of each pixel through a cost matrix S after cost aggregation, and the parallax is generally calculated by using a Winner-to-All algorithm (WTA), that is, a parallax corresponding to a minimum cost value is selected as an optimal parallax from cost values of All parallaxes of a certain pixel. This means that the value of the aggregation cost matrix S must be able to accurately reflect the correlation between pixels, and also indicates that the last cost aggregation step is a very critical step in stereo matching, directly determining the accuracy of the algorithm.

In an alternative embodiment of the present invention, the disparity optimization for the intermediate disparity map specifically includes the following steps:

(1) Adopting a left-right consistency check algorithm to remove error parallaxes in the middle parallaxes to obtain a first optimized parallaxes;

(2) Removing isolated abnormal points in the first optimized parallax map by adopting a small connected region removing algorithm to obtain a second optimized parallax map;

(3) And carrying out smoothing treatment on the second optimized parallax image by adopting a smoothing algorithm to obtain a target parallax image.

Specifically, the purpose of parallax optimization is to further optimize the parallax map obtained in the previous step (i.e. the middle parallax map), improve the quality of the parallax map, and include steps of error parallax elimination, proper smoothing, subpixel accuracy optimization, etc., and generally adopt a Left-Right consistency Check (Left-Right Check) algorithm to eliminate error parallax caused by shielding and noise; adopting a small connected region eliminating algorithm to eliminate isolated abnormal points; smoothing the parallax image by adopting smoothing algorithms such as Median filtering (Median Filter), bilateral filtering (bipolar Filter) and the like; in addition, methods such as robust plane fitting (Robust Plane Fitting), luminance consistency constraint (Intensity Consistent), local consistency constraint (Locally Consistent) and the like are also commonly used to effectively improve the quality of the disparity map.

Because the parallax value obtained by the WTA algorithm is the whole pixel precision, in order to obtain higher sub-pixel precision, further sub-pixel refinement is required for the parallax value, a common sub-pixel refinement method is a unitary quadratic curve fitting method, a unitary quadratic curve is fitted through the cost value under the optimal parallax and the cost values under the left and right parallaxes, and the parallax value represented by the minimum value point of the quadratic curve is taken as the sub-pixel parallax value.

The global optimization idea finds the optimal parallax result of each pixel so that the global and overall matching cost is minimum, and the process is the process of optimizing an energy function, and the function is generally written in the following form: e (d) =e _data (d)+E _smooth (d)。

The algorithm function is added in the unmanned aerial vehicle core system, so that the unmanned aerial vehicle has the omnidirectional obstacle avoidance function, the unmanned aerial vehicle can fly autonomously, and the safety of the unmanned aerial vehicle in the flight process is greatly improved.

The overall flow of the method of stereo matching of the present invention is shown in fig. 2.

The traditional unmanned aerial vehicle obstacle avoidance system has low robustness and poor real-time performance in the process of guiding the unmanned aerial vehicle to independently avoid the obstacle, and has strong dependence on manual control flight. The algorithm can greatly improve the robustness and the instantaneity, and reduce the dependence on manual domination flight.

Embodiment two:

the embodiment of the invention also provides a stereo matching device which is mainly used for executing the stereo matching method provided in the first embodiment of the invention, and the stereo matching device provided in the embodiment of the invention is specifically introduced below.

Fig. 3 is a schematic view of a stereo matching device according to an embodiment of the present invention, and as shown in fig. 3, the device mainly includes: the feature extraction unit 10, the cost matching calculation unit 20, the cost aggregation unit 30, the parallax calculation unit 40 and the parallax optimization unit 50, wherein:

the feature extraction unit is used for acquiring an image pair shot by the binocular camera, and extracting features of the image pair to obtain features of the image pair;

the parallax optimization unit is used for carrying out parallax optimization on the intermediate parallax image to obtain a target parallax image, and further completing stereo matching of the image pair.

In an embodiment of the present invention, there is provided a stereo matching apparatus, including: acquiring an image pair shot by a binocular camera, and extracting features of the image pair to obtain features of the image pair; performing cost matching calculation according to the characteristics of the image pair to obtain a parallax space image; performing cost aggregation on the parallax space images to obtain an initial parallax image; performing parallax calculation on each pixel according to the initial parallax map to obtain an intermediate parallax map; and performing parallax optimization on the intermediate parallax image to obtain a target parallax image, and further completing stereo matching of the image pair. According to the stereoscopic matching device, the characteristics of the image pairs are automatically extracted in an automatic convolution mode, so that cost matching calculation, cost aggregation, parallax calculation and parallax optimization are carried out, a target parallax image is finally obtained, stereoscopic matching of the image pairs is further completed, processing speed is improved, instantaneity is good, illumination change can be well adapted, the stereoscopic matching device can be also adapted to a scene lacking in monotone, calculation is simple, and the technical problems that an existing stereoscopic matching method is weak in illumination change adaptation capability, inapplicable to a scene lacking in monotone and high in calculation complexity are solved.

Optionally, the feature extraction unit is further configured to: carrying out polar correction on the image pair to obtain an image pair after polar correction; and extracting features of the image pair after polar line correction to obtain features of the image pair.

Optionally, the device is further configured to: and calculating according to the target parallax map, the baseline distance and the pre-calibrated focal length to obtain the distance between the binocular camera and the obstacle.

Optionally, the cost aggregation unit is further configured to: and carrying out cost aggregation on the parallax space images according to the criterion that adjacent pixels have continuous parallax values, and obtaining an initial parallax image.

Optionally, the parallax calculating unit is further configured to: in the initial disparity map, determining the minimum cost value in cost values of all disparities of a target pixel, and taking the disparity corresponding to the minimum cost value as the optimal disparity of the target pixel to obtain an intermediate disparity map, wherein the target pixel is any pixel in all pixels.

Optionally, the parallax optimization unit is further configured to: adopting a left-right consistency check algorithm to remove error parallaxes in the middle parallaxes to obtain a first optimized parallaxes; removing isolated abnormal points in the first optimized parallax map by adopting a small connected region removing algorithm to obtain a second optimized parallax map; and carrying out smoothing treatment on the second optimized parallax image by adopting a smoothing algorithm to obtain a target parallax image.

Optionally, the method of cost matching calculation includes any one of the following: and (5) summing the gray absolute value differences and normalizing the correlation coefficient.

The device provided by the embodiment of the present invention has the same implementation principle and technical effects as those of the foregoing method embodiment, and for the sake of brevity, reference may be made to the corresponding content in the foregoing method embodiment where the device embodiment is not mentioned.

As shown in fig. 4, an electronic device 600 provided in an embodiment of the present application includes: the device comprises a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601, the processor 601 and the memory 602 communicate through the bus when the electronic device is running, and the processor 601 executes the machine-readable instructions to perform the steps of the method of stereo matching as described above.

Specifically, the above memory 602 and the processor 601 can be general-purpose memories and processors, and are not particularly limited herein, and the above stereo matching method can be performed when the processor 601 runs a computer program stored in the memory 602.

The processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in the processor 601 or instructions in the form of software. The processor 601 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but may also be a digital signal processor (Digital Signal Processing, DSP for short), application specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA for short), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory 602, and the processor 601 reads information in the memory 602 and performs the steps of the above method in combination with its hardware.

Corresponding to the above method for stereo matching, the embodiments of the present application further provide a computer readable storage medium storing machine executable instructions, which when invoked and executed by a processor, cause the processor to execute the steps of the above method for stereo matching.

The stereo matching device provided by the embodiment of the application can be specific hardware on equipment or software or firmware installed on the equipment. The device provided in the embodiments of the present application has the same implementation principle and technical effects as those of the foregoing method embodiments, and for a brief description, reference may be made to corresponding matters in the foregoing method embodiments where the device embodiment section is not mentioned. It will be clear to those skilled in the art that, for convenience and brevity, the specific operation of the system, apparatus and unit described above may refer to the corresponding process in the above method embodiment, which is not described in detail herein.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.

As another example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments provided in the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the vehicle marking method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

It should be noted that: like reference numerals and letters in the following figures denote like items, and thus once an item is defined in one figure, no further definition or explanation of it is required in the following figures, and furthermore, the terms "first," "second," "third," etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.

Finally, it should be noted that: the foregoing examples are merely specific embodiments of the present application, and are not intended to limit the scope of the present application, but the present application is not limited thereto, and those skilled in the art will appreciate that while the foregoing examples are described in detail, the present application is not limited thereto. Any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or make equivalent substitutions for some of the technical features within the technical scope of the disclosure of the present application; such modifications, changes or substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application. Are intended to be encompassed within the scope of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of stereo matching comprising:

2. The method of claim 1, wherein capturing an image pair captured by a binocular camera and performing feature extraction on the image pair comprises:

3. The method according to claim 1, wherein the method further comprises:

4. The method of claim 1, wherein cost aggregating the disparity space image comprises:

5. The method of claim 1, wherein performing a disparity calculation for each pixel from the initial disparity map comprises:

6. The method of claim 1, wherein disparity optimizing the intermediate disparity map comprises:

7. The method of claim 1, wherein the method of cost matching calculation comprises any one of: and (5) summing the gray absolute value differences and normalizing the correlation coefficient.

8. A stereo matching device, comprising:

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any of the preceding claims 1 to 7 when the computer program is executed.

10. A computer readable storage medium storing machine executable instructions which, when invoked and executed by a processor, cause the processor to perform the method of any one of the preceding claims 1 to 7.