CN117474922B - Anti-noise light field depth measurement method and system based on inline shielding processing - Google Patents
Anti-noise light field depth measurement method and system based on inline shielding processing Download PDFInfo
- Publication number
- CN117474922B CN117474922B CN202311821225.9A CN202311821225A CN117474922B CN 117474922 B CN117474922 B CN 117474922B CN 202311821225 A CN202311821225 A CN 202311821225A CN 117474922 B CN117474922 B CN 117474922B
- Authority
- CN
- China
- Prior art keywords
- light field
- occlusion
- shielding
- noise
- angle domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000691 measurement method Methods 0.000 title claims abstract description 24
- 238000012545 processing Methods 0.000 title claims description 16
- 238000000034 method Methods 0.000 claims abstract description 60
- 230000008569 process Effects 0.000 claims abstract description 23
- 230000008447 perception Effects 0.000 claims abstract description 15
- 238000001914 filtration Methods 0.000 claims abstract description 14
- 238000005457 optimization Methods 0.000 claims abstract description 14
- 239000011159 matrix material Substances 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 15
- 230000003044 adaptive effect Effects 0.000 claims description 10
- 238000005259 measurement Methods 0.000 claims description 9
- 230000035945 sensitivity Effects 0.000 claims description 6
- 230000000903 blocking effect Effects 0.000 claims description 5
- 230000001788 irregular Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 238000013139 quantization Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims 1
- 238000001514 detection method Methods 0.000 abstract description 8
- 238000003384 imaging method Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010008 shearing Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Geometry (AREA)
- Image Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention relates to the technical field of light field imaging, in particular to an anti-noise light field depth measurement method and system based on an inline shielding process, wherein the method comprises the following steps: acquiring an original image of a light field; by usingThe light field is parameterized so that,in the case of an angular position of the pins,for spatial coordinates, input light fieldBased on candidate depth labelsRemapping to a shear light fieldThe method comprises the steps of carrying out a first treatment on the surface of the Judging the shielding type according to the sheared light field; if the shielding is linear shielding, constructing a partial angle domain cost amount; if the shielding is block shielding, constructing a self-adaptive angle domain cost amount; solving the cost quantity according to the cost minimum criterionIs defined in the original depth map of (a); and adopting various filtering strategies to perform noise perception optimization on the initial depth map and obtaining a final depth map. Therefore, the occlusion map obtained based on the scene three-dimensional model is used for occlusion detection, so that the accuracy of occlusion detection is improved; the adaptability of the algorithm to noise and shielding is improved, and the accuracy of light field depth estimation is improved.
Description
Technical Field
The invention relates to the technical field of light field imaging, and particularly provides an anti-noise light field depth measurement method and system based on an inline shielding process.
Background
In the past decade, light field imaging technology has evolved from a popular topic to an active part of the computer vision field. But most conventional approaches may exhibit poor performance at occlusion pixels when occlusion is present, resulting in excessive smoothing and outliers of the depth map near the occlusion boundary. This directly affects the accuracy of the depth inversion, creating ambiguous shapes in the depth map, and presents a significant challenge to three-dimensional reconstruction, three-dimensional scene understanding.
At present, a traditional light field depth estimation algorithm aiming at shielding is based on a 2D model of a scene in shielding treatment. But occlusion is determined by a 3D model of the scene, and 2D models do not guarantee the accuracy of occlusion detection.
However, on the other hand, due to noise, the important precondition imaging consistency of depth estimation is not strictly satisfied, and thus the depth estimation accuracy is degraded. Particularly, in a noisy scene with complex occlusion, the occlusion detection difficulty is increased by noise, and the quality of depth estimation is further reduced by incorrect occlusion judgment.
Disclosure of Invention
The invention provides an anti-noise light field depth measurement method and system based on an inline occlusion process for solving the problems, wherein an occlusion map directly obtained based on a scene three-dimensional model is used for occlusion detection to improve the accuracy of occlusion detection; on the other hand, the adaptability of the algorithm to noise and shielding is improved, so that the accuracy of light field depth estimation is improved.
The invention provides an anti-noise light field depth measurement method based on an inline occlusion process, which comprises the following steps:
s1, acquiring an original image of a light field;
s2, carrying out the light field original imageLight field Parametrization (PYRIUM)>For angular coordinates +.>For spatial coordinates, input light field +.>Is based on the candidate depth label +.>Remapping to shear light field->；
S3, judging the shielding type according to the sheared light field;
s4, if the shielding is linear shielding, constructing a partial angle domain cost amount; if the occlusion is a block occlusion, constructing a self-adaptive angle domain cost amount;
s5, according to a cost minimum criterion, solving an initial depth map of a cost quantity, wherein the cost quantity comprises the cost quantity of the partial angle domain or the cost quantity of the self-adaptive angle domain;
s6, adopting various filtering strategies to perform noise perception optimization on the initial depth map, and obtaining a final depth map.
Preferably, the shear light fieldSpecifically, the shearing process of (2) comprises:
（1）
in the method, in the process of the invention,（2）
（3）。
preferably, the judging the occlusion type includes:
only one object is used for shielding the background, and the shielding boundary is approximately a linear shielding phenomenon, and judging that the linear shielding is performed;
and judging that the block is blocked if a plurality of objects block the background or an object has an irregular blocking boundary.
Preferably, after S3, the method further includes:
establishing varianceAs a corresponding quantization evaluation index, the calculation formula is as follows:
（4）；
wherein the method comprises the steps ofRepresenting the pixel mean of all angle domains,/->Is->And->The number of samples in the direction;
according to the nature of the light field center view, when the angle coordinatesThe shift of the center view is not accompanied by +.>Is changed by a change of (a) and is always kept at 0, thus, each of the depth tags +.>Is a central view of (2)The formula is as follows:
（5）；
using the center viewInstead of the angular domain pixel mean +.>Equation (4) turns into the following form:
（6）；
to simplify the shear transformation of the light field, refocusing Jiao Gongshi (1) is rewritten as:
（7）
wherein the method comprises the steps ofIs parallax, i.e. the offset of the sub-aperture image;
preferably, if the occlusion is a linear occlusion, constructing the partial angle domain cost amount includes:
obtaining required angle domain pixels by using linear mask, and performing correlation measurement on the angle domain pixels of all the space points in different directionsThe formula of (2) is as follows:
（8）；
wherein,is a 3D binary mask matrix, < >>Is a subscript of the mask matrix, representing different types of mask matrix,/for each mask matrix>Controlling the number of layers in the third dimension of the matrix;
to further control the robustness to noise, a gaussian-like function is used to derive a cost function for the partial angle domainThe formula of (2) is as follows:
（9）；
wherein the method comprises the steps ofThe sensitivity to noise is controlled.
Preferably, if the occlusion is a block occlusion, constructing the adaptive angle domain cost amount includes:
using a block mask to obtain the desired angleA domain pixel; correlation metrics for angle domain pixels of all spatial points over different sub-regionsThe formula of (c) is as follows,
（11）；
wherein,is a 3D binary mask matrix used for defining the angle domain image pixels of a sub-region to participate in operation; />Is a subscript of the mask matrix, representing different types of mask matrix,/for each mask matrix>And controlling the number of the angle domain images to be divided into the subareas.
Calculating correlation coefficients of pixels in different angle domains of each block maskTo realize->Weight fusion of the sub-region correlation coefficients using a weighting function in gaussian form>The formula is as follows:
（12）；
（13）；
wherein the method comprises the steps ofThe magnitude of the weight coefficient is controlled, and the larger the magnitude of the weight coefficient is, the higher the robustness to noise is, and the +.>Is a threshold value for the weight of each of said sub-regions;
will be according to the weight functionFusing the subareas:
（14）；
normalizing cost function of adaptive angle domain using gaussian-like functionThe formula of (2) is as follows:
（15）；
wherein the method comprises the steps ofThe sensitivity to noise is controlled.
Preferably, the step S5 includes:
the angle domain pixel of the object point has a global minimum value at the true value depth;
i.e. initial depth mapThe formula of (2) is as follows:
（10）。
preferably, the step S6 includes:
and performing noise perception optimization on the initial depth map by adopting two or more of edge-preserving filter (EPF), graph Cut (GC), weighted median filtering (WMT), weight Mode Filter (WMF) and noise-aware filter (NAF) strategies so as to improve the accuracy of the depth map and obtain a final depth map.
An anti-noise light field depth measurement system based on inline occlusion processing, comprising:
a first acquisition module: the method comprises the steps of acquiring an original image of a light field;
refocusing Jiao Mokuai: performing the light field original imageLight field Parametrization (PYRIUM)>In the case of an angular position of the pins,for spatial coordinates, input light field +.>Is based on the candidate depth label +.>Remapping to shear light field->；
And a judging module: the method is used for judging the shielding type according to the sheared light field;
and (3) constructing a module: if the shielding is linear shielding, constructing a partial angle domain cost amount; if the occlusion is a block occlusion, constructing a self-adaptive angle domain cost amount;
and a solving module: an initial depth map for solving a cost amount according to a cost minimization criterion, the cost amount comprising the partial angle domain cost amount or the adaptive angle domain cost amount;
and a second acquisition module: and the method is used for carrying out noise perception optimization on the initial depth map by adopting various filtering strategies to obtain a final depth map.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides an anti-noise light field depth measurement method and system based on an inline occlusion process, wherein the anti-noise light field depth measurement method based on the inline occlusion process comprises the following steps: s1, acquiring an original image of a light field; s2, adoptingLight field Parametrization (PYRIUM)>For angular coordinates +.>For spatial coordinates, input light field +.>Is based on the candidate depth label +.>Remapping to shear light field->The method comprises the steps of carrying out a first treatment on the surface of the S3, judging the shielding type according to the sheared light field; s4, if the shielding is linear shielding, constructing a partial angle domain cost amount; if the occlusion is a block occlusion, constructing a self-adaptive angle domain cost amount; s5, according to a cost minimum criterion, solving an initial depth map of a cost quantity, wherein the cost quantity comprises the cost quantity of the partial angle domain or the cost quantity of the self-adaptive angle domain; s6, adopting various filtering strategies to perform noise perception optimization on the initial depth map, and obtaining a final depth map. Therefore, on one hand, the occlusion map obtained directly based on the scene stereoscopic model is used for occlusion detection, so that the accuracy of occlusion detection is improved; on the other hand, the adaptability of the algorithm to noise and shielding is improved, and the accuracy of light field depth estimation is improved.
Drawings
FIG. 1 is a method flow chart of an anti-noise light field depth measurement method based on an inline occlusion process provided in accordance with embodiment 1 of the present invention;
FIG. 2 is a light field parameterized graph of an anti-noise light field depth measurement method based on inline occlusion processing provided in accordance with embodiment 1 of the present invention;
FIG. 3 is a binary mask matrix of a partial angular domain cost amount of an anti-noise light field depth measurement method based on an inline occlusion process according to embodiment 1 of the present invention;
FIG. 4 is a binary mask matrix of the cost amount of the adaptive angular domain of the anti-noise light field depth measurement method based on the inline occlusion process provided in accordance with embodiment 1 of the present invention;
FIG. 5 is a block diagram of an anti-noise light field depth measurement system based on inline occlusion processing, provided in accordance with embodiment 2 of the present invention;
FIG. 6 (a) is an exemplary diagram of a judging occlusion type of an anti-noise light field depth measurement method based on an inline occlusion process according to embodiment 2 of the present invention;
FIG. 6 (b) is an exemplary diagram of a judging occlusion type of an anti-noise light field depth measurement method based on an inline occlusion process according to embodiment 2 of the present invention;
fig. 6 (c) is a diagram showing an example of a judgment occlusion type of the anti-noise light field depth measurement method based on the inline occlusion process according to embodiment 2 of the present invention.
Wherein reference numerals include:
100-an anti-noise light field depth measurement system based on inline occlusion processing;
10-a first acquisition module; 20-refocusing Jiao Mokuai; 30-judging module; 40-constructing a module; 50-a solving module; 60-a second acquisition module.
Detailed Description
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, like modules are denoted by like reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, a detailed description thereof will not be repeated.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not to be construed as limiting the invention.
Example 1
An anti-noise light field depth measurement method (shown in fig. 1) based on an inline occlusion process, comprising: s1, acquiring an original image of a light field; s2, carrying out the light field original imageLight field parametrization (as shown in fig. 2), +.>For angular coordinates +.>For spatial coordinates, input light field +.>Is based on the candidate depth label +.>Remapping to shear light field->The method comprises the steps of carrying out a first treatment on the surface of the S3, judging the shielding type according to the sheared light field; s4, if the shielding is linear shielding, constructing a partial angle domain cost amount; if the occlusion is a block occlusion, constructing a self-adaptive angle domain cost amount; s5, according to a cost minimum criterion, solving an initial depth map of a cost quantity, wherein the cost quantity comprises the cost quantity of the partial angle domain or the cost quantity of the self-adaptive angle domain; s6, adopting various filtering strategies to perform noise perception optimization on the initial depth map, and obtaining a final depth map.
The above-mentioned means to propose a light field depth estimation algorithm with robust occlusion for noise scene from thick to thin and from local to global. The method comprises the steps of obtaining an original image of a light field, obtaining a refocused image of the light field by light field parameterization, defining the occlusion by a proposed light field depth estimation framework as linear occlusion and block occlusion, and respectively constructing a partial angular domain cost amount and a self-adaptive angular domain cost amount. And constructing cost quantities aiming at different shielding types by adopting an inline shielding processing frame, and solving an initial depth map according to a cost minimum criterion. In order to further filter noise, a plurality of filtering strategies are adopted to conduct shielding perception optimization on the initial cost quantity, and a final depth map is obtained.
Therefore, the anti-noise light field depth measurement method based on the inline occlusion processing of the embodiment can perform different anti-occlusion processing according to the occlusion classification of the scene under the condition that prior information such as occlusion edges, directions and the like is not known. An advantage of applying a 3D mask (stereoscopic model of the scene) is that the resulting occlusion map will be closer to the real occlusion pattern. And the cost quantity is subjected to noise perception optimization by utilizing various filtering strategies, so that the depth estimation accuracy of the real light field scene is improved.
Further, in the present embodiment, different occlusion types are defined and corresponding cost amounts are constructed; adopting an inline shielding processing frame, adaptively selecting the optimal cost amount, and obtaining a relatively accurate local depth map; and selecting the optimal optimization method of the depth map for different noisy scenes, further obtaining a final depth map, and improving the accuracy of light field depth estimation.
In this embodiment, the shear light fieldSpecifically, the shearing process of (2) comprises:
（1）
in the method, in the process of the invention,（2）
（3）。
in this embodiment, the judging the occlusion type includes:
only one object is used for shielding the background, and the shielding boundary is approximately a linear shielding phenomenon, and judging that the linear shielding is performed;
and judging that the block is blocked if a plurality of objects block the background or an object has an irregular blocking boundary.
In the above-mentioned embodiment, as shown in fig. 6 (a) -6 (c), it is worth further explanation that the image belongs to no occlusion when the image meets the imaging consistency principle. The occlusion border of the red box area of fig. 6 (a) is approximately a straight line occlusion and there is only one occlusion, where the antenna of the butterfly belongs to a line occlusion; the occlusion boundary of the blue box area of fig. 6 (a) is irregular, in which the leaves belong to the block occlusion. The red and blue box areas of fig. 6 (b) are both of the line-type occlusion type. The red frame area of fig. 6 (c), the barrier is of the line type of occlusion; the occlusion border of the blue box area of fig. 6 (c) is made up of a combination of straight lines and curves, and there are multiple occlusions, the fence and tree belonging to the block occlusion.
In this embodiment, after the step S3, the method further includes:
establishing varianceAs a corresponding quantization evaluation index, the calculation formula is as follows:
（4）；
wherein the method comprises the steps ofRepresenting the pixel mean of all angle domains,/->Is->And->The number of samples in the direction;
the measurement using the above equation has two main problems. First of all,including light from non-imaging targets; second, go up>Often tending to occupy more of the partial pixel values. If the blocked light exceeds half of the whole, the depth value obtained by equation (4) is more prone to the depth of the blocking object, and a depth matching error is introduced.
According to the nature of the light field center view, when the angle coordinatesThe shift of the center view is not accompanied by +.>Is changed by a change of (a) and is always kept at 0, thus, each of the depth tags +.>Is a central view of (2)The formula is as follows:
（5）；
here, when the angle coordinatesThe shift of the center view is not accompanied by +.>Is always kept at 0, in other words, the center pixel of all angle domain images comes from the target object point, regardless of the depth of the object point.
Using the center viewInstead of the angular domain pixel mean +.>Equation (4) turns into the following form:
（6）；
above, the center viewInstead of the angular domain pixel mean +.>The beneficial effects are thatThe pixels of the pixel(s) come from the light rays which are not blocked by the object point, so that the blocking treatment effect can be improved to a certain extent.
To simplify the shear transformation of the light field, refocusing Jiao Gongshi (1) is rewritten as:
（7）；
wherein the method comprises the steps ofIs parallax, i.e. the offset of the sub-aperture image;
it should be further noted that, the four-dimensional light field data is in units of pixels, and thus, the light field is also in units of pixels during light field remapping.
In this embodiment, if the occlusion is a linear occlusion, the constructing the partial angle domain cost amount includes:
obtaining required angle domain pixels by using linear mask, and performing correlation measurement on the angle domain pixels of all the space points in different directionsThe formula of (2) is as follows:
（8）；
wherein,is a 3D binary mask matrix, < >>Is a subscript of the mask matrix, representing different types of mask matrix,/for each mask matrix>The number of layers in the third dimension of the matrix is controlled.
After obtaining the light field images at different depths, the linear mask is used to obtain the required angular domain pixels. In a specific embodiment, equation (8) may beFor example, six line masks:the specific form of (a) is shown in figure 3 (0 is not marked in the matrix).
To further control the robustness to noise, a gaussian-like function is used to derive a cost function for the partial angle domainThe formula of (2) is as follows:
（9）
wherein the method comprises the steps ofThe sensitivity to noise is controlled.
In this embodiment, if the occlusion is a block occlusion, constructing the adaptive angle domain cost amount includes:
using a block mask to obtain the required angle domain pixels; correlation metrics for angle domain pixels of all spatial points over different sub-regionsThe formula of (c) is as follows,
（11）；
wherein,is a 3D binary mask matrix used for defining the angle domain image pixels of a sub-region to participate in operation; />Is a subscript of the mask matrix, representing different types of mask matrix,/for each mask matrix>And controlling the number of the angle domain images to be divided into the subareas.
After the light field images at different depths are obtained, the block mask is used to obtain the required angular domain pixels. In one specific embodiment of the present invention,and (4) by->For example, six block masks:the specific form of (a) is shown in fig. 4 (0 is not marked in the matrix).
Calculating correlation coefficients of pixels in different angle domains of each block maskTo realize->Weight fusion of the sub-region correlation coefficients using a weighting function in gaussian form>The formula is as follows:
（12）
（13）
wherein the method comprises the steps ofThe magnitude of the weight coefficient is controlled, and the larger the magnitude of the weight coefficient is, the higher the robustness to noise is, and the +.>Is a threshold value for the weight of each of said sub-regions;
in the above-mentioned manner,the larger the weight coefficient is, the higher the robustness to noise is, but the processing effect on shielding is reduced. According to different scenes +.>Is a value of (2). For example, in areas with less obstruction or more noise, a larger +.>The method comprises the steps of carrying out a first treatment on the surface of the While in areas with more shielding or less noise, smaller +.>Finer results are obtained. />Is a threshold value of the weight of each sub-region, and the weight function of the sub-region with a particularly large difference in correlation coefficient is set to 0.
Will be according to the weight functionFusing the subareas:
（14）；
normalizing cost function of adaptive angle domain using gaussian-like functionThe formula of (2) is as follows:
（15）；
wherein the method comprises the steps ofThe sensitivity to noise is controlled.
In this embodiment, the S5 includes:
the angle domain pixel of the object point has a global minimum value at the true value depth;
i.e. initial depth mapThe formula of (2) is as follows:
（10）。
the above means that the angle domain pixel of the object point will exhibit the highest correlation along the optimal direction at its true depth, i.e. the cost function has a global minimum at the correct depth.
Preferably, the step S6 includes:
and performing noise perception optimization on the initial depth map by adopting two or more strategies such as edge-preserving filter (EPF), graph cuts (GC, graph cutting), weighted median filtering (WMT, weighted median filtering), weight mode filter (WMF, weight mode filter), noise-aware filter (NAF, noise perception filter) and the like so as to improve the accuracy of the depth map and obtain a final depth map.
Example two
An anti-noise light field depth measurement system based on an inline occlusion process, as shown in FIG. 5, comprising:
the first acquisition module 10: the method comprises the steps of acquiring an original image of a light field;
refocusing module 20: performing the light field original imageLight field Parametrization (PYRIUM)>For angular coordinates +.>For spatial coordinates, input light field +.>Is based on the candidate depth label +.>Remapping to shear light field->；
The judgment module 30: the method is used for judging the shielding type according to the sheared light field;
construction module 40: if the shielding is linear shielding, constructing a partial angle domain cost amount; if the occlusion is a block occlusion, constructing a self-adaptive angle domain cost amount;
the solving module 50: an initial depth map for solving a cost amount according to a cost minimization criterion, the cost amount comprising the partial angle domain cost amount or the adaptive angle domain cost amount;
the second acquisition module 60: and the method is used for carrying out noise perception optimization on the initial depth map by adopting various filtering strategies to obtain a final depth map.
While embodiments of the present invention have been illustrated and described above, it will be appreciated that the above described embodiments are illustrative and should not be construed as limiting the invention. Variations, modifications, alternatives and variations of the above-described embodiments may be made by those of ordinary skill in the art within the scope of the present invention.
The above embodiments of the present invention do not limit the scope of the present invention. Any other corresponding changes and modifications made in accordance with the technical idea of the present invention shall be included in the scope of the claims of the present invention.
Claims (7)
1. An anti-noise light field depth measurement method based on an inline occlusion process, comprising the steps of:
s1, acquiring an original image of a light field;
s2, carrying out the light field original imageLight field Parametrization (PYRIUM)>For angular coordinates +.>For spatial coordinates, input light field +.>Is based on the candidate depth label +.>Remapping to a shear light field；
S3, judging the shielding type according to the sheared light field;
s4, if the shielding is linear shielding, constructing a partial angle domain cost amount; if the occlusion is a block occlusion, constructing a self-adaptive angle domain cost amount;
s5, according to a cost minimum criterion, solving an initial depth map of a cost quantity, wherein the cost quantity comprises the cost quantity of the partial angle domain or the cost quantity of the self-adaptive angle domain;
s6, adopting various filtering strategies to perform noise perception optimization on the initial depth map, and obtaining a final depth map;
in the step S4, if the occlusion is a line occlusion, the constructing a partial angle domain cost amount includes:
obtaining required angle domain pixels by using linear mask, and performing correlation measurement on the angle domain pixels of all the space points in different directionsThe formula of (2) is as follows:
（8）；
wherein,is a 3D binary mask matrix, < >>Is a subscript of the mask matrix, representing different types of mask matrix,/for each mask matrix>Controlling the number of layers in the third dimension of the matrix;
to further control the robustness to noise, a gaussian-like function is used to derive a cost function for the partial angle domainThe formula of (2) is as follows:
（9）；
wherein the method comprises the steps ofControlling sensitivity to noise;
in the step S4, if the occlusion is a block occlusion, the constructing the adaptive angle domain cost amount includes:
using a block mask to obtain the required angle domain pixels; correlation metrics for angle domain pixels of all spatial points over different sub-regionsThe formula of (c) is as follows,
（11）；
wherein,is a 3D binary mask matrix used for defining the angle domain image pixels of a sub-region to participate in operation; />Is a subscript of the mask matrix, representing different types of mask matrix,/for each mask matrix>Controlling the number of the angle domain images divided into the subareas;
calculating correlation coefficients of pixels in different angle domains of each block maskTo realize->Weight fusion of the sub-region correlation coefficients using a weighting function in gaussian form>The formula is as follows:
（12）；
（13）；
wherein the method comprises the steps ofThe magnitude of the weight coefficient is controlled, and the larger the magnitude of the weight coefficient is, the higher the robustness to noise is, and the +.>Is a threshold value for the weight of each of said sub-regions;
will be according to the weight functionAnd fusing the sub-regions:
（14）；
normalizing cost function of adaptive angle domain using gaussian-like functionThe formula of (2) is as follows:
（15）；
wherein the method comprises the steps ofThe sensitivity to noise is controlled.
2. The anti-noise light field depth measurement method based on inline occlusion processing of claim 1, wherein in the S2 the shear light fieldSpecifically, the mapping process of (1) comprises:
（1）
in the method, in the process of the invention,（2）
（3）。
3. the anti-noise light field depth measurement method based on inline occlusion processing of claim 2, wherein in said S3, said determining an occlusion type includes:
only one object is used for shielding the background, and the shielding boundary is approximately a linear shielding phenomenon, and judging that the linear shielding is performed;
and judging that the block is blocked if a plurality of objects block the background or an object has an irregular blocking boundary.
4. The anti-noise light field depth measurement method based on inline occlusion processing of claim 3, further comprising, after said S3:
establishing varianceAs a corresponding quantization evaluation index, the calculation formula is as follows:
（4）；
wherein the method comprises the steps ofRepresenting the pixel mean of all angle domains,/->Is->And->The number of samples in the direction;
according to the nature of the light field center view, when the angle coordinatesThe shift of the center view is not accompanied by +.>Is changed by a change of (a) and is always kept at 0, thus, each of the depth tags +.>Is a central view of (2)The formula is as follows:
（5）；
using the center viewInstead of the angular domain pixel mean +.>Equation (4) turns into the following form:
（6）；
to simplify the shear transformation of the light field, refocusing Jiao Gongshi (1) is rewritten as:
（7）；
wherein the method comprises the steps ofIs parallax, i.e. the offset of the sub-aperture image.
5. The anti-noise light field depth measurement method based on inline occlusion processing of claim 4, wherein S5 comprises:
the angle domain pixel of the object point has a global minimum value at its true value depth;
i.e. initial depth mapThe formula of (2) is as follows:
（10）。
6. the anti-noise light field depth measurement method based on inline occlusion processing of claim 1, wherein S6 includes:
and performing noise perception optimization on the initial depth map by adopting two or more of a guard filter, a graph cutting, a weighted median filtering, a weight mode filter and a noise perception filter strategy so as to improve the accuracy of the depth map and obtain a final depth map.
7. An anti-noise light field depth measurement system based on an inline occlusion process using the anti-noise light field depth measurement method based on an inline occlusion process of claim 1, comprising:
a first acquisition module: the method comprises the steps of acquiring an original image of a light field;
refocusing Jiao Mokuai: performing the light field original imageLight field Parametrization (PYRIUM)>In the case of an angular position of the pins,for spatial coordinates, input light field +.>Is based on the candidate depth label +.>Remapping to shear light field->；
And a judging module: the method is used for judging the shielding type according to the sheared light field;
and (3) constructing a module: if the shielding is linear shielding, constructing a partial angle domain cost amount; if the occlusion is a block occlusion, constructing a self-adaptive angle domain cost amount;
and a solving module: an initial depth map for solving a cost amount according to a cost minimization criterion, the cost amount comprising the partial angle domain cost amount or the adaptive angle domain cost amount;
and a second acquisition module: and the method is used for carrying out noise perception optimization on the initial depth map by adopting various filtering strategies to obtain a final depth map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311821225.9A CN117474922B (en) | 2023-12-27 | 2023-12-27 | Anti-noise light field depth measurement method and system based on inline shielding processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311821225.9A CN117474922B (en) | 2023-12-27 | 2023-12-27 | Anti-noise light field depth measurement method and system based on inline shielding processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117474922A CN117474922A (en) | 2024-01-30 |
| CN117474922B true CN117474922B (en) | 2024-04-02 |
Family
ID=89627816
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311821225.9A Active CN117474922B (en) | 2023-12-27 | 2023-12-27 | Anti-noise light field depth measurement method and system based on inline shielding processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117474922B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107038719A (en) * | 2017-03-22 | 2017-08-11 | 清华大学深圳研究生院 | Depth estimation method and system based on light field image angle domain pixel |
| CN108154528A (en) * | 2017-12-05 | 2018-06-12 | 西北工业大学 | Based on broad sense to the light field depth estimation method of polar plane figure |
| CN111325763A (en) * | 2020-02-07 | 2020-06-23 | 清华大学深圳国际研究生院 | Occlusion prediction method and device based on light field refocusing |
| CN114511609A (en) * | 2022-04-18 | 2022-05-17 | 清华大学 | Unsupervised light field parallax estimation system and method based on occlusion perception |
| CN114897952A (en) * | 2022-05-30 | 2022-08-12 | 中国测绘科学研究院 | Method and system for estimating accurate depth of single light field image in self-adaptive shielding manner |
| CN116029919A (en) * | 2022-12-09 | 2023-04-28 | 中国科学院微电子研究所 | Intelligent optimization method for compressed three-dimensional light field display |
| CN116402908A (en) * | 2023-03-23 | 2023-07-07 | 宁波大学 | Dense light field image reconstruction method based on heterogeneous imaging |
| CN116485868A (en) * | 2022-01-11 | 2023-07-25 | 香港城市大学深圳研究院 | Four-dimensional light field image depth acquisition method and device |
-
2023
- 2023-12-27 CN CN202311821225.9A patent/CN117474922B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107038719A (en) * | 2017-03-22 | 2017-08-11 | 清华大学深圳研究生院 | Depth estimation method and system based on light field image angle domain pixel |
| CN108154528A (en) * | 2017-12-05 | 2018-06-12 | 西北工业大学 | Based on broad sense to the light field depth estimation method of polar plane figure |
| CN111325763A (en) * | 2020-02-07 | 2020-06-23 | 清华大学深圳国际研究生院 | Occlusion prediction method and device based on light field refocusing |
| CN116485868A (en) * | 2022-01-11 | 2023-07-25 | 香港城市大学深圳研究院 | Four-dimensional light field image depth acquisition method and device |
| CN114511609A (en) * | 2022-04-18 | 2022-05-17 | 清华大学 | Unsupervised light field parallax estimation system and method based on occlusion perception |
| CN114897952A (en) * | 2022-05-30 | 2022-08-12 | 中国测绘科学研究院 | Method and system for estimating accurate depth of single light field image in self-adaptive shielding manner |
| CN116029919A (en) * | 2022-12-09 | 2023-04-28 | 中国科学院微电子研究所 | Intelligent optimization method for compressed three-dimensional light field display |
| CN116402908A (en) * | 2023-03-23 | 2023-07-07 | 宁波大学 | Dense light field image reconstruction method based on heterogeneous imaging |
Non-Patent Citations (3)
| Title |
|---|
| "Occlusion-Aware Cost Constructor for Light Field Depth Estimation";Wang yingqian etal.;《CVPR》;20221231;第19809-19818页 * |
| "Trust your Model:Light Field Depth Estimation with inline Occlusion Handling";Hendrik Schilling etal.;《CVPR》;20181231;第4530-4538页 * |
| "基于光场内联遮挡处理的噪声场景深度获取";吴迪 等;《光电工程》;20211231;第48卷(第7期);第200422-1至200422-14页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117474922A (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110569704B (en) | Multi-strategy self-adaptive lane line detection method based on stereoscopic vision | |
| Schilling et al. | Trust your model: Light field depth estimation with inline occlusion handling | |
| CN108596965B (en) | Light field image depth estimation method | |
| EP2386998B1 (en) | A Two-Stage Correlation Method for Correspondence Search | |
| US8326025B2 (en) | Method for determining a depth map from images, device for determining a depth map | |
| RU2382406C1 (en) | Method of improving disparity map and device for realising said method | |
| US8897545B2 (en) | Apparatus and method for determining a confidence value of a disparity estimate | |
| CN107844750A (en) | A kind of water surface panoramic picture target detection recognition methods | |
| CN110246151B (en) | Underwater robot target tracking method based on deep learning and monocular vision | |
| EP3293700B1 (en) | 3d reconstruction for vehicle | |
| CN109859249B (en) | Scene flow estimation method based on automatic layering in RGBD sequence | |
| CN104200434B (en) | Non-local mean image denoising method based on noise variance estimation | |
| Lo et al. | Depth map super-resolution via Markov random fields without texture-copying artifacts | |
| Chen et al. | A color-guided, region-adaptive and depth-selective unified framework for Kinect depth recovery | |
| Krauss et al. | Deterministic guided lidar depth map completion | |
| CN117474922B (en) | Anti-noise light field depth measurement method and system based on inline shielding processing | |
| Schauwecker et al. | A comparative study of stereo-matching algorithms for road-modeling in the presence of windscreen wipers | |
| Gong et al. | Near-real-time stereo matching with slanted surface modeling and sub-pixel accuracy | |
| CN110490877B (en) | Target segmentation method for binocular stereo image based on Graph Cuts | |
| Dong et al. | Stereo matching via dual fusion | |
| CN117824625B (en) | High dam large warehouse underwater environment sensing and composition method based on improved visual odometer | |
| Zhang et al. | Insights into local stereo matching: Evaluation of disparity refinement approaches | |
| CN118172488A (en) | Three-dimensional reconstruction method based on low texture region | |
| Hamzah et al. | Gradient Magnitude Differences and Guided Filter for Stereo Video Matching Algorithm | |
| Liu et al. | Semiglobal Stereo Matching Algorithm Based on Improved Census Transformation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |