CN104933684B - A kind of light field method for reconstructing - Google Patents

A kind of light field method for reconstructing Download PDF

Info

Publication number
CN104933684B
CN104933684B CN201510323863.7A CN201510323863A CN104933684B CN 104933684 B CN104933684 B CN 104933684B CN 201510323863 A CN201510323863 A CN 201510323863A CN 104933684 B CN104933684 B CN 104933684B
Authority
CN
China
Prior art keywords
msub
mrow
mtd
light field
dictionary
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510323863.7A
Other languages
Chinese (zh)
Other versions
CN104933684A (en
Inventor
尹宝才
王玉萍
王立春
孔德慧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Technology
Original Assignee
Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Technology filed Critical Beijing University of Technology
Priority to CN201510323863.7A priority Critical patent/CN104933684B/en
Publication of CN104933684A publication Critical patent/CN104933684A/en
Application granted granted Critical
Publication of CN104933684B publication Critical patent/CN104933684B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Analysis (AREA)

Abstract

The present invention discloses a kind of light field method for reconstructing, and it can keep the structural information between each dimension of high dimensional data, more effectively carries out rarefaction representation to data, reduces the memory space of dictionary, greatly improves the quality for rebuilding light field image.This light field method for reconstructing, is expressed as tensor structure in training dictionary by light field atom, trains multiple small dictionaries simultaneously by each pattern of tensor, then multiple small dictionaries are merged into a big dictionary by Kronecker product.

Description

A kind of light field method for reconstructing
Technical field
The invention belongs to calculate the technical field of photography, more particularly to a kind of light field method for reconstructing.
Background technology
Compared to traditional colour or gray level image, light field image, which integrates, more precisely reliably to be retouched as scene offer is a kind of State, and the performance of computer vision field each task can be improved.In recent years, increasing light-field camera continues to bring out Out, one kind is directly to obtain light field by the design to optics or the rearrangement added to inducing pixel；Another kind of is to obtain Part light field data is taken, calculating after acquisition data progress is handled to the light field rebuild.
In SIGGRAPH 2013, Marwah et al. proposes a kind of new type of compression light-field camera, is by the optical path Insert mask and combine compressive sensing theory and rebuild to realize.This method is a Typical Representative in above-mentioned second class.Should The image-forming principle of camera is as shown in Figure 1.One image i (x) by mask f (ξ) codings is as light field image collection l's (x, v) Compression sampling, the good dictionary of combined training carry out sparse reconstruction.The process is exactly that classical compressive sensing theory is led in optical field acquisition A kind of application in domain.It is during dictionary training, the form by light field atomic arrangement into column vector.But light field is as one Kind high dimensional data, vectorization operate the structural information for destroying high dimensional data, it is thus impossible to carry out sparse table to data well Show, the memory space of dictionary is bigger, rebuilds the of poor quality of light field image.
The content of the invention
The technology of the present invention solves problem：Overcome the deficiencies in the prior art, there is provided a kind of light field method for reconstructing, it can The structural information between each dimension of high dimensional data is kept, rarefaction representation more effectively is carried out to data, the storage for reducing dictionary is empty Between, greatly improve the quality for rebuilding light field image.
The present invention technical solution be：Light field atom, is expressed as by this light field method for reconstructing in training dictionary Tensor structure, multiple small dictionaries are trained simultaneously by each pattern of tensor, then merge multiple small dictionaries by Kronecker product Into a big dictionary.
The present invention improves the process of dictionary training, and light field atom is expressed as into tensor structure, same by each pattern of tensor The multiple small dictionaries of Shi Xunlian, then multiple small dictionaries are merged into by a big dictionary, this training method energy by Kronecker product The structural information between each dimension of high dimensional data is enough kept, rarefaction representation more effectively is carried out to data, reduces the storage of dictionary Space, greatly improve the quality for rebuilding light field image.
Brief description of the drawings
Fig. 1 shows a kind of image-forming principle of camera of the prior art；
Fig. 2 shows matrix description schematic diagram corresponding to light filed acquisition process in Fig. 1.
Embodiment
This light field method for reconstructing, is expressed as tensor structure in training dictionary by light field atom, by each mould of tensor Formula trains multiple small dictionaries simultaneously, then multiple small dictionaries are merged into a big dictionary by Kronecker product.
The present invention improves the process of dictionary training, and light field atom is expressed as into tensor structure, same by each pattern of tensor The multiple small dictionaries of Shi Xunlian, then multiple small dictionaries are merged into by a big dictionary, this training method energy by Kronecker product The structural information between each dimension of high dimensional data is enough kept, rarefaction representation more effectively is carried out to data, reduces the storage of dictionary Space, greatly improve the quality for rebuilding light field image.
Preferably, this method comprises the following steps：
(1) training sample set is givenWherein P is the light field original for training
The number of son, training process are described with formula (2)
WhereinIt is D(r)I-th row, IPBe dimension be P unit matrix, K0It is normal for one Number；
(2) sparse coding：The process fixes D(i), i value is 1 ..., n, seeks S, and formula (2) deteriorates to formula (3)：
The formula (3) is solved by OMP methods；
(3) dictionary updating：The process fixes S, seeks D(i), i value is 1 ..., n, according to formula (4) by tensorPoint A series of sum of the solution into tensors of order 1：
Wherein
For serial number (p1, p2..., pn) single atom, fixed S and D(i)In after its dependent variable, wherein i value For 1 ..., n, formula (2) is converted into formula (5)
Wherein
(4) by sparse vectorDeteriorate to the vector of only nonzero element
Wherein KtIt is the number of nonzero element,Also it is right It should be reduced toIt is rightCP decomposition is done, CP factors conduct corresponding to each pattern The atom of renewal.
Preferably, this method comprises the following steps：
(I) input：Training dataInitial dictionaryTarget sparse degree K0, iterations Q；
(II) ensure：Rarefaction representation
(III) m=1 ..., Q is performed：
So that | | S (：...,：, i) | |0≤K0；
(IV) m=1 ..., Q is performed：
(V)
(VI) it is rightCarry out CP decomposition；
(VII) dictionary atom is updatedJ value is 1 ..., n；
(VIII) S corresponding to renewal；
(IX) end step (IV)；
(X) end step (III)；
(XI) D is returned(j), j value is 1 ..., n；.
Above method is specifically described below.
It is well known that compressed sensing coding/decoding method is divided into observation and rebuilds two important steps.For the present invention, mainly exist Light field is improved on Marwah et al. working foundations and rebuilds part, that is, proposes a kind of dictionary training method suitable for light field data.
The image-forming principle of the light-field camera and corresponding compressed sensing technology are briefly introduced first.The light-field camera be A mask is inserted in the light path of general camera, to realize the purpose that light field data is encoded to observed image.In mathematical terms Expressing imaging process is：
I=Φ l
Wherein, i and l is respectively the collection image and light field image collection of vectorization, and Φ is observing matrix.All angular lights Field picture is all stacked in l.The matrixing description of the imaging process is as shown in Figure 2.
The process that known i and Φ solve l is a underdetermined problem.Purpose is solved, it is necessary to be subject to l characteristic in order to reach Constraint.It is a kind of effective strategy to find a dictionary that rarefaction representation can be carried out to l, now solves optimization problem
α*=argminα||i-ΦDα||2+λ||α||1 (1)
Wherein D is dictionary, and α is sparse coefficients of the l under dictionary D, D α*To rebuild light field l out.
In whole framework described above, wherein a key issue for influenceing reconstruction light field quality is D training Journey.In SIGGRAPH 2013, Marwah et al. is by the form for the atomic arrangement of training into column vector, afterwards using K- SVD method training dictionaries.But light field data changes data script as a kind of high dimensional data structure, this arrangement mode Structural information.Fortunately, tensor form can effectively keep the structure attribute of high dimensional data.Therefore, the present invention is intended to improve Dictionary training method in the process of reconstruction of light field image collection.Light field atom is expressed as tensor structure, by each mould of tensor Formula trains multiple small dictionaries simultaneously, then multiple small dictionaries are merged into a big dictionary by Kronecker product.Specific training word The process of allusion quotation is briefly discussed below：
The mark of dictionary training needs is described first.Make tensorRepresent light field block, different n generations The different light field structure of table, n value can be 1,2,3,4, be one-dimensional vector during such as n=1, be two-dimensional matrix during n=2, M1..., MnIt is the resolution sizes on each roads of light field block L respectively.Following Tucker decomposition can be carried out to tensor L：
L=S ×1D(1)…×nD(n)
WhereinFor 1,2 ..., n,× i is tensor S and matrix D(i)'s I- pattern products.Using the property of tensor operation, above formula is equivalent to
Wherein L(1)For the tensor L matrixings under corresponding 1- patterns, S(1)To correspond to the tensor s-matrix under 1- patterns, For Kronecker product.Using the property of Kronecker product, above formula can be deformed into
Wherein, vec () is that matrix is transformed into vector form by row.D(1)..., D(n)For the small dictionary to be trained,For the big dictionary in corresponding solving model (1).
D is trained in lower mask body introduction(i)Process, wherein i values are 1 ..., n.Given training sample setWherein P is the number of the light field atom for training.Training process can use following optimization problem Describe (formula (2))
WhereinIt is D(r)I-th row, IPBe dimension be P unit matrix, K0It is normal for one Number.The optimization problem can proceed in two phases iterative：
(1) sparse coding：The process fixes D(i), wherein i value is 1 ..., n, seeks S.Optimization problem
(2) single argument optimization problem (formula (3)) is deteriorated to：
The optimization problem can be solved by OMP methods.
(2) dictionary updating：The process fixes S, seeks D(i), wherein i value is 1 ..., n,.Specifically, tensorCan be with A series of sum of tensors of order 1 is resolved into, i.e.,
Wherein,
By simple calculations, for serial number (p1, p2..., pn) single atom, fixed S and D(i)In
After its dependent variable, wherein i value is 1 ..., n, and formula (2) is converted into formula (5)
Wherein,
For solving-optimizing problem (formula (5)), using similar K-SVD method, by sparse vector Deteriorate to the vector of only nonzero elementWherein KtIt is the number of nonzero element.MeanwhileAlso correspond to and be reduced toAt this moment, it is right CP decomposition is done, atom of the CP factors corresponding to each pattern as renewal.
Said process is summarized as algorithm 1.
Algorithm 1：
Input：Training dataInitial dictionaryTarget sparse degree K0, Iterations Q.
Ensure：Rarefaction representation
Iterative process：
1. couple m=1 ..., Q is performed：
2.
3. couple m=1 ..., Q is performed：
4.
5.
6. pairCarry out CP decomposition
7. update dictionary atomWherein j value is 1 ..., n
8. S corresponding to renewal
9. terminate 3
10. terminate 1
11. return to D(j), wherein j value is 1 ..., n.
Experimental verification has been carried out to above-mentioned model, and has achieved obvious effect.In an experiment, from SIGGRAPH2013 Data set used in middle Marwah et al., and choose n=2 and carry out compared with Marwah et al. method.In experiment, light field atom The size of block is 5 × 5 × 8 × 8, corresponding hereinbefore M1=25, M2=64.
From experimental result it is concluded that：When dictionary size is suitable, the reconstruction quality of the inventive method is excellent In Marwah et al. methods, in other words, when reconstruction quality is suitable, the dictionary size needed for the inventive method is far smaller than Marwah et al. methods.The reconstruction qualities of two methods is compared by experiment, dictionary size of the present invention be 8817 (=25 × 25,64 × 128) when, PSNR=26.1034dB.When Marwah et al. methods dictionary size is 800000 (=1600 × 500), PSNR=25.9139dB.And reconstruction quality corresponding to the situation of other dictionary sizes is enumerated in table 1.Pass through experimental demonstration The above reliability of analysis and conclusion.
Table 1
It is described above, be only presently preferred embodiments of the present invention, any formal limitation not made to the present invention, it is every according to Any simple modification, equivalent change and modification made according to the technical spirit of the present invention to above example, still belong to the present invention The protection domain of technical scheme.

Claims (2)

  1. A kind of 1. light field method for reconstructing, it is characterised in that：Light field atom is expressed as tensor structure in training dictionary, by tensor Each pattern train multiple small dictionaries simultaneously, then multiple small dictionaries are merged into a big dictionary by Kronecker product；Should Method comprises the following steps：
    (1) training sample set is given It is set of real numbers, wherein n is the dimension of atom, for light field Data n takes 1,2,3,4；M1..., MnIt is the size of each dimension of atom, P is the number of the light field atom for training, training Process is described with formula (2)
    Wherein | | | |FIt is the Frobenius norms of matrix, for i=1,2 ..., n+1, ×iFor the Matrix Multiplication under i patterns Method, for i=1 .., n,For corresponding small dictionary under i-th of dimension of atom, its dimension is Mi×Ni,It is D(r)Jth row,For core tensor, IPBe P × P dimension unit matrix, K0For a constant, And markRepresent that it is i's to fix the (n+1)th dimension indicatorSub- tensor；
    (2) sparse coding：Fixed D(i), i value is 1 ..., n, seeks S, and formula (2) deteriorates to formula (3)：
    The formula (3) is solved by orthogonal matching pursuit OMP methods；
    (3) dictionary updating：Fixed S, seeks D(i), i value is 1 ..., n, according to formula
    (4) by tensorResolve into a series of sum of tensors of order 1：
    WhereinRepresent apposition,Represent core tensorN dimension indicators are (k before fixation1..., km) subvector, and
    <mrow> <mi>t</mi> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mn>1</mn> </msub> <mo>,</mo> <mn>...</mn> <mo>,</mo> <msub> <mi>t</mi> <mi>n</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mfenced open = "{" close = ""> <mtable> <mtr> <mtd> <mrow> <msub> <mi>k</mi> <mn>1</mn> </msub> <mo>,</mo> </mrow> </mtd> <mtd> <mrow> <mi>n</mi> <mo>=</mo> <mn>1</mn> <mo>;</mo> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mi>k</mi> <mn>2</mn> </msub> <mo>+</mo> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mn>1</mn> </msub> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> <msub> <mi>N</mi> <mn>2</mn> </msub> <mo>,</mo> </mrow> </mtd> <mtd> <mrow> <mi>n</mi> <mo>=</mo> <mn>2</mn> <mo>;</mo> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mi>k</mi> <mn>3</mn> </msub> <mo>+</mo> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mn>2</mn> </msub> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> <msub> <mi>N</mi> <mn>3</mn> </msub> <mo>+</mo> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mn>1</mn> </msub> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> <msub> <mi>N</mi> <mn>2</mn> </msub> <msub> <mi>N</mi> <mn>3</mn> </msub> <mo>,</mo> </mrow> </mtd> <mtd> <mrow> <mi>n</mi> <mo>=</mo> <mn>3</mn> <mo>;</mo> </mrow> </mtd> </mtr> <mtr> <mtd> <mrow> <msub> <mi>k</mi> <mn>4</mn> </msub> <mo>+</mo> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mn>3</mn> </msub> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> <msub> <mi>N</mi> <mn>4</mn> </msub> <mo>+</mo> <mn>...</mn> <mo>+</mo> <mrow> <mo>(</mo> <msub> <mi>k</mi> <mn>1</mn> </msub> <mo>-</mo> <mn>1</mn> <mo>)</mo> </mrow> <msub> <mi>N</mi> <mn>2</mn> </msub> <msub> <mi>N</mi> <mn>3</mn> </msub> <msub> <mi>N</mi> <mn>4</mn> </msub> <mo>,</mo> </mrow> </mtd> <mtd> <mrow> <mi>n</mi> <mo>=</mo> <mn>4.</mn> </mrow> </mtd> </mtr> </mtable> </mfenced> </mrow>
    For serial number (p1, p2..., pn) single atom, it is fixedAnd D(i)In after its dependent variable, wherein i value is 1 ..., n, formula (2) are converted into formula (5)
    Wherein
    (4) by sparse vectorDeteriorate to the vector of only nonzero elementWherein KtIt is the number of nonzero element,Also correspond to and be reduced toIt is rightCP decomposition is done, atom of the CP factors corresponding to each pattern as renewal.
  2. 2. light field method for reconstructing according to claim 1, it is characterised in that：This method comprises the following steps：
    (I) input：Training dataInitial dictionaryTarget sparse degree K0, iteration Number Q；
    (II) ensure：Rarefaction representation
    (III) m=1 ..., Q is performed：
    So that
    (IV) to sequence number (p1..., pn) single atom traversal perform：
    <mrow> <mi>I</mi> <mo>:</mo> <mo>=</mo> <mo>{</mo> <mi>i</mi> <mo>|</mo> <mn>1</mn> <mo>&amp;le;</mo> <mi>i</mi> <mo>&amp;le;</mo> <mi>P</mi> <mo>,</mo> <msub> <mi>s</mi> <mrow> <mi>t</mi> <mrow> <mo>(</mo> <msub> <mi>p</mi> <mn>1</mn> </msub> <mo>,</mo> <mn>....</mn> <mo>,</mo> <msub> <mi>p</mi> <mi>n</mi> </msub> <mo>)</mo> </mrow> </mrow> </msub> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>&amp;NotEqual;</mo> <mn>0</mn> <mo>}</mo> <mo>;</mo> </mrow>
    <mrow> <mo>(</mo> <mi>V</mi> <mo>)</mo> </mrow>
    Wherein Currently to use the sample set that atom is corresponded in set I；For sparse vectorIt is punctured into only non- The vector of neutral element；
    (VI) it is rightCarry out CP decompositionWherein u1..., un+1The component decomposed for CP；
    (VII) dictionary atom is updatedJ value is 1 ..., n；
    (VIII) corresponding to renewal
    (IX) end step (IV)；
    (X) end step (III)；
    (XI) D is returned(j), j value is 1 ..., n.
CN201510323863.7A 2015-06-12 2015-06-12 A kind of light field method for reconstructing Active CN104933684B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510323863.7A CN104933684B (en) 2015-06-12 2015-06-12 A kind of light field method for reconstructing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510323863.7A CN104933684B (en) 2015-06-12 2015-06-12 A kind of light field method for reconstructing

Publications (2)

Publication Number Publication Date
CN104933684A CN104933684A (en) 2015-09-23
CN104933684B true CN104933684B (en) 2017-11-21

Family

ID=54120839

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510323863.7A Active CN104933684B (en) 2015-06-12 2015-06-12 A kind of light field method for reconstructing

Country Status (1)

Country Link
CN (1) CN104933684B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105634498A (en) * 2015-12-25 2016-06-01 北京工业大学 Observation matrix optimization method
CN105654119B (en) * 2015-12-25 2019-08-16 北京工业大学 A kind of dictionary optimization method
CN105825529B (en) * 2016-03-18 2019-02-26 武汉大学 It is a kind of based on non local and low-rank decomposition method for compressing high spectrum image
CN108494409B (en) * 2018-03-14 2021-07-13 电子科技大学 Underground high-speed real-time compression method of neutron logging-while-drilling instrument based on small dictionary
CN109921799B (en) * 2019-02-20 2023-03-31 重庆邮电大学 Tensor compression method based on energy-gathering dictionary learning
CN112884645B (en) * 2021-01-18 2024-05-03 北京工业大学 Tensor sparse constraint-based light field filling method and device

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104036489A (en) * 2014-05-09 2014-09-10 北京工业大学 Light field acquisition method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040114807A1 (en) * 2002-12-13 2004-06-17 Dan Lelescu Statistical representation and coding of light field data

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104036489A (en) * 2014-05-09 2014-09-10 北京工业大学 Light field acquisition method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Stable, Robust, and Super Fast Reconstruction of Tensors Using Multi-Way Projections;Cesar F. Caiafa et al.;《IEEE Transactions on Signal Processing》;20150201;第63卷(第3期);第780-781页 *
Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting;Gordon Wetzstein et al.;《ACM Transactions on Graphics》;20120731;第31卷(第4期);第5页 *
字典学习模型、算法及其应用研究进展;练秋生 等;《自动化学报》;20150228;第41卷(第2期);第240-260页 *

Also Published As

Publication number Publication date
CN104933684A (en) 2015-09-23

Similar Documents

Publication Publication Date Title
CN104933684B (en) A kind of light field method for reconstructing
CN113360673B (en) Entity alignment method, device and storage medium of multi-mode knowledge graph
CN110097550B (en) Medical image segmentation method and system based on deep learning
Yang et al. Reconstructing the dark matter and dark energy interaction scenarios from observations
Oriti et al. Emergent Friedmann dynamics with a quantum bounce from quantum gravity condensates
Jin et al. Trseg: Transformer for semantic segmentation
CN104867119B (en) The structural missing image fill method rebuild based on low-rank matrix
CN107730451A (en) A kind of compressed sensing method for reconstructing and system based on depth residual error network
CN105608690B (en) A kind of image partition method being combined based on graph theory and semi-supervised learning
CN110659727A (en) Sketch-based image generation method
CN105118053B (en) A kind of full reference picture assessment method for encoding quality based on compressed sensing
CN109410289A (en) A kind of high lack sampling hyperpolarized gas lung MRI method for reconstructing of deep learning
CN107194912A (en) The brain CT/MR image interfusion methods of improvement coupling dictionary learning based on rarefaction representation
CN104077761B (en) Multi-focus image fusion method based on self-adaption sparse representation
DallʼAgata et al. de Sitter vacua in N= 8 supergravity and slow-roll conditions
Vasseur et al. Operator content of the critical Potts model in d dimensions and logarithmic correlations
CN109191376A (en) High-resolution terahertz image reconstruction method based on SRCNN improved model
CN108550108A (en) A kind of Fourier&#39;s lamination image method for reconstructing minimized based on phase iteration
CN108305297A (en) A kind of image processing method based on multidimensional tensor dictionary learning algorithm
CN104463148B (en) Face identification method based on Image Reconstruction and hash algorithm
CN104778665B (en) Compression ghost image reconstruction method and system based on the driving of natural image block priori
CN104091364B (en) Single-image super-resolution reconstruction method
CN111862261B (en) FLAIR modal magnetic resonance image generation method and system
CN104915935B (en) Compressed spectrum imaging method with dictionary learning is perceived based on non-linear compression
Berg et al. An orientifold with fluxes and branes via T-duality

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant