CN102036095A - Resolution compensating device and method applied to three-dimensional (3D) image display and 3D television - Google Patents
Resolution compensating device and method applied to three-dimensional (3D) image display and 3D television Download PDFInfo
- Publication number
- CN102036095A CN102036095A CN 201010620694 CN201010620694A CN102036095A CN 102036095 A CN102036095 A CN 102036095A CN 201010620694 CN201010620694 CN 201010620694 CN 201010620694 A CN201010620694 A CN 201010620694A CN 102036095 A CN102036095 A CN 102036095A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- compensated
- module
- observation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention provides a resolution compensating device and a resolution compensating method applied to three-dimensional (3D) image display and a 3D television. The device comprises an acquiring module for acquiring an observation low resolution (LR) image group according to a time frame sequence, a setting module for taking a first image in the observation LR image group as the current image to be compensated, and a resolution compensating module for compensating the resolution of the current image to be compensated by using a super-resolution algorithm. By using the resolution compensating device and the resolution compensating method applied to the 3D image display and the 3D television which are disclosed by the embodiment of the invention, the resolution of the acquired compensated images is further improved, so that the watching experience of users can be improved when the users watch television programs consisting of the compensated high-resolution images.
Description
Technical field
The present invention relates to image processing field, particularly relate to a kind of resolution compensation device, 3D television set and method that 3D rendering shows that be applied to.
Background technology
3D (3Dimensions, 3 dimensions) TV, exactly by on liquid crystal panel, adding special accurate cylindrical lens screen, 3D video image through encoding process is handled the right and left eyes of independently sending into the people, thereby make the user need not can bore hole experience stereo perception by anaglyph spectacles, simultaneously can compatible 2D picture.
The television set of using polarization type 3D technology (being also referred to as polarization type 3D technology) is called polarization type 3D television set.Polarization type 3D technology belongs to passive type 3D technology, and is very high to the brightness requirement of output equipment.Polarization type 3D technology mainly utilizes light to have the principle of " direction of vibration " to decompose original image, earlier vertical to polarised light and level two group of pictures to polarised light by a secondary original image is divided into, two eyeglasses adopt the polarized lenses of different polarization direction respectively about the 3D glasses then, just make user's right and left eyes see the smooth picture that different light rays is originated, pass through brain compound stereoscopic image again, produce the 3D visual effect with this.
In realizing process of the present invention, the inventor finds that there are the following problems at least in the prior art: what the polarization type 3D system of prior art was the most commonly used is RealD 3D system, but polarization type 3D system of the prior art is owing to need carry out the processing of branch frame, can make level reduce half to the resolution of polarised light, resolution has tangible reduction to the user when watching polarization type 3D television set putting on one's glasses, and then influences user's viewing experience.
Therefore, need the urgent technical problem that solves of those skilled in the art to be exactly at present: how can propose a kind of resolution compensation method, in order to solve the lower problem of image quality of polarization type 3D television set in the prior art.
Summary of the invention
Technical problem to be solved by this invention provides resolution compensation device, 3D television set and the method that 3D rendering shows that be applied to, in order to solve the lower problem of resolution of polarization type 3D television set in the prior art, make the resolution of polarization type 3D television set significantly to promote, thereby improve the image quality of polarization type 3D television set, and then can promote user's viewing experience.
For solving the problems of the technologies described above, the embodiment of the invention provides following technical scheme:
On the one hand, provide a kind of resolution compensation device that 3D rendering shows that is applied to, having comprised:
Acquisition module is used for obtaining observation low resolution LR image sets according to the time frame order;
Module is set, is used for observing the LR image as current image to be compensated first width of cloth of described observation LR image sets;
The resolution compensation module is used to adopt super-resolution algorithms that described current image to be compensated is carried out resolution compensation.
On the other hand, provide a kind of 3D television set, having comprised: aforesaid resolution compensation device, described resolution compensation device are arranged between the video input and transport module and polarization type 3D display module of described 3D television set.
Again on the one hand, provide a kind of resolution compensation method that 3D rendering shows that is applied to, having comprised:
Obtain observation low resolution LR image sets according to the time frame order;
With first width of cloth image in the described observation LR image sets as current image to be compensated;
Adopt super-resolution algorithms that described current image to be compensated is carried out resolution compensation.
Compared with prior art, a technical scheme in the technique scheme has following advantage or beneficial effect: in embodiments of the present invention, because the resolution compensation device has adopted super-resolution algorithms that the image to be compensated of low resolution has been carried out resolution compensation, so the resolution of the compensating images that obtains is improved, thereby in the time of can making the user watch the TV programme that the high-definition picture after the compensation forms, can promote image quality and user's the embodiment of watching.Be understandable that above-mentioned technique effect only is the effect of embodiment, rather than all whole effects of the present invention.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, to do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below, apparently, accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the structural representation that is applied to the resolution compensation device embodiment 1 of 3D rendering demonstration of the present invention;
Fig. 2 is the structural representation that is applied to the resolution compensation device embodiment 2 of 3D rendering demonstration of the present invention;
Fig. 3 is the structural representation of back projection's module 205 among apparatus of the present invention embodiment 2;
Fig. 4 is the principle schematic of four pixels of initial HR image of in actual applications fractional part of side-play amount being added to;
Fig. 5 is the structural representation of 3D television set embodiment of the present invention;
Fig. 6 is the flow chart that is applied to the resolution compensation method embodiment 1 of 3D rendering demonstration of the present invention;
Fig. 7 is the flow chart that is applied to the resolution compensation method embodiment 2 of 3D rendering demonstration of the present invention;
Fig. 8 is the flow chart of step 708 among the inventive method embodiment 2.
Embodiment
For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, the present invention is further detailed explanation below in conjunction with the drawings and specific embodiments.
The phenomenon that reduces for the image quality of improving polarization type 3D television set, simultaneously also need not supporting 3D glasses are improved, one of main thought of the present invention comprises, video signal transmission circuit at polarization type 3D television set improves, between video decode and digital signal processing circuit and image adjustment circuit, consider to increase a resolution compensation module, play the function that promotes image quality before to be implemented in video flowing.Can consider during specific implementation to carry out at first it being carried out resolution compensation according to interpolation factor f before the 3D processing at original image, make the resolution of original image bring up to f*f doubly, carry out image after the resolution compensation and carry out polarisation 3D again when dividing frame output, the image resolution ratio that the user watches can be equal to or higher than the resolution of original image, thereby has promoted user's viewing experience.
When being carried out resolution compensation, original image can adopt super-resolution algorithms, promptly be to utilize an existing frame or multiframe low resolution (Low Resolution, LR) image is rebuild width of cloth high-resolution (High Resolution clearly by certain algorithm, HR) image, according to the interpolation factor f that is provided with in the process of reconstruction, the super-resolution image resolution after the reconstruction will be f*f times of original image.When the algorithm of employing super-resolution carries out resolution compensation, the liquid crystal display screen of former polarization type 3D TV and 3D glasses, perhaps other auxiliary evaluation equipments all need not change, and only need increase a resolution compensation device in the video signal transmission circuit, can solve the lower problem of resolution of polarization type 3D television set in the prior art, make the resolution of polarization type 3D television set significantly to promote, and then promote image quality and user's viewing experience.
With reference to figure 1, show a kind of flow chart that is applied to the resolution compensation device embodiment 1 of 3D rendering demonstration of the present invention, can comprise with lower module:
When getting access to the observation LR image sets of free order, need by be provided with module 102 with first width of cloth image as current image to be compensated, promptly be with the most forward or that width of cloth image of playing at first as current image to be compensated, other observation LR image in should observation LR image sets is then as with reference to image.
The resolution compensation module utilizes super-resolution algorithms just can carry out resolution compensation to current image to be compensated, and the resolution of carrying out the final compensating images behind the resolution compensation improves greatly than original image, thereby has promoted user's viewing experience.Resolution compensation described in the present embodiment is meant and utilizes one or more low-resolution image utilization super-resolution algorithms to obtain a panel height distinct image.Specifically in actual applications, described resolution compensation module 103 can adopt based on the super-resolution algorithms of iterative backprojection described current image to be compensated is carried out resolution compensation.In the present embodiment, entering image by the video flowing at polarization type 3D television set adjusts before the circuit, just first image to be compensated in the observation LR image sets compensated, can obtain the high-resolution HR image of first image correspondence to be compensated, and the resolution of this HR image has promoted several times than original LR image, and then when on the 3D display screen of polarization type 3D television set, showing the video flowing that this HR image forms, can promote the resolution of original image and then promote image quality, thereby make video flowing in playing process, can promote user's viewing experience.
Need to prove, present embodiment introduce just to the compensation process of first width of cloth image, if desired second width of cloth image is carried out resolution compensation, need serve as that the image to be compensated of next observation LR image sets obtains new observation LR image sets and carries out resolution compensation with second width of cloth image then, promptly be the final compensating images that the method for carrying out a present embodiment can obtain to observe the piece image in the LR image sets.
Because the kind of super-resolution algorithms is a lot of in practice, for example: iterative backprojection algorithm (Iterated Back Projection), bicubic cube interpolation (Bicubic) algorithm, POCS convex set sciagraphy, PapoullisGerchberg algorithm and regularization convolution algorithm etc.Those skilled in the art's understanding for convenience, below be example with bicubic cube interpolation and convex set sciagraphy earlier just, simply introduce the principle that this two kinds of algorithm application down show in 3D rendering.
For bicubic cube interpolation algorithm, bicubic cube interpolation is the two dimensional form of a cube interpolation (Cubic), is to close on the algorithm that 16 points carry out interpolation to known in the image.The Cubic interpolation is the interpolation method of one dimension form, is example with the interpolation direction for column direction is carried out interpolation, for cubic interpolation formula f (x)=ax
3+ bx
2+ cx+d, independent variable are the ordinate of these row, and functional value is the brightness value of this pixel.If there are four point function values of closing on known, then between these four points,, can try to achieve by this cubic polynomial if will carry out interpolation to a certain coordinate position.
And bicubic cube interpolation is the two dimensional form of Cubic interpolation, is to the known algorithm that 16 points carry out interpolation that closes on.To carry out interpolation at a two-dimensional grid is example, can use a cube interpolation formula to make up bicubic cube interpolation formula, and cube interpolation formula is as follows:
This cube interpolation formula need meet the following conditions: when x got x=-1, x=0, x=1 and x=2 respectively, functional value was respectively a, b, c and d.
Below again according to cube interpolation formula derivation bicubic cube interpolation formula.Suppose 16 some p are arranged
Ij, i and j get from 0 to 3 numerical value, p respectively
IjBe positioned at that (at first the column direction to rectangular area [0,1] * [0,1] carries out interpolation for i-1, j-1) zone, then the column direction interpolation result is carried out the interpolation of horizontal direction.So above-mentioned cube interpolation method is with regard to a bicubic cube interpolation:
g(x，y)＝f(x，f(y，p
00，p
01，p
02，p
03)，f(y，p
10，p
11，p
12，p
13)，f(y，p
20，p
21，p
22，p
23)，f(y，p
30，p
31，p
32，p
33))
Suppose 16 pixels in 4 * 4 squares in the known image, its coordinate is transformed to x from 0 to 3 respectively, the y coordinate, adopt bicubic cube interpolation formula to carry out the integral multiple interpolation, default interpolation factor is factor, then, will be inserted into factor-1 point in two adjacent points along x direction or y direction; After adopting an above-mentioned bicubic cube interpolation formula interpolation to finish, the resolution of image becomes factor*factor times of original resolution, thereby has realized the lifting to 3D rendering resolution.The high-definition picture total quality that adopts bicubic cube interpolation algorithm to obtain is more even, and real-time is also better.
Convex set sciagraphy (POCS algorithm) then with prioris such as the roomage response characteristic of imaging system and noise statisticses as the constraints of carrying out image reconstruction, each constraints is corresponding to whole imaging space R
N1 * N2In a convex set that contains desirable high-definition picture, any point in the common factor of this convex set all is considered to acceptable image reconstruction result.POCS just is meant that from imaging space a bit beginning projection arbitrarily navigates to process on the common factor of convex set.On calculating, the POCS algorithm adopts the method that repeats to revise, i.e. certain since convex set point utilizes the constraint of other set to revise the position of this point, certain in navigating to common factor a bit.In general, the point that projects on the common factor is not only, and therefore final result of calculation is often relevant with choosing of initial value.
When being applied to the 3D rendering demonstration, the realization of convex set sciagraphy needs earlier low-resolution image to be estimated, just set up high-resolution reference frame image, image sequence is revised reference frame according to the observation then, until obtaining acceptable reconstruction result (final compensating images).The foundation of reference frame image is taked that generally the two field picture in the observation image sequence is carried out bilinear interpolation and is made it to reach desired high-resolution mode.Before revising reference frame, the POCS algorithm can adopt the diamond search algorithm based on the piece coupling to carry out estimation, can ask for motion vector quickly and accurately like this, has guaranteed that all points are projected onto position correct in the reference frame image in the observation image.
The method of correction reference frame is based on point spread function (PSF), just observing all images spot projection in the image sequence on reference frame image, find out the pixel of reference frame image in its PSF scope, adopt PSF to calculate the gray scale estimated value of this pixel, again the actual value in itself and the observation sequence is compared, if error exceeds allowed band, just the pixel gray value in the reference frame is revised, the error of gray scale estimated value and actual value is reduced in the desired scope.When all pixels do not exceed this allowed band, finish correction at last, and the reference frame image that will revise at last is as final compensating images to reference frame image.
Though bicubic side's interpolation algorithm and convex set sciagraphy can promote the resolution of image, but the inventor is through repeatedly experiment discovery, the real-time of the iterative backprojection algorithm in the super-resolution algorithms and the image quality of output image are better with respect to other algorithms, thus below be the process that example is described the image quality of superlift polarization type 3D TV in detail at first with the iterative backprojection algorithm.
When being applied to the 3D rendering demonstration, if be provided with interpolation factor f in the super-resolution algorithms process of reconstruction, the resolution of the super-resolution image after rebuilding so will be f*f times of original image.And in process of reconstruction, at first need to be to determine an iteration variable based on the principle of the super-resolution algorithm of iterative backprojection, this iteration variable can be directly or indirectly constantly by old value recursion value of making new advances, when being applied to the embodiment of the invention, this iteration variable promptly is observation LR image and the error amount of simulating the LR image; Secondly also to set up the iterative relation formula, it promptly is the relational expression of how releasing its next value from the previous value of iteration variable, when being applied to the embodiment of the invention, whether this iterative relation promptly is to observe the error amount of LR image and simulation LR image less than a default threshold value; Also needing at last iterative process is controlled, promptly is to stop iteration satisfying under which kind of condition; When being applied to the embodiment of the invention, can be when the error amount of observation LR image and simulation LR image during less than default threshold value, just stop iteration, if observation LR image is not less than default threshold value with the error amount of simulation LR image, just carry out the process that the error image back projection of observing LR image and corresponding simulating LR image arrives initial HR image, this initial HR image is that the image to be compensated resolution that processing obtains through bilinear interpolation is brought up to f*f image doubly.Mentioned back projection promptly is be added to process on the initial HR image of this error image in the embodiment of the invention.
Those skilled in the art understand the embodiment of above-mentioned iterative backprojection better for convenience, with reference to figure 2, show a kind of flow chart that is applied to the resolution compensation device embodiment 2 of 3D rendering demonstration of the present invention, present embodiment can comprise with lower module:
Need to prove in device embodiment 2, described resolution compensation module 103 specifically can realize by calculating sub-pixel displacement amount module 201, bilinearity difference interpolation processing module 202, the processing module that degrades 203, first judge module 204 and back projection's module 205, wherein:
Described calculating sub-pixel displacement amount module 201 is used to calculate the sub-pixel displacement amount of reference picture and described current image to be compensated; Described reference picture is other observation LR image except described current image to be compensated in the observation LR image sets.
When module 102 being set selecting image current to be compensated in the observation LR image sets, calculate the sub-pixel displacement amount of reference pictures and described current image to be compensated by calculating sub-pixel displacement amount module 201; Described reference picture promptly is other LR images except described current image to be compensated in the observation LR image sets.
Suppose that observing the image number of LR image sets in the present embodiment is K (wherein, can select 4 under the K preferable case), then first width of cloth image g
0(m
1, m
2) be image to be compensated, wherein m
1And m
2Pixel in the resolution expression image to be compensated on level and the vertical direction, g
0Represent first width of cloth image, then reference picture has (K-1) width of cloth, and the sub-pixel displacement amount (x of reference picture that calculates and image to be compensated
0, y
0)
K-1Also one total (K-1) is individual.
Described bilinearity difference interpolation processing module 202 is used for that described current image to be compensated is carried out bilinear interpolation and handles, to obtain and the corresponding initial high resolution HR image of described current image to be compensated.
This bilinearity difference interpolation processing module 202 can be utilized following formula that described current image to be compensated is carried out bilinear interpolation and handle:
F (x, y)=[f (1,0)-f (0,0)] x+[f (0,1)-f (0,0)] y+[f (1,1)+f (0,0)-f (0,1)-f (1,0)] xy+f (0,0), to obtain and the corresponding initial high resolution HR image of described current image to be compensated
The resolution of this initial HR image respectively expands f to doubly at x and y direction, and therefore this initial HR image has expanded f*f to doubly than the resolution of image to be compensated.
In order to make those skilled in the art be more readily understood the processing procedure of bilinearity difference interpolation processing module 202, the principle to bilinear interpolation describes in detail below.
Bilinear interpolation is called bilinear interpolation again, and on mathematical meaning, bilinear interpolation is the linear interpolation expansion that the interpolating function of two variablees is arranged, and its core concept is to carry out the once linear interpolation respectively at both direction.
If we expect that (x, value y) suppose that our known function func is at Q to unknown function func at a P=
11=(x1, y1), Q
12=(x1, y2), Q
21=(x2, y1) and Q
22=(x2, y2) value of four points.With reference to shown in Figure 3, be the bilinear interpolation schematic diagram.Wherein, Q
11, Q
12, Q
21And Q
22Be known data point, R
1And R
2The point that obtains for the x directional interpolation; With R
1And R
2Carry out linear interpolation along the y direction again, just obtain the value of unknown function f, promptly put the value of P at interpolation point.
At first carry out linear interpolation, can obtain following formula in the x direction:
Carry out linear interpolation in the y direction then, can obtain following formula:
The arrangement foregoing can obtain desired f as a result (x, y), shown in the following formula:
If select a coordinate system to make the Q of f
11, Q
12, Q
21And Q
22Coordinate be respectively (0,0), (0,1), (1,0) and (1,1), so f coordinate (x, the interpolation formula of y) locating just can abbreviation be:
f(x，y)≈f(0，0)(1-x)(1-y)+f(1，0)x(1-y)+f(0，1)(1-x)y+f(1，1)xy.
Put above-mentioned abbreviation formula in order promptly:
f(x，y)＝[f(1，0)-f(0，0)]x+[f(0，1)-f(0，0)]y+[f(1，1)+f(0，0)-f(0，1)-f(1，0)]xy+f(0，0)
The described processing module 203 that degrades is used for according to described sub-pixel displacement amount and default interpolation factor to the processing that degrades of described initial HR image, to obtain and described observation LR image sets corresponding simulating LR image sets.
After obtaining initial HR image, according to sub-pixel displacement amount and the default interpolation factor f processing that degrades again.Promptly be that this initial HR image is carried out f times of down-sampling earlier particularly, obtain then and this initial HR image corresponding simulating LR image, this LR image is added the sub-pixel side-play amount that K-1 is individual more successively, just obtain the K-1 width of cloth simulation LR image (being the estimated image of former K-1 width of cloth observation LR image) corresponding to former K-1 width of cloth reference picture, just obtained and described observation LR image sets corresponding simulating LR image sets this moment.In this simulation LR image sets also is K width of cloth image, comprises the described and initial corresponding LR image of HR image, and adds the K-1 width of cloth analog image that the sub-pixel side-play amount obtains.
Whether described first judge module 204, the error amount that is used to judge described observation LR image sets and simulation LR image sets less than predetermined threshold value, if, then with the final compensating images of described initial HR image as current image to be compensated; If not, then trigger back projection's module 205.
In actual applications, the error amount of Dui Ying calculating observation LR image sets and simulation LR image sets successively, can adopt formula:
Realize.
Wherein, n is the current iteration number of times, calculates error amount and whether judges this error amount again less than a predetermined threshold value afterwards, and the size of this threshold value can change to some extent according to the difference of actual conditions, and for example, value is 0.0001.When the error amount that obtains less than predetermined threshold value, just directly with the initial HR image of image correspondence to be compensated final compensating images as current image to be compensated, follow-up final compensating images can output, if and error amount is not less than predetermined threshold value, then triggers back projection's module 205 and carry out.
Described back projection module 205, be used for will observation LR image and corresponding simulating LR image error image back projection to described initial HR image, and trigger second judge module 206.
When error amount was not less than predetermined threshold value, the error image back projection that then will observe LR image and corresponding simulating LR image was to described initial HR image.In the present embodiment, back projection specifically refers to be added to process on the described initial HR image of the observation LR image in the observation LR image sets and the error image between the simulation LR image in the corresponding simulating LR image sets.
With reference to shown in Figure 3, described back projection module 205 specifically can comprise:
Calculating sub module 301 is used to calculate the described error image of observing LR image and corresponding simulating LR image.
When the error image of calculating observation LR image and corresponding simulating LR image, can be directly one group of observation LR image of correspondence be directly subtracted each other with the pixel of simulation LR image, the image after then subtracting each other promptly is an observation LR image and the error image of simulation LR image
Calculating sub module 301 can obtain K width of cloth error image.
Interpolation processing sub 302 is used for according to default interpolation factor described error image being carried out interpolation and handles.
Default interpolation factor is f, according to default interpolation factor described error image is carried out interpolation again this moment and handles, and obtains the error image that the K width of cloth is handled through interpolation
The error image of handling through interpolation all expands f to doubly at x and y direction, and the displacement of error image (x, y)
k=(fx
0, fy
0)
k
Stack submodule 303 is used for the described initial HR image that will be added to through the error image of interpolation processing.
Again by the error image of stack submodule 303 with the processing of K width of cloth process interpolation
Described initial HR image successively superposes
On.Need to prove, carry out after the back projection that 303 of the submodules that superpose this moment are to be added to through the error image that interpolation is handled through the initial HR image of back projection's processing, only superpose at current HR image when stack submodule 303 is handled again.
The displacement of stack submodule 303 is (x
k, y
k)=(x, y)
k,, therefore it need be divided into integer part and fractional part because displacement can not guarantee to be integer in actual applications.For example for x
k=[x
k]+x '
k, wherein [x] is integer part, x '
kBe fractional part, in like manner can get that y direction side-play amount is y
k=[y
k]+y '
k Stack submodule 303 is when handling, and initial HR image at first is added to integer part
On, and because the fractional part of side-play amount can't a mobile integer pixel, so each fractional part is added to
The time will act on four pixels simultaneously, and to the superposition value of each pixel by its area coverage decision in this pixel.With reference to shown in Figure 4, be the initial HR image that in actual applications fractional part of side-play amount is added to
The principle schematic of four pixels.Wherein,
Be the side-play amount of fractional part, wherein, four pixels are respectively:
With
Whether second judge module 206 is used to judge the current iteration number of times greater than default maximum iteration time, if, then will be through the initial HR image of back projection's processing final compensating images as current image to be compensated; If not, then trigger the described processing module 203 that degrades.
After the error image back projection that will observe LR image and corresponding simulating LR image is to described initial HR image, judge that by second judge module 206 whether the current iteration number of times is greater than default maximum iteration time again, this maximum iteration time also can adopt different numerical value according to the difference of actual demand, and for example 100 is inferior.If the result of second judge module 206 is for being, then the initial HR image that will handle through back projection is as the final compensating images of current image to be compensated.And if the result of second judge module 206 is not for, then explanation also needs to continue iteration, triggers the processing that degrades of initial HR image that the processing module that degrades handles this process back projection this moment, thus triggering iterative process next time.
Need to prove that the iterations here refers to the execution number of times that aforementioned calculating sub-pixel displacement amount module 201, bilinearity difference interpolation processing module 202, the processing module that degrades 203, first judge module 204 and back projection's module 205 continue successively and trigger.For example, when the resolution to current image to be compensated compensates, after calculating sub-pixel displacement amount module 201 has been calculated the sub-pixel displacement amount of reference picture and described current image to be compensated, bilinearity difference interpolation processing module 202 continues described current image to be compensated is carried out the bilinear interpolation processing, obtained initial high resolution HR image, the processing module that degrades 203 then according to this pixel displacement amount that calculates and default interpolation factor to the processing that degrades of described initial HR image, obtained and described observation LR image sets corresponding simulating LR image sets, judge that by first judge module 204 whether the error amount of described observation LR image sets and simulation LR image sets is less than predetermined threshold value this moment again,, then trigger back projection's module 205 and carry out for not the time in first judge module, 204 results.Aforesaid each module implementation of handling that continues successively is called iteration one time.
The concrete employing of present embodiment is treated compensating images based on the super-resolution algorithms of iterative backprojection and is handled, and then can obtain HR image after the final compensation of this image to be compensated, and, in this step also to having increased judgement to iterations, just can not make the super-resolution algorithms based on iterative backprojection enter endless loop, guaranteed that the resolution compensation for the treatment of compensating images can carry out smoothly, and the image after the compensation can promote the resolution of image and user's viewing experience.
With reference to figure 5, the invention also discloses the structural representation of a kind of polarization type 3D television set embodiment, wherein, this polarization type 3D television set specifically can comprise the described resolution compensation device 502 of aforementioned arbitrary embodiment, and this polarization type 3D television set also comprises: the video input is adjusted circuit 503 and polarization type 3D liquid crystal indicator 504 with transport module 501, image.In this polarization type 3D television set embodiment, vision signal transfers in the resolution compensation device 502 with transport module 501 via the video input, resolution compensation device 502 at first obtains observation low resolution LR image sets to video flowing according to the time frame order, again with first width of cloth image in the described observation LR image sets as current image to be compensated; Adopt super-resolution algorithms that described current image to be compensated is carried out resolution compensation at last.The video flowing that image after will compensating is at last formed transfers to image again and adjusts circuit, so that the image after the compensation when playing on polarization type 3D liquid crystal indicator, can promote the resolution of image, thereby promote user's viewing experience.
With reference to figure 6, show a kind of structured flowchart that is applied to the resolution compensation method embodiment 1 of 3D rendering demonstration of the present invention, can may further comprise the steps:
Step 601: the resolution compensation device obtains observation low resolution LR image sets according to the time frame order.
Step 602: the resolution compensation device with first width of cloth image in the described observation LR image sets as current image to be compensated.
Step 603: the resolution compensation device adopts super-resolution algorithms that described current image to be compensated is carried out resolution compensation.
Wherein, the described super-resolution algorithms in step 603 is specifically as follows the super-resolution algorithms based on iterative backprojection.
In the present embodiment, entering image by the video flowing at polarization type 3D television set adjusts before the circuit, just first image to be compensated in the observation LR image sets compensated, can obtain the HR image of first image correspondence to be compensated, thereby when on the 3D display screen, showing the video flowing of this HR image composition, can promote the resolution of original image, thereby make video flowing in playing process, can promote user's viewing experience.
With reference to figure 7, show a kind of structured flowchart that is applied to the resolution compensation method embodiment 2 of 3D rendering demonstration of the present invention, can may further comprise the steps:
Step 701: obtain observation low resolution LR image sets according to the time frame order.
Step 702: with first width of cloth image in the described observation LR image sets as current image to be compensated.
Step 703: the sub-pixel displacement amount of calculating reference picture and described current image to be compensated; Described reference picture is other LR images except described current image to be compensated in the observation LR image sets.
Step 704: utilize following formula that described current image to be compensated is carried out bilinear interpolation and handle:
F (x, y)=[f (1,0)-f (0,0)] x+[f (0,1)-f (0,0)] y+[f (1,1)+f (0,0)-f (0,1)-f (1,0)] xy+f (0,0); To obtain and the corresponding initial high resolution HR image of described current image to be compensated.
Step 705: according to described sub-pixel displacement amount and default interpolation factor to the processing that degrades of described initial HR image, to obtain and described observation LR image sets corresponding simulating LR image sets.
Step 706: whether the error amount of judging described observation LR image sets and simulation LR image sets less than predetermined threshold value, if, then enter step 707, if not, then enter step 708.
Step 707: with the final compensating images of described initial HR image as current image to be compensated.
The process that step 703～step 707 is carried out once can be called iteration once.
Step 708: the error image back projection that will observe LR image and corresponding simulating LR image is to described initial HR image.
The process that step 703～step 708 is carried out once also can be called iteration once, and just execution in step 707 still is a step 708 after step 706, with the judged result of step 706 relation is arranged.
With reference to shown in Figure 8, described step 708 specifically can may further comprise the steps:
Step 801: the error image that calculates described observation LR image and corresponding simulating LR image.
Step 802: according to default interpolation factor described error image is carried out interpolation and handle.
Step 803: on the described initial HR image that will be added to through the error image that interpolation is handled.
Step 709: whether judge the current iteration number of times greater than default maximum iteration time, if, then enter step 710, if not, then return step 705.
Described iterations refers to the execution number of times of step 703～step 707, perhaps the execution number of times of step 703～step 708.
Step 710: the initial HR image that will handle through back projection is as the final compensating images of current image to be compensated.
Need to prove, for aforesaid each method embodiment, for simple description, so it all is expressed as a series of combination of actions, but those skilled in the art should know, the present invention is not subjected to the restriction of described sequence of movement, because according to the present invention, some step can adopt other orders or carry out simultaneously.Secondly, those skilled in the art also should know, the embodiment described in the specification all belongs to preferred embodiment, and related action and module might not be that the present invention is necessary.
Need to prove that each embodiment in this specification all adopts the mode of going forward one by one to describe, what each embodiment stressed all is and the difference of other embodiment that identical similar part is mutually referring to getting final product between each embodiment.For method class embodiment, because it is similar substantially to device embodiment, so description is fairly simple, relevant part gets final product referring to the part explanation of device embodiment.
One of ordinary skill in the art will appreciate that all or part of step that realizes in the foregoing description method is to instruct relevant hardware to finish by program, described program can be stored in the computer read/write memory medium, this program is when carrying out, comprise the steps: (step of method), described storage medium, as: ROM/RAM, magnetic disc, CD etc.
Need to prove, in this article, relational terms such as first and second grades only is used for an entity or operation are made a distinction with another entity or operation, and not necessarily requires or hint and have the relation of any this reality or in proper order between these entities or the operation.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thereby make and comprise that process, method, article or the equipment of a series of key elements not only comprise those key elements, but also comprise other key elements of clearly not listing, or also be included as this process, method, article or equipment intrinsic key element.Do not having under the situation of more restrictions, the key element that limits by statement " comprising ... ", and be not precluded within process, method, article or the equipment that comprises described key element and also have other identical element.
More than be applied to resolution compensation device, 3D television set and method that 3D rendering shows and be described in detail provided by the present invention, used specific case herein principle of the present invention and execution mode are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, the part that all can change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.
Claims (11)
1. one kind is applied to the resolution compensation device that 3D rendering shows, it is characterized in that, comprising:
Acquisition module is used for obtaining observation low resolution LR image sets according to the time frame order;
Module is set, is used for observing the LR image as current image to be compensated first width of cloth of described observation LR image sets;
The resolution compensation module is used to adopt super-resolution algorithms that described current image to be compensated is carried out resolution compensation.
2. device as claimed in claim 1 is characterized in that, described resolution compensation module is used to adopt the super-resolution algorithms based on iterative backprojection that described current image to be compensated is carried out resolution compensation.
3. device as claimed in claim 2 is characterized in that, described resolution compensation module comprises:
Calculate sub-pixel displacement amount module, be used to calculate the sub-pixel displacement amount of reference picture and described current image to be compensated; Described reference picture is other observation LR image except described current image to be compensated in the observation LR image sets;
The bilinear interpolation processing module is used for that described current image to be compensated is carried out bilinear interpolation and handles, to obtain and the corresponding initial high resolution HR image of described current image to be compensated;
The processing module that degrades is used for according to described sub-pixel displacement amount and default interpolation factor to the processing that degrades of described initial HR image, to obtain and described observation LR image sets corresponding simulating LR image sets;
Whether first judge module, the error amount that is used to judge described observation LR image sets and simulation LR image sets less than predetermined threshold value, if, then with the final compensating images of described initial HR image as current image to be compensated; If not, then trigger back projection's module;
Back projection's module, be used for will observation LR image and corresponding simulating LR image error image back projection to described initial HR image, and trigger second judge module;
Whether second judge module is used to judge the current iteration number of times greater than default maximum iteration time, if, then will be through the initial HR image of back projection's processing final compensating images as current image to be compensated; If not, then trigger the described processing module that degrades.
4. device as claimed in claim 3 is characterized in that, described back projection module comprises:
Calculating sub module is used to calculate the described error image of observing LR image and corresponding simulating LR image;
The interpolation processing sub is used for according to default interpolation factor described error image being carried out interpolation and handles;
The stack submodule is used for the described initial HR image that will be added to through the error image of interpolation processing.
5. as claim 3 or 4 described devices, it is characterized in that described bilinear interpolation processing module is used to utilize following formula that described current image to be compensated is carried out the bilinear interpolation processing:
f(x，y)＝[f(1，0)-f(0，0)]x+[f(0，1)-f(0，0)]y+[f(1，1)+f(0，0)-f(0，1)-f(1，0)]xy+f(0，0)。
6. a 3D television set is characterized in that, comprising: as each described resolution compensation device of claim 1-5, described resolution compensation device is arranged between the video input and transport module and polarization type 3D display module of described 3D television set.
7. one kind is applied to the resolution compensation method that 3D rendering shows, it is characterized in that, comprising:
Obtain observation low resolution LR image sets according to the time frame order;
With first width of cloth image in the described observation LR image sets as current image to be compensated;
Adopt super-resolution algorithms that described current image to be compensated is carried out resolution compensation.
8. method as claimed in claim 7 is characterized in that, described super-resolution algorithms is the super-resolution algorithms based on iterative backprojection.
9. method as claimed in claim 8 is characterized in that, described employing is carried out resolution compensation based on the super-resolution algorithms of iterative backprojection to described current image to be compensated, comprising:
Calculate the sub-pixel displacement amount of reference picture and described current image to be compensated; Described reference picture is other LR images except described current image to be compensated in the observation LR image sets;
Described current image to be compensated is carried out bilinear interpolation handle, to obtain and the corresponding initial high resolution HR image of described current image to be compensated;
According to described sub-pixel displacement amount and default interpolation factor to the processing that degrades of described initial HR image, to obtain and described observation LR image sets corresponding simulating LR image sets;
Whether the error amount of judging described observation LR image sets and simulation LR image sets less than predetermined threshold value, if, then with the final compensating images of described initial HR image as current image to be compensated; If not, then will observe the error image back projection of LR image and corresponding simulating LR image to described initial HR image; And at the current iteration number of times during greater than default maximum iteration time, will be through the initial HR image of back projection's processing final compensating images as current image to be compensated; Perhaps when the current iteration number of times is not more than default maximum iteration time, carry out the step that described sub-pixel displacement amount of described foundation and default interpolation factor degrade and handle described initial HR image.
10. method as claimed in claim 9 is characterized in that, the described error image back projection that will observe LR image and corresponding simulating LR image comprises to described initial HR image:
Calculate the error image of described observation LR image and corresponding simulating LR image;
According to default interpolation factor described error image being carried out interpolation handles;
To be added to through the error image that interpolation is handled on the described initial HR image.
11., it is characterized in that as claim 9 or 10 described methods, describedly described current image to be compensated is carried out bilinear interpolation handle, for:
Utilizing following formula that described current image to be compensated is carried out bilinear interpolation handles:
f(x，y)＝[f(1，0)-f(0，0)]x+[f(0，1)-f(0，0)]y+[f(1，1)+f(0，0)-f(0，1)-f(1，0)]xy+f(0，0)。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010620694 CN102036095A (en) | 2010-12-31 | 2010-12-31 | Resolution compensating device and method applied to three-dimensional (3D) image display and 3D television |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201010620694 CN102036095A (en) | 2010-12-31 | 2010-12-31 | Resolution compensating device and method applied to three-dimensional (3D) image display and 3D television |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102036095A true CN102036095A (en) | 2011-04-27 |
Family
ID=43888312
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201010620694 Pending CN102036095A (en) | 2010-12-31 | 2010-12-31 | Resolution compensating device and method applied to three-dimensional (3D) image display and 3D television |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102036095A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102360498A (en) * | 2011-10-27 | 2012-02-22 | 江苏省邮电规划设计院有限责任公司 | Reconstruction method for image super-resolution |
| CN103379351A (en) * | 2012-04-28 | 2013-10-30 | ***通信集团山东有限公司 | Video processing method and video processing device |
| CN103780863A (en) * | 2014-01-17 | 2014-05-07 | Tcl集团股份有限公司 | High-resolution image generation method and device |
| CN105100778A (en) * | 2015-08-31 | 2015-11-25 | 深圳凯澳斯科技有限公司 | Method and device for converting multi-view stereoscopic video |
| CN107155096A (en) * | 2017-04-19 | 2017-09-12 | 清华大学 | A kind of super resolution ratio reconstruction method and device based on half error back projection |
| CN107771394A (en) * | 2015-06-19 | 2018-03-06 | 微软技术许可有限责任公司 | The application streaming of the low latency of usage time frame conversion |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1529500A (en) * | 2003-10-16 | 2004-09-15 | 上海交通大学 | Three-dimensional video format conversion method based on motion adaption and marginal protection |
| US20050001930A1 (en) * | 2003-07-01 | 2005-01-06 | Ching-Lung Mao | Method of using three-dimensional image interpolation algorithm to achieve frame rate conversions |
| KR20050049623A (en) * | 2003-11-22 | 2005-05-27 | 한국전자통신연구원 | Apparatus and method for display of stereo-scopic video |
-
2010
- 2010-12-31 CN CN 201010620694 patent/CN102036095A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050001930A1 (en) * | 2003-07-01 | 2005-01-06 | Ching-Lung Mao | Method of using three-dimensional image interpolation algorithm to achieve frame rate conversions |
| CN1529500A (en) * | 2003-10-16 | 2004-09-15 | 上海交通大学 | Three-dimensional video format conversion method based on motion adaption and marginal protection |
| KR20050049623A (en) * | 2003-11-22 | 2005-05-27 | 한국전자통신연구원 | Apparatus and method for display of stereo-scopic video |
Non-Patent Citations (1)
| Title |
|---|
| 《计算机科学与探索》 20090531 郭伟伟,章品正 《基于迭代反投影的超分辨率图像重建》 第3卷, 第3期 2 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102360498A (en) * | 2011-10-27 | 2012-02-22 | 江苏省邮电规划设计院有限责任公司 | Reconstruction method for image super-resolution |
| CN102360498B (en) * | 2011-10-27 | 2013-09-18 | 江苏省邮电规划设计院有限责任公司 | Reconstruction method for image super-resolution |
| CN103379351A (en) * | 2012-04-28 | 2013-10-30 | ***通信集团山东有限公司 | Video processing method and video processing device |
| CN103379351B (en) * | 2012-04-28 | 2016-03-02 | ***通信集团山东有限公司 | A kind of method for processing video frequency and device |
| CN103780863A (en) * | 2014-01-17 | 2014-05-07 | Tcl集团股份有限公司 | High-resolution image generation method and device |
| CN103780863B (en) * | 2014-01-17 | 2017-08-25 | Tcl集团股份有限公司 | A kind of high-definition picture generation method and device |
| CN107771394A (en) * | 2015-06-19 | 2018-03-06 | 微软技术许可有限责任公司 | The application streaming of the low latency of usage time frame conversion |
| US10554713B2 (en) | 2015-06-19 | 2020-02-04 | Microsoft Technology Licensing, Llc | Low latency application streaming using temporal frame transformation |
| CN105100778A (en) * | 2015-08-31 | 2015-11-25 | 深圳凯澳斯科技有限公司 | Method and device for converting multi-view stereoscopic video |
| CN107155096A (en) * | 2017-04-19 | 2017-09-12 | 清华大学 | A kind of super resolution ratio reconstruction method and device based on half error back projection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11438566B2 (en) | Three dimensional glasses free light field display using eye location | |
| CN102036095A (en) | Resolution compensating device and method applied to three-dimensional (3D) image display and 3D television | |
| Takahashi et al. | From focal stack to tensor light-field display | |
| US9924153B2 (en) | Parallel scaling engine for multi-view 3DTV display and method thereof | |
| KR101137753B1 (en) | Methods for fast and memory efficient implementation of transforms | |
| US20130063571A1 (en) | Image processing apparatus and image processing method | |
| US20050185048A1 (en) | 3-D display system, apparatus, and method for reconstructing intermediate-view video | |
| US20100290529A1 (en) | Real-time superresolution and video transmission | |
| US20110249889A1 (en) | Stereoscopic image pair alignment apparatus, systems and methods | |
| US11570418B2 (en) | Techniques for generating light field data by combining multiple synthesized viewpoints | |
| JPH07193746A (en) | Image processing system | |
| US9483812B2 (en) | Enhancing motion pictures with accurate motion information | |
| Jeong et al. | Multi-frame example-based super-resolution using locally directional self-similarity | |
| CN104320647A (en) | Three-dimensional image generating method and display device | |
| US8149913B2 (en) | Moving picture converting apparatus and method, and computer program | |
| CN114514746B (en) | System and method for motion adaptive filtering as pre-processing for video encoding | |
| US20150035953A1 (en) | Electronic display numerical aberration correction | |
| CN104732491A (en) | Edge priority guide single-frame remote sensing image super-resolution processing method | |
| KR101657367B1 (en) | Method for generating computer generated hologram using look-up table and block-based motion compensation | |
| CN103079074B (en) | Block matching method, method for estimating and module, device | |
| CN115170402A (en) | Frame insertion method and system based on cyclic residual convolution and over-parameterized convolution | |
| CN107155096B (en) | A kind of super resolution ratio reconstruction method and device based on half error back projection | |
| Takeda et al. | Locally adaptive kernel regression for space-time super-resolution | |
| CN106507089A (en) | Method and apparatus for generating image | |
| TWI533663B (en) | Apparatus and method for processing 3-d image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20110427 |