KR102022527B1

KR102022527B1 - Stereoscopic image display device and disparity calculation method thereof

Info

Publication number: KR102022527B1
Application number: KR1020130116836A
Authority: KR
Inventors: 이승용; 권경준; 허천
Original assignee: 엘지디스플레이 주식회사
Priority date: 2013-09-30
Filing date: 2013-09-30
Publication date: 2019-11-04
Also published as: KR20150037319A

Abstract

The stereoscopic image display device according to the present invention indicates a difference between first monocular image data to be displayed on an Nth line (N is a positive integer) and second monocular image data located within a first range from the first monocular image data. A first cost calculator for calculating an AD value; A second cost calculator configured to calculate a census value using the first monocular data and its peripheral data and the second monocular image data and its peripheral data; A third cost calculator configured to calculate a smoothness value based on an initial disparity obtained from the N-th line adjacent to the N-th line; An initial matching value calculator configured to calculate an initial matching value by adding the AD value, the census value, and the smoothness value; An initial matching sum value calculating unit configured to calculate an initial matching sum value by adding an initial matching value of a current pixel with initial matching values of a peripheral area thereof; And an initial disparity calculator configured to calculate a minimum displacement from among the initial matching sum values as an initial disparity of the current pixel.

Description

Stereoscopic Display and Disparity Calculation Method {STEREOSCOPIC IMAGE DISPLAY DEVICE AND DISPARITY CALCULATION METHOD THEREOF}

The present invention relates to a stereoscopic image display device for generating multi-view image data from 3D image data for implementing a stereoscopic image, and a disparity calculation method thereof.

Recently, as interest in 3D stereoscopic images increases, various stereoscopic image display apparatuses have been developed. In general, the three-dimensional sense perceived by a person is caused by the degree of change in the thickness of the lens depending on the position of the object to be observed, the difference in angle between the two eyes and the object, and the difference in the position and shape of the visible objects in the left and right eyes, A parallax and other effects caused by various psychological and memory effects are combined. Among them, binocular disparity, which appears as the human eyes are positioned about 6 to 7 cm apart in the horizontal direction, can be said to be the most important factor of the three-dimensional effect. That is, when binocular parallax produces images that are different from each other's eyes, the human brain can fuse these two pieces of information with each other to feel the original 3D stereoscopic image.

There are two methods of realizing stereoscopic images using binocular disparity. Among these, the glasses-free method generally uses optical plates such as parallax barriers and lenticular lenses to separate stereoscopic images of left and right parallax images to realize stereoscopic images. Because of the convenience that users can watch stereoscopic images without wearing shutter glasses or polarized glasses, the glasses-free method has recently been used in small and medium-sized displays such as smart phones, tablets, and notebooks. Is being applied. The autostereoscopic method implements a stereoscopic image by displaying a multiview image including n (n is a natural number of two or more) view images in n view regions using an optical plate to reduce 3D crosstalk. 3D crosstalk means that a plurality of view images are superimposed on the user's single eye (left eye or right eye), and as the 3D crosstalk increases, the quality of the stereoscopic image decreases.

The multi-view image may be generated by capturing an image of an object and separating n cameras by the binocular spacing of the general public. Multi-view images are not as easy to produce as video content compared to 3D images including left and right eye images (or two view images). Video content is lacking a lot. Accordingly, a method of generating a multi-view image using a left eye image and a right eye image of a 3D image has been widely used. In order to generate a multi-view image using a 3D image, a disparity map should be calculated by first analyzing a left eye image and a right eye image of the 3D image. Disparity refers to a parallax (or coordinate difference) of pixels for shifting a left eye image and a right eye image to form a three-dimensional effect. In order to obtain a disparity map, it is necessary to obtain a disparity through stereo matching between a left eye image and a right eye image.

FIG. 1A is a first reference image and FIG. 1B is an exemplary diagram of a disparity map obtained from the first reference image. 2A is a second reference image and FIG. 2B is an exemplary diagram of a disparity map obtained from the second reference image. The disparity maps of FIGS. 1B and 2B include initial disparities represented by gray level values.

In FIG. 1B, the circled portions indicate flat regions of low depth in the reference image of FIG. 1A. According to the existing disparity calculation method, the disparity is not calculated at a constant value in these portions, and the disparity is incorrectly calculated in some of the regions as shown in FIG. 1B.

In FIG. 2B, the circled portions indicate a flat area, etc., in which image matching is poor in the reference image of FIG. 2A. In the case of the conventional disparity calculation method, the disparity is not calculated at a constant value in these parts, but is incorrectly calculated in some of the regions as shown in FIG. 2B.

As described above, in the conventional disparity calculation process, a large amount of noise is included in the flat areas. The noise components of these flat areas are not corrected correctly even after the post-processing process, which reduces the accuracy of stereo matching.

Accordingly, it is an object of the present invention to provide a stereoscopic image display device and a disparity calculation method thereof to improve stereo matching accuracy in flat areas.

In order to achieve the above object, a stereoscopic image display device according to an embodiment of the present invention is located within the first range from the first monocular image data and the first monocular image data to be displayed on the N-th (N is a positive integer) line A first cost calculator configured to calculate an AD value indicating a difference between the second monocular image data; A second cost calculator configured to calculate a census value using the first monocular data and its peripheral data and the second monocular image data and its peripheral data; A third cost calculator configured to calculate a smoothness value based on an initial disparity obtained from the N-th line adjacent to the N-th line; An initial matching value calculator configured to calculate an initial matching value by adding the AD value, the census value, and the smoothness value; An initial matching sum value calculating unit configured to calculate an initial matching sum value by adding an initial matching value of a current pixel with initial matching values of a peripheral area thereof; And an initial disparity calculator configured to calculate a minimum displacement from among the initial matching sum values as an initial disparity of the current pixel.

The disparity calculation method of the stereoscopic image display apparatus according to the embodiment of the present invention is located within a first range from the first monocular image data to be displayed on the Nth line (N is a positive integer) and the first monocular image data. Calculating an AD value indicating a difference between the second monocular image data; Calculating a census value using the first monocular data and its peripheral data and the second monocular image data and its peripheral data; Calculating a smoothness value based on an initial disparity obtained from an N-th line adjacent to the N-th line; Calculating an initial matching value by adding the AD value, the census value, and the smoothness value; Calculating an initial matching sum value by summing an initial matching value of the current pixel with initial matching values of the surrounding area; And calculating a minimum displacement among the initial matching sum values as an initial disparity of the current pixel.

The present invention calculates an initial matching value by considering the smoothness value in addition to the AD value and the census value, and calculates the initial disparities based on the initial matching value, thereby reducing the amount of noise included in the flat areas to reduce the flat area. To increase the accuracy of stereo matching. Furthermore, the present invention can further improve the accuracy of stereo matching in the flat areas by further reducing the amount of noise included in the flat areas by further correcting the initial disparity based on the reliability.

1A and 1B illustrate an example of a disparity map obtained from a first reference image and a first reference image, respectively.
2A and 2B illustrate an example of a disparity map obtained from a second reference image and a second reference image, respectively.
3 is a block diagram schematically illustrating a stereoscopic image display device according to an exemplary embodiment of the present invention.
Figure 4 is an exemplary view showing a stereoscopic image implementation method of the autostereoscopic 3D display device according to an embodiment of the present invention.
5 is a block diagram illustrating in detail the image processing circuit of FIG. 3.
6 is an exemplary diagram illustrating left eye image data, right eye image data, and view image data according to an embodiment of the present invention.
7 is a block diagram showing in detail the disparity calculator of FIG.
8 is a flowchart illustrating a disparity calculation method of a disparity calculator in detail.
9 is a block diagram illustrating in detail an initial disparity generator.
10 is a flowchart illustrating a method of generating an initial disparity in an initial disparity generating unit in detail.
11 illustrates an operation of a first cost calculator.
12 illustrates an operation of a second cost calculator.
13A to 14 illustrate operations of the third cost calculator.
15 is a view showing the operation of the summing kernel component.
16 is a diagram illustrating an operation of an initial matching sum value calculating unit.
17 illustrates an operation of an initial disparity calculator.
18A to 18C illustrate an operation of an initial disparity corrector.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Like numbers refer to like elements throughout. In the following description, when it is determined that a detailed description of known functions or configurations related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. Component names used in the following description may be selected in consideration of ease of specification, and may be different from actual product part names.

3 is a block diagram schematically illustrating a stereoscopic image display device according to an exemplary embodiment of the present invention. Referring to FIG. 3, a stereoscopic image display device according to an exemplary embodiment of the present invention may include a display panel 10, an optical plate 30, a gate driving circuit 110, a data driving circuit 120, a timing controller 130, An image processing circuit 140, a host system 150, and the like. The display panel 10 of the stereoscopic image display device according to an embodiment of the present invention is a liquid crystal display (LCD), a field emission display (FED), a plasma display panel (PDP) ), And a flat panel display device such as an organic light emitting diode (OLED). In the following embodiment, the display panel 10 is exemplarily implemented as a liquid crystal display device, but the present invention is not limited thereto.

The display panel 10 includes an upper substrate and a lower substrate facing each other with the liquid crystal layer interposed therebetween. The display panel 10 includes a pixel array including liquid crystal cells arranged in a matrix by a cross structure of the data lines D and the gate lines G (or scan lines). Each pixel of the pixel array drives the liquid crystal of the liquid crystal layer by adjusting a voltage difference between a pixel electrode charged with a data voltage through a TFT (Thin Film Transistor) and a common electrode applied with a common voltage, thereby adjusting the amount of light transmitted. Display. The black matrix and the color filter are formed on the upper substrate of the display panel 10. The common electrode is formed on the upper substrate in the case of a vertical electric field driving method such as twisted nematic (TN) mode and vertical alignment (VA) mode, and is similar to the in-plane switching (IPS) mode and the ring field switching (FFS) mode In the case of the horizontal electric field driving method, it may be formed on the lower substrate together with the pixel electrode. The liquid crystal mode of the display panel 10 may be implemented in any liquid crystal mode as well as a TN mode, a VA mode, an IPS mode, and an FFS mode. A polarizing plate is attached to each of the upper and lower substrates of the display panel 10 to form an alignment layer for setting a pre-tilt angle of the liquid crystal. A spacer is formed between the upper substrate and the lower substrate of the display panel 10 to maintain a cell gap of the liquid crystal layer.

The display panel 10 may be implemented in any form, such as a transmissive liquid crystal display panel, a transflective liquid crystal display panel, or a reflective liquid crystal display panel. In the transmissive liquid crystal display panel and the transflective liquid crystal display panel, a backlight unit is required. The backlight unit may be implemented as a direct type backlight unit or an edge type backlight unit.

The multiview image includes first to kth (k is 3 or more natural numbers) view images. The multi-view image may be generated by spaced apart by k cameras and capturing an image of an object by the binocular spacing of the general public. The optical plate 30 is disposed on the display panel 10 to advance the first to k th view images displayed on the pixels of the display panel 10 to the first to k th view regions. The first to k th view images are matched one-to-one with the first to k th view regions. That is, the optical plate 30 advances the t-th (t is a natural number satisfying 1 ≦ t ≦ k) displayed on the pixels to the t-th view area. The optical plate 30 of the stereoscopic image display device according to an embodiment of the present invention may be any parallax barrier, a switchable barrier, a lenticular lens, a switchable lens, or the like. It may also be implemented in the form. Meanwhile, when the optical plate 30 is implemented as a switchable barrier or a switchable lens, an optical plate driving circuit for driving the optical plate 30 is required. The optical plate driving circuit may turn on / off the optical separation operation of the switchable barrier or the switchable lens by supplying a driving voltage to the optical plate 30. Hereinafter, the stereoscopic image implementation method using the optical plate 30 will be described in detail with reference to FIG. 4.

4 is an exemplary view illustrating a stereoscopic image implementation method of an autostereoscopic 3D display device according to an exemplary embodiment of the present invention. In FIG. 4, for convenience of explanation, the display panel 10 displays four view images V1, V2, V3, and V4, and the optical plate 30 displays four view images displayed on the display panel 10. The description has been made mainly on advancing (V1, V2, V3, V4) into four view regions VP1, VP2, VP3, VP4. In FIG. 4, the optical plate 30 is exemplarily implemented as a lenticular lens, but the optical plate 30 according to the embodiment of the present invention may be implemented in any form such as a parallax barrier, a switchable barrier, a switchable lens, and the like. It should be noted.

Referring to FIG. 4, the optical plate 30 advances the first view image V1 displayed on the pixels to the first view area VP1 and removes the second view image V2 displayed on the pixels. The display proceeds to the second view area VP2, the third view image V3 displayed on the pixels is advanced to the third view area VP3, and the fourth view image V4 displayed on the pixels is viewed in the fourth view. Proceed to area VP4. When the left eye of the user is located in the t-th view area VPt and the right eye is located in the t-1 view area VPt-1, the user views the t-view image Vt with the left eye, The t-1 view image Vt-1 may be viewed. Therefore, the user can feel a three-dimensional effect by binocular parallax. For example, when the left eye of the user is located in the second view area VP2 and the right eye is located in the first view area VP1 as shown in FIG. 3, the user views the second view image V2 with the left eye and the right eye. Since the first view image V1 may be viewed, the user may feel a three-dimensional effect due to binocular parallax.

The data driving circuit 120 includes a plurality of source drive integrated circuits (hereinafter, referred to as ICs). The source drive ICs convert the multi-view image data MVD into positive / negative gamma compensation voltages under the control of the timing controller 130 to generate positive / negative analog data voltages. The positive / negative analog data voltages output from the source drive ICs are supplied to the data lines D of the display panel 10.

The gate driving circuit 110 sequentially supplies gate pulses (or scan pulses) to the gate lines G of the display panel 10 under the control of the timing controller 130. The gate driver 110 may include a plurality of gate drive integrated circuits each including a shift register, a level shifter for converting an output signal of the shift register into a swing width suitable for driving a TFT of a liquid crystal cell, and an output buffer. have. The gate drive integrated circuit may be directly formed on the lower substrate of the display panel 10 according to a gate driver in panel (GIP) method.

The timing controller 130 receives the multi-view image data MVD and timing signals from the image processing circuit 140. The timing signals may include a vertical synchronization signal, a horizontal synchronization signal, a data enable signal, a clock signal, and the like.

The timing controller 130 generates a gate control signal GCS for controlling the gate driving circuit 110 based on the timing signals, and generates a data control signal DCS for controlling the data driving circuit 120. do. The timing controller 130 supplies the gate control signal GCS to the gate driving circuit 110. The timing controller 130 supplies the multi-view image data MVD and the data control signal DCS to the data driving circuit 120.

The host system 150 includes a system on a chip having a built-in scaler to convert 3D image data RGB3D input from an external video source device into a data format having a resolution suitable for display on the display panel 10. on Chip). The host system 150 supplies 3D image data RGB3D and timing signals to the image processing circuit 140 through interface circuits such as a low voltage differential signaling (LVDS) interface and a transition minimized differential signaling (TMDS) interface.

The image processing circuit 140 generates the multiview image data MVD from the 3D image data RGB3D received from the host system 150 and outputs the multiview image data MVD to the timing controller 130. The 3D image data RGB3D includes first monocular image data and second monocular image data (or two view image data). Hereinafter, for convenience of description, the first monocular image data is left eye image data and the second monocular image data is right eye image data.

As a result, the 3D image display apparatus according to an exemplary embodiment generates multi-view image data (MVD) by using the image processing circuit 140, even if the 3D image data RGB3D is input to the display panel 10. The multi view image can be displayed. As a result, the stereoscopic image display device according to an embodiment of the present invention can increase the quality of the stereoscopic image. Hereinafter, a method of generating multiview image data (MVD) of the image processing circuit 140 will be described in detail with reference to FIGS. 5 and 6.

5 is a block diagram illustrating in detail the image processing circuit 140 of FIG. 3. 6 is an exemplary diagram illustrating left eye image data, right eye image data, and view image data according to an exemplary embodiment of the present invention.

Referring to FIG. 5, the image processing circuit 140 includes a disparity calculator 200 and a multi-view image generator 300.

5 and 6, the disparity calculator 200 uses the left eye image data RGBL and the right eye image data RGBR of the 3D image data RGB3D to determine the disparities DIS. Calculate and output The disparity DIS refers to a value for shifting the left eye image data RGBL or the right eye image data RGBR to form a three-dimensional effect. A detailed configuration of the disparity calculator 200 and a disparity calculation method thereof will be described later with reference to FIGS. 7 and 8.

The multi-view image generator 300 shifts the left eye image data RGBL or the right eye image data RGBR according to the disparities DIS calculated by the disparity calculator 200 to multi-view image data MVD. Create In detail, the multi-view image generator 300 sets the left eye image data RGBL to the first view image data V1 and the right eye image data RGBR to the k-th view image data Vk as shown in FIG. 6. And shifting the left eye image data RGBL or the right eye image data RGBR using the disparities DIS and generating second to k-1th view image data V2 to Vk-1. The multi-view image data MVD including three view image data may be generated. For example, the t-th view image data Vt may be generated by shifting the left eye image data RGBL in the first horizontal direction by a value obtained by multiplying the disparities DIS by (t / k−1).

The multi-view image generating method of the multi-view image generator 300 may be applied to any method known in the art. In addition, the multi-view image generator 300 may arrange the multi-view image data MVD in accordance with the 3D display arrangement of the display panel 10 using the 3D formatter and then output the multi-view image generator 300 to the timing controller 130.

FIG. 7 is a block diagram illustrating in detail the disparity calculator 200 of FIG. 5. 8 is a flowchart illustrating a disparity calculation method of the disparity calculator 200 in detail.

Referring to FIG. 7, the disparity calculator 200 includes a gain value generator 210, an initial disparity generator 220, and a post processor 230. Hereinafter, the disparity calculation method of the disparity calculator according to an embodiment of the present invention will be described in detail with reference to FIGS. 7 and 8.

The gain value generating unit 210 calculates the gain value G of the Nth frame by analyzing the disparities, the left eye image data, and the right eye image data of the Nth frame. The gain value generator 210 may include a memory for storing the gain value G. The gain value G calculated in the N-th frame is stored in the memory. The gain value generator 210 outputs the gain value calculated in the N-th frame to the initial disparity generator 220 during the N-th frame, and stores the gain value G calculated in the N-th frame in the memory. (S201)

The initial disparity generator 220 receives the gain value G from the gain value generator 210 and receives the left eye image data RGBL and the right eye image data RGBR of the Nth frame from the host system 150. 3D image data (RGB3D) is included. The initial disparity generator 220 calculates initial disparities IDIS using the gain value G, the left eye image data RGBL, and the right eye image data RGBR of the Nth frame (S202).

The initial disparity generator 220 calculates an initial matching value according to a new method as shown in FIGS. 9 and 10, and calculates initial disparities IDIS based on the initial matching value, thereby providing a flat area. By reducing the amount of noise included, the accuracy of stereo matching in flat areas can be improved. Furthermore, the initial disparity generator 220 further performs a correction process on the initial disparity IDIS calculated as shown in FIGS. 9 and 10, thereby further reducing the amount of noise included in the flat regions to further reduce the flat regions. Can further increase the accuracy of stereo matching. A detailed description of an initial disparity IDIS generation method of the initial disparity generator 220 will be described later with reference to FIGS. 9 to 18C.

The post processor 230 post-processes the initial disparities IDIS to calculate the disparities DIS. The post-processing unit 230 post-processes the initial disparities IDIS using any one of various filters such as a median filter, a weighted median filter, and a weighted voting filter. can do. The median filter is a filter that converts data at the center coordinates of the mask into a median of data in the mask. The weighted median filter is a filter that arranges the data in the mask by applying the weight of the weighted mask, selects a median value, and converts data at the center coordinates in the mask to the median value. The weighted mode filter is a filter that selects a mode after generating a histogram by applying a weight of a weighted mask to data in the mask, and converts data at the center coordinates in the mask into the mode. The post processor 230 outputs the post-processed disparities DIS to the multi-view image generator 300 (S203).

9 is a block diagram illustrating the initial disparity generator 220 in detail. 10 is a flowchart illustrating an initial disparity generation method of the initial disparity generator 220 in detail. 11 to 18C are diagrams for describing an initial disparity generating method in detail.

Referring to FIG. 9, the initial disparity generator 220 may include a first cost calculator 221, a second cost calculator 222, a third cost calculator 223, a line memory 224, and initial matching. A value calculator 225, a sum kernel configuration unit 226, an initial matching sum value calculator 227, an initial disparity calculator 228, and an initial disparity corrector 229 are included. Hereinafter, the initial disparity generation method of the initial disparity generator 220 will be described in detail with reference to FIGS. 10 and 11.

The initial disparity generator 220 may set one of the left eye image data and the right eye image data as reference image data, and set the other as the comparison image data to generate an initial disparity (IDIS). In the embodiment of the present invention, it should be noted that the left eye image data is reference image data and the right eye image data is comparative image data.

The first cost calculator 221 receives 3D image data RGB3D of the Nth frame from the host system 150. The 3D image data RGB3D of the Nth frame includes left eye image data RGBL for implementing the left image IL and right eye image data RGBR for implementing the right image IR. The first cost calculator 221 calculates an AD value by analyzing the left eye image data RGBL and the right eye image data RGBR of the Nth frame. The AD value means a difference between the left eye image data RGBL and the right eye image data RGBR located within a first range from the left eye image data RGBL.

The first cost calculator 221 sets center coordinates in the left eye image data that is the reference image data. For example, as illustrated in FIG. 11, the first cost calculator 221 may set the (x, y) coordinate as the center coordinate. In this case, the AD value calculator 221 uses the left eye image data RGBL (x, y) at the (x, y) coordinates and the right eye image data (RGBR () at the (xd, y) coordinates as shown in Equation (1). The absolute value of the difference of xd, y)) is calculated as AD value (Cad (x, y, d)) corresponding to (x, y, d). d has a value from 0 to d _max .

For example, if d _max is 60, the first cost calculator 221 determines left eye image data at (x, y) coordinates and right eye image at (x, y) to (x-60, y) coordinates. The difference of each data is calculated as AD values (Cad (x, y, 0) to Cad (x, y, 60)) corresponding to (x, y, 0) to (x, y, 60). (S221)

The second cost calculator 222 receives 3D image data RGB3D of the Nth frame from the host system 150. The second cost calculator 222 calculates a census value Ccen by using the left eye image data and its surrounding data, the right eye image data, and its surrounding data of the N-th frame.

In detail, the second cost calculator 222 sets the first census window CW1 based on the left eye image data at the (x, y) coordinate as the center coordinate CC as shown in FIG. 12. In FIG. 12, the first census window CW1 is implemented with a size of 3 × 3. However, the present invention is not limited thereto, and may be implemented with a size of p × q (p, q is a natural number). The second cost calculator 222 determines the first value when the left eye image data at one coordinate is greater than or equal to the left eye image data at the center coordinate CC in the first census window CW1. If it is smaller than that, the census transformation is performed to assign the second value to the value of the coordinate. The first value may be "1" and the second value may be "0". For example, as shown in FIG. 12, when the left eye image data of one coordinate in the first census window CW1 is greater than or equal to "85" which is the left eye image data at the center coordinate CC, the value of the coordinate is "." 1 is assigned, and if it is smaller than "85", "0" may be assigned as the value of the coordinate.

The second cost calculator 222 sets the second census window CW2 based on the right eye image data at the (x-d, y) coordinate as the center coordinate CC. In FIG. 12, the second census window CW1 is implemented with a size of 3 × 3. However, the present invention is not limited thereto and may be implemented with a size of p × q. The second cost calculator 222 allocates the first value to the coordinate value when the right eye image data at one coordinate is greater than or equal to the right eye image data at the center coordinate in the second census window CW2. If it is smaller than that, the census transformation is performed to assign the second value to the value of the coordinate. The first value may be "1" and the second value may be "0". For example, when the right eye image data of one coordinate is greater than or equal to "30", which is the right eye image data at the center coordinate, in the second census window CW2 as shown in FIG. If it is smaller than "30", "0" may be assigned as the value of the coordinate.

The second cost calculator 222 converts the census transformed values in the first census window CW1 into a first bit string BS1 and converts the census transformed in the second census window CW2 as shown in FIG. 12. After the values are made into the second bit string BS2, an exclusive OR operation is performed to form the third bit string BS3. The second cost calculator 222 calculates a census value Ccen (x, y, d) corresponding to (x, y, d) by summing bit values of the third bit string BS3. In FIG. 12, the census value Ccen (x, y, r) corresponding to (x, y, d) may be calculated as “2”. Meanwhile, the census value Ccen (x, y, d) corresponding to (x, y, d) sets the left eye image data RGBL (x, y) at (x, y) as the center coordinate. , census value calculated by setting the right eye image data RGBR (xd, y) at the (xd, y) coordinate as the center coordinate. (S221)

Subsequently, the third cost calculator 223 calculates a smoothness value Cs using the initial IDIS value obtained from the previous line in order to increase the accuracy of stereo matching in the flat areas. As shown in FIG. 13A, it is easy to calculate the initial disparity IDIS having the lowest value based on the AD value Ca and the census value Ccen calculated above in the edge region of the object. However, as shown in FIG. 13B, when the initial disparity IDIS is calculated based on the AD value Cad and the census value Ccen calculated above, noise may be generated due to a miscalculation problem. It gets worse. As shown in FIG. 13C, the initial disparity IDIS can be calculated relatively accurately only by the AD value Cad and the census value Ccen in the edge region, but the AD value Cad and the census value Ccen in the flat region. It is not possible to calculate the IDIS exactly by itself.

The third cost calculator 223 receives the initial disparity IDIS of the previous stage line (ie, the upper line, the N-th line) stored in the line memory 224, as shown in Equation 2 below. The smoothness value Cs is calculated by the difference between the initial disparity IDIS of the previous line and the initial disparity IDIS of the current line (N-th line, N is a positive integer).

In Equation 2, Cs (x, y, d) represents a smoothness value (Cs) corresponding to (x, y, d) at the present stage, and Dini (x, y-1) represents a (x, y) coordinate. Denotes the initial disparity (IDIS) at the (x, y-1) coordinates of the previous preceding line up, and d represents the initial disparity (IDIS) of the current end line. Csmax represents a preset maximum smoothness value Cs.

According to the present invention, an initial disparity (IDIS) -coast curve for a flat area is made as shown in FIG. 14 by using the smoothness value Cs calculated by the third cost calculator 223, thereby providing an initial disparity in the flat area. (IDIS) is calculated correctly (S222).

Subsequently, the initial matching value calculator 225 receives the AD value Cad from the first cost calculator 221, the census value Ccen from the second cost calculator 222, and receives a third cost. The smoothness value Cs is input from the calculator 223. The initial matching value calculating unit 225 applies the gain value G input from the gain value generating unit 210 to the AD value Cad, the census value Ccen, and the smoothness value Cs, and thus the initial matching value Cx. ) Is calculated. In detail, the initial matching value calculator 225 adds the AD value Ca, the census value Ccen, and the smoothness value Cs to which the gain value G is applied, as shown in Equation 3 below, to match the initial matching value Cx. (x, y, d))

When calculating the initial matching value Cx, the gain value G may be adjusted according to the edge degree of the image. To this end, the gain value generator 210 adds the gain values λad, λcen, and Cn according to the characteristics of the image when the three costs Cad, Ccen, and Cs are summed together to calculate the initial matching value Cx. λs) can be adjusted to improve the performance of the matched image. The gain value generator 210 increases the relative census value Ccen in the initial matching value Cx (x, y, d) as the image includes more edges or complex parts, and vice versa. In this case (ie, as the image includes more flat portions), the ratio of the smoothness value Cs to the initial matching value Cx (x, y, d) may be increased by increasing λs relatively. If this is expressed as an equation, Equation 4 below. λad may be preselected by the user between 0 and 1. FIG.

The sum of the gain values λad, λcen, and λs is 1, and the larger the change in the image, the greater the specificity of the census value Ccen, and the smaller the change in the image, the greater the specificity of the smoothness value Cs (S223).

The summing kernel configuration unit 226 constitutes an adaptive shape kernel. The adaptive shape kernel is composed of pixels whose color difference between the corresponding pixel and the neighboring pixel does not exceed a certain level, so that only initial matching values for the same object are added in the initial matching value summation process. For example, the adaptive shape kernel may be configured as shown in FIG. 15. The method of constructing the adaptive shape kernel is firstly searched vertically along Vx- and Vx + for different pixels, and secondly, hq- for different pixels starting from all pixels between Vx- and Vx +. And in the horizontal direction along hq +, and third, " Np " Here, "Np" indicates the peripheral area of the pixel for the summation.

The initial matching sum calculating unit 227 receives the initial matching value Cx from the initial matching value calculating unit 223, and receives the peripheral area Np for the summating from the summing kernel configuration unit 226. The initial matching sum calculator 227 calculates the initial matching sum Ex by adding the initial matching value Cx of the current pixel with the initial matching values of the peripheral area Np. In general, since images have similar colors in succession, it is difficult to find a corresponding point only by using an initial matching value using only the color difference between two pixels. Therefore, it is effective to perform matching using the current pixel and the surrounding pixels together. To this end, a corresponding point may be found by selecting disparity having a small sum of initial matching values of neighboring pixels.

In detail, the initial matching sum value calculating unit 227 sets a mask based on the initial matching value Cx (x, y) corresponding to (p (x, y), d) as shown in FIG. 16. The initial matching values of each of the coordinates in the mask are summed to calculate an initial matching sum value Ex (x, y, d) at (p (x, y), d). The mask may be implemented to include i × j (i, j is two or more natural numbers) initial matching values (S224).

Subsequently, the initial disparity calculator 228 receives initial matching sum values Ex from the initial matching sum calculator 227. The initial disparity calculator 228 calculates the initial disparity IDIS by analyzing the initial matching sum values Ex.

In detail, the initial disparity calculator 228 may include initial matching sum values Ex (x, y, 0) to Ex corresponding to (x, y, d) to (x, y, d _max ) as shown in FIG. 17. The displacement d of the initial matching sum value having the minimum value among (x, y, d _max )) is calculated as the initial disparity IDIS (x, y) in the p (x, y) coordinate. For example, the initial disparity calculator 228 may include initial matching sum values Ex (x, y, 0) to Ex (x, corresponding to (x, y, d) to (x, y, d _max ). y, d _max )), if the initial matching sum corresponding to (x, y, 10) has the minimum value, then "10" is the initial disparity in the (x, y) coordinates (IDIS (x, y)). It can be calculated as (S225)

The present invention may further include an initial disparity corrector 229 that corrects an initial disparity (IDIS) in order to further reduce the amount of noise included in the flat areas to further increase the accuracy of stereo matching in the flat areas. .

The initial disparity correcting unit 229 divides the image into a plurality of blocks, calculates a reliability level of the initial disparity IDIS of the current pixel, and then calculates the first reliability level of the current pixel. If it is lower than the reference value, the first average value of the initial disparities IDIS having a certain level or higher in the reliability level in the block including the current pixel is obtained. The initial disparity corrector 229 may replace the initial disparity of the current pixel with a first average initial disparity indicating the first average value. On the other hand, when the number of initial disparities IDIS having a certain level of confidence level or higher in the block including the current pixel is smaller than the second reference value, the first disparities of the initial disparities IDIS having a certain level or higher level of confidence level in the whole image are included. 2 Find the average. The initial disparity corrector 229 may replace the initial disparity of the current pixel with a second average initial disparity indicating the second average value.

To this end, the initial disparity corrector 229 calculates reliability of the initial disparities IDIS input from the initial disparity calculator 228. In detail, the initial disparity corrector 229 calculates a reliability level of each pixel by using a difference between the initial disparity of the left eye image data and the initial disparity of the right eye image data. The smaller the difference between the initial disparities of the left and right eyes, the higher the confidence level of the disparity. The initial disparity corrector 229 may create a reliability map by calculating the reliability of the initial disparities at all positions. This confidence map is shown at 18a. Referring to FIG. 18A, the reliability maps show the reliability of disparities represented by gray values. Higher gradation values mean higher reliability, and lower gradation values mean lower reliability. The initial disparity corrector 229 divides the image into n × m blocks as illustrated in FIG. 18B (4 × 4 blocks are illustrated in FIG. 18C), and the first reliability level of the current pixel is set in advance. Compare with baseline.

If the reliability level of the initial disparity for the current pixel is lower than the first reference value, the initial disparity corrector 229 may substitute the first average initial disparity or the second average initial disparity as described above. Can be output as the initial disparity for.

On the other hand, when the reliability level of the initial disparity for the current pixel is equal to or greater than the first reference value, the initial disparity correcting unit 229 outputs the initial disparity for the current pixel as it is (S226).

As described above, the present invention calculates an initial matching value in consideration of the smoothness value in addition to the AD value and the census value, and calculates initial disparities based on the initial matching value, thereby including noise included in the flat areas. Reducing the amount of H can increase the accuracy of stereo matching in flat areas. Furthermore, the present invention can further improve the accuracy of stereo matching in the flat areas by further reducing the amount of noise included in the flat areas by further correcting the initial disparity based on the reliability.

Those skilled in the art will appreciate that various changes and modifications can be made without departing from the technical spirit of the present invention. Therefore, the present invention should not be limited to the details described in the detailed description but should be defined by the claims.

10: display panel 30: optical plate
110: gate driving circuit 120: data driving circuit
130: timing controller 140: image processing circuit
150: host system 200: disparity calculator
210: gain value generation unit 220: initial disparity generation unit
221: first cost calculator 222: second cost calculator
223: third cost calculator 224: line memory
225: initial matching value calculation unit 226: summing kernel component
227: initial matching sum value calculating unit 228: initial disparity calculating unit
229: initial disparity correction unit 230: post-processing unit
300: multi view image generator

Claims

First cost calculation for calculating an AD value indicating a difference between first monocular image data to be displayed on an Nth line (N is a positive integer) and second monocular image data located within a first range from the first monocular image data part;
A second cost calculator configured to calculate a census value using the first monocular data and its peripheral data and the second monocular image data and its peripheral data;
A third cost calculator configured to calculate a smoothness value based on an initial disparity obtained from the N-th line adjacent to the N-th line;
An initial matching value calculator configured to calculate an initial matching value by adding the AD value, the census value, and the smoothness value;
An initial matching sum value calculating unit configured to calculate an initial matching sum value by adding an initial matching value of a current pixel with initial matching values of a peripheral area thereof; And
And an initial disparity calculator configured to calculate a minimum displacement from among the initial matching sum values as an initial disparity of a current pixel.

The method of claim 1,
The smoothness value may refer to a difference between an initial disparity of a neighboring pixel neighboring the current pixel and the initial disparity of the current pixel obtained in the N-th line.

The method of claim 1,
The third cost calculator calculates the initial matching value by adding the AD value to which the first gain value is applied, the census value to which the second gain value is applied, and the smoothness value to which the third gain value is applied;
The first to third gain values are adjusted differently according to the edge degree of the input image.

The method of claim 3, wherein
The higher the second gain value as the input image includes more edges or complex portions, the lower the third gain value;
And the second gain value is lower as the input image includes more flat portions, while the third gain value is higher.

The method of claim 1,
And an initial disparity corrector configured to correct an initial disparity of the current pixel based on a confidence level.

The method of claim 5,
The initial disparity correction unit,
After dividing an input image into a plurality of blocks, calculating a reliability level of the initial disparity for the current pixel, and when the calculated reliability level of the current pixel is lower than a first reference value, the current pixel is included. And obtaining a first average value of the initial disparities having a confidence level equal to or greater than a predetermined level in the block, and replacing the initial disparity of the current pixel with a first average initial disparity indicating the first average value. .

The method of claim 5,
The initial disparity correction unit,
After dividing an input image into a plurality of blocks, calculating a reliability level of the initial disparity for the current pixel, the number of initial disparities whose reliability level is higher than or equal to a predetermined level in the block including the current pixel is the second. In the case where the reference value is smaller than the reference value, a second average value of the initial disparities having a certain level of confidence level or higher in the entire image is obtained, and the initial disparity of the current pixel is substituted with a second average initial disparity indicating the second average value. Stereoscopic image display device.

Calculating an AD value indicating a difference between first monocular image data to be displayed on an Nth line (N is a positive integer) and second monocular image data located within a first range from the first monocular image data;
Calculating a census value using the first monocular data and its peripheral data and the second monocular image data and its peripheral data;
Calculating a smoothness value based on an initial disparity obtained from an N-th line adjacent to the N-th line;
Calculating an initial matching value by adding the AD value, the census value, and the smoothness value;
Calculating an initial matching sum value by summing an initial matching value of the current pixel with initial matching values of the surrounding area; And
And calculating a minimum displacement among the initial matching sum values as an initial disparity of a current pixel.

The method of claim 8,
The smoothness value means a difference between an initial disparity of a neighboring pixel neighboring the current pixel and an initial disparity of the current pixel obtained in the N-th line. Way.

The method of claim 8,
The calculating of the smoothness value may include calculating the initial matching value by adding the AD value to which the first gain value is applied, the census value to which the second gain value is applied, and the smoothness value to which the third gain value is applied. ;
The first to third gain values are adjusted differently according to the edge degree of the input image.

The method of claim 10,
The higher the second gain value as the input image includes more edges or complex portions, the lower the third gain value;
The second gain value is lowered as the input image includes more flat portions, while the third gain value is increased.

The method of claim 8,
And correcting an initial disparity of the current pixel based on a confidence level.

The method of claim 12,
Correcting the initial disparity of the current pixel,
After dividing an input image into a plurality of blocks, calculating a reliability level of the initial disparity for the current pixel, and when the calculated reliability level of the current pixel is lower than a first reference value, the current pixel is included. And obtaining a first average value of the initial disparities having a confidence level equal to or greater than a predetermined level in the block, and replacing the initial disparity of the current pixel with a first average initial disparity indicating the first average value. Disparity calculation method of the.

The method of claim 12,
Correcting the initial disparity of the current pixel,
After dividing an input image into a plurality of blocks, calculating a reliability level of the initial disparity for the current pixel, the number of initial disparities whose reliability level is higher than or equal to a predetermined level in the block including the current pixel is the second. In the case where the reference value is smaller than the reference value, a second average value of the initial disparities having a certain level of confidence level or higher in the entire image is obtained, and the initial disparity of the current pixel is substituted with a second average initial disparity indicating the second average value. A disparity calculation method of a stereoscopic image display apparatus.