CN102457722B - Processing method and device for Bayer image - Google Patents

Processing method and device for Bayer image Download PDF

Info

Publication number
CN102457722B
CN102457722B CN201010519524.3A CN201010519524A CN102457722B CN 102457722 B CN102457722 B CN 102457722B CN 201010519524 A CN201010519524 A CN 201010519524A CN 102457722 B CN102457722 B CN 102457722B
Authority
CN
China
Prior art keywords
pixel
pixel value
contiguous
pixels
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201010519524.3A
Other languages
Chinese (zh)
Other versions
CN102457722A (en
Inventor
李永盛
原顺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Allwinner Technology Co Ltd
Original Assignee
Allwinner Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Allwinner Technology Co Ltd filed Critical Allwinner Technology Co Ltd
Priority to CN201010519524.3A priority Critical patent/CN102457722B/en
Publication of CN102457722A publication Critical patent/CN102457722A/en
Application granted granted Critical
Publication of CN102457722B publication Critical patent/CN102457722B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Color Television Image Signal Generators (AREA)

Abstract

The invention discloses a processing method and device for a Bayer image. The processing method comprises the following steps of: reading the original data of the Bayer image in an external memory; for pixels of red R/green G/blue B in the image, and retaining pixel values of the pixels as corresponding R/G/B components of the pixel values after the pixels are processed; carrying out interpolation processing on the original image of the Bayer image, obtaining deletion components of the pixel values after the pixels are processed; caching the data repeatedly read in the interpolation processing process to an on-chip memory; transforming a RGB (Red, Green and Blue) image into a YUV (Luma and Chroma) image; and carrying out video coding on the YUV image. The processing device comprises an external storage, an image interpolation processing unit, the on-chip memory, an image converter and a video coder. According to the processing method and device provided by the invention, only data repeatedly read in a template in the interpolation processing process is needed to be cached to the on-chip memory, thereby the capacity of the on-chip memory and the cost of the interpolation processing of a chip are saved.

Description

The processing method of BAYER image and device
Technical field
The present invention relates to a kind of processing method and device of BAYER image, particularly relate to a kind of processing method and the device that can save the BAYER image of on-chip memory capacity.
Background technology
Along with the extensive use of camera in field of consumer electronics, cmos sensor is as the core devices of camera, the BAYER image that it produces is alternately to occur at a line Green G pixel and blue B pixel, and alternately occur at next line red R pixel and green G pixel, green G pixel and blue B pixel replace existing and red R pixel and green G pixel and replace and existingly also alternately occur simultaneously.Because the data of original BAYER image do not meet the standard of Video coding pattern of the input, thus must carry out video input preliminary treatment to BAYER image, to meet the input demand of video encoder.
First the processing of BAYER image is the interpolation processing to the initial data of BAYER image, through interpolation processing, the pixel value that converts each pixel to has the RGB image of red R, green G and tri-kinds of color components of blue B, then RGB image is obtained to YUV image through colour space transformation, finally YUV image is encoded, obtain encoded video.
Fig. 1 is the structural representation of processing unit of the BAYER image of prior art, as shown in Figure 1, the processing of traditional BAYER image is as independent modular design, CSI controller is under the triggering of clock pulse, the image information data of the BAYER form that cmos sensor is caught is in real time sent to image interpolation process unit, carry out the reception of data, process, the RGB image obtaining after processing is carried out to colour space transformation, be met the YUV image of Video coding pattern of the input standard, then store YUV image into external memory storage, when video encoder need to be to YUV coded image data, from external memory storage, read YUV view data again.
There is the defect of following two aspects in the BAYER image processing apparatus of prior art: on the one hand, in order to complete the real-time interpolation processing of grating scanning mode, must there is a larger row buffering on-chip memory, storage has image certain line number and that have certain width, can make like this cost of interpolation processing chip very high; On the other hand, although RGB image greatly reduces view data after converting YUV image to, but the data of YUV image are still greater than the initial data of the BAYER image before interpolation processing, so the memory space of the external memory storage that the BAYER image processing apparatus of prior art needs is larger, the image transmitting bandwidth of requirement is also larger.
Summary of the invention
The object of the present invention is to provide a kind of processing method of BAYER image, the capacity of on-chip memory while processing to reduce image, the cost of saving interpolation processing chip.
Another object of the present invention is to provide a kind of processing unit of BAYER image, to save the capacity of on-chip memory, save the cost of interpolation processing chip.
For realizing the processing method of a kind of BAYER image that object of the present invention provides, comprising:
The initial data of the BAYER image in reading external memory;
For the R/G/B pixel in image, the corresponding R/G/B component of the pixel value after the pixel value that retains described R/G/B pixel is processed as each pixel;
The initial data of BAYER image is carried out to interpolation processing, obtains the disappearance component of the pixel value after described R/G/B pixel is processed, and by the data buffer storage that needs to repeat to read in interpolation processing process to on-chip memory;
The RGB image obtaining after interpolation processing is carried out to colour space transformation, obtain YUV image;
YUV image is carried out to Video coding, obtain encoded video.
For realizing the processing unit of a kind of BAYER image that object of the present invention provides, comprising: external memory storage, image interpolation process unit, on-chip memory, image converter, video encoder;
External memory storage, for storing the initial data of the BAYER image that cmos sensor catches;
Image interpolation process unit, the initial data of storing for reading external memory, to the R/G/B pixel in image, the corresponding R/G/B component of the pixel value after the pixel value of reservation R/G/B pixel is processed as each pixel, and the initial data of BAYER image is carried out to interpolation processing by interpolation processing template, obtain the disappearance component of the pixel value after described R/G/B pixel is processed;
On-chip memory, needs for memory image interpolation processing process the data that repeat to read; Image converter, carries out colour space transformation for the RGB image to obtaining after interpolation processing, obtains YUV image;
Video encoder, for YUV image is encoded, obtains encoded video.
More preferably, the processing method of BAYER image, adopts 5 * 5 interpolation processing template to carry out interpolation processing to the initial data of BAYER image.
More preferably, the processing method of BAYER image, adopts above-mentioned interpolation processing template to carry out interpolation calculation to the disappearance component of R/G/B pixel and specifically comprises the steps:
For G pixel, by the pixel value of described G pixel, the pixel value of 4 G pixels the most contiguous of described G pixel, the pixel value of 2 R/B pixels the most contiguous and the pixel value weighted average of 2 time being close to G pixels the most contiguous with the most contiguous described R/B pixel, calculate the corresponding R/B component of the pixel value after described G pixel is processed, wherein, the pixel value of described G pixel, the weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of described G pixel and 2 contiguous G pixels is 0, the weight coefficient sum of the pixel value of described 2 R/B pixels the most contiguous is 1,
For R/B pixel, by the pixel value weighted average of the pixel value of 4 G pixels the most contiguous of the pixel value of described R/B pixel, described R/B pixel and 4 contiguous R/B pixels, calculate the corresponding G component of the pixel value after described R/B pixel is processed, wherein, the weight coefficient sum of the pixel value of 4 G pixels the most contiguous of described R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of described R/B pixel and described R/B pixel is 0;
For R/B pixel, by the pixel value weighted average of the pixel value of 4 B/R pixels the most contiguous of the pixel value of described R/B pixel, described R/B pixel, 4 contiguous R/B pixels, calculate the corresponding B/R component of the pixel value after described R/B pixel is processed, wherein, the pixel value weight coefficient sum of 4 B/R pixels the most contiguous of described R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of described R/B pixel and described R/B pixel is 0.
More preferably, the processing method of BAYER image, while asking the R/B component of G pixel, the weight coefficient of the pixel value of G pixel is 3/4, the weight coefficient that the weight coefficient of the pixel value of the weight coefficient of the pixel value of 4 G pixels the most contiguous of G pixel and 2 contiguous G pixels is the pixel value of-1/8,2 R/B pixel the most contiguous is 1/2.
More preferably, the processing method of BAYER image, while asking the G component of R/B pixel, the weight coefficient of the pixel value of 4 G pixels the most contiguous of R/B pixel is 1/4, the weight coefficient of the pixel value of R/B pixel is that the weight coefficient of the pixel value of the inferior vicinity R/B of 4 of 1/2, R/B pixel pixels is-1/8;
While asking the B/R component of R/B pixel, the weight coefficient of the pixel value of 4 B/R pixels the most contiguous of described R/B pixel is 1/4, the weight coefficient of the pixel value of described R/B pixel is 1/2, and the weight coefficient of the pixel value of 4 contiguous R/B pixels of described R/B pixel is-1/8.
More preferably, the interpolation processing template that the interpolation processing template of the processing unit of BAYER image is 5 * 5.
More preferably, the image interpolation process unit of the processing unit of BAYER image, for adopting above-mentioned 5 * 5 interpolation processing template to carry out interpolation calculation to the disappearance component of R/G/B pixel, account form is specially:
For G pixel, by the pixel value of described G pixel, the pixel value of 4 G pixels the most contiguous of described G pixel, the pixel value of 2 R/B pixels the most contiguous and the pixel value weighted average of 2 time being close to G pixels the most contiguous with the most contiguous described R/B pixel, calculate the corresponding R/B component of the pixel value after described G pixel is processed, wherein, the pixel value of described G pixel, the weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of described G pixel and 2 contiguous G pixels is 0, the weight coefficient sum of the pixel value of described 2 R/B pixels the most contiguous is 1,
For R/B pixel, by the pixel value weighted average of the pixel value of 4 G pixels the most contiguous of the pixel value of described R/B pixel, described R/B pixel and 4 contiguous R/B pixels, calculate the corresponding G component of the pixel value after described R/B pixel is processed, wherein, the weight coefficient sum of the pixel value of 4 G pixels the most contiguous of described R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of described R/B pixel and described R/B pixel is 0;
For R/B pixel, by the pixel value weighted average of the pixel value of 4 B/R pixels the most contiguous of the pixel value of described R/B pixel, described R/B pixel, 4 contiguous R/B pixels, calculate the corresponding B/R component of the pixel value after described R/B pixel is processed, wherein, the pixel value weight coefficient sum of 4 B/R pixels the most contiguous of described R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of described R/B pixel and described R/B pixel is 0.
More preferably, the output of the image converter of the processing unit of BAYER image and the input of video encoder are directly connected.
More preferably, the capacity of the on-chip memory of the processing unit of BAYER image is 400 bytes.
Beneficial effect of the present invention: the processing method of BAYER image of the present invention and device, in interpolation processing process, do not need to store the image that there is certain line number and there is certain width, only need to be by the data buffer storage that repeats to read in image interpolation process process to on-chip memory, so saved to a great extent the capacity of on-chip memory, saved the cost of interpolation processing chip.In addition, the processing method of BAYER image of the present invention and device, external memory stores be the initial data of BAYER image before interpolation processing, and the data of the initial data of the BAYER image image after with respect to interpolation processing are wanted much less before interpolation processing, so also saved the capacity of external memory storage, saved the cost that image is processed.
Accompanying drawing explanation
Fig. 1 is the structural representation of processing unit of the BAYER image of prior art;
Fig. 2 is the structural representation of processing unit one embodiment of BAYER image of the present invention;
Fig. 3 is the flow chart of processing method one embodiment of BAYER image of the present invention;
Fig. 4 is the position view of each pixel in the first interpolation processing template in embodiments of the invention;
Fig. 5 is the position view of each pixel in the second interpolation processing template in embodiments of the invention;
Fig. 6 is the position view of each pixel in the 3rd interpolation processing template in embodiments of the invention;
Fig. 7 is the position view of each pixel in the 4th interpolation processing template in embodiments of the invention.
Embodiment
In order to make object of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and the specific embodiments, the processing method of BAYER image of the present invention and BAYER image processing apparatus are described in detail further.
Fig. 3 is the flow chart of processing method one embodiment of BAYER image of the present invention, and as shown in Figure 3, the method comprises the steps:
31 steps: the initial data of the BAYER image in reading external memory;
32 steps: for the blue B pixel of the green G/ of the red R/ in image, the corresponding R/G/B component of the pixel value after the pixel value that retains the blue B pixel of the green G/ of described red R/ is processed as each pixel;
33 steps: the initial data of BAYER image is carried out to interpolation processing, obtains the disappearance component of the pixel value after the blue B pixel of the green G/ of described red R/ is processed, and by the data buffer storage that repeats to read in interpolation processing process to on-chip memory;
34 steps: the RGB image obtaining after interpolation processing is carried out to colour space transformation, obtain YUV image;
35 steps: YUV image is carried out to Video coding, obtain encoded video.
BAYER image is processed while starting, under the triggering of clock pulse, the initial data of the BAYER image of storing in external memory storage is read into image interpolation process unit, for the blue B pixel of the green G/ of the red R/ in image, the corresponding R/G/B component of the pixel value after the pixel value that retains the blue B pixel of the green G/ of red R/ is processed as each pixel, the disappearance component of the pixel value after processing is by carrying out interpolation processing acquisition to the initial data of BAYER image; During for fear of image interpolation process, from external memory storage, repeat to read the data interpolation template, by the data buffer storage that repeats to read in interpolation processing process to on-chip memory, accelerated so to a great extent the process that image is processed, also saved the capacity of on-chip memory, saved the cost of interpolation processing chip simultaneously.
In order to make the view data after processing meet Video coding pattern of the input standard, the RGB image obtaining is carried out to colour space transformation, be met the YUV image of Video coding pattern of the input standard.General RGB image obtains having the image of brightness signal Y and two color difference signal R-Y (being U), B-Y (being V) through matrixer, it is YUV image, RGB image converts YUV image to can adopt common method of the prior art, as disclosed method of China Patent No. 200610138960.x etc.
When clock pulse triggers, YUV image is outputed to video encoder, for certain video encoding standard, adopt basic encryption algorithm as the inter prediction of employing motion compensation, two-dimensional dct, VLC Run-Length Coding etc., YUV image is encoded and obtained encoded video.
Because the data that the relative RGB image of initial data of the BAYER image of storing in external memory storage is converted to after YUV image are less, so also greatly saved the capacity of external memory storage, further reduced the cost that image is processed.
More preferably, in order to improve the quality of image interpolation process, save the capacity of on-chip memory, the interpolation processing template of the processing method employing 5 * 5 of BAYER image of the present invention is carried out interpolation processing to the initial data of BAYER image.
More preferably, in order further to improve the quality of image interpolation process, the processing method of BAYER image of the present invention adopts above-mentioned 5 * 5 interpolation template to carry out interpolation processing to the initial data of BAYER image, obtain the disappearance component of the pixel value of R/G/B pixel, in interpolation processing process, for the impact on the corresponding R/B component of the pixel value of the G pixel of processing of the G pixel of effectively eliminating territory, neighbor cell, the impact of the R pixel in territory, neighbor cell on the corresponding G/B component of the pixel value of the R pixel of processing, and the impact of the B pixel in territory, neighbor cell on the corresponding G/R component of the pixel value of the B pixel of processing, interpolation calculation specifically comprises the steps:
Step 1, for G pixel, by the pixel value of G pixel, the pixel value of 4 G pixels the most contiguous of G pixel, the pixel value of 2 R/B pixels the most contiguous and the pixel value weighted average of 2 time being close to G pixels the most contiguous with the most contiguous R/B pixel, calculate the corresponding R/B component of the pixel value after G pixel is processed, wherein, the pixel value of G pixel, the weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of G pixel and 2 contiguous G pixels is 0, the weight coefficient sum of the pixel value of 2 R/B pixels the most contiguous is 1.
In step 1, in order more effectively to eliminate the impact of contiguous G pixel on the corresponding R/B component of the pixel value of the G pixel of processing, when calculating the corresponding R/B component of the pixel value after G pixel is processed, to take into account and be weighted with the pixel values of the most contiguous 2 contiguous G pixels of the most contiguous R/B pixel, thereby further improved the quality that image is processed.
Step 2, for R/B pixel, by the pixel value weighted average of the pixel value of 4 G pixels the most contiguous of the pixel value of R/B pixel, R/B pixel and 4 contiguous R/B pixels, calculate the corresponding G component of the pixel value after R/B pixel is processed, wherein, the weight coefficient sum of the pixel value of 4 G pixels the most contiguous of R/B pixel is that the weight coefficient sum of the pixel value of the pixel value of 1, R/B pixel and 4 contiguous R/B pixels of R/B pixel is 0.
Step 3, for R/B pixel, by the pixel value weighted average of the pixel value of 4 B/R pixels the most contiguous of the pixel value of R/B pixel, R/B pixel, 4 contiguous R/B pixels, calculate the corresponding B/R component of the pixel value after R/B pixel is processed, wherein, the weight coefficient sum of the pixel value of 4 B/R pixels the most contiguous of R/B pixel is that the weight coefficient sum of the pixel value of the pixel value of 1, R/B pixel and 4 contiguous R/B pixels of R/B pixel is 0.
More preferably, in order to obtain the better image of performance, when asking the G component of R/B pixel of BAYER image, the weight coefficient of the pixel value of 4 G pixels the most contiguous of R/B pixel is 1/4, the weight coefficient of the pixel value of R/B pixel is that the weight coefficient of the pixel value of the inferior vicinity R/B of 4 of 1/2, R/B pixel pixels is-1/8.While asking the B/R component of R/B pixel, the weight coefficient of the pixel value of 4 B/R pixels the most contiguous of R/B pixel is 1/4, the weight coefficient of the pixel value of R/B pixel is that the weight coefficient of the pixel value of 4 of 1/2, R/B pixel contiguous R/B pixels is-1/8.
Fig. 4 is the position view of each pixel in the first interpolation processing template in embodiments of the invention, and as shown in Figure 4, in this template, to replace the pixel value of the G pixel that the capable j row of existing i be G for BAYER image Green G pixel and blue B pixel i, j, retain the G component G of the pixel value after this pixel value is processed as this G pixel gi, j, i.e. G gi, j=G i, j;
The R component G of the pixel value of this G pixel ri, j/ B component G bi, jaccording to the pixel value G of G pixel i, j, G pixel the pixel value G of 4 G pixels the most contiguous i-1, j-1, G i-1, j+1, G i+1, j-1and G i+1, j+1, the pixel value R of 2 R/B pixels the most contiguous i-1, jand R i+1, j/ B i, j-1and B i, j+1and the pixel value G that 2 time be close to G pixels the most contiguous with the most contiguous R/B pixel i-2, jand G i+2, j/ G i, j-2and G i, j+2weighted average obtains, wherein the pixel value G of G pixel i, j, the pixel value G of 4 G pixels the most contiguous of G pixel i-1, j-1, G i-1, j+1, G i+1, j-1and G i+1, j+1and the pixel value G of 2 contiguous G pixels i-2, jand G i+2, j/ G i, j-2and G i, j+2weight coefficient sum be the pixel value R of 0,2 R/B pixel the most contiguous i-1, jand R i+1, j/ B i, j-1and B i, j+1weight coefficient sum be 1.
More preferably, in order to make the more excellent performance after image interpolation process, the better effects if that image is processed, asks the R/B component G of G pixel ri, j/ G bi, jtime, the pixel value G of G pixel i, jweight coefficient be the pixel value G of 4 G pixels the most contiguous of 3/4, G pixel i-1, j-1, G i-1, j+1, G i+1, j-1and G i+1, j+1weight coefficient and the pixel value G of 2 contiguous G pixels i-2, jand G i+2, j/ G i, j-2and G i, j+2weight coefficient be the pixel value R of-1/8,2 R/B pixel the most contiguous i-1, jand R i+1, j/ B i, j-1and B i, j+1weight coefficient be 1/2.The R component G of the pixel value after this G pixel interpolation processing ri jwith B component G bi, jconcrete calculating formula be:
G ri , j = R i - 1 , j + R i + 1 , j 2 + 3 G i , j 4 - G i - 1 , j - 1 + G i - 1 , j + 1 + G i + 1 , j - 1 + G i + 1 , j + 1 + G i - 2 , j + G i + 2 , j 8
G bi , j = B i , j - 1 + B i , j + 1 2 + 3 G i , j 4 - G i - 1 , j - 1 + G i - 1 , j + 1 + G i + 1 , j - 1 + G i + 1 , j + 1 + G i , j - 2 + G i , j + 2 8 .
Fig. 5 is the position view of each pixel in the second interpolation processing template in embodiments of the invention, and as shown in Figure 5, in this template, to replace the pixel value of the G pixel that the capable j row of existing i be G for BAYER image Green G pixel and red R pixel i, j, retain this pixel value as the G component G of the pixel value of this G pixel after processing gi, j, i.e. G gi, j=G i, j; The R component G of the pixel value of this G pixel ri, j/ B component G bi, jaccording to the pixel value G of G pixel i, j, G pixel the pixel value G of 4 G pixels the most contiguous i-1, j-1, G i-1, j+1, G i+1, j-1and G i+1, j+1, the pixel value R of 2 R/B pixels the most contiguous i, j-1and R i, j+1/ B i-1, jand B i+1, jand the pixel value G that 2 time be close to G pixels the most contiguous with the most contiguous R/B pixel i, j-2and G i, j+2/ G i-2, jand G i+2, jweighted average obtains, wherein the pixel value G of G pixel i, j, the pixel value G of 4 G pixels the most contiguous of G pixel i-1, j-1, G i-1, j+1, G i+1, j-1and G i+1, j+1and the pixel value G of 2 contiguous G pixels i, j-2and G i, j+2/ G i-2, jand G i+2, jweight coefficient sum be the pixel value R of 0,2 R/B pixel the most contiguous i, j-1and R i, j+1/ B i-1, jand B i+1, jweight coefficient sum be 1.
More preferably, in order to make the more excellent performance after image interpolation process, the better effects if that image is processed, asks the R/B component G of G pixel ri, j/ G bi, jtime, the pixel value G of G pixel i, jweight coefficient be the pixel value G of 4 G pixels the most contiguous of 3/4, G pixel i-1, j-1, G i-1, j+1, G i+1, j-1and G i+1, j+1weight coefficient and the pixel value G of 2 contiguous G pixels i, j-2and G i, j+2/ G i-2, jand G i+2, jweight coefficient be the pixel value R of-1/8,2 R/B pixel the most contiguous i, j-1and R i, j+1/ B i-1, jand B i+1, jweight coefficient be 1/2.The R component G of the pixel value after this G pixel interpolation processing ri, jwith B component G bi, jconcrete calculating formula be:
G ri , j = B i , j - 1 + B i , j + 1 2 + 3 G i , j 4 - G i - 1 , j - 1 + G i - 1 , j + 1 + G i + 1 , j - 1 + G i + 1 , j + 1 + G i , j - 2 + G i , j + 2 8 .
G bi , j = R i - 1 , j + R i + 1 , j 2 + 3 G i , j 4 - G i - 1 , j - 1 + G i - 1 , j + 1 + G i + 1 , j - 1 + G i + 1 , j + 1 + G i - 2 , j + G i + 2 , j 8
Fig. 6 is the position view of each pixel in the 3rd interpolation processing template in embodiments of the invention, and as shown in Figure 6, in this template, in BAYER image, the pixel value of the R pixel of the capable j row of i is R i, j, retain the R components R of the pixel value after this pixel value is processed as this R pixel ri, j, i.e. R ri, j=R i, j; The G components R of the pixel value of this R pixel gi, jaccording to the pixel value R of R pixel i, j, R pixel the pixel value G of 4 G pixels the most contiguous i, j-1, G i, j+1, G i-1, jand G i+1, jand the pixel value R of 4 contiguous R pixels i, j-2, R i, j+2, R i-2, jand R i+2, jweighted average calculation obtains, wherein, and the pixel value G of 4 G pixels the most contiguous of R pixel i, j-1, G i, j+1, G i-1, jand G i+1, jweight coefficient sum be the pixel value R of 1, R pixel i, jand the pixel value R of 4 contiguous R pixels of R pixel i, j-2, R i, j+2, R i-2, jand R i+2, jweight coefficient sum be 0.
The B components R of the pixel value of this R pixel bi, jaccording to the pixel value R of R pixel i, j, R pixel the pixel value B of 4 B pixels the most contiguous i-1, j-1, B i-1, j+1, B i+1, j-1and B i+1, j+1, 4 contiguous R pixels pixel value R i, j-2, R i, j+2, R i-2, jand R i+2, jweighted average, the corresponding B components R of the pixel value after calculating R pixel and processing bi, j, wherein, the pixel value B of 4 B pixels the most contiguous of R pixel i-1, j-1, B i-1, j+1, B i+1, j-1and B i+1, j+1weight coefficient sum be the pixel value R of 1, R pixel i, jand the pixel value R of 4 contiguous R pixels of R pixel i, j-2, R i, j+2, R i-2, jand R i+2, jweight coefficient sum be 0.
More preferably, in order to make the more excellent performance of image after image interpolation process, the better effects if that image is processed, when asking the G component of R pixel of BAYER image, the pixel value G of 4 G pixels the most contiguous of R pixel i, j-1, G i, j+1, G i-1, jand G i+1, jweight coefficient be the pixel value R of 1/4, R pixel i, jweight coefficient be the pixel value R of 4 of 1/2, R pixel contiguous R pixels i, j-2, R i, j+2, R i-2, jand R i+2, jweight coefficient be-1/8.Ask the B components R of R pixel bi, jtime, the pixel value B of 4 B pixels the most contiguous of R pixel i-1, j-1, B i-i, j+1, B i+1, j-1and B i+1, j+1weight coefficient be the pixel value R of 1/4, R pixel i, jweight coefficient be the pixel value R of 4 of 1/2, R pixel contiguous R pixels i, j-2, R i, j+2, R i-2, jand R i+2, jweight coefficient be-1/8.
The G components R of the pixel value after this R pixel interpolation processing gi, jwith B components R bi, jconcrete calculating formula be:
R gi , j = G i , j - 1 + G i , j + 1 + G i - 1 , j + G i + 1 , j 4 + R i , j 2 - R i , j - 2 + R i , j + 2 + R i - 2 , j + R i + 2 , j 8
R bi , j = B i - 1 , j - 1 + B i - 1 , j + 1 + B i + 1 , j - 1 + B i + 1 , j + 1 4 + R i , j 2 - R i , j - 2 + R i , j + 2 + R i - 2 , j + R i + 2 , j 8 .
Fig. 7 is the position view of each pixel in the 4th interpolation processing template in embodiments of the invention, and as shown in Figure 7, in this template, in BAYER image, the pixel value of the B pixel of the capable j row of i is B i, j, retain this pixel value as the B component B of the pixel value of this B pixel after processing bi, j, i.e. B bi, j=B i, j; For this B pixel, by the pixel value B of B pixel i, j, B pixel the pixel value G of 4 G pixels the most contiguous i, j-1, G i, j+1, G i-1, jand G i+1, jand the pixel value B of 4 contiguous B pixels i, j-2, B i, j+2, B i-2, jand B i+2, jweighted average, the corresponding G component B of the pixel value after calculating B pixel and processing gi, j, wherein, the pixel value G of 4 G pixels the most contiguous of B pixel i, j-1, G i, j+1, G i-1, jand G i+1, jweight coefficient sum be the pixel value B of 1, B pixel i, jand the pixel value B of 4 contiguous B pixels of B pixel i, j-2, B i, j+2, B i-2, jand B i+2, jweight coefficient sum be 0;
For this B pixel, by the pixel value B of B pixel i, j, B pixel the pixel value R of 4 R pixels the most contiguous i-1, j-1, R i-1, j+1, R i+1, j-1and R i+1, j+1, 4 contiguous B pixels pixel value B i, j-2, B i, j+2, B i-2, jand B i+2, jweighted average, the corresponding R component B of the pixel value after calculating B pixel and processing ri, j, wherein, the pixel value R of 4 R pixels the most contiguous of B pixel i-1, j-1, R i-1, j+1, R i+1, j-1and R i+1, j+1weight coefficient sum be the pixel value B of 1, B pixel i, jand the pixel value B of 4 contiguous B pixels of B pixel i, j-2, B i, j+2, B i-2, jand B i+2, jweight coefficient sum be 0.
More preferably, in order to obtain the better image of performance, the G component B of the B pixel of BAYER image in asking this template gi, jtime, the pixel value G of 4 G pixels the most contiguous of B pixel i, j-1, G i, j+1, G i-1, jand G i+1, jweight coefficient be the pixel value B of 1/4, B pixel i, jweight coefficient be the pixel value B of 4 of 1/2, B pixel contiguous B pixels i, j-2, B i, j+2, B i-2, jand B i+2, jweight coefficient be-1/8.Ask the R component B of B pixel ri, jtime, the pixel value R of 4 R pixels the most contiguous of B pixel i-1, j-1, R i-1, j+1, R i+1, j-1and R i+1, j+1weight coefficient be the pixel value B of 1/4, B pixel i, jweight coefficient be the pixel value B of 4 of 1/2, B pixel contiguous B pixels i, j-2, B i, j+2, B i-2, jand B i+2, jweight coefficient be-1/8.
The G component B of the pixel value after this B pixel interpolation processing gi, jwith R component B ri, jconcrete calculating formula be:
B gi , j = G i , j - 1 + G i , j + 1 + G i - 1 , j + G i + 1 , j 4 + B i , j 2 - B i , j - 2 + B i , j + 2 + B i - 2 , j + B i + 2 , j 8
B ri , j = R i - 1 , j - 1 + R i - 1 , j + 1 + R i + 1 , j - 1 + R i + 1 , j + 1 4 + B i , j 2 - B i , j - 2 + B i , j + 2 + B i - 2 , j + B i + 2 , j 8 .
In the interpolation processing process of BAYER image of the present invention, only have the data that repeat to read in interpolation processing process to be stored in on-chip memory, so just save the capacity of on-chip memory, also accelerated the process that image is processed simultaneously, saved the cost of interpolation processing chip.The initial data of the BAYER image of simultaneously storing in external memory storage is relatively less, so also greatly saved the capacity of external memory storage, has further reduced the cost that image is processed.
Fig. 2 is the structural representation of processing unit one embodiment of BAYER image of the present invention, as shown in Figure 2, BAYER image processing apparatus of the present invention comprises for storing external memory storage, image interpolation process unit, on-chip memory, image converter, the video encoder of the initial data of the BAYER image that cmos sensor catches.
CSI controller is under the triggering of clock pulse, and the initial data of the BAYER image that cmos sensor is caught outputs to external memory storage.
The initial data that image interpolation process unit is stored for reading external memory, to the blue B pixel of the green G/ of the red R/ in image, the corresponding R/G/B component of the pixel value after the pixel value that retains the blue B pixel of the green G/ of red R/ is processed as each pixel, and the initial data of BAYER image is carried out to interpolation processing by interpolation processing template, obtain the disappearance component of the pixel value after the blue B pixel of the green G/ of red R/ is processed, thereby convert BAYER image to RGB image.
In the interpolation processing process of BAYER image, on-chip memory needs for memory image interpolation processing process the data that repeat to read, and to avoid repeating reading out data from external memory storage, thereby has accelerated the process that image is processed.
Image converter, carries out colour space transformation for the RGB image to obtaining after interpolation processing, obtains YUV image, to meet Video coding pattern of the input standard, meets the input demand of video encoder.
Video encoder, for YUV image is encoded, obtains encoded video.
In order to improve the performance of image after interpolation processing and effectively to save the capacity of on-chip memory, image interpolation process unit adopts 5 * 5 interpolation processing template to carry out interpolation calculation to the disappearance component of the R/G/B pixel in BAYER image, and account form is specially:
For G pixel, by the pixel value of G pixel, the pixel value of 4 G pixels the most contiguous of G pixel, the pixel value of 2 R/B pixels the most contiguous and the pixel value weighted average of 2 time being close to G pixels the most contiguous with the most contiguous R/B pixel, calculate the corresponding R/B component of the pixel value after G pixel is processed, wherein, the pixel value of G pixel, the weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of G pixel and 2 contiguous G pixels is 0, the weight coefficient sum of the pixel value of 2 R/B pixels the most contiguous is 1.
For R/B pixel, by the pixel value weighted average of the pixel value of 4 G pixels the most contiguous of the pixel value of R/B pixel, R/B pixel and 4 contiguous R/B pixels, calculate the corresponding G component of the pixel value after R/B pixel is processed, wherein, the weight coefficient sum of the pixel value of 4 G pixels the most contiguous of R/B pixel is that the weight coefficient sum of the pixel value of the pixel value of 1, R/B pixel and 4 contiguous R/B pixels of R/B pixel is 0.
For R/B pixel, by the pixel value weighted average of the pixel value of 4 B/R pixels the most contiguous of the pixel value of R/B pixel, R/B pixel, 4 contiguous R/B pixels, calculate the corresponding B/R component of the pixel value after R/B pixel is processed, wherein, the pixel value weight coefficient sum of 4 B/R pixels the most contiguous of R/B pixel is that the weight coefficient sum of the pixel value of the pixel value of 1, R/B pixel and 4 contiguous R/B pixels of R/B pixel is 0.
The output of image converter is directly connected with the input of video encoder, has avoided so further use external memory storage, has saved the capacity of external memory storage.The YUV image of image converter output, under the triggering of clock pulse, directly outputs to video encoder, by video encoder encodes, obtains encoded video.
Due to the interpolation processing template interpolation processing template that is 5 * 5, for meeting general video encoder as the input requirements of video encoder H.264, H.264 video encoder once need to read 16 * 16 pixel and forms a macro block and encode from external memory storage, the macro block of encoding needs the pixel of 20 * 20 BAYER image, and the on-chip memory of selected like this 400 bytes can complete the high-quality image interpolation process of arbitrary resolution.
Finally it should be noted that obviously, those skilled in the art can carry out various changes and modification and not depart from the spirit and scope of the present invention the present invention.Like this, if within of the present invention these are revised and modification belongs to the scope of the claims in the present invention and equivalent technologies thereof, the present invention is also intended to comprise these changes and modification.

Claims (6)

1. a processing method for BAYER image, is characterized in that, the method comprises:
The initial data of the BAYER image in reading external memory;
For the R/G/B pixel in image, the corresponding R/G/B component of the pixel value after the pixel value that retains described R/G/B pixel is processed as each pixel;
The initial data of BAYER image is carried out to interpolation processing, obtains the disappearance component of the pixel value after described R/G/B pixel is processed, and by the data buffer storage that repeats to read in interpolation processing process to on-chip memory;
The interpolation processing template of employing 5 * 5 is carried out interpolation processing to the initial data of BAYER image;
The initial data of BAYER image is carried out to interpolation processing, and the disappearance component that obtains the pixel value after described R/G/B pixel is processed specifically comprises the steps:
For G pixel, by the pixel value of described G pixel, the pixel value of 4 G pixels the most contiguous of described G pixel, the pixel value of 2 R/B pixels the most contiguous and the pixel value weighted average of 2 time being close to G pixels the most contiguous with the most contiguous described R/B pixel, calculate the corresponding R/B component of the pixel value after described G pixel is processed, wherein, the pixel value of described G pixel, the weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of described G pixel and 2 contiguous G pixels is 0, the weight coefficient sum of the pixel value of described 2 R/B pixels the most contiguous is 1,
For R/B pixel, by the pixel value weighted average of the pixel value of 4 G pixels the most contiguous of the pixel value of described R/B pixel, described R/B pixel and 4 contiguous R/B pixels, calculate the corresponding G component of the pixel value after described R/B pixel is processed, wherein, the weight coefficient sum of the pixel value of 4 G pixels the most contiguous of described R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of described R/B pixel and described R/B pixel is 0;
For R/B pixel, by the pixel value weighted average of the pixel value of 4 B/R pixels the most contiguous of the pixel value of described R/B pixel, described R/B pixel, 4 contiguous R/B pixels, calculate the corresponding B/R component of the pixel value after described R/B pixel is processed, wherein, the weight coefficient sum of the pixel value of 4 B/R pixels the most contiguous of described R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of described R/B pixel and described R/B pixel is 0;
The RGB image obtaining after interpolation processing is carried out to colour space transformation, obtain YUV image;
YUV image is carried out to Video coding, obtain encoded video.
2. the processing method of BAYER image according to claim 1, it is characterized in that, while asking the R/B component of G pixel, the weight coefficient of the pixel value of described G pixel is 3/4, the weight coefficient of the pixel value of the weight coefficient of the pixel value of 4 G pixels the most contiguous of described G pixel and 2 contiguous G pixels is-1/8, and the weight coefficient of the pixel value of described 2 R/B pixels the most contiguous is 1/2.
3. the processing method of BAYER image according to claim 1, is characterized in that:
While asking the G component of R/B pixel, the weight coefficient of the pixel value of 4 G pixels the most contiguous of described R/B pixel is 1/4, the weight coefficient of the pixel value of described R/B pixel is 1/2, and the weight coefficient of the pixel value of 4 contiguous R/B pixels of described R/B pixel is-1/8;
While asking the B/R component of R/B pixel, the weight coefficient of the pixel value of 4 B/R pixels the most contiguous of described R/B pixel is 1/4, the weight coefficient of the pixel value of described R/B pixel is 1/2, and the weight coefficient of the pixel value of 4 contiguous R/B pixels of described R/B pixel is-1/8.
4. a processing unit for BAYER image, is characterized in that, comprising: external memory storage, image interpolation process unit, on-chip memory, image converter, video encoder;
Described external memory storage, for storing the initial data of the BAYER image that cmos sensor catches;
Described image interpolation process unit, for reading the initial data that described external memory storage is stored, to the R/G/B pixel in image, the corresponding R/G/B component of the pixel value after the pixel value that retains described R/G/B pixel is processed as each pixel, and the initial data of BAYER image is carried out to interpolation processing by interpolation processing template, obtain the disappearance component of the pixel value after described R/G/B pixel is processed;
The interpolation processing template that described interpolation processing template is 5 * 5;
Described image interpolation process unit, for adopting described interpolation processing template to carry out interpolation calculation to the disappearance component of R/G/B pixel, account form is specially:
For G pixel, by the pixel value of described G pixel, the pixel value of 4 G pixels the most contiguous of described G pixel, the pixel value of 2 R/B pixels the most contiguous and the pixel value weighted average of 2 time being close to G pixels the most contiguous with the most contiguous described R/B pixel, calculate the corresponding R/B component of the pixel value after described G pixel is processed, wherein, the pixel value of described G pixel, the weight coefficient sum of the pixel value of the pixel value of 4 G pixels the most contiguous of described G pixel and 2 contiguous G pixels is 0, the weight coefficient sum of the pixel value of described 2 R/B pixels the most contiguous is 1,
For R/B pixel, by the pixel value weighted average of the pixel value of 4 G pixels the most contiguous of the pixel value of described R/B pixel, described R/B pixel and 4 contiguous R/B pixels, calculate the corresponding G component of the pixel value after described R/B pixel is processed, wherein, the weight coefficient sum of the pixel value of 4 G pixels the most contiguous of described R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of described R/B pixel and described R/B pixel is 0;
For R/B pixel, by the pixel value weighted average of the pixel value of 4 B/R pixels the most contiguous of the pixel value of described R/B pixel, described R/B pixel, 4 contiguous R/B pixels, calculate the corresponding B/R component of the pixel value after described R/B pixel is processed, wherein, the pixel value weight coefficient sum of 4 B/R pixels the most contiguous of described R/B pixel is 1, and the weight coefficient sum of the pixel value of 4 contiguous R/B pixels of the pixel value of described R/B pixel and described R/B pixel is 0;
Described on-chip memory, needs for memory image interpolation processing process the data that repeat to read;
Described image converter, carries out colour space transformation for the RGB image to obtaining after interpolation processing, obtains YUV image;
Described video encoder, for YUV image is encoded, obtains encoded video.
5. the processing unit of BAYER image according to claim 4, is characterized in that, the output of described image converter is directly connected with the input of described video encoder.
6. the processing unit of BAYER image according to claim 5, is characterized in that, the capacity of described on-chip memory is 400 bytes.
CN201010519524.3A 2010-10-26 2010-10-26 Processing method and device for Bayer image Active CN102457722B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201010519524.3A CN102457722B (en) 2010-10-26 2010-10-26 Processing method and device for Bayer image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201010519524.3A CN102457722B (en) 2010-10-26 2010-10-26 Processing method and device for Bayer image

Publications (2)

Publication Number Publication Date
CN102457722A CN102457722A (en) 2012-05-16
CN102457722B true CN102457722B (en) 2014-01-22

Family

ID=46040317

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201010519524.3A Active CN102457722B (en) 2010-10-26 2010-10-26 Processing method and device for Bayer image

Country Status (1)

Country Link
CN (1) CN102457722B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014010224A1 (en) * 2012-07-09 2014-01-16 株式会社ニコン Video compression device, video decoding device, image compression device, image decoding device, imaging device, and program
CN103716606B (en) * 2013-12-30 2015-10-14 上海富瀚微电子股份有限公司 A kind of method of Bayer area image down-sampling, device and picture pick-up device
CN105847770B (en) * 2015-01-15 2017-08-08 展讯通信（上海）有限公司 The caching method of view data in picture processing chip and the chip
CN105611256B (en) * 2015-12-28 2017-08-01 南京航空航天大学 A kind of Bayer format interpolation method based on gradient and the display device based on FPGA
CN106412542A (en) * 2016-02-03 2017-02-15 周彩章 Image processing method and system
CN107016343A (en) * 2017-03-06 2017-08-04 西安交通大学 A kind of traffic lights method for quickly identifying based on Bel's format-pattern
CN110049337B (en) * 2019-05-24 2021-05-25 安翰科技(武汉)股份有限公司 Compression processing method and system for capsule endoscope bayer image
WO2021007742A1 (en) * 2019-07-15 2021-01-21 上海极清慧视科技有限公司 Compression method for obtaining video file, decompression method, system, and storage medium
CN114788280A (en) * 2020-12-31 2022-07-22 深圳市大疆创新科技有限公司 Video coding and decoding method and device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1595454A (en) * 2004-06-25 2005-03-16 闫宇松 A high compression ratio portrait coding-decoding method
CN1632829A (en) * 2004-12-27 2005-06-29 北京中星微电子有限公司 A method for implementing image zoom
CN1761312A (en) * 2005-11-03 2006-04-19 西安交通大学 Method and circuit for zooming digital video image based on dual quadratic interpolation
CN1799492A (en) * 2005-12-02 2006-07-12 清华大学 Quasi-lossless image compression and decompression method of wireless endoscope system
US7139022B1 (en) * 2002-06-19 2006-11-21 Neomagic Corp. Edge enhancer for RGB-Beyer to YUV 4:2:0 converter with sharpened-Y feedback to U, V transformer

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7139022B1 (en) * 2002-06-19 2006-11-21 Neomagic Corp. Edge enhancer for RGB-Beyer to YUV 4:2:0 converter with sharpened-Y feedback to U, V transformer
CN1595454A (en) * 2004-06-25 2005-03-16 闫宇松 A high compression ratio portrait coding-decoding method
CN1632829A (en) * 2004-12-27 2005-06-29 北京中星微电子有限公司 A method for implementing image zoom
CN1761312A (en) * 2005-11-03 2006-04-19 西安交通大学 Method and circuit for zooming digital video image based on dual quadratic interpolation
CN1799492A (en) * 2005-12-02 2006-07-12 清华大学 Quasi-lossless image compression and decompression method of wireless endoscope system

Also Published As

Publication number Publication date
CN102457722A (en) 2012-05-16

Similar Documents

Publication Publication Date Title
CN102457722B (en) Processing method and device for Bayer image
US11758184B2 (en) Line-based compression for digital image data
CN1791219B (en) Dual layer bus architecture for system-on-a-chip
JP5520122B2 (en) Data converter
CN107483850B (en) low-delay high-definition digital image transmission method
US9813652B2 (en) CMOS image sensor and imaging method implementing correlated double sampling and compression
CN102356635A (en) Video acquisition and processing systems
CN101960858A (en) Moving image coding device, imaging device and moving image coding method
CN109155850A (en) parallel video coding
WO2009087783A1 (en) Data generator for coding, method of generating data for coding, decoder and decoding method
US8224103B2 (en) Image encoding method and device, image decoding method and device, and imaging device
US20050212933A1 (en) Luminance and color difference signal generation apparatus, image data compression apparatus, and image processing system
KR100789221B1 (en) Image processor, imaging device, and image processing system
JP2011015347A (en) Apparatus and method for processing image, program and recording medium
JP2017192078A (en) Picture encoder and control method thereof
JP2010098352A (en) Image information encoder
CN103533260A (en) Lossless compression method of pixel values of CMOS image sensor
US7613351B2 (en) Video decoder with deblocker within decoding loop
US20080252740A1 (en) Image Pickup Device and Encoded Data Transferring Method
US20080266415A1 (en) Image Pickup Device and Encoded Data Outputting Method
Lin et al. Adjusted 4: 2: 2 chroma subsampling strategy for compressing mosaic videos with arbitrary RGB color filter arrays in HEVC
CN101742279B (en) Compression ratio adjustment method and electronic device of JPEG (Joint Photographic Experts Group) image
JP2009124278A (en) Imaging device
CN116828200B (en) Image processing method, processing device, equipment and medium
JP4522640B2 (en) Data transfer system and data transfer method

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant