WO2011043045A1 - 撮像装置、固体撮像素子、画像生成方法、およびプログラム - Google Patents
撮像装置、固体撮像素子、画像生成方法、およびプログラム Download PDFInfo
- Publication number
- WO2011043045A1 WO2011043045A1 PCT/JP2010/005911 JP2010005911W WO2011043045A1 WO 2011043045 A1 WO2011043045 A1 WO 2011043045A1 JP 2010005911 W JP2010005911 W JP 2010005911W WO 2011043045 A1 WO2011043045 A1 WO 2011043045A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel
- unit
- signal
- pixels
- image
- Prior art date
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 94
- 238000000034 method Methods 0.000 title claims description 18
- 230000003287 optical effect Effects 0.000 claims description 13
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 15
- 238000009825 accumulation Methods 0.000 description 14
- 238000004364 calculation method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 230000035945 sensitivity Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 210000004027 cell Anatomy 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000001444 catalytic combustion detection Methods 0.000 description 1
- 239000013078 crystal Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 210000002763 pyramidal cell Anatomy 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
- H04N25/46—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by combining or binning pixels
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/50—Control of the SSIS exposure
- H04N25/53—Control of the integration time
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
- H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
Definitions
- the present invention relates to an imaging technique using the relationship between the spatial resolution and temporal resolution of an image.
- the human retina has cone cells that sense the color of light (RGB) and rod cells that sense light and dark.
- the number of pyramidal cells and rod cells is said to be about 6.5 million and about 120 million, respectively. From this point of view, it is considered that the human eye is equivalent to a color camera with 6.5 million pixels in terms of color discrimination.
- the number of pixels of the current solid-state imaging device is already sufficiently large. For example, the resolution of a full high-definition image is 1920 ⁇ 1080, and even a high-definition image has a resolution of about 4096 ⁇ 2160.
- the progress speed of increasing the resolution of the display device does not exceed the progress speed of increasing the number of pixels of a solid-state imaging device currently being developed.
- the signal readout time for one image becomes longer in proportion to the number of pixels.
- the amount of data increases in proportion to the number of pixels.
- a plurality of pixels can be processed as one pixel.
- the imaging sensitivity can be increased by adding signals of a plurality of pixels.
- by adding a large number of pixel signals it is possible to deal with a low-luminance subject and improve the dynamic range at low illuminance. This produces the same effect as the case where an image sensor having a plurality of pixels with different aperture ratios is disclosed in Patent Document 2 and Patent Document 3.
- JP 2001-346103 A Japanese Patent Laid-Open No. 4-298175 JP 2006-174404 A
- the present invention provides a new imaging technique for generating a high frame rate moving image without performing pixel signal addition processing inside the imaging device.
- the imaging device of the present invention is a pixel array including a plurality of pixels arranged two-dimensionally on an imaging surface, and each includes N pixels (N is an integer of 2 or more) (M is 2).
- N is an integer of 2 or more
- M is 2.
- a pixel array divided into unit pixel blocks of the above integer) and a pixel signal reading unit for reading out pixel signals from each pixel at regular time intervals T, the N pixel pixels included in each unit pixel block A pixel signal reading unit that reads the pixel signal from each pixel at different timing; and an image generation unit that generates an image based on each pixel signal read from the pixel signal reading unit.
- the pixel signal reading unit reads the pixel signal from the N pixels included in each unit pixel block at different timings by time T / N.
- the image generation unit generates the image based on M ⁇ N pixel signals read by the pixel signal reading unit during the time interval T.
- the pixel signal read out from the j-th unit pixel block (j is an integer not less than 1 and not more than M) by the pixel signal readout unit i-th (i is an integer not less than N + 1) P (i, j), when the absolute value of the difference between P (i, j) and P (i-1, j) is larger than a preset value, the image generator j) ⁇ P (i ⁇ N + 1, j) ⁇ (N ⁇ 1) / N is used as a signal indicating the pixel value of each pixel at the i-th readout time in the j-th unit pixel block.
- the image generation unit may generate P (i + k, j when the absolute value of the difference between P (i, j) and P (i-1, j) is greater than a preset value. ) ⁇ P (i + k ⁇ 1, j) + (k ⁇ P (i + k ⁇ N, j) ⁇ (k ⁇ 1) ⁇ P (i + k ⁇ N ⁇ 1, j)) / N (k is 1 or more and less than N
- the image is generated as a signal indicating the pixel value of each pixel at the time of i + k reading in the j-th unit pixel block.
- the pixel signal reading unit changes the time interval T according to the brightness of the subject.
- the shape of the arrangement area of each unit pixel block is a cross shape.
- the imaging apparatus includes an optical low-pass filter for removing a frequency component higher than a spatial frequency defined by the size of the unit pixel block from the image.
- the image generation unit performs low-pass filter processing for removing a frequency component higher than a spatial frequency defined by the size of the unit pixel block from the image.
- the solid-state imaging device of the present invention is a pixel array including a plurality of pixels arranged two-dimensionally on an imaging surface, and each includes N (N is an integer of 2 or more) pixels (M is an integer).
- N is an integer of 2 or more
- M is an integer.
- a pixel array divided into unit pixel blocks of two or more integers), and a pixel signal reading unit for reading out pixel signals from each pixel at regular time intervals T, the N pixel units included in each unit pixel block A pixel signal reading unit that reads out the pixel signal from each of the pixels at different timings.
- the moving image generation method of the present invention is a pixel array including a plurality of pixels arranged two-dimensionally on an imaging surface, and each includes N pixels (N is an integer of 2 or more) (M is an integer).
- a pixel array divided into unit pixel blocks of two or more integers), and a pixel signal reading unit for reading out pixel signals from each pixel at regular time intervals T, the N pixel units included in each unit pixel block A method of generating a moving image based on a signal acquired by an imaging apparatus including a pixel signal reading unit that reads the pixel signal from each pixel at different timings, and is continuously read out in each unit pixel block Determining whether or not the difference between the two pixel signals is greater than a specified value; and if the difference is greater than the specified value, Generates image, if the difference is smaller than the prescribed value, and generating a video relatively low frame rate at a relatively high resolution.
- the moving image generation program of the present invention is a pixel array including a plurality of pixels arranged two-dimensionally on an imaging surface, and each includes N (N is an integer of 2 or more) pixels (M is an integer).
- a program for generating a moving image based on a signal acquired by an imaging device including a pixel signal reading unit that reads the pixel signal from each pixel at different timings, and is read continuously in each unit pixel block.
- the imaging apparatus and the solid-state imaging device of the present invention it is possible to generate a high frame rate moving image without performing pixel signal addition processing inside the imaging device. Furthermore, according to one embodiment of the present invention, a high resolution and high light utilization image is generated when the subject is not moving, and a low resolution and high frame rate video is generated when the subject is moving. can do.
- FIG. 1 is a block diagram illustrating a basic configuration example of an imaging apparatus according to the present invention.
- the figure which shows an example of the signal storage time in each pixel of the imaging device of this invention 1 is a block diagram showing a schematic configuration of an imaging apparatus according to Embodiment 1 of the present invention. Schematic diagram showing the arrangement of the lens and the imaging device in Embodiment 1 of the present invention.
- the flowchart which shows the image generation method in Embodiment 1 of this invention.
- the top view which shows the unit pixel block in Embodiment 2 of this invention The top view which shows the arrangement
- FIG. 1 is a block diagram showing an example of a schematic configuration of an imaging apparatus according to the present invention.
- the imaging apparatus includes a pixel array 200 including a plurality of pixels 20 arranged two-dimensionally on the imaging surface, a pixel signal reading unit 30 that reads pixel signals from each pixel, and an image generation unit 32.
- the pixel array 200 and the pixel signal reading unit 30 are included in the solid-state imaging device 7, but a part of the image signal reading unit 30 may be provided outside the solid-state imaging device 7. .
- the pixel array 200 is divided into a plurality of unit pixel blocks 40, and each unit pixel block 40 includes a plurality of pixels 20.
- the number of unit pixel blocks 40 included in the pixel array 200 is M (M is an integer of 2 or more), and the number of pixels 20 included in each unit pixel block 40 is N (N is 2 or more). Integer). That is, the pixel array 200 includes M ⁇ N pixels 20.
- Each pixel 20 receives light incident during exposure and outputs a photoelectric conversion signal (pixel signal) corresponding to the amount of received light (light reception amount).
- the number of pixels used for image generation among the pixels 20 included in the pixel array 200 is M ⁇ N, but the pixel array 200 may include pixels that are not used for image generation.
- the pixel signal reading unit 30 reads pixel signals from each pixel 20 at regular time intervals T. However, pixel signals are not read at the same timing for all the pixels 20, but pixel signals are read from N pixels in each unit pixel block 40 at different timings.
- FIG. 2 is a diagram illustrating the signal accumulation time of each pixel 20 included in one unit pixel block 40.
- N 4
- the timing at which pixel signals are read from each pixel 20 is equally spaced (every T / 4). To do.
- the present invention is not limited to such an example, and the timing of reading from each pixel may not be equal.
- the pixel signal from the time t n are stored, pixel signals are read at time t n + 4.
- the pixel signal is accumulated from time t n + 1 , and the pixel signal is read out at time t n + 5 .
- the pixel signal is accumulated from time t n + 2 , and the pixel signal is read out at time t n + 6 .
- the pixel signal is accumulated from time t n + 3 , and the pixel signal is read out at time t n + 7 .
- the accumulated signal is reset and signal accumulation starts again. Thereafter, pixel signals are read at each time interval T in each pixel.
- the pixel signal is read from each pixel 20 by the pixel signal reading unit 30 at the above timing, so that N pixel signals whose accumulation times are shifted by T / N are obtained every time T.
- the image generation unit 32 can generate one image from a total of M ⁇ N pixel signals obtained by N for each unit pixel block 40 during the time T. If the time interval T is set sufficiently long according to the brightness of the subject, this image becomes an image with a sufficient amount of light. Further, since an image is generated based on the pixel signal read for each pixel 20, an image with high resolution can be obtained. Thus, according to the present invention, an image (still image) with high resolution and high sensitivity can be obtained.
- the image generation unit 32 is a signal accumulated between two successive reading times by calculation using the pixel signal read out in one unit pixel block 40. The amount can be calculated. Details of the arithmetic processing will be described later. If the signal amount is set to the pixel values of all the pixels included in the unit pixel block, the resolution is lowered, but a moving image having a frame rate higher than the frame rate defined by the time interval T can be generated. If the time interval T is set to a normal frame interval used in moving image generation, it is possible to generate a moving image with a higher frame rate than usual.
- the image generation unit 32 can detect the presence or absence of movement of the subject by comparing the pixel signals.
- the image generation unit 32 performs a difference calculation between two pixel signals read continuously in each unit pixel block 40, and detects the presence or absence of movement of the subject based on whether or not the difference is larger than a specified value. To do. If it is determined that the subject is not moving, the image generation unit 32 generates an image by the method for acquiring the still image. In this case, a moving image is generated at a frame rate defined by the time interval T.
- the image generation unit 32 If it is determined that the subject is moving, the image generation unit 32 generates an image using the signal obtained by the calculation between the two pixel signals as the pixel signal of each pixel in the unit pixel block 40. Thereafter, if the same signal calculation is executed in accordance with the readout timing of the pixel signal, the resolution is lowered, but a moving image with a higher frame rate is generated than when the subject is determined not to move. Details of the moving image generation method by the image generation unit 32 will be described later.
- FIG. 3 is a block diagram showing the overall configuration of the imaging apparatus according to the first embodiment of the present invention.
- the imaging apparatus of this embodiment is a digital video camera, and includes an imaging unit 100 and a video signal processing unit 10 that generates a video signal (video signal) based on a signal transmitted from the imaging unit 100. Yes.
- the imaging unit 100 includes a condensing lens 5 for forming an image of a subject, an optical filter 6, and a solid-state image pickup device that converts optical information imaged through the condensing lens 5 and the optical filter 6 into an electrical signal by photoelectric conversion. 7 (image sensor).
- the imaging unit 100 further generates a basic signal for driving the solid-state imaging device 7, receives an output signal from the solid-state imaging device 7, and sends it to the video signal processing unit 10.
- An element driving unit 9 that drives the image sensor 7 based on the basic signal generated by the generation / reception unit 8 is provided.
- the element driving unit 9 may be incorporated in the signal generation / reception unit 8.
- the condenser lens 5 is a known lens and may be a lens unit having a plurality of lenses.
- the optical filter 6 includes an infrared cut filter for removing infrared rays in addition to a crystal low-pass filter for reducing moire patterns generated due to pixel arrangement.
- the image sensor 7 is typically a CMOS sensor and is manufactured by a known semiconductor manufacturing technique.
- the signal generation / reception unit 8 and the element driving unit 9 are configured by an LSI.
- the video signal processing unit 10 processes a signal sent from the imaging unit 100 to generate a video signal, and an image memory unit 11 that stores various data generated in the video signal generation process. And a video interface unit 13 for sending the generated video signal to the outside.
- the video signal generation unit 12 can be suitably realized by a combination of hardware such as a known digital signal processor (DSP) and software that executes image processing including video signal generation processing.
- DSP digital signal processor
- the video signal generation unit 12 and the image memory unit 11 are included in the image generation unit 32 in the present invention.
- the image memory unit 11 is configured by a DRAM or the like.
- the image memory unit 11 records the signal transmitted from the imaging unit 100 and temporarily records the image data generated by the video signal generation unit 12 and the compressed image data. These image data are sent to a recording medium (not shown) or a display unit via the video interface unit 13.
- the imaging apparatus may include known components such as an electronic shutter, a viewfinder, a power source (battery), and a flashlight, but a description thereof is omitted because it is not particularly necessary for understanding the present invention.
- known components such as an electronic shutter, a viewfinder, a power source (battery), and a flashlight, but a description thereof is omitted because it is not particularly necessary for understanding the present invention.
- the above configuration is merely an example, and in the present invention, known components can be suitably combined with the components not related to the pixel signal reading operation in the solid-state imaging device 7.
- FIG. 4 is a diagram schematically showing a state in which light transmitted through the condenser lens 5 enters the image sensor 7 during exposure.
- the lens 5 can be generally composed of a plurality of lenses arranged in the optical axis direction, but is drawn as a single lens for simplicity.
- a pixel array including a plurality of pixels arranged two-dimensionally is arranged on the imaging surface 7a of the imaging element 7.
- Each pixel is typically an electronic circuit including a photodiode, and outputs a pixel signal corresponding to the amount of incident light by photoelectric conversion.
- Light (visible light) that has passed through the condenser lens 5 and the optical filter 6 enters the imaging surface 7a.
- FIG. 5A is a plan view showing a pixel arrangement in the present embodiment.
- the pixel array 200 has a plurality of pixels arranged in a square lattice pattern on the imaging surface 7a.
- the pixel array 200 is divided into a plurality of unit pixel blocks 40, and each unit pixel block 40 includes four pixels 20a, 20b, 20c, and 20d arranged in two rows and two columns.
- the pixel array is not such a square lattice array, but may be, for example, an oblique array as shown in FIG. 5B or another array.
- FIG. 6 is a flowchart showing a schematic procedure when a moving image is generated by the imaging apparatus of the present embodiment.
- imaging starts, charges corresponding to the amount of incident light are accumulated in each pixel (S1).
- pixel signals are sequentially read pixel by pixel at a constant time interval T (S2).
- T a constant time interval
- S3 a specified value
- FIG. 7 schematically shows read signal lines 50a, 50b, 50c, and 50d connected to four pixels 20a, 20b, 20c, and 20d included in one unit pixel block 40, and output signal lines 52a and 52b.
- the readout signal is generated by the element driving unit 9 and is given to the pixels 20a, 20b, 20c, and 20d via the input terminals 21a, 21b, 21c, and 21d, respectively.
- a readout signal is given to each pixel, each pixel outputs a pixel signal corresponding to the accumulated charge amount.
- Pixel signals output from the pixels 20a and 20c are given to the output signal line 52a via the output terminals 22a and 22c, respectively.
- Pixel signals output from the pixels 20b and 20d are given to the output signal line 52b via the output terminals 22b and 22d, respectively.
- the read signal lines 50a to 50d and the output signal lines 52a and 52b function as a part of the signal read unit 30 in the present invention.
- the configuration of the readout signal line and the output signal line is not limited to the configuration illustrated in FIG. 7, and may be any configuration as long as pixel signals to be described later can be read out.
- the readout signal is input from the element driving unit 9 at every constant time interval T for all the pixels.
- FIG. 8 is a diagram schematically showing two consecutive frames in an image generated by using the pixel signal output at each time T from each pixel of the solid-state imaging device 7 in this embodiment as it is as the pixel value of each pixel.
- Image 3 represents an image one frame before image 2.
- a symbol P (i, 1) or the like written in each pixel of the image represents the pixel value of the pixel.
- a unit pixel block 1 having 4 pixels (P (i, 1), P (i + 1, 1), P (i + 2, 1), P (i + 3, 1)) at the upper left corner of the image 2 as one unit.
- x i, i + 1, i + 2, i + 3.
- the subscript j in the pixel signal P (i, j) is the number of the unit pixel block in one whole image, and is a number that does not change with time.
- the subscript i is a number indicating the reading order of the pixels in the unit pixel block, and is a number that changes over time.
- the signals P (i ⁇ 4,1), P (i ⁇ 3,1), P (i ⁇ 2,1), and P (i ⁇ 1,1) are first to second, respectively.
- the pixels are sequentially read out with a time interval of T / 4 from the four pixels.
- the signal P (i, 1), the signal P (i + 1, 1), the signal P (i + 2, 1), and the signal P (i + 3, 1) from the first to fourth pixels are each time interval T / 4. They are read out in order.
- the subscript i if the number of the upper left pixel included in each unit pixel block is i, the upper right pixel number is i + 1, the lower left pixel number is i + 2, and the lower right pixel number is i + 3.
- the pixel signal P (i, 1) at the upper left corner of the entire image is represented by P (i-4, 1) before one frame and P (i + 4, 1) after one frame.
- FIG. 9 is a timing chart showing an example of signal readout timing in each pixel of the unit pixel block 1 at the upper left end of the pixel array 200 of the solid-state imaging device 7 in the present embodiment.
- the signal accumulation time T is set to four times the normal accumulation time, but in the present invention, the signal accumulation time T can be set to an arbitrary value.
- the signal accumulation time T is preferably set to an optimum time according to the brightness of the subject.
- the optical information imaged by the condenser lens 5 is photoelectrically converted by the solid-state imaging device 7, read by the signal generation / reception unit 8, and stored in the image memory unit 11.
- T time
- the image data is sent to the video signal generation unit 12 when data for one frame is prepared in the image memory unit 11.
- the video signal generation unit 12 Based on the transmitted image data, the video signal generation unit 12 generates a video signal, and the generated video signal is transmitted to the outside through the video interface unit 13.
- One image data is stored in the image memory unit 11, and then the data is transmitted to the video signal generation unit 12, and then signal processing is performed in units of image frames.
- the signal processing is performed in units of frames, but is performed simultaneously for each unit pixel block.
- signal processing performed in the unit pixel block 1 at the upper left end of the image will be described as an example.
- each pixel signal of all unit pixel blocks is compared with the pixel signal data read immediately before and the difference between them is all smaller than a preset value Zth, the value of each pixel signal is set to the pixel value of that pixel.
- a video signal of one image is generated. That is, it is determined that the image is in a still state, and only one frame is output every four frames (one image at time T), but the sensitivity is improved four times the normal and an image with no decrease in resolution is created. Therefore, in this case, a moving image having a normal 1 ⁇ 4 frame rate is generated.
- the time t n + 3 is between the t n + 4 P1, the time t n + 4 from t n + 5 between the P2, the time t n + 5 between the t n + 6 from P3, the time t n It is assumed that a signal of P4 amount is accumulated in each pixel between +6 and t n + 7 . Here, it is assumed that the pixel signal does not change within the range of one unit pixel block. Based on this premise, it can be determined that the pixel signal P (i-1, 1) before P (i, 1) is read out is still.
- P (i, 1) is a signal accumulated between times t n and t n + 4
- the signal P2 during the time t n + 4 to t n + 5 , the signal P3 during the time t n + 5 to t n + 6 , and the time t n + 6 to t n + 7 The signal P4 can also be generated by the following equations 2 to 4, respectively.
- the signals P3 and P4 may be obtained by operations shown in the following equations 5 and 6, respectively.
- P4 P (i + 3,1) ⁇ P (i + 2,1) + P (i ⁇ 1,1) / 4
- all the unit pixel blocks are compared with the pixel signal read immediately before, and it is determined whether the subject is moving based on whether the difference is larger than the specified value. Judgment may be made by this method. For example, the above determination may be performed for each unit pixel block, and only a moving part of the image may be a moving image having a low resolution and a relatively high frame rate. In addition, the detection of the presence or absence of the movement of the subject may be determined based on a comparison with the pixel signal of the previous frame instead of the previous pixel signal.
- a new image may not be generated until it is determined that the subject has movement. Further, once the image is generated, the determination as to whether or not the subject is moving may not be performed for a while, and the above determination may be performed again after a predetermined time (for example, several seconds) has elapsed.
- the pixel array of the solid-state imaging device 7 is divided into a plurality of unit pixel blocks, the accumulation time in each pixel of each unit pixel block is set longer than usual, and the accumulation timing is changed.
- multiple images having different times can be obtained.
- a moving image with a relatively high light utilization rate and a low frame rate can be obtained.
- the resolution is lowered by the calculation based on Equations 1 to 4, but a moving image can be obtained at a relatively high frame rate.
- the imaging method can be returned in accordance with the presence or absence of the movement of the subject.
- the pixel signal when it is determined that the subject is moving, the pixel signal does not change in the range of the unit pixel block, and the pixel signals up to P (i ⁇ 1,1) that have been read are not changed. Is not processed.
- the unit pixel block is composed of four pixels arranged in 2 rows and 2 columns.
- the unit pixel block is not limited to this, and any number of pixels may be included in the unit pixel block as long as it is plural. .
- the imaging apparatus of the present embodiment is generalized as follows.
- the number of unit pixel blocks is M
- the element driving unit 9 generates a read signal so that pixel signals are read from N pixels included in each unit pixel block at different timings by time T / N.
- the signal generation / reception unit 8 sequentially reads out pixel signals from these pixels at different timings by time T / N.
- the pixel signal read i-th i is an integer equal to or greater than N + 1) from the j-th unit pixel block (j is an integer greater than or equal to 1 and equal to or smaller than M) is P (i, j).
- the signal Pijk shown in the following Expression 9 is processed as a signal indicating the pixel value of each pixel at the i + k-th (k is an integer less than or equal to 1 and less than N) readout in the j-th unit pixel block.
- Pijk P (i + k, j) ⁇ P (i + k ⁇ 1, j) + (k ⁇ P (i + k ⁇ N, j) ⁇ (k ⁇ 1) ⁇ P (i + k ⁇ N ⁇ 1, j) ) / N
- the imaging apparatus of the present embodiment has a function of dynamically detecting the presence or absence of movement of a subject during moving image shooting and dynamically changing the imaging method.
- the present invention does not necessarily include such a function. There is no need.
- only a function of generating a moving image may be provided in a method that is executed when it is determined that there is a movement of the subject.
- only a function for generating a still image by processing performed when it is determined that the subject is not moving may be provided.
- each pixel included in each unit pixel block 40 it is assumed that substantially the same amount of light is incident on each pixel included in each unit pixel block 40. This can be realized by removing an image component having a frequency higher than the spatial frequency defined by the size of each pixel block 40 by the optical low-pass filter included in the optical filter 6. However, even if the imaging apparatus does not include an optical low-pass filter, a similar effect can be obtained by performing a low-pass filter process that removes high-frequency components of an image based on the read pixel signal.
- the signal calculation processing in the present embodiment is executed by the video generation unit 12, but the imaging device does not necessarily have to be configured to perform the signal calculation processing described above.
- the imaging device may only generate a pixel signal, and read out the pixel signal and generate a video signal may be executed by an external information processing device. In that case, it is realizable by making the said information processing apparatus run the program which prescribes
- FIG. 10A and 10B show unit pixel blocks in the present embodiment.
- FIG. 10A shows one unit pixel block 4
- FIG. 10B is a pixel plan view showing a situation in which a plurality of unit pixel blocks 4 are arranged adjacent to each other.
- the difference between the present embodiment and the first embodiment is only the number of pixels included in the unit pixel block and the shape of the arrangement area of the unit pixel block, and the configuration and signal processing of the imaging apparatus are all the same. Therefore, in the following description, only differences from the first embodiment will be described.
- the unit pixel block 4 in this embodiment has a cross shape using five pixels. Further, as shown in FIG. 10B, adjacent unit pixel blocks are brought close to each other. As a result, the center of the unit pixel block 4 approaches the center of gravity of the adjacent unit pixel block in the horizontal and vertical directions, and is only one pixel away.
- the shape of the unit pixel block 4 is a square of 2 ⁇ 2 pixels as in the first embodiment, the distance between the centers of the adjacent regions is two pixels. The interval is approaching. Therefore, the imaging apparatus of the present embodiment can improve the resolution in the horizontal and vertical directions as compared with the imaging apparatus of the first embodiment. That is, the resolution of the moving image generated by the imaging device of the present embodiment using the cross-shaped unit pixel block is higher than that of the moving image generated by the imaging device of the first embodiment using the square unit pixel block. It will be.
- each unit pixel block includes five pixels arranged in a cross shape, and by bringing the centers of the unit pixel blocks closer to each other, an effect of improving the resolution in a moving image can be obtained.
- the number of unit pixel blocks is five.
- the shape of the arrangement area of each unit pixel block is a cross shape, the number of pixels included in the unit pixel block is not limited to five, and is more than six. May be.
- the solid-state imaging device and the imaging apparatus of the present invention are effective for all cameras using the solid-state imaging device.
- it can be used for consumer cameras such as digital still cameras and digital video cameras, and industrial solid-state surveillance cameras.
- Unit pixel block consisting of 4 pixels (P (i, 1), P (i + 1,1), P (i + 2,1), P (i + 3,1)) at the upper left corner of one image 2
- One frame 3 Image 1 frame 4 cruciform unit pixel block consisting of 5 pixels 5 condensing lens 6 optical plate 7 solid-state imaging device 8 signal generation and pixel signal reception unit 9 element driving unit 10 video signal processing unit 11 image memory unit 12 video signal generation unit 13
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
Description
図3は、本発明の第1の実施形態による撮像装置の全体構成を示すブロック図である。本実施形態の撮像装置は、デジタル式のビデオカメラであり、撮像部100と、撮像部100から送出される信号に基づいて映像信号(ビデオ信号)を生成する映像信号処理部10とを備えている。
(式1) P1=P(i,1)-P(i-1,1)×3/4
(式2) P2=P(i+1,1)-P(i,1)+P(i-3,1)/4
(式3) P3=P(i+2,1)-P(i+1,1)+(2P(i-2,1)-P(i-3,1))/4
(式4) P4=P(i+3,1)-P(i+2,1)+(3P(i-1,1)-2P(i-2,1))/4
(式5) P3=P(i+2,1)-P(i+1,1)+P(i-2,1)/4
(式6) P4=P(i+3,1)-P(i+2,1)+P(i-1,1)/4
(式7) Pm=(P(i-4,1)+P(i-3,1)+P(i-2,1)+P(i-1,1))/4
(式8)Pij=P(i,j)-P(i-N+1,j)×(N-1)/N
(式9)Pijk=P(i+k,j)-P(i+k-1,j)+(k×P(i+k-N,j)-(k-1)×P(i+k-N-1,j))/N
次に本発明の第2の実施形態について、図面を用いて説明する。図10A、10Bは本実施形態における単位画素ブロックを示している。図10Aは1つの単位画素ブロック4を示し、図10Bは複数の単位画素ブロック4が隣接して配置された状況を示す画素平面図である。ここで、本実施形態と実施形態1との相違点は単位画素ブロックに含まれる画素数および単位画素ブロックの配置エリアの形状だけであり、撮像装置の構成や信号処理については全て同じである。よって、以下の説明では、実施形態1と異なる点のみを説明する。
2 画像の1フレーム
3 画像の1フレーム
4 5画素からなる十字型の単位画素ブロック
5 集光レンズ
6 光学板
7 固体撮像素子
8 信号発生及び画素信号受信部
9 素子駆動部
10 映像信号処理部
11 画像メモリー部
12 ビデオ信号生成部
13 ビデオインターフェース部
20 画素
30 画素信号読出し部
32 画像生成部
40 単位画素ブロック
50a、50b、50c、50d 読出し信号線
52a、52b 出力信号線
100 撮像部
200 画素アレイ
Claims (12)
- 撮像面に2次元状に配列された複数の画素を含む画素アレイであって、各々がN個(Nは2以上の整数)の画素を含むM個(Mは2以上の整数)の単位画素ブロックに分割された画素アレイと、
各画素から一定の時間間隔Tごとに画素信号を読出す画素信号読出し部であって、各単位画素ブロックに含まれる前記N個の画素からそれぞれ異なるタイミングで前記画素信号を読み出す画素信号読出し部と、
前記画素信号読出し部から読み出された各画素信号に基づいて画像を生成する画像生成部と、
を備える撮像装置。 - 前記画素信号読出し部は、各単位画素ブロックに含まれる前記N個の画素から時間T/Nずつ異なるタイミングで前記画素信号を読み出す、請求項1に記載の撮像装置。
- 前記画像生成部は、前記時間間隔Tの間に前記画素信号読出し部によって読み出されたM×N個の画素信号に基づいて前記画像を生成する、請求項2に記載の撮像装置。
- 前記画素信号読出し部によってj番目(jは1以上M以下の整数)の単位画素ブロックからi回目(iはN+1以上の整数)に読み出された画素信号をP(i、j)とするとき、
前記画像生成部は、P(i,j)とP(i-1,j)との差の絶対値が予め設定された値よりも大きい場合には、P(i,j)-P(i-N+1,j)×(N-1)/Nを、j番目の単位画素ブロックにおけるi回目の読出し時点での各画素の画素値を示す信号として前記画像を生成する、
請求項3に記載の撮像装置。 - 前記画像生成部は、P(i,j)とP(i-1,j)との差の絶対値が予め設定された値よりも大きい場合には、P(i+k,j)-P(i+k-1,j)+(k×P(i+k-N,j)-(k-1)×P(i+k-N-1,j))/N (kは1以上N未満の整数)を、j番目の単位画素ブロックにおけるi+k回目の読出し時点での各画素の画素値を示す信号として前記画像を生成する、請求項4に記載の撮像装置。
- 前記画素信号読出し部は、被写体の明るさに応じて前記時間間隔Tを変化させる、請求項1から5のいずれかに記載の撮像装置。
- 各単位画素ブロックの配置エリアの形状は十字形である、請求項1から6のいずれかに記載の撮像装置。
- 前記単位画素ブロックのサイズで規定される空間周波数よりも高い周波数成分を前記画像から除去するための光学的ローパスフィルタを備える請求項1から7のいずれかに記載の撮像装置。
- 前記画像生成部は、前記単位画素ブロックのサイズで規定される空間周波数よりも高い周波数成分を前記画像から除去するローパスフィルタ処理を行う、請求項1から7のいずれかに記載の撮像装置。
- 撮像面に2次元状に配列された複数の画素を含む画素アレイであって、各々がN個(Nは2以上の整数)の画素を含むM個(Mは2以上の整数)の単位画素ブロックに分割された画素アレイと、
各画素から一定の時間間隔Tごとに画素信号を読出す画素信号読出し部であって、各単位画素ブロックに含まれる前記N個の画素からそれぞれ異なるタイミングで前記画素信号を読み出す画素信号読出し部と、
を備える固体撮像素子。 - 撮像面に2次元状に配列された複数の画素を含む画素アレイであって、各々がN個(Nは2以上の整数)の画素を含むM個(Mは2以上の整数)の単位画素ブロックに分割された画素アレイと、各画素から一定の時間間隔Tごとに画素信号を読出す画素信号読出し部であって、各単位画素ブロックに含まれる前記N個の画素からそれぞれ異なるタイミングで前記画素信号を読み出す画素信号読出し部とを備える撮像装置によって取得された信号に基づいて動画を生成する方法であって、
各単位画素ブロックにおいて、連続して読み出された2つの画素信号の差分が規定値よりも大きいか否かを判断するステップと、
前記差分が前記規定値よりも大きい場合に、相対的に低い解像度で相対的に高いフレームレートの動画を生成し、前記差分が前記規定値よりも小さい場合に、相対的に高い解像度で相対的に低いフレームレートの動画を生成するステップと、
を含む動画生成方法。 - 撮像面に2次元状に配列された複数の画素を含む画素アレイであって、各々がN個(Nは2以上の整数)の画素を含むM個(Mは2以上の整数)の単位画素ブロックに分割された画素アレイと、各画素から一定の時間間隔Tごとに画素信号を読出す画素信号読出し部であって、各単位画素ブロックに含まれる前記N個の画素からそれぞれ異なるタイミングで前記画素信号を読み出す画素信号読出し部とを備える撮像装置によって取得された信号に基づいて動画を生成するためのプログラムであって、
各単位画素ブロックにおいて、連続して読み出された2つの画素信号の差分が規定値よりも大きいか否かを判断するステップと、
前記差分が前記規定値よりも大きい場合に、相対的に低い解像度で相対的に高いフレームレートの動画を生成し、前記差分が前記規定値よりも小さい場合に、相対的に高い解像度で相対的に低いフレームレートの動画を生成するステップと、
を含む動画生成プログラム。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/141,711 US8970721B2 (en) | 2009-10-07 | 2010-10-01 | Imaging device, solid-state imaging element, image generation method, and program |
JP2011511929A JPWO2011043045A1 (ja) | 2009-10-07 | 2010-10-01 | 撮像装置、固体撮像素子、画像生成方法、およびプログラム |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2009-233286 | 2009-10-07 | ||
JP2009233286 | 2009-10-07 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011043045A1 true WO2011043045A1 (ja) | 2011-04-14 |
Family
ID=43856529
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2010/005911 WO2011043045A1 (ja) | 2009-10-07 | 2010-10-01 | 撮像装置、固体撮像素子、画像生成方法、およびプログラム |
Country Status (3)
Country | Link |
---|---|
US (1) | US8970721B2 (ja) |
JP (1) | JPWO2011043045A1 (ja) |
WO (1) | WO2011043045A1 (ja) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015111761A (ja) * | 2013-12-06 | 2015-06-18 | 株式会社ニコン | 電子機器 |
JP2017223675A (ja) * | 2016-06-17 | 2017-12-21 | 株式会社ミツトヨ | 超解像度ボア撮像システム |
US9979914B2 (en) | 2016-03-17 | 2018-05-22 | Panasonic Intellectual Property Management Co., Ltd. | Imaging device |
JP2018174592A (ja) * | 2018-08-15 | 2018-11-08 | 株式会社ニコン | 電子機器 |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017126899A2 (en) | 2016-01-19 | 2017-07-27 | Samsung Electronics Co., Ltd. | Methods and systems for reconstructing a high frame rate high resolution video |
WO2020045278A1 (en) * | 2018-08-31 | 2020-03-05 | Canon Kabushiki Kaisha | Imaging device with motion dependent pixel binning |
JP7356266B2 (ja) * | 2018-08-31 | 2023-10-04 | キヤノン株式会社 | 撮像装置、撮像システム、および撮像装置の駆動方法 |
US20230388662A1 (en) * | 2022-05-24 | 2023-11-30 | Samsung Electronics Co., Ltd. | Color filter array for de-mosaicing and binning |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH04315378A (ja) * | 1991-04-15 | 1992-11-06 | Olympus Optical Co Ltd | 固体撮像素子 |
JP2000209509A (ja) * | 1999-01-19 | 2000-07-28 | Canon Inc | 固体撮像装置と撮像システム |
JP2002369083A (ja) * | 2001-06-07 | 2002-12-20 | Olympus Optical Co Ltd | 撮像装置 |
JP2006157124A (ja) * | 2004-11-25 | 2006-06-15 | Olympus Corp | 撮像装置 |
JP2007228460A (ja) * | 2006-02-27 | 2007-09-06 | Mitsumasa Koyanagi | 集積センサを搭載した積層型半導体装置 |
JP2008035279A (ja) * | 2006-07-28 | 2008-02-14 | Sanyo Electric Co Ltd | 画素情報読出方法および撮像装置 |
JP2009159067A (ja) * | 2007-12-25 | 2009-07-16 | Olympus Imaging Corp | 撮像表示方法および撮像表示装置 |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4140077B2 (ja) * | 1998-02-18 | 2008-08-27 | ソニー株式会社 | 固体撮像素子の駆動方法及び固体撮像素子、並びにカメラ |
TW439285B (en) * | 1998-11-30 | 2001-06-07 | Toshiba Corp | Solid-state imaging device |
US7202897B2 (en) * | 2001-07-11 | 2007-04-10 | Sony Corporation | Solid-state image pickup exposure control system and method |
JP4142340B2 (ja) * | 2002-05-22 | 2008-09-03 | オリンパス株式会社 | 撮像装置 |
JP3898606B2 (ja) * | 2002-09-12 | 2007-03-28 | 株式会社東芝 | 動きベクトル検出方法及び装置並びにフレーム補間画像作成方法及び装置 |
US7283164B2 (en) * | 2002-09-18 | 2007-10-16 | Micron Technology, Inc. | Method for detecting and correcting defective pixels in a digital image sensor |
US7694231B2 (en) * | 2006-01-05 | 2010-04-06 | Apple Inc. | Keyboards for portable electronic devices |
JP4888081B2 (ja) * | 2006-01-23 | 2012-02-29 | セイコーエプソン株式会社 | 撮像装置、撮像方法、撮像システム及び画像処理装置 |
US7944482B2 (en) * | 2006-07-28 | 2011-05-17 | Sanyo Electric Co., Ltd. | Pixel information readout method and image pickup apparatus |
-
2010
- 2010-10-01 JP JP2011511929A patent/JPWO2011043045A1/ja active Pending
- 2010-10-01 US US13/141,711 patent/US8970721B2/en not_active Expired - Fee Related
- 2010-10-01 WO PCT/JP2010/005911 patent/WO2011043045A1/ja active Application Filing
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH04315378A (ja) * | 1991-04-15 | 1992-11-06 | Olympus Optical Co Ltd | 固体撮像素子 |
JP2000209509A (ja) * | 1999-01-19 | 2000-07-28 | Canon Inc | 固体撮像装置と撮像システム |
JP2002369083A (ja) * | 2001-06-07 | 2002-12-20 | Olympus Optical Co Ltd | 撮像装置 |
JP2006157124A (ja) * | 2004-11-25 | 2006-06-15 | Olympus Corp | 撮像装置 |
JP2007228460A (ja) * | 2006-02-27 | 2007-09-06 | Mitsumasa Koyanagi | 集積センサを搭載した積層型半導体装置 |
JP2008035279A (ja) * | 2006-07-28 | 2008-02-14 | Sanyo Electric Co Ltd | 画素情報読出方法および撮像装置 |
JP2009159067A (ja) * | 2007-12-25 | 2009-07-16 | Olympus Imaging Corp | 撮像表示方法および撮像表示装置 |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015111761A (ja) * | 2013-12-06 | 2015-06-18 | 株式会社ニコン | 電子機器 |
US9979914B2 (en) | 2016-03-17 | 2018-05-22 | Panasonic Intellectual Property Management Co., Ltd. | Imaging device |
JP2017223675A (ja) * | 2016-06-17 | 2017-12-21 | 株式会社ミツトヨ | 超解像度ボア撮像システム |
JP2018174592A (ja) * | 2018-08-15 | 2018-11-08 | 株式会社ニコン | 電子機器 |
Also Published As
Publication number | Publication date |
---|---|
US8970721B2 (en) | 2015-03-03 |
US20110317053A1 (en) | 2011-12-29 |
JPWO2011043045A1 (ja) | 2013-03-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2011043045A1 (ja) | 撮像装置、固体撮像素子、画像生成方法、およびプログラム | |
JP4448888B2 (ja) | 撮像装置及び撮像装置の信号処理方法 | |
US7978240B2 (en) | Enhancing image quality imaging unit and image sensor | |
KR100827238B1 (ko) | 고화질 영상을 위한 영상 표시 방법 및 장치 | |
JP2009212909A (ja) | 固体撮像装置、固体撮像装置のフリッカ検出方法、制御プログラム、可読記録媒体および電子情報機器 | |
JP5608820B2 (ja) | 撮像装置及び合焦制御方法 | |
JP2007104178A (ja) | 撮像素子および撮像装置 | |
JP2013120813A (ja) | 固体撮像素子および電子機器 | |
US8830384B2 (en) | Imaging device and imaging method | |
CN109474781A (zh) | 摄像装置、摄像装置的控制方法、记录介质 | |
JPS60142683A (ja) | 固体撮像装置 | |
KR20210114290A (ko) | 이미지 센서 및 이를 포함하는 촬영 장치 | |
JP4538337B2 (ja) | 固体撮像素子 | |
JP2000350101A (ja) | 固体撮像装置及び画像情報取得装置 | |
WO2018021034A1 (ja) | 撮像制御装置および撮像制御方法、並びに撮像素子 | |
JP5106055B2 (ja) | 撮像装置及びそのフリッカ検出方法 | |
JP6137539B2 (ja) | 固体撮像素子及びその駆動方法、並びに電子機器 | |
JPH05115041A (ja) | 固体撮像装置 | |
JP2008187614A (ja) | 撮影装置 | |
WO2015008635A1 (ja) | 固体撮像素子及びその駆動方法、並びに電子機器 | |
JP2010283669A (ja) | 固体撮像素子、撮像システム、および固体撮像素子の駆動方法 | |
JP4840991B2 (ja) | 光電変換装置及びその制御方法並びに撮像装置 | |
JP2009303020A (ja) | 撮像装置及び欠陥画素補正方法 | |
JP2006140581A (ja) | 撮像素子及び画像入力装置 | |
JP2003153090A (ja) | X−y走査型撮像素子及びこれを備えた撮像装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 2011511929 Country of ref document: JP |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10821722 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13141711 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 10821722 Country of ref document: EP Kind code of ref document: A1 |