WO2004047427A1 - フリッカ低減方法、撮像装置およびフリッカ低減回路 - Google Patents
フリッカ低減方法、撮像装置およびフリッカ低減回路 Download PDFInfo
- Publication number
- WO2004047427A1 WO2004047427A1 PCT/JP2003/013237 JP0313237W WO2004047427A1 WO 2004047427 A1 WO2004047427 A1 WO 2004047427A1 JP 0313237 W JP0313237 W JP 0313237W WO 2004047427 A1 WO2004047427 A1 WO 2004047427A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- flicker
- component
- input image
- signal
- image signal
- Prior art date
Links
- 230000009467 reduction Effects 0.000 title claims description 124
- 238000000034 method Methods 0.000 title claims description 78
- 238000012545 processing Methods 0.000 claims abstract description 77
- 238000001228 spectrum Methods 0.000 claims abstract description 41
- 238000010606 normalization Methods 0.000 claims abstract description 16
- 238000003384 imaging method Methods 0.000 claims description 56
- 238000005286 illumination Methods 0.000 claims description 43
- 230000010354 integration Effects 0.000 claims description 16
- 239000003086 colorant Substances 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 abstract description 37
- 230000008859 change Effects 0.000 description 42
- 238000001514 detection method Methods 0.000 description 41
- 238000010586 diagram Methods 0.000 description 30
- 230000002829 reductive effect Effects 0.000 description 24
- 238000012937 correction Methods 0.000 description 15
- 230000007704 transition Effects 0.000 description 13
- 230000000694 effects Effects 0.000 description 9
- 230000000295 complement effect Effects 0.000 description 8
- 238000005070 sampling Methods 0.000 description 6
- 239000002131 composite material Substances 0.000 description 4
- 230000014509 gene expression Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000004091 panning Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000007423 decrease Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000000926 separation method Methods 0.000 description 3
- 230000003595 spectral effect Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000002238 attenuated effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007562 laser obscuration time method Methods 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 238000011946 reduction process Methods 0.000 description 2
- 230000004043 responsiveness Effects 0.000 description 2
- 238000003775 Density Functional Theory Methods 0.000 description 1
- 108010076504 Protein Sorting Signals Proteins 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000002542 deteriorative effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/76—Circuitry for compensating brightness variation in the scene by influencing the image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/745—Detection of flicker frequency or suppression of flicker wherein the flicker is caused by illumination, e.g. due to fluorescent tube illumination or pulsed LED illumination
Definitions
- the present invention relates to an image pickup device when an object is photographed by an XY address scan type image pickup device (imager, image sensor) such as a CMOS (complementary metal oxide semiconductor) image pickup device under the illumination of a fluorescent lamp.
- an imaging device such as a video camera using an XY address scanning type image sensor such as a CMOS image sensor, a digital still camera, and the like.
- the present invention relates to a fritting force reducing circuit used in an imaging device.
- the frequency of the luminance change (fluctuation change) of the fluorescent lamp tilt the commercial AC power frequency
- the difference in frequency causes a temporal change in light and dark in the video signal of the shooting output, that is, a so-called fluorescent light flickering force.
- the commercial AC power frequency is 50 Hz
- the period of one field is 1/60 seconds
- the period of the luminance change of the fluorescent lamp is 100 seconds, so that the change in the luminance of the fluorescent lamp
- the exposure timing of each field shifts, and the exposure amount of each pixel changes for each field.
- the exposure amount is different even for the same exposure time, and when the exposure time is shorter than 1/60 second (however, During the period bl, b2, and b3, the exposure amount is different even during the same exposure time. Since the exposure timing for the change in the luminance of the fluorescent lamp returns to the original timing every three fields, the change in brightness due to the fritting force is repeated every three fields. In other words, the luminance ratio of each field (the appearance of the fritting force) changes depending on the exposure period, but the period of the fritting force does not change. However, in the case of a progressive camera such as a digital camera and the vertical synchronization frequency is 30 Hz, the light / dark changes are repeated every three frames.
- fluorescent lamps generally use a plurality of phosphors, for example, red, green, and blue phosphors, to emit white light.
- each of these phosphors has a unique afterglow characteristic, and emits light with attenuated light with each afterglow characteristic during the period from the stop of discharge existing during the cycle of luminance change to the start of the next discharge. Therefore, during this period, the light that was initially white is attenuated while gradually changing its hue. If the exposure timing is shifted as described above, not only the light and dark changes but also the hue changes.
- fluorescent lamps have a unique spectral characteristic that a strong peak exists at a specific wavelength, the signal fluctuation component differs depending on the color.
- the exposure time is set to an integral multiple of the cycle of change in the brightness of the fluorescent lamp (100 times 100 seconds)
- the exposure amount can be set regardless of the exposure timing. Is constant, and no fritting force is generated.
- the exposure time is set to an integer multiple of 110 seconds when under fluorescent light illumination by detecting the presence of fluorescent light illumination by user operation or by signal processing in the camera. A scheme is being considered. According to this method, it is possible to completely prevent the generation of the frit force by a simple method.
- the flit force will be a repetition period of 3 fields, and the average value of the video signal in each field will be constant
- the exposure timing of each pixel is sequentially shifted by one period of a read clock (pixel clock) in the horizontal direction of the screen, and the exposure timing is increased for all pixels. Because of the different lighting, the above method cannot sufficiently suppress flicker.
- Figure 29 shows this situation. As described above, the exposure timing of each pixel is sequentially shifted even in the horizontal direction of the screen. Since the normal cycle is sufficiently short, the exposure timing of each line in the vertical direction of the screen is shown, assuming that the exposure timing is the same for pixels on the same line. In practice, there is no problem with this assumption.
- the exposure timing differs for each line (F1 shows the state in a certain field), and the exposure is performed in each line. Because of the difference in light intensity, light and dark changes and color changes due to fringe force occur not only between fields but also within fields, and stripes appear on the screen (the stripes themselves are horizontal, and the stripes are vertical. Direction).
- this stripe pattern corresponds to five periods (five wavelengths) in three fields (three screens), and appears to flow vertically when viewed continuously.
- FIGS. 30 and 31 show only the change in brightness due to flicker, but in fact, the above-mentioned color change is also added, and the image quality is significantly deteriorated. In particular, the color fretting force becomes remarkable as the shutter speed increases, and the image quality degradation becomes more conspicuous in an XY address scanning type image sensor because the effect appears in the screen.
- the exposure time can be set to an integral multiple of the cycle of the luminance change of the fluorescent lamp (1/100 second), regardless of the exposure timing.
- the exposure amount is constant, and no fluorescent lamp flicker including in-screen flicker occurs.
- the electronic shutter speed is made variable with a CMOS image sensor or the like, the imaging device becomes complicated.
- the exposure time can be set only to an integral multiple of 1Z100 seconds to prevent flicker, the exposure adjustment to obtain the appropriate exposure The degree of freedom of the means is reduced.
- Patent Document 1 Japanese Patent Application Laid-Open No. 2000-3500102
- Patent Document 2 Japanese Patent Application Laid-Open No. 2000-230400
- Patent Document 3 Japanese Unexamined Patent Application Publication No. 2000-16508 discloses a first electronic shutter value suitable for the current external light condition and a flicker cycle of a fluorescent lamp having a predetermined relationship.
- Patent Document 4 Japanese Unexamined Patent Publication No. 11-1641642 discloses a method in which the state of a change in brightness under fluorescent light illumination is recorded in advance in a memory as a correction coefficient, while the video signal There is a method of detecting the phase of the frit component from the video signal from the image sensor using the difference in frequency between the component and the flicker component, and correcting the video signal with a correction coefficient in the memory according to the detection result. It is shown.
- Patent Document 5 Japanese Unexamined Patent Application Publication No. 2000-0-167052 discloses that two video signals exposed with a time difference such that the phase of the fritting force is inverted by exactly 180 degrees. It shows a method of calculating a correction coefficient and correcting a video signal with the calculated correction coefficient.
- a method of estimating a flicker component by measuring the amount of light of a fluorescent lamp using a light receiving element or a photometric element is to add a light receiving element or a photometric element to an imaging device. Therefore, the size / cost of the imaging device system increases.
- the method of estimating a flicker component by photographing two kinds of images under different shirt conditions also has a disadvantage that the system of the imaging device is complicated. Moreover, this method has a disadvantage that it is not suitable for shooting moving images.
- the method of using a coefficient prepared in a memory as a correction signal does not allow correction coefficients to be prepared for all types of fluorescent lamps. Depending on the type of lamp, there is a drawback that the fritz force component cannot be accurately detected and reduced.
- a black component having a very small frit component is used in the method of extracting a flicker component from a video signal by using a difference in frequency between the video signal component and the flicker component. It is difficult to detect flicker components separately from video signal components in the background and low-illumination areas, and when there is a moving object on the screen, the flicker component detection performance is poor. Significantly reduced.
- Patent Document 5 a method of estimating a frit component by capturing two images under different timing conditions is similar to the method described in Patent Document 3, in which the system of the imaging device is used. There are drawbacks that make it more complicated and unsuitable for movie shooting.
- the present invention uses the simple signal processing without the use of a light receiving element or the like to reduce the fluorescent light flit force inherent in an XY address scanning type imaging element such as a CMOS imaging element to a subject or a video signal. Regardless of the type of fluorescent lamp, etc. Is made possible. Disclosure of the invention
- the flicker reduction method of the first invention is as follows.
- the flicker reduction method of the second invention is as follows.
- the method for reducing flicker of the third invention is as follows.
- a signal component other than the frit force component is removed as the integrated value or the difference value after the normalization, so that the flicker is reduced regardless of the object.
- a signal that can easily estimate the frit component with high accuracy is obtained even in a black background portion or a low illuminance portion where the component is minute, and the normalized integrated value or the difference value up to the appropriate order
- the Fritz force component can be estimated with high accuracy regardless of the type of fluorescent lamp or the waveform of luminance change, and even in the area where the signal component is discontinuous due to the influence of the subject.
- the flicker component can be reliably and sufficiently reduced from the input image signal.
- the fritting force component is detected for each color signal of each color obtained as a video signal, or for each luminance signal and each color signal of each color. Therefore, it is possible to detect the fluorescent light frit power including the color frit power with high accuracy, and to surely and sufficiently reduce it.
- FIG. 1 is a diagram showing a system configuration of an embodiment of the imaging apparatus of the present invention.
- FIG. 2 is a diagram illustrating an example of a digital signal processing unit of a primary color system.
- FIG. 3 is a diagram illustrating an example of a digital signal processing unit of a color capturing system.
- FIG. 3 is a diagram illustrating an example of FIG.
- FIG. 5 is a diagram illustrating an example of an operation block when a saturation region is considered.
- FIG. 6 is a diagram showing a second example of the fritting force reducing unit.
- FIG. 7 is a diagram illustrating a third example of the fritting force reducing unit.
- FIG. 8 is a diagram illustrating an example of a flicker reduction unit when non-fluorescent lighting is considered.
- FIG. 9 is a diagram illustrating another example of the flicker reduction unit when considering under non-fluorescent lighting.
- FIG. 10 is a diagram illustrating an example of an imaging apparatus in consideration of a case where a subject changes greatly in a short time due to an operation or operation of a photographer.
- FIG. 11 is a diagram illustrating an example of an imaging apparatus in a case where a case where flicker reduction processing is not required depending on a shooting state is considered.
- FIG. 12 is a diagram illustrating another example of the imaging apparatus in a case where the case where the flicker reduction processing is not required depending on the imaging state is considered.
- FIG. 13 shows the basic structure of an example where the estimated flicker component is adjusted.
- FIG. 14 is a diagram illustrating a first specific example in the case of adjusting the estimated flicker component.
- FIG. 15 is a diagram illustrating a second specific example in the case where the estimated flicker component is adjusted.
- FIG. 16 is a diagram provided for explanation of the examples of FIG. 14 and FIG.
- FIG. 17A and FIG. 17B are diagrams showing equations for explanation of each example.
- FIG. I8A and FIG. 18B are diagrams showing equations for explanation of each example.
- FIG. 19A and FIG. 19B are diagrams showing equations used for explaining each example.
- FIG. 20A to FIG. 20E are diagrams showing equations used to explain each example.
- FIG. 21A to FIG. 21C are diagrams showing equations for explanation of each example.
- FIGS. 22A and 22B are diagrams for explaining the examples of FIGS. 8 and 9.
- FIG. 23 is a diagram showing a subject used in the test.
- FIG. 24 is a diagram showing integral values in the case of the subject shown in FIG.
- FIG. 25 is a diagram showing difference values in the case of the subject in FIG.
- FIG. 26 is a diagram showing the difference value after the normalization in the case of the subject in FIG. 23.
- FIG. 27 is a diagram showing the estimated flicker coefficient in the case of the subject in FIG.
- FIG. 28 is a diagram for describing the flickering force of a fluorescent lamp in the case of a CCD image sensor.
- FIG. 29 is a diagram provided to explain the flickering force of a fluorescent lamp in the case of an XY address scanning type image sensor.
- FIG. 30 shows the fluorescent lamp flit force in the case of the XY address scanning type image sensor.
- FIG. 3 is a diagram showing a stripe pattern in one screen.
- FIG. 31 is a diagram showing a stripe pattern over three screens in which the fluorescent lamp flit force is continuous in the case of an XY address scanning type image sensor.
- FIGS. 1 to 3 [Embodiment of imaging apparatus: FIGS. 1 to 3]
- FIG. 1 shows a system configuration of an embodiment of an imaging apparatus according to the present invention, which is a case of a video camera using a CMOS image sensor as an XY address scanning type image sensor.
- the imaging device of this embodiment that is, the video camera
- light from a subject enters the CMOS image sensor 12 via the imaging optical system 11, and is photoelectrically converted by the CMOS image sensor 12.
- An analog video signal is obtained from the image sensor 12.
- the CMOS image sensor 12 consists of a photodiode (photogate), a transfer gate (Shatta transistor), a switching transistor (address transistor), an amplifying transistor, and a reset transistor (reset gate) on a CMOS substrate.
- a plurality of pixels having such a configuration are formed in a two-dimensional array, and a vertical scanning circuit, a horizontal scanning circuit, and a video signal output circuit are formed.
- the CMOS image sensor 12 may be either a primary color system or a color capture system as described later.
- the analog video signal obtained from the CMOS image sensor 12 is a primary color signal of each RGB color or a color signal of the color capture system. is there.
- the analog video signal from the CMOS image sensor 12 is sampled and held for each color signal in an analog signal processing unit 13 configured as an integrated circuit (IC), and gain is controlled by an automatic gain control (AGC). Is controlled and converted to a digital signal by A / D conversion.
- IC integrated circuit
- AGC automatic gain control
- the digital video signal from the analog signal processing unit 13 is processed as described later in a digital signal processing unit 20 configured as an IC.
- the flicker component is reduced by the method of the present invention for each signal component as described later, and finally the luminance signal Y and
- the signals are converted to red and blue color difference signals R—Y and ⁇ — ⁇ , and output from the digital signal processor 20.
- the system controller 14 is composed of a microcomputer or the like and controls each part of the camera.
- a lens drive control signal is supplied from a system controller 14 to a lens drive driver 15 constituted by an IC, and the lens of the imaging optical system 11 is driven by the lens drive driver 15.
- a timing control signal is supplied from the system controller 14 to the timing generator 16, various timing signals are supplied from the timing generator 16 to the CMOS image sensor 12, and the CMOS image sensor 12 is driven.
- the detection signal of each signal component is taken into the system controller 14 from the digital signal processing unit 20, and the analog signal is sent from the system controller 14 to the analog signal processing unit 13 as described above.
- the system controller 14 controls the signal processing in the digital signal processing unit 20.
- a camera shake sensor 17 is connected to the system controller 14. If the subject changes greatly in a short time due to the operation of the photographer, this is determined from the output of the camera shake sensor 17.
- the flicker reduction unit 25 is controlled as described later.
- the system controller 14 also includes an operation unit 18a and a display unit 18b that constitute a user interface 18 via an interface (I / F) 19 configured by a microcomputer or the like.
- an operation unit 18a When connected, the setting and selection operations on the operation unit 18a are performed by the system controller 1. 4 and the camera setting state and control state are displayed on the display section 18 b by the system controller 14.
- the photographer changes the subject in a short period of time by performing a camera operation such as a zoom operation on the operation unit 18a, this is detected by the system controller 14 and the camera will be shifted as described later.
- the force reduction unit 25 is controlled.
- FIG. 2 shows an example of the digital signal processing unit 20 in the case of a primary color system.
- the imaging optical system 11 of FIG. 1 has a separation optical system that separates light from a subject into color light of each color of RGB, and has a CMOS image sensor for each color of RGB as the CMOS image sensor 12.
- CMOS image sensor for each color of RGB as the CMOS image sensor 12.
- CMOS image sensor 12 As a three-chip system, or as a CMOS image sensor 12, a single CMOS image sensor with a color filter for each color of RGB on the light-incident surface, which is arranged repeatedly in the horizontal direction of the screen, one pixel at a time. System. In this case, primary color signals of each of the RGB colors are read out in parallel from the CMOS image sensor 12.
- the clamp circuit 21 clamps the black level of the input RGB primary color signal to a predetermined level
- the gain adjustment circuit 22 clamps the RGB primary colors according to the amount of exposure.
- the gain of the signal is adjusted, and the flicker force reduction units 25R, 25G, and 25B reduce the flicker component in the RGB primary color signal after the gain adjustment according to the method of the present invention. Is done.
- the white balance adjustment circuit 27 adjusts the white balance of the RGB primary color signal after flicker reduction.
- the gamma correction circuit 28 converts the gradation of the RGB primary color signal after white balance adjustment, and the composite matrix circuit 29 outputs the output luminance signal from the RGB primary color signal after gamma correction. Y and color difference signals R_Y, BY are generated.
- the luminance signal Y is generally generated after all the RGB primary color signal processing is completed as shown in Fig. 2, so that as shown in Fig. 2, the RGB primary color signal is processed in the RGB primary color signal in a free-range manner.
- frits-force reduction units 25R, 25G, and 25B are arranged as shown in FIG. 2, but the arrangement is not necessarily limited thereto.
- FIG. 3 shows an example of the digital signal processing unit 20 in the case of a complementary color system.
- the complementary color system is a one-chip system having one CMOS image sensor having a complementary color filter formed on the light incident surface as the CMOS image sensor 12 of FIG.
- a color filter 1 in FIG. 3 at every other horizontal line position Lo, the green color filter 1 G and the magenta color filter lMg are horizontal.
- the cyan color filter section ICy and the yellow color filter section lYe are arranged one by one in the horizontal direction. They are sequentially and repeatedly arranged.
- the clamp circuit 21 clamps the black level of the color capture signal to a predetermined level, and the gain adjustment circuit 22 adjusts the gain of the complementary color signal after clamping according to the amount of exposure. Is adjusted, the luminance signal Y is generated from the complementary color signal after the gain adjustment in the luminance synthesis circuit 23, and the primary color separation circuit 24 generates the RGB primary color signal from the color capture signal after the gain adjustment. Is done.
- the fritting force reducing unit 25 Y reduces the fritting force component in the luminance signal Y from the luminance synthesizing circuit 23 by the method of the present invention.
- flicker components in the RGB primary color signals from the primary color separation circuit 24 are reduced by the fritting force reduction units 25R, 25G, and 25B.
- the gamma correction circuit 26 captures the gradation of the luminance signal after flicker reduction to obtain the luminance signal Y of the output, and the white balance adjustment circuit In 27, the white balance of the RGB primary color signal after the reduction of the frit force is adjusted, and in the gamma correction circuit 28, the gradation of the RGB primary color signal after the white balance adjustment is converted, and the composite matrix circuit 2 In step 9, the color difference signals R—Y and ⁇ — ⁇ are generated from the gamma-corrected RGB primary color signals.
- a luminance signal and an RGB primary color signal are generated relatively before the digital signal processing unit 20. This is because the luminance signal can be easily generated by a simple addition process from the above synthesized signal, and an RGB primary color signal is generated from the above synthesized signal by difference processing, and a luminance signal is generated from the RGB primary color signal. Then, the S / N of the luminance signal deteriorates.
- frits-force reduction units 25Y, 25R, 25G, and 25B are arranged as shown in FIG. 3, but it is not necessarily limited to this.
- the input image signal is an RGB primary color signal or a luminance signal before flicker reduction, which is input to the flit force reduction unit 25, respectively, and the output image signal is a flit signal, respectively.
- This is the RGB primary color signal or the luminance signal output from the power reduction unit 25 after flicker reduction.
- the subject is photographed by an NTSC (60 Hz vertical synchronization frequency) CMOS camera under fluorescent lighting in an area where the commercial AC power supply frequency is 50 Hz. Therefore, when the fluorescent light frit power is not reduced, as shown in FIGS. 29 to 31, the light / dark change and color change due to the frit force occur not only between the fields but also within the field, and the Above is the case where three fields (three screens) appear as stripes for five periods (five wavelengths).
- NTSC 60 Hz vertical synchronization frequency
- FIG. 4 shows a first example of the fritted force reducing section 25.
- FIGS. 30 and 31 show the case where the subject is uniform.
- the 'fritz force component is proportional to the signal intensity of the subject.
- the input image signal (RGB primary color signal or luminance signal before the reduction of the frit force) of a general subject in an arbitrary field n and an arbitrary pixel ( X , y) is defined as In, (x, y).
- I n ′ (x, y) is expressed by the equation (1) in FIG. 17A as the sum of a signal component that does not include a flicker component and a flicker component proportional to the signal component.
- In (x, y) is a signal component
- ⁇ ⁇ (y) * In (x, y) is a frit component
- ⁇ ⁇ (y) is a frit component.
- One horizontal period is sufficiently shorter than the emission period of a fluorescent lamp (1/100 second), and the flicker coefficient can be regarded as constant for the same line in the same field. ⁇ ⁇ n (y).
- ⁇ mn indicates the initial phase of each of the following fritting force components, and is determined by the emission cycle (1/100 second) of the fluorescent lamp and the exposure timing. However, since ⁇ has the same value every three fields, the value of ⁇ m ⁇ between the previous field and The difference is expressed by equation (3) in Fig. 17A.
- the integration block 31 uses the equation (4) in FIG. As shown in), integration is performed over one line in the horizontal direction of the screen, and an integrated value F n (y) is calculated.
- a n (y) in Eq. (4) is the integral value over one line of the signal component In (X, y), as expressed by Eq. (5) in FIG. 17B.
- the calculated integrated value F n (y) is stored and held in the integrated value holding block 32 for detecting the fritting force in the subsequent fields.
- the integrated value holding block 32 is configured to hold the integrated value of at least two fields. If the subject is uniform, the integral value an (y) of the signal component I n (x, y) becomes constant, so the integral value F n (y) of the input image signal I n '(x, y) It is easy to extract the flicker component an (y) * ⁇ n (y) from
- an (y) also includes the m * cuo component, the luminance component and the color component as the frit component, and the luminance component and the color component as the subject's own signal component Cannot be separated, and it is not possible to purely extract only the frit component.
- the flicker component of the second term is very small compared to the signal component of the first term in equation (4), the flicker component is almost completely buried in the signal component.
- Fig. 24 shows the integral value F n (y) of the subject shown in Fig. 23 (actually a color image) in three consecutive fields. This is for the red signal.
- Field: N + 0 (solid line), Fie 1 d: N + 1 (dashed line), Fie 1 d: N + 2 (dotted line) are three consecutive fields. These are the first, second, and third fields in.
- the Fritz force component is directly extracted from the integral F n (y). Is impossible.
- the integral value holding block 3 2 when calculating the integral value F n (y), the integral value holding block 3 2 The integrated value F n — 2 (y) of the same line is read out, and the average calculation block 3 3 is used to average the three integrated values F n (y), F n — l (y), and F n — 2 (y) The value AVE [F n (y)] is calculated.
- the integral value holding block 3 2 is configured to hold the integral values of at least (j ⁇ 1) fields.
- the example in FIG. 4 is a case where the approximation of equation (7) in FIG. 18A holds.
- the difference calculation block 34 further calculates the integration value F n (y) of the field from the integration block 3 1 and the integration value F n__l (y) of the previous field from the integration value holding block 3 2. Is calculated, and a difference value Fn (y) -Fn-1 (y) represented by the equation (8) in FIG. 18B is calculated. Equation (8) also assumes that the approximation of equation (7) holds.
- FIG. 25 shows difference values F n (y) —F n—1 (y) in three consecutive fields for the subject shown in FIG.
- the difference value Fn (y) -Fn-1 (y) sufficiently removes the influence of the subject, so that the difference between the difference value Fn (y) and the integral value Fn (y) shown in FIG.
- the appearance of the force component clearly appears.
- the difference value F n from the difference calculation block 3 4 (y) - F n- 1 (y) is the average value AVE of the average calculation block 3 3 Normalization is performed by dividing by [F n (y)], and the normalized difference value gn (y) is calculated.
- the difference value F n (y) -F n_l (y) is affected by the signal strength of the subject, so the level of luminance change and color change due to flicker differs depending on the area. Thus, the luminance change and the color change due to flicker can be adjusted to the same level over the entire area.
- FIG. 26 shows normalized difference values g n (y) in three consecutive fields for the subject shown in FIG.
- , 0 m which is expressed by the equation (11a) (lib) in Fig. 19B, is the amplitude of the next-order spectrum of the normalized difference value gn (y). If the normalized difference value gn (y) is Fourier-transformed to detect the amplitude I Am I of each next-order spectrum and the initial phase 0 m, the equation (1 2a) By using (1 2b), the amplitude ⁇ m and initial phase ⁇ of each order flicker component shown in equation (2) in Fig. 17A can be obtained. Therefore, in the example of FIG. 4, in the DF ⁇ block 51, the difference value gn (y) after the normalization from the normalization block 35 is equivalent to one wavelength (L line) of the Fritz force. Perform discrete Fourier transform on the data.
- Equation (13) is DFT [g n (y)] and the DFT result of order m is Gn (m)
- the DFT operation is expressed by equation (13) in FIG. 20B.
- W in equation (13) is represented by equation (14).
- Equations (15a) and (15) in FIG. 20C the relationship between Equations (11a) (lib) and Equation (13) is represented by Equations (15a) and (15) in FIG. 20C.
- the equations (16a) and (16b) of FIG. The amplitude 7 m and the initial phase ⁇ can be obtained.
- the data length of the DFT calculation is set to one wavelength (L line) of the fritting force. This means that a discrete vector group that is an integral multiple of COo can be directly obtained. Because it can be.
- FFT Fast Fourier Transform
- DFT Downward Fast Fourier Transform
- the spectrum is extracted by the DFT operation defined by the expression (13), and then the next free-range is calculated by the operation of the expressions (16a) and (16b).
- the amplitude ⁇ m of the force component and the initial phase ⁇ are estimated.
- the estimated value of ⁇ , ⁇ m ⁇ from the DF block 51 is used to calculate the flicker expressed by the equation (2) in FIG.
- the force coefficient ⁇ ⁇ (y) is calculated.
- the Fritz force component can be sufficiently approximated, so that the Fritz force coefficient ⁇ ⁇ ( In calculating y), the total order is not infinity but can be limited to a predetermined order, for example, the second order.
- FIG. 27 shows the Fritz force coefficient ⁇ ⁇ (y) in three consecutive fields for the subject shown in FIG.
- the flicker force component is completely buried in the signal component, and the flicker component is a small area such as a black background portion or a low illuminance portion.
- the frit component can be detected with high accuracy. be able to. '
- estimating the flicker component from the spectrum up to an appropriate order approximates the normalized difference value gn (y) without completely reproducing it.
- the frit component of that part can be accurately estimated.
- the arithmetic block 40 adds 1 to the flicker coefficient ⁇ n (y) from the fritz force generation block 53 and inputs the sum [1 + ⁇ (y)].
- the image signal In '(x, y) is divided.
- the frit component contained in the input image signal In ′ (X, y) is almost completely removed, and the output block 40 outputs the output image signal (the RGB primary colors after the frit reduction) from the arithmetic block 40.
- a signal or a luminance signal a signal component I n (x, y) substantially free from flicker components is obtained.
- the calculation block 40 is used by utilizing the fact that the flicker is repeated every three fields.
- a function is provided to hold the Fritz force coefficient ⁇ ⁇ (y) over three fields, and the held Fritz force coefficient is stored for the input image signal I n ′ (x, y) after three fields ⁇ ⁇ (y) may be calculated. (Example when the saturation region is considered: Fig. 5)
- the arithmetic block 40 when the level of the 'input image signal In' (x, y) is in the saturation region, when the operation of the expression (17) is performed by the operation block 40, the signal component (color) Component or luminance component). Therefore, it is desirable that the arithmetic block 40 be configured as shown in the example of FIG.
- the operation block 40 in the example of FIG. 5 is an addition circuit 41 that adds 1 to the flicker coefficient ⁇ n (y) from the flicker generation block 53, and the sum [1 + ⁇ n (y)] It is composed of a divider circuit 42 that divides the input image signal I n ′ (x, y), a switch 43 on the input side, a switch 44 on the output side, and a saturation level determination circuit 45.
- the level determination circuit 45 determines, for each pixel, whether or not the level of the input image signal In, (X, y) is equal to or higher than the threshold level of the saturation region.
- the switches 43 and 44 are shifted to the opposite side by the saturation level determination circuit 45 in the pixel.
- the input image signal I n ′ (x, y) is output as it is from the operation block 40 as an output image signal.
- Figure 6 shows an example of this case.
- the integrated value F n (y) from the integration block 31 is replaced by the average value AVE [F n (y) from the average calculation block 33. ]
- the normalized difference value gn (y) is obtained.
- each of the DFT blocks is calculated by the equations (16a) and (16b). While the amplitude of the next Fritz force component ⁇ m and the initial phase ⁇ are estimated, in the example of Fig. 6, after extracting the spectrum by the DFT operation defined by equation (13), The amplitude ⁇ m and the initial phase ⁇ of each order flicker component are estimated by the operations of the equations (20a) and (20b). Subsequent processing is the same as in the example of FIG.
- the flicker reduction unit 25 can be simplified accordingly. Also in this example, it is desirable that the arithmetic block 40 be configured as in the example of FIG.
- the average value AVE [F n (y)] used for normalization in the example of FIG. 4 is expressed as an (6) when an approximation of equation (7) in FIG. y) and the second term [an (y) * ⁇ n (y)] in equation (4) in Fig. 17B is sufficiently smaller than the first term an (y). The effect of the second term on the impact is very small.
- the difference value Fn (y) —Fn—1 (y) from the difference calculation block 34 is converted to the integration value Fn ( Normalize by dividing by y). Subsequent processing is the same as in the example of FIG.
- the integral value holding block 32 only needs to be able to hold the integral value for one field, and does not require the average value calculation block 33, so that the fritting force reduction section 25 can be simplified. it can.
- the arithmetic block 40 be configured as in the example of FIG.
- FIG. 8 shows an example in which the flicker reduction section 25 is configured as described above.
- the normalized integration value calculation block 30 is configured as in the example of FIG. 4, FIG. 6, or FIG. In the examples of FIGS. 4 and 7, not the integral value F n (y) but the differential value F n (y) -one F n — 1 (y) is normalized, but for convenience, the normalized integral value is calculated. Called block.
- a fluorescent light illumination judgment block 52 is provided between the DFT block 51 and the fritting force generation block 53.
- the level (amplitude) ⁇ ⁇ of each next component estimated and calculated in the DFT block 51 is as shown in Fig. 22 ⁇ under the fluorescent lamp illumination. While the level is sufficiently larger than a certain threshold value T h and decreases rapidly as m increases, under non-fluorescent lighting, as shown in Fig. 22B, the level of each next component becomes the threshold value T h or less.
- the spectrum should be zero, but in practice, because the subject moves, the normalized difference value gn (y ) Or the integrated value gn (y) 11 inevitably contains a small number of frequency components.
- the fluorescent light illumination determination block 52 determines that the illumination is not performed by the non-fluorescent lamp, and for all the orders m, ⁇ Zero the estimate of m.
- the flicker coefficient ⁇ n (y) also becomes zero, and the input image signal In, (x, y) is output from the arithmetic block 40 as an output image signal as it is.
- Figure 9 shows another example.
- the fluorescent light illumination determination block 52 determines whether or not it is under fluorescent light illumination, as in the example of FIG. 8, but if it is determined that it is under non-fluorescent light illumination, The detection flag COMP—OFF is set to stop the processing in the flicker generation block 53 and the operation block 40, and the input image signal I n ′ (X, y) is used as it is as the output image signal. Output from the calculation block 40. If it is under fluorescent light illumination, the detection flag COMP-OFF is reset, and the flicker reduction processing is executed as described above.
- the imaging apparatus is configured as shown in, for example, FIG.
- FIG. 10 As the fritting force reducing unit 25, in the example of FIG. 4, FIG. 6 or FIG. 5 6 and frits force holding block 57 are provided, A detection flag DET-OFF, which will be described later, output from the system controller 14 is supplied to the switches 55 and 56 as a switching signal. It is assumed that the flicker holding block 57 can hold the flicker coefficient ⁇ n (y) for three fields, and the flicker force coefficient ⁇ ⁇ (y) is changed every time the processing for one field is completed. In addition to storing the data for each field, the read output can be switched by repeating every three fields.
- the system controller 14 controls the driving of the lens according to the photographer's zoom operation such as telephoto or wide-angle operation.
- the camera shake of the photographer is detected by the camera shake sensor 17 and the camera shake information is sent to the system controller 14.
- the system controller 14 controls camera shake correction based on the camera shake information. Panning / tilting is also detected by the system controller 14, and the system controller 14 performs control such as weakening camera shake correction during pan junging.
- Such detection control is generally the same as that performed by a camera. Further, in the example of FIG.
- the system controller 14 when the system controller 14 detects a photographer's operation or movement in which the subject changes greatly in a short time, the system controller 14 sets the detection flag DET-OFF, and If no operation or operation is performed, reset the detection flag DET—OFF. Then, in a normal state where the subject does not change greatly in a short time, the detection flag DET-OFF is reset, so that the switch 55 is switched to the flicker generation block 5 by the flicker force reduction unit 25. Switched to 3 side, The current flit force coefficient ⁇ (y) from the flit force generation block 53 is supplied to the operation block 40, and the flicker reduction processing is executed, and the switch 56 is turned on. Then, the flicker force coefficient ⁇ n (y) at that time is stored in the flicker holding block 57.
- the detection flag DET-OFF is set, and the flicker reduction section 25 causes the switch 55 to flicker.
- the holding block 57 side reading from the fritting force holding block 57 instead of the fritting force coefficient ⁇ ⁇ ⁇ (y) with the poor detection accuracy at that time from the fritting force generating block 53
- the flit force coefficient ⁇ ⁇ n (y) force S with good detection accuracy is supplied to the calculation block 40 to reduce the flit force.
- the switch 56 is turned off, and the flicker coefficient ⁇ ⁇ n (y) with poor detection accuracy at that time is prevented from being stored in the fritz force holding block 57.
- the detection flag DET-OFF is also sent to the normalized integration value calculation block 30, the DFT block 51, and the flicker generation block 53, and the subject is operated or operated by the photographer.
- the detection flag DET-OFF is set, and the normalized integration value calculation block 30, the DFT block 51, and the fritz force generation block 53 are set. Is stopped. Therefore, in this example, power consumption can also be reduced.
- the fritz force coefficient ⁇ ⁇ (y) is replaced with the immediately preceding signal, but a signal at a previous stage, for example, the integral value F n (y) is replaced with the immediately preceding signal. You may be comprised so that it may be replaced with a number.
- the first situation in which the fritting force reduction processing is unnecessary is, for example, the case where a still image is captured by a video camera / digital still camera capable of capturing both moving images and still images.
- the exposure timing (exposure start and exposure end timing) of all pixels in one screen can be made the same, and the It is possible to avoid the generation of a light fritting force.
- Readout from the image sensor can be performed slowly with the mechanical shutter closed and shielded from light because there is no restriction on the frame rate as in the case of shooting a moving image.
- the shooting conditions that do not require fritz force reduction processing are as follows.
- the exposure time electronic shutter time
- the exposure time can be adjusted to the brightness of the fluorescent lamp.
- it is set to an integral multiple of the change cycle (1/100 second).
- Whether it is under fluorescent light illumination can be detected from the level of the spectrum extracted by the DFT block 51, as shown in the examples of FIGS.
- the system controller 14 can directly detect the presence of non-fluorescent lighting even from the same non-fluorescent lighting under the same non-fluorescent lighting in the system controller 14.
- the exposure time is set to an integral multiple of the cycle of the luminance change of the fluorescent lamp (1Z100 seconds). Occasionally, no fluorescent light fringing force including the in-screen fringing force is generated.
- the system controller 14 can directly detect whether or not the exposure time has been set to an integral multiple of the cycle of the luminance change of the fluorescent lamp by adjusting the exposure amount or the like.
- the system controller 14 determines that the shooting state does not require the fringe force reduction processing, the flicker reduction processing is not performed, and the input image signal I n ′ ( The system is configured so that x, y) is output as-is from the flicker reduction unit 25 as an output image signal.
- FIG. 11 shows an example of such a system configuration.
- a zeroing block 59 is provided between the DFT block 51 and the flicker generation block 53 to reduce the flit force from the system controller 14.
- the zeroing block 59 is controlled by the on / off control signal.
- the flicker reduction on / off control signal is set to the on state, and the zeroing block 59 is controlled by the DFT block 51.
- the estimated values of ym and ⁇ are output to the flicker generation block 53 as they are. Therefore, in this case, the flicker reduction processing is executed as described above.
- the system controller 14 determines that the flicker force reduction processing is unnecessary, the flicker reduction on / off control signal is turned off, and The zeroing block 59 sets the estimated value of ⁇ to zero for all orders m. Therefore, in this case, the flicker coefficient ⁇ n (y) also becomes zero, and the input image signal In, (x, y) force S is output from the operation block 40 as an output image signal as it is.
- FIG. 12 shows another example.
- the operation block 40 of the fritting force reduction section 25 has the addition circuit 41, the division circuit 42, and the switches 43, 44 shown in the example of FIG.
- the switch is configured so as not to include the determination circuit 45, and the switches 43 and 44 are switched by a flicker reduction on / off control signal from the system controller 14.
- the switches 43 and 44 are switched to the division circuit 42 side, and as described above, the equation (17) ) Is output from the operation block 40 as an output image signal.
- the switches 43 and 44 are switched to the opposite side, and the input image signal I n ′ (X, y) remains unchanged. It is output from the operation block 40 as an output image signal.
- the fritting force reduction on / off control signal is also sent to the normalized integral value calculation block 30, the DFT block 51, and the fritting force generation block 53, and the system controller 14.
- the processing in the normalized integral value calculation block 30, the DFT block 51, and the flicker generation block 53 is stopped. Therefore, in this example, power consumption can also be reduced.
- an average calculation or a difference calculation between a plurality of fields is performed in detecting a frit force component, so that the instant when the switch of the fluorescent lamp is turned on or off, or under the fluorescent lamp illumination, In a transitional unstable state, such as when entering a room or exiting a room under fluorescent lighting, the fritz force component cannot be detected accurately. Therefore, if the flicker reduction process is performed using the flicker component obtained in such a state, an undesirable correction may be performed on the input image signal.
- the horizontal angle of view change caused by camera operation such as horizontal subject movement or panning / zooming or camera shake is reliably and stably flipped.
- flit force reduction performance is slightly reduced for vertical movement of the subject due to camera movements such as vertical subject movement, tilting and zooming, or camera shake. Lower.
- LPF low-pass filter
- the following example is a case in which the amplitude and the phase of the estimated flicker component, which are parameters related to the flicker force reduction, are adjusted.
- Figure 13 shows the basic configuration of this example.
- the data of the estimated amplitude ⁇ m and the initial phase ⁇ of the flicker force component obtained from the DFT block 51 of the above-described flicker force reduction unit 25 are transmitted to the system controller 14.
- the data is then adjusted by the parameter control unit 14a in the system controller 14 as described later, and the data of the adjusted amplitude ym and the initial phase ⁇ ′ is Input to the block generation block 53.
- the flicker coefficient ⁇ n (y) expressed by (2) is calculated. That is, in this example, ⁇ ⁇ and ⁇ in Equation (2) in FIG. 17A are replaced with ⁇ ⁇ , and ⁇ '.
- FIG. 13 shows the normalized integral value calculation block 3 of the frits force reduction unit 25.
- the normalized integral value calculation block 30 may have the configuration shown in FIG. 6 or FIG.
- FIG. 14 shows a first specific example in this case.
- the data of the amplitude ⁇ m and the initial phase ⁇ mn as the input signals of the parameter control unit 14a actually have m systems per field, but are shown here as one system. The same applies to the data of the amplitude ⁇ m ′ and the initial phase ⁇ ′ which are output signals of the parameter control unit 14a.
- the data of the amplitude ⁇ m and the initial phase ⁇ mn from the DF ⁇ block 51 are supplied to the digital LPFs (low-pass filters) 61 and 62 respectively, and the output data of the digital LPF 61 is
- the output data of the gain adjustment circuit 63 is supplied to the flicker generation block 53 as data of the adjusted amplitude ⁇ ', and the digital LPF
- the output data of 62 is input to the flicker generation block 53 as data of the adjusted initial phase ⁇ m ⁇ ,.
- the time constant Ta of the digital LPF 61 and the time constant Tp of the digital LPF 62 are set by the time constant setting block 65.
- the gain (multiplication coefficient) Ka in the gain adjustment circuit 63 is set by the gain setting block 66.
- the commercial AC power source frequency is 5 0 Eta Zeta
- the vertical synchronizing frequency of the camera is 6 0 Eta zeta
- the initial phase ⁇ ⁇ ⁇ the same value every three fields, between the previous field Produces the difference (difference) expressed by equation (3) in Fig. 17 1.
- the digital LPF 62 it is necessary to configure one LPF for data having the same phase in consideration of the variation of the initial phase ⁇ . That is, when the period of the fluctuation of the initial phase ⁇ m ⁇ is 3 fields as in the above example, three LPFs are provided as the digital LPF 62, and the data of the initial phase ⁇ ⁇ ⁇ are Input to two LPFs.
- the data of the amplitude ⁇ m and the initial phase ⁇ ⁇ ⁇ and the ⁇ ⁇ (auto exposure) control information and the AWB (auto white balance) control information obtained in the system controller 14 are Input to status detection block 68.
- the control information is, specifically, information on the brightness of the screen, and the AWB control information is, specifically, information indicating whether or not the camera is under a color temperature or fluorescent lighting.
- the state detection block 68 determines whether or not the current shooting environment is under fluorescent light illumination, or whether or not the switch of the fluorescent light is turned on or off, from the input information as described below.
- Imaging that affects the occurrence of fluorescent lamp flicker such as whether the transition state is from non-fluorescent lighting to fluorescent lighting or from fluorescent lighting to non-fluorescent lighting. The situation is detected, and the control mode is determined according to the detection result.
- the determined control mode is presented to the time constant setting block 65 and the gain setting block 66 by the control mode presentation signal, and in response to this, the time constant setting block 65 digitally converts the PF 61 and 6 Set the time constants T a and T p of 2 and gain setting block 6 6 Set the gain Ka in circuit 63.
- FIG. 16 shows an example of the criterion for the above-described state detection in the state detection block 68.
- the flit force is constantly and stably generated under the fluorescent lamp illumination.
- the brightness of the screen fluctuates at a substantially constant cycle under fluorescent lighting, so it can be sufficiently judged from the brightness information of the A control that indicates that the screen is under fluorescent lighting. .
- the light source is estimated from the detected color information, and it is determined whether or not the light source is a fluorescent lamp. You can also determine that there is.
- the accuracy of detection is increased by comprehensively judging information beyond the past multiple fields.
- the state detection block 68 sets the control mode to the mode A described later when it is determined that the flicker is constantly and stably generated under the fluorescent lamp illumination. .
- the estimated amplitude ⁇ m of the frit force component fluctuates randomly near zero due to only the noise component, and The phase ⁇ fluctuates randomly due to noise.
- the fritting force reduction processing is not required under the non-fluorescent lamp illumination.
- there is no periodicity in the fluctuation of the screen brightness Therefore, it can be sufficiently determined from the AE control brightness information that indicates that the screen is under non-fluorescent lighting. it can. It is also possible to determine that the subject is under non-fluorescent lighting from the light source estimation information of the AWB control described above.
- the accuracy of detection is increased by comprehensively judging information beyond the past multiple fields.
- the control mode is set to the control mode. Is set to mode B described later.
- the time constant setting block 65 and the gain setting block 66 specifically determine the time constants Ta and Tp of the digital LPFs 61 and 62 and the gain. What value should be set as the gain Ka in the adjustment circuit 63 is determined as follows according to the system configuration and the requirements for the system.
- the time constant Ta of the digital LPF 61 As described above, although the estimated amplitude ⁇ m of the flicker component varies depending on whether it is near zero or not, the mode A (fluorescence In the state where the frit force is constantly and stably generated under the lighting of the lamp, or in the mode B (the state where the frit force is not generated constantly under the non-fluorescent lighting), the value is almost constant. Become. The value does not become almost constant when there is a disturbance.
- the time constant T a is set to be short, with emphasis on the latter followability.
- the time constant T a becomes longer, and when transitioning from mode A to mode B or from mode B to mode A, the time constant T a becomes shorter.
- the time constant Ta can be dynamically controlled.
- mode ⁇ (a state in which no fritting force is constantly generated under non-fluorescent lighting)
- the initial phase ⁇ ⁇ ⁇ continues to take a random value, so the time constant T p is set longer.
- the time constant ⁇ ⁇ can be set to an arbitrary value due to the effect of gain adjustment described later.
- the time constant Ta or ⁇ ⁇ may be switched between mode ⁇ ⁇ and mode ⁇ .
- the gain Ka in the gain adjustment circuit 63 in mode A (the state in which the frit force is constantly and stably generated under the fluorescent light) is represented by the amplitude ⁇ as shown in FIG. Since m is almost constant, the gain Ka should basically be set to 1.
- K a the capture rate of the amplitude ⁇ m can be directly controlled.
- the gain Ka is not limited to 1, but can be set to a value larger than 1 or a value smaller than 1.
- a system may be configured.
- mode B a state in which no fritting force is constantly generated under non-fluorescent lighting
- the amplitude ⁇ m becomes a random value near zero due to noise.
- the gain Ka is set to zero so that unnecessary processing is not performed.
- Mode A or Mode B the state in which the frit force is generated constantly or not.
- the frit component is extracted by an average operation or a difference operation between a plurality of fields.
- a part of each signal sequence used for the averaging operation or the difference operation includes a flicker component, and the other part does not include a flicker component.
- the gain setting block 66 detects the transition state of the control mode and controls the value of the gain Ka according to the transition state.
- the reliability of the amplitude ⁇ m and the initial phase ⁇ is sufficiently high because the reliability of the amplitude ym and the initial phase ⁇ mn is still low at the start of the transition.
- the gain Ka is switched from zero to 1 and the flicker force reduction processing in the flicker generation block 53 and the calculation block 40 is executed, or the gain Ka is gradually reduced.
- the flicker generation block 53 and the calculation block 40 are used to smoothly execute flicker reduction processing.
- FIG. 15 shows a second specific example.
- This example is obtained by adding storage units 71 to 74, switches 75 to 78, and a state detection block 69 to the example shown in FIG.
- the storage unit 71 stores data of the amplitude ⁇ m
- the storage unit 72 stores data of the initial phase ⁇
- the storage unit 73 stores the output data of the gain adjustment circuit 63.
- the storage unit 74 stores the output data of the digital LPF 62
- the switches 75 to 78 store the storage units 71 to 7 in accordance with the detection results of the state detection block 69, respectively. 4 input and output data
- the output data of switch 75 is supplied to digital LPF 61
- the output data of switch 76 is supplied to digital PF 62
- the output data of switch 77 is selected.
- the output data of the switch 78 is input to the flicker generation block 53 as the data of the initial phase ⁇ ′. .
- Zooming information and camera shake information are input to the state detection block 69.
- the state detection block 69 determines from the zooming information whether a large angle of view change has occurred due to zooming, and from the camera shake information, a large angle of view due to panning / tilting or large-amplitude camera shake. It is determined whether or not a change has occurred.
- the state detection block 69 switches the switches 75 to 78 to the sides other than the storage units 71 to 74, respectively. That is, normally, similarly to the example of FIG. 14, the fritting force reduction processing is executed.
- the state detection block 69 switches the switches 75 to 78 to the storage units 71 to 74, respectively.
- the reliability of the amplitude ⁇ ⁇ and the initial phase ⁇ ⁇ ⁇ decreases, and the amplitudes obtained in the past and stored in the storage units 73 and 74 are reduced.
- the data and the initial phase data are input to the fritting force generation block 53 as the amplitude ⁇ m and the initial phase ⁇ ′.
- switches 777 and 788 are then switched to non-storage units 73 and 74. Immediately after this, the amplitude ⁇ m and the initial phase ⁇ ⁇ , contain errors.
- the state detection block 69 not only switches the switches 77 and 78 to the storage units 73 and 74 but also switches the switches 5 and 76 are switched to the storage units 7 1 and 7 2, and the unreliable data at that time is stored in the storage units 7 1 and 7 2 without being input to the digital LPFs 6 1 and 6 2.
- the control is performed so that highly reliable data before large changes in the angle of view are digitized and input to the PFs 61 and 62.
- the reliability of the amplitude y m and the initial phase ⁇ ⁇ n is determined separately not only for the zooming information and the camera shake information, and the reliability level information of the determination result is used as input information of the state detection block 69, If the reliability level information indicates that the reliability of the amplitude ⁇ m and the initial phase ⁇ ⁇ ⁇ is low, the switches 75 to 78 are switched to the storage units 71 to 74 as described above. Therefore, it is also possible to configure so that past reliable data is used.
- the input image signal I n ′ (x, y) is integrated over one line, but the integration of the input image signal In, (x, y) is This is to obtain the sampling value of the flicker component with less influence, so that the sampling may be performed not only for one line but also for a plurality of lines.
- the input image signal In, (x, y ) can be integrated over a time that is several times or more than 10 times the horizontal period. Also, the integration time does not have to be an integral multiple of the horizontal period, such as 2.5 horizontal periods.
- the load of the DFT operation in the DFT block 51 can be reduced, and when the subject moves in the vertical direction of the screen. In addition, the effect can be reduced.
- the flicker reduction components 25R, 25G, and 25B detect and reduce the frit component for each primary color signal of each RGB color as shown in Fig. 2.
- a flicker reducing unit 25 as in the above example is provided on the output side of the luminance signal Y of the composite matrix circuit 29 to detect a flicker component in the luminance signal Y. , May be configured to reduce.
- the digital signal processing unit 20 including the flits force reduction unit 25 is configured by hardware.
- the flicker reduction unit 25 or the digital signal processing unit 20 is not required. Some or all of them may be configured by software.
- the vertical synchronization frequency is 60 Hz (one field cycle is 1 to 60 seconds).
- the present invention is, for example, a progressive camera such as a digital camera, It can also be applied when the vertical synchronization frequency is 30 Hz (one frame period is 1/30 second).
- the three-frame period (1/10 seconds) is an integral multiple of the fluorescent light emission period (1Z100 seconds). (The fringe fringe pattern is equivalent to 10 wavelengths in three frames.) Therefore, the fields in the above-described embodiment may be replaced with frames.
- the present invention can also be applied to a case where an XY address scanning type imaging element other than the CMOS imaging element is used.
- the fluorescent lamp flicker unique to an XY address scanning type image sensor such as a CMOS image sensor can be removed by subjecting the subject to simple signal processing without using a light receiving element or the like. Regardless of the signal level, video signal level, and type of fluorescent lamp, it can be detected with high accuracy and can be reliably and sufficiently reduced.
- the method for reducing fringe power of the present invention when used for the primary color signals of RGB colors, it is necessary to detect not only bright and dark fringe powers but also color fringe powers with high accuracy and to reduce them reliably and sufficiently. Can be.
- the reverse calculation is performed by performing the calculation of the flicker reduction. It is possible to prevent the signal component from being changed.
- the flicker reduction calculation is performed. This can prevent the image quality from being affected.
- the influence of disturbance is reduced in a steady state under fluorescent lamp illumination or non-fluorescent lamp illumination. It is not easy to receive, but it also has good responsiveness at the time of transition. ⁇ It is possible to realize a process with good follow-up, and also at the time of state transition, angle of view change, or when the reliability of flicker detection parameter is low. Appropriate processing can be performed smoothly and without discomfort.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Color Television Image Signal Generators (AREA)
- Image Processing (AREA)
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/535,114 US7656436B2 (en) | 2002-11-18 | 2003-10-16 | Flicker reduction method, image pickup device, and flicker reduction circuit |
EP03756640A EP1566962A4 (en) | 2002-11-18 | 2003-10-16 | FLAG REDUCTION PROCEDURE, IMAGING DEVICE AND FLAG REDUCTION SWITCHING |
AU2003302113A AU2003302113A1 (en) | 2002-11-18 | 2003-10-16 | Flicker reduction method, image pickup device, and flicker reduction circuit |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2002333158 | 2002-11-18 | ||
JP2002-333158 | 2002-11-18 | ||
JP2003173642A JP4423889B2 (ja) | 2002-11-18 | 2003-06-18 | フリッカ低減方法、撮像装置およびフリッカ低減回路 |
JP2003-173642 | 2003-06-18 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2004047427A1 true WO2004047427A1 (ja) | 2004-06-03 |
Family
ID=32328307
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2003/013237 WO2004047427A1 (ja) | 2002-11-18 | 2003-10-16 | フリッカ低減方法、撮像装置およびフリッカ低減回路 |
Country Status (6)
Country | Link |
---|---|
US (1) | US7656436B2 (ja) |
EP (1) | EP1566962A4 (ja) |
JP (1) | JP4423889B2 (ja) |
KR (1) | KR101040842B1 (ja) |
AU (1) | AU2003302113A1 (ja) |
WO (1) | WO2004047427A1 (ja) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1725023A2 (en) * | 2005-05-16 | 2006-11-22 | Sony Corporation | Image process apparatus and image pickup apparatus |
EP1814310A1 (en) * | 2004-11-15 | 2007-08-01 | Sony Corporation | Flicker correcting method, flicker correcting circuit, and imaging device using them |
CN100477744C (zh) * | 2005-04-19 | 2009-04-08 | 索尼株式会社 | 闪烁校正方法与设备以及成像设备 |
US7538799B2 (en) | 2005-01-14 | 2009-05-26 | Freescale Semiconductor, Inc. | System and method for flicker detection in digital imaging |
CN1882047B (zh) * | 2005-06-13 | 2011-06-15 | 索尼株式会社 | 图像处理设备和图像拾取设备 |
CN105430289A (zh) * | 2015-11-19 | 2016-03-23 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | 一种基于cmos图像传感器检测led闪烁频率的方法 |
Families Citing this family (54)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4259998B2 (ja) * | 2003-12-19 | 2009-04-30 | 三洋電機株式会社 | フリッカ検出装置及び撮像装置 |
JP4796755B2 (ja) * | 2004-07-01 | 2011-10-19 | Hoya株式会社 | 像ブレ補正装置 |
US7683948B2 (en) * | 2005-03-31 | 2010-03-23 | Freescale Semiconductor, Inc. | System and method for bad pixel replacement in image processing |
JP4207923B2 (ja) | 2005-04-19 | 2009-01-14 | ソニー株式会社 | フリッカ補正方法、フリッカ補正装置及び撮像装置 |
JP4207926B2 (ja) | 2005-05-13 | 2009-01-14 | ソニー株式会社 | フリッカ補正方法、フリッカ補正装置及び撮像装置 |
JP4539449B2 (ja) * | 2005-06-10 | 2010-09-08 | ソニー株式会社 | 画像処理装置および撮像装置 |
JP4453648B2 (ja) * | 2005-06-13 | 2010-04-21 | ソニー株式会社 | 画像処理装置および撮像装置 |
JP4483744B2 (ja) | 2005-08-26 | 2010-06-16 | ソニー株式会社 | 撮像装置及び撮像制御方法 |
JP4631611B2 (ja) | 2005-08-30 | 2011-02-16 | ソニー株式会社 | フリッカ検出装置とフリッカ除去装置と撮像装置およびフリッカ検出方法 |
JP4904749B2 (ja) * | 2005-09-08 | 2012-03-28 | ソニー株式会社 | フリッカ低減方法、フリッカ低減回路及び撮像装置 |
KR100721664B1 (ko) | 2005-12-22 | 2007-05-23 | 매그나칩 반도체 유한회사 | 이미지 센서의 플리커 검출 회로 및 방법 |
JP4371108B2 (ja) | 2005-12-27 | 2009-11-25 | ソニー株式会社 | 撮像装置および方法、記録媒体、並びにプログラム |
US8068148B2 (en) * | 2006-01-05 | 2011-11-29 | Qualcomm Incorporated | Automatic flicker correction in an image capture device |
US7593040B2 (en) | 2006-01-30 | 2009-09-22 | Omnivision Technologies, Inc. | Image anti-shake in digital cameras |
JP5175441B2 (ja) * | 2006-03-03 | 2013-04-03 | 日本電気株式会社 | 撮影装置及び方法並びにプログラム |
JP4840578B2 (ja) * | 2006-06-29 | 2011-12-21 | 京セラ株式会社 | 撮像装置のフリッカ検出方法と装置 |
US20100002149A1 (en) * | 2006-11-14 | 2010-01-07 | Koninklijke Philips Electronics N.V. | Method and apparatus for detecting slow motion |
KR100891434B1 (ko) * | 2007-05-16 | 2009-04-03 | 주식회사 코아로직 | 플리커 노이즈 검출 장치, 방법 및 그 기록매체 |
JP4971956B2 (ja) * | 2007-11-27 | 2012-07-11 | キヤノン株式会社 | フリッカ補正装置、フリッカ補正方法並びに撮像装置 |
JP5111100B2 (ja) * | 2007-12-28 | 2012-12-26 | キヤノン株式会社 | 画像処理装置、画像処理方法、プログラム及び記憶媒体 |
US8144213B2 (en) | 2008-02-04 | 2012-03-27 | Panasonic Corporation | Imaging device, integrated circuit, and imaging method for encoding image frames based on detected flicker of a light source |
JP5035025B2 (ja) * | 2008-03-03 | 2012-09-26 | ソニー株式会社 | 画像処理装置、フリッカ低減方法、撮像装置及びフリッカ低減プログラム |
JP2010021645A (ja) * | 2008-07-08 | 2010-01-28 | Nikon Corp | 画像処理装置、撮像装置、および画像処理プログラム |
JP5106283B2 (ja) * | 2008-07-15 | 2012-12-26 | 三洋電機株式会社 | ビデオカメラ |
US7911505B2 (en) * | 2008-08-20 | 2011-03-22 | Eastman Kodak Company | Detecting illuminant flicker |
JP4626689B2 (ja) * | 2008-08-26 | 2011-02-09 | ソニー株式会社 | 撮像装置、補正回路および補正方法 |
WO2010058567A1 (ja) * | 2008-11-20 | 2010-05-27 | パナソニック株式会社 | フリッカ低減装置、集積回路及びフリッカ低減方法 |
JP5218239B2 (ja) * | 2009-04-14 | 2013-06-26 | ソニー株式会社 | 画像データ処理装置および撮像装置 |
EP2262228A1 (en) | 2009-06-09 | 2010-12-15 | Global IP Solutions (GIPS) AB | Detection and supression of flicker in a sequence of images |
JP2011030093A (ja) * | 2009-07-28 | 2011-02-10 | Sanyo Electric Co Ltd | ビデオカメラ |
JP4724767B2 (ja) * | 2009-08-19 | 2011-07-13 | アキュートロジック株式会社 | 撮像装置のフリッカ検出方法及びフリッカ検出装置、フリッカ検出プログラム |
JP2011091775A (ja) | 2009-10-26 | 2011-05-06 | Toshiba Corp | 固体撮像装置 |
JP5523124B2 (ja) * | 2010-01-26 | 2014-06-18 | キヤノン株式会社 | 撮像装置 |
JP5625371B2 (ja) * | 2010-01-29 | 2014-11-19 | ソニー株式会社 | 画像処理装置、および信号処理方法、並びにプログラム |
JP2011163947A (ja) * | 2010-02-10 | 2011-08-25 | Seiko Epson Corp | 光特性測定方法およびその装置 |
JP5682196B2 (ja) * | 2010-09-22 | 2015-03-11 | セイコーエプソン株式会社 | 画像補正回路、撮影装置および画像補正プログラム |
JP2012109794A (ja) | 2010-11-17 | 2012-06-07 | Toshiba Corp | 固体撮像装置 |
US9418270B2 (en) | 2011-01-31 | 2016-08-16 | Hand Held Products, Inc. | Terminal with flicker-corrected aimer and alternating illumination |
US8542292B2 (en) | 2011-02-04 | 2013-09-24 | Panasonic Corporation | Flicker correction device, flicker correction method and flicker correction program |
JP5296112B2 (ja) * | 2011-02-04 | 2013-09-25 | パナソニック株式会社 | フリッカ補正装置、フリッカ補正方法、およびフリッカ補正プログラム |
US9088727B2 (en) | 2011-04-06 | 2015-07-21 | Pelco, Inc. | Spatially-varying flicker detection |
JPWO2012147337A1 (ja) * | 2011-04-28 | 2014-07-28 | オリンパス株式会社 | フリッカ検出装置、フリッカ検出方法およびフリッカ検出プログラム |
TWI469632B (zh) * | 2011-11-09 | 2015-01-11 | Silicon Motion Inc | 抗閃爍攝錄裝置與攝錄方法 |
US8736711B2 (en) * | 2012-03-19 | 2014-05-27 | Htc Corporation | Camera with anti-flicker capability and image capturing method with anti-flicker capability |
WO2014167865A1 (ja) * | 2013-04-12 | 2014-10-16 | パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ | フリッカ低減装置、撮像装置およびフリッカ低減方法 |
US9344640B2 (en) | 2013-06-14 | 2016-05-17 | Panasonic Intellectual Property Corporation Of America | Imaging device, integrated circuit, and flicker reduction method |
JP6060824B2 (ja) | 2013-06-20 | 2017-01-18 | 株式会社Jvcケンウッド | 撮像装置、及びフリッカ低減方法 |
JP6525525B2 (ja) | 2014-07-31 | 2019-06-05 | キヤノン株式会社 | 撮像装置及びその制御方法、プログラム、記憶媒体 |
JP2016034094A (ja) | 2014-07-31 | 2016-03-10 | ソニー株式会社 | 画像処理装置、画像処理方法、プログラム、およびイメージセンサ |
WO2016031402A1 (ja) * | 2014-08-27 | 2016-03-03 | ソニー株式会社 | 撮像装置および撮像装置の制御方法 |
JP6300118B2 (ja) * | 2015-09-24 | 2018-03-28 | カシオ計算機株式会社 | 撮像装置、フリッカ検出方法及びプログラム |
KR102523301B1 (ko) * | 2015-12-15 | 2023-04-19 | 삼성전자주식회사 | 플리커를 제거하는 이미지 센서 회로 및 이를 포함하는 카메라 장치 |
JP2018093275A (ja) * | 2016-11-30 | 2018-06-14 | オリンパス株式会社 | 撮像装置およびフリッカ判定方法 |
KR20220048090A (ko) | 2020-10-12 | 2022-04-19 | 삼성전자주식회사 | 주파수 도메인을 이용한 이미지 센서의 검사 방법 및 이를 수행하는 검사 시스템 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11122513A (ja) * | 1997-10-09 | 1999-04-30 | Matsushita Electric Ind Co Ltd | 蛍光灯フリッカー補正装置及びそれを使用したカメラ |
JPH11252446A (ja) * | 1998-03-03 | 1999-09-17 | Matsushita Electric Ind Co Ltd | フリッカ補正装置 |
EP1091571A2 (en) | 1999-10-08 | 2001-04-11 | Matsushita Electric Industrial Co., Ltd. | Apparatus and methods for detection and compensation of illumination flicker, detection and measurement of AC line frequency |
JP2001119708A (ja) * | 1999-10-19 | 2001-04-27 | Matsushita Electric Ind Co Ltd | フリッカ検出・補正装置およびフリッカ検出・補正方法 |
JP2003198932A (ja) * | 2001-12-27 | 2003-07-11 | Sharp Corp | フリッカ補正装置およびフリッカ補正方法、並びにフリッカ補正プログラムを記録した記録媒体 |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6195127B1 (en) * | 1996-07-18 | 2001-02-27 | Sanyo Electric Co., Ltd. | Digital camera, having a flash unit, which determines proper flash duration through an assessment of image luminance and, where needed, a preliminary flash emission |
US6850278B1 (en) * | 1998-11-27 | 2005-02-01 | Canon Kabushiki Kaisha | Solid-state image pickup apparatus |
US7298401B2 (en) * | 2001-08-10 | 2007-11-20 | Micron Technology, Inc. | Method and apparatus for removing flicker from images |
-
2003
- 2003-06-18 JP JP2003173642A patent/JP4423889B2/ja not_active Expired - Fee Related
- 2003-10-16 EP EP03756640A patent/EP1566962A4/en not_active Withdrawn
- 2003-10-16 KR KR1020057008810A patent/KR101040842B1/ko not_active IP Right Cessation
- 2003-10-16 AU AU2003302113A patent/AU2003302113A1/en not_active Abandoned
- 2003-10-16 US US10/535,114 patent/US7656436B2/en active Active
- 2003-10-16 WO PCT/JP2003/013237 patent/WO2004047427A1/ja active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11122513A (ja) * | 1997-10-09 | 1999-04-30 | Matsushita Electric Ind Co Ltd | 蛍光灯フリッカー補正装置及びそれを使用したカメラ |
JPH11252446A (ja) * | 1998-03-03 | 1999-09-17 | Matsushita Electric Ind Co Ltd | フリッカ補正装置 |
EP1091571A2 (en) | 1999-10-08 | 2001-04-11 | Matsushita Electric Industrial Co., Ltd. | Apparatus and methods for detection and compensation of illumination flicker, detection and measurement of AC line frequency |
JP2001119708A (ja) * | 1999-10-19 | 2001-04-27 | Matsushita Electric Ind Co Ltd | フリッカ検出・補正装置およびフリッカ検出・補正方法 |
JP2003198932A (ja) * | 2001-12-27 | 2003-07-11 | Sharp Corp | フリッカ補正装置およびフリッカ補正方法、並びにフリッカ補正プログラムを記録した記録媒体 |
Non-Patent Citations (1)
Title |
---|
See also references of EP1566962A4 |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1814310A1 (en) * | 2004-11-15 | 2007-08-01 | Sony Corporation | Flicker correcting method, flicker correcting circuit, and imaging device using them |
CN100484197C (zh) * | 2004-11-15 | 2009-04-29 | 索尼株式会社 | 闪烁校正方法、闪烁校正电路、以及使用它们的摄像装置 |
EP1814310A4 (en) * | 2004-11-15 | 2010-09-08 | Sony Corp | FLACKER CORRECTION, FLACKER CORRECTION AND IMAGING DEVICE |
US8040392B2 (en) | 2004-11-15 | 2011-10-18 | Sony Corporation | Flicker correcting method, flicker correcting circuit, and imaging device using them |
US7538799B2 (en) | 2005-01-14 | 2009-05-26 | Freescale Semiconductor, Inc. | System and method for flicker detection in digital imaging |
CN100477744C (zh) * | 2005-04-19 | 2009-04-08 | 索尼株式会社 | 闪烁校正方法与设备以及成像设备 |
EP1725023A2 (en) * | 2005-05-16 | 2006-11-22 | Sony Corporation | Image process apparatus and image pickup apparatus |
EP1725023A3 (en) * | 2005-05-16 | 2007-08-15 | Sony Corporation | Image process apparatus and image pickup apparatus |
US8416337B2 (en) | 2005-05-16 | 2013-04-09 | Sony Corporation | Image process apparatus and method for processing a color image signal |
CN1882047B (zh) * | 2005-06-13 | 2011-06-15 | 索尼株式会社 | 图像处理设备和图像拾取设备 |
CN105430289A (zh) * | 2015-11-19 | 2016-03-23 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | 一种基于cmos图像传感器检测led闪烁频率的方法 |
Also Published As
Publication number | Publication date |
---|---|
JP2004222228A (ja) | 2004-08-05 |
KR20050075425A (ko) | 2005-07-20 |
EP1566962A4 (en) | 2009-09-16 |
US20060055823A1 (en) | 2006-03-16 |
EP1566962A1 (en) | 2005-08-24 |
KR101040842B1 (ko) | 2011-06-14 |
AU2003302113A1 (en) | 2004-06-15 |
US7656436B2 (en) | 2010-02-02 |
JP4423889B2 (ja) | 2010-03-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2004047427A1 (ja) | フリッカ低減方法、撮像装置およびフリッカ低減回路 | |
JP4483744B2 (ja) | 撮像装置及び撮像制御方法 | |
JP5035025B2 (ja) | 画像処理装置、フリッカ低減方法、撮像装置及びフリッカ低減プログラム | |
JP3826904B2 (ja) | 撮像装置およびフリッカ低減方法 | |
JP4904749B2 (ja) | フリッカ低減方法、フリッカ低減回路及び撮像装置 | |
JP4106554B2 (ja) | 撮影環境判定方法および撮像装置 | |
WO2005120047A1 (ja) | 撮像装置および信号処理方法 | |
WO2017090300A1 (ja) | 画像処理装置、および画像処理方法、ならびにプログラム | |
US20140063294A1 (en) | Image processing device, image processing method, and solid-state imaging device | |
KR20060128649A (ko) | 화상 처리 장치 및 촬상 장치 | |
US10412356B2 (en) | Image processing apparatus and image processing method | |
KR20060129954A (ko) | 화상 처리 장치 및 촬상 장치 | |
EP1725023B1 (en) | Image process apparatus and image pickup apparatus | |
JP2012151566A (ja) | 撮像装置 | |
JP2007158964A (ja) | 画像処理装置および撮像装置 | |
JP2004179839A (ja) | 固体撮像装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
REEP | Request for entry into the european phase |
Ref document number: 2003756640 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2003756640 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2006055823 Country of ref document: US Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 10535114 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020057008810 Country of ref document: KR |
|
WWE | Wipo information: entry into national phase |
Ref document number: 20038A3546X Country of ref document: CN |
|
WWP | Wipo information: published in national office |
Ref document number: 1020057008810 Country of ref document: KR |
|
WWP | Wipo information: published in national office |
Ref document number: 2003756640 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 10535114 Country of ref document: US |