CA2240343C - Image signal processor having an enlarged correction range for picture blurring caused by movement of the hands - Google Patents
Image signal processor having an enlarged correction range for picture blurring caused by movement of the hands Download PDFInfo
- Publication number
- CA2240343C CA2240343C CA 2240343 CA2240343A CA2240343C CA 2240343 C CA2240343 C CA 2240343C CA 2240343 CA2240343 CA 2240343 CA 2240343 A CA2240343 A CA 2240343A CA 2240343 C CA2240343 C CA 2240343C
- Authority
- CA
- Canada
- Prior art keywords
- size
- picture blurring
- image signal
- pixels
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Studio Devices (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Studio Circuits (AREA)
Abstract
An image signal processor for use in an imaging system. Each field of an input image signal is stored in a memory. If the picture blurring is not within a predetermined range, a zoom controller disables an optical zoom-in operation and determines the size of a reduced-size readout frame such that the picture blurring is just fit in a predetermined range. On receiving the size, an address genetator generates addresses to cause the pixels to be read out of the reduced-size readout frame, in the memory, which is located at a position shifted from the center of the field by the PV vector and which has the reduced size. In response to receiving the size of the reduced-size readout frame, the interpolator interpolates the read out pixels such that the read out pixels of the reduced-size are magnified up to the predetermined output size.
Description
CS- q8~ ~ -IMAGE SIGNAL PROCESSOR HAVING AN ENLARGED CORRECTION RANGE
FOR PICTURE BLURRING CAUSED BY MOVEMENT OF THE HANDS
BACKGROUND OF THE INVENTION
l . Field of the Invention The invention relates to a video camera and more particularly to an image signal processor, for use in a video camera, for preventing blurring of resultant television pictures caused by unintentional movements of the hands or the camera during optical zooming in. The image signal processor is capable of correcting picture blurring of an enhanced magnitude caused by a hand movement during an optical zoom-in operation.
10 2. Description of the Prior Art Unintentional movements of the hands or the camera (hereinafter referred to as "hand movements") in shooting result in blurring of resultant television pictures (hereinafter referred to as "picture blurring"). It is especially true when the user is zooming in because the higher the zoom power becomes, the more the picture blurring 15 increases in magnitude. For this, the present invention contemplates the picture blurring caused by hand movements in zoom-in shooting.
Various systems which deal with problems of such picture blurring have been proposed so far.
A basic scheme to correct the picture blurring is to store a field of shot and 20 A/D-converted image in a field memory, to read out, from the field memory, pixel data in a reduced-size frame at a position moved from the last frame position in the direction of and by the distance of the picture blurring, and to interpolate the read-out pixel data to restore the reduced-size frame to an original size. Doing this enables the correction of the picture blurring as long as both of horizontal and vertical components of the magnitude of 25 the picture blurring are smaller than halves of horizontal and vertical differences, respectively, between the original size and the reduced-size frame (the latter quantities are referred to as "allowable dimensions of picture blurring" or "allowable picture blurring dimensions"). Thus, the basic correction scheme is only available to picture blurring within the allowable picture blurring dimensions, which we call an allowable dimension problem.
It is noted that the allowable picture blurring dimensions comprise horizontal 5 and vertical components of the allowable magnitude of picture blurring, which is a function of not only a hand movement but also the zoom m~gnification or power as described above.
One of the solutions of the allowable dimension problem is to enlarge the allowable picture blurring dimensions by simply increasing the field memory in size or 10 capacity. In this solution, the added memory will not be used in ordinary shooting conditions or very low zoom power, becoming less efficient.
Another solution of the allowable dimension problem is disclosed in Japanese unexamined patent publication No. HeiS- l 30,48 l (l 993) titled "IMAGE PICKUP
DEVICE." In the image pickup device, if the zoom power is not less than the maximum 15 zoom power which is calculated from a rocking angle of the image pickup device caused by a hand movement and the capacity of the field memory, the zoom-in operation is disabled so that the resultant picture blurring remains within the allowable picture blurring dimensions. Thus, since the zoom power is limited by the magnitude of picture blurring, the zoom power may not be raised to a satisfactory extent due to a hand 20 movement.
Also, U.S. Patent 5,502,484 issued March 26, l 996 discloses a video camera and a video signal reproducing apparatus which has a CCD (charge coupled device) and an image memory for PAL (Phase Alternation Line) to provide a hand movement-corrected output image signal for NTSC (National Television System Committee). A
25 hand movement correction in the vertical direction is mainly achieved by shifting a reading out position (the readout starting line) of the CCD, and a hand movement correction in the horizontal direction is mainly achieved by horizontally shifting a fixed-size cut-out range by a correction amount associated by coefficients of a low pass filter with the way angle of the video camera. Though "by ch~nging the coefficient of the low pass filter according to the m~gnification, a wide correction range can be obtained", it is difficult and makes the system complicated to associate the electronic zoom 5 m~gnification with the coefficient such that the image memory is fully utilized. This is not only because the association has to be achieved experimentally and needs additional table but also because, as for angular velocity sensors used for the detection of horizontal and vertical angles, the characteristic may very from sensor to sensor.
It is therefore an object of the invention to provide an image signal processor 10 and an im~ging system which is capable of correcting picture blurring of an enhanced magnitude caused by a hand movement during an optical zoom-in operation by switching into an electronic or digital zoom-in operation when the magnitude of picture blurring exceeds the allowable picture blurring dimensions (or the allowable magnitude of picture blurring).
SUMMARY OF THE INVENTION
The foregoing object is achieved by an im~ging system such as a video camera which comprises an optical zoom arrangement, an im~ging device such as an M X N-pixel CCD for providing an image signal, an analog-to-digital converter for converting the image signal into a digital image signal, and a picture blurring (PV) compensation &
20 digital zoom unit or an image signal processor.
In the image signal processor, a field of pixels in the digital image signal is stored in a field memory for M X N pixels. For each field, a test is made to see if the picture blurring is within a predetermined range, i.e., if a readout frame is included in a correctable range (or in the M XN pixels). If so, a conventional PV compensation is 25 performed by an address generator reading out pixel data in a readout frame which has a predetermined size of Mr XNr pixels (Mr<M, Nr<N) and which is at a position shifted from the center of the field by a PV vector and an interpolator interpolating the read out pixel data into a predetermined output size of Mo X No pixels.
If the picture blurring is not within a predetermined range, a zoom controller disables the optical zoom-in operation and determines the size (Mr' XNr') of a reduced-6 size readout frame such that the picture blurring is just fit in the predetermined range. In response to receiving the size of the reduced-size readout frame, the address generator reads out pixels in the reduced-size readout frame, in the field memory, which is located at a position shifted from the center of the field by the PV vector and which has the reduced size. In response to receiving the size of the reduced-size readout frame, the 10 interpolator interpolates the read out pixels such that the read out pixels of the reduced-size are m~gnified up to the predetermined output size.
BRIEF DESCRIPTION OF THE DRAWING
Further objects and advantages of the present invention will be apparent from the following description of the preferred embodiment of the invention as illustrated in 15 the accompanying drawing, in which:
FIG. l is a schematic block diagram showing a part of an image pickup device such as a video camera which incorporates an image signal processor according to an illustrative embodiment of the invention;
FIG. 2 is a diagram of the field memory 62;
FIG. 3 is a flowchart showing an exemplary operation of the zoom controller 66;
FIG. 4 is a diagram illustrating a situation where the picture blurring is within the correctable range;
FIG. 5 is a diagram illustrating how blurring of a picture within the correctable 25 region is corrected;
FIG. 6 is a diagram illustrating a situation where the picture blurring has exceeded the correctable region in the direction BD running vertically between the two diagonals Dl and D2 of the field memory 62; and FIG. 7 is a flowchart showing the operation of the readout frame size calculator68 when the calculator 68 has received either DH or Dv.
Throughout the drawing, the same elements when shown in more than one figure are designated by the same reference numerals.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
FIG. l is a schematic block diagram showing a part of an image pickup device l such as a video camera which incorporates an illustrative embodiment of an image 10 signal processor 60 for effectuating picture blurring compensation and/or digital zoom in accordance with the invention. The image pickup device l comprises an optical system l0 including a lens group (not shown) for use in a zoom operation, an im~ging device 20 comprising a CCD (charge coupled device) or a set of Cads, an analog to digital (A/D) converter 30 the input of which is coupled to the im~ging device 20. The image pickup 15 device l further comprises a zoom control switch 40 which permits the user to select one of three positions, i.e., a zoom-in position (denoted as IN) to raise the zoom power, a zoom-out position (denoted as OUT) to lower the zoom power and an OFF position to maintain the current zoom power. The image pickup device 1 still further comprises a optical zoom section which comprises the above-mentioned lens group (not shown) 20 included in the optical system l 0, an actuator 52 such as a motor for mechanically adjust a lens position to increase and decrease the zoom power, and an actuator driver 54 for electrically driving the actuator 52. Also, the image pickup device 1 comprises the image signal processor 60.
The image signal processor 60 comprises a field memory 62 for storing a field 25 of picked-up and A/D converted image signal from the A/D converter 30; a picture blurring (PV) vector generator 64 having an input thereof coupled to the A/D converter 30 for providing a picture blurring vector (or PV vector) for each field; a zoom controller 66 for receiving the PV vector and a zoom control switch 40 output and for providing an optical zoom control signal to the actuator driver 54 and providing a correction amount calculated from the PV vector and horizontal and vertical correction margins Mh and Mv as detailed later; a readout frame size calculator 68 for receiving the correction amount from the zoom controller 66 and for providing a readout frame size as described in detail later; an address generator 70 for receiving the PV vector and the readout frame size and for providing addresses to the field memory 62; and an interpolator 72 which is coupled to the field memory 62, which receives the readout frame size and which provides an 10 picture blurring compensated image signal as an output of the image signal processor 60.
Referring now to FIG. l, the operation of the image pickup device l will be described in the following. If the user is shooting an object, then an image of the object is formed by the optical system l 0 on the surface of the im~ging device 20, which provides a picked-up image signal. Assuming that the im~ging device comprises an array of M XN
15 im~ging elements, each field of the picked-up image signal comprises M XN pixels, i.e., N horizontal lines each comprising M pixels (M and N are any natural numbers). The picked-up image signal is converted by the A/D converter 30 into a digital image signal, which is supplied to the image signal processor or a PV compensation and digital zoom unit 60.
In the image signal processor 60, each of the fields in the digital image signal is stored in the field memory 62. The size of the field memory 62 is equal to that of the im~ging element array of the imaging device 20, i.e., M X N. Increasing the im~ging element array size and the memory 62 size results in an increase in the allowable picture blurring dimensions (or magnitude) and accordingly results in an increase in an allowable magnitude of hand movements and/or in a allowable zoom m~gnification.
In order to enable the picture blurring to be corrected when the picture blurring is within the allowable magnitude (or dimensions), i.e., when hand movements and/or the zoom power is relatively small, the designer of the image signal processor 60 first determines the size, in pixels, of a readout (or cut-out) frame or window (hereinafter referred to as "the readout frame size"), Mr X Nr (Mr<M, Nr<N). FIG. 2 is a diagram of the field memory 62. In FIG. 2, it is assumed that the readout frame 621 is located at the 5 center of the M-by-N pixel block or the field memory 62. The rest 623 of the field memory (referred to as a correction margin) is used for the picture blurring correction.
Letting the horizontal and the vertical correction margins be 2 X Mh and 2 X Mv, then we obtain 2XMh = M - Mr, and 2XMv=N-Nr.
The number No of horizontal SC~nning lines and the number Mo of pixels of each sc~nning line in the output image signal from the image signal processor 60, i.e., the interpolator 72 are preferably determined according to one of standard television systems.
Thus, assuming that the horizontally-inserted pixel number (i.e., the number of pixels 15 inserted in each of Nr read-out lines for interpolation) and the vertically-inserted pixel number (i.e., the number of lines inserted among the Nr read-out lines for interpolation) are Mi and Ni, respectively, then the horizontally- and vertically-inserted pixel numbers are expressed as Mi = Mo - Mr, and Ni = No - Nr.
It should be noted that the size of the im~ging element array of the im~ging 20 device 20 and the field memory 62, i.e., M and N may be set either equal to or larger than that of each field of the output image signal (or an output picture size), i.e., Mo and No.
Thus, if the size of the im~ging element array, M X N, is equal to the output picture size Mo XNo, the hand movement correction involves a digital zoom enlargement, which cause a certain reduction of the resolution. If the size M XN of the im~gin~; element array 25 is set for such a value as to provide sufficient horizontal and vertical margins Mh and Mr, the readout frame size Mr X Nr can be set equal to the output picture size. In this case, the hand movement correction can be achieved without a digital zoom enlargement.
Returning now to FIG. l, the digital image signal from the A/D converter 30 is also supplied to the PV vector generator 64. The PV vector generator 64 calculates quantities indicative of the direction and the magnitude (i.e., a PV vector B) of the picture blurring between the current field and the last field. In this illustrative embodiment, the vector generator 64 calculates as such quantities, for example, horizontal and vertical elements Bh and Bv of the PV vector B in a well-known manner in the art. Thus, B = (Bh, Bv) in this example. The vector (Bh, Bv) is passed to the zoom controller 66 and the address generator 70.
FIG. 3 is a flowchart showing an exemplary operation of the zoom controller 66.
10 In response to the reception of the vector(Bh, Bv), the zoom controller 66 enters the operation of FIG. 3. The zoom controller 66 first makes a test in steps 300 and 302 to see if the picture blurring is within the correctable range, i.e., if the current readout frame 62 l completely included in the field memory 62 as shown in FIG. 4. If so, i.e., if lBhl < Mh in step 300 and lBvl < Mv in step 302, then the zoom controller 66 sends (as a correction 15 amount) a signal indicative of correctable blurring to the readout frame size calculator in step 304. Then, the zoom controller 66 passes the output signal from the zoom control switch 40 as it is to the actuator dirver 54 in step 306, and ends the operation. Thus, in this case, if the user has kept and is keeping the zoom control switch 40 at the IN position, the optical zoom operation continues.
Before the description of cases where the picture blurring is not within the correctable range, we discuss the operation of correcting the picture blurring within the correctable range. The operation is performed by the address generator 70 and the interpolator 72.
In response to the reception of the signal indicative of correctable blurring, the 25 readout frame size calculator 68 in turn sends a signal to this effect to the address generator 70. On receiving the signal, the address generator 70 generates addresses for use in reading the readout frame which is at the position shifted by the vector B (= (Bh, Bv)) from the center of the field memory 62 and has the predetermined size of Mr X Nr.
Though the position of the readout frame 621 is indicated by the center of the readout frame 621 in FIG. 4, the position may be indicated by any convenient point such as any corner of the readout frame 621. The readout of pixel data from the memory 62 isperformed line by line in synchronism with the operation of the interpolator 72. The read out pixel data from the memory 62 are temporarily stored in two Mo-pixel line memory in the interpolator 72 and interpolated in a well-known manner such that each field of the output image signal from the interpolator 72 has a size of Mo X No. Each of Mi pixels inserted in each of the Nr lines of the readout frame is given as an average of the two 10 adjacent pixels, and each pixel of each of the Ni lines inserted among the Nr lines is given as an average of the two vertically adjacent pixels. Thus, blurring of a picture within the correctable region is corrected as shown in FIG. 5.
Returning now to FIG. 3, cases where the picture blurring is not within the correctable range will be described. In these cases, the correctable region is virtually 15 expanded by reducing the readout frame size and raising the digital zoom power accordingly. If the readout frame is vertically out of the correctable range, i.e., if the test result is NO in step 302, then the zoom controller 66 outputs a vertical correction amount Dv calculated as:
Dv= lBvl - Mv 20 in step 308 and send a zoom OFF signal as a zoom control signal to the actuator driver 54 to stop the optical zoom operation in step 310. If the readout frame is horizontally out of the correctable range, i.e., if lBhl > Mh in step 300 and lBvl < Mv in step 312, then the zoom controller 66 outputs a horizontal correction amount DH calculated as:
DH= lBhl - Mh 2~ in step 316 and again send a zoom OFF signal as a zoom control signal to the actuator driver 54 to stop the optical zoom operation in step 310.
If the readout frame is both horizontally and vertically out of the correctable range, i.e., if the test result is NO in step 312, then the zoom controller 66 determines dominant one of the horizontal and vertical directions which is the decisive factor to define the size of the readout frame by seeing if lDv/Dhl S N/M in step 314. If so, meaning the direction BD of the picture blurring, (denoted with ~ = arctan(Bv/Bh)), runs horizontally between the two diagonals D 1 and D2 of the field memory 62 (+ c~ =
+arctan(N/M)), then the zoom controller 66 outputs the above-mentioned vertical correction amount Dv in step 308. Otherwise (shown in FIG. 6), the zoom controller 66 outputs the horizontal correction amount DH in step 316. After step 308 or 316, the controller 66 performs step 310 and ends the operation of FIG.3.
In response to the reception of a horizontal or a vertical correction amount DH
or Dv, respectively, the readout frame size calculator 68 determines the size of the reduced readout frame 626 which is defined by a horizontal pixel number Mr' and a vertical pixel number Nr' by means of a proportional arithmetic as shown in FIG. 6. FIG.
7 is a flowchart showing the operation of the readout frame size calculator 68 when the 15 calculator 68 has received either DH or Dv. In step 702, the calculator 68 makes a test to see if it received, say, a vertical correction amount Dv. If so, then, in step704, the calculator 68 calculates Nr' and Mr' as follows:
Nr' = Nr - 2Dv, and (1) Mr' = Mr - 2Dv Mr/Nr.
Otherwise, in step 706, the calculator 68 calculates Mr' and Nr' as follows:
Mr' = Mr - 2Dh, and (2) Nr' = Nr - 2Dh-Nr/Mr.
25 After step 704 or 706, the calculator 68 outputs the horizontal and vertical pixel numbers Mr' and Nr', respectively, to the field memory 62 and the interpolator 72.
In response to receiving the above mentioned PB vector B (= (Bh, Bv)) and the reduced readout frame size Mr' and Nr', the address calculator 70 generates addresses for use in reading a reduced readout frame which is at the position shifted by the vector (Bh, Bv) from the center of the field memory 62 and has a reduced size of Mr' X Nr'. On the other hand, in receiving the reduced readout frame size Mr' and Nr', the interpolator 72 calculates the horizontally- and vertically-inserted pixel numbers Mi' and Ni', respectively, 5 by the following equations:
Mi' = Mo - Mr', and (3) Ni' = No - Nr'.
Then, the interpolator 72 performs interpolation in cooperation with the address10 generator 70 in the above-mentioned manner by inserting Mi' pixels in each of the Nr' lines of the readout frame and inserting Ni' lines among the Nr' lines such that each field of the output image signal from the interpolator 72 has a size of Mo X No pixels. In this way, a video camera or an image signal processor can correct picture blurring of an extended magnitude without increasing a field memory in capacity.
The foregoing merely illustrates the principles of the invention. Thus, though the above-described illustrative embodiment has used a picture blurring vector generator 64 which calculates the PB vector from the current field and the last field, it is possible to implement the present invention by using a pitch and a yaw angular velocity sensors to obtain the pitch and the yaw angles. However, in this case, the embodiment requires 20 means for associating the angles with the horizontal and the vertical elements Bh and Bv of the PV vector, e.g., a reference table. The reference table will have to be created through experiments. Considering these, it is much preferable to obtain PV vectors through the calculation of the image signal.
The above illustrative embodiment has used separate hardware for the zoom 26 controller and the readout frame size calculator. However, these functions may be performed by a single controller including a microprocessor.
Though the above-described image signal processor or PV compensation &
digital zoom unit is shown as comprising discrete components, the whole or a part of it is realized either as a single dedicated integrated circuit or as a part of an integrated circuit.
Also, the zoom controller and/or the readout frame size calculator may be included in a controller which controls the entire of the video camera.
Many widely different embodiments of the present invention may be constructed without departing from the spirit and scope of the present invention. It should be understood that the present invention is not limited to the specific embodiments described in the specification, except as defined in the appended claims.
FOR PICTURE BLURRING CAUSED BY MOVEMENT OF THE HANDS
BACKGROUND OF THE INVENTION
l . Field of the Invention The invention relates to a video camera and more particularly to an image signal processor, for use in a video camera, for preventing blurring of resultant television pictures caused by unintentional movements of the hands or the camera during optical zooming in. The image signal processor is capable of correcting picture blurring of an enhanced magnitude caused by a hand movement during an optical zoom-in operation.
10 2. Description of the Prior Art Unintentional movements of the hands or the camera (hereinafter referred to as "hand movements") in shooting result in blurring of resultant television pictures (hereinafter referred to as "picture blurring"). It is especially true when the user is zooming in because the higher the zoom power becomes, the more the picture blurring 15 increases in magnitude. For this, the present invention contemplates the picture blurring caused by hand movements in zoom-in shooting.
Various systems which deal with problems of such picture blurring have been proposed so far.
A basic scheme to correct the picture blurring is to store a field of shot and 20 A/D-converted image in a field memory, to read out, from the field memory, pixel data in a reduced-size frame at a position moved from the last frame position in the direction of and by the distance of the picture blurring, and to interpolate the read-out pixel data to restore the reduced-size frame to an original size. Doing this enables the correction of the picture blurring as long as both of horizontal and vertical components of the magnitude of 25 the picture blurring are smaller than halves of horizontal and vertical differences, respectively, between the original size and the reduced-size frame (the latter quantities are referred to as "allowable dimensions of picture blurring" or "allowable picture blurring dimensions"). Thus, the basic correction scheme is only available to picture blurring within the allowable picture blurring dimensions, which we call an allowable dimension problem.
It is noted that the allowable picture blurring dimensions comprise horizontal 5 and vertical components of the allowable magnitude of picture blurring, which is a function of not only a hand movement but also the zoom m~gnification or power as described above.
One of the solutions of the allowable dimension problem is to enlarge the allowable picture blurring dimensions by simply increasing the field memory in size or 10 capacity. In this solution, the added memory will not be used in ordinary shooting conditions or very low zoom power, becoming less efficient.
Another solution of the allowable dimension problem is disclosed in Japanese unexamined patent publication No. HeiS- l 30,48 l (l 993) titled "IMAGE PICKUP
DEVICE." In the image pickup device, if the zoom power is not less than the maximum 15 zoom power which is calculated from a rocking angle of the image pickup device caused by a hand movement and the capacity of the field memory, the zoom-in operation is disabled so that the resultant picture blurring remains within the allowable picture blurring dimensions. Thus, since the zoom power is limited by the magnitude of picture blurring, the zoom power may not be raised to a satisfactory extent due to a hand 20 movement.
Also, U.S. Patent 5,502,484 issued March 26, l 996 discloses a video camera and a video signal reproducing apparatus which has a CCD (charge coupled device) and an image memory for PAL (Phase Alternation Line) to provide a hand movement-corrected output image signal for NTSC (National Television System Committee). A
25 hand movement correction in the vertical direction is mainly achieved by shifting a reading out position (the readout starting line) of the CCD, and a hand movement correction in the horizontal direction is mainly achieved by horizontally shifting a fixed-size cut-out range by a correction amount associated by coefficients of a low pass filter with the way angle of the video camera. Though "by ch~nging the coefficient of the low pass filter according to the m~gnification, a wide correction range can be obtained", it is difficult and makes the system complicated to associate the electronic zoom 5 m~gnification with the coefficient such that the image memory is fully utilized. This is not only because the association has to be achieved experimentally and needs additional table but also because, as for angular velocity sensors used for the detection of horizontal and vertical angles, the characteristic may very from sensor to sensor.
It is therefore an object of the invention to provide an image signal processor 10 and an im~ging system which is capable of correcting picture blurring of an enhanced magnitude caused by a hand movement during an optical zoom-in operation by switching into an electronic or digital zoom-in operation when the magnitude of picture blurring exceeds the allowable picture blurring dimensions (or the allowable magnitude of picture blurring).
SUMMARY OF THE INVENTION
The foregoing object is achieved by an im~ging system such as a video camera which comprises an optical zoom arrangement, an im~ging device such as an M X N-pixel CCD for providing an image signal, an analog-to-digital converter for converting the image signal into a digital image signal, and a picture blurring (PV) compensation &
20 digital zoom unit or an image signal processor.
In the image signal processor, a field of pixels in the digital image signal is stored in a field memory for M X N pixels. For each field, a test is made to see if the picture blurring is within a predetermined range, i.e., if a readout frame is included in a correctable range (or in the M XN pixels). If so, a conventional PV compensation is 25 performed by an address generator reading out pixel data in a readout frame which has a predetermined size of Mr XNr pixels (Mr<M, Nr<N) and which is at a position shifted from the center of the field by a PV vector and an interpolator interpolating the read out pixel data into a predetermined output size of Mo X No pixels.
If the picture blurring is not within a predetermined range, a zoom controller disables the optical zoom-in operation and determines the size (Mr' XNr') of a reduced-6 size readout frame such that the picture blurring is just fit in the predetermined range. In response to receiving the size of the reduced-size readout frame, the address generator reads out pixels in the reduced-size readout frame, in the field memory, which is located at a position shifted from the center of the field by the PV vector and which has the reduced size. In response to receiving the size of the reduced-size readout frame, the 10 interpolator interpolates the read out pixels such that the read out pixels of the reduced-size are m~gnified up to the predetermined output size.
BRIEF DESCRIPTION OF THE DRAWING
Further objects and advantages of the present invention will be apparent from the following description of the preferred embodiment of the invention as illustrated in 15 the accompanying drawing, in which:
FIG. l is a schematic block diagram showing a part of an image pickup device such as a video camera which incorporates an image signal processor according to an illustrative embodiment of the invention;
FIG. 2 is a diagram of the field memory 62;
FIG. 3 is a flowchart showing an exemplary operation of the zoom controller 66;
FIG. 4 is a diagram illustrating a situation where the picture blurring is within the correctable range;
FIG. 5 is a diagram illustrating how blurring of a picture within the correctable 25 region is corrected;
FIG. 6 is a diagram illustrating a situation where the picture blurring has exceeded the correctable region in the direction BD running vertically between the two diagonals Dl and D2 of the field memory 62; and FIG. 7 is a flowchart showing the operation of the readout frame size calculator68 when the calculator 68 has received either DH or Dv.
Throughout the drawing, the same elements when shown in more than one figure are designated by the same reference numerals.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
FIG. l is a schematic block diagram showing a part of an image pickup device l such as a video camera which incorporates an illustrative embodiment of an image 10 signal processor 60 for effectuating picture blurring compensation and/or digital zoom in accordance with the invention. The image pickup device l comprises an optical system l0 including a lens group (not shown) for use in a zoom operation, an im~ging device 20 comprising a CCD (charge coupled device) or a set of Cads, an analog to digital (A/D) converter 30 the input of which is coupled to the im~ging device 20. The image pickup 15 device l further comprises a zoom control switch 40 which permits the user to select one of three positions, i.e., a zoom-in position (denoted as IN) to raise the zoom power, a zoom-out position (denoted as OUT) to lower the zoom power and an OFF position to maintain the current zoom power. The image pickup device 1 still further comprises a optical zoom section which comprises the above-mentioned lens group (not shown) 20 included in the optical system l 0, an actuator 52 such as a motor for mechanically adjust a lens position to increase and decrease the zoom power, and an actuator driver 54 for electrically driving the actuator 52. Also, the image pickup device 1 comprises the image signal processor 60.
The image signal processor 60 comprises a field memory 62 for storing a field 25 of picked-up and A/D converted image signal from the A/D converter 30; a picture blurring (PV) vector generator 64 having an input thereof coupled to the A/D converter 30 for providing a picture blurring vector (or PV vector) for each field; a zoom controller 66 for receiving the PV vector and a zoom control switch 40 output and for providing an optical zoom control signal to the actuator driver 54 and providing a correction amount calculated from the PV vector and horizontal and vertical correction margins Mh and Mv as detailed later; a readout frame size calculator 68 for receiving the correction amount from the zoom controller 66 and for providing a readout frame size as described in detail later; an address generator 70 for receiving the PV vector and the readout frame size and for providing addresses to the field memory 62; and an interpolator 72 which is coupled to the field memory 62, which receives the readout frame size and which provides an 10 picture blurring compensated image signal as an output of the image signal processor 60.
Referring now to FIG. l, the operation of the image pickup device l will be described in the following. If the user is shooting an object, then an image of the object is formed by the optical system l 0 on the surface of the im~ging device 20, which provides a picked-up image signal. Assuming that the im~ging device comprises an array of M XN
15 im~ging elements, each field of the picked-up image signal comprises M XN pixels, i.e., N horizontal lines each comprising M pixels (M and N are any natural numbers). The picked-up image signal is converted by the A/D converter 30 into a digital image signal, which is supplied to the image signal processor or a PV compensation and digital zoom unit 60.
In the image signal processor 60, each of the fields in the digital image signal is stored in the field memory 62. The size of the field memory 62 is equal to that of the im~ging element array of the imaging device 20, i.e., M X N. Increasing the im~ging element array size and the memory 62 size results in an increase in the allowable picture blurring dimensions (or magnitude) and accordingly results in an increase in an allowable magnitude of hand movements and/or in a allowable zoom m~gnification.
In order to enable the picture blurring to be corrected when the picture blurring is within the allowable magnitude (or dimensions), i.e., when hand movements and/or the zoom power is relatively small, the designer of the image signal processor 60 first determines the size, in pixels, of a readout (or cut-out) frame or window (hereinafter referred to as "the readout frame size"), Mr X Nr (Mr<M, Nr<N). FIG. 2 is a diagram of the field memory 62. In FIG. 2, it is assumed that the readout frame 621 is located at the 5 center of the M-by-N pixel block or the field memory 62. The rest 623 of the field memory (referred to as a correction margin) is used for the picture blurring correction.
Letting the horizontal and the vertical correction margins be 2 X Mh and 2 X Mv, then we obtain 2XMh = M - Mr, and 2XMv=N-Nr.
The number No of horizontal SC~nning lines and the number Mo of pixels of each sc~nning line in the output image signal from the image signal processor 60, i.e., the interpolator 72 are preferably determined according to one of standard television systems.
Thus, assuming that the horizontally-inserted pixel number (i.e., the number of pixels 15 inserted in each of Nr read-out lines for interpolation) and the vertically-inserted pixel number (i.e., the number of lines inserted among the Nr read-out lines for interpolation) are Mi and Ni, respectively, then the horizontally- and vertically-inserted pixel numbers are expressed as Mi = Mo - Mr, and Ni = No - Nr.
It should be noted that the size of the im~ging element array of the im~ging 20 device 20 and the field memory 62, i.e., M and N may be set either equal to or larger than that of each field of the output image signal (or an output picture size), i.e., Mo and No.
Thus, if the size of the im~ging element array, M X N, is equal to the output picture size Mo XNo, the hand movement correction involves a digital zoom enlargement, which cause a certain reduction of the resolution. If the size M XN of the im~gin~; element array 25 is set for such a value as to provide sufficient horizontal and vertical margins Mh and Mr, the readout frame size Mr X Nr can be set equal to the output picture size. In this case, the hand movement correction can be achieved without a digital zoom enlargement.
Returning now to FIG. l, the digital image signal from the A/D converter 30 is also supplied to the PV vector generator 64. The PV vector generator 64 calculates quantities indicative of the direction and the magnitude (i.e., a PV vector B) of the picture blurring between the current field and the last field. In this illustrative embodiment, the vector generator 64 calculates as such quantities, for example, horizontal and vertical elements Bh and Bv of the PV vector B in a well-known manner in the art. Thus, B = (Bh, Bv) in this example. The vector (Bh, Bv) is passed to the zoom controller 66 and the address generator 70.
FIG. 3 is a flowchart showing an exemplary operation of the zoom controller 66.
10 In response to the reception of the vector(Bh, Bv), the zoom controller 66 enters the operation of FIG. 3. The zoom controller 66 first makes a test in steps 300 and 302 to see if the picture blurring is within the correctable range, i.e., if the current readout frame 62 l completely included in the field memory 62 as shown in FIG. 4. If so, i.e., if lBhl < Mh in step 300 and lBvl < Mv in step 302, then the zoom controller 66 sends (as a correction 15 amount) a signal indicative of correctable blurring to the readout frame size calculator in step 304. Then, the zoom controller 66 passes the output signal from the zoom control switch 40 as it is to the actuator dirver 54 in step 306, and ends the operation. Thus, in this case, if the user has kept and is keeping the zoom control switch 40 at the IN position, the optical zoom operation continues.
Before the description of cases where the picture blurring is not within the correctable range, we discuss the operation of correcting the picture blurring within the correctable range. The operation is performed by the address generator 70 and the interpolator 72.
In response to the reception of the signal indicative of correctable blurring, the 25 readout frame size calculator 68 in turn sends a signal to this effect to the address generator 70. On receiving the signal, the address generator 70 generates addresses for use in reading the readout frame which is at the position shifted by the vector B (= (Bh, Bv)) from the center of the field memory 62 and has the predetermined size of Mr X Nr.
Though the position of the readout frame 621 is indicated by the center of the readout frame 621 in FIG. 4, the position may be indicated by any convenient point such as any corner of the readout frame 621. The readout of pixel data from the memory 62 isperformed line by line in synchronism with the operation of the interpolator 72. The read out pixel data from the memory 62 are temporarily stored in two Mo-pixel line memory in the interpolator 72 and interpolated in a well-known manner such that each field of the output image signal from the interpolator 72 has a size of Mo X No. Each of Mi pixels inserted in each of the Nr lines of the readout frame is given as an average of the two 10 adjacent pixels, and each pixel of each of the Ni lines inserted among the Nr lines is given as an average of the two vertically adjacent pixels. Thus, blurring of a picture within the correctable region is corrected as shown in FIG. 5.
Returning now to FIG. 3, cases where the picture blurring is not within the correctable range will be described. In these cases, the correctable region is virtually 15 expanded by reducing the readout frame size and raising the digital zoom power accordingly. If the readout frame is vertically out of the correctable range, i.e., if the test result is NO in step 302, then the zoom controller 66 outputs a vertical correction amount Dv calculated as:
Dv= lBvl - Mv 20 in step 308 and send a zoom OFF signal as a zoom control signal to the actuator driver 54 to stop the optical zoom operation in step 310. If the readout frame is horizontally out of the correctable range, i.e., if lBhl > Mh in step 300 and lBvl < Mv in step 312, then the zoom controller 66 outputs a horizontal correction amount DH calculated as:
DH= lBhl - Mh 2~ in step 316 and again send a zoom OFF signal as a zoom control signal to the actuator driver 54 to stop the optical zoom operation in step 310.
If the readout frame is both horizontally and vertically out of the correctable range, i.e., if the test result is NO in step 312, then the zoom controller 66 determines dominant one of the horizontal and vertical directions which is the decisive factor to define the size of the readout frame by seeing if lDv/Dhl S N/M in step 314. If so, meaning the direction BD of the picture blurring, (denoted with ~ = arctan(Bv/Bh)), runs horizontally between the two diagonals D 1 and D2 of the field memory 62 (+ c~ =
+arctan(N/M)), then the zoom controller 66 outputs the above-mentioned vertical correction amount Dv in step 308. Otherwise (shown in FIG. 6), the zoom controller 66 outputs the horizontal correction amount DH in step 316. After step 308 or 316, the controller 66 performs step 310 and ends the operation of FIG.3.
In response to the reception of a horizontal or a vertical correction amount DH
or Dv, respectively, the readout frame size calculator 68 determines the size of the reduced readout frame 626 which is defined by a horizontal pixel number Mr' and a vertical pixel number Nr' by means of a proportional arithmetic as shown in FIG. 6. FIG.
7 is a flowchart showing the operation of the readout frame size calculator 68 when the 15 calculator 68 has received either DH or Dv. In step 702, the calculator 68 makes a test to see if it received, say, a vertical correction amount Dv. If so, then, in step704, the calculator 68 calculates Nr' and Mr' as follows:
Nr' = Nr - 2Dv, and (1) Mr' = Mr - 2Dv Mr/Nr.
Otherwise, in step 706, the calculator 68 calculates Mr' and Nr' as follows:
Mr' = Mr - 2Dh, and (2) Nr' = Nr - 2Dh-Nr/Mr.
25 After step 704 or 706, the calculator 68 outputs the horizontal and vertical pixel numbers Mr' and Nr', respectively, to the field memory 62 and the interpolator 72.
In response to receiving the above mentioned PB vector B (= (Bh, Bv)) and the reduced readout frame size Mr' and Nr', the address calculator 70 generates addresses for use in reading a reduced readout frame which is at the position shifted by the vector (Bh, Bv) from the center of the field memory 62 and has a reduced size of Mr' X Nr'. On the other hand, in receiving the reduced readout frame size Mr' and Nr', the interpolator 72 calculates the horizontally- and vertically-inserted pixel numbers Mi' and Ni', respectively, 5 by the following equations:
Mi' = Mo - Mr', and (3) Ni' = No - Nr'.
Then, the interpolator 72 performs interpolation in cooperation with the address10 generator 70 in the above-mentioned manner by inserting Mi' pixels in each of the Nr' lines of the readout frame and inserting Ni' lines among the Nr' lines such that each field of the output image signal from the interpolator 72 has a size of Mo X No pixels. In this way, a video camera or an image signal processor can correct picture blurring of an extended magnitude without increasing a field memory in capacity.
The foregoing merely illustrates the principles of the invention. Thus, though the above-described illustrative embodiment has used a picture blurring vector generator 64 which calculates the PB vector from the current field and the last field, it is possible to implement the present invention by using a pitch and a yaw angular velocity sensors to obtain the pitch and the yaw angles. However, in this case, the embodiment requires 20 means for associating the angles with the horizontal and the vertical elements Bh and Bv of the PV vector, e.g., a reference table. The reference table will have to be created through experiments. Considering these, it is much preferable to obtain PV vectors through the calculation of the image signal.
The above illustrative embodiment has used separate hardware for the zoom 26 controller and the readout frame size calculator. However, these functions may be performed by a single controller including a microprocessor.
Though the above-described image signal processor or PV compensation &
digital zoom unit is shown as comprising discrete components, the whole or a part of it is realized either as a single dedicated integrated circuit or as a part of an integrated circuit.
Also, the zoom controller and/or the readout frame size calculator may be included in a controller which controls the entire of the video camera.
Many widely different embodiments of the present invention may be constructed without departing from the spirit and scope of the present invention. It should be understood that the present invention is not limited to the specific embodiments described in the specification, except as defined in the appended claims.
Claims (10)
1. An image signal processing circuit for correcting picture blurring of an enhanced magnitude caused by a hand movement during an optical zoom-in operation, the image signal processing circuit comprising:
means for storing a field of input pixels from the outside;
means for determining if said picture blurring is within a predetermined range;
means for reading out pixels in a readout frame which is located at a position shifted from a center of said field by said picture blurring and which, if a specified size of said readout frame is received, has said specified size and, otherwise, has a predetermined size;
means responsive to a negative determination by said determining means for disabling said optical zoom-in operation and for determining said specified size of said readout frame such that said picture blurring is just fit in said predetermined range; and means responsive to receiving said specified size of said readout frame for interpolating said read out pixels such that said read out pixels of said specified size are magnified up to a predetermined output size.
means for storing a field of input pixels from the outside;
means for determining if said picture blurring is within a predetermined range;
means for reading out pixels in a readout frame which is located at a position shifted from a center of said field by said picture blurring and which, if a specified size of said readout frame is received, has said specified size and, otherwise, has a predetermined size;
means responsive to a negative determination by said determining means for disabling said optical zoom-in operation and for determining said specified size of said readout frame such that said picture blurring is just fit in said predetermined range; and means responsive to receiving said specified size of said readout frame for interpolating said read out pixels such that said read out pixels of said specified size are magnified up to a predetermined output size.
2. An image signal processing circuit as defined in claim 1, wherein said means for determining if said picture blurring is within a predetermined range includes means for calculating a vector indicative of said picture blurring from a current field and a last field of said input pixels from the outside.
3. An image signal processing circuit as defined in claim 1, wherein all of said means are realized as an single integrated circuit.
4. An image signal processing circuit as defined in claim 1, wherein at least a part of said means is realized as a part of an integrated circuit.
5. A method for correcting picture blurring of an enhanced magnitude caused by a hand movement during an optical zoom-in operation in an imaging device for providing an image signal comprising pixels wherein said imaging device includes a field memory, the method comprising the steps of:
storing a field of said pixels from said imaging device in said field memory;
determining if said picture blurring is within a predetermined range;
reading out pixels in a readout frame, in said field memory, which is located at a position shifted from a center of said field by said picture blurring and which, if a specified size of said readout frame is received, has said specified size and, otherwise, has a predetermined size;
in response to a negative determination by said determining step, disabling said optical zoom-in operation and determining said specified size of said readout frame such that said picture blurring is just fit in said predetermined range; and in responsive to receiving said specified size of said readout frame, interpolating said read out pixels such that said read out pixels of said specified size are magnified up to a predetermined output size.
storing a field of said pixels from said imaging device in said field memory;
determining if said picture blurring is within a predetermined range;
reading out pixels in a readout frame, in said field memory, which is located at a position shifted from a center of said field by said picture blurring and which, if a specified size of said readout frame is received, has said specified size and, otherwise, has a predetermined size;
in response to a negative determination by said determining step, disabling said optical zoom-in operation and determining said specified size of said readout frame such that said picture blurring is just fit in said predetermined range; and in responsive to receiving said specified size of said readout frame, interpolating said read out pixels such that said read out pixels of said specified size are magnified up to a predetermined output size.
6. A method as defined in claim 5, wherein said step of determining if said picture blurring is within a predetermined range includes the step of calculating a vector indicative of said picture blurring from a current field and a last field of said input pixels from said imaging device.
7. An imaging system capable of correcting picture blurring of an enhanced magnitude caused by a hand movement during an optical zoom-in operation, the imaging system comprising:
means for effectuating said optical zoom-in operation;
means subjected to said optical zoom-in operation for providing an image signal comprising pixels;
means for storing a field of said pixels from said means for providing an image signal;
means for determining if said picture blurring is within a predetermined range;
means for reading out pixels in a readout frame which is located at a position shifted from a center of said field by said picture blurring and which, if a specified size of said readout frame is received, has said specified size and, otherwise, has a predetermined size;
means responsive to a negative determination by said determining means for disabling said optical zoom-in operation and for determining said specified size of said readout frame such that said picture blurring is just fit in said predetermined range; and means responsive to receiving said specified size of said readout frame for interpolating said read out pixels such that said read out pixels of said specified size are magnified up to a predetermined output size.
means for effectuating said optical zoom-in operation;
means subjected to said optical zoom-in operation for providing an image signal comprising pixels;
means for storing a field of said pixels from said means for providing an image signal;
means for determining if said picture blurring is within a predetermined range;
means for reading out pixels in a readout frame which is located at a position shifted from a center of said field by said picture blurring and which, if a specified size of said readout frame is received, has said specified size and, otherwise, has a predetermined size;
means responsive to a negative determination by said determining means for disabling said optical zoom-in operation and for determining said specified size of said readout frame such that said picture blurring is just fit in said predetermined range; and means responsive to receiving said specified size of said readout frame for interpolating said read out pixels such that said read out pixels of said specified size are magnified up to a predetermined output size.
8. A system as defined in claim 7, wherein said means for determining if said picture blurring is within a predetermined range includes means for calculating a vector indicative of said picture blurring from a current field and a last field of said input pixels from the outside.
9. A system as defined in claim 7, wherein all of said storing means, said determining means, said reading means, said disabling and determining means, said interpolating means are realized as an single integrated circuit.
10. A system as defined in claim 7, wherein at least a part of said storing means, said determining means, said reading means, said disabling and determining means, said interpolating means are realized as a part of an integrated circuit.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP9-155606 | 1997-06-12 | ||
| JP9155606A JPH114379A (en) | 1997-06-12 | 1997-06-12 | Image signal processing unit |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2240343A1 CA2240343A1 (en) | 1998-12-12 |
| CA2240343C true CA2240343C (en) | 2000-10-24 |
Family
ID=15609709
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA 2240343 Expired - Fee Related CA2240343C (en) | 1997-06-12 | 1998-06-11 | Image signal processor having an enlarged correction range for picture blurring caused by movement of the hands |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JPH114379A (en) |
| CA (1) | CA2240343C (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4162333B2 (en) * | 1999-07-09 | 2008-10-08 | 株式会社東芝 | Monitoring system |
| JP2005198700A (en) * | 2004-01-13 | 2005-07-28 | Olympus Corp | Microscope system for surgical operation |
| JP4446193B2 (en) | 2005-11-11 | 2010-04-07 | ソニー株式会社 | Image processing apparatus and method, and program |
| JP4952891B2 (en) * | 2006-05-08 | 2012-06-13 | カシオ計算機株式会社 | Movie shooting device and movie shooting program |
| JP4702233B2 (en) * | 2006-09-11 | 2011-06-15 | ソニー株式会社 | Image data processing apparatus and image data processing method |
| JP5905130B2 (en) * | 2015-01-30 | 2016-04-20 | オリンパス株式会社 | Image processing apparatus, endoscope apparatus, and operation method of image processing apparatus |
-
1997
- 1997-06-12 JP JP9155606A patent/JPH114379A/en active Pending
-
1998
- 1998-06-11 CA CA 2240343 patent/CA2240343C/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| JPH114379A (en) | 1999-01-06 |
| CA2240343A1 (en) | 1998-12-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP3701759B2 (en) | Image stabilization circuit | |
| US5867213A (en) | Image pickup apparatus having image-shake correcting devices determined according to zoom magnifying power | |
| US5990942A (en) | Photographing apparatus using image information for blurring correction | |
| US7260271B2 (en) | Digital image data correction apparatus, digital image data correction method and digital image pickup apparatus | |
| CA2240343C (en) | Image signal processor having an enlarged correction range for picture blurring caused by movement of the hands | |
| JP2005175581A (en) | Image processor | |
| JPH01125064A (en) | Picture fluctuation correcting image pickup device | |
| JP3342041B2 (en) | Imaging device | |
| JP2956527B2 (en) | Video device with image memory function | |
| JPH06153065A (en) | Image pickup device | |
| JPH06197261A (en) | Image pickup device | |
| JPH06237412A (en) | Video processor | |
| JP4503106B2 (en) | Information processing apparatus and method, information processing system, and recording medium | |
| JP2885010B2 (en) | Image motion compensation device | |
| JPH06237413A (en) | Video processor | |
| JPH0846856A (en) | Hand shake correcting device | |
| JPH06237411A (en) | Video processor | |
| JPH0767025A (en) | Video processor | |
| JP2621468B2 (en) | Video camera equipment | |
| JP2735264B2 (en) | Motion compensated imaging device | |
| JP3437189B2 (en) | Electronic magnification changer | |
| EP0989746B1 (en) | Imaging apparatus having a high-speed imaging function with electronic zoom circuit | |
| JP2003078809A (en) | Device and method for correcting image blur and imaging apparatus | |
| JP4195950B2 (en) | Image motion correction device | |
| JP2936793B2 (en) | Imaging device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |