GB2269506A - Intergrated-circuit image sensor - Google Patents

Abstract

An integrated-circuit image sensor comprises on the same substrate a two-dimensional array (50) of image-sensing cells (20), a processing means (60) and a comparison means (70). The image content for each of a repetitive sequential pair of frame scans of the array (22) is separately integrated in the processing means (60) and the result of the two integrations stored in the comparison means (70). The comparison means (70) then compares the stored image content for the two frames and if sufficient difference exists between them, a signal (72) is sent out to indicate this. The time interval between the two frames can be varied to adjust the sensitivity of the image sensor to image changes, either by varying the array scan rate or by arranging for the comparison means to compare non-sequential image frames. Uses of the image sensor include a movement detector for use, for example, in an intruder alarm, and an instrumentation monitor. <IMAGE>

Description

INTEGRATED-CIRCUIT IMAGE SENSOR The invention relates to an integrated-circuit image sensor and in particular to an integrated-circuit image sensor comprising an image processing device having a two-dimensional array of sensing cells.

Image processing devices comprising a two-dimensional array of sensing cells are well known and are based on one of two techniques: the charge coupling (CCD) technique and the diode array technique. Both of these techniques conventionally require that scanning, sensing and storage of a received image take place off the sensing chip, and this in turn means considerable expenditure in production costs and sacrifice in terms of space, even if something as basic as a simple camera is to be realised.

Recent developments in image array technology, however, have made possible the radical compression of the space needed to accommodate on a single substrate an image array together with its associated scanning and sensing circuitry, and it is this that forms the springboard for the present invention.

The aim of the invention is to provide an integrated-circuit image sensor which overcomes the above-mentioned drawbacks associated with the conventional techniques.

Accordingly, the invention provides an integrated-circuit image sensor, comprising on the same substrate a two-dimensional array of image-sensing cells, for sensing a two-dimensional image field in successive frames, processing means for processing the contents of the image-sensing cells at each successive frame, and comparison means for comparing the results of the processing in the processing means for one frame with the results of the processing in the processing means for another frame, the image sensor being arranged to detect and measure a change in the image between the said pair of frames, to compare the change to a threshold level and, if a significant change has occurred, to output a signal to indicate the occurrence.

The image-sensing cells may be scanned in rows and the processing means may include means for integrating the contents of the scanned image-sensing cells across the rows of the image field and forming a quantity representing the total integration for those rows in a particular frame.

The comparison means may include means for storing the results of the processing for the said one frame while the contents of the image-sensing cells in the said another frame are being processed and means for comparing the contents of the storage means for the said one frame with the results of the processing of the image-sensing cells for the said another frame and outputting the said output signal if a significant difference exists between the two compared quantities.

For each successive comparison in the comparison means the old, preceding said another frame may become the new said one frame and the new, succeeding frame may become the new said another frame.

The image-sensing cells may be scanned and their contents processed on a continuous basis and the said one and said another frames may be successive frames.

The image-sensing cells may be scanned and their contents processed on a continuous basis and the said one and said another frames may be non-successive frames.

The image-sensing cells may be scanned and their contents processed on a non-continuous but repetitive basis, the time interval between frame scans determining the sensitivity of the image sensor to changes in image.

The storage means may be capacitors.

The contents of the array of image-sensing cells may be processed at each successive frame in two half-frames, an upper and a lower, the results of the processing in the upper half-frame of the said one frame being compared with the results of the processing in the upper half-frame of the said another frame, the results of the processing in the lower half-frame of the said one frame being compared with the results of the processing in the lower half-frame of the said another frame, the image sensor being arranged to detect a change in the image in either the upper half-frame of the said pair of frames or the lower half-frame of the said pair of frames.

The comparison means may include a storage means for each half-frame.

The processing of the two half-frames may occur sequentially or simultaneously.

The image array may be a diode array.

The invention also provides a security system including one or more integrated-circuit image sensors as described above.

Figure 1 is a schematic diagram of an image array sensor utilising local preliminary charge sensing and common output charge sensing; Figure 2 is a schematic diagram of a diode sensing cell as shown in Figure 1; Figure 3 is a schematic diagram of a preliminary charge-sensing stage used in the image array sensor; Figure 4 is a schematic diagram showing the general principle of operation of the image sensor according to the invention; Figure 5 is a schematic diagram showing one method of scanning the image array in the image sensor according to the invention; Figure 6 is a schematic diagram of one embodiment of the processing means and comparison means in the image sensor according to the invention; Figure 7 is a schematic diagram of an alternative image array addressing arrangement according to the invention;; Figure 8 is a schematic diagram of a second embodiment of the comparison means in the image sensor according to the invention, and Figure 9 is a schematic diagram of a security system including the image sensor according to the invention.

Referring now to the drawings, there is shown in Figure 1 an imaging arrangement 10 which consists of a matrix 22 of light-sensing cells 20, each of which takes the form of a light-sensitive diode 32 and a switching transistor 31, as shown in Figure 2. This matrix of diodes constitutes a pixel array and although this array has been shown in Figure 1 as consisting of only 9 pixels (3 x 3), it will in practice comprise more like 300 x 300 pixels or, where broadcast-quality images are desired, even more (e.g.

500 x 500).

Light incident on the reverse-biased junction of each diode 32 causes a change in the diode's reverse leakage current and this in turn induces a change in charge on the diode's junction capacitance which serves to back off the applied reverse bias potential. When any particular row of pixels is addressed by the row select line 27 associated with that row, the transistors 31 associated with those pixels are switched on and the above-mentioned charge is passed on to the column select lines 25. The column select lines 25 then pass the charge on to the preliminary charge-sensing stages 40.

One of the preliminary charge-sensing sensing stages 40 is shown in more detail in Figure 3. It is based around an integrator 42 comprising an amplifier 41 and a feedback capacitor 44. When a particular row has been addressed by means of its row select line 27, light-induced charge on each column of that row is taken by column select line 25 to the integrator 42 for each row, where it is integrated across capacitor 44. Note that before this takes place, charge on capacitor 44 from a previous integration is first nulled by operation of a switch 48. The resulting voltage appearing at the output of amplifier 41 is then taken by the closing of a switch 45 to a capacitor 46, where it is held until readout of this voltage to the common output charge-sensing stage 24 via line 30 is initiated by the closing of a switch 47.The common output charge-sensing stage 24 is likewise realised as an integrator in exactly the same way as the preliminary sensing stages, so that the charge appearing on the capacitor 46 is transferred to an integrator capacitor in the common output stage 24, the resulting voltage on the output of this stage being then taken to further circuitry. In practice switches 48, 45 and 47 are realised by the use of analogue switching devices, e.g MOS transmission gates.

Signals on the column select lines 29 ensure that only one pixel in a row is processed at any one time by the charge-sensing integrators 42, 24. Thus Figure 3 shows a reset switch 49 (likewise for example a MOS transistor in practice) which normally keeps the inverting input of amplifier 41 at high (+Vdd) potential until the column in question is about to be scanned, at which point transfer of charge from the pixel diode concerned to the capacitor 44 via column select line 25 is enabled by the opening of this reset switch.

The invention extends the usefulness of this basic scheme by incorporating on the same substrate as the imaging arrangement 10 a processing means for processing the contents of the image-sensing cells at each successive scan frame and a comparison means for comparing the results of the processing in the processing means for pairs of frames.

The outline of the integrated-circuit image sensor according to the invention is shown in Figure 4. Figure 4 shows a substrate 80 containing the basic imaging arrangement 10 of Figure 1 and also a processing means 60 and comparison means 70. Serial image data from each row in turn of the array 22 are sent along line 26 from the common output charge-sensing stage 24 to the processing means 60, where they are processed to produce one value representative of the video content of a number of rows of each frame of the image field held in the array 22. (The block 50 in Figure 4 represents the diode array 22 plus the preliminary charge-sensing stages 40 of Figure 1). This value is output along line 62 to the comparison means 70, where it is stored temporarily while another frame of the image field is likewise processed by the processing means 60.The results of the processing of this second field are then compared with the first value stored in the comparison means 70 and if the two values are sufficiently different an output signal is produced on line 72 to inform the user that a significant change in video content has occurred. In practice a significant change in video content over one or more frames may indicate that some movement has taken place in the 'scene being monitored by the array 22, and in the appropriate context, e.g. security monitoring, may signal the presence of an intruder. When a change in scene has been detected, the image seen by the array 22 can be read out along video output line 78.

One method of processing the image data from the array 22 is to integrate the data for each pixel 20 along each row for a number of rows of the image field. This is illustrated in Figure 5. In Figure 5 the image field, which corresponds to the image array 22, is divided into two halves, an upper and a lower half corresponding to rows 0 to 3 and 4 to 7, respectively. (In reality the number of rows would be much greater than this: at least 300). In one embodiment of the processing stage the row and column select lines of the array 22 are driven so that row 3 is scanned first, followed by row 2, then row 1, and finally row 0.

Then the lower half is scanned, starting with row 4 and ending with row 7, as shown by the arrows in Figure 5.

One practical embodiment of this is given in Figure 6.

In Figure 6 the output from the common output charge-sensing stage 24 is sent along line 26 to a switch 55, where it is routed to either an upper-half-frame integrator 61-1 or a lower-half-frame integrator 61-2. The timing of the routing is controlled by means of the row and column drive signals 27 and 29, respectively, and is arranged to follow the scanning scheme of Figure 5, i.e. the pixel data relative to the upper half-frame are integrated by integrator 61-1, while those for the lower half-frame are integrated by integrator 61-2. The outputs of the integrators 61-1, 61-2 are taken via switches 66-1, 66-2 respectively to the comparison means 70. The switches 66-1, 66-2 are arranged to close at some point near the end of the last row in the respective half-frame, to allow the result of the integration to pass through to the comparison stage.In Figure 6 switch 66-1 is shown as closing when scanning of the upper half frame reaches the final pixel for that half (i.e. row 0, column 7), while switch 66-2 is shown as closing in a similar manner, in this case at row 7, column 7. Closing of switches 66-1, 66-2 is thus momentary, but is arranged to be long enough to ensure that the respective capacitors 63, 64 are charged to their new charge level each time. Normally, some form of charge nulling is necessary at the start of the recharging process to ensure that each capacitor takes on its new value of charge. (This is not shown for the sake of clarity).

As an alternative to having the switch 55, it is possible to arrange for the array 22 to be addressed in two halves :simultaneously by means of two sets of row select lines 27-1, 27- 2 and to output data from the two halves simultaneously via two output charge-sensing stages 24-1, 24-2. This is shown in Figure 7.

Since integration results from two different frames are required to be compared, some form of memory is required to hold the results of one frame while the results for the other frame are being collected. One embodiment of this is shown in Figure 6. The output of each integrator 61-1, 61-2 is taken to a switch 67-1, 67-2 where the output is stored on either an even-frame capacitor 63-1, 63-2 or an odd-frame capacitor 64-1, 64-2. The switches 67-1, 67-2 toggle between the two frames whenever the image field is being rescanned, and in this embodiment this can be achieved by operating each switch whenever the "row 3" select line is energised, since this marks the beginning of the scanning process for each new frame. Thus capacitors 63-1, 63-2, 64-1, 64-2 can be seen to hold the results of the integration process for their respective half-frames during almost an entire rescan, until, that is, switches 66-1, 66-2 are commanded to close at the end of each half of the rescan to store the new integration value on the respective capacitor.

Finally, the values on each pair of capacitors 63, 64 are compared in a comparator 71-1, 71-2 and a signal is output on line 72 if either of these comparators registers a significant difference between the values for the two frames.

The above embodiment of the comparator stage compares alternate frames of image data. However, in many situations, e.g. intrusion detection, where the movement to be detected is slow, this could give rise to video content differences that are too small to be detected without the threshold being set so low as to give an excessive number of false alarms. A second embodiment of the comparator means 70 is illustrated in Figure 8 and involves the comparing of integration results for frames that are more widely spaced apart. Thus a counter 74 is provided in Figure 8 which uses the "row 3" select line as a clock at its input 76 and outputs on its output lines 75 signals which can be gated in a gating means 77 and used to toggle the switches 67-1, 67-2 between their two changeover states.

Assuming a division ratio of n:l in the counter 74, capacitor 63-1 will store integration values related to frames 1, 1+2n, 1+4n, etc, while capacitor 64-1 will store values related to frames 1+n, 1+3n, 1+5n, etc. The same applies two the lower-half-frame capacitors 63-2, 64-2, which are not shown in Figure 8 for the sake of clarity.

Since now integration data from the integrators 61 must only be admitted to the storage stage (i.e. capacitors 63, 64) at the end of the appropriate frame, the switches 66-1, 66-2 are arranged to close momentarily at the end of their respective half-frames after switches 67-1 and 67-2 have changed over. Thus, for instance, if n=3, switches 67-1, 67-2 will change over at the start of frame 4 and at the end of frame 4 (row 0, col 7/row 7 column 7) switches 66-1, 66-2 will momentarily close. Then at the start of frame 7 switches 67-1 67-2 will change over again and at the end of that frame (row 0, column 7, etc) switches 66-1, 66-2 will again close momentarily and so on. This is achieved by means of appropriate gating from the counter outputs 75 and the row/column select lines 27/29.

All the switches so far described are in practice realised by analogue switching devices, e.g. MOS transmission gates.

The method, shown in Figure 8, of comparing the video content of two widely separated frames can be substituted by a simpler arrangement involving the "alternate frame" method of Figure 6 and an array addressing scheme in which scanning of the image field is not carried out on a continuous basis, as has hitherto been assumed to be the case, but on an interrupted basis. Thus it can be arranged, for instance, to instruct the row/column select lines 27/29 to perform a complete frame scan only once every 200mS, say, which on a normal 50Hz frame rate would be equivalent to instructing the counter 74 in the system shown in Figure 8 to divide by 10. Then, once a large enough change in video content has been detected, the interrupted scan can be changed to the more normal continuous scan to allow an adequate video readout of the new image to take place along the line 78.

A further embodiment of the comparator stage is possible, in which instead of waiting a number of frames before the integration values are stored on the capacitors 63, 64, the results for the first frame, say, are stored on capacitor means 63 and the results for the next m frames are fed in turn to the capacitor means 64 and are compared with the results for the first frame. Thus if m equalled, for example, 10, the comparison stage would compare the processing results stored on capacitors 64-1, 64-2 for each of frames 2 to 11 in turn (i.e. according to each half-frame) with the results for frame 1 stored on capacitors 63-1, 63-2 and would output a signal on line 72 if any of those ten frames showed a significant difference with respect to frame 1. It can be seen from this that frame 1 acts as a kind of reference frame.Then a new comparison cycle would start, with frame 11 acting as a new reference, though the processing results for this new reference frame would appear on capacitors 64-1, 64-2 instead of capacitors 63-1, 63-2 as in the previous cycle.

This new reference frame would then be compared with the results for the next 10 frames (frames 12 to 21), which would appear in turn on capacitors 63-1, 63-2.

This can be achieved in practice by arranging for the changeover switch 67-1/67-2 in Figure 8 to dwell in its upper position (i.e. the position associated with the capacitor 63- 1/63-2) for one frame and then switch over to its lower position where it would dwell for the next, say, ten frames. The tenth frame of this group of ten frames would be compared with the earlier reference, (as would also the first nine frames) but would then form the new reference frame for the next comparison cycle. Having done that, the changeover switch 67-1/67-2 would switch over to its upper position again for the following ten frames, and so on back and forth. In this particular concrete example, then, the counter 74 would be set to divide by 10.

Ciearly, since in this last embodiment of the comparison means the processing results are required to be stored at each frame instead of once every number of frames, the gating of switch 66-1 (and 66-2) by the gating element 77 shown in Figure 8 must be omitted.

In all the embodiments described above, the comparison means are understood to include some means of establishing a threshold below which any detected change in image is not signalled. This effectively acts as a sensitivity control and could, on the basis of the simple comparator schemes illustrated in Figures 4, 6 and 8, take the form of a hysteresis adjustment in the comparators themselves. By hysteresis adjustment is meant an adjustment of the degree of positive feedback used between the output and the non-inverting input of each comparator. However, the function of sensitivity control is also performed by the choice of "n" in several of the above embodiments, since this governs the threshold below which the speed of movement of an image is not sensed.

The scanning scheme shown in Figure 5 is only one example and others are possible, e.g. simple scanning from top to bottom, as in conventional imaging techniques. Also, the image field does not have to be divided into two, but the video information for the whole field (i.e. frame) can be integrated in just one integrator 61, removing the need for the other half-frame components (i.e. the second set of storage and comparator stages). The memory stages in the comparator means 70 can also take forms other than the capacitors shown.

Uses for the integrated-circuit image sensor of the invention are numerous and involve any application in which a change in an image is required to be signalled. One obvious use is as a movement detector, and this might in practice take the form of some sort of security supervision device. An example of such an arrangement is given in Figure 9. Figure 9 shows a number of image sensors 100 according to the invention, each outputting two signals: the result-of-comparison output signal 72 and the video output signal 78 containing the serial image data from the array 22. Each image sensor 100 monitors a particular geographical area, e.g. a doorway or window, with the aid of a lens arrangement 110 mounted in close proximity to the image array 22. If, say, detector 1 picks up movement, it sends a "movement detected" signal along line 72 to a video routing device 140 which changes a switch 120 over to position 1. The video output from detector 1 is then routed via switch 120 to a common monitor 130. In this way it is not necessary for security personnel to monitor as many displays 130 as there are image sensors 100, since the detection of movement by one of the detectors determines which detector comes up on the monitor display 130. This sort of arrangement becomes very attractive vis-a-vis the more conventional one-monitor-per-detector system when a large number of detectors are incorporated into a system, and this is something that is encouraged by the very small dimensions of the integrated circuit involved.Indeed, the small dimensions of the image sensor according to the invention allow the detector to be mounted in places which would be a bar to more conventional image array based devices; one could even be mounted in a door handle or a door peephole, for instance.

This latter possibility for the siting of the image sensor hints at another use for the invention, namely as a home intruder monitor. An image sensor such as heretofore described could be mounted in each of the outside doors of a house, and when the detector picked up movement outside, the doorbell could be arranged to ring automatically and the picture on the in-house television set could be instructed to change over to the view as seen by the triggered detector.

Other applications are as a fire monitor (the onset of fire would result in a change of image) and as an instrumentation monitor. In the latter case the image array would be set up to look at an instrumentation panel, preferably of the type comprising a multiplicity of luminant indicators such as lamps, LEDs and so on. Then when one or more of the indicators failed, the image sensor would pick this up as a change in image and would signal the failure to the supervising personnel. Both these last two applications involve not so much the monitoring of movement as the monitoring of changes in luminance of a scene.

Although the image array employed in the above embodiments has been assumed to be a diode array, it could alternatively be a CCD array. Also, the method of processing described in the various embodiments, whereby the contents of the image sensing cells are integrated across each row of the array 22, is only one of a number of methods. An alternative technique, for example, would be to compare each pixel of the first frame with the corresponding pixel of the later frame.

Claims (16)

1. An integrated-circuit image sensor, comprising on the same substrate: a two-dimensional array of image-sensing cells, for sensing a two-dimensional image field in successive frames; processing means for processing the contents of the image-sensing cells at each successive frame, and comparison means for comparing the results of the processing in the processing means for one frame with the results of the processing in the processing means for another frame, the image sensor being arranged to detect and measure a change in the image between the said pair of frames, to compare the change to a threshold level and, if a significant change has occurred, to output a signal to indicate the occurrence.

2. An integrated-circuit image sensor, according to claim 1, in which the image-sensing cells are scanned in rows and the processing means includes means for integrating the contents of the scanned image-sensing cells across the rows of the image field and forming a quantity representing the total integration for those rows in a particular frame.

3. An integrated-circuit image sensor, according to claim 1 or 2, in which the comparison means includes means for storing the results of the processing for the said one frame while the contents of the image-sensing cells in the said another frame are being processed and means for comparing the contents of the storage means for the said one frame with the results of the processing of the image-sensing cells for the said another frame and outputting the said output signal if a significant difference exists between the two compared quantities.

4. An integrated-circuit image sensor, according to claim 3, in which for each successive comparison in the comparison means the old, preceding said another frame becomes the new said one frame and the new, succeeding frame becomes the new said another frame.

5. An integrated-circuit image sensor, according to claim 4, in which the image-sensing cells are scanned and their contents processed on a continuous basis and the said one and said another frames are successive frames.

6. An integrated-circuit image sensor, according to claim 4, in which the image-sensing cells are scanned and their contents processed on a continuous basis and the said one and said another frames are non-successive frames.

7. An integrated-circuit image sensor, according to claim 4, in which the image-sensing cells are scanned and their contents processed on a non-continuous but repetitive basis, the time interval between frame scans determining the sensitivity of the image sensor to changes in image.

8. An integrated-circuit image sensor, according to any of claims 4 to 7, in which the storage means are capacitors.

9. An integrated-circuit image sensor, according to any of claims 1 to 8, in which the contents of the array of image-sensing cells are processed at each successive frame in two half-frames, an upper and a lower, the results of the processing in the upper half-frame of the said one frame being compared with the results of the processing in the upper half-frame of the said another frame, the results of the processing in the lower half-frame of the said one frame being compared with the results of the processing in the lower half-frame of the said another frame, the image sensor being arranged to detect a change in the image in either the upper half-frame of the said pair of frames or the lower half-frame of the said pair of frames.

10. An integrated-circuit image sensor, according to claim 9, in which the comparison means includes a storage means for each half-frame.

11. An integrated-circuit image sensor, according to claim 9 or 10, in which the processing of the two half-frames occurs sequentially.

12. An integrated-circuit image sensor, according to claim 9 or 10, in which the processing of the two half-frames occurs simultaneously.

13. An integrated-circuit image sensor, according to any preceding claim, in which the image array is a diode array.

14. A security system including one or more integrated-circuit image sensors according to any of the preceding claims.

15. An integrated-circuit image sensor substantially as shown in or as hereinbefore described with reference to Figures 4 to 8 of the drawings.

16. A security system substantially as shown in or as hereinbefore described with reference to Figures 4 to 9 of the drawings.