WO2019178939A1 - 像素采集电路、光流传感器和光流及图像信息采集*** - Google Patents

像素采集电路、光流传感器和光流及图像信息采集*** Download PDF

Info

Publication number
WO2019178939A1
WO2019178939A1 PCT/CN2018/088261 CN2018088261W WO2019178939A1 WO 2019178939 A1 WO2019178939 A1 WO 2019178939A1 CN 2018088261 W CN2018088261 W CN 2018088261W WO 2019178939 A1 WO2019178939 A1 WO 2019178939A1
Authority
WO
WIPO (PCT)
Prior art keywords
optical flow
signal
unit
pixel
timing
Prior art date
Application number
PCT/CN2018/088261
Other languages
English (en)
French (fr)
Inventor
陈守顺
Original Assignee
上海芯仑光电科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海芯仑光电科技有限公司 filed Critical 上海芯仑光电科技有限公司
Priority to EP18910634.7A priority Critical patent/EP3758365A4/en
Priority to SG11202009370QA priority patent/SG11202009370QA/en
Priority to JP2021500323A priority patent/JP6992238B2/ja
Publication of WO2019178939A1 publication Critical patent/WO2019178939A1/zh
Priority to US17/024,929 priority patent/US12022211B1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H04N25/44—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array
    • H04N25/443—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array by reading pixels from selected 2D regions of the array, e.g. for windowing or digital zooming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H04N25/44—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled by partially reading an SSIS array
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/57—Control of the dynamic range
    • H04N25/571—Control of the dynamic range involving a non-linear response
    • H04N25/573—Control of the dynamic range involving a non-linear response the logarithmic type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/62—Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/703—SSIS architectures incorporating pixels for producing signals other than image signals
    • H04N25/707—Pixels for event detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/745—Circuitry for generating timing or clock signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/75—Circuitry for providing, modifying or processing image signals from the pixel array
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
    • H04N25/772—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising A/D, V/T, V/F, I/T or I/F converters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/779—Circuitry for scanning or addressing the pixel array
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/7795—Circuitry for generating timing or clock signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/78—Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L27/00—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate
    • H01L27/14—Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including semiconductor components sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation
    • H01L27/144—Devices controlled by radiation
    • H01L27/146—Imager structures
    • H01L27/14643—Photodiode arrays; MOS imagers

Definitions

  • the present invention relates to the field of image acquisition technologies, and in particular, to a pixel collection circuit, an optical flow sensor, and an optical flow and image information acquisition system.
  • the optical flow method can determine the motion of the target object based on the image information.
  • the optical flow method can be applied in many fields such as military aerospace, traffic supervision, information science, meteorology and medicine.
  • optical flow was first proposed by Gibson in 1950.
  • the object in reality can be imaged through the photosensitive element of the camera, and the points in the image are in one-to-one correspondence with the points on the object.
  • the projection of the target object on the two-dimensional image plane forms a motion, and the motion of the motion expressed in the image plane luminance mode is called an optical flow.
  • optical flow algorithms are often complex and computationally intensive.
  • the front-end image acquisition device needs to perform image acquisition at a high frame rate, and then perform high-computation optical flow analysis by a subsequent image processing device.
  • the existing optical flow sensor realizes optical flow calculation by integrating an active pixel sensor and a separate image information processor.
  • Conventional optical flow calculation methods mostly calculate motion information of moving objects by calculating multi-frame images in the time domain.
  • active pixel sensors need to operate at very high frame rates, which increases the amount of optical flow calculations and increases the hardware requirements for the back-end information processor.
  • the present invention proposes a new optical flow and image information acquisition scheme.
  • the invention provides a new technical solution for optical flow collection, which effectively solves at least one of the above problems.
  • a pixel collection circuit comprising: a photodetection unit adapted to output a first electrical signal indicative of an intensity of an optical signal illuminating thereon in real time; an optical flow information timing triggering unit, An input end is coupled to the output end of the photodetecting unit, and a second input end is coupled to the optical flow information resetting line, and is adapted to: after the reset signal from the optical flow information reset line is cancelled, and the first electrical signal is detected to satisfy the predetermined When the trigger condition is met, the optical flow information timing trigger signal is output; the optical flow information timing control unit has a first input end coupled to the optical flow information timing trigger unit, and a second input end coupled to the optical flow information reset line, suitable for The timing start signal is output after the reset signal from the optical flow information reset line is cancelled and the optical flow information timing trigger signal is received; the optical flow information timing unit has a first input end coupled to the optical flow information timing control unit, and the first The two input ends are coupled to the optical flow information reset line
  • the image information acquisition unit further includes a first input end coupled to the output end of the photodetection unit, and a second input end coupled to the image information collection line.
  • the image output line selection line is coupled to receive an image line selection signal from the image output line selection line, and to buffer and output the first electrical signal of the current reception time when the image line selection signal is valid.
  • an optical flow sensor comprising: a pixel collection circuit array comprising a plurality of pixel collection circuits as described above; an optical flow information readout unit adapted to read a pixel acquisition circuit array a timing signal output by at least a portion of the pixel acquisition circuit; a global control unit adapted to generate a reset signal and output to the pixel acquisition circuit in the pixel collection circuit array through the optical flow information reset line.
  • the optical flow sensor according to the present invention further includes an image information reading unit adapted to read the first electrical signal output by at least a part of the pixel collecting circuits in the pixel collecting circuit array; and the global control unit is further adapted The image information acquisition signal is generated and output to each pixel acquisition circuit in the pixel collection circuit array through the image information acquisition line.
  • an optical flow and image information collecting system comprising: an optical flow sensor as described above; an optical flow pre-processor coupled to the optical flow information reading unit in the optical flow sensor a time-mapped image adapted to acquire a timing signal and generate an optical flow trigger pixel; and an image pre-processor coupled to the image information readout unit in the optical flow sensor, adapted to acquire the first electrical signal and generate a full pixel point Grayscale image.
  • the optical flow sensor provides a scheme for efficiently extracting motion optical flow information, which can realize effective optical flow information extraction for moving objects of various speeds; meanwhile, image information can be acquired.
  • the background information is provided for the extracted optical flow information by obtaining a clear and time-free full-scale grayscale image.
  • the optical flow sensors output optical flow frames and image frame information in parallel and independently of one another. In this way, the optical flow and image information collecting system according to the present invention can acquire the optical flow and the image information completely in parallel without interfering with each other, and the acquisition and readout speeds of the optical flow information and the image information can be adjusted according to different needs. .
  • FIG. 1 shows a schematic diagram of an optical flow and image information collection system 100 in accordance with some embodiments of the present invention
  • FIGS. 2A, 2B, 2C, and 2D respectively illustrate schematic diagrams of the operation of optical flow sensor 110 in accordance with some embodiments of the present invention
  • 3A-3F are schematic diagrams showing optical flow information extraction of the optical flow sensor 110 according to an embodiment of the present invention.
  • FIG. 4 shows a schematic diagram of a pixel collection circuit 400 in accordance with some embodiments of the present invention.
  • 5A, 5B, 5C, and 5D respectively show schematic diagrams of a photodetection unit 410 in accordance with an embodiment of the present invention
  • 6A, 6B, 6C, and 6D respectively show schematic diagrams of a filter amplification module according to an embodiment of the present invention
  • FIG. 7A, 7B, 7C, and 7D respectively show schematic diagrams of a threshold comparison module in accordance with one embodiment of the present invention.
  • FIGS. 8A and 8B are respectively schematic views of an optical flow information timing unit according to an embodiment of the present invention.
  • FIG. 9 shows a schematic diagram of a pixel collection circuit 900 in accordance with one embodiment of the present invention.
  • FIG. 10 shows a schematic diagram of an optical flow sensor 110 in accordance with some embodiments of the present invention.
  • FIG. 1 shows a schematic diagram of an optical flow and image information collection system 100 in accordance with some embodiments of the present invention.
  • optical flow and image information acquisition system 100 includes an optical flow sensor 110, an image pre-processor 120, and an optical flow pre-processor 130.
  • the optical flow sensor 110 includes a pixel collection circuit array 111 composed of a plurality of pixel collection circuits (i.e., pixel units), an image information readout unit 112, an optical flow information readout unit 113, and a global control unit 114.
  • Each of the pixel collection circuits in the array 111 is coupled to the image information readout unit 112, the optical flow information readout unit 113, and the global control unit 114 via a bus.
  • the pixel collection circuit array 111 generally includes a plurality of rows of pixel acquisition circuits, each of which includes one or more pixel collection circuits. As shown in FIG. 1, a 3*2 total of 6 pixel acquisition circuits is illustrated, but is not limited thereto.
  • the global control unit 114 includes an image acquisition unit 1142 and an optical flow acquisition unit 1144, and controls the entire pixel collection circuit array 111 by generating a global image information acquisition signal and a reset signal, respectively. Specifically, the pixel collection circuit in the array 111 can simultaneously acquire image information and optical flow information under the action of the global control signal given by the global control unit 114.
  • the image information reflects the size of the illumination intensity perceived by the array 111 in the field of view
  • the optical flow information reflects the spatiotemporal information of the motion in the field of view.
  • the two kinds of information are independent of each other by the image information readout unit 112 and the optical flow information readout unit 113.
  • the control is sent to the image pre-processor 120 and the optical stream pre-processor 130, respectively.
  • array 111 can synchronously monitor changes in light intensity within the field of view (eg, illuminance variation and rate of change) Wait). Specifically, each pixel acquisition circuit in array 111 monitors the intensity information of the illumination thereon in real time. A pixel acquisition circuit enters an optical flow trigger state when it is determined that the change of the light intensity information satisfies a predetermined condition (for example, both the illuminance change amount and the change rate exceed the respective threshold values); if the condition is not satisfied, the original state remains unchanged.
  • a predetermined condition for example, both the illuminance change amount and the change rate exceed the respective threshold values
  • the pixel acquisition circuit that enters the optical flow trigger state immediately activates the internal optical flow information timing unit, which implements a timer function inside the pixel acquisition circuit, which is activated after the pixel acquisition circuit optical flow is triggered and generates a time-dependent Monotonically varying timing signals to characterize time information; subsequently, optical flow pre-processor 130 reads the timing of the timing signals (at the time of reading) of all pixel acquisition circuits in pixel array 111 by optical flow information readout unit 113
  • the value can obtain a two-dimensional time map image (optical stream frame), and the timing signal of each pixel point indicates whether the pixel acquisition circuit has triggered the optical flow at the readout time (by detecting whether the signal is a valid timing signal) And how long it has been triggered (by detecting the magnitude of the effective timing signal).
  • the mapping of the optical flow frame in the pixel array plane appears as a two-dimensional time slope plane, and further passes the time slope plane.
  • the analysis can extract the information of the direction and velocity of the motion in the field of view, thereby realizing the calculation of the optical flow.
  • the optical flow information timing unit is a ramp signal generator that is activated only once during the primary optical flow information detection process and locally generates a ramp signal that varies linearly with time.
  • the global control unit 114 gives a (global) image information acquisition signal, and the image information acquisition unit of all the pixel acquisition circuits in the signal control array 111 simultaneously collects and buffers the time and light. Strongly correlated first electrical signal.
  • the intensity information (image frame) of the entire image is obtained by acquiring the first electrical signal buffered by each pixel acquisition circuit.
  • the optical flow sensor 110 is completely independent of the acquisition of the optical flow and the image information, and thus the two kinds of information can be output in parallel.
  • the pixel acquisition circuit in the array 111 can simultaneously collect the optical flow and the image information through the independent image information acquisition unit and the optical flow information timing unit; in addition, the two kinds of information are controlled by the respective readout unit through the independent data bus. Read out to their respective preprocessors. In this way, the optical flow sensor 110 can output the optical flow and image information in full parallel without interfering with each other, and the acquisition and readout speeds of the optical flow information and the image information can be adjusted according to different needs.
  • the optical flow sensor 110 can output optical flow information and image information in parallel.
  • the optical flow information reading unit 113 scans the timing signals of the optical flow information timing unit of the entire pixel collection circuit array 111 at a certain frequency to acquire continuous optical flow frames.
  • Each optical flow frame includes time information of a triggering time of the triggered pixel acquisition circuit, and analyzes the aspect and slope information of the time slope formed by the two-dimensional space mapping of the pixel array 111 to obtain the optical flow information of the motion.
  • the image information readout unit 112 scans the first electric signal collected by the image information acquisition unit of the entire pixel array at a certain frequency to acquire a continuous image frame.
  • Each image frame contains a grayscale image of the full intensity of the light intensity.
  • the grayscale image of the full pixel point acquired by the image frame is used as the background image for the optical flow information mark, and the time slope plane acquired by the optical flow frame can conveniently extract the optical flow information.
  • a typical mode of operation of the optical flow sensor 110 is exemplified in the following with reference to Figures 2A, 2B, 2C and 2D.
  • Figures 2A, 2B, 2C, and 2D squares filled with diagonal lines represent optical flow frames, and unfilled squares represent image frames.
  • the optical flow sensor 110 outputs the optical flow frame and the image frame independently in parallel, and the frame rates of the two may be separately set according to different application scenarios, as shown in FIG. 2A; or may be set to the same frame rate, as shown in FIG. 2B. In some applications, it may only be necessary to continuously transmit optical frame output to achieve motion detection, or only continuous image frame output, as shown in Figures 2C and 2D.
  • operation mode of the optical flow sensor 110 of the present invention may also be configured as other equivalent alternatives than the above examples, and details are not described herein again.
  • optical flow frame and image frame will be further described below.
  • the illumination of some of the pixel acquisition circuits in the array 111 may be changed.
  • the pixel acquisition circuit activates an internal timing unit, that is, the optical flow information acquisition unit, when the illumination change exceeds the threshold, and most simply, by locally generating a ramp signal that changes linearly with time; no change in intensity or light is detected.
  • the pixel acquisition circuit with insufficient strong change maintains its untriggered state, and the internal optical flow information timing unit is not activated.
  • the optical flow information reading unit 113 reads the timing signal of the optical flow information timing unit of each pixel acquisition circuit in the array 111 and transmits it to the optical flow pre-processor 130 at the rear end, which is thereby triggered by the optical flow.
  • a time-mapped image of a pixel For example, it can scan the entire array 111 in rows and send the amplitude of the ramp signal in each optical flow information acquisition unit to the external optical flow pre-processor 130.
  • the timing signal (ramp amplitude signal) of each pixel tells whether the pixel has been triggered at the readout time (by detecting whether the signal is a valid timing signal) and how long has been triggered (by detecting the valid timing signal) Amplitude).
  • the optical stream pre-processor 130 further performs amplification, analog-to-digital conversion processing, and encoding operations on the amplitude signal to obtain corresponding time information from the ramp signal amplitude.
  • the pixel collection circuit in the array 111 is triggered only when the light intensity change exceeding the threshold value is detected for the first time and the primary optical flow information timing unit is activated, and the optical flow information timing unit of all the pixel arrays 111 is scanned. It is possible to obtain an optical flow frame that contains the time and the time information of the triggering time of all the triggered pixel units up to the scanning time.
  • the first optical flow frame output is located at the initial moment of the primary optical flow information detection, the number of useful pixel units acquired by the first optical flow frame is small (sparse effective pixel units are insufficient to form an effective optical flow frame) The time slope plane, so the analysis of the moving light flow cannot be achieved). Since the reading of the optical flow information does not affect the normal operation of the pixel acquisition circuit, as the optical flow frame is continuously read, more and more pixel acquisition circuits enter the light due to the change of the light intensity caused by the detection of the field of view motion. The flow triggers the state and starts the internal optical flow information timing unit. By continuously outputting the optical flow frame, all accumulated time information of all activated optical flow information timing units up to the current frame readout time can be acquired.
  • the time required for the primary optical flow information detection process is also different for the speed of the detected motion. Since the fast motion can accumulate enough trigger pixel units in a relatively short time to form an effective time slope, the optical flow information detection process is relatively short, that is, less optical flow frames are required to complete one motion detection. For slow motion, it takes a long enough time to form an effective motion trajectory that can be detected, and thus it takes a long time to complete a motion detection, that is, multiple optical flow frames are required. As described above, since the reading of the optical flow frame does not affect the normal operation of the pixel unit, the time information of the triggered pixel unit will remain valid until it exceeds its timing range, so that the detection time is long enough to form an effective one.
  • the time slope is used for optical flow analysis.
  • the global control unit 114 For the acquisition of image frames, the global control unit 114 provides a global image information acquisition signal, and the image acquisition unit of all pixel acquisition circuits in the control array 111 samples the first electrical signal associated with the light intensity.
  • the image pre-processor 120 reads the first electrical signal related to the light intensity at the effective moment of the image information acquisition signal by all the pixel collection circuits through the image information readout unit 112 and can be configured to perform amplification processing and analog-to-digital conversion on the signal. Processing, the data it outputs can be used to create a complete background grayscale image on which the extracted optical flow information can be marked. Since the pixel acquisition array 111 synchronously samples the first electrical signal associated with the light intensity, this global shutter-like control mechanism can acquire a clear and hysteresis-free background image.
  • optical flow sensor 110 of the present invention extracts optical flow information
  • the process of extracting optical flow information using the optical flow frame output of the optical flow sensor 110 will be further explained below in conjunction with FIGS. 3A-3F.
  • a simple scene is selected for explanation.
  • a cube A having a uniform shape and uniformity moves in the X direction at a certain speed into the field of view of the optical flow sensor 110.
  • the object A is filled with diagonal lines to indicate that the brightness of the object A is lower than the brightness of the background.
  • the optical flow sensor of the present invention can effectively detect scenes with complex shapes and shapes moving along different motion trajectories at different speeds.
  • the array 111 of optical flow sensors 110 includes only 8 x 8 pixel acquisition circuits, where P(x, y) is used to represent the pixels of the xth and yth columns.
  • the image seen in the scene is shown in Figure 3B.
  • the field of view of the pixels P(3,1) to P(6,2) is occupied by the object A and is represented as gray, and the remaining pixels detect only the background with higher brightness and are represented as white.
  • the motion of object A those pixel units whose field of view area is covered by the motion track of object A will enter the optical flow trigger state, because these pixel units will detect the light intensity from the initial high-brightness background to the low-brightness object. Change (other pixel units are not triggered because the detected light intensity has not changed).
  • the optical flow information timing unit within the pixel unit will be activated and report relative time information from the optical flow frame readout time.
  • FIG. 3C shows a timing signal generated by the optical flow information timing unit in the pixel unit.
  • the timing signal is only a representation selected for convenience of description, and may actually be implemented in other forms.
  • the timing signal is a single ramp signal that is activated at the moment the pixel unit is triggered, then slowly decays, and the slope of its amplitude as a function of time is known and adjustable.
  • the optical flow information reading unit continuously scans the optical flow information timing unit of the entire pixel array to obtain a continuous optical flow frame output. In a single optical flow frame, the pixel unit triggering time distance can be obtained by detecting the amplitude of the timing signal. Relative time information of the current optical frame output.
  • the amplitude of the entire ramp signal is 10 units and its slope with time is 1 unit/msec, then within a certain optical frame, if the amplitude of the read timing signal is 7 (as shown in the figure) 3C shows the point A), then it can be concluded that the pixel unit to the optical stream frame readout time has been triggered for 3 milliseconds; if the read timing signal has a magnitude of 2 (as shown by point B in Figure 3C) Then, it can be concluded that the pixel unit has been triggered for 8 milliseconds until the optical stream frame readout time.
  • the timing information of the pixel unit read by each optical flow frame actually represents how long each pixel unit has been triggered when the optical flow frame is read out, and the time information is obtained by the optical flow preprocessor.
  • the amplitude of the transient timing signal output by each pixel unit is calculated. The earlier the pixel unit is triggered, the lower the amplitude to be read; the later the triggering time, the higher the amplitude of the reading, the amplitude of the timing signal that the untriggered and triggered pixels are read. The values are very low, as can be seen from Figure 3D.
  • the figure shows the timing signals generated by six pixel units P(3,3) to P(3,8).
  • the pixel unit will in turn trigger and activate the corresponding optical flow information timing unit, each of which generates a timing signal that appears as a translation on the time axis.
  • the trigger time information of the P(3,3) pixel unit For the optical stream frame output at 1st time, since only P(3,3) triggers up to that moment, it can acquire the trigger time information of the P(3,3) pixel unit as 0.5ms; for the 6th optical flow frame output Since these pixel units are triggered, it can acquire the trigger time information of these pixel units.
  • P(3,3) triggers the earliest
  • P(3,8) triggers the latest
  • the amplitude of the corresponding timing signal is read by Low to high, according to the slope of the timing signal and the transient amplitude of each timing signal at the readout time, it can be obtained that P(3,3) has been triggered for 5.5ms at the readout time, P(3,8) ) 0.5 ms has been triggered at this readout time.
  • the pixel units P(3,3) to P(6,3) are triggered at the 0.5ms (because at this time, half of the field of view corresponding to these pixel units is covered by the moving object), and the respective internals are activated at this moment.
  • Optical flow information acquisition timing unit Similarly, as the object continues to move, the pixel units P(3, 4) through P(6, 4) will trigger at 1.5 ms and activate the internal optical flow information acquisition timing unit. At this time, since the pixel units of P(3,3) to P(6,3) have been triggered for 1 ms, their timing units have been clocked for 1 ms.
  • the pixel units (rows 3 to 6) of the following columns 5 to 8 will enter the optical flow trigger state at 2.5 to 5.5 ms and activate the respective optical flow information acquisition timing units.
  • the optical flow information reading unit of the optical flow sensor also continuously scans the pixel array to read the timing signal of the optical flow information collecting timing unit of the pixel unit and generates a two-dimensional time information mapping image.
  • the optical flow sensor 110 outputs an optical flow frame every 1 ms, and the time-mapped image formed by each optical flow frame is as shown in FIG. 3E.
  • a to F respectively represent the optical flow read out from 1 to 6 ms.
  • the time-mapped image of the frame is assumed that the optical flow sensor 110 outputs an optical flow frame every 1 ms, and the time-mapped image formed by each optical flow frame is as shown in FIG. 3E.
  • a to F respectively represent the optical flow read out from 1 to 6 ms.
  • the time-mapped image of the frame is assumed that the optical flow sensor 110 outputs an optical flow frame every 1
  • the frame can only acquire the timing signals of these pixel units for 0.5ms, which forms The time information map image is shown as A in Figure 3E.
  • the optical stream frame can detect P(3,3) to P(6). , 3) and P (3, 4) to P (6, 4) these trigger pixel units.
  • the detected timing signals of the P (3, 3) to P (6, 3) pixel units are 1.5 ms
  • the timing signals of the P (3, 4) to P (6, 4) pixel units are For 0.5 ms
  • the time information map image formed by it is shown as B in Fig. 3E.
  • the time-mapped image formed by the optical flow frames outputted from the 3rd to 6th ms is as shown in C to F in FIG. 3E.
  • the subsequent optical flow frame can detect more excitation pixel units than the previous optical flow frame, and thus a more efficient time slope plane can be formed on the time map image.
  • the optical flow sensor 110 can achieve effective detection for different speed motions.
  • the optical flow sensor 110 can still output a clear image with high speed and no lag while the optical flow information is detected.
  • the optical flow analysis can be conveniently performed by using the time slope information mapped by the optical flow frame generated by the optical flow information readout unit 113.
  • the steepness of the time slope and the aspect of the slope characterize the velocity of the object and the direction of motion.
  • (1) in Fig. 3F shows the time slope formed when the object passes through the field of view at different movement speeds in the opposite direction of the X-axis. Since the direction of motion is the same, the slope directions of the three time slopes are consistent, but the difference in speed causes the steepness of the time slope to be different.
  • the three time slopes from high to low correspond to a moving speed of 0.5 pixels, 1 pixel, and 2 pixels per millisecond.
  • FIG. 4 shows a schematic diagram of a pixel collection circuit 400 in accordance with some embodiments of the present invention.
  • the pixel collection circuit 400 includes at least a photodetection unit 410, an optical flow information timing trigger unit 420, an optical flow information timing control unit 430, an optical flow information timing unit 440, and a row selection output unit 450.
  • the pixel acquisition circuit further includes an image information acquisition unit 460.
  • the optical flow information reset line and the image information acquisition line shown in FIG. 4 are given by the optical flow acquisition unit 1144 and the image acquisition unit 1142 of the global control unit 114; the image output line selection line is output by the image information readout unit 112; The stream output row selection line is output by the optical stream information readout unit 113; the data output bus is sent to the image information readout unit 112 and the optical flow information readout unit 113, respectively.
  • the photodetection unit 410 outputs a first electrical signal characterizing the intensity of the optical signal illuminated thereon in real time.
  • the photodetection unit 410 is, for example, a plurality of logarithmic photodetectors, but is not limited thereto.
  • 5A, 5B, 5C, and 5D show schematic views of a photodetecting unit 410, respectively, in accordance with an embodiment of the present invention.
  • photodetection unit 410 includes an anode-grounded photodiode PD 1 and a first transistor T 1 .
  • the source of the first transistor T 1 is connected to the cathode of the photodiode PD 1 , and the drain and the gate thereof are connected to the power source VDD.
  • photodiode PD 1 generates current I after receiving an illumination signal.
  • the voltage change generated between the source and the gate of T 1 is linearly related to lnI.
  • the first electrical signal of the photodetecting unit 410 in this embodiment is in a logarithmic relationship with the intensity of the illumination light signal.
  • the photodetection unit 410 includes a grounded anode of the photodiode PD 1, a first transistor T 1 and the first amplifier A 1.
  • the source of the first transistor T 1 is connected to the cathode of the photodiode PD 1 and the drain thereof is connected to the power source VDD.
  • the first amplifier A 1 is connected between the cathode of the photodiode PD 1 and the gate of the first transistor T 1 .
  • the first amplifier A 1 can increase the response speed at which a voltage change occurs between the source and the gate of T 1 . In other words, the first amplifier A 1 increases the speed at which the pixel acquisition circuit detects the change in light intensity.
  • the photodetection unit includes an anode-grounded photodiode PD 1 and N transistors in series, where N ⁇ 2 (N in FIG. 5C is 2, but is not limited thereto), the first the source of the transistor and the cathode of the photodiode PD 1 is connected to the drain of the N transistor is connected to the VDD power source, gate and drain of each transistor, the second to the N transistor of each transistor is connected to a source To the drain of the first transistor.
  • the N transistors in series can increase the current (voltage) gain of the photodetector.
  • the photodetecting unit 410 shown in FIG. 5D is further configured with a first amplifier A 1 based on the embodiment shown in FIG. 5C.
  • the first amplifier A 1 is connected between the cathode of the photodiode PD 1 and the gate of the first transistor.
  • the present invention can also employ a variety of well-known high-real-time photodetecting units, which are not described herein.
  • the conventional photodetection technology usually needs to charge the capacitor, and then perform continuous exposure (capacitance continuous discharge), and then determine the accumulated light intensity according to the remaining capacity of the capacitor.
  • the photodetection unit 410 according to the present invention does not require an additional exposure time when generating the first electrical signal representative of the intensity of the optical signal. Therefore, the photodetection unit 410 can output the first electrical signal without delay.
  • the collection of optical flow information in the pixel unit is performed by the optical flow information timing trigger unit 420, the optical flow information timing control unit 430, and the optical flow information timing unit 440. All three modules are controlled by the global reset signal output from the optical flow acquisition unit 1144, and when the signal is valid, the three units associated with the optical flow information detection are reset. The detection of the primary optical flow information starts after the reset signal is cancelled, and the optical flow information readout unit 113 continuously scans the optical flow information timing unit 440 of all the pixel units until a sufficient amount of optical flow analysis is accumulated in the optical flow frame. Effective pixel unit. The specific functions and implementations of these three units are given below.
  • the first input end of the optical flow information timing trigger unit 420 is coupled to the output end of the photodetection unit 410, and the second input end is coupled to the optical flow information reset line.
  • the optical flow information timing trigger unit 420 includes a filter amplification module 422 and a threshold comparison module 424.
  • the optical flow information timing triggering unit 420 is reset at an initial stage of primary optical flow information detection, and then is responsible for, when the first electrical signal (representative light intensity) output by the photodetecting unit 410 changes to meet a predetermined trigger condition during the optical flow information detecting process.
  • the optical flow information timing trigger signal is generated and sent to the optical flow information timing control unit 430 at the rear end.
  • the predetermined trigger condition is set such that the first electrical signal changes rapidly enough and the duration of the change is long enough. Since the first electrical signal responds in real time to the light intensity signal that is incident on the photodetecting unit 410, the requirement for the first electrical signal as described above is also the requirement for the light intensity signal. A change in light intensity that is fast enough and high enough can be considered to be caused by the effective motion of the object. Conversely, changes in light intensity that are slow or slow in intensity may be due to the drift or jitter of the background light without the value of analysis.
  • the optical flow information timing trigger unit 420 includes a filter amplification module 422 and a threshold comparison module 424.
  • the filter amplification module 422 performs a pre-processing operation on the first electrical signal to generate a second electrical signal, wherein the pre-processing operation includes at least one of an amplification operation and a filtering operation; the threshold comparison module 424 determines whether the second electrical signal is greater than the first The threshold and/or is less than the second threshold, and generates an optical flow information timing trigger signal when the second electrical signal is greater than the first threshold or less than the second threshold.
  • the amplification operation is to increase the sensitivity of the pixel acquisition circuit to light intensity detection, but this is not essential.
  • the filtering operation is generally high-pass filtering, that is, only responding to high-frequency, that is, fast enough light intensity changes, thereby filtering out those slow-changing light intensity changes.
  • the filter amplification module 422 can employ a variety of well-known filtering and amplification techniques, but is not limited thereto.
  • Figures 6A, 6B, 6C, 6D of the present invention illustrate schematic views of a filter amplification module 422, respectively, in accordance with some embodiments of the present invention.
  • the filter amplification module 422 includes a second amplifier A 2 and a high-pass filter
  • the second amplifier A 2 is input to the output terminal of the positive-connected photo-detection unit 410, and the input negative-electrode is connected with the first resistor R 1 that is pulled down
  • a second resistor R 2 is connected between the output terminal and the input negative pole
  • the second amplifier A 2 and the first resistor R 1 and the second resistor R 2 provide an amplification function
  • the gain is related to the ratio of the resistors R 2 and R 1 .
  • a high pass filter coupled to the second amplifier A 2 is configured to filter out signal components below the frequency threshold in the amplified first electrical signal to output a second electrical signal.
  • the filter amplification module 422 includes a first capacitor C 1 , a second amplifier A 2 , a second capacitor C 2 , a third resistor R 3 , and a first switch K 1 .
  • the first end of the first capacitor C 1 is connected to the output end of the photodetecting unit 410; the input positive pole of the second amplifier A 2 is connected to a fixed potential, and the input negative pole is connected to the second end of the first capacitor C 1 ;
  • the second capacitor C 2 , the third resistor R 3 and the first switch K 1 are both connected in parallel between the input negative terminal and the output terminal of the second amplifier A 2 , wherein when the reset signal is valid, the first switch K 1 is closed, when reset After the signal is cancelled, the first switch K 1 is turned off.
  • a first capacitor C 1 may be isolated from the DC component of the first electrical signal.
  • the ratio of the first capacitor C 1 and the second capacitor C 2 is proportional to the gain of the filter amplification module 422.
  • the second capacitor C 2 and the third resistor R 3 constitute a filter, and R 3 is an adjustable resistor.
  • the filter can filter out signal components below the frequency threshold in the AC component of the first electrical signal.
  • the frequency threshold depends on the resistance of R 3 .
  • R 3 may, for example, be adjusted by the system 100 via an indication signal transmitted by the global control unit 114, which may be determined based on user input, but is not limited thereto.
  • the filter amplification module 422 includes a second amplifier A 2 , a first capacitor C 1 , a first switch K 1 , and a high pass filter.
  • the input anode of the second amplifier A 2 is connected to the output of the photodetecting unit 410; the first end of the first capacitor C 1 is connected to the input negative terminal of the second amplifier A 2 , and the second end thereof is connected to a fixed potential (usually The fixed potential is a ground potential, but is not limited thereto; the first switch K 1 is connected between the first end of the first capacitor C 1 and the output end of the photo detecting unit 410; the high pass filter, and the second amplifier A 2 connected to filter out a signal component below the predetermined frequency threshold in the amplified first electrical signal to generate a second electrical signal.
  • the first switch K 1 when the reset signal is active, the first switch K 1 is closed, when the reset signal is canceled, the first switch K 1 is turned off.
  • the first control switch K 1 of the first storage capacitor C 1 of the first electrical signal output from the photodetection unit 410 as the reference level then the second amplifier A 2 detects and amplifies a first electrical signal in real time The difference from this reference level.
  • the embodiment shown in Figures 6A, 6B, and 6C is an amplification and filtering operation on the relative amount of change of the first electrical signal.
  • the filter amplification module 422 can also detect the light intensity change by directly detecting the rate of change of the first electrical signal, as shown in FIG. 6D.
  • the filter amplification module 422 includes a first capacitor C 1 , a current replicator P 1 , a second amplifier A 2 , a second capacitor C 2 , a first switch K 1 , and a third resistor R 3 .
  • the first end of the first capacitor C 1 is connected to the output end of the photodetecting unit 410; the first end of the current replicator P 1 is connected to the second end of the first capacitor C 1 ; the input positive pole of the second amplifier A 2 Connected to a fixed potential, the input negative terminal is connected to the second end of the current replicator P 1 ; the second capacitor C 2 , the first switch K 1 and the third resistor R 3 are both connected in parallel to the input negative terminal and output of the second amplifier A 2 between the ends, wherein, when the reset signal is active, the first switch K 1 is closed, when the reset signal is canceled, the first switch K 1 is turned off.
  • the first end and the second end of the current replicator P 1 are respectively connected with a first current source I 1 and a second current source I 2 , and the current replicator P 1 clamps the first end thereof at a fixed level, and A current of a capacitor C 1 is copied to the second current source I 2 via the first current source I 1 .
  • the current replicator P 1 can be implemented in accordance with a variety of well-known techniques and will not be described herein.
  • One end of the first capacitor C 1 is connected to the first electrical signal output by the photodetecting unit 410, and the other end is connected to a fixed level.
  • the current of the first capacitor C 1 is proportional to the rate of change of the first electrical signal, and the current is copied to the current source I 2 via the current source I 1 and the second capacitor C 2 is charged and discharged.
  • a second electrical signal is generated.
  • the amplitude of the second electrical signal is determined by the speed at which the first electrical signal changes (the magnitude of the charge and discharge current I 2 ) and the duration of the change (the duration of the charge and discharge current I 2 ). Only a first electrical signal (light intensity) that lasts longer and changes faster can generate a larger second electrical signal.
  • the third resistor R 3 is adjustable and serves as a leakage resistor, and can cancel the charging effect of the second capacitor C 2 when the second current source I 2 is very small, thereby filtering out the AC component of the first electrical signal below the frequency threshold.
  • the signal component enables high-pass filtering.
  • An input of the threshold comparison module 424 is coupled to an output of the filter amplification module 422.
  • the threshold comparison module 424 is configured to determine whether the second electrical signal is greater than the first threshold and/or is less than the second threshold, and generate an optical flow information collection trigger signal when the value is greater than the first threshold or less than the second threshold.
  • the threshold comparison module 424 may only determine whether the second electrical signal is greater than the first threshold; or may only determine whether the second electrical signal is less than the second threshold; or determine whether the second electrical signal is greater than the first threshold And determining whether the second electrical signal is less than the second threshold, and the second threshold is less than the first threshold.
  • the threshold comparison module 424 can detect whether the amount of change in the illumination intensity on the pixel collection circuit 400 is large (light intensity can be made larger or smaller).
  • a threshold comparison module 424 comprises a first voltage comparator VC 1, which is connected to the inverting input terminal of a first threshold value to provide a signal line which is connected with the inverting input terminal of the filter output terminal 422 of the amplifier module .
  • the threshold comparison module 424 of the present embodiment can determine whether the second electrical signal is greater than the first threshold.
  • the threshold comparison module 424 includes a second voltage comparator VC 2, its noninverting input terminal and providing a signal line of the second threshold value is connected to its inverting input connected to the filter amplifier module output terminal 422 .
  • the threshold comparison module 424 of the present embodiment can determine whether the second electrical signal is less than the second threshold.
  • the threshold comparison module 424 includes: a first voltage comparator VC 1, VC 2 second voltage comparator and an OR gate.
  • a first voltage comparator VC 1 is connected to the inverting input terminal of a first threshold value to provide a signal line connected to the noninverting input terminal of the output of the filter amplifier module 422; noninverting input of the second voltage comparator VC 2 is provided with a second The signal line of the threshold is connected, and the inverting input end is connected to the output end of the filter amplifying module 422; the first input end of the OR gate is coupled to the output end of the first voltage comparator, and the second input end is connected to the second voltage comparator The output is coupled to perform an OR logic operation on the output of the first voltage comparator and the output of the second voltage comparator.
  • the threshold comparison module 424 includes a differential pressure comparator VD 1 having a first input coupled to a pull-down third capacitor C 3 and a second input coupled to a pull-down fourth capacitor C. 4, for outputting a first difference signal input terminal and a second input terminal; a second switch K 2, is provided between the output terminal of the third capacitor C 3 and the amplifier module 422 of the filter; sequentially third capacitance C 3 connected in series And a first buffer B 1 and a third switch K 3 between the fourth capacitor C 4 ; the first voltage comparator VC 1 has an inverting input terminal connected to a signal line providing a first threshold, and a non-inverting input terminal thereof The output of the differential pressure comparator; the second voltage comparator VC 2 has a non-inverting input connected to a signal line providing a second threshold, and an inverting input connected to an output of the differential pressure comparator; or a gate, the first The input end is coupled to the output end of the first voltage comparator, the second voltage comparator V 2
  • the optical flow information timing control unit 430 receives the optical flow information timing trigger signal generated by the front end optical flow information timing triggering unit 420, and generates a timing start signal of the back end optical flow information timing unit 440, which is mainly composed of two parts: latching And pulse shaper.
  • the latch is reset when the reset signal is active and is asserted when the optical flow information timing trigger signal is first received, and then remains set until the next reset signal is asserted (next optical flow) Information detection begins).
  • the pulse shaper receives the latch signal output by the latch and generates a narrow pulse signal as the timing enable signal for the back end optical flow information timing unit 440 when the latch is asserted.
  • a pulse shaper is not required and it may be post-incorporated into the optical flow information timing unit 440.
  • the optical flow information timing control unit 430 adopts such a configuration to ensure that, in the primary optical flow information detection process, the optical flow information timing unit 440 of the pixel unit detects the light intensity change for the first time in the pixel unit to satisfy the threshold requirement (optical flow information timing)
  • the trigger unit 420 is activated once when the timing trigger signal is output, and is not repeatedly activated. This is because the latch in the optical flow information timing control unit 430 is not actively reset after the first assertion (until the next reset signal is asserted). This feature is necessary for optical flow analysis to obtain sufficient and accumulated information of the effective pixel unit time by continuously updating the optical flow frame during the detection of the optical flow information.
  • the optical flow information timing unit 440 is a timing unit inside the pixel unit that is reset at the initial stage of the primary optical flow information detection, and is started at the pixel unit optical flow timing trigger timing and starts timing.
  • the time information of the pixel activation time can be obtained by the current timing information of the optical flow timing unit 440.
  • FIG. 8A shows an embodiment of the optical flow information timing unit 440.
  • the optical flow information timing unit 440 includes: a fourth switch K 4 , a fifth switch K 5 , a fifth capacitor C 5 , and a third a transistor T 3 , wherein the fifth switch K 5 , the fifth capacitor C 5 , and the third transistor T 3 are both connected in parallel between the fourth switch K 4 and a fixed potential (not shown here in the figure);
  • the first end of the four switch K 4 is connected to the fifth switch K 5 , the fifth capacitor C 5 and the third transistor T 3 connected in parallel, and the second end thereof is connected to another fixed potential (illustrated as power supply but not limited thereto) ).
  • the gate voltage of the third transistor T 3 is a bias voltage input from the outside, and thus T 3 can be equivalent to a current source, and the current magnitude can be adjusted according to the application scenario.
  • the fifth switch (K 5 ) is closed, the upper plate of the fifth capacitor C 5 is discharged to the ground potential; and when the timing start pulse outputted by the front end optical flow information timing control unit 430 is received, the fourth switch is K 4 is closed for a short period of time and then disconnected, the upper plate of the fifth capacitor C 5 is quickly pulled up to the power supply voltage (K 4 is closed), and then slowly discharged (K 4 is turned off) by the third transistor T 3 , thereby A high-to-low ramp signal is formed. As shown in FIG.
  • the ramp signal is actually a timing signal.
  • the time information of the pixel unit triggering time can be known. The higher the amplitude of the signal, the later the pixel unit triggers (the later the corresponding motion occurs), the lower the amplitude of the signal, indicating that the pixel unit is triggered earlier (the earlier the corresponding motion occurs). Since the lowest potential can only be the ground potential (the temperature cannot be further decreased after the ground potential is lowered), the range of the effective time information that the optical flow information timing unit 440 can give is limited, and the time information beyond the range can only be fixed. The value is expressed.
  • the slope of the ramp signal determines the timing range of the optical flow information timing unit 440, and the slope thereof is determined by the discharge current of the third transistor T 3 and the magnitude of the fifth capacitor C 5 .
  • the smaller the discharge current, the fifth capacitor The larger the C 5 is, the smaller the slope of the ramp signal is, the slower the attenuation is, and the longer the effective timing range is. Conversely, the larger the slope, the faster the discharge, and the shorter the effective timing range.
  • a slowly varying ramp signal is effective for detecting slow motion, because in this case it usually takes a long enough time to accumulate enough trigger pixel units corresponding to the motion trajectory for optical flow analysis, due to The effective timing range is long, and the early triggered pixel unit can still give valid timing information after a long enough time, so that the ideal time slope information can be obtained.
  • a ramp signal with a large slope can be used for fast motion detection, since a sufficient number of pixel trigger units can be accumulated in a short time.
  • the optical flow information timing unit 440 can also be implemented by the embodiment shown in FIG. 8B.
  • the fifth switch K 5 , the fifth capacitor C 5 , and the fourth The resistor R 4 is connected in parallel between the fourth switch K 4 and the fixed potential; the first end of the fourth switch K 4 is connected to the fifth switch K 5 , the fifth capacitor C 5 and the fourth resistor R 4 connected in parallel, The two terminals are connected to another fixed potential, and when the reset signal is valid, the fifth switch (K 5 ) is closed. Thus, the charge stored on the fifth capacitor C 5 is discharged through the fourth resistor R 4 .
  • the timing signal is an exponentially attenuated voltage signal, and the speed of the attenuation is determined by the magnitudes of the fourth resistor R 4 and the fifth capacitor C 5 .
  • the fourth resistor R 4 requires a large resistance value, which can be achieved by a crystal in an off state or a reverse biased diode.
  • the first input end of the row selection output unit 450 is coupled to the optical flow information timing unit 440, the second input end is coupled to the optical flow output line selection line, and the timing signal from the optical flow information timing unit 440 is buffered and When the light pop selection signal is valid, it is sent to the optical flow data output bus, wherein the optical pop selection signal is from the optical flow information readout unit 113, and the optical flow data output bus is sent to the optical flow information readout unit 113.
  • the optical flow information readout unit 113 processes the data of the optical flow output bus and transmits it to the external optical flow preprocessor 130.
  • the pixel collection circuit 400 further includes an image information collection unit 460 for acquiring image information, the first input end of which is coupled to the output end of the photodetection unit 410, and the second input end is coupled to the image information collection line.
  • an image information collection unit 460 for acquiring image information, the first input end of which is coupled to the output end of the photodetection unit 410, and the second input end is coupled to the image information collection line.
  • image information acquisition unit 460 is generally comprised of a simple sample and hold circuit including a control switch and a sampling capacitor.
  • the first input end of the control switch is connected to the first electrical signal outputted by the photodetecting unit, the second end is connected to the first end of the sampling capacitor, and the second end of the sampling capacitor is connected to a fixed level.
  • the control signal of the control switch is a global image information acquisition signal.
  • the control switch When the signal is valid, the control switch is closed, and the sampling capacitance of all the pixel units samples the first electrical signal output by the photodetection unit 410 in the pixel unit; when the image information is collected After becoming invalid, the control switch is turned off, the sampling capacitor holds the sampled first electrical signal and is then read out by the image information readout unit 110 to the image preprocessor 120 to form an image frame.
  • the third input end of the row selection output unit 450 is coupled to the image information collection unit 460, and the fourth input end is coupled to the image output line selection line so as to be coupled to the image information collection unit 460.
  • the input signal ie, the first electrical signal
  • the input signal is buffered and sent to the image data output bus when the image line selection signal is valid, and the image line selection signal is from the row selection unit of the image information readout unit 112, the image data
  • the output bus is sent to the image information readout unit 112.
  • the image information readout unit 112 processes the data of the image output bus and transmits it to the external image preprocessor 120.
  • the pixel collection circuit 400 can simultaneously output image and optical flow information, and since two pieces of information are transmitted to an external preprocessor through independent readout channels, the image frame and optical flow of the optical flow sensor 110 The acquisition of frames is parallel and independent of each other.
  • the collection of optical flow information requires global reset signals and control of the optical flow information readout unit.
  • the detection of primary optical flow information may require multiple optical flow frames.
  • the reset signal is valid.
  • all the units related to the optical flow information detection in the pixel unit (such as the optical flow information timing trigger unit, the optical flow information timing control unit, and the optical flow information timing unit) are Reset.
  • the reset signal is cancelled, the optical flow information detection starts, and the optical flow information readout unit scans the timing information of the optical flow information timing unit of the entire pixel array to form an optical flow frame.
  • the acquisition of the image frame is controlled by the image information acquisition signal and the image information readout unit given by the global control unit.
  • the image information acquisition signal is valid for a short period of time
  • the image information acquisition unit in the pixel array samples and maintains the first electrical signal output by the photodetection unit, and then the image information readout unit scans the entire pixel one by one.
  • the first electrical signal associated with the light intensity stored by the image information acquisition unit in the array thereby obtaining a complete image frame.
  • FIG. 9 shows an exemplary schematic diagram of pixel acquisition circuit 900, taking one of them as an example.
  • the pixel collection circuit 900 includes a photodetection unit 910, an image information acquisition unit 920, a filter amplification module 931, a threshold comparison module 932, an optical flow information timing control unit 940, an optical flow information timing unit 950, and a row selection. Output unit 960.
  • the filter amplification module 931 and the threshold comparison module 932 constitute an optical flow information timing trigger unit 930 (not shown).
  • the photodetecting unit 910 shown in FIG. 9 is arranged as a photodetecting unit as shown in FIG. 5B. Of course, the photodetecting unit 910 can also be provided as any one of FIGS. 5A, 5C and 5D, or any other capable of implementing the present invention.
  • the filter amplification module 931 shown in FIG. 9 is set as the filter amplification module shown in FIG. 6B, and it should be noted that the filter amplification module 931 can also be arranged as any of FIGS. 6A, 6C, and 6D.
  • the threshold comparison module 932 shown in FIG. 9 is configured as the threshold comparison module shown in FIG. 7C. It should be noted that the threshold comparison module 932 can also be configured as any of FIGS. 7A, 7B, and 7D, and details are not described herein again.
  • the optical flow information timing unit 950 shown in Fig. 9 is configured as an optical flow information timing unit as described in Fig. 8A, and it should be noted that the optical flow information timing unit 950 can also be configured as shown in Fig. 8B, here No longer.
  • the reset signal When primary light detection start flow information, the reset signal is active, the filter-amplifier module of a first switch 931 K 1 is closed, the second electrical signal is output to a fixed reference potential, which is filtered and amplified with a second amplifier module 931
  • the reference potential of the input positive connection of A 2 is identical, and is generally located in the middle of the first threshold and the second threshold of the backend threshold comparison module 932.
  • the optical flow information timing trigger unit 930 does not give an optical flow information timing trigger signal.
  • the first switch K 1 After the reset signal is canceled, the first switch K 1 is turned off, the first capacitor C 1 samples the first electric signal output from the photodetection unit 910 at that time as an initial reference level for subsequent detection.
  • the second electrical signal is responsive to a difference between the first electrical signal of the real time and the initial reference level, and when the second electrical signal exceeds the threshold detection range of the threshold comparison module 932, the optical flow information timing trigger signal is provided.
  • the second capacitor C 2 and the adjustable resistor R 3 in the filter amplification module 931 constitute a high-pass filter such that the slow-changing low-frequency components in the first electrical signal are shielded. That is, although the difference between the second electrical signal in response to the first electrical signal in real time to the initial reference level, the initial reference level is not entirely determined by the first electrical signal when the first switch K 1 is turned off.
  • the OR logic unit ie, OR gate
  • the threshold comparison module 932 sends or outputs the outputs of the first two comparison units to the optical flow information timing control unit 940, which ensures a change in light intensity in different directions (by strong Detection to weak and weak to strong).
  • the optical flow information timing control unit 940 receives the timing trigger signal generated from the front end optical flow information timing trigger unit 930, and generates a timing start signal of the back end optical flow information timing unit 950.
  • the optical flow information timing control unit 940 is mainly composed of two parts: a latch and a pulse shaper.
  • the latch of the front end is used to characterize whether the pixel unit has been triggered during the detection of the optical flow information, its reset signal is a global reset signal, and the set signal is from the local optical flow information timing trigger signal. Therefore, at the beginning of the primary optical flow information detection, the latch of the pixel unit in the array is forcibly reset.
  • the pixel unit that detects the change in the effective light intensity sets the latch by the optical flow information trigger signal. Since the automatic reset is not possible, the set state is maintained until the next reset signal is valid (the next optical flow information detection starts). Thus, the moment when the latch is set represents the moment when the active unit detects the effective motion for the first time in the optical flow detection process, although the optical flow information triggering unit still responds to the change of the light intensity and can give the timing trigger signal again. However, since the latch has been set, the later timing trigger signal is masked.
  • the pulse shaper detects the latch signal output by the latch and generates a pulse signal as a timing enable signal for the back end optical flow information timing unit 950 when the latch is set.
  • the pixel acquisition circuit responds only to the light intensity change that satisfies the condition for the first time, and activates the optical flow information timing unit 950 having the monotonic gain characteristic.
  • the time information of the spatial and optical flow information timing unit of the triggered pixel unit in the array has an exact mapping relationship with the spatial and temporal information of the motion evolution, thereby obtaining a time-mapped image in an effective optical flow mode.
  • the optical information measuring unit shown in the flow, the time signal (K 5 closed) 950 is reset when the reset signal is asserted to the ground potential, then decreased at a constant slope when starting (K 4 closed) is rapidly pulled to the supply voltage, To ground potential.
  • the time information of the triggering moment of the pixel unit can be obtained by reading the amplitude information of the ramp signal at different times.
  • the row select output unit 960 is configured as a buffer with two select switches.
  • the two buffers are independent for outputting the first electrical signal given by the image information acquisition unit 920 and the time signal given by the optical flow information timing unit 950.
  • the image output line selection line is valid, the selection switch connected to the first electrical signal line is closed, and the first electrical signal given by the image information collection unit 920 is buffered and sent to the first electrical signal line.
  • the optical flow output line selection line is valid, the selection switch connected to the time signal line is closed, and the timing signal given by the optical flow information timing unit 950 is buffered and sent to the time signal line.
  • FIG. 10 shows a schematic diagram of an optical flow sensor 110 in accordance with some embodiments of the present invention.
  • the optical flow sensor 110 includes at least a pixel collection circuit array 111, an optical flow information readout unit 113, and a global control unit 114.
  • the optical flow sensor 110 may further include an image information readout unit 112.
  • the pixel collection circuit array 111 includes a plurality of pixel collection circuits 400.
  • the optical flow information readout unit 113 reads the timing signal output by at least a part of the pixel collection circuits in the pixel collection circuit array 111, and the global control unit 114 generates a reset signal, and outputs the reset signal to the pixel collection circuit array 111 through the optical flow information reset line.
  • the image information reading unit 112 reads the first electrical signal output by at least a part of the pixel collecting circuits in the pixel collecting circuit array 111, and the global control unit 114 generates an image information collecting signal, and outputs the image information to the pixel collecting circuit through the image information collecting line.
  • Each pixel in the array 111 acquires circuitry.
  • the global control unit 114 includes an image acquisition module 1142 and an optical flow acquisition module 1144.
  • the image acquisition module 1142 is coupled to each pixel acquisition circuit in the pixel collection circuit array 111 by an image information acquisition line for controlling the image information acquisition unit of the pixel array to obtain the current time and the light intensity in an initial stage of image frame acquisition.
  • the optical flow acquiring module 1144 is coupled to each of the pixel collecting circuits of the pixel collecting circuit array 111 through the optical flow information resetting line for generating a reset signal, and resetting all the information related to the optical flow information detection in an initial stage of detecting the optical flow information. Unit/module.
  • the optical flow information readout unit 113 includes an optical flow output line scanner 1132 (i.e., a line selection unit), a time signal scanner 1134 (i.e., a column selection unit), and an optical flow readout controller 1136.
  • the optical flow output line scanner 1132 is coupled to a row of pixel acquisition circuits in the pixel collection circuit array through an optical flow output line selection line, and the optical flow readout controller 1136 instructs the optical flow output line scanner 1132 to collect the pixels line by line.
  • the optical flow output row select line in the circuit array is asserted and the time signal scanner 1134 is instructed to sequentially read the timing signals output by the row of pixel capture circuits.
  • the image information readout unit 112 includes an image output line scanner 1122 (i.e., a line selection unit), a first electric signal scanner 1124 (i.e., a column selection unit), and an image readout controller 1126.
  • the image output line scanner 1122 is coupled to a row of pixel acquisition circuits in the pixel collection circuit array through an image output line selection line; the image readout controller 1126 instructs the image output line scanner 1122 to row-by-row the pixel acquisition circuit array.
  • the image output row select line is asserted and instructs the first electrical signal scanner 1124 to sequentially read the first electrical signal output by the row of pixel capture circuits.
  • a partial pixel acquisition circuit of array 111 is shown in FIG. This part of the pixel acquisition circuit can be divided into rows i-1, i and i+1 (corresponding to the image/optical stream output row selection line), which can be divided into columns j-1, j and j+1 by column ( Corresponding to the data output bus: the first electrical signal line and the time signal line), but is not limited thereto.
  • Each row in the pixel collection circuit array 111 (for example, the i-th row in FIG. 9) is coupled to the image output line scanner 1122 through an image output line selection line, and is further scanned by an optical flow output line selection line and an optical flow output line.
  • the device 1132 is coupled.
  • Each column of the pixel collection circuit array 111 is coupled to the first electrical signal scanner 1124 and the time signal scanner 1134 via a data line transmitting a first electrical signal and a data line transmitting a time signal, respectively.
  • the image output line scanner 1122 and the first electric signal scanner 1124 scan the first electric signal output from the entire pixel collection circuit array 111 under the control of the image readout controller 1126 to acquire image frame information.
  • the optical flow output line scanner 1132 and the time signal scanner 1134 scan the time signal output from the entire pixel collection circuit array 111 under the control of the optical flow readout controller 1136 to acquire optical flow frame information.
  • the image readout controller 1126 first instructs the image output line scanner 1122 to select the first row of pixel acquisition circuits, and the image output line scanner 1122 immediately sets the image output row selection line of the first row to be valid (the remaining row selection lines are invalid) .
  • the first row of pixel acquisition circuits sends the first electrical signals collected by themselves to the first electrical signal scanner 1124 through the row selection output unit.
  • the image readout controller 1126 then controls the first electrical signal scanner 1124 to scan the first electrical signals output by all of the pixel acquisition circuits of the row one by one.
  • the image readout controller 1126 sends a line feed signal to the image output line scanner 1122 to instruct the latter to jump to the second line, and controls the first electrical signal scanner 1124 to scan one by one.
  • the first electrical signal information of all the second row of pixel acquisition circuits is output. And so on, until the first electrical signal information of all the pixel acquisition circuits in the entire array 111 is read.
  • the operation of the optical flow information output mode is the same as that of the optical flow information output controller 1136, the optical flow output line scanner 1132, and the time signal scanner 1134.
  • the optical flow sensor 110 provides a solution for efficiently extracting motion optical flow information, which can realize effective optical flow information extraction for moving objects of various speeds; at the same time, image information can be acquired to obtain A clear, hysteresis-free full-scale grayscale image provides background information for the extracted optical flow information.
  • the optical flow sensor 110 outputs the optical flow frame and image frame information in parallel.
  • Each optical flow frame includes a space of the triggered pixel acquisition circuit and time information of the trigger time.
  • the pixel unit labeled with the spatiotemporal information directly associates the motion of the object in the three-dimensional space, and the time information is analyzed in the pixel array by two-dimensionally.
  • the aspect and slope information of the time slope plane formed by the spatial mapping can effectively extract the optical flow information of the motion.
  • the acquisition of each image frame does not require an exposure time, similar to the global shutter control mechanism, so that a clear and hysteresis-free grayscale image can be obtained even at high speeds, which can be extracted from the optical flow frame.
  • the optical flow information provides background information and is also compatible with existing grayscale image based processing algorithms.
  • the optical flow information timing unit arranged in the pixel acquisition circuit provides time information of the pixel unit triggering time and is not interfered by the optical flow information readout operation, and the non-destructive readout mechanism ensures continuous output of the optical flow frame.
  • Sex that is, the subsequent optical flow frame can obtain the time and space information of all the previously triggered pixel units.
  • This feature ensures that the optical flow sensor can detect both fast moving objects and medium and low speed moving objects that are more common in practical applications. More specifically, since the slow motion requires a long time to form a motion trajectory extension, the pixel unit trigger time interval in the corresponding field of view is long, and the non-destructive readout mechanism ensures that the continuously output optical flow frame can always be acquired.
  • the complete spatio-temporal information of the pixel unit (which corresponds to the field of motion trajectory) available for optical flow analysis.
  • the optical flow sensor 110 is completely independent of the acquisition of the optical flow and the image information, and the optical flow sensor can output the optical flow and the image information completely in parallel without interfering with each other, and the optical flow information and the image information are acquired.
  • the read speed can be adjusted according to different needs.
  • modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be positioned differently than the devices in this example. In one or more devices.
  • the modules in the foregoing examples may be combined into one module or may be further divided into a plurality of sub-modules.
  • modules in the devices of the embodiments can be adaptively changed and placed in one or more devices different from the embodiment.
  • the modules or units or components of the embodiments may be combined into one module or unit or component, and further they may be divided into a plurality of sub-modules or sub-units or sub-components.
  • any combination of the features disclosed in the specification, including the accompanying claims, the abstract and the drawings, and any methods so disclosed, or All processes or units of the device are combined.
  • Each feature disclosed in this specification (including the accompanying claims, the abstract and the drawings) may be replaced by alternative features that provide the same, equivalent or similar purpose.
  • the invention discloses together:
  • the pixel collection circuit of A1, wherein the photodetection unit comprises: an anode-grounded photodiode (PD 1 ); N transistors in series, wherein N ⁇ 2, a source of the first transistor and The photodiode (PD 1 ) is connected to the cathode, the drain of the Nth transistor is connected to the power source (VDD), the gate of each transistor is connected to the drain, and the source of each of the 2nd to Nth transistors is connected to The drain of the first transistor.
  • the photodetection unit comprises: an anode-grounded photodiode (PD 1 ); N transistors in series, wherein N ⁇ 2, a source of the first transistor and The photodiode (PD 1 ) is connected to the cathode, the drain of the Nth transistor is connected to the power source (VDD), the gate of each transistor is connected to the drain, and the source of each of the 2nd to Nth transistors is connected to The drain of the first transistor.
  • the pixel acquisition circuit of A1 wherein the photodetection unit comprises: an anode-grounded photodiode (PD 1 ); N transistors connected in series, wherein N ⁇ 2, a source of the first transistor and The photodiode (PD 1 ) is connected to the cathode, the drain of the Nth transistor is connected to the power supply (VDD), and the source of each of the second to Nth transistors is connected to the drain of the previous transistor, the second each gate and a drain connected to the N-th transistor; a first amplifier (a 1), is connected between the gate of the photo cathode of the diode (PD 1) and a first transistor.
  • a 1 is connected between the gate of the photo cathode of the diode (PD 1) and a first transistor.
  • the filter and amplifier module comprises: a second amplifier (A 2), which is connected to the positive input terminal of the output photodetection unit, which is connected to the negative input of the first pull-down a resistor (R 1 ), a second resistor (R 2 ) is connected between the output end and the input negative pole, and is adapted to amplify the first electrical signal output by the photodetecting unit; and connect the second amplifier (A 2 ) a high pass filter adapted to filter out signal components below the frequency threshold in the amplified first electrical signal to output the second electrical signal.
  • a pixel acquisition circuit wherein the filter amplification module comprises: a first capacitor (C 1 ), a first end of which is connected to an output end of the photodetection unit; and a second amplifier (A 2 )
  • the input positive terminal is connected to a fixed potential
  • the input negative terminal is connected to the second end of the first capacitor (C 1 );
  • the second capacitor (C 2 ), the third resistor (R 3 ) and the first switch (K 1 ) Both are connected in parallel between the input negative terminal and the output terminal of the second amplifier (A 2 ), wherein when the reset signal is valid, the first switch (K 1 ) is closed, and when the reset signal is cancelled, the first switch (K 1 ) is turned off. open.
  • a pixel acquisition circuit comprising: a second amplifier (A 2 ) having an input positive pole connected to an output end of the photodetection unit; a first capacitor (C 1 ), The first end is connected to the input negative terminal of the second amplifier (A 2 ), the second end thereof is connected to the fixed potential; the first switch (K 1 ) is connected to the first end of the first capacitor (C 1 ) Between the output ends of the photodetecting unit; a high pass filter connected to the second amplifier (A 2 ), adapted to filter out signal components below the predetermined frequency threshold in the amplified first electrical signal, to generate the The second electrical signal, wherein when the reset signal is active, the first switch (K 1 ) is closed, and when the reset signal is cancelled, the first switch (K 1 ) is turned off.
  • the pixel acquisition circuit of A4, wherein the filter amplification module comprises: a first capacitor (C 1 ), a first end of which is connected to an output end of the photodetection unit; and a current replicator (P 1 ) a first end connected to the second end of the first capacitor (C 1 ); a second amplifier (A 2 ) having an input positive terminal connected to a fixed potential, the input negative terminal and the second end of the current replicator (P 1 ) Connecting; the second capacitor (C 2 ), the first switch (K 1 ), and the third resistor (R 3 ) are both connected in parallel between the input negative terminal and the output terminal of the second amplifier (A 2 ), wherein when the reset signal is valid When the first switch (K 1 ) is closed, when the reset signal is cancelled, the first switch (K 1 ) is turned off.
  • the threshold comparison module comprises: a second voltage comparator (VC 2), its non-inverting input terminal and the signal line is provided connecting the second threshold value, which is an inverting input The end is connected to the output of the filter amplification module.
  • the threshold comparison module comprises: a first voltage comparator (VC 1), an inverting input terminal and the signal line is connected to the first threshold value, its noninverting input The end is connected to the output end of the filter amplification module; the second voltage comparator (VC 2 ) has a non-inverting input terminal connected to the signal line providing the second threshold, and an inverting input terminal connected to the output of the filter amplification module The first input terminal is coupled to the output end of the first voltage comparator, and the second input end is coupled to the output end of the second voltage comparator, and is adapted to be coupled to the first voltage comparator.
  • the output and the output of the second voltage comparator are ORed or logically operated.
  • the threshold comparison module comprises: a differential pressure comparator (VD 1 ), the first input end of which is connected with a pull-down third capacitor (C 3 ), and the second input end Connected with a pull-down fourth capacitor (C 4 ) adapted to output a difference signal between the first input terminal and the second input terminal; a second switch (K 2 ) disposed at the third capacitor (C 3 ) and the filter amplification Between the outputs of the module; a first buffer (B 1 ) and a third switch (K 3 ) connected in series between the third capacitor (C 3 ) and the fourth capacitor (C 4 ); the first voltage comparator (VC 1 ), an inverting input terminal thereof is connected to a signal line for providing the first threshold value, and a non-inverting input terminal is connected to an output end of the differential pressure comparator; a second voltage comparator (VC 2 ) having a non-inverting input terminal thereof Providing a signal line connection of the second threshold, an inverting
  • the pixel collection circuit of A1, wherein the optical flow information timing unit comprises: a fifth switch (K 5 ), a fifth capacitor (C 5 ), and a third transistor (T 3 ), all connected in parallel a fourth switch (K 4 ) and a fixed potential; a fourth switch (K 4 ), a first end thereof and a fifth switch (K 5 ), a fifth capacitor (C 5 ) and a third transistor (T 3 ) connected in parallel Connected, its second end is connected to another fixed potential, wherein the fifth switch (K 5 ) is closed when the reset signal is active.
  • the optical flow information timing unit comprises: a fifth switch (K 5 ), a fifth capacitor (C 5 ), and a third transistor (T 3 ), all connected in parallel a fourth switch (K 4 ) and a fixed potential; a fourth switch (K 4 ), a first end thereof and a fifth switch (K 5 ), a fifth capacitor (C 5 ) and a third transistor (T 3 ) connected in parallel Connected
  • the pixel collection circuit of A1 wherein the optical flow information timing unit comprises: a fifth switch (K 5 ), a fifth capacitor (C 5 ), and a fourth resistor (R 4 ), all connected in parallel. a fourth switch (K 4 ) and a fixed potential; a fourth switch (K 4 ), a first end thereof and a fifth switch (K 5 ), a fifth capacitor (C 5 ) and a fourth resistor (R 4 ) connected in parallel Connected, its second end is connected to another fixed potential, wherein the fifth switch (K 5 ) is closed when the reset signal is active.
  • optical flow information readout unit comprises: an optical flow output line scanner, through one optical flow output row selection line and one row in the pixel collection circuit array
  • the pixel acquisition circuit is coupled to the time signal scanner;
  • the optical flow readout controller is adapted to instruct the optical flow output line scanner to set the optical flow output line selection line in the pixel acquisition circuit array to be valid line by line, and to indicate the time signal scanning
  • the device sequentially reads the timing signals output by the row of pixel acquisition circuits.
  • the optical flow sensor of B22 wherein the image information readout unit comprises: an image output line scanner coupled to a row of pixel acquisition circuits in the pixel collection circuit array through an image output line selection line a first electrical signal scanner; the readout controller is adapted to instruct the image output line scanner to assert the image output line selection line in the pixel acquisition circuit array line by line, and instruct the first electrical signal scanner to sequentially read The first electrical signal output by the row of pixel acquisition circuits.
  • the global control unit comprises: an optical flow acquisition module, coupled to each pixel acquisition circuit in the pixel collection circuit array by an optical flow information reset line, Generate a reset signal.
  • the optical flow sensor of B24, wherein the global control unit further comprises: an image acquisition module coupled to each pixel acquisition circuit in the pixel collection circuit array by an image information acquisition line, suitable for generating Image information acquisition signal.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Nonlinear Science (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Studio Devices (AREA)

Abstract

本发明公开了一种像素采集电路，至少包括：光电探测单元、光流信息计时触发单元、光流信息计时控制单元、光流信息计时单元和行选输出单元。本发明一并公开了包含上述像素采集电路的光流传感器和光流及图像信息采集***。

Description

像素采集电路、光流传感器和光流及图像信息采集*** 技术领域
本发明涉及图像采集技术领域，尤其涉及像素采集电路、光流传感器和光流及图像信息采集***。
背景技术
随着信息技术的不断发展，计算机视觉及图像信息处理变得越来越重要。其中，光流法可以基于图像信息而确定目标对象的运动情况。光流法可以应用在军事航天、交通监管、信息科学、气象和医学等多个领域中。
光流的概念最初由Gibson于1950年首先提出。现实中的物体可以透过摄像头的感光元件成像，且所成图像中的点与物体上的点一一对应。将三维空间中的目标对象和场景对应于二维图像平面运动时，目标对象在二维图像平面的投影就形成了运动，这种运动以图像平面亮度模式表现出来的流动称为光流。
目前，光流算法通常复杂且计算量巨大。为了进行光流计算，前端图像采集设备需要以高帧率进行图像采集，然后由后续的图像处理设备进行高计算量的光流分析。
现有的光流传感器通过集成有源像素传感器和独立的图像信息处理器实现光流计算。传统的光流计算方法大多通过对时域中的多帧图像进行计算以得到运动物体的运动信息。为获得对高速运动物体的实时光流分析，有源像素传感器需要以非常高的帧率运行，这就增大了光流计算量并提高了对后端信息处理器的硬件要求。
因此，本发明提出了一种新的光流及图像信息的采集方案。
发明内容
本发明提供一种新的光流采集的技术方案，有效地解决了上述中至少一个问题。
根据本发明的一个方面，提供了一种像素采集电路，包括：光电探测单元，适于实时输出表征照射在其上的光信号的强度的第一电信号；光流信息计时触发单元，其第一输入端与光电探测单元的输出端耦接，其第二输入端与光流信息复位线耦接，适于在来自光流信息复位线的复位信号撤销后且检测到第一电信号满足预定触发条件时，输出光流信息计时触发信号；光流信息计时控制单元，其第一输入端与光流信息计时触发单元耦接，其第二输入端与光流信息复位线耦接，适于在来自光流信息复位线的复位信号撤销后且接收到光流信息计时触发信号时输出计时启动信号；光流信息计时单元，其第一输入端与光流信息计时控制单元耦接，其第二输入端与光流信息复位线耦接，适于在来自光流信息复位线的复位信号撤销后且接收到计时启动信号时开始计时，生成计时信号；以及行选输出单元，其第一输入端与光流信息计时单元耦接，其第二输入端与光流输出行选择线耦接，适于接收来自光流输出行选择线的光流行选择信号，并在所述光流行选择信号有效时，缓存并输出本次接收时刻的计时信号。
可选地，在根据本发明的像素采集电路中，还包括图像信息采集单元，其第一输入端与光电探测单元的输出端耦接，其第二输入端与图像信息采集线耦接，适于接收来自图像信息采集线的图像信息采集信号，并采样和缓存本次接收时刻的第一电信号；行选输出单元的第三输入端与图像信息采集单元耦接，其第四输入端与图像输出行选择线耦接，以便接收来自图像输出行选择线的图像行选择信号，并在图像行选择信号有效时，缓存并输出本次接收时刻的第一电信号。
根据本发明的又一个方面，提供了一种光流传感器，包括：像素采集电路阵列，包括多个如上所述的像素采集电路；光流信息读出单元，适于读取像素采集电路阵列中至少一部分像素采集电路所输出的计时信号；全局控制单元，适于生成复位信号，并通过光流信息复位线输出至像素采集电路阵列中的每个像素采集电路。
可选地，在根据本发明的光流传感器中，还包括图像信息读出单元，适于读取像素采集电路阵列中至少一部分像素采集电路所输出的第一电信号；以及全局控制单元还适于生成图像信息采集信号，并通过图像信息采集线输出至像素采集电路阵列中的每个像素采集电路。
根据本发明的又一个方面，提供了一种光流及图像信息采集***，包括：如上所述的光流传感器；光流预处理器，与光流传感器中的光流信息读出单元耦接，适于获取计时信号并生成光流触发像素点的时间映射图像；以及图像预处理器，与光流传感器中的图像信息读出单元耦接，适于获取第一电信号并生成全像素点的灰度图像。
综上，根据本发明的光流传感器提供了一种有效提取运动光流信息的方案，该方案对各种速度的运动物体均可实现有效的光流信息提取；同时，又可以采集图像信息，以获取清晰无滞后的全幅灰度图像为提取的光流信息提供背景信息。根据一种实现方式，光流传感器并行且相互独立地地输出光流帧和图像帧信息。这样，根据本发明的光流及图像信息采集***就可以完全并行地获取光流和图像信息而不会相互干扰，且光流信息和图像信息地获取和读出速度可以根据不同的需求进行调节。
附图说明
为了实现上述以及相关目的，本文结合下面的描述和附图来描述某些说明性方面，这些方面指示了可以实践本文所公开的原理的各种方式，并且所有方面及其等效方面旨在落入所要求保护的主题的范围内。通过结合附图阅读下面的详细描述，本公开的上述以及其它目的、特征和优势将变得更加明显。遍及本公开，相同的附图标记通常指代相同的部件或元素。
图1示出了根据本发明一些实施例的光流及图像信息采集***100的示意图；
图2A、2B、2C和2D分别示出了根据本发明的一些实施例的光流传感器110的工作方式示意图；
图3A-3F示出了本发明实施例的光流传感器110的光流信息提取的示意图；
图4示出了根据本发明一些实施例的像素采集电路400的示意图；
图5A、5B、5C和5D分别示出了根据本发明实施例的光电探测单元410的示意图；
图6A、6B、6C和6D分别示出了根据本发明实施例的滤波放大模块的示意图；
图7A、7B、7C和7D分别示出了根据本发明一个实施例的阈值比较模块的示意图；
图8A和8B分别示出了根据本发明一个实施例的光流信息计时单元的示意图；
图9示出了根据本发明一个实施例的像素采集电路900的示意图；以及
图10示出了根据本发明一些实施例的光流传感器110的示意图。
具体实施方式
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。
图1示出了根据本发明一些实施例的光流及图像信息采集***100的示意图。如图1所示，光流及图像信息采集***100包括光流传感器110、图像预处理器120和光流预处理器130。光流传感器110包括由多个像素采集电路(即，像素单元)组成的像素采集电路阵列111、图像信息读出单元112、光流信息读出单元113和全局控制单元114。阵列111中每个像素采集电路通过总线与图像信息读出单元112、光流信息读出单元113和全局控制单元114耦接。像素采集电路阵列111通常包括多行像素采集电路，每行又包括一个或多个像素采集电路，如图1中示出了3*2共6个像素采集电路，但不限于此。其中，全局控制单元114包括图像获取单元1142和光流获取单元1144，并通过他们分别生成全局的图像信息采集信号和复位信号，来控制整个像素采集电路阵列111。具体地，阵列111中的像素采集电路在全局控制单元114给出的全局控制信号的作用下可以同时采集图像信息和光流信息。其中，图像信息反映视场中阵列111所感知的光照强度大小，光流信息反映视场中运动的时空信息，这两种信息在相互独立的图像信息读出单元112和光流信息读出单元113的控制下分别发送至图像预处理器120和光流预处理器130。
根据一种实现方式，对于光流信息的采集，真实世界的运动反映为传感器视场中感知的光强变化，阵列111可以同步实时监测视场内光强的变化(例如照度变化量和变化速率等)。具体而言，阵列111中每个像素采集电路实时监测其上照射的光强信息。一个像素采集电路在确定光强信息的变化满足预定条件(例如，照度变化量和变化速率都超过各自的阈值)时进入光流触发状态；未满足条件的则维持原状态不变。进入光流触发状态的像素采集电路立刻启动内部的光流信息计时单元，该单元在像素采集电路内部实现计时器的功能，该计时器在像素采集电路光流触发后被启动并生成一个随时间单调变化的计时信号来表征时间信息；随后，光流预处理器130通过光流信息读出单元113读取像素阵列111中所有像素采集电路的计时信号(在被读出时刻的)瞬态幅值可得到 二维的时间映射图像(光流帧)，每个像素点的计时信号告知了在该读出时刻该像素采集电路是否已光流触发(通过检测该信号是否为有效的计时信号)以及触发了多久(通过检测该有效计时信号的幅值大小)。当光流帧获取到足量的光流触发像素单元输出的光流计时信息时，该光流帧在像素阵列平面的映射表现为一个二维的时间坡平面，通过对该时间坡平面的进一步分析可以提取出视场中运动的方向和速度的信息，从而实现光流的计算。可选地，光流信息计时单元是一个斜坡信号发生器，它在一次光流信息检测过程中仅启动一次并在本地生成一个随时间线性变化的斜坡信号。
根据一种实现方式，对于图像信息的采集，全局控制单元114给出(全局)图像信息采集信号，该信号控制阵列111中的所有像素采集电路的图像信息采集单元同时采集并缓存该时刻与光强相关的第一电信号。在随后的读出过程中，通过获取每个像素采集电路缓存的第一电信号即可得到整幅图像的光强信息(图像帧)。
需要说明的是，光流传感器110对光流和图像两种信息的获取是完全独立的，因而可以并行地输出这两种信息。首先，阵列111中的像素采集电路通过独立的图像信息采集单元和光流信息计时单元可以同时采集光流和图像信息；此外，这两种信息在各自的读出单元控制下通过独立的数据总线被读出到各自的预处理器。这样，光流传感器110可以完全并行地输出光流和图像信息而不会相互干扰，而光流信息和图像信息地获取和读出速度可以根据不同的需求进行调节。
根据以上描述，光流传感器110可以并行地输出光流信息和图像信息。光流信息读出单元113按照一定的频率扫描整个像素采集电路阵列111的光流信息计时单元的计时信号以获取连续的光流帧。每个光流帧包含有已触发的像素采集电路触发时刻的时间信息，通过分析该时间信息在像素阵列111二维空间映射形成的时间坡的坡向和坡度信息以得到运动的光流信息。另一方面，图像信息读出单元112按照一定的频率扫描整个像素阵列的图像信息采集单元采集的第一电信号以获取连续的图像帧。每个图像帧包含一幅全像素点的与光强相关的灰度图像。图像帧获取的全像素点的灰度图像作为供光流信息标记的背景图像，而光流帧获取的时间坡平面则可以方便地提取光流信息。
下面结合图2A、2B、2C和2D，对光流传感器110的典型的工作方式进行示例性说明。在图2A、2B、2C和2D中，用斜线填充的方块代表光流帧，用无填充的方块代表图像帧。光流传感器110并行独立地输出光流帧和图像帧，二者的帧率也可以根据不同的应用场景进行单独的设置，如图2A所示；也可以设置为相同帧率，如图2B所示；在某些应用中，可能只需要连续地光流帧输出来实现运动的检测、或者仅需要连续的图像帧输出，如图2C和2D所示。
应注意的是，本发明的光流传感器110的工作方式还可以被配置为上述示例之外的其他等同替换方式，这里不再赘述。
下面将分别对光流帧和图像帧做进一步的说明。
在一次光流信息检测过程中，当运动对象经过光流传感器110的视场时，可以引起阵列111中部分像素采集电路的光照发生变化。像素采集电路在光照变化超出阈值时启动内部的一个计时单元，即光流信息采集单元，最简单地，通过在本地生成一个随时间线性变化的斜坡信号来实现；未检测到光强变化或光强变化不足的像素采集电路则维持其未触发状态，内部光流信息计时单元未启动。与此同时，光流信息读出单元113读取阵列111中每个像素采集电路的光流信息计时单元的计时信号并发送至后端的光流预处理器130，后者据此得到光流触发像素点的时间映射图像。例如，它可以按行扫描整个阵列111，并将每个光流信息采集单元中的斜坡信号的幅值送至外部的光流预处理器130。每个像素点的计时信号(斜坡幅值信号)告知了在该读出时刻该像素点是否已触发(通过检测该信号是否为有效的计时信号)以及已经触发了多久(通过检测该有效计时信号的幅值)。
光流预处理器130进一步对该幅值信号进行放大、模数转化处理以及编码操作，以便从斜坡信号幅值获取相应的时间信息。在一次光流信息检测过程中，阵列111中的像素采集电路仅在首次检测到超过阈值的光强变化时被触发且启动一次光流信息计时单元，扫描所有像素阵列111的光流信息计时单元就可以获取到一个光流帧，它包含了截至该扫描时刻所有已触发的像素单元的空间和触发时刻的时间信息。由于第一个光流帧输出位于一次光流信息检测的初始时刻，因此第一个光流帧获取的有用像素单元的数目较少(稀疏的有效像素单元不足以在光流帧中形成有效的时间坡平面，因此不能够实现运动光流的分析)。由于光流信息的读出并不影响像素采集电路的正常工作，随着光流帧地不断读出，越来越多的像素采集电路因检测到视场运动所引起的光强变化而进入 光流触发状态并启动内部光流信息计时单元。通过不断地输出光流帧可以获取所有累积的截至当前帧读出时刻所有已启动的光流信息计时单元的时间信息。通过不断地累积这些时间信息(光流帧连续扫描)，它们在像素阵列111二维空间上的投影便会逐渐形成一个坡平面(时间坡)，通过分析该时间坡的坡度和坡向信息可以提取出有效的光流信息，从而实现运动方向和速度大小的检测。至此，一次完整的光流信息检测过程结束，下一次光流信息检测可以开始。
根据一种实现方式，针对被检测运动的快慢，一次光流信息检测过程所需的时间也不同。由于快速运动在相对较短的时间内便可积累出足够多的触发像素单元来形成有效的时间坡，因而光流信息检测过程相对较短，即需要较少的光流帧完成一次运动检测。对于慢速运动，它需要足够长的时间才可以形成可被检测的有效运动轨迹，因而需要较长时间完成一次运动检测，即需要多幅光流帧。如上所述，由于光流帧的读出不影响像素单元的正常工作，已触发的像素单元的时间信息会一直有效(直至超出其计时范围)，因此只要检测的时间足够长就可以形成有效的时间坡来进行光流分析。
对于图像帧的采集，全局控制单元114提供一个全局图像信息采集信号，控制阵列111中的所有像素采集电路的图像采集单元采样与光强相关的第一电信号。图像预处理器120通过图像信息读出单元112读取所有像素采集电路在图像信息采集信号有效时刻的与光强相关的第一电信号并可以被配置为对该信号进行放大处理和模数转化处理，其输出的数据可以用于建立一张可供在其上标记已提取的光流信息的完整背景灰度图像。由于像素采集阵列111同步采样与光强相关的第一电信号，这种类似于全局快门的控制机制可以获取清晰且无滞后的背景图像。
为更形象地说明本发明的光流传感器110如何提取光流信息，以下将结合图3A-3F对利用光流传感器110的光流帧输出来提取光流信息的过程做进一步的解释说明。
为了叙述的方便，选取一个简单的场景进行阐述，如图3A所示，一个形状规则质地均匀的立方体A沿X方向以一定速度运动进入光流传感器110的视场范围。如图3A所示，以斜线填充物体A来表示物体A的亮度低于背景的亮度。应当指出的是，该场景的选取只是为了说明的方便，本发明所涉及的光流传感器对质地形状复杂的物体沿着复杂的运动轨迹以不同的速度运动的场景均可做出有效的检测。另外，为简便，光流传感器110的阵列111仅包含8×8个像素采集电路，其中P(x,y)用来表示第x行第y列的像素。在不同时刻，场景中看到的图像如图3B所示。在初始时刻(t＝0)，像素P(3，1)到P(6,2)的视场区域被物体A占据而表示为灰色，其余像素只检测到亮度较高的背景而表示为白色。随着物体A的运动，那些视场区域被物体A运动轨迹所覆盖的像素单元将会进入光流触发状态，因为这些像素单元将会检测到由初始的高亮度背景到低亮度物体的光强变化(其他像素单元因检测的光强未发生变化而未触发)。如上所述，一旦被触发，像素单元内的光流信息计时单元将被启动并报告距光流帧读出时刻的相对时间信息。随着运动的演进以及光流帧的不断输出，后续光流帧会因为检测到更多的触发像素单元而形成更加完整的时间映射图像。在该场景中，为了定量描述的方便，假设物体成像的运动速度为每毫秒一个像素，并认为像素单元被物体成像覆盖一半时进入光流触发状态。那么在t＝1ms时，物体A的成像将会向右推进一列(因为沿X方向运动且速度为每毫秒一个像素)；在t＝2ms时，物体A的成像将会向右推进二列，随着时间的推移，视场中的成像如图3B所示。
下面首先说明光流帧读出操作如何通过像素单元内的光流信息计时单元的计时信号获取光流触发像素单元的触发时间信息。
图3C给出了像素单元内的光流信息计时单元生成的计时信号，需要说明的是，该计时信号仅仅是为了叙述的方便而选取的一种表示方式，实际上还可使用其他形式来实现同样的目的。该计时信号是一个单斜坡信号，它在像素单元触发时刻被启动，随后慢慢衰减，其幅值随时间变化的斜率是已知并可调的。光流信息读出单元不断地扫描整个像素阵列的光流信息计时单元以获取连续的光流帧输出，在单个光流帧内，通过检测该计时信号的幅值可以得到该像素单元触发时刻距离本次光流帧输出的相对时间信息。例如，如果整个斜坡信号的幅值为10个单位，其随时间变化的斜率为1单位/毫秒，那么在某个光流帧内，如果读取到的计时信号的幅值为7(如图3C中A点所示)，那么可以得出像素单元至该光流帧读出时刻已触发了3毫秒；如果读取到的计时信号的幅值为2(如图3C中B点所示)，那么可以得出像素单元至该光流帧读出时刻已触发了8毫秒。因此每个光流帧读取到的像素单元的计时信息实际上表征了每个像素单元至该光流帧读出时已经触发了多长时间，该时间信息是光流预处理器通过获取到的每个像素单元输 出的瞬态计时信号的幅值计算得到的。像素单元触发的时刻越早，被读取的幅值就越低；触发时刻越晚，被读取的幅值就越高，未触发的和触发很久的像素单元被读取的计时信号的幅值均很低，这点可以从图3D看出。该图给出了P(3,3)到P(3,8)六个像素单元生成的计时信号，按照上文描述的场景，P(3,3)到P(3,8)这几个像素单元将依次触发并启动相应的光流信息计时单元，其各自生成的计时信号在时间轴上表现为平移。对于1st时刻的光流帧输出，由于截至该时刻仅有P(3,3)触发，因此它能获取P(3,3)像素单元的触发时间信息为0.5ms；对于6th的光流帧输出，由于这些像素单元均触发，它能获取这些像素单元的触发时刻信息，由于P(3,3)触发最早，P(3,8)触发最晚，其相应计时信号被读取的幅值由低到高，根据计时信号随时间变化的斜率和该读出时刻各个计时信号的瞬态幅值，可以得到P(3,3)在该读出时刻已触发了5.5ms，P(3,8)在该读出时刻已触发了0.5ms。
下面接着说明利用这些提取的时间信息获取二维时间坡平面以及利用该时间坡平面提取出有用的光流信息。
像素单元P(3，3)到P(6，3)在第0.5ms被触发(因为此时，这些像素单元对应的视场区域有一半被运动物体覆盖)，并在该时刻启动各自内部的光流信息采集计时单元。同理，随着物体的继续运动，像素单元P(3，4)到P(6，4)将在1.5ms时刻触发并启动内部的光流信息采集计时单元。此时，由于P(3，3)到P(6，3)的像素单元已经触发了1ms，因此它们的计时单元已经计时了1ms。按此分析，后面的第5至8列的像素单元(第3至6行)将分别在第2.5至5.5ms进入光流触发状态并启动各自的光流信息采集计时单元。另一方面，光流传感器的光流信息读出单元也在不断的扫描像素阵列读取像素单元内部光流信息采集计时单元的计时信号并生成二维的时间信息映射图像。为便于分析，假设光流传感器110每隔1ms输出一个光流帧，那么各个光流帧形成的时间映射图像如图3E所示，图中A至F分别表示第1至6ms读出的光流帧的时间映射图像。对于第1ms输出的光流帧，由于此时仅有P(3,3)到P(6,3)的像素单元触发，因此该帧仅能获取这些像素单元的计时信号为0.5ms，其形成的时间信息映射图像如图3E中A所示。对于第2ms的光流帧，由于又有P(3,4)到P(6,4)的像素单元在1.5ms触发，因此该光流帧能够检测到P(3,3)到P(6,3)和P(3,4)到P(6,4)这些触发像素单元。对于该光流帧，其检测到的P(3,3)到P(6,3)像素单元的计时信号为1.5ms，P(3，4)到P(6，4)像素单元的计时信号为0.5ms，其形成的时间信息映射图像为图3E中B所示。依此，第3至6ms输出的光流帧所形成的时间映射图像如图3E中C至F所示。从该图3E可以看出，后续光流帧能比前面的光流帧检测到更多的激发像素单元，因而在时间映射图像上可以形成更加有效的时间坡平面。对于快速的运动，由于运动轨迹延伸的速度快，因而在相对较短的时间内可累积到足量的像素单元，以形成有效的时间坡平面，也就是说，需要较少的光流帧即可实现运动光流信息的检测；对于慢速的运动，由于运动轨迹延伸的速度慢，因而需要在相对较长的时间以便累积到足量的像素单元，以形成有效的时间坡平面，也就是说，需要较多的光流帧方可完成运动光流信息的检测。因此，只要有足够时间积累起有效的时间坡平面，光流传感器110对于速度不同的运动均可实现有效的检测。此外，由于光流信息检测和图像信息输出完全独立，在光流信息检测的同时，光流传感器110依然可以输出高速无滞后的清晰图像。
根据一种实现方式，利用光流信息读出单元113生成的光流帧所映射的时间坡信息可以方便地进行光流分析。该时间坡的陡峭程度以及坡向表征物体的运动速度大小和运动方向信息。图3F中(1)给出了该物体沿X轴反方向以不同运动速度经过视场时形成的时间坡。由于运动方向相同，三个时间坡的坡向是一致的，但是速度的不同导致时间坡的陡峭程度不同。坡度由高到低的三个时间坡分别对应运动速度为每毫秒0.5个像素、1个像素以及2个像素。由此可见，运动速度越慢的物体形成的时间坡越陡峭；反之，运动速度越快的物体形成的时间坡越平缓。另外，图3F中(2)、(3)分别给出了物体沿着Y轴反方向以及X轴逆时针45°方向运动的时间坡。由图3F中指示的时间坡坡向在XY平面的投影可以方便的获得物体的运动方向。
下面结合图4对像素采集电路阵列111中的像素采集电路进一步说明。图4示出了根据本发明一些实施例的像素采集电路400的示意图。
如图4所示，像素采集电路400至少包括光电探测单元410、光流信息计时触发单元420、光流信息计时控制单元430、光流信息计时单元440和行选输出单元450。根据一种实现方式，像素采集电路还包括图像信息采集单元460。图4所示的光流信息复位线、图像信息采集线由全局控制单元114的光流获取单元1144和图像获取单元1142给出；图像输出行选择线由图像信息读出单元112中输出；光流输出行选择线由光流信 息读出单元113输出；数据输出总线再分别送至图像信息读出单元112和光流信息读出单元113。
光电探测单元410实时输出表征照射在其上的光信号的强度的第一电信号。这里，光电探测单元410例如是多种对数式光电探测器，但不限于此。图5A、5B、5C和5D分别示出了根据本发明实施例的光电探单元410的示意图。
在图5A所示的实施例中，光电探测单元410包括阳极接地的光电二极管PD 1和第一晶体管T 1。其中，第一晶体管T 1的源极与光电二极管PD 1阴极连接，其漏极与栅极连接到电源VDD。在一个应用场景中，光电二极管PD 1接收到光照信号后产生电流I。在此基础上，T 1的源极和栅极之间产生的电压变化与lnI线性相关。换言之，本实施例中光电探测单元410的第一电信号与照射光信号强度成对数关系。
在图5B所示的实施例中，光电探测单元410包括阳极接地的光电二极管PD 1、第一晶体管T 1和第一放大器A 1。其中，第一晶体管T 1的源极与光电二极管PD 1阴极连接，其漏极与电源VDD连接；第一放大器A 1连接在光电二极管PD 1的阴极与第一晶体管T 1的栅极之间。这里，第一放大器A 1可以提高T 1的源极和栅极之间产生电压变化的响应速度。换言之，第一放大器A 1提高像素采集电路检测光强变化的速度。
在图5C所示的实施例中，光电探测单元包括阳极接地的光电二极管PD 1、串联的N个晶体管，其中，N≥2(图5C中N为2，但不限于此)，第1个晶体管的源极与光电二极管PD 1阴极连接，第N个晶体管的漏极连接到电源VDD，每个晶体管的栅极与漏极连接，第2至第N个晶体管中每个晶体管的源极连接到前1个晶体管的漏极。这里，串联的N个晶体管可以提高光电探测器的电流(电压)增益。
图5D示出的光电探测单元410是在图5C示出实施例的基础上，进一步配置有第一放大器A 1。第一放大器A 1连接在光电二极管PD 1的阴极与第1个晶体管的栅极之间。除了上述多个光电探测单元410的实施例之外，本发明还可以采用多种公知的高实时性的光电探测单元，这里不再赘述。
需要说明的是，传统的光电探测技术通常需要进行电容充电，然后进行持续曝光(电容持续放电)，然后根据电容的剩余电量来确定累积的光照强度。根据本发明的光电探测单元410在生成代表光信号强度的第一电信号时，并不需要额外的曝光时间。因此，光电探测单元410可以无延迟的输出第一电信号。
像素单元内光流信息的采集由光流信息计时触发单元420、光流信息计时控制单元430和光流信息计时单元440共同完成。这三个模块均受到全局的来自光流获取单元1144输出的复位信号的控制、且当该信号有效时，这三个与光流信息检测相关的单元均复位。一次光流信息的检测在复位信号撤销后开始，光流信息读出单元113不断地扫描所有像素单元的光流信息计时单元440直至在光流帧中积累到足够多的可供光流分析的有效像素单元。下面给出这三个单元具体的功能和实现。
光流信息计时触发单元420的第一输入端与光电探测单元410的输出端耦接，第二输入端与光流信息复位线耦接。根据本发明的一个实施例，光流信息计时触发单元420包括滤波放大模块422和阈值比较模块424。光流信息计时触发单元420在一次光流信息检测的初始阶段复位，随后负责在光流信息检测过程中，当光电探测单元410输出的第一电信号(代表光强)变化满足预定触发条件时，生成光流信息计时触发信号并送至后端的光流信息计时控制单元430。
为有效提取光流信息，预定触发条件设为：第一电信号变化速度够快且变化持续时间够长。由于第一电信号实时响应照射到光电探测单元410的光强信号，因而如上所述的对第一电信号的要求也就是对光强信号的要求。速度够快且强度够高的光强变化可以被认为是由物体有效运动所引起的。相反，速度变化缓慢或者强度较低的光强变化则可能是由于背景光强的漂移或抖动而没有分析的价值。为实现上述要求，光流信息计时触发单元420包括滤波放大模块422和阈值比较模块424。其中，滤波放大模块422对第一电信号进行预处理操作以生成第二电信号，其中预处理操作包括放大操作和滤波操作中的至少一个；阈值比较模块424判断第二电信号是否大于第一阈值和/或是否小于第二阈值，并在第二电信号大于第一阈值或小于第二阈值时生成光流信息计时触发信号。根据本发明的一种实现方式，预处理操作中，放大操作是为了增加像素采集电路对光强检测的灵敏度，但这不是必须的。滤波操作一般是高通滤波，即只对高频也就是速度足够快的光强变化响应，从而过滤掉那些速度缓慢的光强变化。
滤波放大模块422可以采用多种公知的滤波和放大技术，但不限于此。本发明的图 6A、6B、6C、6D分别示出了根据本发明一些实施例的滤波放大模块422的示意图。
如图6A所示，滤波放大模块422包括第二放大器A 2和高通滤波器，第二放大器A 2输入正极连接光电探测单元410的输出端，输入负极连接有下拉的第一电阻R 1，且其输出端与输入负极之间连接有第二电阻R 2，第二放大器A 2以及第一电阻R 1、第二电阻R 2提供放大功能，增益与电阻R 2与R 1的比值有关。与第二放大器A 2连接的高通滤波器用来滤除经过放大的第一电信号中低于频率阈值的信号成分，以输出第二电信号。
在图6B示出的实施例中，滤波放大模块422包括：第一电容C 1、第二放大器A 2、第二电容C 2、第三电阻R 3和第一开关K 1。其中，第一电容C 1的第一端与光电探测单元410的输出端连接；第二放大器A 2的输入正极连接到固定电位，其输入负极与第一电容C 1的第二端连接；第二电容C 2、第三电阻R 3和第一开关K 1均并联在第二放大器A 2的输入负极和输出端之间，其中，当复位信号有效时，第一开关K 1闭合，当复位信号撤销后，第一开关K 1断开。在图6B所示的实施中，第一电容C 1可以隔离第一电信号中的直流成分。第一电容C 1和第二电容C 2比值与滤波放大模块422的增益成正比。另外，第二电容C 2和第三电阻R 3组成了滤波器，R 3为可调电阻。该滤波器可以滤除第一电信号的交流成分中低于频率阈值的信号成分。这里，频率阈值取决于R 3的阻值。R 3例如可以由***100通过全局控制单元114所发送的指示信号来调节阻值，该指示信号可以是根据用户输入来确定的，但不限于此。
在图6C示出的实施例中，滤波放大模块422包括：第二放大器A 2、第一电容C 1、第一开关K 1和高通滤波器。其中，第二放大器A 2的输入正极与光电探测单元410的输出端连接；第一电容C 1的第一端与第二放大器A 2的输入负极连接，其第二端连接到固定电位(通常该固定电位为地电位，但不限于此)；第一开关K 1连接在第一电容C 1的第一端与光电探测单元410的输出端之间；高通滤波器，与该第二放大器A 2相连，用于滤除经过放大的第一电信号中低于预定频率阈值的信号成分，生成第二电信号。且当复位信号有效时，第一开关K 1闭合，当复位信号撤销后，第一开关K 1断开。光流输出模式开始时，第一开关K 1控制第一电容C 1存储光电探测单元410输出的第一电信号作为参考电平，随后，第二放大器A 2检测并放大实时的第一电信号与该参考电平的差值。
图6A、6B、6C所示的实施例是对第一电信号的相对变化量进行放大与滤波操作。此外，滤波放大模块422还可以通过直接检测第一电信号的变化速率来实现光强变化检测，如图6D所示。滤波放大模块422包括：第一电容C 1、电流复制器P 1、第二放大器A 2、第二电容C 2、第一开关K 1和第三电阻R 3。其中，第一电容C 1的第一端与光电探测单元410的输出端连接；电流复制器P 1的第一端与第一电容C 1的第二端连接；第二放大器A 2的输入正极连接到固定电位，其输入负极与电流复制器P 1的第二端连接；第二电容C 2、第一开关K 1和第三电阻R 3均并联在第二放大器A 2的输入负极和输出端之间，其中，当复位信号有效时，第一开关K 1闭合，当复位信号撤销后，第一开关K 1断开。电流复制器P 1的第一端和第二端分别连接有第一电流源I 1和第二电流源I 2，电流复制器P 1将其第一端钳位于一个固定电平，并将第一电容C 1的电流经第一电流源I 1复制给第二电流源I 2。电流复制器P 1可以根据多种公知的技术实现，在此不在赘述。第一电容C 1一端连接光电探测单元410输出的第一电信号，另一端连接固定电平。根据电容两端的电压电流关系，第一电容C 1的电流与第一电信号的变化率成正比，该电流经电流源I 1复制到电流源I 2并对第二电容C 2进行充放电以生成第二电信号。如此，第二电信号的幅值则由第一电信号变化的速度(充放电电流I 2的大小)和变化持续时间(充放电电流I 2的持续时间)决定。只有持续时间较长且变化速度较快的第一电信号(光强)才可以生成较大的第二电信号。另外，第三电阻R 3可调且作为泄漏电阻，可以抵消当第二电流源I 2非常小时对第二电容C 2的充电效果，从而滤除第一电信号的交流成分中低于频率阈值的信号成分，实现高通滤波的功能。
阈值比较模块424的输入端与滤波放大模块422的输出端耦接。如前所述，阈值比较模块424用于判断第二电信号是否大于第一阈值和/或是否小于第二阈值，并在大于第一阈值或小于第二阈值时产生光流信息采集触发信号。取决于期望的配置，阈值比较模块424可以只判断第二电信号是否大于第一阈值；也可以只判断第二电信号是否小于第二阈值；或者，既判断第二电信号是否大于第一阈值、又判断第二电信号是否小于第二阈值，且第二阈值小于第一阈值。这样，根据本发明的阈值比较模块424可以检测像素采集电路400上光照强度的变化量是否较大(光照强度可以变大或变小)。如图7A、7B、7C和7D，分别示出了根据本发明一些实施例的阈值比较模块424的示意图，应当说明的是，以下对阈值比较模块424的说明仅是示例性的，本发明对此均不作限制。
在图7A所示的实施例中，阈值比较模块424包括第一电压比较器VC 1，其反相输入端与提供第一阈值的信号线连接，其同相输入端连接滤波放大模块422的输出端。这样，本实施例的阈值比较模块424就可以判断第二电信号是否大于第一阈值。
在图7B示出的实施例中，阈值比较模块424包括第二电压比较器VC 2，其同相输入端与提供第二阈值的信号线连接，其反相输入端连接滤波放大模块422的输出端。这样，本实施例的阈值比较模块424就可以判断第二电信号是否小于第二阈值。
在图7C所示的实施例中，阈值比较模块424包括：第一电压比较器VC 1、第二电压比较器VC 2和或门。第一电压比较器VC 1的反相输入端与提供第一阈值的信号线连接，其同相输入端连接滤波放大模块422的输出端；第二电压比较器VC 2的同相输入端与提供第二阈值的信号线连接，其反相输入端连接滤波放大模块422的输出端；或门的第一输入端与第一电压比较器的输出端耦接，第二输入端与第二电压比较器的输出端耦接，对第一电压比较器的输出和第二电压比较器的输出进行或逻辑操作。
在图7D所示的实施例中，阈值比较模块424包括：压差比较器VD 1，其第一输入端连接有下拉的第三电容C 3，第二输入端连接有下拉的第四电容C 4，用于输出第一输入端和第二输入端的差值信号；第二开关K 2，设置在第三电容C 3与滤波放大模块422的输出端之间；依次串联在第三电容C 3和第四电容C 4之间的第一缓存器B 1和第三开关K 3；第一电压比较器VC 1，其反相输入端与提供第一阈值的信号线连接，其同相输入端连接压差比较器的输出端；第二电压比较器VC 2，其同相输入端与提供第二阈值的信号线连接，其反相输入端连接压差比较器的输出端；或门，其第一输入端与第一电压比较器的输出端耦接，其第二输入端与第二电压比较器的输出端耦接，其输出端与第三开关K 3耦接，其中，在压差比较器输出的差值信号大于第一阈值或者小于第二阈值时，该阈值比较模块424输出光流信息计时触发信号，并且按照时间顺序依次断开第二开关K 2、闭合第三开关K 3、断开第三开关K 3和闭合第二开关K 2。这样，阈值比较模块424可以将第二输入端保持的信号更新为当前第一输入端的第二电信号。
光流信息计时控制单元430接收来自前端光流信息计时触发单元420生成的光流信息计时触发信号，并生成后端光流信息计时单元440的计时启动信号，它主要由两部分组成：锁存器和脉冲整形器。根据一种实现方式，锁存器在复位信号有效时被复位、并在首次接收到光流信息计时触发信号时被置位，随后一直保持置位状态直至下一次复位信号有效(下一次光流信息检测开始)。脉冲整形器接收锁存器输出的锁存信号，并在锁存器置位时生成一个窄脉冲信号作为后端光流信息计时单元440的计时启动信号。在根据本发明的一些实施例中，脉冲整形器不是必须的，它也可以被后置到光流信息计时单元440中。
光流信息计时控制单元430采用这种结构可以确保在一次光流信息检测过程中，像素单元的光流信息计时单元440仅在该像素单元首次检测到光强变化满足阈值要求(光流信息计时触发单元420输出计时触发信号)时被启动一次，而不会被反复启动。这是因为光流信息计时控制单元430中的锁存器在首次置位后不会被主动复位(直至下一次复位信号有效时才会被被强制复位)。该特性对于在一次光流信息检测过程中，通过光流帧的不断更新来获取足够多的、不断累积的有效像素单元的时间信息来进行光流分析是必要的。
光流信息计时单元440是像素单元内部的一个计时单元，其在一次光流信息检测的初始阶段被复位，在像素单元光流计时触发时刻被启动并开始计时。当像素单元被光流信息读出单元读出时，通过该光流计时单元440当前的计时信息就可以得到该像素激活时刻的时间信息。
图8A给出了光流信息计时单元440的一个实施例，如图8A所示，光流信息计时单元440包括：第四开关K 4、第五开关K 5、第五电容C 5和第三晶体管T 3，其中，第五开关K 5、第五电容C 5和第三晶体管T 3均并联在第四开关K 4和固定电位(图中标示为地电位但不限于此)之间；第四开关K 4的第一端与并联的第五开关K 5、第五电容C 5和第三晶体管T 3相连，其第二端连接到另一固定电位(图中标示为电源但不限于此)。第三晶体管T 3的栅极电压是一个由外部输入的偏置电压，因而T 3可以等效为一个电流源，而且电流大小可以根据应用场景的不同进行调节。当复位信号有效时，第五开关(K 5)闭合，第五电容C 5的上极板放电到地电位；当接收到前端光流信息计时控制单元430输出的计时启动脉冲时，第四开关K 4闭合一小段时间然后断开，第五电容C 5的上极板被迅速拉升至电源电压(K 4闭合)，然后通过第三晶体管T 3缓慢放电(K 4断开)，由此形成一个由高至低的斜坡信号，如图3C所示，该斜坡信号实际上是一个计时信号， 通过读取该信号不同的幅值信息可以得知像素单元触发时刻的时间信息。该信号的幅值越高，表明像素单元触发越晚(对应的运动发生越晚)，该信号的幅值越低，表明像素单元触发越早(对应的运动发生越早)。由于最低电位只能是地电位(降低到地电位后无法继续下降)，因此光流信息计时单元440所能给出的有效时间信息的范围是有限的，超出该范围的时间信息只能以固定值表示。由上分析可知，斜坡信号的斜率决定了光流信息计时单元440的计时范围，其斜率由第三晶体管T 3的放电电流和第五电容C 5的大小决定，放电电流越小，第五电容C 5越大，该斜坡信号的斜率越小、衰减越慢、有效计时范围越长；反之，斜率越大、放电越快、有效计时范围越短。缓慢变化的斜坡信号对于检测慢速运动是有效的，因为在这种情况下，通常需要等待足够长的时间才能累积到足够的可供光流分析的与运动轨迹相对应的触发像素单元，由于有效计时范围长，早期触发的像素单元在经过足够长时间后仍能给出有效的计时信息，因此可以获取理想的时间坡信息。对于快速运动的检测，由于较短时间内便可积累足够的像素触发单元，因此可以采用斜率较大的斜坡信号。
根据电流源实现的方式不同，光流信息计时单元440也可以通过图8B所示的实施例来实现，在光流信息计时单元440中，第五开关K 5、第五电容C 5和第四电阻R 4均并联在第四开关K 4和固定电位之间；第四开关K 4的第一端与并联的第五开关K 5、第五电容C 5和第四电阻R 4相连，其第二端连接到另一固定电位，当复位信号有效时，第五开关(K 5)闭合。这样，第五电容C 5上存储的电荷通过第四电阻R 4进行放电。光流信息计时单元440启动后，该计时信号是一个呈指数衰减的电压信号，衰减的速度由第四电阻R 4和第五电容C 5的大小决定。一般而言，第四电阻R 4需要的阻值很大，它可以通过处于关断状态的晶体或者反向偏置的二极管来实现。
行选输出单元450的第一输入端与光流信息计时单元440耦接，其第二输入端与光流输出行选择线耦接，对来自光流信息计时单元440的计时信号进行缓存并在光流行选择信号有效时将其送至光流数据输出总线，其中，光流行选择信号来自光流信息读出单元113，光流数据输出总线送至光流信息读出单元113。光流信息读出单元113对光流输出总线的数据进行处理并发送至外部的光流预处理器130。
另外，像素采集电路400中还包括用于采集图像信息的图像信息采集单元460，其第一输入端与光电探测单元410的输出端耦接，其第二输入端与图像信息采集线耦接，接收来自图像信息采集线的图像信息采集信号，并采样和缓存本次接收时刻的第一电信号。由于图像信息采集信号为全局信号，因此所有像素采集电路的图像信息采集单元460同步地采样本地光电探测单元410输出的第一电信号，这样当所有图像信息采集单元460被读出至图像预处理器120后，就可得到一幅完整的灰度图像。
可选地，图像信息采集单元460一般由简单的采样保持电路组成，包括一个控制开关和一个采样电容。其中控制开关的第一输入端连接光电探测单元输出的第一电信号，第二端连接采样电容的第一端，采样电容的第二端连接一固定电平。控制开关的控制信号是全局的图像信息采集信号，当该信号有效时，控制开关闭合，所有像素单元的采样电容采样本像素单元中光电探测单元410输出的第一电信号；当图像信息采集信号变为无效后，控制开关断开，采样电容保持已采样的第一电信号并在随后被图像信息读出单元110读出至图像预处理器120以形成图像帧。
根据本发明的又一实施方式，行选输出单元450的第三输入端与图像信息采集单元460耦接，其第四输入端与图像输出行选择线耦接，以便对来自图像信息采集单元460的输入信号(即，第一电信号)进行缓存、并在图像行选择信号有效时将其送至图像数据输出总线，图像行选择信号来自于图像信息读出单元112的行选择单元，图像数据输出总线送至图像信息读出单元112。图像信息读出单元112对图像输出总线的数据进行处理并发送至外部的图像预处理器120。
综上所述，根据本发明的像素采集电路400可以同时输出图像和光流信息，而且由于两路信息通过独立的读出通道发送至外部的预处理器，因此光流传感器110的图像帧和光流帧的采集是并行且相互独立的。
总的来说，光流信息的采集需要全局的复位信号以及光流信息读出单元的控制。需要说明的是，一次光流信息的检测可能需要多个光流帧。在一次光流信息检测的开始，复位信号有效，此时像素单元中所有与光流信息检测相关的单元(如光流信息计时触发单元、光流信息计时控制单元和光流信息计时单元)均被复位。随后，复位信号撤销，光流信息检测开始，光流信息读出单元扫描整个像素阵列的光流信息计时单元的计时信息从而形成光流帧。而图像帧的采集由全局控制单元给出的图像信息采集信号以及图像 信息读出单元控制。在一个图像帧采集的开始，图像信息采集信号在一小段时间内有效，像素阵列中的图像信息采集单元采样并保持光电探测单元输出的第一电信号，随后图像信息读出单元逐个扫描整个像素阵列中图像信息采集单元所存储的与光强相关的第一电信号，从而得到一个完整的图像帧。
结合关于图4-图8B的描述，通过不同的组合方式，像素采集电路可以被设置为多种形式，此处不再一一列举。图9以其中一种为例，示出了像素采集电路900的一个示例性示意图。
如图9所示，像素采集电路900包括光电探测单元910、图像信息采集单元920、滤波放大模块931、阈值比较模块932、光流信息计时控制单元940、光流信息计时单元950、以及行选输出单元960。其中滤波放大模块931和阈值比较模块932构成光流信息计时触发单元930(图中未示出)。图9中示出的光电探测单元910设置为如图5B所示的光电探测单元，当然，光电探测单元910还可以设置为图5A、5C和5D中任一种，或者其他任何能实现本发明目的的光电探测单元，这里不再赘述。同样地，图9中示出的滤波放大模块931被设置为图6B所示的滤波放大模块，应当指出，滤波放大模块931也可以布置为图6A、6C、6D中任意一种。图9中示出的阈值比较模块932被配置为图7C所示的阈值比较模块，应当指出，阈值比较模块932也可以被配置为图7A、7B和7D中任一种，这里不再赘述。图9中示出的光流信息计时单元950被配置为如图8A中所述的光流信息计时单元，应当指出，光流信息计时单元950还可以被配置为如图8B中所示，这里不再赘述。
在一次光流信息检测开始时，复位信号有效，滤波放大模块931的第一开关K 1闭合，将其输出的第二电信号固定于一个参考电位，该电位与滤波放大模块931的第二放大器A 2的输入正极连接的参考电位一致，一般位于后端阈值比较模块932的第一阈值和第二阈值的中间。这样光流信息计时触发单元930不会给出光流信息计时触发信号。复位信号撤销后，第一开关K 1断开，第一电容C 1采样该时刻的光电探测单元910输出的第一电信号作为后续检测的初始参考电平。第二电信号响应实时的第一电信号与初始参考电平的差值，并在第二电信号超出阈值比较模块932的阈值检测范围时，给出光流信息计时触发信号。在这里，滤波放大模块931中的第二电容C 2和可调电阻R 3组成了高通滤波器，使得第一电信号中的变化缓慢的低频成分被屏蔽。也就是说，虽然第二电信号响应的是实时的第一电信号与初始参考电平的差值，但初始参考电平并非完全由第一开关K 1断开时的第一电信号决定。如果第一电信号在开关K 1断开后有缓慢的(低频)变化，那么初始参考电平也会随之实时地缓慢变化，从而确保了对第一电信号低频成分的有效过滤。阈值比较模块932中的或逻辑单元(即，或门)将前面两个比较单元的输出进行或操作后送至光流信息计时控制单元940，这确保了对不同方向的光强变化(由强到弱以及由弱到强)的检测。
光流信息计时控制单元940接收来自前端光流信息计时触发单元930生成的计时触发信号，并生成后端光流信息计时单元950的计时启动信号。如前文所述，光流信息计时控制单元940主要由两部分组成：锁存器和脉冲整形器。前端的锁存器用于表征在一次光流信息检测过程中，该像素单元是否已被触发，它的复位信号是来自全局的复位信号，置位信号来自本地的光流信息计时触发信号。因此，在一次光流信息检测开始时，阵列中像素单元的该锁存器都被强制复位。在随后的光流信息检测过程中，检测到有效光强变化(由运动造成)的像素单元通过光流信息触发信号置位该锁存器。由于无法自动复位，该置位状态一直保持至下一次复位信号有效(下一次光流信息检测开始)。由此，锁存器被置位时刻即代表本像素单元在一次光流检测过程中首次检测到有效运动的时刻，虽然随后光流信息触发单元仍然响应光强变化并可以再次给出计时触发信号，但由于锁存器已经被置位，因此后期的计时触发信号被屏蔽。脉冲整形器检测锁存器输出的锁存信号，并在锁存器置位时生成一个脉冲信号作为后端光流信息计时单元950的计时启动信号。这确保了在一次光流信息检测过程中，像素采集电路只响应于第一次满足条件的光强变化、并启动一次具有单调增益特性的光流信息计时单元950。阵列中已触发像素单元的空间和光流信息计时单元的时间信息与运动演进的空间和时间信息具有确切的映射关系，从而得到有效的光流模式下的时间映射图像。如光流信息计时单元950所示，该时间信号在复位信号有效时(K 5闭合)被复位至地电位，在启动时(K 4闭合)被迅速拉至电源电压，随后以固定的斜率下降至地电位。通过在不同时刻读取该斜坡信号的幅值信息就可获取该像素单元触发时刻的时间信息。
行选输出单元960被配置成两路带选择开关的缓冲器。两路缓冲器是独立的，分别 用于输出图像信息采集单元920给出的第一电信号和光流信息计时单元950给出的时间信号。当图像输出行选择线有效时，与第一电信号线相连的选择开关闭合，图像信息采集单元920给出的第一电信号经缓存后送至第一电信号线。当光流输出行选择线有效时，与时间信号线相连的选择开关闭合，光流信息计时单元950给出的计时信号经过缓存后送至时间信号线。
图10示出了根据本发明一些实施例的光流传感器110的示意图。如图10所示，光流传感器110至少包括像素采集电路阵列111、光流信息读出单元113和全局控制单元114。当然，为便于光流传感器110对图像信息的采集，光流传感器110还可以包括图像信息读出单元112。
根据一些实现方式，像素采集电路阵列111中包含多个像素采集电路400，关于像素采集电路400的介绍可参见前文所述，此处不再赘述。光流信息读出单元113读取像素采集电路阵列111中至少一部分像素采集电路所输出的计时信号，全局控制单元114生成复位信号，并通过光流信息复位线输出至像素采集电路阵列111中的每个像素采集电路。同样，图像信息读出单元112读取像素采集电路阵列111中至少一部分像素采集电路所输出的第一电信号，全局控制单元114生成图像信息采集信号，并通过图像信息采集线输出至像素采集电路阵列111中的每个像素采集电路。
更具体地，全局控制单元114包括：图像获取模块1142和光流获取模块1144。图像获取模块1142通过图像信息采集线与像素采集电路阵列111中的每个像素采集电路耦接，用于在一个图像帧采集初始阶段控制像素阵列的图像信息采集单元获取当前时刻与光强相关的第一电信号。光流获取模块1144通过光流信息复位线与像素采集电路阵列111中的每个像素采集电路耦接，用于生成复位信号，在一次光流信息检测的初始阶段复位所有与光流信息检测相关的单元/模块。
光流信息读出单元113包括光流输出行扫描器1132(即，行选择单元)、时间信号扫描器1134(即，列选择单元)和光流读出控制器1136。其中，光流输出行扫描器1132通过一条光流输出行选择线与像素采集电路阵列中的一行像素采集电路耦接，光流读出控制器1136指示光流输出行扫描器1132逐行将像素采集电路阵列中的光流输出行选择线置为有效，并指示时间信号扫描器1134依次读取该行像素采集电路所输出的计时信号。
图像信息读出单元112包括图像输出行扫描器1122(即，行选择单元)、第一电信号扫描器1124(即，列选择单元)和图像读出控制器1126。其中，图像输出行扫描器1122通过一条图像输出行选择线与像素采集电路阵列中的一行像素采集电路耦接；图像读出控制器1126指示图像输出行扫描器1122逐行将像素采集电路阵列中的图像输出行选择线置为有效，并指示第一电信号扫描器1124依次读取该行像素采集电路所输出的第一电信号。
图10中示出了阵列111的部分像素采集电路。这部分像素采集电路按行可以分为第i-1、i和i+1行(对应图像/光流输出行选择线)，按列可以分为第j-1、j和j+1列(对应数据输出总线：第一电信号线和时间信号线)，但不限于此。像素采集电路阵列111中每一行(例如图9中第i行)通过一条图像输出行选择线与图像输出行扫描器1122耦接，此外还通过一条光流输出行选择线与光流输出行扫描器1132耦接。像素采集电路阵列111中每一列通过一条传输第一电信号的数据线和一条传输时间信号的数据线分别与第一电信号扫描器1124和时间信号扫描器1134耦接。图像输出行扫描器1122和第一电信号扫描器1124在图像读出控制器1126的控制下扫描整个像素采集电路阵列111输出的第一电信号以获取图像帧信息。光流输出行扫描器1132和时间信号扫描器1134在光流读出控制器1136的控制下扫描整个像素采集电路阵列111输出的时间信号以获取光流帧信息。
根据本发明的实现方式，图像帧输出和光流帧输出的控制原理是相同的，下面以图像帧输出为例进行详细说明。图像读出控制器1126首先指示图像输出行扫描器1122选中第一行像素采集电路，图像输出行扫描器1122随即将第一行的图像输出行选择线置为有效(其余行选择线均无效)。第一行像素采集电路通过行选输出单元将自身采集到的第一电信号发送至第一电信号扫描器1124。然后图像读出控制器1126控制第一电信号扫描器1124按顺序逐个扫描该行所有的像素采集电路输出的第一电信号。当首行最后一个像素采集电路信息输出完毕时，图像读出控制器1126向图像输出行扫描器1122发出换行信号指示后者跳转至第二行、并控制第一电信号扫描器1124逐个扫描输出所有第二行像素采集电路的第一电信号信息。依此类推，直至整个阵列111中的所有像素 采集电路的第一电信号信息读取完毕。光流信息输出模式的操作与此相同，只是通过光流读出控制器1136、光流输出行扫描器1132和时间信号扫描器1134来完成。
综上，本光流传感器110提供了一种有效提取运动光流信息的方案，该方案对各种速度的运动物体均可实现有效的光流信息提取；同时，又可以采集图像信息，以获取清晰无滞后的全幅灰度图像为提取的光流信息提供背景信息。根据一种实现方式，光流传感器110并行地输出光流帧和图像帧信息。其中，每个光流帧包含有已触发的像素采集电路的空间以及触发时刻的时间信息，被标记时空信息的像素单元直接关联三维空间中物体的运动，通过分析该时间信息在像素阵列二维空间映射形成的时间坡平面的坡向和坡度信息可以有效地提取运动的光流信息。而每个图像帧的获取不需要曝光时间，类似于全局快门的控制机制，使得即便在高速运动时也可以获得清晰且无滞后的灰度图像，该图像既可以为从光流帧中提取的光流信息提供背景信息，也可以兼容现有的基于灰度图像的处理算法。
同时，布置在像素采集电路中的光流信息计时单元提供像素单元触发时刻的时间信息、且不受光流信息读出操作的干扰，这种非破坏性读出机制确保了光流帧输出的连续性，即后续光流帧可以获取之前所有已触发像素单元的时间和空间信息。这种特性确保了光流传感器既可以对快速运动的物体进行检测，也可以对实际应用中更加常见的中低速运动的物体进行检测。更具体地说，因为慢速运动需要较长的时间形成运动轨迹延伸，对应视场中的像素单元触发时间间隔较长，非破坏性读出机制确保了连续输出的光流帧中总可以获取完整的可供光流分析的(与视场运动轨迹相对应的)触发像素单元的时空信息。
此外，光流传感器110对光流和图像两种信息的获取是完全独立的，光流传感器可以完全并行地输出光流和图像信息而不会相互干扰，且光流信息和图像信息地获取和读出速度可以根据不同的需求进行调节。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下被实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
类似地，应当理解，为了精简本公开并帮助理解各个发明方面中的一个或多个，在上面对本发明的示例性实施例的描述中，本发明的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释成反映如下意图：即所要求保护的本发明要求比在每个权利要求中所明确记载的特征更多特征。更确切地说，如下面的权利要求书所反映的那样，发明方面在于少于前面公开的单个实施例的所有特征。因此，遵循具体实施方式的权利要求书由此明确地并入该具体实施方式，其中每个权利要求本身都作为本发明的单独实施例。
本领域那些技术人员应当理解在本文所公开的示例中的设备的模块或单元或组件可以布置在如该实施例中所描述的设备中，或者可替换地可以定位在与该示例中的设备不同的一个或多个设备中。前述示例中的模块可以组合为一个模块或者此外可以分成多个子模块。
本领域那些技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且把它们设置在与该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或组件组合成一个模块或单元或组件，以及此外可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过程或者单元中的至少一些是相互排斥之外，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。
本发明一并公开了：
A8、如A1所述的像素采集电路，其中，所述光电探测单元包括：阳极接地的光电二极管(PD 1)；串联的N个晶体管，其中，N≥2，第1个晶体管的源极与光电二极管(PD 1)阴极连接，第N个晶体管的漏极连接到电源(VDD)，每个晶体管的栅极与漏极连接，第2至第N个晶体管中每个晶体管的源极连接到前1个晶体管的漏极。A9、如A1所述的像素采集电路，其中，所述光电探测单元包括：阳极接地的光电二极管(PD 1)；串联的N个晶体管，其中，N≥2，第1个晶体管的源极与光电二极管(PD 1)阴极连接，第N个晶体管的漏极连接到电源(VDD)，第2个至第N个晶体管中每个 晶体管的源极连接到前1个晶体管的漏极，第2个至第N个晶体管中每个的栅极与漏极连接；第一放大器(A 1)，连接在光电二极管(PD 1)的阴极与第1个晶体管的栅极之间。A10、如A4所述的像素采集电路，其中，所述滤波放大模块包括：第二放大器(A 2)，其输入正极连接所述光电探测单元的输出端，其输入负极连接有下拉的第一电阻(R 1)，其输出端与输入负极之间连接有第二电阻(R 2)，适于对所述光电探测单元输出的第一电信号进行放大处理；连接该第二放大器(A 2)的高通滤波器，适于滤除经过放大的第一电信号中低于频率阈值的信号成分，以输出所述第二电信号。A11、如A4所述的像素采集电路，其中，所述滤波放大模块包括：第一电容(C 1)，其第一端与所述光电探测单元的输出端连接；第二放大器(A 2)，其输入正极连接到固定电位，其输入负极与第一电容(C 1)的第二端连接；第二电容(C 2)、第三电阻(R 3)和第一开关(K 1)，均并联在第二放大器(A 2)的输入负极和输出端之间，其中，当复位信号有效时，第一开关(K 1)闭合，当复位信号撤销后，第一开关(K 1)断开。A12、如A11所述的像素采集电路，其中，所述第三电阻为可调电阻。A13、如A4所述的像素采集电路，其中，所述滤波放大模块包括：第二放大器(A 2)，其输入正极与所述光电探测单元的输出端连接；第一电容(C 1)，其第一端与第二放大器(A 2)的输入负极连接，其第二端连接到固定电位；第一开关(K 1)，其连接在第一电容(C 1)的第一端与所述光电探测单元的输出端之间；高通滤波器，与该第二放大器(A 2)相连，适于滤除经过放大的第一电信号中低于预定频率阈值的信号成分，生成所述第二电信号，其中，当复位信号有效时，第一开关(K 1)闭合，当复位信号撤销后，第一开关(K 1)断开。A14、如A4所述的像素采集电路，其中，所述滤波放大模块包括：第一电容(C 1)，其第一端与所述光电探测单元的输出端连接；电流复制器(P 1)，其第一端与第一电容(C 1)的第二端连接；第二放大器(A 2)，其输入正极连接到固定电位，其输入负极与电流复制器(P 1)的第二端连接；第二电容(C 2)、第一开关(K 1)和第三电阻(R 3)均并联在第二放大器(A 2)的输入负极和输出端之间，其中，当复位信号有效时，第一开关(K 1)闭合，当复位信号撤销后，第一开关(K 1)断开。A15、如A4所述的像素采集电路，其中，所述阈值比较模块包括：第一电压比较器(VC 1)，其反相输入端与提供所述第一阈值的信号线连接，其同相输入端连接所述滤波放大模块的输出端。A16、如A4所述的像素采集电路，其中，所述阈值比较模块包括：第二电压比较器(VC 2)，其同相输入端与提供所述第二阈值的信号线连接，其反相输入端连接所述滤波放大模块的输出端。A17、如A4所述的像素采集电路，其中，所述阈值比较模块包括：第一电压比较器(VC 1)，其反相输入端与提供所述第一阈值的信号线连接，其同相输入端连接所述滤波放大模块的输出端；第二电压比较器(VC 2)，其同相输入端与提供所述第二阈值的信号线连接，其反相输入端连接所述滤波放大模块的输出端；或门，其第一输入端与第一电压比较器的输出端耦接，其第二输入端与第二电压比较器的输出端耦接，适于对所述第一电压比较器的输出和所述第二电压比较器的输出进行或逻辑操作。A18、如A4所述的像素采集电路，其中，所述阈值比较模块包括：压差比较器(VD 1)，其第一输入端连接有下拉的第三电容(C 3)，第二输入端连接有下拉的第四电容(C 4)，适于输出第一输入端和第二输入端的差值信号；第二开关(K 2)，设置在第三电容(C 3)与所述滤波放大模块的输出端之间；依次串联在第三电容(C 3)和第四电容(C 4)之间的第一缓存器(B 1)和第三开关(K 3)；第一电压比较器(VC 1)，其反相输入端与提供所述第一阈值的信号线连接，其同相输入端连接压差比较器的输出端；第二电压比较器(VC 2)，其同相输入端与提供所述第二阈值的信号线连接，其反相输入端连接压差比较器的输出端；或门，其第一输入端与第一电压比较器的输出端耦接，其第二输入端与第二电压比较器的输出端耦接，其输出端与所述第三开关(K 3)耦接，其中，在所述压差比较器输出的差值信号大于第一阈值或者小于第二阈值时，该阈值比较模块输出所述光流信息计时触发信号，并且按照时间顺序依次断开第二开关(K 2)、闭合第三开关(K 3)、断开第三开关(K 3)和闭合第二开关(K 2)。A19、如A1所述的像素采集电路，其中，所述光流信息计时单元包括：第五开关(K 5)、第五电容(C 5)和第三晶体管(T 3)，均并联在第四开关(K 4)和固定电位之间；第四开关(K 4)，其第一端与并联的第五开关(K 5)、第五电容(C 5)和第三晶体管(T 3)相连，其第二端连接到另一固定电位，其中，当复位信号有效时，第五开关(K 5)闭合。A20、如A1所述的像素采集电路，其中，所述光流信息计时单元包括：第五开关(K 5)、第五电容(C 5)和第四电阻(R 4)，均并联在第四开关(K 4)和固定电位之间；第四开关(K 4)，其第一端与并联的第五开关(K 5)、第五电容(C 5)和第四电阻(R 4)相连，其第二端连接到另一固定电位，其中，当复位信号有效时，第五开关(K 5)闭合。
B23、如B21或22所述的光流传感器，其中，所述光流信息读出单元包括：光流输出行扫描器，通过一条光流输出行选择线与所述像素采集电路阵列中的一行像素采集电路耦接；时间信号扫描器；光流读出控制器，适于指示光流输出行扫描器逐行将像素采集电路阵列中的光流输出行选择线置为有效，并指示时间信号扫描器依次读取该行像素采集电路所输出的计时信号。B24、如B22所述的光流传感器，其中，所述图像信息读出单元包括：图像输出行扫描器，通过一条图像输出行选择线与所述像素采集电路阵列中的一行像素采集电路耦接；第一电信号扫描器；像读出控制器，适于指示图像输出行扫描器逐行将像素采集电路阵列中的图像输出行选择线置为有效，并指示第一电信号扫描器依次读取该行像素采集电路所输出的第一电信号。B25、如B24所述的光流传感器，其中，所述全局控制单元包括：光流获取模块，通过光流信息复位线与所述像素采集电路阵列中的每个像素采集电路耦接，适于生成复位信号。B26、如B24所述的光流传感器，其中，所述全局控制单元还包括：图像获取模块，通过图像信息采集线与所述像素采集电路阵列中的每个像素采集电路耦接，适于生成图像信息采集信号。
此外，本领域的技术人员能够理解，尽管在此所述的一些实施例包括其它实施例中所包括的某些特征而不是其它特征，但是不同实施例的特征的组合意味着处于本发明的范围之内并且形成不同的实施例。例如，在下面的权利要求书中，所要求保护的实施例的任意之一都可以以任意的组合方式来使用。
此外，所述实施例中的一些在此被描述成可以由计算机***的处理器或者由执行所述功能的其它装置实施的方法或方法元素的组合。因此，具有用于实施所述方法或方法元素的必要指令的处理器形成用于实施该方法或方法元素的装置。此外，装置实施例的在此所述的元素是如下装置的例子：该装置用于实施由为了实施该发明的目的的元素所执行的功能。
如在此所使用的那样，除非另行规定，使用序数词“第一”、“第二”、“第三”等等来描述普通对象仅仅表示涉及类似对象的不同实例，并且并不意图暗示这样被描述的对象必须具有时间上、空间上、排序方面或者以任意其它方式的给定顺序。
尽管根据有限数量的实施例描述了本发明，但是受益于上面的描述，本技术领域内的技术人员明白，在由此描述的本发明的范围内，可以设想其它实施例。此外，应当注意，本说明书中使用的语言主要是为了可读性和教导的目的而选择的，而不是为了解释或者限定本发明的主题而选择的。因此，在不偏离所附权利要求书的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。对于本发明的范围，对本发明所做的公开是说明性的，而非限制性的，本发明的范围由所附权利要求书限定。

Claims (10)

  1. 一种像素采集电路，包括：
    光电探测单元，适于实时输出表征照射在其上的光信号的强度的第一电信号；
    光流信息计时触发单元，其第一输入端与所述光电探测单元的输出端耦接，其第二输入端与光流信息复位线耦接，适于在来自光流信息复位线的复位信号撤销后且检测到第一电信号满足预定触发条件时，输出光流信息计时触发信号；
    光流信息计时控制单元，其第一输入端与光流信息计时触发单元耦接，其第二输入端与光流信息复位线耦接，适于在来自光流信息复位线的复位信号撤销后且接收到光流信息计时触发信号时输出计时启动信号；
    光流信息计时单元，其第一输入端与光流信息计时控制单元耦接，其第二输入端与光流信息复位线耦接，适于在来自光流信息复位线的复位信号撤销后且接收到计时启动信号时开始计时，生成计时信号；以及
    行选输出单元，其第一输入端与光流信息计时单元耦接，其第二输入端与光流输出行选择线耦接，适于接收来自光流输出行选择线的光流行选择信号，并在所述光流行选择信号有效时，缓存并输出本次接收时刻的计时信号。
  2. 如权利要求1所述的像素采集电路，还包括：
    图像信息采集单元，其第一输入端与所述光电探测单元的输出端耦接，其第二输入端与图像信息采集线耦接，适于接收来自图像信息采集线的图像信息采集信号，并采样和缓存本次接收时刻的第一电信号；
    所述行选输出单元的第三输入端与图像信息采集单元耦接，其第四输入端与图像输出行选择线耦接，以便接收来自图像输出行选择线的图像行选择信号，并在所述图像行选择信号有效时，缓存并输出本次接收时刻的第一电信号。
  3. 如权利要求2所述的像素采集电路，其中，
    所述图像信息采集单元适于在接收到的图像信息采集信号有效时，采样本次接收时刻的第一电信号，以及，在接收到的图像信息采集信号无效时，缓存已采样的第一电信号。
  4. 如权利要求1-3中任一项所述的像素采集电路，其中，所述光流信息计时触发单元包括：
    滤波放大模块，适于对所述第一电信号进行预处理操作以生成第二电信号，其中所述预处理操作包括放大操作和滤波操作中的至少一个；以及
    阈值比较模块，适于判断第二电信号是否大于第一阈值和/或是否小于第二阈值，并在第二电信号大于第一阈值或小于第二阈值时生成光流信息计时触发信号。
  5. 如权利要求1-4中任一项所述的像素采集电路，其中，所述光流信息计 时控制单元包括：
    锁存器，适于在复位信号有效时被复位，并在首次接收到光流信息计时触发信号时被置位；以及
    脉冲整形器，适于在锁存器被置位时生成计时启动信号。
  6. 如权利要求1所述的像素采集电路，其中，所述光电探测单元包括：
    阳极接地的光电二极管(PD 1)；
    第一晶体管(T 1)，其源极与光电二极管(PD 1)阴极连接，其漏极与栅极连接到电源(VDD)。
  7. 如权利要求1所述的像素采集电路，其中，所述光电探测单元包括：
    阳极接地的光电二极管(PD 1)；
    第一晶体管(T 1)，其源极与光电二极管(PD 1)阴极连接，其漏极与电源(VDD)连接；
    第一放大器(A 1)，连接在光电二极管(PD 1)的阴极与第一晶体管(T 1)的栅极之间。
  8. 一种光流传感器，包括：
    像素采集电路阵列，包括多个如权利要求1-20中任一项所述的像素采集电路；
    光流信息读出单元，适于读取像素采集电路阵列中至少一部分像素采集电路所输出的计时信号；
    全局控制单元，适于生成复位信号，并通过光流信息复位线输出至像素采集电路阵列中的每个像素采集电路。
  9. 如权利要求8所述的光流传感器，还包括：
    图像信息读出单元，适于读取像素采集电路阵列中至少一部分像素采集电路所输出的第一电信号；以及
    所述全局控制单元还适于生成图像信息采集信号，并通过图像信息采集线输出至像素采集电路阵列中的每个像素采集电路。
  10. 一种光流及图像信息采集***，包括：
    如权利要求9所述的光流传感器；
    光流预处理器，与所述光流传感器中的光流信息读出单元耦接，适于获取计时信号并生成光流触发像素点的时间映射图像；以及
    图像预处理器，与所述光流传感器中的图像信息读出单元耦接，适于获取第一电信号并生成全像素点的灰度图像。
PCT/CN2018/088261 2018-03-23 2018-05-24 像素采集电路、光流传感器和光流及图像信息采集*** WO2019178939A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP18910634.7A EP3758365A4 (en) 2018-03-23 2018-05-24 PIXEL ACQUISITION CIRCUIT, OPTICAL FLOW SENSOR AND IMAGE AND OPTICAL FLOW INFORMATION ACQUISITION SYSTEM
SG11202009370QA SG11202009370QA (en) 2018-03-23 2018-05-24 Pixel collection circuit, optical flowsensor, and optical flow and image information collection system
JP2021500323A JP6992238B2 (ja) 2018-03-23 2018-05-24 画素収集回路、オプティカルフローセンサー、オプティカルフロー及び画像情報収集システム
US17/024,929 US12022211B1 (en) 2018-03-23 2020-09-18 Pixel collection circuit, optical flow sensor, and optical flow and image information collection system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810247041.9A CN108449557B (zh) 2018-03-23 2018-03-23 像素采集电路、光流传感器和光流及图像信息采集***
CN201810247041.9 2018-03-23

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/024,929 Continuation US12022211B1 (en) 2018-03-23 2020-09-18 Pixel collection circuit, optical flow sensor, and optical flow and image information collection system

Publications (1)

Publication Number Publication Date
WO2019178939A1 true WO2019178939A1 (zh) 2019-09-26

Family

ID=63196948

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/088261 WO2019178939A1 (zh) 2018-03-23 2018-05-24 像素采集电路、光流传感器和光流及图像信息采集***

Country Status (6)

Country Link
US (1) US12022211B1 (zh)
EP (1) EP3758365A4 (zh)
JP (1) JP6992238B2 (zh)
CN (1) CN108449557B (zh)
SG (1) SG11202009370QA (zh)
WO (1) WO2019178939A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023501237A (ja) * 2019-11-29 2023-01-18 オムニビジョン センサー ソリューション （シャンハイ） カンパニー リミテッド ピクセル取得回路及び画像センサ

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109151348B (zh) * 2018-09-28 2021-01-08 维沃移动通信有限公司 一种图像处理方法、电子设备及计算机可读存储介质
CN109842767B (zh) * 2019-01-09 2020-07-14 上海芯仑光电科技有限公司 一种防闪光电路组件及图像传感器
CN109842768B (zh) * 2019-01-29 2020-05-15 上海芯仑光电科技有限公司 一种像素采集电路及图像传感器
CN111510650B (zh) * 2020-04-26 2021-06-04 豪威芯仑传感器(上海)有限公司 一种图像传感器
CN112636758B (zh) * 2020-12-22 2022-05-06 电子科技大学 一种用于快照式读出电路中的采样保持电路
CN113365004B (zh) * 2021-06-07 2022-03-29 豪威芯仑传感器(上海)有限公司 一种像素采集电路及图像传感器

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104205169A (zh) * 2011-12-21 2014-12-10 皮埃尔和玛利居里大学(巴黎第六大学) 基于异步光传感器估计光流的方法
CN106340514A (zh) * 2016-11-01 2017-01-18 京东方科技集团股份有限公司 一种有源像素传感器、驱动电路及驱动方法
US20180035067A1 (en) * 2015-07-01 2018-02-01 Brian M. Tyrrell Method and Apparatus for On-Chip Per-Pixel Pseudo-Random Time Coded Exposure

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8929601B2 (en) 2007-12-05 2015-01-06 John Caulfield Imaging detecting with automated sensing of an object or characteristic of that object
CN102464221B (zh) * 2010-11-17 2016-03-16 集成彩票***股份有限公司 分配方法和分配***
IL212289A (en) * 2011-04-13 2016-08-31 Semi-Conductor Devices - An Elbit Systems - Rafael Partnership Circuit and method for reading image signals
EP3047647B1 (en) 2013-09-16 2020-05-27 Prophesee Dynamic, single photodiode pixel circuit and operating method thereof
CN104978728B (zh) * 2014-04-08 2017-11-14 南京理工大学 一种光流法的图像匹配***
US9986179B2 (en) 2014-09-30 2018-05-29 Qualcomm Incorporated Sensor architecture using frame-based and event-based hybrid scheme
KR102347249B1 (ko) * 2014-10-21 2022-01-04 삼성전자주식회사 외부 물체의 움직임과 연관된 이벤트에 응답하여 화면을 디스플레이하는 장치 및 방법
CN104535077A (zh) * 2014-12-29 2015-04-22 上海交通大学 一种基于智能移动终端设备的行人步长估计方法
CN106093917A (zh) * 2016-06-01 2016-11-09 中国科学院合肥物质科学研究院 基于fpga技术的高精度星载激光高度计地面定标***
CN106162000B (zh) * 2016-07-08 2019-03-15 上海芯仑光电科技有限公司 像素采集电路、图像传感器及图像采集***
EP3267678B1 (en) * 2016-07-08 2022-10-05 OmniVision Sensor Solution (Shanghai) Co., Ltd Pixel acquisition circuit, image sensor and image acquisition system
CN108632546B (zh) * 2017-03-17 2021-06-04 豪威芯仑传感器(上海)有限公司 像素采集电路、光流传感器及图像采集***
CN107396009B (zh) * 2017-08-25 2020-06-09 电子科技大学 脉冲频率调制型图像传感器电路及其处理方法
CN108827461B (zh) * 2018-04-25 2019-05-03 上海芯仑光电科技有限公司 像素采集电路及光流传感器
CN109842767B (zh) * 2019-01-09 2020-07-14 上海芯仑光电科技有限公司 一种防闪光电路组件及图像传感器

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104205169A (zh) * 2011-12-21 2014-12-10 皮埃尔和玛利居里大学(巴黎第六大学) 基于异步光传感器估计光流的方法
US20180035067A1 (en) * 2015-07-01 2018-02-01 Brian M. Tyrrell Method and Apparatus for On-Chip Per-Pixel Pseudo-Random Time Coded Exposure
CN106340514A (zh) * 2016-11-01 2017-01-18 京东方科技集团股份有限公司 一种有源像素传感器、驱动电路及驱动方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3758365A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023501237A (ja) * 2019-11-29 2023-01-18 オムニビジョン センサー ソリューション （シャンハイ） カンパニー リミテッド ピクセル取得回路及び画像センサ
JP7428307B2 (ja) 2019-11-29 2024-02-06 オムニビジョン センサー ソリューション （シャンハイ） カンパニー リミテッド ピクセル取得回路及び画像センサ

Also Published As

Publication number Publication date
CN108449557B (zh) 2019-02-15
US12022211B1 (en) 2024-06-25
US20240236516A1 (en) 2024-07-11
EP3758365A1 (en) 2020-12-30
JP2021516934A (ja) 2021-07-08
SG11202009370QA (en) 2020-10-29
EP3758365A4 (en) 2022-01-05
CN108449557A (zh) 2018-08-24
JP6992238B2 (ja) 2022-01-13

Similar Documents

Publication Publication Date Title
WO2019178939A1 (zh) 像素采集电路、光流传感器和光流及图像信息采集***
US12010447B2 (en) Dynamic vision sensor architecture
CN108632546B (zh) 像素采集电路、光流传感器及图像采集***
US10909824B2 (en) System and method for pulsed light pattern capturing using a dynamic vision sensor
KR101613133B1 (ko) 3차원 영상 처리 장치 및 그 방법
WO2019205217A1 (zh) 像素采集电路及光流传感器
US10582178B2 (en) Systems and methods for active depth imager with background subtract
CN113556528A (zh) 用于通过深度传感相机使用光发射在弱光条件下捕获视频影像的方法和***
JP7297850B2 (ja) 動画のためのｌｅｄフリッカ軽減
US11233966B1 (en) Breakdown voltage monitoring for avalanche diodes
US11626446B2 (en) Pixel circuit and method of operating the same in an always-on mode
TW201923308A (zh) 具有雷射測距（ｌｒｆ）能力之模ｒｏｉｃ像素
JP3822056B2 (ja) 複数のイメージキャプチャモードを有する画像処理システム
EP3376754B1 (en) Pixel acquisition circuit, optical flow sensor, and image acquisition system
Brajovic et al. Temporal photoreception for adaptive dynamic range image sensing and encoding
EP3358367B1 (en) Event-triggered imaging pixels
JP2022522952A (ja) 飛行時間型装置および３ｄ光学検出器
WO2023093986A1 (en) A monolithic image sensor, a camera module, an electronic device and a method for operating a camera module
WO2024008305A1 (en) An image sensor system, a camera module, an electronic device and a method for operating a camera module for detecting events using infrared
JP2001145124A (ja) ３次元画像検出装置
WO2013030561A1 (en) Pixel circuit

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18910634

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021500323

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018910634

Country of ref document: EP

Effective date: 20200923