CN111343398B - CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology - Google Patents
CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology Download PDFInfo
- Publication number
- CN111343398B CN111343398B CN202010272251.0A CN202010272251A CN111343398B CN 111343398 B CN111343398 B CN 111343398B CN 202010272251 A CN202010272251 A CN 202010272251A CN 111343398 B CN111343398 B CN 111343398B
- Authority
- CN
- China
- Prior art keywords
- circuit
- sensing
- light intensity
- data
- sampling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 34
- 238000005516 engineering process Methods 0.000 title claims abstract description 19
- 230000000295 complement effect Effects 0.000 title abstract description 4
- 229910044991 metal oxide Inorganic materials 0.000 title abstract description 4
- 150000004706 metal oxides Chemical class 0.000 title abstract description 4
- 239000004065 semiconductor Substances 0.000 title abstract description 4
- 238000005070 sampling Methods 0.000 claims abstract description 51
- 238000001514 detection method Methods 0.000 claims abstract description 26
- 238000004364 calculation method Methods 0.000 claims abstract description 17
- 230000000875 corresponding effect Effects 0.000 claims abstract description 11
- 230000002596 correlated effect Effects 0.000 claims abstract description 7
- 230000008859 change Effects 0.000 claims description 32
- 239000003990 capacitor Substances 0.000 claims description 21
- 230000010354 integration Effects 0.000 claims description 16
- 238000000034 method Methods 0.000 claims description 8
- 230000003071 parasitic effect Effects 0.000 claims description 7
- 238000005286 illumination Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 230000001276 controlling effect Effects 0.000 claims description 4
- 230000003287 optical effect Effects 0.000 claims description 4
- 239000012072 active phase Substances 0.000 claims description 3
- 230000003321 amplification Effects 0.000 claims description 2
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 36
- 238000012545 processing Methods 0.000 description 4
- 238000009825 accumulation Methods 0.000 description 3
- 238000007599 discharging Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 210000003792 cranial nerve Anatomy 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
- H04N25/75—Circuitry for providing, modifying or processing image signals from the pixel array
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
The invention discloses a CMOS (complementary metal oxide semiconductor) sensing and calculating integrated circuit structure based on a dynamic visual sensing technology. The invention mainly comprises a sensing circuit module based on an AER mode and a storage and calculation integrated circuit module, wherein the sensing circuit module based on the AER mode comprises: the dynamic visual sensing active phase element circuit is used for sensing an external light intensity signal and converting the external light intensity signal into a corresponding electric signal; the correlated double sampling circuit is used for carrying out double sampling on the electric signal; the differential comparator is used for making difference on the subsampled data and comparing the subsampled data with a reference voltage; the light intensity detection circuit is used for sensing whether the dynamic light intensity changes or not; the logic decision circuit is used for combining the data of the differential comparator and the light intensity detection circuit to carry out validity decision and outputting a decision result. The purposes of saving storage area, reducing calculation power consumption and improving calculation speed are achieved.
Description
Technical Field
The invention belongs to the technical field of image sensing technology and integrated circuits, and particularly relates to a CMOS (complementary metal oxide semiconductor) sensing and storage integrated circuit structure based on a dynamic visual sensing technology.
Background
The development of big data technology and the rise of deep learning technology wave taking a neural network as a core are taken as representatives, and higher requirements are put forward on the calculation power of the traditional mainstream hardware platform. Since the deep learning algorithm needs to process streaming data in the calculation, when a hardware platform based on the von neumann computing architecture is used for processing related tasks, a large amount of data can flow between the computing unit and the storage unit. The latter read-write speed is much slower than the former computing speed, the operation process of accessing the memory occupies most of the overall energy consumption and delay, and the processing speed of the data is limited, which is called as von neumann bottleneck or memory bottleneck. The memory bottleneck causes the computing system to have the defects of high power consumption, low speed and the like. In a calculation task centered on a large data volume, the problem of memory separation is more prominent.
Under the background, a calculation and storage integrated framework similar to a cranial nerve structure is gradually developed and is used as a model similar to a human brain, a data storage unit and a calculation unit are fused into a whole, so that the data transportation is reduced, and the calculation parallelism and the energy efficiency are greatly improved. It is certain that the integrated computing and storing chip and its specific application will be quickly dropped to the ground driven by the gradual mature technology and the simultaneous application requirement.
The current research on the storage and computation integrated circuit focuses only on two aspects of storage and computation, and the research on the combination of the storage and computation integrated circuit and other applications is little and less, particularly on a sensing and computation integrated circuit combined with a sensing circuit.
Disclosure of Invention
In order to solve the problems in the prior art, research on the existing CMOS active-source pixel circuit finds that the CMOS active-source pixels exist in the form of a large-scale pixel array, that is, the pixel array, the storage unit and the operation unit are independent circuit modules. The von Neumann structure inevitably causes the requirement of separate data buses, address buses and control buses among different circuit modules, corresponding decoding circuits and control circuits, and the working mode has low operation speed and large generated power consumption. Therefore, the invention adopts the idea of integration of sensing and storage, and can realize the acquisition, storage and linear operation of dynamic visual sensing data in one unit by combining the CMOS dynamic visual sensing active pixel circuit and the storage and calculation integrated circuit. The purposes of saving storage area, reducing calculation power consumption and improving calculation speed are achieved.
The technical scheme adopted by the invention is as follows:
the CMOS sensing and storage integrated circuit structure based on the dynamic visual sensing technology is characterized by comprising a sensing circuit module based on an AER mode and a storage integrated circuit module;
the sensing circuit module based on the AER mode is used for sensing dynamic image data and specifically comprises a dynamic visual sensing active pixel circuit, a related secondary sampling circuit, a differential comparator circuit, a light intensity change detection circuit and a logic judgment circuit; wherein,
the dynamic visual sensing active pixel circuit is used for sensing an input optical signal and converting the input optical signal into an electric signal;
the correlated secondary sampling circuit is coupled to the output end of the dynamic visual sensing active-phase pixel circuit and is used for sampling and holding the voltage signal which is converted and output in a reset period and an integration period respectively;
the differential comparator circuit is coupled to the output end of the related secondary sampling circuit and is used for carrying out differential operation on the secondary sampling result and comparing the operation result with the reference voltage;
the light intensity change detection circuit is used for sensing whether the dynamic light intensity changes or not;
the logic judgment circuit is respectively coupled to the differential comparator circuit and the light intensity change detection circuit and is used for carrying out validity judgment by combining data of the differential comparator and the light intensity detection circuit and outputting a judgment result; the method specifically comprises the following steps: only when the differential comparator judges that the light intensity is changed and the light intensity detection circuit judges that the dynamic light intensity is changed, effective data '1' is output and stored in the SRAM, otherwise, the effective data '1' is stored in the SRAM;
the storage and calculation integrated circuit module is used for storing and calculating dynamic image data and specifically comprises an SRAM unit, a weight write-in circuit, a digital logic unit, an analog accumulator and a linear amplifier; wherein,
the SRAM unit is coupled to the output end of the logic judgment circuit and is used for storing external weight data and sensing data;
the weight value writing circuit is coupled to the input end of the SRAM unit and is used for writing external weight value data into the SRAM unit;
the digital logic unit is coupled to the output end of the SRAM unit, and the circuit structure of the digital logic unit comprises a multiplier for multiplying an external weight value and sensing data, a counter for identifying a multiplication result, and a pulse generating unit for generating a pulse signal;
the analog accumulator is coupled to the output end of the pulse generating unit and used for accumulating the number of the pulse signals in a capacitance charging mode and converting the pulse signals into analog voltage signals;
the linear amplifier is coupled to the output end of the analog accumulator and is used for further linearly amplifying the analog voltage signal.
Furthermore, the dynamic visual sensing active-phase pixel circuit mainly comprises a photodiode, a reset diode, a source follower and a row selection switch tube; wherein,
the photodiode is used for sensing the external illumination intensity and converting the illumination intensity into an induced current;
the reset diode periodically works in a reset period and an integration period under the control of a reset signal; in a reset period, a reset diode is turned on to charge a negative terminal node of the photodiode; during an integration period, the reset diode is turned off, and the charge on the parasitic capacitance of the negative terminal node of the photodiode is linearly discharged by the induced current;
the source follower is used as a buffer, so that the charge of the negative terminal node of the photodiode can be prevented from leaking in the process of reading signals;
and the row selection switch tube is used for controlling the output of signals.
Furthermore, the related secondary sampling circuit consists of an MOS switching tube controlled by a clock signal, a sampling capacitor and a sampling signal reading circuit;
the MOS switching tube controlled by the clock signal is used for controlling the switching tube to be opened at the tail ends of a reset period and an integration period respectively and sampling voltage signals at the two moments;
the sampling capacitor is used for holding sampling data;
the sampling signal reading circuit is formed by connecting three P-type MOS tubes in series and is used as a buffer for reading a sampling voltage result and lifting the level.
Further, the differential comparator circuit is composed of a differential operation circuit for realizing a differential function;
the differential operation circuit for realizing the differential function consists of an operational amplifier and four corresponding equal resistors, can perform equal-proportion difference operation on two sampling voltages and output a differential operation result, and the differential operation result represents light intensity information.
Furthermore, the light intensity change detection circuit can judge whether the dynamic light intensity changes, namely, whether the variation of the light intensity is equal in different periods or not, namely whether the representative image is dynamic change or not, and only dynamically changed light intensity data is sampled.
Furthermore, the SRAM cell stores two portions of data, one of which is external weight data, and the other is dynamic image sensing data.
Further, the weight writing circuit is used for writing external weight data into the SRAM unit, and comprises 4 writing transistors and 2 parasitic capacitors. Because the SRAM requires the data lines to be pre-charged and pre-discharged before writing data, the accuracy and stability of the written data can be ensured.
Furthermore, the digital logic unit circuit structure comprises a multiplier for multiplying an external weight value and sensing data, a counter for identifying a multiplication result, and a pulse generating unit for generating a pulse signal;
the specific algorithm of the multiplier is that the sum operation is carried out on each bit of the external input weight and the sensing data;
the counter is used for counting the number of '1' in the data of the multiplication result;
the pulse generating unit emits an equal number of short-time pulse signals according to the counting result of the counter.
Furthermore, the analog accumulator is composed of a charging transistor, a discharging transistor, a summing capacitor and a buffer, and the working principle is as follows: when a pulse is received once, the charging transistor is turned on, the summing capacitor is charged once by small current, and the voltage on the capacitor is accumulated in equal proportion, so that a digital signal result can be converted into an analog voltage signal; and when the capacitor is fully charged, the charge is fully discharged by turning on the discharge transistor.
Furthermore, the linear amplifier is composed of an operational amplifier and a corresponding resistor, and the input of the operational amplifier is coupled with the output end of the analog accumulator, so that the output result of the analog accumulator can be subjected to linear operation and further amplified.
The invention has the beneficial effects that: the invention combines the existing CMOS memory-computation integrated chip with a dynamic visual sensing circuit adopting Address-Event triggering (AER) technology, thereby having the characteristics of high sampling rate, high speed, high precision and low time delay.
Drawings
FIG. 1 is a schematic diagram of an overall structure of a CMOS sensing and computing integrated circuit based on dynamic visual sensing technology;
FIG. 2 is a schematic diagram of the specific circuit configuration of FIG. 1;
FIG. 3 is a schematic diagram of a dynamic visual sensing active pixel circuit configuration;
FIG. 4 is a schematic diagram of the dynamic visual sense active pixel circuit output voltage;
FIG. 5 is a schematic diagram of a circuit configuration of a correlated double sampling circuit;
FIG. 6 is a schematic diagram of the operation of the correlated double sampling circuit;
FIG. 7 is a schematic circuit diagram of the differential comparator circuit;
FIG. 8 is a schematic diagram of a circuit configuration of the light intensity change detection circuit;
FIG. 9 is a schematic circuit configuration diagram of a logic decision circuit incorporating a differential comparator circuit and a light intensity change detection circuit;
FIG. 10 is a schematic diagram showing the output waveforms of the key signals of the logic decision circuit for the same light intensity transition during two consecutive periods;
FIG. 11 is a schematic diagram showing the output waveforms of the key signals of the logic decision circuit when the light intensity changes are different in two consecutive periods;
FIG. 12 is a circuit diagram of a weight write circuit;
FIG. 13 is a schematic circuit diagram of an SRAM cell;
FIG. 14 is a schematic diagram of the overall circuit structure of the weight write circuit and the SRAM cell;
FIG. 15 is a schematic diagram of a digital logic circuit;
FIG. 16 is a schematic diagram of a circuit configuration of an analog accumulation circuit;
FIG. 17 is a schematic circuit diagram of a linear amplifier;
FIG. 18 is a diagram illustrating the output waveforms of the analog accumulator and the linear amplifier for the pulse signal corresponding to the operation result;
FIG. 19 is a schematic diagram of another digital circuit structure for performing multiplication of 4-bit weight and 1-bit sensing data.
Detailed Description
The invention provides a CMOS sensing and calculating integrated circuit structure based on a dynamic visual sensing technology on the basis of the prior art, and the circuit structure can realize the functions of dynamic visual sensing, storage and calculation on the same circuit. The pixel array constituted by such circuits can realize not only sensing and storage of moving image data of a larger scale but also more complicated parallel arithmetic processing.
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail by referring to specific embodiments in the accompanying drawings, it being understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.
As shown in fig. 1, a CMOS sensing and storage integrated circuit overall architecture based on dynamic visual sensing technology is composed of a sensing circuit module 1 based on an AER method and a storage integrated circuit module 2; the sensing circuit module 1 based on the AER mode comprises a dynamic visual sensing active pixel unit 3, a related secondary sampling circuit 5, a differential comparator circuit 7, a light intensity change detection circuit 9 and a logic judgment circuit 12; the storage and calculation integrated circuit module 2 comprises a weight value writing circuit 15, an SRAM unit 17, a digital logic unit 19, an analog accumulator 21 and a linear amplifier circuit 23; the signal flow when the circuit works is as follows: firstly, an external light intensity signal 14 is sensed by the DVS active pixel unit 3 and generates a sensing voltage signal 4; the related secondary sampling circuit 5 respectively samples the induction voltage signals 4 with two different periods and outputs a sampling result 6; the differential comparator circuit 7 performs differential operation on the sampling result 6 to obtain differential operation results 8 and 10; the light intensity detection circuit 9 obtains light intensity change information 11 according to the multiplication result 8; the logic decision circuit 12 comprehensively processes the difference operation result 10 and the light intensity change information 11 to obtain final sensing data 13; the SRAM cell 17 stores the external input weight 23 written by the weight write circuit 15 and the sensing data 13 at the same time, and transmits the storage data 18 to the digital logic unit 19; processing the external input weight 23 and the sensing data 13 in the digital logic unit 19 to obtain a multiplication result and sending pulse signal data 20 to the analog accumulation circuit 21; the analog accumulation circuit 21 transfers the accumulated result 22 to the linear amplification circuit 23, and obtains a final operation result 25.
Fig. 2 is a schematic diagram of a specific circuit structure corresponding to fig. 1.
Fig. 3 is a schematic structural diagram of the dynamic visual sensing active pixel circuit 3. This dynamic visual sensing active pixel cell is referred to as a 4-tube structure, and includes a reset tube 26, a source follower tube 27 and a row select tube 28, as well as a transistor 30 providing bias, in addition to a photodiode 31 in the pixel cell. The working principle of the pixel is as follows: the photodiode 31 goes through two cycles per operation under the control of the reset signal 21: a charging period and an integration period. In the charging period, the reset signal 21 keeps high level, the reset tube 26 is opened and the upper parasitic capacitance of the node 32 is charged to VDD-Vth, wherein Vth is the threshold voltage of the reset tube 26; during the integration period, the reset tube 26 is turned off and the photodiode 31 is illuminated with a certain intensity, and the charge stored in the parasitic capacitor on the node 32 is discharged by the induced current generated by the photodiode, so that the voltage on the node 32 is linearly decreased. The two working periods are performed circularly, so that the external light intensity change information can be sensed in real time through the photodiode 31, and the voltage signal on the node 32 can be read out without loss through the source follower 27 and the row selection switch 28, namely, the voltage signal is the output voltage signal 29.
Fig. 4 is a schematic diagram of the output voltage of the dynamic visual sense active pixel circuit 3. Each period T is divided into a charging period and a sub-period; during the charging period 0-t 1, node 32 is charged to VDD-Vth resulting in output node 29 being charged to Vr, which is about VDD-2 Vth; during the integration period t 1-t 2, the voltage at node 32 drops approximately linearly due to the presence of the induced current, and thus the voltage at output node 39 drops approximately linearly to vs 1; the situation of the second period T can be analyzed similarly.
As shown in fig. 5, the correlated double sampling circuit 5 has two identical sampling paths, and the circuit is composed of sampling tubes 33 and 40, sampling capacitors 35 and 42, source followers 38 and 45, switching tubes 37 and 44, and bias tubes 36 and 43. The working principle of the circuit is as follows: at the end of the charging period, the sampling tube 33 is opened through the control signal 34, the output voltage 29 is sampled and stored in the sampling capacitor 35, the sampling voltage is read out through the source follower 38, and the charging period sampling voltage 39 is obtained at the source node of the switching tube 37; the integration period operates as described above to obtain an integration period sample voltage 46.
Fig. 6 is a schematic diagram illustrating the operation principle of the correlated double sampling circuit 5. According to the output voltage change of the dynamic visual sensing active pixel circuit 3, at the end of the charging period, when the voltage of the node 29 is charged to VR, the control signal 34 of the sampling tube 33 generates a pulse signal to sample the voltage VR of the node 29 into the sampling capacitor 35, since the charging period will make the voltage of the node 29 rise to VR each time, so that VR remains unchanged thereafter, and at the same time, the sampled voltage is read out to the node VR without loss through the follower 38 and the switching tube 37 and remains unchanged, wherein VR is about VR + Vth; at the end of the integration period the voltage at node 29 decreases to a minimum value for one period, at which time the control signal 41 of the sampling tube 40 generates a pulse signal that samples the voltage VS1 at node 29 into the sampling capacitor 42, and the sampled voltage is read through the follower 45 and the switching tube 44 lossless to node VS, which is approximately VS1+ Vth until the next sampling. The situation of the second period T is obtained by the same analysis.
As shown in fig. 7, a schematic diagram of the differential comparator circuit 7 is shown, and the differential comparator mainly includes a differential circuit and a comparator circuit. The differential circuit consists of three equal resistors 40, 42 and 43 (all R), a feedback resistor 42 (Rf) and a high-precision operational amplifier 44, and if the resistor R is selected to be Rf, the output of the operational amplifier can be derived from the principles of "virtual short" and "virtual break" as follows: therefore, by comparing VD with the reference voltage Vrefh, it can be determined whether or not a sufficient light intensity is sensed during one period T. If VD is large, it means that the voltage change of VS is large during the integration period, i.e. enough light intensity information is sensed, so that the comparator 45 outputs a high level, otherwise it outputs a low level.
As shown in fig. 8, it is a schematic diagram of the light intensity change detection circuit 9, which is also composed of a differential comparison circuit and a dual comparator circuit. The differential comparison circuit is composed of three equal resistors 46, 47, 48 (all values are R), a feedback resistor 49 (the value is Rf) and a high-precision operational amplifier 50, and if the resistance value R is selected to be Rf, the above analysis can obtain: Δ V is VD2-VD1 ', where VD2 represents the voltage VD of the node 54 calculated by the differential comparison circuit in the period T, and VD1 ' represents the voltage VD of the node 54 calculated by the differential comparison circuit in the previous period T, where VD1 ' can be implemented by a buffer 53, so that the detection of the change of the VD voltage in two adjacent periods T can be implemented. Then comparing the obtained Δ V with two reference voltages + -Vrefl, if the variation of Δ V is higher than + -Vrefl or lower than-Vrefl, the circuit outputs a high level to indicate that the dynamic variation of light intensity is detected, otherwise the circuit outputs a low level. The circuit can realize the detection of the dynamic change of the light intensity, namely, the light intensity data in the period T can be detected only when the light intensity change in the period T is different from the light intensity change in the previous period T.
As shown in fig. 9, is a schematic circuit configuration diagram of the logic decision circuit 12 in combination with the differential comparator circuit 7 and the light intensity change detection circuit 9. The circuit mainly comprises a comparator 44 in a differential comparator circuit 7, comparators 51 and 52 in a light intensity conversion detection circuit 9, a logic OR gate 55 and a logic AND gate 56, and the working principle is as follows: only when the differential comparator circuit 7 senses the illumination intensity information of sufficient intensity and the light intensity detection circuit 9 detects that the light intensity change of the period T is inconsistent with the light intensity change of the previous period T, it is determined that the dynamic light intensity information is detected as a whole and a high level is output, otherwise a low level is output.
Fig. 10 is a schematic diagram showing output waveforms of the key signals VC, VCD, and VA of the logic decision circuit 12 when the light intensity transitions are the same in two consecutive periods; it can be seen from the change in Vph in the graph that sufficient light intensity is sensed during two consecutive periods T, resulting in the same amount of voltage change. Therefore, although the differential comparator circuit 7 outputs a high level indicating that the intensity of light is sensed, the light intensity change detection circuit 9 does not sense the dynamically changing light intensity information, and therefore outputs a low level. While the logic decision circuit also outputs a low level.
Fig. 11 is a schematic diagram showing output waveforms of the key signals VC, VCD, and VA of the logic determination circuit 12 when the light intensity conversion conditions are different in two consecutive periods; it can be seen from the change of Vph in the graph that in two consecutive periods T, sufficient light intensity is sensed, but the amount of voltage change generated in the two periods is not the same. Therefore, the differential comparator circuit 7 outputs a high level indicating that the intensity of the light is sensed, and the light intensity change detection circuit 9 also senses the dynamically changing light intensity information, thereby outputting a high level. At the same time the logic decision circuit also outputs a high level, representing that dynamic sense data is recognized, and stores the data into the SRAM cell 17.
As shown in fig. 12, it is a schematic structural diagram of the weight write circuit 15, which mainly includes write tubes 59 and 60 and bias tubes 57 and 58, where 61 and 62 are parasitic capacitances, and the operation principle is as follows: when the corresponding weight data W needs to be written, the signal 63 and the inverted signal 64 thereof need to be simultaneously input, assuming that data "1" needs to be written into the SRAM cell 17, the input signal 63 is at a low level, the inverted signal 64 thereof is at a high level, the node 65(BL) is pulled down to GND, the node 66(BLB) is charged to VDD, and the written data "0" can be analyzed in the same manner, and then the weight data can be stably written into the SRAM cell 17 by cooperating with the corresponding control signal of the SRAM cell 17.
As shown in fig. 13, which is a schematic diagram of a single memory cell structure of the SRAM cell 17, a CMOS inverter formed by a PMOS transistor 67 and an NMOS transistor 71 is connected end to end with a CMOS inverter formed by a PMOS transistor 68 and an NMOS transistor 72, which is a classic SRAM structure and can be used to store 1-bit data. To store a data "1", it is necessary that signal 65(BL) remain high and signal 66(BLB) remain low; to store a data "0," it is desirable that signal 65(BL) remain low and signal 66(BLB) remain high. When data needs to be stored into the SRAM cell 17, preprocessing by the weight value writing circuit 15 is required, that is, when the weight value writing circuit 15 precharges the data node 65(BL) and the node 66(BLB), and after the written data is ready, the write control signal 73 keeps a high level to allow data writing, and at this time, writing of the weight value data is completed; the sensing data is also stored in the SRAM cell 17, and the writing principle is the same as described above.
Fig. 14 is a schematic diagram of the overall circuit structure of the weight value writing circuit and the SRAM cell.
As shown in fig. 15, it is a schematic diagram of the architecture of the digital logic unit 19, and the digital logic unit 19 is implemented by using a digital circuit, which mainly includes a multiplier 76, a counter 77, and a pulse generating unit 78. Assuming that the external weight data and the sensing data are both 1bit data, the specific algorithm of the multiplier 76 is to perform an and operation on the external input weight and the sensing data; the counter is used for counting the number of '1' appearing in the data of the multiplication result; and the pulse generating unit sends out an equal number of short-time pulse signals according to the counting result of the counter.
As shown in fig. 16, it is a schematic diagram of a circuit structure of the analog accumulator 21, which mainly comprises a charging tube 80, a discharging tube 81, a summing capacitor 84 and a buffer 86, and the working principle is as follows: the hold signal 83 is kept low, so long as when the pulse generating unit 78 delivers a pulse, the charging tube 80 charges the summing capacitor 84 for a short period of time with a small current, and the charging current forms a summing voltage on the summing capacitor 84, which is read out by the buffer 86 to a subsequent circuit, and the summing voltage can be cleared through the discharging tube 81.
As shown in fig. 17, the linear amplifier 23 is a schematic diagram, and is composed of a high-precision operational amplifier 90, an input resistor 87, a matching resistor 88, and a feedback resistor 89, and can be derived from the principles of "virtual short" and "virtual short" of the operational amplifier: and VOUT is-R2/R1 multiplied by Va, namely, the output result of the analog accumulator can be subjected to linear operation and further amplified to obtain the final output VOUT of the system.
As shown in fig. 18, it is a schematic diagram of the output voltage when the analog accumulator 21 and the linear amplifier 23 operate. As can be seen from the figure, when the digital logic unit 19 obtains the result of the multiplication operation, the pulse generating unit 78 sends a corresponding pulse signal to the analog accumulator 21, and charges the analog accumulator 21. The voltage VQ across the analog accumulator 21 increases by Δ v for each pulse signal generated, and can be read by the linear amplifier 23 via the voltage follower 86 and amplified by a factor of-R2/R1.
Fig. 19 is a schematic diagram of another digital circuit structure for realizing multiplication of 4-bit weight and 1-bit sensing data. The weight writing circuit 15, the logic decision circuit 12, and the structure and the operation principle of the SRAM cell are not changed, and the external input weight 23 stores 4-bit weight data into the SRAM cell 17. The weight data of 4 bits and the sensing data of 1bit are operated, and the specific rule of multiplication operation is changed into that: performing AND operation on the weight data of 4 bits and the sensing data of 1 bit; that is, every time the logic decision circuit 12 stores a "1" into the SRAM cell 17, 4-bit weight data is accumulated once into the 8-bit adder 100; if the data stored by the logic decision circuit 12 into the SRAM cell 17 is "0", the data in the previous 8-bit adder 100 is saved until the 8-bit adder 100 overflows, and the 8-bit adder 100 is cleared at this time. The initial value of the 8-bit adder 100 is set to 8' b 00000000. The digital logic unit 98 may also be implemented by digital circuitry.
Claims (6)
1. The CMOS sensing and storage integrated circuit structure based on the dynamic visual sensing technology is characterized by comprising a sensing circuit module based on an AER mode and a storage integrated circuit module;
the sensing circuit module based on the AER mode is used for sensing dynamic image data and specifically comprises a dynamic visual sensing active pixel circuit, a related secondary sampling circuit, a differential comparator circuit, a light intensity change detection circuit and a logic judgment circuit; wherein,
the dynamic visual sensing active pixel circuit is used for sensing an input optical signal and converting the input optical signal into an electric signal; the dynamic visual sensing active pixel circuit consists of a photodiode, a reset diode, a source follower and a row selection switch tube; wherein,
the photodiode is used for sensing the external illumination intensity and converting the illumination intensity into an induced current;
the reset diode periodically works in a reset period and an integration period under the control of a reset signal; in a reset period, a reset diode is turned on to charge a negative terminal node of the photodiode; during an integration period, the reset diode is turned off, and the charge on the parasitic capacitance of the negative terminal node of the photodiode is linearly discharged by the induced current;
the source follower is used as a buffer, so that the charge of the negative terminal node of the photodiode can be prevented from leaking in the process of reading signals;
the row selection switch tube is used for controlling the output of signals;
the correlated secondary sampling circuit is coupled to the output end of the dynamic visual sensing active-phase pixel circuit and is used for sampling and holding the voltage signal which is converted and output in a reset period and an integration period respectively; the related secondary sampling circuit consists of an MOS switching tube controlled by a clock signal, a sampling capacitor and a sampling signal reading circuit;
the MOS switching tube controlled by the clock signal is used for controlling the switching tube to be opened at the tail ends of a reset period and an integration period respectively and sampling voltage signals at the two moments;
the sampling capacitor is used for holding sampling data;
the sampling signal reading circuit is formed by connecting three P-type MOS tubes in series and is used as a buffer for reading a sampling voltage result and raising the level;
the differential comparator circuit is coupled to the output end of the related secondary sampling circuit and is used for carrying out differential operation on the secondary sampling result and comparing the operation result with the reference voltage; the differential comparator circuit is composed of a differential operation circuit for realizing a differential function; the differential operation circuit for realizing the differential function consists of an operational amplifier and four corresponding equal resistors, can perform equal-proportion difference calculation on two sampling voltages and output a differential operation result, wherein the differential operation result represents light intensity information;
the light intensity change detection circuit is used for sensing whether the dynamic light intensity changes or not; the light intensity change detection circuit can judge whether the dynamic light intensity changes, namely, whether the variable quantity of the light intensity is equal in different periods or not, namely, whether the representative image is dynamic change or not, and only dynamically changed light intensity data is sampled;
the logic judgment circuit is respectively coupled to the differential comparator circuit and the light intensity change detection circuit and is used for carrying out validity judgment by combining data of the differential comparator and the light intensity detection circuit and outputting a judgment result; the method specifically comprises the following steps: only when the differential comparator judges that the light intensity is changed and the light intensity detection circuit judges that the dynamic light intensity is changed, effective data '1' is output and stored in the SRAM, otherwise, the effective data '1' is stored in the SRAM;
the storage and calculation integrated circuit module is used for storing and calculating dynamic image data and specifically comprises an SRAM unit, a weight write-in circuit, a digital logic unit, an analog accumulator and a linear amplifier; wherein,
the SRAM unit is coupled to the output end of the logic judgment circuit and is used for storing external weight data and sensing data;
the weight value writing circuit is coupled to the input end of the SRAM unit and is used for writing external weight value data into the SRAM unit;
the digital logic unit is coupled to the output end of the SRAM unit, and the circuit structure of the digital logic unit comprises a multiplier for multiplying an external weight value and sensing data, a counter for identifying a multiplication result, and a pulse generating unit for generating a pulse signal;
the analog accumulator is coupled to the output end of the pulse generating unit and used for accumulating the number of the pulse signals in a capacitance charging mode and converting the pulse signals into analog voltage signals;
the linear amplifier is coupled to the output end of the analog accumulator and is used for further linearly amplifying the analog voltage signal.
2. The CMOS sensing-accumulating integrated circuit structure based on dynamic visual sensing technology as claimed in claim 1, wherein said SRAM cells store two parts of data respectively, one part being external weight data and the other part being dynamic image sensing data.
3. The CMOS-based integrated circuit structure for dynamic visual sensing technology according to claim 2, wherein the weight write circuit is configured to write external weight data to the SRAM cell, which includes 4 write transistors and 2 parasitic capacitances.
4. The CMOS sensing and computing integrated circuit structure based on dynamic visual sensing technology according to claim 3, wherein the digital logic unit circuit structure comprises a multiplier for multiplying an external weight value and sensing data, a counter for identifying the multiplication result, and a pulse generating unit for generating a pulse signal;
the specific algorithm of the multiplier is that the sum operation is carried out on each bit of the external input weight and the sensing data;
the counter is used for counting the number of '1' in the data of the multiplication result;
the pulse generating unit sends out an equal number of short-time pulse signals according to the counting result of the counter.
5. The CMOS sensing and calculating integrated circuit structure based on dynamic visual sensing technology according to claim 4, wherein the analog accumulator is composed of a charge transistor, a discharge transistor, a summing capacitor and a buffer, and its working principle is as follows: when a pulse is received once, the charging transistor is turned on, the summing capacitor is charged once by small current, and the voltage on the capacitor is accumulated in equal proportion, so that a digital signal result can be converted into an analog voltage signal; and when the capacitor is fully charged, the charge is fully discharged by turning on the discharge transistor.
6. The CMOS integrated circuit structure for sensing and accumulating based on dynamic visual sensing technology as claimed in claim 5, wherein said linear amplifier is composed of an operational amplifier and a corresponding resistor, and the input of the operational amplifier is coupled to the output terminal of said analog accumulator, so as to perform linear operation on the output result of the analog accumulator for further amplification.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010272251.0A CN111343398B (en) | 2020-04-09 | 2020-04-09 | CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010272251.0A CN111343398B (en) | 2020-04-09 | 2020-04-09 | CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111343398A CN111343398A (en) | 2020-06-26 |
CN111343398B true CN111343398B (en) | 2021-10-26 |
Family
ID=71187529
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010272251.0A Active CN111343398B (en) | 2020-04-09 | 2020-04-09 | CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111343398B (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112601037B (en) * | 2020-12-14 | 2021-10-26 | 电子科技大学 | Floating gate device-based image sensing and storage integrated pixel unit and pixel array |
CN113052024B (en) * | 2021-03-12 | 2022-08-23 | 电子科技大学 | Sense and memory integrated circuit structure based on memristor and sensor |
CN113596361B (en) * | 2021-08-02 | 2022-07-12 | 电子科技大学 | Sense-memory-computation integrated circuit structure for realizing positive and negative weight calculation in pixel |
CN115988350B (en) * | 2022-10-20 | 2024-07-23 | 电子科技大学 | CMOS image sensing and storing integrated circuit integrating sampling calculation |
CN115906735B (en) * | 2023-01-06 | 2023-05-05 | 上海后摩智能科技有限公司 | Multi-bit number storage and calculation integrated circuit, chip and calculation device based on analog signals |
CN115756388B (en) * | 2023-01-06 | 2023-04-18 | 上海后摩智能科技有限公司 | Multi-mode storage and calculation integrated circuit, chip and calculation device |
CN117479027B (en) * | 2023-11-07 | 2024-05-24 | 上海宇勘科技有限公司 | SRAM-based method for simulating dynamic vision sensor array |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101309360A (en) * | 2007-05-17 | 2008-11-19 | 索尼株式会社 | Video input processor, imaging signal-processing circuit, and method for reducing noises in imaging signals |
CN102710906A (en) * | 2012-04-16 | 2012-10-03 | 天津大学 | CMOS (complementary metal-oxide-semiconductor transistor) image sensor for realizing two-dimensional discrete cosine transformation |
WO2018237110A1 (en) * | 2017-06-22 | 2018-12-27 | Newvoicemedia Us Inc. | Using conversational similarity for dimension reduction in deep analytics |
CN110870297A (en) * | 2017-07-19 | 2020-03-06 | 高通股份有限公司 | Motion detection on CMOS image sensor die using inter-pixel grid relationships |
CN110891152A (en) * | 2018-09-07 | 2020-03-17 | 三星电子株式会社 | Image sensor including CMOS image sensor pixels and dynamic vision sensor pixels |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10311454B2 (en) * | 2017-06-22 | 2019-06-04 | NewVoiceMedia Ltd. | Customer interaction and experience system using emotional-semantic computing |
-
2020
- 2020-04-09 CN CN202010272251.0A patent/CN111343398B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101309360A (en) * | 2007-05-17 | 2008-11-19 | 索尼株式会社 | Video input processor, imaging signal-processing circuit, and method for reducing noises in imaging signals |
CN102710906A (en) * | 2012-04-16 | 2012-10-03 | 天津大学 | CMOS (complementary metal-oxide-semiconductor transistor) image sensor for realizing two-dimensional discrete cosine transformation |
WO2018237110A1 (en) * | 2017-06-22 | 2018-12-27 | Newvoicemedia Us Inc. | Using conversational similarity for dimension reduction in deep analytics |
CN110870297A (en) * | 2017-07-19 | 2020-03-06 | 高通股份有限公司 | Motion detection on CMOS image sensor die using inter-pixel grid relationships |
CN110891152A (en) * | 2018-09-07 | 2020-03-17 | 三星电子株式会社 | Image sensor including CMOS image sensor pixels and dynamic vision sensor pixels |
Also Published As
Publication number | Publication date |
---|---|
CN111343398A (en) | 2020-06-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111343398B (en) | CMOS (complementary Metal oxide semiconductor) sensing and calculating integrated circuit structure based on dynamic visual sensing technology | |
JP2020503752A (en) | Dynamic vision sensor architecture | |
CN113596361B (en) | Sense-memory-computation integrated circuit structure for realizing positive and negative weight calculation in pixel | |
CN111309670A (en) | CMOS (complementary Metal oxide semiconductor) inductive storage and calculation integrated circuit structure | |
US8450673B2 (en) | Pixel circuit, imaging integrated circuit, and method for image information acquisition | |
CN101009762A (en) | Read-out circuit of infrared focal plane | |
CN114038492B (en) | Multiphase sampling memory internal computing circuit | |
CN110751279B (en) | Ferroelectric capacitance coupling neural network circuit structure and multiplication method of vector and matrix in neural network | |
CN110113548B (en) | CMOS image sensor and signal transmission method thereof | |
US11626153B2 (en) | Low power static random-access memory | |
CN116092553A (en) | Memory with multiplication and addition functions | |
CN112700810B (en) | CMOS sense-memory integrated circuit structure integrating memristors | |
CN102036019B (en) | Photoelectric conversion apparatus and image pickup system | |
Kaiser et al. | Neuromorphic-p2m: processing-in-pixel-in-memory paradigm for neuromorphic image sensors | |
CN115954029A (en) | Multi-bit operation module and in-memory calculation circuit structure using the same | |
CN115988350B (en) | CMOS image sensing and storing integrated circuit integrating sampling calculation | |
CN114974337A (en) | Time domain memory computing circuit based on spin magnetic random access memory | |
JP4628586B2 (en) | Photodetector | |
CN201242868Y (en) | Sensitive amplifier for EEPROM and read circuit comprised by the same | |
CN112468747A (en) | Current type reading circuit applied to large-area array image sensor and control method thereof | |
CN112737590A (en) | Current type reading circuit applied to high-speed large-area array ADC (analog to digital converter) | |
WO2024082377A1 (en) | D6t in-memory computing accelerator capable of always linearly discharging and reducing numeric steps | |
CN101532874B (en) | Optical sensor for display device | |
JP2743842B2 (en) | Infrared detector | |
CN111355907B (en) | Column-level ADC (analog to digital converter) for CMOS (complementary metal oxide semiconductor) image sensor and implementation method thereof |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |