WO2014077783A1

WO2014077783A1 - Image capture device and image capture system

Info

Publication number: WO2014077783A1
Application number: PCT/SG2013/000489
Authority: WO
Inventors: Mei Yan; Kiat Seng Yeo; Hao Yu
Original assignee: Nanyang Technological University
Priority date: 2012-11-15
Filing date: 2013-11-15
Publication date: 2014-05-22

Abstract

An image capture device may be provided, which may include: an array of dual-mode pixel elements configured to operate in a first mode and a second mode, wherein, in the first mode, each dual-mode pixel element generates a first signal representing a pixel value of a photographic image of an ion-carrying electrolyte, and wherein, in the second mode, each dual-mode pixel element generates a second signal representing a measurement of a concentration of ions in the ion-carrying electrolyte.

Description

IMAGE CAPTURE DEVICE AND IMAGE CAPTURE SYSTEM

Technical Field

[0001] Various aspects relate to an image capture device and an image capture system.

Background

[0002] When ions (e.g. in an ion-carrying electrolyte) are introduced at the surface of an ion-sensitive field-effect transistor (ISFET) sensor array, each ISFET sensor senses the ion density variation and converts it into small current signal with a linear response. The ISFET sensor array may amplify this small current signal and drive it off-chip for detection. Accordingly, the accuracy and sensitivity of the ISFET sensor array may be limited, e.g. by the performance of readout circuitry that may detect the small current signal.

Summary

[0003] According to various embodiments, an image capture device is provided. The image capture device may include: an array of dual-mode pixel elements configured to operate in a first mode and a second mode, wherein, in the first mode, each dual-mode pixel element generates a first signal representing a pixel value of a photographic image of an ion-carrying electrolyte, and wherein, in the second mode, each dual-mode pixel element generates a second signal representing a measurement of a concentration of ions in the ion-carrying electrolyte. Brief Description of the Drawings

[0004] In the drawings, like reference characters generally refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various aspects of the invention are described with reference to the following drawings, in which:

[0005] FIG. 1 shows an architecture of an image capture device.

[0006] FIG. 2A shows a pixel element of an array of pixel elements coupled to an amplifier, a sampler, and a memory.

[0007] FIG. 2B shows a timing diagram for a sampler.

[0008] FIG. 3 A to FIG. 3C show a working principle of a single- frame super-resolution image reconstruction, which may be executed by a processing circuit 1 12.

[0009] FIG. 4A to FIG. 4C show photographic images generated by the image capture device shown in FIG. 1.

[0010] FIG. 5 shows a conventional ISFET sensor device.

[0011] FIG. 6 shows an array of dual-mode pixel elements, which may be configured to operate in a first mode and a second mode.

[0012] FIG. 7 shows a schematic view of a dual-mode pixel element of the array of dual- mode pixel elements shown in FIG. 6.

[0013] FIG. 8 shows a cross-sectional view a dual-mode pixel element of the array of dual- mode pixel elements shown in FIG. 6.

[0014] FIG. 9 shows a top-level chip architecture of an image capture device including an array of dual-mode pixel elements. [0015] FIG. 10 shows a testing setup for a image capture device including an array of dual- mode pixel elements.

[0016] FIG. 1 1 shows a photographic image and a pH map generated based on first and second signals generated by the array of dual-mode pixel elements shown in FIG. 6.

[0017] FIG. 12 shows a sensitivity of an image capture device 100 to ion concentration in an ion-carrying electrolyte.

[0018] FIG. 13 shows a simplified schematic of a dual-mode pixel element.

[0019] FIG. 14 shows a comparison of sensitivities of the dual-mode pixel element shown in

FIG. 13 and the conventional ISFET sensor device shown in FIG. 5.

[0020] FIG. 15A to FIG. 15D show images generated from a measurement of a concentration of ions in the ion-carrying electrolyte.

[0021] FIG. 16 shows an image capture system.

Description

[0022] The following detailed description refers to the accompanying drawings that show, by way of illustration, specific details and aspects in which the invention may be practised. These aspects are described in sufficient detail to enable those skilled in the art to practice the invention. Other aspects may be utilized and structural, logical, and electrical changes may be made without departing from the scope of the invention. The various aspects are not necessarily mutually exclusive, as some aspects can be combined with one or more other aspects to form new aspects. Various aspects are described for structures or devices, and various aspects are described for methods. It may be understood that one or more (e.g. all) aspects described in connection with structures or devices may be equally applicable to the methods, and vice versa. [0023] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any aspect or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0024] The terms "coupled" and/or "electrically coupled" and/or "connected" and/or "electrically connected", used herein to describe a feature being connected to at least one other implied feature, are not meant to mean that the feature and the at least one other implied feature must be directly coupled or connected together; intervening features may be provided between the feature and at least one other implied feature.

[0025] Directional terminology, such as e.g. "upper", "lower", "top", "bottom", "left-hand", "right-hand", etc., may be used with reference to the orientation of figure(s) being described. Because components of the figure(s) may be positioned in a number of different orientations, the directional terminology is used for purposes of illustration and is in no way limiting. It is to be understood that structural or logical changes may be made without departing from the scope of the invention.

[0026] An image sensor may include, or may be, a device that may convert photons (e.g. light photons) into an electrical signal, which may subsequently be represented as a photographic image. An image sensor may be used in various applications and devices, including but not limited to mobile phones, cameras, and biomedical devices. An image sensor may include, or may be, a charge-coupled device (CCD) image sensor and/or a complementary metal-oxide-semiconductor (CMOS) image sensor.

[0027] A CMOS image sensor may be compatible to a standard CMOS process. Consequently, a CMOS image sensor may have low manufacturing cost, small size (e.g. small footprint and/or small real-estate), high processing speed, and low power consumption. W

Furthermore, a CMOS image sensor may be easy to integrate, e.g. with at least one other device. The CMOS image sensor and the at least one other device may be a part of a chip, e.g. an integrated chip. Easy integration of a CMOS image sensor may lead to the CMOS image sensor gaining much attention in the mobile imaging market.

[0028] On the other hand, although a CCD image sensor may have higher power consumption and slow processing speed (e.g. slow readout time) compared to a CMOS image sensor, a CCD image sensor may still have a market in security and medical section due at least in part to the CCD image sensor having high light sensitivity.

[0029] Improving the light sensitivity may be a key challenge for CMOS image sensor technology, especially as the number of pixel elements in a CMOS image sensor keeps increasing and pixel size keeps decreasing (e.g. pixels sizes less than 1 μιη).

[0030] For a fixed process technology and pixel architecture, shrinking a pixel size may downgrade pixel performance. For example, without compensating technologies, smaller pixels may have lower dynamic range, lower fill factor, lower light sensitivity, higher dark signals, and higher non-uniformity. While it may be possible to compensate for smaller pixel sizes by increasing the exposure time, a consequence of such an approach may be an decrease in the processing speed (e.g. increased readout time) of the CMOS image sensor.

[0031] Mobile imaging applications have driven innovations in CMOS image sensor technologies that can significantly compensate for the expected degradation in performance with decreasing pixel sizes. Process modifications including back-side illumination, improved micro- lenses, pinned photodiode, dual-gate oxide, floating diffusion, circuit techniques such as device sharing, and active reset may compensate for the factors that would otherwise reduce W

performance of small pixel sizes. However, such approaches may be costly and there may be a physical limit beyond which pixel performance cannot be improved further.

[0032] In order to maintain good light sensitivity, a large pixel size may be preferred, which in turn may degrade the resolution. Super-resolution algorithms have been explored where at least one low resolution image is used to generate a high resolution image. For example, multi- frame super-resolution algorithms have been explored to improve resolution from a system design perspective. Such approaches may overcome spatial pixel size limitation by using the image information from a few previous frames to generate an image with improved resolution. An example may be a field-programmable gate array (FPGA)-level implementation of a super- resolution algorithm. However, such an approach may be limited by off-chip data transferring speed and data size, and therefore it cannot be applied to high-speed image sensor operation.

[0033] Accordingly, there may be a need for an image capture device (e.g. a CMOS image sensor) that may achieve high-speed and high light sensitivity while maintaining good spatial resolution. Such a need may, for example, be satisfied by the image capture device 100 shown in FIG. 1.

[0034] FIG. 1 shows an architecture of an image capture device 100.

[0035] The image capture device 100 may include an array of pixel elements 102.

[0036] The array of pixel elements 102 may also be referred to as a sensor array. The array of pixel elements 102 may be arranged as a plurality of rows and a plurality of columns. Each pixel element of the array of pixel elements 102 may generate a signal (e.g. an analog signal), which may represent a pixel value of an image (e.g. a photographic image) of an object, e.g. that may be exposed to the array of pixel elements 102. [0037] In the example shown in FIG. 1, the array of pixel elements 102 may include a 128x128 array of pixel elements (i.e. 128 rows and 128 columns). However, in another example, the array of pixel elements 102 may have more than 128 rows of pixel elements or less than 128 rows of pixel elements. In like manner, in another example, the array of pixel elements 102 may have more than 128 columns of pixel elements or less than 128 columns of pixel elements.

[0038] The image capture device 100 may further include at least one of an amplifier 104, a sampler 106, a memory 108, a selector 1 10, and a processing circuit 1 12, which may be coupled (e.g. electrically coupled) to the array of pixel elements 102. The word "circuit" is used herein to mean any kind of a logic implementing entity, which may be special purpose circuitry or processor executing software stored in a memory, firmware, or any combination thereof. Thus, in one or more examples, a "circuit" may be a hard-wired logic circuit or a programmable logic circuit such as a programmable processor, e.g. a microprocessor (e.g. a Complex Instruction Set Computer (CISC) processor or a Reduced Instruction Set Computer (RISC) processor). A "circuit" may also be a processor executing software, e.g. any kind of computer program, e.g. a computer program using a virtual machine code such as e.g. Java. Different circuits can thus also be implemented by the same component, e.g. by a processor executing two different programs.

[0039] The image capture device 100 may be configured as a column-parallel image sensor. In other words, each column of the array of pixel elements 102 may be coupled to a respective device and/or circuit element. Stated differently, each pixel element of a respective column of the array of pixel elements 102 may be coupled to the same device and/or circuit element.

[0040] The image capture device 100 may include an amplifier 104. The amplifier 104 may be configured to amplify the signal (e.g. analog signal) generated by each pixel element of the array of pixel elements 102. As described above, the image capture device 100 may be configured as a column-parallel image sensor. In other words, each column of the array of pixel elements 102 may be coupled to a respective amplifier 104.

[0041] The image capture device 100 may include a sampler 106. The sampler 106 may be configured to sample a signal (e.g. an analog signal), which may be provided to it by the amplifier 104. The sampler 106 may include, or may be, an analog-to-digital convertor. In such an example, the sampler 106 (e.g. analog-to-digital convertor) may be configured to convert the signal (e.g. analog signal), e.g. amplified by the amplifier 104, to a digital signal. As described above, the image capture device 100 may be configured as a column-parallel image sensor. Accordingly, each column of the array of pixel elements 102 may be coupled to a respective sampler 106.

[0042] The image capture device 100 may include a memory 108, which may be configured to store a signal (e.g. digital signal) provided to it by the sampler 106 (e.g. analog-to-digital convertor). As described above, the image capture'device 100 may be configured as a column- parallel image sensor. Accordingly, each column of the array of pixel elements 102 may be coupled to a respective memory 108.

[0043] The image capture device 100 may include a selector 110, which may be configured to select data stored in the memory 108. The selector 1 10 may be further configured to provide the selected data to the processing circuit 1 12. The selector 1 10 may include, or may be, at least one of a column decoder, a column driver, a row decoder and a row driver of the array of pixel elements 102. [0044] The processing circuit 1 12 may be configured to increase a spatial resolution of an image that may be represented by the signal (e.g. analog signal) generated by each pixel element of the array of pixel elements 102.

[0045] In the description that follows, an example is described where each pixel element of the array of pixel elements 102 may achieve a spatial resolution of about 10 μιη. Accordingly, the image that may be represented by the signal (e.g. analog signal) generated by each pixel element of the array of pixel elements 102 may have a spatial resolution of about 10 μχη. The processing circuit 1 12 may, for example, increase a spatial resolution of the image, thus resulting in another image with a spatial resolution of less than about 10 μτη. It is noted that while the description that follows may provide an example where the spatial resolution achieved by each pixel element of the array of pixel elements 102 is about 10 /mi, in another example, the spatial resolution achieved by each pixel element of the array of pixel elements 102 may be less than about 10 μηι or greater than about 10 μηι.

[0046] FIG. 2 A shows a pixel element 102a of the array of pixel elements 102 coupled to the amplifier 104, the sampler 106, and the memory 108.

[0047] FIG. 2B shows a timing diagram 202 for the sampler 106.

[0048] Reference signs in FIG. 2A and FIG. 2B that are the same as in FIG. 1 denote the same or similar elements as in FIG. 1. Thus, those elements will not be described in detail again here; reference is made to the description above.

[0049] The pixel element 102a may include, or may consist of, a plurality of circuit elements. The plurality of circuit elements of the pixel element 102a may include, or may be, at least one of a transistor, a resistor, a capacitor, and a diode (e.g. photo-diode), although other circuit elements may be possible as well. [0050] For example, in the example shown in FIG. 2A, the plurality of circuit elements of the pixel element 102a may include, or may be, a plurality of transistors RST, SEL and a photo- diode 102P. It may be noted that in the example shown in FIG. 2A, the pixel element 102a may include, or may be, a 3-transistor (3T) advanced photo system (APS) pixel element (also referred to as 3T-APS pixel element). As described above, the pixel element 102a may be a pixel element of the array of pixel elements 102. Accordingly, in the example shown in FIG. 2A, the array of pixel elements 102 may include, or may consist of, 128x 128 3T-APS pixel elements.

[0051] The plurality of circuit elements of the pixel element 102a may be configured to generate the signal (e.g. analog signal) that may represent a pixel value of an image (e.g. a photographic image), e.g. of an object that may be exposed to the array of pixel elements 102.

[0052] A spacing between adjacent pixel elements (i.e. pitch) may determine the spatial resolution of an image (e.g. a photographic image) that may be generated from the signal (e.g. analog signal) generated by the array of pixel elements 102. In the example shown in FIG. 1, the pitch may be about 10 /mi, and thus the pixel size may be chosen as about lO/rni; e.g. to achieve good low-light sensitivity. When one row of the array of pixel elements 102 is active, the signal (e.g. analog signal) from each column (e.g. each of the 128-column outputs) may be amplified (e.g. by means of the amplifier 104), sampled (e.g. by means of sampler 106) and processed (e.g. by means of the processing circuit 112). The signal (e.g. analog signal) from each column (e.g. each of the 128-column outputs) may go through the aforementioned processes simultaneously.

[0053] As described above, the amplifier 104 may, for example, amplify a signal (e.g. analog signal) generated by the pixel element 102a of the array of pixel elements 102. W

[0054] The amplifier 104 may include, or may consist of, a plurality of circuit elements. The plurality of circuit elements of the amplifier may include, of may be, at least one of an op-amp, a switch, a transistor, a resistor, a capacitor, and a diode, although other circuit elements may be possible as well. For instance, in the example shown in FIG. 2A, the amplifier 104 may include, or may be, a single-ended switch-capacitance amplifier.

[0055] A gain of the amplifier 104 (e.g. single-ended switch-capacitance amplifier) may be variable. In other words, the gain of the amplifier 104 may be adjusted and/or selected, e.g. based on a selection of the plurality of circuit elements. For example, the gain of the amplifier 104 may depend on a ratio of capacitances of a plurality of capacitors that may be included in the amplifier 104. For example, the gain of the amplifier 104 may be adjusted as and/or selected from IX, 2X and 4X gain, although other gain values may be possible as well.

[0056] The gain of the amplifier 104 may be based on a lighting condition to which the pixel element 102a may exposed to. In other words, the amplifier gain may be adjustable according to different lighting conditions. This design may allow the following: i) to amplify the signal and increase pixel sensitivity; ii) to reduce readout noise as the amplifying stage is close to the pixel element 102a; iii) to provide the flexibility to adjust the signal voltage level of analog-to-digital (A/D) conversion (ADC) in next stage (e.g. sampling).

[0057] The amplified signal may be converted to a digital output, e.g. by the sampler 106 (e.g. analog-to-digital converter (ADC)). The sampler 106 (e.g. analog-to-digital converter) may, for example, implement a 10-bit single-slope ADC (SS-ADC) with up-down counter in each column. The sampler 106 (e.g. analog-to-digital converter) not only converts an analog signal to a digital signal but may also carry out correlated double sampling (CDS) directly in the digital domain. A timing diagram for the CDS carried out by sampler 106 (e.g. analog-to-digital convertor) is shown in the timing diagram 202 in FIG. 2B.

[0058] After sampling, the digital outputs of each row of the array of pixel elements 102 may be latched and stored in the memory 108 (e.g. static random-access memory (SRAM), e.g. column-wise standard 6-transistor (6T) SRAM).

[0059] The selector 1 10 (e.g. column decoder) shown in FIG. 1 may select data stored in the memory 108 (e.g. SRAM), e.g. in serial. The selector 110 may read out the selected data by means of a sense amplifier (SAMP) 204. The selector 110 may subsequently provide the selected and readout data to the processing circuit 112.

[0060] The data provided to the processing circuit 112 may be referred to as captured imaging data. The processing circuit 112 may analyze the captured imaging data in real-time. The processing circuit 112 may be configured to increase a spatial resolution of a photographic image whose pixel value is represented by the signal (e.g. analog signal) generated by each pixel element the array of pixel elements 102. For example, the processing circuit 1 12 may be configured to execute a super-resolution image reconstruction algorithm (described below). The higher-resolution image data may be driven off-chip, i.e. external to the image capture device 100.

[0061] Driven by lower cost and higher resolution, pixel sizes may be becoming smaller and smaller, even into the sub-micron range. However, as described above, a small pixel size may degrade the performance of the sensor array, such as quantum efficiency and SNR sensitivity under the low light condition. A large pixel size (e.g. 10 μπι pitch of adjacent pixel elements of the plurality of pixel elements 102) may be selected to achieve superior low-light sensitivity. The light exposure time for large pixel may be less, which in turn may speed up the frame rate of the image capture device 100. However, one consequence is that the spatial resolution of the sensor array may be degraded, which may mean that the sensor array cannot recognize an object less than e.g. 10 μχη in size.

[0062] As described above, the processing circuit 1 12 may increase a spatial resolution of an image represented by the signal (e.g. analog signal) generated by the array of pixel elements 102. This may be accomplished on-chip. As such, the physical size limitation may be overcome and spatial resolution of the resultant image may be improved.

[0063] Super-resolution (SR) based image reconstruction may include, or may be, an imaging processing technique that may include converting an original low-resolution image into a high-resolution image. SR based imaging reconstruction may include at least one of single- frame based SR image reconstruction and multi-frame based SR image reconstruction.

[0064] The multi-frame based SR image reconstruction may include capturing a plurality (i.e. multiple) low-resolution images of an object to be imaged and reconstructing it into a high- resolution image by assuming that the plurality of low-resolution images can provide different imaging perspectives of the object.

[0065] The single-frame based SR image reconstruction may be based on a single low- resolution image by assuming that the image may be spatially smooth and can be approximately reconstructed by polynomials such as bilinear functions. Since multi-frame SR approach may require large memory to process data, while the single-frame SR approach may process the data in real time, a single-frame SR approach may be implemented by the processing circuit 1 12. This may also allow for on-chip system-on-chip (SoC) integration.

[0066] FIG. 3A to FIG. 3D show a working principle of a single-frame super-resolution image reconstruction, which may be executed by the processing circuit 1 12. [0067] The single-frame SR based image reconstruction may be designed on chip through the standard application-specific integrated circuit (ASIC) synthesis flow.

[0068] The signal (amplified and sampled signal) from the array of pixel elements 102 (e.g. entire sensor array, e.g. all 128-rows and 128-columns) may be provided to the processing circuit 112 (e.g. from memory 108, by means of selector 110) in serial into an input data buffer. The processing circuit 112 may include the input data buffer. In the example shown in FIG. 3 A, the 128 pixel output data from a row (e.g. first row, e.g. represented by pixels Nl, N2 ...N128) may be transferred to the input data buffer 301 along with at least one pixel from the next row (e.g. first 2 pixels of the second row, e.g. represented by pixels (N+l)l, (N+l)2).

[0069] As shown in FIG. 3B, a targeted pixel (e.g. pixel Nl) may be reconstructed into a plurality of high resolution pixels 303 (e.g. reconstructed into 4x4 "new" high-resolution pixels represented by pixels Nl, PI, P2 ... P15). The reconstruction into the plurality of high resolution pixels 303 may be performed by means of an interpolation (e.g. bilinear interpolation). The plurality of high resolution pixels (e.g. 4x4 "new" high-resolution pixels represented by pixels Nl, PI, P2 ... P15) may be generated based on the targeted pixel (e.g. pixel Nl) and a plurality of reference pixels (e.g. reference pixels represented by pixels N2, (N+l)l, (N+l)2). The plurality of reference pixels may (e.g. reference pixels represented by pixels N2, (N+l)l, (N+l)2), for example, be neighboring pixels of the targeted pixel (e.g. pixel Nl). The plurality of high resolution pixels 303 (e.g. represented by pixels Nl, PI , P2 ... PI 5) may be subsequently driven off-chip through an output buffer 302 shown in FIG. 3C and used to represent the targeted original pixel (e.g. pixel Nl).

[0070] As shown in FIG. 3D, data may be continuously pumped into the input buffer 301 and a new targeted pixel (e.g. pixel N2) may then be processed. For the new targeted (e.g. pixel N2), a plurality of new reference pixels may include, or may be, neighboring pixels of the new targeted pixel (e.g. pixels N3, (N+l)2, (N+l)3) as shown in FIG. 3C. Consequently, the processing circuit 1 12 may generate another plurality of high resolution pixels 304 based on the new targeted pixel (e.g. pixel N2) and the new plurality of reference pixels (e.g. pixels N3, (N+l)2, (N+l)3), which may be subsequently driven off-chip through the output buffer 302.

[0071] When a special condition occurs that the plurality of reference pixels (e.g. all three reference pixels) are not available during the interpolation, an average interpolation may be employed instead. As seen in FIG. 3 A to FIG. 3D, the processing circuit 1 12 may only need one processing core and two data buffers (e.g. input buffer 301 and output buffer 302) to store the data before and after processing. In other words, the processing circuit 1 12 may include, or may consist of, one processing core and two data buffers. This may be conveniently demonstrated by on-chip ASIC implementation.

[0072] In the example described above, the original data of 128x 128 sensor array with l OjUm resolution can generate processed data of 512x512 with 2.5 μτα. resolution at a frame rate of 1750fps. Accordingly, the image capture device 100 may achieve high-speed, high-sensitivity and high-resolution.

[0073] It is noted that while the description gives an example of single-frame SR. based image reconstruction based on one row of an image, it can also applied to multiple rows to achieve better spatial information in a vertical direction. This may be referred to as a single- frame SR multi-row implementation. It may also be noted that a single-frame SR multi-row implementation may not limited to column-parallel image sensor architecture, it can also apply to any other sensor architecture, such as global-readout architecture. W

[0074] FIG. 4A to FIG. 4C show photographic images generated by the image capture device 100.

[0075] FIG. 4A shows an image 400 of colloid particle flowing in a microfluidic device. The particle may have a diameter 15.7μπι similar to the size of a cancer cell. The trajectory 402 of the particle may be tracked by means of the generating high-speed and high-sensitivity photographic images over time.

[0076] FIG. 4B shows an original 8x8 image of a particle and FIG. 4C shows a processed 32x32 image of the image shown in FIG. 4B. The image in FIG. 4C was generated by the above-described single-frame SR based image reconstruction (e.g. shown in FIG. 3A to FIG. 3D). As shown in FIG. 4C, when compared to the original image in FIG. 4B, the image after super-resolution processing shows more details with 4X improved resolution of the particle.

[0077] The image capture device 100 may be used for biomedical applications as well as applied to commercial image sensors, such as mobile imaging, security etc., where a high resolution image may be desired.

[0078] The image capture device 100 described above may include, or may be, a column- parallel single-frame super-resolution CMOS image sensor that may achieve high-speed, high- sensitivity while maintain good spatial resolution.

[0079] As described above, a large pixel may be implemented to obtain both high sensitivity and high speed, while fine to moderate spatial resolution may be achieved with an on-chip single-frame super-resolution algorithm. Single-frame SR based image reconstruction described above may be a promising alternative to conventional multi-frame super-resolution approaches which may require multiple frames of memory and which may not be feasible for on-chip implementation. Single-frame SR based image reconstruction may provide high resolution image through reconstruction using polynomials such as bilinear functions based on multiple rows instead of multiple frames. Single-frame SR based image reconstruction may be especially important to certain applications where limited memory and computational resources are required.

[0080] Existing CMOS image sensor devices may include a column-parallel image sensor chip with small pixel and a FPGA-based super-resolution algorithm using large pixels. Table 1 shows a comparison between these existing CMOS image sensor devices. As shown in Table 1, column-parallel image sensor can achieve high speed benefit from the high speed column- parallel readout architecture, but small pixels show poor light sensitivity. As shown in Table 1 , a CMOS image sensor device with large pixel employed with FPGA-based super-resolution can improve both sensitivity and spatial resolution, however the readout speed is limited by the off- chip FPGA imaging analysis. In contrast, the image capture device 100, which may include, or may be, a column-parallel super-resolution image sensor device, can achieve high sensitivity, high speed and high spatial resolution.

Table 1: Comparison between existing CMOS image sensor design and proposed super- resolution ima e sensor desi n

[0081] In addition to the above-described needs for devices that can achieve high sensitivity, high speed and high spatial resolution, there is an emerging need for devices that may be configured to determine (e.g. measure) a concentration of ions with high temporal resolution and high sensitivity, e.g. by means of generating a voltage and/or a current that may vary with the concentration of ions. For example, a device may generate a voltage and/or a current that may be proportional (e.g. linearly proportional) to the concentration of ions, e.g. in an ion- carrying solution (e.g. an ion-carrying electrolyte). Therefore, the concentration of ions may be determined (e.g. measured) based on the voltage and/or current generated by the device.

[0082] Determining (e.g. measuring) a concentration of ions may be useful in various industries and/or applications. For example, in the food and drug industry, testing the safety of samples (e.g. food, drug and/or biological samples) may include, or may consist of, determining (e.g. measuring) a concentration of ions, e.g. by means of devices such as pH detectors and ion- channel detectors. The description that follows provides examples in the context of the food and drug industry. However, the description may analogously apply to other industries and/or applications.

[0083] As modern customers become increasingly aware of food and drug contamination, food and drug safety testing may be a critical issue worldwide. In food and drug safety testing, a concentration of ions in a sample being tested (e.g. food, drug and/or biological sample) may depend, at least in part, on ion channels of the sample.

[0084] In food and drug safety testing, ion channels (e.g. for calcium, chlorine, and/or other ions) may be tested (e.g. by means of measuring a concentration of ions in an ion-carrying electrolyte), and may be primary test targets (i.e. frequently tested). Ion channels may be primary test targets due, at least in part, to the fact that ion channels play a crucial role in the human central nervous system and/or in controlling the heart and brain. Chemicals contained in seafood, water, milk, or juice may regulate ion channels and may reveal a potential link with human diseases, including cancer. In addition, some drugs can show serious side-effects via ion W

channels, in such a manner that testing guidelines for drugs may now require ion channel screening for pharmaceutical safety.

[0085] Existing methods for measuring a concentration of ions in an ion-carrying electrolyte may include, or may consist of, using at least one of a patch-clamp, a fluorescence measurement, an electrode, and an ion-sensitive field-effect transistor (ISFET) sensor device.

[0086] As described above, a patch-clamp may be used for measuring a concentration of ions in an ion-carrying electrolyte. Such a technique may be referred to as a patch-clamp technique. A patch-clamp technique may control (or clamp) the electrical potential (e.g. voltage) difference in a sample being tested (e.g. food, drug and/or biological sample). For example, a patch-clamp technique may control (or clamp) the electrical potential (e.g. voltage) difference across a small patch of membrane of an entire cell, and may directly measure the current carried by ions crossing the membrane. Although the patch-clamp technique may provide high quality data of ion channel function, this technology may require skilled operation, which may result in a relatively low throughput. The patch-clamp technique may also be quite expensive (e.g. for academic institutions and/or small corporations).

[0087] As described above, a fluorescence mesurement may be used for measuring a concentration of ions in an ion-carrying electrolyte. In fluorescence measurement, a particular dye (which may be linked to a specified ion) may be introduced at each measurement. The dye may be ion-concentration dependent or membrane-potential dependent. Since the dye is either ion-concentration dependent or membrane-potential dependent, the fluorescence measurement may change when dyes are loaded into the cell membrane through the ion channels. Therefore, ion flux information can be obtained. A fluorescence measurement (e.g. a fluorescence assay) may be a popular preliminary screening method because it may be easy to set up and may achieve high throughput. However, the data provided may have false negatives and/or false positive information due to an indirect readout in this approach, e.g. since the fluorescence measurement may be an indirect measure of membrane-potential dependent or ion-concentration dependent fluorescent signal changes, rather than a direct measure of changes in ionic current. In addition, cell activity (e.g. live-time cell activity) may also be largely affected by the introduced dye.

[0088] As described above, an electrode may be used for measuring a concentration of ions in an ion-carrying electrolyte. This technique may measure specific ion flux by placing two or more ion-selective electrodes and measuring the voltage difference to detect the ion flux concentration gradient. The flux measurement (e.g. flux assay) may be preferred by many screening laboratories because it may measure ionic flux directly and may provide robust data against false positive and/or false negative information seen in fluorescence-based measurements (e.g. fluorescence-based assays). This technique may also give higher throughput measurement than the patch-clamp approach. However, using an electrode may not detect ion movement in real-time, and may also suffer from low temporal resolution and/or low spatial resolution.

[0089] As described above, an ion-sensitive field-effect transistor (ISFET) sensor device may be used for measuring a concentration of ions in an ion-carrying electrolyte. The ISFET sensor device used for testing an ion channel may include, or may be, at least one of an ISFET sensor device including an ion-sensitive membrane and an ISFET sensor device manufactured in accordance with metal-oxide-semiconductor (MOS) technology (e.g. p-type MOS (PMOS), n- type MOS (NMOS), complementary MOS (CMOS)). Unlike an ISFET device that may include an ion-sensitive membrane by expensive post-processing, ISFET sensor devices manufactured in accordance with a MOS process (e.g. standard CMOS process) have drawn much attention recently, due at least in part, by the use of a passivation layer (e.g. top most passivation layer, e.g. S13N4) for measuring a concentration of ions in an ion-carrying electrolyte.

[0090] An emerging application for an ISFET sensor device manufactured in accordance with a MOS process may be to detect a hybridization process of a complimentary stranded DNA. Since there may be protons (i.e. H⁺ ions) released in this process, the pH of a solution (e.g. an electrolyte) surrounding a reaction chamber containing the complimentary stranded DNA may have a correlated change in the reaction chamber. In this example, the pH of the solution (e.g. an electrolyte) may measure a concentration of ions (i.e. H⁺ ions) in an ion- carrying electrolyte. This technique may thereby provide portable and label-free biological diagnosis for DNA sequencing as well as bacterial biofilm. It may be noted that a micro-bead may be utilized to attach and carry one slice of DNA. With detecting the pH of a plurality of micro-beads which may be spatially localized (e.g. thousands of spatial-localized micro-beads), a long-chain DNA sequence may be decoded within a single test. However, there are many challenges unresolved for high-throughput pH detection. For example, a slice of DNA may need to be located at a precise spatial position and a local pH response may need to be generated at the precise spatial position with accurate data correlation.

[0091] A plurality of ISFET sensor devices may be arranged as an ISFET sensor array. When an ion-carrying electrolyte is introduced at a surface of the ISFET sensor array, each ISFET sensor device may sense a local concentration of ions (e.g. a local ion density variation) and may convert this into a small current signal, e.g. with a linear response. Since the ISFET sensor array may measure the ion concentration of an ion-carrying electrolyte that may, for example, be facing the ISFET sensor array, each ISFET sensor device may have a function that may be similar to a sensor array (e.g. photon sensor array) in a digital camera system. For example, while the sensor array in a digital camera system may generate signals representing a pixel value of a photographic image of an object exposed to the sensor array, the ISFET sensor array may generate signals representing a measurement of a concentration of ions in an ion- carrying electrolyte, which may subsequently be used to generate an ion concentration map of the ion-carrying electrolyte.

[0092] Each ISFET sensor device may amplify the small current signal and may drive it off- chip (i.e. external to the ISFET sensor array) for detection. Thus, the accuracy of the ISFET sensor device and/or the ISFET sensor array may be limited by the performance of external readout circuitry that may be used for detection. Since the small current signal is readout directly, it may be challenging to design a high-sensitivity readout circuit that may produce a large change in signal with a small change in ion concentration. Furthermore, a mismatch between adjacent ISFET sensor devices of the ISFET sensor array may need to be reduced for high accuracy during readout by an ISFET sensor array (e.g. a large ISFET sensor array).

[0093] FIG. 5 shows a conventional ISFET sensor device 500.

[0094] The ISFET sensor device 500 may, for example, be a conventional ISFET sensor device that may be used for ion channel characterization, ion channel screening and/or measuring a concentration of ions in an ion-carrying electrolyte, as described above.

[0095] A plurality of ISFET sensor devices 500 may, for example, be arranged as an ISFET sensor array.

[0096] The ISFET sensor device 500 may include a plurality of transistors PI, P2, P3, Nl . Only four circuit transistors PI, P2, P3, Nl are shown as an example. However, the number of transistors may be less than four (e.g. one, two, three) or may be greater than four and may, for example, be five, six, seven, eight, nine, or tens of transistors.

[0097] The transistor PI (e.g. PMOS transistor PI) may include, or may be, a biasing transistor (namely, a transistor that may provide a biasing current). The transistor P2 (e.g. PMOS transistor P2) may include, or may be, a sensing transistor (namely, a transistor that may sense the ion concentration of an ion-carrying electrolyte). When an ion-carrying electrolyte is introduced at a gate P2g of the transistor P2 (e.g. PMOS transistor P2), a gate potential and a channel current may change in accordance with different ion densities in the ion-carrying electrolyte. In other words, the gate potential and the channel current may vary with the ion concentration of the ion-carrying electrolyte. Stated in another way, the circuit element P2 (e.g. PMOS transistor P2) may function as a sensing unit, which may sense the ion concentration through its gate P2g. The channel current may pass through a transmission switch (which may include, or consist of, transistors P3 and Nl). The channel current may be sent off-chip (i.e. external to the ISFET sensor device 100), e.g. through read out circuitry.

[0098] As described above, the ISFET sensor device 100 may directly measure a small current, which may limit a sensitivity of the ISFET sensor device 100 to the ion concentration of the ion-carrying electrolyte. As described above, the ISFET sensor device 100 may drive the channel current off-chip, which may limit a spatial and/or temporal resolution of the ISFET sensor device 100.

[0099] Accordingly, the existing ion-channel characterization and/or screening methods (e.g. patch-clamp, fluorescence, electrode, ISFET sensor device approaches) may not achieve high throughput and accuracy (e.g. suffer from low temporal resolution and/or low spatial resolution), and may also be prone to false negative and/or false positives errors. As such, in food and drug safety testing, the ion channel pre-screening stage may be skipped, and direct animal testing may be performed, which may be time-consuming and expensive.

[00100] FIG. 6 shows an array of dual-mode pixel elements 600, which may be configured to operate in a first mode and a second mode.

[00101] In the first mode, each dual-mode pixel element of the array of dual-mode pixel elements 600 may generate a first signal representing a pixel value of a photographic image of an ion-carrying electrolyte. In the second mode, each dual-mode pixel element of the array of dual-mode pixel elements 600 may generate a second signal representing a measurement of a concentration of ions in the ion-carrying electrolyte.

[00102] The first mode of the array of dual-mode pixel elements 600 may be referred to as an image mode of the array of dual-mode pixel elements 600. In the image mode, the array of dual- mode pixel elements 600 may be configured to generate a photographic image of the ion- carrying electrolyte (e.g. of an object in the ion-carrying electrolyte).

[00103] The second mode of the array of dual-mode pixel elements 600 may be referred to as a chemical mode of the array of pixel elements 600. In the chemical mode, the array of dual- mode pixel elements 604 may be configured to measure a concentration of ions in the ion- carrying electrolyte.

[00104] The array of dual-mode pixel elements 600 may, for example, be included in an image capture device (e.g. the image capture device 100 shown in FIG. 1, where the array of pixel elements 102 is replaced by the array of dual-mode pixel elements 600).

[00105] In other words, an image capture device may include, or may be, a photographic camera combined with an ion camera. In the first mode, the image capture device may operate as a photographic camera that may take a photographic image of an ion-carrying electrolyte by means of the array of dual-mode pixel elements 600. In the second mode, the image capture device may operate as an ion camera that may take a "chemical image" of an ion-carrying electrolyte, also by means of the array of dual-mode pixel elements 600. The "chemical image" may show a concentration of ions in the ion-carrying electrolyte. For example, the "chemical image" may include, or may be, a pH map of the ion-carrying electrolyte.

[00106] FIG. 7 shows a schematic view 700 of a dual-mode pixel element of the array of dual -mode pixel elements 600 shown in FIG. 6.

[00107] . FIG. 8 shows a cross-sectional view 800 a dual-mode pixel element of the array of dual-mode pixel elements 600 shown in FIG. 6.

[00108] As shown in FIG. 7, the dual-mode pixel element 700 may have a structure that may be similar to a hybrid of a CMOS image sensor (CIS) (e.g. shown in the schematic view 702) and an ISFET sensor device (e.g. shown in the schematic view 708).

[00109] In the first mode, the dual-mode pixel element 700 may be configured to function as a a CMOS image sensor (e.g. as shown in the schematic view 702). The dual-mode pixel element may include a photo-diode 704, which in the first mode of the dual-mode pixel element 700, may be configured to convert light energy (e.g. photons) into electrical charge.

[00110] The dual-mode pixel element 700 may include a switch M6. In the example shown in FIG. 7, the switch M6 may include, or may be, a transistor including a transfer gate 706. In the first mode of the dual-mode pixel element 700, the switch M6 may be configured to provide the electrical charge from the photo-diode 704 of the dual-mode pixel element 700 to a storage device Ml of the dual-mode pixel element 700. The storage device Ml of the dual-mode pixel element 700 may be configured to store the electrical charge provided to it (e.g. by the switch M6). The storage device Ml may generate a voltage based on the electrical charge stored in it. [00111] In the example shown in FIG. 7, the storage device Ml of the dual-mode pixel element 700 may include, or may be, a floating diffusion node FD, and may generate a voltage signal V_FD.

[00112] The voltage generated by the storage device Ml may be readout by a readout device M2, M3. In the example shown in FIG. 7, the readout device M2, M3 may include, or may be, a source follower.

[00113] As described above, the array of dual-mode pixel elements 600 may, for example, replace the array of pixel elements 102 of the image capture device 100. In such an example, each dual-mode pixel element 700 may generate a first signal representing a pixel value of a photographic image of an ion-carrying electrolyte (e.g. shown as reference sign 808 in FIG. 8) that may be contacted (e.g. directly contacted) on a surface (e.g. the surface 804 shown in FIG. 8) of the image capture device 100. Accordingly, a photographic image (e.g. a shadow image) can be captured directly by photographic imaging (e.g. contact photographic imaging) without bulky optical lens.

[00114] In the second mode, the dual-mode pixel element may be configured to function as an ISFET sensor device (e.g. as shown in the schematic view 708). The readout device M2, M3, may include a sensing element M2 configured to generate a voltage, which may be correlated to the ion concentration of the ion-carrying electrolyte. For instance, in the example shown in FIG. 7, the readout device M2, M3 may include a plurality of transistors, and the sensing element M2 may be a transistor of the plurality of transistors, whose gate (e.g. poly-gate) 802 shown in FIG. 8 may be connected to a metal and/or passivation layer 804 (e.g. top metal/passivation layer), which may act as ion-sensitive membrane. [00115] As described above, readout device M2, M3, may include a sensing element M2 configured to generate a voltage, which may be correlated to the ion concentration of the ion- carrying electrolyte. In the example shown in FIG. 7 and FIG. 8, the voltage may include, or may be, a threshold-voltage Vj of the metal and/or passivation layer 804, and may be written as: v —v - m + y ^{φ &} Qox + Qss + Q^p οφ , where V_ref is the potential of the reference-

1 ^Cox

electrode (shown as reference sign 806 in FIG. 8 and reference VREF in FIG. 7), and 0i_nt is the potential across electrolyte-insulator interface that depends on the concentration of ions (e.g. H⁺ ions) of the ion-carrying electrolyte (shown as reference 808 in FIG. 8). As such, V_T may be correlated to the ion concentration (e.g. pH) of the ion-carrying electrolyte.

[00116] A change in the voltage (e.g. V_T) generated by the sensing element M2 of the readout device M2, M3 may lead to a change in an output voltage Vout of a readout element M3 of the readout device M2, M3. For example, when a biasing current of the dual-mode pixel element 700 is fixed, a change in V_T may lead to the change in drain voltage Vout, which may be read out as the output of the pixel element 700. Therefore, the ion concentration (e.g. pH) can be represented by Vout. In other words, the second signal representing a measurement of a concentration of ions in the ion-carrying electrolyte may include, or may be, the signal Vout.

[00117] By generating a first signal representing a pixel value of a photographic image of an ion-carrying electrolyte and a second signal representing a measurement of a concentration of ions in the ion-carrying electrolyte, the dual-mode pixel element may be configured to improve data correlation between the local ion concentration (e.g. pH) and the spatial location of an object (e.g. micro bead, membrane, etc.) in the ion-carrying electrolyte.

[00118] The top-level chip architecture 914 shown in FIG. 9 describes a 64*64 array of dual- mode pixel elements 600. However, in other examples, the array of dual-mode pixel elements 600 may be of other sizes. In the example shown in FIG. 9, the row-readout time may be designed to achieve 13μβ for sampling, amplification and digitization, which may result in a frame rate of 1/(13μ5><64)=1200ίρ5 for a 64*64 array of dual-mode pixel elements 600.

[00119] As described above in relation to FIG. 1, the image capture device 100 may include an amplifier 104. The amplifier 104 may be included in the top-level chip architecture 914 of the image capture device, and may amplify the first and/or second signals. The amplifier 104 may include, or may be, a switched-capacitor amplifier, whose amplifier gain Ci_n/C_f can be variable, and/or selected (e.g. from among a plurality of gains, e.g. IX, 2X, 4X gain), e.g. under different input signal levels with improved sensitivity and dynamic range. The amplifier 104 (e.g. switched-capacitor amplifier) may include an opamp, which may utilize a telescopic topology with open-loop gain of about 68dB and bandwidth of about 628MHz. These values of open-loop gain and bandwidth are illustrative and not meant to be limiting. The resultant high- GBW (gain bandwidth product) may enable a high-speed readout with 10MHz column-wise readout speed.

[00120] As described above in relation to FIG. 1, the image capture device 100 may include a sampler 106 (e.g. analog-to-digital convertor). The sampler 106 (e.g. analog-to-digital converter) may be included in the top-level chip architecture 914 of the image capture device, and may digitize the first and/or second signals. The sampler 106 (e.g. analog-to-digital convertor) shown in FIG. 9 may, for example, implement a 12-bit pipelined ADC, which may include one sample- and-ho Id input stage, ten serially connected 1.5-bit pipeline stages, and one 2-bit flash stage. These values are illustrative and not meant to be limiting. The sampler 106 (e.g. analog-to-digital convertor) may include a digital correction circuit that may generate a 12- bit output code by redundant-signed-digit (RSD). According to post-layout simulation, the maximum differential non-linearity (DNL) may be about 0.44LSB, the maximum integral non- linearity (INL) may be about 0.61LSB, the effective number of bits (ENOB) may be about 1 1.4 bits and the signal to noise and distortion ratio (SNDR) may be about 70.35dB.

[00121] The image capture device 100 including the array of dual-mode pixel elements 600 (e.g. in place of the array of pixel elements 102) may be fabricated in an 0.18μιη CMOS image sensor process with a total area of 2.5x5 mm². An area of the image capture device 100 that may be used for optical sensing (e.g. when the array of dual-mode pixels elements 600 operate in the first mode) may be about 20. Ιμιη² with 18.1% fill factor, and the chemical sensing area (e.g. when the array of dual-mode pixels elements 600 operate in the second mode) may be about 22.3 Ltm with 20.1% fill factor. The power may be about 32mA at 3.3V supply voltage.

[00122] FIG. 10 shows a testing setup for the image capture device 100 including the array of dual-mode pixel elements 600.

[00123] The image capture device 100 including the array of dual-mode pixel elements 600 (e.g. in place of the array of pixel elements 102) may be configured as a chip (e.g. an integrated chip) and may be packaged by liquid-friendly encapsulation with sensing area open. The chip may be integrated with a testing printed circuit board (PCB) 1002, as shown in FIG. 10.

[00124] The image capture device 600 may also be referred to as an ion camera 600, which may be configured to generate a photographic image of the ion-carrying electrolyte and measure a concentration of ions in the ion-carrying electrolyte.

[00125] FIG. 1 1 shows a photographic image 1 102 and a pH map 1 104 generated based on the first and second signals, respectively, generated by the array of dual-mode pixel elements 600. [00126] As shown in FIG. 11, during image mode (i.e. the first mode), the image capture device (e.g. chip) may be able to capture a photographic image 1102 of an ion-carrying electrolyte, e.g. micro-beads in an ion-carrying electrolyte with diameters of 6um, 15um and 23um, e.g. by contact imaging. A distance between the ion-carrying electrolyte (e.g. microbeads in the ion-carrying electrolyte) and the array of dual-mode pixel elements 600 may be less than ΙΟΟμτη. Once the location of micro-beads becomes spatially addressable in image mode (i.e. once the location of micro-beads is known, e.g. from the photographic image 1102), a measurement of a concentration of ions in the ion-carrying electrolyte can be performed, e.g. by means of localized pH measurement in chemical mode with accurate data correlation. In FIG. 1 1, the pH map 1104 may be a 64*64 map of ion concentration (e.g. pH map) and may be generated to correlate the ion concentration (e.g. pH) with the photographic image 1 102. In other words, each pH entry in the pH map 1 104 may be locally associated with one micro-bead of the photographic image 1 102.

[00127] FIG. 12 shows a sensitivity of the image capture device 100 including the array of dual-mode pixel elements 600 to ion concentration in an ion-carrying electrolyte.

[00128] The image capture device 100 including the array of dual-mode pixel elements 600 (e.g. in place of the array of pixel elements 102) may have improved sensitivity to ion concentration and/or photo intensity. When water is used as buffer solution, the pH can be changed by adding HCL and NaOH, thus changing the ion concentration in an ion-carrying electrolyte. With standard 0.18μπι CIS process, the sensitivity (e.g. pH sensitivity) of the image capture device 100 including the array of dual-mode pixel elements 600 may be determined as about 26.2mV/pH on average when output amplifier gain=l (as shown in curve 1202). the sensitivity (e.g. pH sensitivity) of the image capture device 100 including the array of dual- mode pixel elements 600 may be improved to about 103.8mV/pH when output amplifier gain=4 (as shown in curve 1204). FIG. 12 also shows the testing pH (i.e. measure of H+ ion concentration) (e.g. as shown in curves 1206, 1208) result of a bacteria {E. Coli) culture solution with glucose. The measured result by the image capture device 100 (indicated by curve 1206) including the array of dual-mode pixel elements 600 may correlate well with the commercial pH meter (indicated by curve 1208) .

[00129] As described above, based on the fact that an ISFET sensor device has a function similar as the photon sensor (pixel) in digital camera system, an image capture device, which may be referred to as an "ion camera" may be provided. The ion camera may integrate a two- dimensional ISFET pixel array (which may converts the ion information to charge domain) with existing mature and well-developed high-precision high-speed digital camera readout circuit.

[00130] The ion camera may be a unique real-time high-sensitive and high-resolution ion detector based on the readout circuit of existing imager camera. In other words, it can be integrated with general imager sensor product on the market, for example, Aptina MT9P031 (5M digital image sensor), Ominivision OV2640 (2M camera chip), and readout architecture can be either global and column-parallel.

[00131] FIG. 13 shows a simplified schematic of a dual-mode pixel element 1400.

[00132] As shown in FIG. 13, the dual-mode pixel element 1400 may be similar to a CMOS image sensor device (e.g. a 4-transistor (4T) pixel). The presence of ions in an ion-carrying electrolyte may cause a threshold voltage Vt change, which converts the ion concentration information into charge. The charge may be integrated at the node FD and may be read out through in-pixel amplifier 104 (e.g. source follower) and send off-pixel through "output" node. [00133] FIG. 14 shows a comparison of sensitivities of the dual-mode pixel element shown in FIG. 13 and the conventional ISFET sensor device shown in FIG. 5.

[00134] As shown in FIG. 14, the charge domain conversion 1502 (e.g. corresponding to the dual-mode pixel element shown in FIG. 13) can improve the sensitivity by integrating the current continuously. The sensitivity can be improved up-to 20x more compared to direct reading of the current 1504 (e.g. corresponding to the conventional ISFET sensor device shown in FIG. 5).

[00135] FIG. 15A to FIG. 15D show images generated from a measurement of a concentration of ions in the ion-carrying electrolyte.

[00136] With the proposed image capture device including the array of dual-mode pixel elements, a still-image with ion-density distribution as well as a real-time-image showing ion- flux movement may be obtained.

[00137] FIG. 15A shows a two-dimensional ion density mapping with 16x16 sensor arrays. Different grayscales represent different ion densities, therefore the grayscale map may able to represent the ion density map or ion concentration map. The proposed image capture device including the array of dual-mode pixel elements not only captures the static ion concentration information (e.g. represented as a picture) but also the dynamic information (e.g. represented as a video). FIG. 15C shows the image of ion flux movement at different time periods, where FIG. 15B and FIG. 15D indicates the physical ion movement on the surface of proposed image capture device including the array of dual-mode pixel elements.

[00138] The proposed image capture device including the array of dual-mode pixel elements may allow for fast detection by integrating existing well-developed high-speed high-precision CMOS image sensor (digital camera) readout circuit directly on sensor array chip. Instead of reading through a single off-chip, a shared amplifier 104 and analog-to-digital converter 106 may be used. The proposed image capture device including the array of dual-mode pixel elements may include dedicated column-wise amplifiers and ADCs on-chip, which improve the speed at least 100X.

[00139] Instead of directly measuring small current like the conventional ISFET sensor device shown in FIG. 5, the proposed image capture device including the array of dual-mode pixel elements may convert the current into charge domain and may read the charge out through CMOS image sensor readout method. This charge domain measurement can improve the sensitivity by 20X. On-chip low noise readout circuit design can further increase sensitivity by improving the signal-to-noise ratio.

[00140] The conventional ISFET sensor device may have a small array size due to the limitation of readout circuit. However, the proposed image capture device including the array of dual-mode pixel elements solution can support at least an 96x96 array, and hence the spatial resolution can be extended with 100X improvement.

[00141] The proposed image capture device including the array of dual-mode pixel elements can detect ion distribution and concentration in an ion-carrying electrolyte and movement in real-time. As a result, the proposed image capture device including the array of dual-mode pixel elements may be capable to fast detect variety types of ions with a wide range of applications such as food and drug safety tests.

[00142] Compared to existing ion detection systems, the proposed image capture device including the array of dual-mode pixel elements may be the first system-on-chip (SoC) solution that may provide real-time, high-sensitivity and high-resolution photographic images as well as measurements of an ion concentration of an ion carrying electrolyte. [00143] The proposed image capture device including the array of dual-mode pixel elements may improve the sensitivity 20X by adapting charge-domain accumulation method which is effective in CMOS image sensor pixel design, therefore the dual-mode pixel element may be able to measure the ion density into ppm level. Nevertheless, with integrated on-chip readout architecture, real-time measurement can be carried out with 95% consistency. In addition, with two-dimensional ISFET-sensor array instead of single ISFET-sensor, accurate spatial information can be obtained with 1 OOx improvement of resolution.

[00144] . The proposed image capture device including the array of dual-mode pixel elements may shows a promising solution to compete for the market of food safety and drug pre- screening.

[00145] The proposed image capture device including the array of dual-mode pixel elements may be widely applied. One of applications is to detect ion-channel behavior based on the observation of large demanding on accurate ion channel detector at both food safety and drug discovery market. One possible embodiment of the proposed image capture device is shown in FIG. 16.

[00146] FIG. 16 shows an image capture system 1700.

[00147] The image capture system 1700 may include an image capture device 1702, a container 1704 configured to hold an ion carrying electrolyte and a cavity 1706 configured to contain a sample. The image capture device 1702 may include, or may be, the image capture device including the array of dual -mode pixel elements, as described above. The cavity 1706 may face the image capture device 1702 (e.g. face the array of dual-mode pixel elements).

[00148] The image capture system 1700 can provide a solution to resolve the aforementioned technical challenges for both food and drug safety. The cavity 1706 may hold a sample, e.g. a cell, e.g. brain cell or nerve cell. When the testing drug/food is introduced into an ion-carrying electrolyte that may be contained in the container 1704, the ion density of the ion-carrying electrolyte may change accordingly by in-flux or out-flux through cell ion channels in the sample. This can be measured and observed by the image capture device disposed below the cavity 1706 in a real-time fashion as well.

[00149] Food safety market has been witnessed an unprecedented level of growth in the past few years. Rising consumer awareness and stringent legal requirements by government are among the key factors driving the demand for effective food safety testing globally. As a result, both developed and developing countries are investing much more money into this market to ensure healthy food for their citizens. For example, the overall food safety testing market size in USA is estimated around 2-billion with estimated growth at 10%.

[00150] For the food safety testing, there is a gap between the food that we test and the food that we eat due to lack of fast and accurate prescreening tool. Government food safety and inspection agency has to randomly select the food sample on the market and do a complicated test to examine the compound. Less than 1% of the world food is tested despite billions of dollars in losses through product-recall, food poisoning, prophylactic-measures and food-borne illness. Ion channels should be considered as preliminary targets in food safety toxicological studies. Fast and accurate ion-channel prescreening tool is strongly demanding at food safety testing market as a potential early indicator of life-threatening disease conditions.

[00151] Ion channels may be proteins on the membrane of cells which allow the flow of ions into the cell. They are highly intriguing biophysical entities that play an incredibly subtle role inter- and intra-cellular communication. Regulated and selective transport of ions mediated by ion channels underpins numerous fundamental physiological processes. This includes electrical signaling in the heart and the nervous system, fluid secretion in the lung, and a variety of other key processes such as hormone secretion, the immune response, and tumor cell proliferation. Therefore, ion channels are crucial for the vitality of all living organisms and hence become the important drug targets. Despite being such a rich source of drug targets with attention from the pharmaceutical industry, ion channels are traditionally difficult to characterize or screen for both drug discovery and safety testing. As a result, the existing ion channel drugs were discovered and tested by traditional tissue- or animal-based pharmacological methods without any knowledge of their molecular-level targets. The animal study costs around $300K and 6 month for identifying one drug candidate. Nevertheless, without accurate pre- screening tools, the drug candidates are randomly chosen during animal test. As such, the success-rate is unpredictable and the side-effect is unsafe to verify. Therefore, an accurate pre-screening tool, such as the proposed image capture device including the array of dual-mode pixel elements can save million dollars per year for drug companies.

[00152] According to various examples presented herein, an image capture device is provided. The image capture device may include: an array of dual-mode pixel elements configured to operate in a first mode and a second mode, wherein, in the first mode, each dual- mode pixel element generates a first signal representing a pixel value of a photographic image of an ion-carrying electrolyte, and wherein, in the second mode, each dual-mode pixel element generates a second signal representing a measurement of a concentration of ions in the ion- carrying electrolyte.

[00153] The image capture device may further include an amplifier coupled to the array of dual-mode pixel elements, wherein the amplifier is configured to amplify the first and second signals of each dual-mode pixel element. [00154] The amplifier may include, or may be, a single-ended switch-capacitance amplifier.

[00155] The amplifier may include, or may consist of, at least one of an operational amplifier, a transistor a resistor, a capacitor, and a diode.

[00156] A gain of the amplifier may vary with a lighting condition to which each dual-mode pixel element is exposed to.

[00157] The image capture device may further include a sampler configured to sample the first and second signals.

[00158] . The sampler may be configured to sample the first and second signals subsequent to an amplification of the first and second signals.

[00159] The sampler may include, or may be, an analog-to-digital convertor.

[00160] The sampler may be configured to perform correlated double sampling of the first and second signals relative to a reference signal.

[00161] The image capture device may further include a memory configured to store the first and second signals.

[00162] The memory may be configured to store the first and second signals subsequent to a sampling of the first and second signals.

[00163] The image capture device may further include a selector configured to select at least one of the first and second signals.

[00164] The selector may be configured to select the first and second signals stored in a memory.

[00165] The selector may be further configured to readout at least one of the selected first and second signals. [00166] The selector may be further configured to provide at least one of the selected first and second signals to a processing circuit configured to process the first and second signals.

[00167] The image capture device may further include a processing circuit configured to increase a spatial resolution of a photographic image whose pixel value is represented by the first signal and to increase a spatial resolution of a second image whose pixel value is represented by the second signal.

[00168] Each dual-mode pixel element may include, or consist of, a photo-diode, which in the first mode, is configured to convert light energy into electrical charge.

[00169] Each dual-mode pixel element may further include a switch, which in the first mode, is configured to provide the electrical charge to a storage device.

[00170] Each dual-mode pixel element may include a storage device, which in the first mode, is configured to store an electrical charge.

[00171] The storage device may include, or may be, a floating diffusion node.

[00172] The storage device may be configured to generate a first voltage from the electrical charge, wherein the first voltage is readout by a readout device as the first signal.

[00173] Each dual-mode pixel element may include a readout device, which in the first mode, is configured to readout a first voltage stored in a storage device as the first signal.

[00174] The readout device may include, or may consist of, a sensing element and a readout element.

[00175] The sensing element, in the second mode, may be configured to generate a second voltage which is correlated to the concentration of ions in the ion-carrying electrolyte.

[00176] The second voltage may be configured to generate an output voltage at an output of the readout element, wherein the second signal may include, or may be, the output voltage. [00177] According to various examples presented herein, an image capture system may be provided. The image capture system may include: an image capture device, including: an array of dual-mode pixel elements configured to operate in a first mode and a second mode, wherein, in the first mode, each dual-mode pixel element generates a first signal representing a pixel value of a photographic image of an ion-carrying electrolyte, and wherein, in the second mode, each dual-mode pixel element generates a second signal representing a measurement of a concentration of ions in the ion-carrying electrolyte; and a container configured to hold the ion- carryingdectrolyte.

[00178] The image capture system may further include a cavity configured to contain sample, wherein the sample changes the concentration of ions in the ion-carrying electrolyte.

[00179] The image capture device of the image capture system may further include an amplifier coupled to the array of dual-mode pixel elements, wherein the amplifier is configured to amplify the first and second signals of each dual-mode pixel element.

[00180] The image capture device of the image capture system may further include a sampler configured to sample the first and second signals.

[00181] The image capture device of the image capture system may further include a memory configured to store the first and second signals.

[00182] The memory may be configured to store the first and second signals subsequent to a sampling of the first and second signals.

[00183] The image capture device of the image capture system may further include a selector configured to select at least one of the first and second signals.

[00184] The selector may be configured to select the first and second signals stored in a memory. [00185] The selector may be further configured to readout at least one of the selected first and second signals.

[00186] The selector may be further configured to provide at least one of the selected first and second signals to a processing circuit configured to process the first and second signals.

[00187] The image capture device of the image capture system may further include a processing circuit configured to increase a spatial resolution of a photographic image whose pixel value is represented by the first signal and to increase a spatial resolution of a second image whose pixel value is represented by the second signal.

[00188] While various aspects have been particularly shown and described with reference to these aspects of this disclosure, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the disclosure as defined by the appended claims. The scope of the disclosure is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced.

Claims

Claims What is claimed is:

1. An image capture device comprising:

an array of dual-mode pixel elements configured to operate in a first mode and a second mode,

wherein, in the first mode, each dual-mode pixel element generates a first signal representing a pixel value of a photographic image of an ion-carrying electrolyte, and wherein, in the second mode, each dual-mode pixel element generates a second signal representing a measurement of a concentration of ions in the ion-carrying electrolyte.

2. The image capture device of claim 1, further comprising an amplifier coupled to the array of dual-mode pixel elements, wherein the amplifier is configured to amplify the first and second signals of each dual-mode pixel element.

3. The image capture device of claim 2, wherein the amplifier comprises a single-ended switch-capacitance amplifier.

4. The image capture device of claim 2, wherein the amplifier comprises at least one of an operational amplifier, a transistor a resistor, a capacitor, and a diode.

5. The image capture device of claim 2, wherein a gain of the amplifier varies with a lighting condition to which each dual-mode pixel element is exposed to.

6. The image capture device of claim 1, further comprising a sampler configured to sample the first and second signals.

7. The image capture device of claim 6, wherein the sampler is configured to sample the first and second signals subsequent to an amplification of the first and second signals.

8. The image capture device of claim 6, wherein the sampler comprises an analog-to-digital converter.

9. The image capture device of claim 6, wherein the sampler is configured to perform

correlated double sampling of the first and second signals relative to a reference signal.

10. The image capture device of claim 1, further comprising a memory configured to store the first and second signals.

1 1. The image capture device of claim 10, wherein the memory is configured to store the first and second signals subsequent to a sampling of the first and second signals.

12. The image capture device of claim 1, further comprising a selector configured to select at least one of the first and second signals.

13. The image capture device of claim 12, wherein the selector is configured to select the first and second signals stored in a memory.

14. The image capture device of claim 12, wherein the selector is further configured to

readout at least one of the selected first and second signals.

15. The image capture device of claim 12, wherein the selector is further configured to

provide at least one of the selected first and second signals to a processing circuit configured to process the first and second signals.

16. The image capture device of claim 1, further comprising a processing circuit configured to increase a spatial resolution of a photographic image whose pixel value is represented by the first signal and to increase a spatial resolution of a second image whose pixel value is represented by the second signal.

17. The image capture device of claim 1, wherein each dual-mode pixel element comprises a photo-diode, which in the first mode, is configured to convert light energy into electrical charge.

18. The image capture device of claim 17, wherein each dual-mode pixel element further comprises a switch, which in the first mode, is configured to provide the electrical charge to a storage device.

19. The image capture device of claim 1, wherein each dual-mode pixel element comprises a storage device, which in the first mode, is configured to store an electrical charge.

20. The image capture device of claim 19, wherein the storage device comprises a floating diffusion node.

21. The image capture device of claim 19, wherein the storage device is configured to

generate a first voltage from the electrical charge, wherein the first voltage is readout by a readout device as the first signal.

22. The image capture device of claim 1, wherein each dual-mode pixel element comprises a readout device, which in the first mode, is configured to readout a first voltage stored in a storage device as the first signal.

23. The image capture device of claim 22, wherein the readout device comprises a sensing element and a readout element.

24. The image capture device of claim 23, wherein, in the second mode, the sensing element is configured to generate a second voltage which is correlated to the concentration of ions in the ion-carrying electrolyte.

25. The image capture device of claim 24, wherein the second voltage is configured to generate an output voltage at an output of the readout element, wherein the second signal comprises the output voltage.

26. An image capture system, comprising:

an image capture device, comprising:

wherein, in the first mode, each dual-mode pixel element generates a first signal representing a pixel value of a photographic image of an ion-carrying electrolyte, and wherein, in the second mode, each dual-mode pixel element generates a second signal representing a measurement of a concentration of ions in the ion- carrying electrolyte; and

a container configured to hold the ion-carrying electrolyte.

27. The image capture system of claim 26, further comprising a cavity configured to contain sample, wherein the sample changes the concentration of ions in the ion-carrying electrolyte.

28. The image capture system of claim 26, wherein the image capture device further

comprises an amplifier coupled to the array of dual-mode pixel elements, wherein the amplifier is configured to amplify the first and second signals of each dual-mode pixel element.

29. The image capture system of claim 26, wherein the image capture device further comprises a sampler configured to sample the first and second signals.

30. The image capture system of claim 26, wherein the image capture device further comprises a memory configured to store the first and second signals.

31. The image capture system of claim 30, wherein the memory is configured to store the first and second signals subsequent to a sampling of the first and second signals.

32. The image capture system of claim 26, wherein the image capture device further comprises a selector configured to select at least one of the first and second signals.

33. The image capture system of claim 32, wherein the selector is configured to select the first and second signals stored in a memory.

34. The image capture system of claim 32, wherein the selector is further configured to readout at least one of the selected first and second signals.

35. The image capture system of claim 32, wherein the selector is further configured to provide at least one of the selected first and second signals to a processing circuit configured to process the first and second signals. The image capture system of claim 26, wherein the image capture device further comprises a processing circuit configured to increase a spatial resolution of a photographic image whose pixel value is represented by the first signal and to increase a spatial resolution of a second image whose pixel value is represented by the second signal.