CN110753221B - Real-time correction system for serial image data training of CMOS image sensor - Google Patents

Abstract

A real-time correction system for serial image data training of a CMOS image sensor relates to a real-time correction system for serial image data training of a CMOS image sensor and solves the problems that the prior system has data sampling errors and the like due to the change of external environment temperature or working voltage, wherein a serial data jumping edge changes along a relative position; the output serial image data is converted into stable parallel data determined by the output effective data position through the iodelay, the iserdes and the shift register with controllable output position controlled by the training module, and then the image data format meeting the application requirement is output through the data conditioning module. The real-time correction system can ensure stable and reliable image output by monitoring and adjusting in the line blanking period, and does not interrupt the normal shooting task.

Description

Real-time correction system for serial image data training of CMOS image sensor

Technical Field

The invention relates to a serial image data training and real-time correction system of a CMOS image sensor, in particular to a serial image data training and real-time correction method of the CMOS image sensor applied to a rolling shutter and a linear array detector.

Background

After the CMOS detector is just powered on and trained, random codes can appear in some channel images occasionally; and training is carried out again, so that the acquired image is stable. The reason for this is that the temperature of the detector itself is different just after power-up and working for a period of time, so that the relative delay of the detector output serial data changes, and the change of the relative phase can be ignored after the temperature reaches balance. The timing margin for detector training depends primarily on the frequency of the serial image data and the magnitude of the temperature change of the detector. In the aspect of guarantee measures of training, an idelay _ ctrl module is arranged in Virtex 5 and higher-level FPGAs, delay variation caused by working voltage and temperature variation of the FPGAs can be corrected, and therefore adaptability adjustment of temperature variation of the detectors becomes a key.

Disclosure of Invention

The invention provides a real-time correction system for serial image data training of a CMOS image sensor, aiming at solving the problems that in the prior art, due to the change of external environment temperature or working voltage, the relative position of serial data jumping is changed, and further data sampling errors are caused.

The serial image data training and real-time correction system of the CMOS image sensor comprises the CMOS image sensor, a driver, a level shifter and a single chip controller; the single chip controller comprises a time sequence control module, a data conditioning module, a training module, a variable delayer (iodelay), a controllable serial-parallel converter (iserdes) and a shift register with controllable output position; the time sequence control module in the single chip controller outputs a driving time sequence signal and a control time sequence signal for charge transfer, and the driving time sequence signal and the control time sequence signal are respectively sent to the CMOS image sensor after passing through the driver and the level converter; serial CMOS image data output by the CMOS image sensor are converted into stable parallel data determined by an output effective data position through an iodelay, iserdes and a shift register with controllable output position controlled by a training module, and then the stable parallel data are output in an image data format meeting application requirements after passing through a data conditioning module.

The invention has the beneficial effects that:

firstly, the real-time correction system can detect the change of the relative position and then carry out adaptive adjustment.

The real-time correction system can acquire the relative delay change rule of each channel of the detector in the imaging process through real-time monitoring;

the real-time correction system can ensure stable and reliable image output by monitoring and adjusting in the line blanking period, and does not interrupt the normal shooting task.

Drawings

FIG. 1 is a block diagram of a serial image data training and real-time calibration system for a CMOS image sensor according to the present invention;

FIG. 2 is a timing diagram of the CMOS driving for each row;

FIG. 3 is a flow chart of full tap detection;

FIG. 4 is a flow chart of detection based on the sampling eye center position;

fig. 5 is a flow chart of detection of edge positions based on transitions.

Detailed Description

In a first embodiment, the present embodiment is described with reference to fig. 1 to 5, in which a serial image data training and real-time correction system of a CMOS image sensor includes a CMOS image sensor, a driver, a level shifter and a single chip controller; the single chip controller comprises a time sequence control module, a data conditioning module, a training module, an iodelay, iserdes and a shift register with controllable output position; the time sequence control module in the single chip controller outputs a driving time sequence signal and a control time sequence signal for charge transfer, and the driving time sequence signal and the control time sequence signal are respectively sent to the CMOS image sensor after passing through the driver and the level converter; serial CMOS image data output by the CMOS image sensor are converted into stable parallel data determined by an output effective data position through an iodelay, iserdes and a shift register with controllable output position controlled by a training module, and then the stable parallel data are output in an image data format meeting application requirements after passing through a data conditioning module.

In this embodiment, the stability detection of the CMOS serial image data channel is performed sequentially channel by channel, and the detection is performed only during the redundant blanking period of each line; the redundant blanking period of each line is only detected for 1 or more channels; the number of the circulating lines is the total number of channels/p of the CMOS serial image data connected with the single-chip FPGA. p is an integer greater than 0.

As shown in fig. 2, the driving timing of each row of CMOS is set to: the charge uniform transfer operation is performed in one row while including an effective readout timing operation in one row and an additionally added redundancy timing operation. The charge transfer operation is continued for the entire row period; in an effective readout timing operation stage, the train signal is set to a low level; in the redundant time sequence operation stage (blanking period) of each row, the train signal is set to be high level, the delay of the iodelay is carried out, and the stability detection of the sampled parallel data is carried out (whether the data is stable or not is detected in a specified range, if not, the offset adjustment of the sampling position is required); if the data is detected to be stable or the jumping position of the data is not changed, the change of the primary iodelay position is not needed; if instability is detected or the position of the data transition is not changed, the increase or decrease of the iodelay position is required.

The timing overhead for parallel data stability detection and adaptive adjustment in a row must be less than the time t of the redundant timing operation stage_blank：

t_{sync_rising_delay}+t_{sync_falling_delay}+n_tap×t_{stability_check}+t_{delay_set}≤t_blankt_{sync_rising_delay}Delay from rising edge of train to LVDS output training word; t is t_{sync_falling_delay}Delay for the falling edge of train to LVDS output the photosensitive image; t is t_{stability_check}Delay detection time for tap; n is_tapThe number of delay taps for the used iododelay; t is t_{delay_set}The time for the final delay value is set for iodelay.

In this embodiment, performing circuit delay by using the resource iodelay inside the FPGA divides the input reference clock period into a plurality of portions, where the length corresponding to each portion is the length of a single tap. The n tap values represent the total length of the n sections after the delay lengths are superposed.

In this embodiment, three modes of full tap detection (if the blanking time is long enough, parallel data received at each tap position, for example, 64 positions can be compared with the training word, and a bit correction process of the entire training process can be realized), detection based on the center position of the sampling eye (if the blanking time is short, comparison is performed only in a limited number of taps, for example, 10 taps, whose centers are the center of the sampling eye), and detection based on the transition edge position (if the blanking time is short, comparison is performed only in a limited number of taps, for example, 10 taps, whose centers are the unstable positions of the original sampling) can be set according to the length of the blanking period.

In the embodiment described with reference to fig. 3, the procedure of the total tap detection is as follows: when receiving the detection command, first, the IDLE phase subchannel _ IDLE _ state, which does not perform any operation, jumps to the pre-detection RESET phase subchannel _ RESET _1_ state, and performs RESET (delayed RESET to a default value) of the current channel IODELAY 1. And then jumping to a data stability detection stage subhannle _ EYE _ SAMPLE _ state, continuously sampling for r times to detect whether the acquired data is stable, and finally giving a conclusion of whether the acquired data is stable. Then entering a sampling EYE position judgment stage subhangle _ EYE _ CHECK _ state, and initializing and increasing the tap position count; judging the tap direction after detecting the unstable region; judging the state of the detected jumping edge; and judging the width of the stable sampling eye. When all taps are detected to be traversed or the length q greater than the stable sampling region of the taps is detected (the stable sampling region of the taps is greater than q, the stable sampling region is considered to be stable, and q is an integer greater than 3) and a jump edge is detected, jumping to a sampling EYE position calculation stage subhannle _ EYE _ CALC _ state, calculating the middle position of the sampling EYE, otherwise jumping to a DELAY stage subhannle _ EYE _ DELAY _ state, and delaying the iododelay by one tap; the DELAY stage subhandshake _ EYE _ DELAY _ state jumps to the sampling EYE position decision stage subhandshake _ EYE _ CHECK _ state. And jumping to a post-detection RESET stage Subchanle _ RESET _2_ state from a sampling EYE position calculation stage Subchanle _ EYE _ CALC _ state, resetting the current channel iodelay, and delaying to RESET to a default value. Then jump to the final sample EYE position setting stage subchanle _ EYE _ CENTER _ state, the iodelay delay value is set to the middle position of the optimal sample EYE. Finally, the transition is made back to the IDLE phase subhannle _ IDLE _ state.

The process of detecting the center position of the sampling eye according to this embodiment will be described with reference to fig. 4Comprises the following steps: when receiving the detection command, firstly jumping from the IDLE stage, i.e. IDLE _ state, in which no operation is performed, to the reset stage, i.e. pre-detection _ delay _1_ state, the tap delay value of the iodelay is set to be the current (tap value) plus or minus n_tap. And then jumping to a data stability detection stage subhannle _ EYE _ SAMPLE _ state, continuously sampling for r times to detect whether the acquired data is stable, and finally giving a conclusion of whether the acquired data is stable. Then entering a sampling EYE position judgment stage subhangle _ EYE _ CHECK _ state, and counting and increasing the tap position; judging the state of the detected jumping edge; and (5) performing tap position recording of unstable positions. When the detection traverses 2n +1 taps or a stable sampling region of a number with the length larger than a preset tap is detected and a jump edge is detected, jumping to a sampling EYE position calculation stage subhandshake _ EYE _ CALC _ state, calculating the middle position of a sampling EYE, otherwise jumping to a DELAY stage subhandshake _ EYE _ DELAY _ state, and performing DELAY of one tap by using the iododelay; the DELAY stage subhandshake _ EYE _ DELAY _ state jumps to the sampling EYE position decision stage subhandshake _ EYE _ CHECK _ state. Jumping from the sampling EYE position calculation stage subcrank _ EYE _ CALC _ state to the final sampling EYE position setting stage subcrank _ EYE _ CENTER _ state, the iodelay delay value is set to the middle position of the optimal sampling EYE. Finally, the transition is made back to the IDLE phase subhannle _ IDLE _ state.

The process of detecting the position of the jumping edge according to the present embodiment is described with reference to fig. 5 as follows: when receiving a detection command, firstly jumping from an IDLE stage, i.e. not doing any operation, of subhangle _ IDLE _ state to a pre-detection reset stage, i.e. subhangle _ delay _1_ state, the tap delay value of the iodelay is set as the current (tap value) plus or minus n_tap. And then jumping to a data stability detection stage subhannle _ EYE _ SAMPLE _ state, continuously sampling for r times to detect whether the acquired data is stable, and finally giving a conclusion of whether the acquired data is stable. Then entering a sampling EYE position judgment stage subhangle _ EYE _ CHECK _ state, and counting and increasing the tap position; judging the state of the detected jumping edge; and (5) performing tap position recording of unstable positions. When the detection traverses 2n +1 taps or a number stable sampling region with a length greater than a predetermined tap has been detected, and a transition edge has been detectedJumping to a sampling EYE position calculation stage subhandshake _ EYE _ CALC _ state, calculating the middle position of the sampling EYE, otherwise jumping to a DELAY stage subhandshake _ EYE _ DELAY _ state, and performing a tap DELAY by the iodelay; the DELAY stage subhandshake _ EYE _ DELAY _ state jumps to the sampling EYE position decision stage subhandshake _ EYE _ CHECK _ state. Jumping from the sampling EYE position calculation stage subcrank _ EYE _ CALC _ state to the final sampling EYE position setting stage subcrank _ EYE _ CENTER _ state, the iodelay delay value is set to the middle position of the optimal sampling EYE. Finally, the transition is made back to the IDLE phase subhannle _ IDLE _ state.

In this embodiment, the method further includes a processing method after detecting the full tap, detecting the center position of the sampling eye, and detecting the position of the jumping edge:

the processing mode after the detection of the full tap is the same as that of the prior art, and the processing mode of the prior patent of 'a new training method for multi-channel low-frequency CMOS serial image data' and 'a training method for multi-channel low-frequency CMOS serial image data' can be combined.

For the detection of 2n +1 taps near the center position of the sampling eye, if an unstable position (equal to a training word) is not detected, the original setting position is unchanged; if the unstable position is detected, moving the central sampling position according to the difference value of the original central sampling position and the unstable position; if the unstable value is smaller than the tap value, the delay value is increased by the distance between the unstable value and the tap value; if the tap value is large, the delay value is reduced by the distance between the two.

n_{tap_now}＝n_{tap_before}+n_{tap_change1}；n_{tap_now}Delay tap, n set for current iodelay_{tap_before}Delay tap, n set for last time training iodelay_{tap_change1}Is the difference between the sampled position and the unstable position.

For the detection of 2n +1 taps near the position of the jumping edge, comparing the relative change of the detected position of the jumping edge, and if no change exists, keeping the delay value of the iodelay unchanged; if the value obtained by comparison is larger, the value is increased gradually, and the value is decreased and decreased gradually. n is_{tap_now}＝n_{tap_before}+n_{tap_change}；n_{tap_now}Delay t set for current iodelayap，n_{tap_before}Delay tap, n set for last time training iodelay_{tap_change}Two detected transitions vary along the edge.

In the blanking period of imaging, the real-time correction system of the embodiment allows the detector to be in a training state, monitors the relative change of the unstable sampling position of the serial data, and then corrects the sampling position. Aiming at the application of a rolling shutter and a linear array detector, in a redundant time sequence stage, a train signal is pulled up, then, delay of iodelay is carried out, whether received parallel data are equal to training words or not is judged, and the range equal to the training words is counted. From the perspective of reducing the temperature deviation between the training state when the detector is just powered on and the normal imaging. From the angle of time sequence control, the time-sharing and step-by-step electrification of various power supplies is not carried out, but all the power supplies are electrified simultaneously when electrified; meanwhile, in the training stage of just powering on, the driving and control time sequence signal is not in a default fixed level state, but is in a high-low level alternating state of normal time sequence output, so that the power consumption during just powering on is improved.

The CMOS image sensor described in this embodiment is a TDI CMOS image sensor of long-photospeed core company; the driver adopts EL7457 of Intersil company; the level shifter adopts 54AC163245 of ST company; the monolithic controller employs a Virtex 5 FPGA from Xilinx.

Claims (7)

1. The real-time correction system for serial image data training of a CMOS image sensor comprises the CMOS image sensor, a driver, a level shifter and a single chip controller; the single chip controller comprises a time sequence control module, a data conditioning module, a training module, a variable delayer, a controllable serial-parallel converter iserdes and a shift register with controllable output position; the method is characterized in that:

   the time sequence control module in the single chip controller outputs a driving time sequence signal and a control time sequence signal for charge conversion, and the driving time sequence signal and the control time sequence signal for charge conversion are respectively sent to the CMOS image sensor after passing through the driver and the level converter; the CMOS image sensor outputs serial CMOS image data, the serial CMOS image data are converted into stable parallel data determined by an output effective data position through an iodelay, an iser and a shift register with controllable output position controlled by a training module, and the parallel data output image data meeting application requirements after passing through a data conditioning module;

   the stability detection of the serial CMOS image data channel is carried out in a channel-by-channel sequence mode, and detection is carried out in a redundant blanking period of each line;

   the driving timing of each line of CMOS image data is set to: the charge transfer operation is continued for the entire row period; in an effective readout timing operation stage, the train signal is set to a low level; in the redundant time sequence operation stage of each row, a train signal is set to be at a high level, delay of iodelay is carried out, and stability detection of sampled parallel data is carried out; if the data is detected to be stable or the jumping position of the data is not changed, the original iodelay position is not changed; if the data is detected to be unstable or the jumping position of the data is not changed, increasing or decreasing the iodelay position;

   the single chip controller is set to be full tap detection, detection according to the center position of a sampling eye and detection according to the position of a jumping edge according to the length of a blanking period;

   the specific process of the full tap detection is as follows: after receiving a detection command, jumping from an IDLE stage, which does not perform any operation, to a pre-detection RESET stage, namely, subidle _ IDLE _ RESET _1_ state, of a current channel IODELAY1, resetting the current channel IODELAY1, then jumping to a data stability detection stage, namely, subidle _ EYE _ SAMPLE _ state, detecting whether acquired data is stable or not through continuous sampling for r times, finally giving a result of stability or not, entering a sampling EYE position judgment stage, namely, subidle _ EYE _ CHECK _ state, initializing and increasing the count of tap positions, judging the tap direction after detecting an unstable region, judging the detection jumping edge state, and judging the stable sampling EYE width; when all taps are detected to be traversed or the length is detected to be larger than the length of a tap stable sampling region and a jump edge is detected, jumping to a sampling EYE position calculation stage subhannle _ EYE _ CALC _ state, calculating the middle position of a sampling EYE, otherwise jumping to a DELAY stage subhannle _ EYE _ DELAY _ state, and delaying the position of one tap; jumping to a sampling EYE position judgment stage subhandshake _ EYE _ CHECK _ state from a DELAY stage subhandshake _ EYE _ DELAY _ state, jumping to a post-detection RESET stage Subchankle _ RESET _2_ state from a sampling EYE position calculation stage subhandshake _ EYE _ CALC _ state, resetting the current channel iodelay, and delaying to RESET to a default value; jumping to a final sampling EYE position setting stage subhangle _ EYE _ CENTER _ state, and setting an iododelay value as the middle position of the optimal sampling EYE; finally, the transition is made back to the IDLE phase subhannle _ IDLE _ state.
2. 2. The serial image data trained real-time correction system of a CMOS image sensor of claim 1, wherein: the redundant blanking period of each line is detected only for one or more channels; the number of circulating lines is the total number of channels/p of the CMOS serial image data connected with the single-chip FPGA, and p is an integer larger than 0.
3. 3. The serial image data trained real-time correction system of a CMOS image sensor of claim 1, wherein: the timing overhead for setting the stability detection and adaptive adjustment of parallel data in a row must be less than the time t of the redundant timing operation stage_blank：

   t_{sync_rising_delay}+t_{sync_falling_delay}+n_tap×t_{stability_check}+t_{delay_set}≤t_blank

   t_{sync_rising_delay}Delay from the rising edge of the train signal to LVDS output training words; t is t_{sync_falling_delay}Delay from the falling edge of the train signal to the LVDS output photosensitive image; t is t_{stability_check}Delay detection time for tap; n is_tapThe number of delay taps for the used iododelay; t is t_{delay_set}The time for the final delay value is set for iodelay.
4. 4. Serial of CMOS image sensors according to claim 1Image data training and real-time correction system, its characterized in that: the process of detecting according to the central position of the sampling eye comprises the following steps: when receiving a detection command, firstly jumping from an IDLE stage, i.e. not doing any operation, of subhangle _ IDLE _ state to a pre-detection reset stage, i.e. subhangle _ delay _1_ state, setting the tap delay value of the iodelay to be the current tap value plus or minus n_tap(ii) a Then jumping to a data stability detection stage subhangle _ EYE _ SAMPLE _ state, continuously sampling for r times to detect whether the acquired data is stable, and finally giving a result of whether the acquired data is stable; then entering a sampling EYE position judgment stage subhangle _ EYE _ CHECK _ state, and counting and increasing the tap position; judging the state of the detected jumping edge; performing tap position recording of unstable positions; when the detection traverses 2n +1 taps or a stable number sampling region with the length larger than the preset tap is detected and a jump edge is detected, jumping to a sampling EYE position calculation stage subhandshake _ EYE _ CALC _ state, calculating the middle position of a sampling EYE, otherwise jumping to a DELAY stage subhandshake _ EYE _ DELAY _ state, and performing DELAY of one tap by using the iododelay; jumping to a sampling EYE position judgment stage subhandshake _ EYE _ CHECK _ state from a DELAY stage subhandshake _ EYE _ DELAY _ state; jumping from a sampling EYE position calculation stage subcrank _ EYE _ CALC _ state to a final sampling EYE position setting stage subcrank _ EYE _ CENTER _ state, and setting an iododelay value as the middle position of an optimal sampling EYE; finally, the transition is made back to the IDLE phase subhannle _ IDLE _ state.
5. 5. The system of claim 1, wherein: the process of detecting according to the position of the jumping edge is as follows: when receiving a detection command, firstly jumping from an IDLE stage, i.e. not doing any operation, of subhangle _ IDLE _ state to a reset stage, i.e. subhangle _ delay _1_ state before detection, and setting the tap delay value of the iodelay to be the current tap value plus or minus n_tap(ii) a Then jumping to a data stability detection stage subhangle _ EYE _ SAMPLE _ state, continuously sampling for r times to detect whether the acquired data is stable, and finally giving a conclusion of whether the acquired data is stable; then enters a sampling EYE position judgment stage subcrank _ EYE _ CHECK _ state,counting and increasing the tap position; judging the state of the detected jumping edge; performing tap position recording of unstable positions; when the detection traverses 2n +1 taps or a stable sampling region of a number with the length larger than a preset tap is detected and a jump edge is detected, jumping to a sampling EYE position calculation stage subhandshake _ EYE _ CALC _ state, calculating the middle position of a sampling EYE, otherwise jumping to a DELAY stage subhandshake _ EYE _ DELAY _ state, and performing DELAY of one tap by using the iododelay; jumping to the subhandshake _ EYE _ CHECK _ state in the DELAY phase subhandshake _ EYE _ DELAY _ state; jumping from a sampling EYE position calculation stage subcrank _ EYE _ CALC _ state to a final sampling EYE position setting stage subcrank _ EYE _ CENTER _ state, and setting an iododelay value as the middle position of an optimal sampling EYE; finally, the transition is made back to the IDLE phase subhannle _ IDLE _ state.
6. 6. The serial image data training and real-time correction system of CMOS image sensor as in claim 4, wherein: aiming at the detection of 2n +1 taps at the central position of the sampling eye, if an unstable position is not detected, the central position of the original sampling eye is unchanged; if the unstable position is detected, moving the central sampling position according to the difference value between the central position of the original sampling eye and the unstable position; if the unstable position is smaller than the original sampling eye center position, adding the distance between the new sampling eye center position and the original sampling eye center position; if the unstable position is larger than the original sampling eye center position, subtracting the distance between the unstable position and the original sampling eye center position from the new sampling eye center position; if the unstable position is equal to the center position of the original sampling eye, the full tap detection is needed;

   n_{tap_now}＝n_{tap_before}+n_{tap_change1}；n_{tap_now}delay tap, n set for current iodelay_{tap_before}Delay tap, n set for last time training iodelay_{tap_change1}The difference between the center position of the sampling eye and the unstable position is obtained.
7. 7. The serial image data training and real-time correction system for CMOS image sensor as in claim 5, wherein the detected transition edge position is compared for the detection of 2n +1 taps of transition edge positionThe relative change value of the position, if the value is not changed, the delay value of the iodelay is not changed; if the value obtained by comparison is larger, the delay value of the iodelay is increased progressively, and if the value is smaller, the delay value of the iodelay is decreased progressively; n is_{tap_now}＝n_{tap_before}+n_{tap_change}；n_{tap_change}The change value is detected for two jump edges.

Images