CN215871605U - Multi-view imaging circuit with self-adaptive cache - Google Patents

Abstract

The utility model provides a multi-view imaging circuit with self-adaptive cache, which solves the problems of low integration level and high power consumption of the existing multi-view imaging system; the upper computer software is required to cooperatively control the multiple independent cameras, so that the development cost of the upper computer software is increased, and the control is more complicated; aiming at different integration time, the problem that exposure starting time can not be set arbitrarily by each detector is solved. The circuit comprises an FPGA, a ping-pong buffer, a data output interface, a power management module and N CMOS detectors; the FPGA is respectively connected with the N CMOS detectors; n cache FIFOs are arranged in the FPGA and respectively store original image signals output by N CMOS detectors; the ping-pong buffer comprises 2 DDR buffers adopting a ping-pong buffer mode, and each DDR buffer comprises N buffer areas for respectively buffering the image data of the N buffer FIFOs; the FPGA is used for carrying out image splicing on the N image data cached by the DDR to obtain a complete multi-view spliced image; and the data output interface is connected with the FPGA and used for outputting the multi-view spliced image.

Description

Multi-view imaging circuit with self-adaptive cache

Technical Field

The utility model relates to a multi-view imaging technology, in particular to a multi-view imaging circuit with adaptive cache.

Background

The quantum dot imaging spectrum technology is characterized in that a superfine strip spectrum resolution quantum dot integrated array made of colloidal quantum dot nano materials is used as a spectrum gating medium to replace complex optical mechanisms such as slit gratings, the limitation of bottleneck problems of the traditional optical light splitting principle on the improvement of the comprehensive performance of the satellite-borne hyperspectral imager is broken through, the application efficiency can be greatly improved under the same mass, and the quantum dot imaging spectrum technology is expected to become a preferred technical scheme of the future hyperspectral imager. At present, a quantum dot imaging spectrometer needs multi-view imaging, each detector corresponds to different wave bands, and the quantum dot imaging spectrometer needs to work under different integration time in order to ensure that the overall DN (remote sensing image pixel brightness value) value of an output image of the multi-detector is close.

The traditional multi-camera imaging system adopts a plurality of cameras to image respectively, each camera needs an independent FPGA, an image cache and an image output interface, and multi-camera cooperative control is carried out through upper computer software, so that the system integration level is low, the complexity is high, and the power consumption is high; and the software of the upper computer is required to carry out cooperative control on the multiple independent cameras, and image data acquired by the multiple cameras are spliced and reconstructed to obtain a multi-view spliced image, so that the development cost of the software of the upper computer is increased, the real-time performance is poor, and the control of the multiple cameras is complex.

In addition, the detector outputs image data immediately after exposure is completed, the conventional multi-view camera needs to enable the multi-view camera to finish exposure at the same time in order to ensure data synchronization, under the condition that the multi-view camera is provided with different integration time, namely, the exposure of each detector is required to be started at different time, but for the application occasions such as airborne push-broom and the like, the exposure starting time of the multi-view camera is required to be the same, and the conventional multi-view imaging system obviously cannot meet the requirement.

SUMMERY OF THE UTILITY MODEL

The system aims to solve the problems that the existing multi-camera imaging system adopts a plurality of cameras to image respectively, the system integration level is low, and the power consumption is high; the upper computer software is needed to carry out cooperative control on the multiple independent cameras, so that the development cost of the upper computer software is increased, and the cooperative control of the multiple cameras is complex; the utility model also provides a multi-view imaging circuit with self-adaptive cache, aiming at the technical problem that exposure starting time can not be set arbitrarily by each detector in different integration time.

In order to achieve the purpose, the technical scheme provided by the utility model is as follows:

a multi-view imaging circuit with adaptive cache is characterized in that: the device comprises an FPGA, a ping-pong buffer, a data output interface, a power management module and N CMOS detectors, wherein N is an integer greater than 1;

the FPGA is respectively connected with the N CMOS detectors and is used for sending a driving signal to the CMOS detectors;

n cache FIFOs are arranged in the FPGA and used for respectively storing original image signals output by N CMOS detectors;

the ping-pong buffer comprises 2 DDR buffers adopting a ping-pong buffer mode, and each DDR buffer comprises N buffer areas for respectively buffering the image data of the N buffer FIFOs;

the FPGA is used for carrying out image splicing on the N image data cached by the DDR to obtain a complete multi-view spliced image;

the data output interface is connected with the FPGA and used for outputting a multi-view spliced image;

the power management module is used for supplying power to the FPGA, the ping-pong buffer and the N CMOS detectors.

Further, N is 2.

Further, the data output interface adopts a CamLink interface.

Further, the FPGA adopts Xilinx corporation industry grade XC7K 325T;

the CMOS detector adopts Lince5M of E2V company.

Compared with the prior art, the utility model has the advantages that:

1. in the multi-view imaging circuit, the multi-view camera shares the ping-pong buffer consisting of the FPGA and the group of 2 DDR buffers, the cooperative control is realized inside the camera, the complexity and the power consumption of the system are greatly reduced, images are spliced inside the FPGA, the development of upper computer software is saved, and the operation is simplified.

2. The multi-view imaging circuit of the utility model opens up storage resources inside the FPGA as a first-level cache, the ping-pong cache is used as a second-level cache, the input ports are multiplexed by DDR cache dynamic address allocation, multi-path image signal caching is realized, image splicing is completed by data in the second-level cache in the FPGA, image processing inside a camera is realized, a complete multi-view spliced image is directly output, the real-time performance is high, and the complexity of a hardware system is greatly reduced.

3. The multi-view imaging circuit can realize the random integration time and the random exposure starting time of the multi-view camera through FIFO and DDR cache in the FPGA, directly obtain a complete multi-view spliced image and provide larger use freedom.

4. The CMOS detectors of the imaging circuit share the hardware circuits such as the FPGA, the ping-pong buffer, the data output interface, the power management module and the like, so that the weight and the volume can be greatly reduced, and the integration level of the imaging system is improved.

5. The imaging circuit of the utility model uses dynamic secondary cache to receive image data, has no fixed requirements on the write-in time and the write-in depth of single-path image data, can work when the data volume and the data format of a plurality of CMOS detectors are inconsistent, such as windowing, interval sampling and the like, can set all parameters of a single CMOS detector, can be independently set, supports the cooperative work of detectors of different models, and has stronger flexibility.

Drawings

FIG. 1 is a schematic block diagram of an embodiment of a multi-view imaging circuit with adaptive caching according to the present invention (a binocular camera is used as an example and a power management module is not shown);

FIG. 2 is a schematic diagram of a DDR cache memory space in an embodiment of the adaptive-cache-equipped multi-view imaging circuit of the present invention (taking a binocular camera as an example);

fig. 3 is a flowchart of the operation of the adaptive-cache multi-view imaging circuit according to the present invention (taking a binocular camera as an example).

Detailed Description

The utility model is described in further detail below with reference to the figures and specific embodiments.

As shown in fig. 1, the multi-view imaging circuit with adaptive cache of the present invention includes an FPGA, a ping-pong cache, a data output interface, a power management module, and a plurality of CMOS detectors; the FPGA is responsible for driving a plurality of detectors and receiving and reconstructing images, the ping-pong buffer is responsible for caching the images, and the data output interface is responsible for outputting the image data; and the power supply management module is used for supplying power to the FPGA, the ping-pong buffer and the plurality of CMOS detectors.

The imaging circuit is suitable for a plurality of detectors, and the plurality of detectors can be detectors with the same model and can also be detectors with different models. In the embodiment, two CMOS detectors are taken as an example, the two CMOS detectors are both connected to the FPGA, the FPGA sends driving signals to the two CMOS detectors, and the two CMOS detectors realize binocular imaging. The data output interface of the embodiment adopts a CamLink interface chip; the FPGA selects Xilinx corporation industrial grade XC7K325T, the model meets the requirement of a binocular camera, and in other embodiments, a higher grade FPGA is needed for more purposes and is mainly limited by IO (input/output) resources of the FPGA. The CMOS detector of the embodiment adopts Lince5M of E2V company, and other types of CMOS detectors can also be adopted, but the following 2 points need to be met: 1. the detector must be internally provided with an AD converter to directly output a digital image signal; 2. the detector must have an external trigger function to allow accurate exposure control.

The core of the dual-detector camera is acquisition and reconstruction of two paths of non-clock domain image signals. The single detector camera takes the DDR cache as an image cache, and image signals obtained by the CMOS detector enter the FPGA for image reconstruction after the DDR cache and are output to the data interface chip for transmission. The DDR buffer works with an internal clock of the FPGA, and the DDR buffer has the function of converting a clock domain of an image signal output by the CMOS detector into a clock domain of the FPGA for working so as to perform subsequent image processing. Two paths of original image signals of the double-detector camera work in different clock domains, and only 1 set of input physical ports of the single DDR cache exists, so that the double-detector requirement cannot be met by adopting a DDR cache scheme.

Because the original image signals output by the two CMOS detectors are difficult to be completely synchronous, the traditional scheme is that two paths of signals are respectively collected and output to an upper computer, image splicing is required to be realized by upper computer software, the development cost of the upper computer software is additionally increased, splicing can be started after image data of a plurality of paths of cameras are completely transmitted to the upper computer, and the real-time performance of multi-view imaging is reduced. The utility model adopts dynamic ping-pong second-level cache, develops storage resources in FPGA as first-level cache, takes ping-pong cache composed of 2 DDR caches as second-level cache, and multiplexes input ports by DDR cache dynamic address allocation to realize multi-path image signal cache; the first-level cache is a plurality of cache FIFOs in the FPGA, the number of the cache FIFOs is the same as that of the CMOS detectors, and the cache FIFOs correspond to the CMOS detectors one by one, specifically, 2 cache FIFOs are arranged in the embodiment, the original image signals output by 2 CMOS detectors are respectively and fixedly stored, and then the original image signals are written into the second-level cache in turn according to the data storage condition of the first-level cache; forming ping-pong buffers by 2 DDR caches, opening a plurality of continuous storage areas in each DDR cache, and respectively using the continuous storage areas as a second-level cache area of a plurality of paths of image signals, specifically in the embodiment, each DDR cache comprises 2 cache areas which are respectively used for caching image data read out by 2 first-level cache FIFOs; after the storage of all the detector images is finished, the detector images are output to the FPGA, the FPGA carries out multi-path image splicing according to task requirements, the obtained multi-purpose spliced image is output to a CamLink interface chip, and the image is converted into a CamLink protocol to be output.

The working process of the multi-view imaging circuit of the embodiment is as follows:

as shown in fig. 1 and fig. 3, the dual detectors are, by definition, a CMOS detector A, CMOS, a buffer FIFO corresponding to the CMOS detector a, and a buffer FIFO corresponding to the CMOS detector B, respectively, are buffer FIFOs a and B; as shown in fig. 2, the storage space of each DDR cache includes a buffer area a corresponding to the buffer FIFO a and a buffer area B corresponding to the buffer FIFO B; the 2 DDR caches are respectively a DDR cache A, DDR cache B, and the DDR cache A is in a writing state and is taken as an initial working state;

1) the FPGA sends driving signals to the CMOS detector A and the CMOS detector B respectively to drive the CMOS detector A and the CMOS detector B to image;

2) an original image signal A output by the CMOS detector A firstly passes through a cache FIFO A, whether a whole line of data is written or not is judged according to the write address of the FIFO A, and the line of data of the cache FIFO A is written into a cache area A of the DDR cache A when the cache FIFO A is full of the whole line of data;

an original image signal B output by the CMOS detector B firstly passes through a buffer FIFO B, whether a whole line of data is written is judged according to the write address of the FIFO B, and the line of data of the buffer FIFO B is written into a buffer area B of the DDR buffer A when the buffer FIFO B is full of the whole line of data;

3) the DDR cache area writes data from an appointed address sequence, specifically, the initial address of the cache area A is 0, the initial address of the cache area B is 50% of the total capacity, two paths of image signal transmission conditions of the CMOS detector A and the CMOS detector B are judged according to the written address numbers in the cache area A and the cache area B in the DDR cache A, when the cache area A and the cache area B in the DDR cache A finish image storage of a whole frame, ping-pong switching is performed by the ping-pong cache, the image data sequence in the cache area A and the cache area B of the DDR cache A is read out to the FPGA according to the address sequence, and meanwhile, the DDR cache B writes in, so that the continuity of image signal caching output by the double detectors is ensured;

4) the FPGA splices the received complete CMOS detector A, CMOS detector B images as required, specifically, in the example, the CMOS detector A image is above and the CMOS detector B image is below, so as to obtain a complete multi-view spliced image and output the image through a CamLink interface.

The binocular camera of the embodiment also supports the operation of double detection at any integration time or exposure starting time. The CMOS detector starts exposure after receiving an exposure request, image data reading can be started immediately after reaching specified integration time, when two CMOS detectors set different integration time parameters, two paths of image data are sent out at different times, and in the worst case, the CMOS detector A finishes image transmission, and the CMOS detector B does not start. If the traditional one-level cache scheme is used, two CMOS detectors must start data transmission synchronously, and the integration time or the exposure starting time of the two detectors cannot be set arbitrarily, so that the application flexibility is greatly limited. According to the utility model, because a dynamic address is adopted in the DDR secondary cache, for example, the CMOS detector A is set according to the integral time or the exposure starting time and finishes exposure before the CMOS detector B, the CMOS detector A outputs image data firstly, the image data of the CMOS detector A is stored on a cache region corresponding to the DDR cache through the primary cache, then the image data of the CMOS detector B is waited, and when two CMOS detectors finish data transmission, the secondary cache outputs two stored continuous and complete images. And performing ping-pong switching after one DDR buffer stores the images of the two CMOSA detectors. In the embodiment, the DDR write clock speed is 4 times of the pixel clock speed of the CMOS detector, and when the CMOS detector A, CMOS and the detector B output data at the same time, the data can be stored timely and completely. Through DDR dynamic cache, image data output by multiple detectors can be written into corresponding addresses of the DDR in time when the image data arrive at any time.

The utility model uses dynamic second-level buffer to receive image data, has no fixed requirements on the writing time and the writing depth of single-path image data, and can work when a plurality of CMOS detectors do not output images at the same time or the data quantity is inconsistent. For example, a multi-slice CMOS detector may set different exposure start times, integration times, numbers of windowing lines, or binning parameters. All parameters which can be set by a single CMOS detector can be independently set for multiple detectors, and even the multiple detectors can adopt different models, so that the flexibility is strong. The scheme can be realized under any number of detectors, specifically, the scheme is limited by the maximum support of 4 detectors by hardware resources in the example, and the multi-view imaging circuit consisting of more detectors can be realized by upgrading a higher-level FPGA and using a DDR cache with higher clock rate.

According to the embodiment, the double detectors are controlled by the single FPGA through dynamic second-level cache, binocular images are spliced inside the FPGA, processing is not needed by upper computer software, and the complexity of a hardware system is greatly reduced. The 2 CMOS detectors of the imaging circuit of the embodiment share hardware circuits such as an FPGA, a ping-pong buffer, a data output interface, a power management module and the like, so that the power consumption, the weight and the volume can be greatly reduced, and the integration level of an imaging system is improved; and the splicing of the images in the camera is realized, the spliced complete binocular images can be directly output, and the real-time performance is high.

The above description is only for the preferred embodiment of the present invention and does not limit the technical solution of the present invention, and any modifications made by those skilled in the art based on the main technical idea of the present invention belong to the technical scope of the present invention.

Claims (4)

1. A multi-view imaging circuit with adaptive cache, characterized in that: the device comprises an FPGA, a ping-pong buffer, a data output interface, a power management module and N CMOS detectors, wherein N is an integer greater than 1;

the FPGA is respectively connected with the N CMOS detectors and is used for sending a driving signal to the CMOS detectors;

n cache FIFOs are arranged in the FPGA and used for respectively storing original image signals output by N CMOS detectors;

the ping-pong buffer comprises 2 DDR buffers adopting a ping-pong buffer mode, and each DDR buffer comprises N buffer areas for respectively buffering the image data of the N buffer FIFOs;

the FPGA is used for carrying out image splicing on the N image data cached by the DDR to obtain a complete multi-view spliced image;

the data output interface is connected with the FPGA and used for outputting a multi-view spliced image;

the power management module is used for supplying power to the FPGA, the ping-pong buffer and the N CMOS detectors.

2. The multi-purpose imaging circuit with adaptive cache of claim 1, wherein: and the N is 2.

3. The multi-purpose imaging circuit with adaptive cache of claim 2, wherein: the data output interface adopts a CamLink interface.

4. The multi-purpose imaging circuit with adaptive cache according to any one of claims 1 to 3, characterized in that: the FPGA adopts Xilinx corporation industrial grade XC7K 325T;

the CMOS detector adopts Lince5M of E2V company.

Images