CN104776919B - Infrared focal plane array ribbon Nonuniformity Correction system and method based on FPGA - Google Patents
Infrared focal plane array ribbon Nonuniformity Correction system and method based on FPGA Download PDFInfo
- Publication number
- CN104776919B CN104776919B CN201510181531.XA CN201510181531A CN104776919B CN 104776919 B CN104776919 B CN 104776919B CN 201510181531 A CN201510181531 A CN 201510181531A CN 104776919 B CN104776919 B CN 104776919B
- Authority
- CN
- China
- Prior art keywords
- module
- image
- histogram
- value
- column
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Transforming Light Signals Into Electric Signals (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of infrared focal plane array ribbon Nonuniformity Correction system and method based on FPGA, belong to infrared imagery technique field.The present invention for core, is optimized with the infrared equalization algorithm of intermediate value (MIRE) for algorithm and FPGA characteristic, including image buffer storage module, parameter calculating module, Nonuniformity Correction module and data outputting module.Parameter calculating module instructs to obtain weight coefficient according to host computer, original image is after caching, Nonuniformity Correction module is sequentially inputted in column direction obtains the histogram of each row image, accumulative histogram and inverse histogram, then by the way that accumulative histogram and inverse histogram are indexed and each row original image standardized onto the histogram of adjacent column respectively, finally the pixel value after each standardization is weighted to obtain final correction result and exported.The present invention can effectively correct infrared focal plane array ribbon heterogeneity, and due to all being realized using FPGA, speed is fast, can be done in real time correction.
Description
Technical Field
The invention belongs to the technical field of infrared imaging, and particularly relates to a system and a method for correcting strip-shaped non-uniformity of an infrared focal plane array by using a median infrared equilibrium algorithm based on an FPGA (field programmable gate array).
Background
Most of the currently used infrared imaging systems are based on infrared focal plane array (IRFPA) devices, but due to the problems of manufacturing process level and materials, the infrared focal plane array devices have a problem of low signal-to-noise ratio, which is, on one hand, fixed pattern noise caused by inconsistent response of the detection units, and on the other hand, stripe-shaped non-uniformity caused by non-uniformity of the readout circuit and bias voltage noise, which seriously affects the quality of the infrared image and limits the application of the infrared focal plane array devices.
Although the conventional non-uniformity correction method has an excellent performance in removing the non-uniformity of the infrared image, the stripe-shaped non-uniformity has both structural and random properties. The existing calibration-based methods (such as single-point calibration and two-point calibration) use the observation of the black body to perform image correction, and are only suitable for the condition that the parameter time domain is fixed. However, the traditional scene-based methods (such as a constant statistical method, a neural network method, a time domain high-pass filtering method, and a Kalman filtering method) generally need to calculate a plurality of continuous frames in a time domain, and have a slow convergence rate, so that the method cannot adapt to the change of parameters in the time domain. It is therefore necessary to study an algorithm for correcting specifically for the banding non-uniformity. At present, the algorithm mainly stays at a software implementation level, has long processing time and can not meet the real-time processing requirement of images. The existing real-time infrared focal plane strip-shaped non-uniformity correction system realized by hardware adopts too simple algorithm and poor processing effect. Therefore, a real-time infrared focal plane array strip non-uniformity correction system with good processing effect needs to be designed.
The median infrared equalization (MIRE) algorithm was proposed in 2010 by y.tendero et al, has a better correction effect on the stripe-like non-uniformity, and has no iterative operation and low complexity. The basic process is as follows: first, calculate each column image C j Histogram of (1) j (ii) a Second, calculate cumulative histogram H of each column of images j (ii) a Thirdly, calculating an inverse histogram of the cumulative histogram of each column of images(ii) a Fourthly, calculating a value cumulative histogram thereofMake it Fifthly, normalizing the column histogram of each column to the median histogram of the column to obtain a corrected image
The following difficulties exist when the FPGA is adopted to realize the median infrared equalization algorithm: five steps in the original algorithm flow can be calculated in the next step after all the calculation of the previous step is completed, a coherent pipeline cannot be formed, the hardware realization is not facilitated, and a quite large storage space is needed to cache intermediate results.
Disclosure of Invention
In view of the above defects or improvement requirements of the prior art, the present invention provides a system and a method for correcting a banding non-uniformity in an infrared focal plane array based on an FPGA, which take a median infrared equalization algorithm (MIRE) as a core, optimize the algorithm and the characteristics of the FPGA, and are intended to fill up the gap of the existing hardware system and method for correcting the banding non-uniformity.
In order to achieve the above object, according to an aspect of the present invention, there is provided an FPGA-based infrared focal plane array stripe non-uniformity correction system, including an image caching module, a parameter calculating module, a non-uniformity correction module, and a data output module, wherein:
the image caching module is used for caching an externally input infrared image, and outputting data to the nonuniformity correction module according to the image in a row after one frame of image is cached;
the parameter calculation module is used for receiving and analyzing an input external instruction to obtain a correction parameter, converting the correction parameter to obtain a weighting coefficient and transmitting the weighting coefficient to the non-uniformity correction module;
the non-uniformity correction module comprising:
the input control and data distribution module is used for controlling the image caching module to output a column of images to the histogram statistical module every time, counting the number of the output columns at the same time, and controlling the image caching module to output the column of image data after secondary caching to the cumulative histogram processing module after all data required for calculating a certain column of image normalized pixels are obtained;
the histogram statistical module is used for performing histogram statistics on the image column data output by the input control and data distribution module and transmitting the histogram information of the previous column of images to the cumulative histogram processing module;
the cumulative histogram processing module is used for calculating the cumulative histogram of each row of images according to the histogram information of each row of images, caching the cumulative histogram, generating a mark value required for calculating the inverse histogram of the cumulative histogram, transmitting the mark value to the mark value forwarding module, and transmitting the index value of the row corresponding to the image data index transmitted by the input control and data distribution module to the index value forwarding module;
the mark value forwarding module is used for caching the mark value and forwarding the mark value to the inverse histogram calculation module group;
the index value forwarding module is configured to forward the index value to the inverse histogram calculation module group;
the inverse histogram calculation module group comprises a plurality of inverse histogram calculation modules, and is used for finishing the calculation of an inverse histogram through the marking value, updating the DPRAM inside the module, then indexing the DPRAM inside the module according to the index value, and forwarding the corresponding data to the normalized calculation module;
the parallel control module is used for controlling each inverse histogram calculation module in the inverse histogram calculation module group to complete calculation, updating and corresponding data forwarding of the inverse histogram; and
the normalized calculation module is used for updating the weighting coefficient through the parameter calculation module, multiplying the output result of the inverse histogram calculation module group by the corresponding weighting coefficient, adding the multiplied results to obtain an image correction result, and transmitting the image correction result to the data output module; and
and the data output module is used for caching and outputting the corrected image.
According to another aspect of the present invention, there is provided an FPGA-based infrared focal plane array stripe non-uniformity correction method, including:
step 1 calculating weighting coefficientsWherein the content of the first and second substances,represents the standard deviation; k represents the serial number of the weighting coefficient, the value is 1,2, \ 8230, 2N +1,2N +1 are the size of the calculation window;
step 2, caching the originally input infrared image;
step 3, performing mirror image expansion on the edge of the originally input infrared image, adjusting the mapping relation between an output column and an output image address, setting the initial address of the originally input infrared image cache space as 0, the number of image columns as L, and the size of a calculation window as 2x +1, and then setting the initial address m when the nth column image is output as:
when outputting the next pixel of the current column, the address shifts the address space of a column of pixels;
step 4, the current column image C j Performing histogram statistics to obtain a histogram I j ；
Step 5 calculating the current image column C j Cumulative histogram of (H) j ；
Step 6, calculating the current image column C j Inverse histogram of cumulative histogram；
Step 7, after the inverse histograms of all the columns in the image correction calculation window of a certain column are calculated, performing secondary cache on the column of image pixels X i,j Inputting, searching the cumulative histogram in the corresponding column region of the buffered cumulative histogram with the gray value of the pixel as the address to obtain the index value H j (X i,j )；
Step 8 is to use the index value H j (X i,j ) Obtaining target column image pixel X as address index i,j Normalizing the pixel values to the histograms of the corresponding columns, respectively
Step 9 normalizes each column of image to the pixel values of the adjacent column histogramBy a corresponding weighting factor psi σ (k) Adding the multiplied results to obtain an image correction result
Step 10, repeating the steps 4 to 9 until all columns of the originally input infrared image are processed;
step 11 buffers and outputs the corrected image data.
Generally, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:
1. compared with the method for realizing the strip-shaped non-uniformity correction of the infrared focal plane array by using the DSP for operation, all the algorithms of the method are finished by the FPGA, the DSP auxiliary operation is not needed, the system processing speed is high, the real-time correction of a large-format infrared image can be realized, and meanwhile, the correction cost is effectively reduced;
2. the original algorithms 4 and 5 are subjected to step equivalent transformation, so that a complete assembly line can be formed by the two steps, the structure of the system is greatly simplified, and meanwhile, the storage space required for storing the median histogram is saved;
3. the simultaneous calculation of the cumulative histogram and the inverse histogram thereof is realized by utilizing the mark value, so that the processing time of the system is saved;
4. the flow processing on the column scale is realized by utilizing the on-chip storage resources to cooperate with ping-pong operation and reasonable planning of the storage area. In the transmission period of a row of images, each level of modules on the production line respectively complete the processing of a row of image data, the intermediate result obtained by the processing is cached by using the DPRAM in a chip, and the intermediate result obtained by the operation in the previous row period is output to the next level of module for processing. The advantages of FPGA parallel computation are fully utilized, and the system operation speed is favorably improved;
5. the FPGA resource occupation is small, the FPGA resource occupation can be realized by using an FPGA device with low cost, the algorithm processing module can also be used as a sub-module and integrated into an FPGA together with other algorithm processing modules, and only the entry logic of the image cache module and the exit logic of the data output module need to be modified to adapt to the time sequence of other modules without modifying the core algorithm module, so that the method is very convenient and the practicability of the method is improved.
Drawings
FIG. 1 is a block diagram of the FPGA-based infrared focal plane array strip non-uniformity correction system of the present invention;
FIG. 2 is a flow chart of the method for correcting the stripe-shaped non-uniformity of the FPGA-based infrared focal plane array of the present invention;
FIG. 3 is a diagram illustrating the update of the weighting factor register of the normalization calculation module according to the present invention;
fig. 4 is a schematic diagram showing the comparison of the original infrared image, the infrared image with the artificially added stripe-like non-uniformity and the processed image according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Fig. 1 is a structural diagram of the infrared focal plane array strip-shaped non-uniformity correction system based on the FPGA, which includes an image caching module, a parameter calculation module, a non-uniformity correction module, and a data output module. The non-uniformity correction module is a main body of the whole system and comprises an input control and data distribution module, a histogram statistic module, an accumulated histogram processing module, a marking value forwarding module, an index value forwarding module, an inverse histogram calculation module group, a parallel control module and a normalized calculation module. Wherein:
and the image caching module is used for caching the infrared image input from the outside by using an external image caching DPRAM (Dual-Port STATIC RAM), sending a data ready signal to the input control and data distribution module of the non-uniformity correction module after caching one frame of image, responding to a control command sent by the input control and data distribution module, and outputting the image according to the sequence of the rows. The image cache module divides an external DPRAM storage area into two parts, and the operation is carried out in a ping-pong mode: and when the current frame image is cached by using the area A, the previous frame image cached in the area B is output to the input control and data distribution module. The image caching module utilizes a DPRAM (not shown in fig. 1) inside the FPGA chip to perform secondary caching on the image while outputting the cached image, and outputs the image under the control of the input control and data distribution module.
And the parameter calculation module adopts a UART (universal asynchronous receiver/transmitter) to realize an interface with the upper computer, receives external instruction input and analyzes the instruction of the upper computer to obtain correction parameters. The weighting coefficients required by calculation are calculated on a computer in advance and stored in an on-chip ROM, and a parameter calculation module obtains corresponding addresses according to conversion of correction parameters and transmits the corresponding weighting coefficients obtained from the ROM to a normalization calculation module.
And the input control and data distribution module is connected with the image cache module, generates logic reset signals of the rest parts of the non-uniformity correction module after receiving data ready signals sent by the image cache module, controls the image cache module to output a column of images to the histogram statistical module each time according to the column sequence, and counts the number of the output columns at the same time. And when all data required for calculating the normalized pixels of a certain column of images are obtained, controlling the image cache module to output the column of image data subjected to secondary cache to the cumulative histogram processing module.
And the histogram statistical module is connected with the input control and data distribution module and used for performing histogram statistics on the image column data output by the input control and data distribution module. The histogram statistical module utilizes the internal DPRAM to perform real-time statistics on the histogram of each column of images, and simultaneously transmits the histogram information of the previous column of images to the cumulative histogram processing module. The histogram statistic module divides an internal DPRAM storage area into two parts, and the operation is carried out in a ping-pong mode: and outputting the histogram of the previous column of image cached in the B area to the cumulative histogram processing module while counting the column histogram of the current image by using the A area.
And the cumulative histogram processing module is connected with the histogram statistical module and the input control and data distribution module, calculates the cumulative histogram of each column of images according to the histogram information of each column of images, caches the cumulative histogram by using the on-chip DPRAM, generates mark information required for calculating the inverse histogram of the cumulative histogram and transmits the mark information to the mark value forwarding module, reads out a corresponding index value according to a DPRAM cache region of a column corresponding to an image data index transmitted by the input control and data distribution module, and transmits the index value to the index value forwarding module.
And the mark value forwarding module is connected with the cumulative histogram processing module, caches the mark information of each column of image cumulative histogram through the internal FIFO and forwards the mark information to the inverse histogram calculation module which needs to be updated correspondingly in the inverse histogram calculation module group. Because the updating speed of the inverse histogram calculation module is lower than the processing speed of the cumulative histogram processing module, the mark value forwarding module internally comprises two FIFOs, and two inverse histogram calculation modules can be ensured to update at the same time.
And the index value forwarding module is connected with the cumulative histogram processing module and forwards the index values obtained by the corresponding column images to the inverse histogram calculation module group.
The inverse histogram calculation module group is connected with the marking value forwarding module, the index value forwarding module and the parallel control module and comprises a plurality of inverse histogram calculation modules, and each inverse histogram calculation module has two states of refreshing and outputting and is controlled by the parallel control module to switch. When the module is in a refresh state, the calculation of the inverse histogram is completed through the mark value transmitted by the mark value forwarding module and the DPRAM in the module is updated, and the specific method comprises the following steps: firstly, reading a mark value, filling the gray value in the mark value into the address corresponding to the height value in the memory, and filling the blank part between the two addresses with the gray value; and when the module is in an output state, indexing the DPRAM in the module according to the index value forwarded by the index value forwarding module and forwarding the corresponding data to the standardized computing module. In the embodiment of the present invention, the number of modules included in the inverse histogram calculation module group is determined by the size of the calculation window, and the corresponding relationship is as follows: when the window size is 2 × N +1,2 × N +4 inverse histogram calculation modules are used to perform parallel processing.
And the parallel control module is connected with the inverse histogram calculation module group, sends a refresh starting signal to the inverse histogram calculation module, controls each individual module in the inverse histogram calculation module group to be switched to a refresh state when the refresh is needed, and switches to an output state when the refresh is finished.
And the normalized calculation module is connected with the inverse histogram calculation module group and the parameter calculation module and updates the weighting coefficient through the parameter calculation module. And a multiplier is arranged at the input interface of each inverse histogram calculation module, the output result of the inverse histogram calculation module is multiplied by the corresponding weighting coefficient, the multiplied results are added to obtain a normalized pixel value, namely an image correction result, and the result is transmitted to the data output module.
And the data output module is used for caching the corrected image by using the off-chip DPRAM, providing an EMIF interface facing the DSP, outputting the result to the DSP for subsequent processing and realizing an external test interface by adopting a VGA interface.
Fig. 2 is a flowchart of the method for correcting the stripe-shaped non-uniformity of the FPGA-based infrared focal plane array, which specifically includes the following steps:
step 1 calculation of the weighting factor psi σ (k)。
Weighting factor psi σ (k) The calculation formula is as follows:wherein, the first and the second end of the pipe are connected with each other,wherein σ represents a standard deviation; k represents the serial number of the weighting coefficient, and takes the value of 1,2, \ 8230;, 2N +1 (assuming that the calculation window size is 2N + 1). In order to save hardware resources occupied by the system, the hardware system does not directly set the standard deviation sigma to calculate the weighting coefficient, but sets a plurality of files, and each file calculates the weighting coefficient psi by software in advance σ (k) And then stored in on-chip ROM for calling. In the embodiment of the invention, the grading mode is as follows: first, the maximum calculation window size (2P + 1) and the minimum calculation window size (2Q + 1) are determined (note that it needs to be an odd number) according to the breadth and characteristics of the imager, and for each window with the size of (2N + 1) (Q ≦ N ≦ P), the non-uniformity coefficient is definedLet psi when Λ is 0.5, 0.4, 0.3, 0.2 and 0.1 respectively σ (k)(k∈[1,2N+1]) As weighting coefficients, all weighting coefficients are scaled up 256 times (i.e., shifted left by 8 bits). Note that in practice there are (2P + 1) weighting coefficients for each gear, for N&In the case of P, the x-th coefficient is mapped to the x + P-N, and the empty coefficient is 0. When each frame of image starts to be processed, firstly detecting whether the upper computer sends a parameter change instruction, if not, judging whether the current frame is a first frame, if so, adopting a default parameter, and if not, not updating the parameter; and if the upper computer sends a parameter change instruction, analyzing a new parameter for updating. Then the parameters are mapped to the corresponding storage space in the coefficient ROM, and the corresponding weighting coefficient psi is read out σ (k) And updating to a normalized calculation module.
And 2, caching the originally input infrared image.
Because the image input is input according to the sequence of lines, and the subsequent processing of the invention requires the image output according to the sequence of columns, the original image needs to be cached, and the processing is carried out when the caching of a frame of complete image is finished.
And 3, carrying out mirror image expansion on the edge of the original image.
The subsequent processing process has the operation between the current column image and the adjacent column image, and the pixels at the edge lack the corresponding data of the adjacent column for processing, so the special processing needs to be carried out on the boundary in the row direction of the original image. The image mirror expansion is realized by adjusting the mapping relation between the output columns and the output image addresses, and if the initial address of the input image cache space is 0, the number of image columns is L, and the size of the calculation window is 2x +1, the initial address m when the nth column of images is output is:
the address is shifted by the address space of a column of pixels each time the next pixel of the current column is output.
Step 4, calculating the current image columnC j Histogram of (1) j 。
Mapping DPRAM addresses with pixel grey values to use on-chip DPRAM for current column image C j Performing histogram statistics to obtain a histogram I j . The details of the histogram statistics are well known to those skilled in the art and will not be described in detail herein.
Step 5 calculating the current image column C j Cumulative histogram of (H) j 。
In order to save processing time, the invention calculates the cumulative histogram H of the current image column in parallel j Inverse histogram of cumulative histogramTherefore, it is necessary to generate information for calculating an inverse histogram while calculating a current cumulative histogram, and the method specifically includes the following sub-steps:
(5-1) reading the height value of the gray value corresponding to the histogram, wherein the gray value is 0 in the initial state;
(5-2) for the height value obtained by each gray scale value, adding the height value and the accumulated height value (0 in the initial state) to obtain a new accumulated height value, and writing the new accumulated height value into an address corresponding to the current gray scale in the DPRAM to update the accumulated histogram;
(5-3) if the height value is 0, executing the step (5-5), otherwise, taking the new accumulated height value and the current gray value as the mark values and downloading the mark values to a mark value forwarding module;
(5-4) if the current height value is 1, filling 0 into the filling flag bit of the mark value, otherwise, filling 1 into the filling flag bit;
(5-5) if the maximum gray value is reached, ending, otherwise, adding 1 to the gray value, and then executing the step (5-1).
Step 6, calculating the current image column C j Inverse histogram of cumulative histogramThe method specifically comprises the following substeps:
(6-1) when new mark data exist, reading out mark value data from the mark value forwarding FIFO, and filling partial data content of the gray value into an address corresponding to the accumulated height value;
(6-2) if the filling flag bit is 0, executing the step (6-3), otherwise, completely filling the address area corresponding to the current accumulated height value and the accumulated height value received last time into the current gray value;
(6-3) ending if the maximum height value has been reached, otherwise continuing to perform step (6-1).
Step 7 calculating index value H j (X i,j )。
After the inverse histogram statistical module calculates the inverse histograms of all the adjacent columns needed by correcting a certain column of image, the image pixel X of the column of image cached for the second time is processed i,j Inputting, searching the accumulated histogram in the corresponding column region of the buffered accumulated histogram with the gray value of the pixel as the address to obtain the index value H j (X i,j )。
Step 8, calculating the pixel X of the image column to be corrected i,j Respectively normalized to the values of adjacent columns after the histogram
Each inverse histogram statistic module in output state indexes value H j (X i,j ) Storing DPRAM as the inverse histogram of the address index itself to obtain the target column image pixel X i,j Normalizing the pixel values respectively after the histograms of the corresponding columns
Step 9 normalizes the values to the adjacent column histogramWeighted average to obtain final correction result
In the case of a window size of 2 XN +1, there are a total of 2 XN +4The data output interface of the inverse histogram statistical module is connected to the normalized calculation module, wherein only the inputs on 2 XN +1 interfaces are effective inputs at the same time. A multiplier is arranged at each input interface of the normalized calculation module, and the output result of each column is converted into a digital signalBy a corresponding weighting factor psi σ (k) In that respect The corresponding weighting coefficients are stored in the registers, and a total of 2 × N +4 registers are provided, wherein the coefficients corresponding to 3 invalid input interfaces in the update state are zero, so that the calculation result is ensured not to be interfered by invalid data. The invention realizes the sliding of the calculation window by circularly assigning and updating the weighting coefficient register, fig. 3 is a schematic diagram showing the updating of the weighting coefficient register of the normalized calculation module of the invention, the upper half of the diagram shows the situation when calculating the normalized pixel values of the N +1 th column of images, the calculation window is the 1 st column to the 2n +1 st column of images, corresponding to the numbers 1 to 2n +1 in the inverse histogram calculation module, and the value in the multiplied weighting coefficient register is exactly corresponding to the psi σ (1) To psi σ (2N + 1). The lower part of the diagram shows the case when normalized pixel values of the N +2 columns of images are calculated, the calculation window is from the 2 nd column to the 2n +2 nd column of images, corresponding to the 2 nd to 2n +2 nd numbers in the inverse histogram calculation module, at this time, the weighting coefficient register is circularly assigned, the 1 st value is assigned to the 2 nd number, the 2 nd number is assigned to the 3 rd number, and so on, the 2n +4 th value is assigned to the 1 st number, so that the value in the weighting coefficient register multiplied by the inverse histogram calculation module just corresponds to the psi σ (1) To psi σ (2N + 1). Since the weighting factor is amplified 256 times, after all the products are added, the result needs to be right-shifted by 8 bits to obtain the final correction result
Step 10 repeats steps 4 to 9 until processing of all columns of the image is completed.
And step 11, outputting a correction result. And caching the corrected image data, outputting the image data to a DSP (digital signal processor) through an EMIF (external memory interface) for subsequent algorithm processing, and displaying the corrected image in a display through a VGA (video graphics array) interface for testing. Such logic is well known to those skilled in the art and will not be described in detail herein.
Fig. 4 is a schematic diagram showing the comparison of the original infrared image, the infrared image with the artificially added stripe-like non-uniformity and the processed image according to the present invention. FIG. 4 (a) is an original infrared image without banding non-uniformities; FIG. 4 (b) is an image of an artificially added banding non-uniformity; fig. 4 (c) shows the processing results obtained by the present invention, and it can be seen that the stripe-like non-uniformity is well corrected and the image has no significant distortion, and the visual effect is significantly improved.
It will be understood by those skilled in the art that the foregoing is only an exemplary embodiment of the present invention, and is not intended to limit the invention to the particular forms disclosed, since various modifications, substitutions and improvements within the spirit and scope of the invention are possible and within the scope of the appended claims.
Claims (10)
1. The utility model provides an infrared focal plane array banding heterogeneity correction system based on FPGA, which is characterized in that includes image buffer module, parameter calculation module, heterogeneity correction module and data output module, wherein:
the image caching module is used for caching an externally input infrared image, and outputting data to the nonuniformity correction module according to the image in a row after one frame of image is cached;
the parameter calculation module is used for receiving and analyzing an input external instruction to obtain a correction parameter, converting the correction parameter to obtain a weighting coefficient and transmitting the weighting coefficient to the non-uniformity correction module;
the non-uniformity correction module comprising:
the input control and data distribution module is used for controlling the image caching module to output a column of images to the histogram statistical module every time, counting the number of the output columns at the same time, and controlling the image caching module to output the column of image data after secondary caching to the cumulative histogram processing module after all data required for calculating a certain column of image normalized pixels are obtained;
the histogram statistical module is used for carrying out histogram statistics on the image column data output by the input control and data distribution module and simultaneously transmitting the histogram information of the last column of images to the cumulative histogram processing module;
the cumulative histogram processing module is used for calculating the cumulative histogram of each row of images according to the histogram information of each row of images, caching the cumulative histogram, generating a mark value required for calculating the inverse histogram of the cumulative histogram, transmitting the mark value to the mark value forwarding module, and transmitting the index value of the row corresponding to the image data index transmitted by the input control and data distribution module to the index value forwarding module;
the mark value forwarding module is used for caching the mark value and forwarding the mark value to the inverse histogram calculation module group;
the index value forwarding module is configured to forward the index value to the inverse histogram calculation module group;
the inverse histogram calculation module group comprises a plurality of inverse histogram calculation modules, and is used for completing the calculation of an inverse histogram through the mark value, updating the DPRAM inside the module, then indexing the DPRAM inside the module according to the index value, and forwarding the corresponding data to the standardized calculation module;
the parallel control module is used for controlling each inverse histogram calculation module in the inverse histogram calculation module group to complete calculation and updating of the inverse histogram and corresponding data forwarding; and
the normalized calculation module is used for updating the weighting coefficient through the parameter calculation module, multiplying the output result of the inverse histogram calculation module group by the corresponding weighting coefficient, adding the multiplied results to obtain an image correction result, and transmitting the image correction result to the data output module; and
and the data output module is used for caching and outputting the corrected image.
2. The FPGA-based infrared focal plane array striping non-uniformity correction system of claim 1 wherein said image cache module caches said externally input infrared image using an external image cache DPRAM and divides the storage area of said external DPRAM into two parts, operating in a ping-pong fashion.
3. The system according to claim 1 or 2, wherein the weighting coefficients are calculated in advance and stored in an on-chip ROM, and the parameter calculation module obtains corresponding addresses through conversion according to the correction parameters, and obtains corresponding weighting coefficients from the on-chip ROM according to the corresponding addresses and transmits the weighting coefficients to the normalization calculation module.
4. The FPGA-based ir focal plane array striping non-uniformity correction system of claim 1 wherein each inverse histogram calculation module within said set of inverse histogram calculation modules has two states of refresh and output, controlled by said parallel control module to switch, when in said refresh state, first reading out a mark value to fill the gray value therein into the corresponding address of its height value in the memory, and then filling the blank part between two addresses with said gray value; and when the module is in the output state, indexing the DPRAM inside the module according to the index value, and forwarding corresponding data to the normalized calculation module.
5. The FPGA-based IR focal plane array banding non-uniformity correction system of claim 4, wherein said parallelization control module sends a refresh start signal to said set of inverse histogram calculation modules, controls each of said set of inverse histogram calculation modules to switch to said refresh state when refresh is needed, and to switch to said output state when refresh is over.
6. An infrared focal plane array strip-shaped non-uniformity correction method based on FPGA is characterized by comprising the following steps:
step 1 calculating weighting coefficientsWherein the content of the first and second substances,σ represents the standard deviation; k represents the serial number of the weighting coefficient, the value is 1,2, \ 8230, 2N +1,2N +1 are the size of the calculation window;
step 2, caching the originally input infrared image;
step 3, performing mirror image expansion on the edge of the originally input infrared image, adjusting the mapping relation between an output column and an output image address, setting the initial address of the originally input infrared image cache space as 0, the number of image columns as L, and the size of a calculation window as 2x +1, and then setting the initial address m when the nth column of image is output as:
when outputting the next pixel of the current column, the address shifts the address space of a column of pixels;
step 4, the current column image C j Performing histogram statistics to obtain a histogram I j ；
Step 5 calculating the current image column C j Cumulative histogram of (H) j ；
Step 6, calculating the current image column C j Inverse histogram of cumulative histogram
Step 7, after the inverse histograms of all the columns in the image correction calculation window of a certain column are calculated, performing secondary cache on the column of image pixels X i,j Inputting, searching accumulation in the corresponding column area of the accumulated histogram with the gray value of the pixel as the addressHistogram yields index value H j (X i,j )；
Step 8 is to use the index value H j (X i,j ) Obtaining target column image pixel X as address index i,j Normalizing the pixel values to the histograms of the corresponding columns, respectively
Step 9 normalizes each column of image to the pixel value of the adjacent column of histogramBy a corresponding weighting factor psi σ (k) Adding the multiplied results to obtain an image correction result
Step 10, repeating the steps 4 to 9 until the processing of all the columns of the original input infrared image is finished;
step 11 buffers and outputs the corrected image data.
7. The method of claim 6 wherein in step 1, a maximum calculation window size 2P +1 and a minimum calculation window size 2Q +1 are determined according to the aspect and characteristics of the imager, wherein Q < N < P for each window of size 2N +1, a coefficient of non-uniformity is definedLet psi when Λ is 0.5, 0.4, 0.3, 0.2 and 0.1 respectively σ (k) As a weighting coefficient, where k ∈ [1,2N +1 +]。
8. The method as claimed in claim 6, wherein in step 5, the current image column C is calculated j Generating information for calculating an inverse histogram while accumulating histograms, comprising the sub-steps of:
(5-1) reading the height value of the corresponding gray value of the histogram, wherein the gray value is 0 in the initial state;
(5-2) for the height value obtained by each gray value, adding the height value and the accumulated height value to obtain a new accumulated height value, and writing the new accumulated height value into an address update accumulated histogram corresponding to the current gray value;
(5-3) if the height value is 0, executing the step (5-5), otherwise, taking the new accumulated height value and the current gray value as marking values;
(5-4) if the current height value is 1, filling the filling flag bit of the mark value with 0, otherwise, filling 1;
(5-5) if the maximum gray value is reached, ending, otherwise, adding 1 to the gray value, and then executing the step to (5-1).
9. The method of claim 8, wherein the step 6 comprises the sub-steps of:
(6-1) when a new mark value exists, reading the new mark value, and filling partial data content of the gray value into an address corresponding to the accumulated height value;
(6-2) if the filling flag bit is 0, executing the step (6-3), otherwise, completely filling the address area corresponding to the current accumulated height value and the accumulated height value received last time into the current gray value;
(6-3) ending if the maximum height value has been reached, otherwise continuing to perform step (6-1).
10. The method according to any one of claims 6 to 9, wherein in step 9, in the case of a window size of 2 × N +1, a total of 2 × N +4 outputs of inverse histogram statistics are provided, wherein only 2 × N +1 inputs are valid inputs at the same time, and the weighting coefficients corresponding to the remaining 3 invalid inputs are zero, and the sliding of the calculation window is implemented by circularly assigning values to the weighting coefficient registers.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510181531.XA CN104776919B (en) | 2015-04-17 | 2015-04-17 | Infrared focal plane array ribbon Nonuniformity Correction system and method based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510181531.XA CN104776919B (en) | 2015-04-17 | 2015-04-17 | Infrared focal plane array ribbon Nonuniformity Correction system and method based on FPGA |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104776919A CN104776919A (en) | 2015-07-15 |
| CN104776919B true CN104776919B (en) | 2018-01-26 |
Family
ID=53618528
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510181531.XA Expired - Fee Related CN104776919B (en) | 2015-04-17 | 2015-04-17 | Infrared focal plane array ribbon Nonuniformity Correction system and method based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104776919B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109640280B (en) * | 2019-01-10 | 2020-12-22 | 深圳市沃特沃德股份有限公司 | Call control method, device, computer equipment and storage medium |
| CN114066759B (en) * | 2021-11-18 | 2023-08-01 | 电子科技大学 | FPGA-based infrared image real-time distortion correction method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102456224A (en) * | 2010-10-19 | 2012-05-16 | 高森 | Real-time digital image enhancement method based on field programmable gate array (FPGA) |
| CN103076096A (en) * | 2013-01-07 | 2013-05-01 | 南京理工大学 | Infrared nonuniformity correcting algorithm based on mid-value histogram balance |
| CN104217399A (en) * | 2013-05-29 | 2014-12-17 | 无锡华润矽科微电子有限公司 | Circuit and method for realizing platform histogram equalization processing function of infrared image |
-
2015
- 2015-04-17 CN CN201510181531.XA patent/CN104776919B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102456224A (en) * | 2010-10-19 | 2012-05-16 | 高森 | Real-time digital image enhancement method based on field programmable gate array (FPGA) |
| CN103076096A (en) * | 2013-01-07 | 2013-05-01 | 南京理工大学 | Infrared nonuniformity correcting algorithm based on mid-value histogram balance |
| CN104217399A (en) * | 2013-05-29 | 2014-12-17 | 无锡华润矽科微电子有限公司 | Circuit and method for realizing platform histogram equalization processing function of infrared image |
Non-Patent Citations (6)
| Title |
|---|
| "Efficient single image non-uniformity correction algorithm";Y.Tendero 等;《Proc Spie》;20101231;第7834卷(第1期);78340E-78340E-12 * |
| "Non-uniformity Correction of Infrared Images by Midway Equalization";Y.Tendero 等;《Image Processing on Line》;20121231;134-146页 * |
| "中值直方图均衡的单帧红外图像非均匀校正算法";贺明 等;《红外与激光工程》;20120930;第41卷(第9期);2550-2553页 * |
| "基于FPGA的直方图均衡实时并行算法及新架构实现";刘延 等;《红外技术》;20100331;第32卷(第3期);148-151页 * |
| "基于FPGA的红外图像非均匀性校正***设计";马欣慰 等;《科技资讯》;20140930;30,32页 * |
| "基于中间均衡直方图的红外图像非均匀性校正";康长青 等;《激光与红外》;20131130;第43卷(第11期);1240-1242页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104776919A (en) | 2015-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210192687A1 (en) | Image processing device and method | |
| CN102663693B (en) | Least square method-based adaptive radiation correction method for linear array push-broom image | |
| CN110555808B (en) | Image processing method, device, equipment and machine-readable storage medium | |
| CN101646091B (en) | Gamma correction method and gamma correction device based on edge detection error diffusion | |
| US11593913B2 (en) | Method and system for correcting a distorted input image | |
| CN108805942B (en) | Infrared image wide dynamic compression method | |
| CN108171645B (en) | Lucky imaging processing system based on FPGA | |
| CN106296616A (en) | A kind of infrared image detail enhancing method and a kind of infrared image details intensifier | |
| CN104776919B (en) | Infrared focal plane array ribbon Nonuniformity Correction system and method based on FPGA | |
| CN113538223A (en) | Noise image generation method, noise image generation device, electronic device, and storage medium | |
| CN115442575A (en) | Video self-adaptive transmission method and system based on video quality evaluation | |
| CN116402679A (en) | Lightweight infrared super-resolution self-adaptive reconstruction method | |
| CN102316283A (en) | Real-time light leak phenomenon removing device for space and line space transfer CCD (charge coupled device) camera | |
| CN115942128A (en) | ISP system design and implementation method based on heterogeneous platform | |
| CN111369449A (en) | Infrared blind pixel compensation method based on generating type countermeasure network | |
| CN117726547A (en) | Defogging implementation method and system based on improved dark channel algorithm | |
| US20060050084A1 (en) | Apparatus and method for histogram stretching | |
| CN104268098A (en) | On-chip cache system for transformation on ultrahigh-definition video frame rates | |
| CN116403200A (en) | License plate real-time identification system based on hardware acceleration | |
| CN109215001B (en) | High temperature difference self-adaptive platform histogram equalization implementation method based on FPGA | |
| CN103177420B (en) | Based on image magnification method and the device of local features correlativity | |
| CN113781368B (en) | Infrared imaging device based on local information entropy | |
| CN110942425A (en) | Reconstruction method and reconstruction system of super-resolution image and electronic equipment | |
| CN111681191A (en) | Color image demosaicing method and system based on FPGA and storage medium | |
| CN104967796A (en) | Super-resolution intelligent image sensor chip |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180126 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |