CN109995965A - A kind of ultrahigh resolution video image real-time calibration method based on FPGA - Google Patents
A kind of ultrahigh resolution video image real-time calibration method based on FPGA Download PDFInfo
- Publication number
- CN109995965A CN109995965A CN201910276412.0A CN201910276412A CN109995965A CN 109995965 A CN109995965 A CN 109995965A CN 201910276412 A CN201910276412 A CN 201910276412A CN 109995965 A CN109995965 A CN 109995965A
- Authority
- CN
- China
- Prior art keywords
- fpga
- data
- image data
- calibration
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
- H04N17/004—Diagnosis, testing or measuring for television systems or their details for digital television systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8455—Structuring of content, e.g. decomposing content into time segments involving pointers to the content, e.g. pointers to the I-frames of the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/20—Adaptations for transmission via a GHz frequency band, e.g. via satellite
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Astronomy & Astrophysics (AREA)
- General Physics & Mathematics (AREA)
- Image Processing (AREA)
Abstract
The present invention provides a kind of ultra high-definition video image real-time calibration method based on FPGA, for carrying out real time calibration to ultra high-definition video image by FPGA, it is characterized in that, include the following steps: step S1, raw image data receiving channel is established, the raw image data of ultra high-definition video image is directly delivered to FPGA；Step S2 carries out calibration process to raw image data, obtains result calibration data；Step S3 establishes calibration result output channel, result calibration data is returned from FPGA, wherein step S2 includes following sub-step: step S2-1 establishes image data frame buffer, and is pre-processed to obtain preprocessing image data to raw image data；Step S2-2 is handled by image calibration algorithm preprocessing image data and is obtained Δ X, Δ Y, Δ θ；Step S2-3 establishes result internal memory cache region, and the Δ X that will be obtained, Δ Y, Δ θ are arranged, and obtains final calibration result data.
Description
Technical field
The invention belongs to field of video image processing, and in particular to a kind of real-time school of ultra high-definition video image based on FPGA
Quasi- processing method.
Background technique
Ultra high-definition video image collimation technique plays an important role in our life.Biomedical engineering,
Very important effect is played in terms of Industrial Engineering, in terms of military public security, in terms of aerospace.With the hair of science and technology
Exhibition, the resolution ratio of video image is higher and higher, and resolution ratio is higher, and bring video image information is abundanter, is more conducive to acquire
More useful information, the resolution applications of 5Kx5K are also gradually common.
However, big video image resolution ratio bring is, bigger calculating power demand can be generated in calibration, that is,
It says, so that the pressure of hardware process speed becomes larger.On the one hand, with the development of computer technology, the processing speed of CPU is
It has been obtained constantly improving, quickly to the processing speed of unit data quantity；On the other hand, in the video figure in many fields
As calibration process requires higher and higher real-time, it is higher that this calculates force request to the hardware in the unit time.But the calculation power of CPU
Promotion be not enough to make up present video image under larger resolution ratio to real-time processing requirement.
To the calibration process of video image, common processing mode has based on PC machine CPU processing mode and based on the processing of DSP
Mode.Processing mode based on CPU is based on von Neumann framework computer platform, and this mode is inherently suitable
What sequence executed, for the video image of a 5Kx5K resolution ratio, using the processing mode of CPU, processing speed is slow, in real time
Property it is not strong, be unable to satisfy demand fast to processing speed in real many scenes.Processing mode based on DSP, at digital signal
Reason device (DSP) has used mathematical operation unit (MAC), takes and configures different grades of pipeline organization on hardware.Its band
What the deficiency come was also evident from, due to the fixation of hardware configuration, the structure of assembly line makes DSP in the mistake of video image processing
Cheng Zhongwei is linearly calculated, have it is clearly disadvantageous, it is also unfriendly for the development process of processing system.
The progress of science and technology, the increasing video image of bring resolution ratio, it includes semantic information it is also more abundant,
More and more advantageous to the information extraction of image, what is supervened is the difficulty for carrying out image calibration, the huge calculation amount pair of bring
The calculation power of hardware is no small load.In face of the process demand of video image calibration real-time, how within the unit time, quickly
Accurately target video image point is analyzed and processed and is calibrated, pass weight is applied to the practical industry landing of video image
It wants.
Hardware calculates the promotion speed of power at present, can no longer meet to the huge need for calculating power in video image calibration algorithm
It asks.Therefore it is directed to limited hardware condition, realizes that the real-time processing of ultra high-definition video image calibration has great importance.
Summary of the invention
To solve the above-mentioned problems, the present invention provides a kind of based at FPGA acceleration for the calibration of ultra high-definition video image
Reason method, this method are quickly handled the calibration of ultra high-definition video image based on effective operation of software and hardware, can be significant
Increase processing speed that ultra high-definition video image is calibrated.Technical scheme is as follows:
The present invention provides a kind of ultrahigh resolution video image real-time calibration method based on FPGA, for passing through FPGA
Real time calibration is carried out to ultra high-definition video image, which comprises the steps of: step S1 establishes raw image data
The raw image data of ultra high-definition video image is directly delivered to FPGA by receiving channel；Step S2, to raw image data into
Row calibration process obtains result calibration data, which includes the centre coordinate of ultra high-definition video image in X-axis
Offset Δ X, offset Δ Y and deflection angle Δ θ in Y-axis；Step S3 establishes calibration result output channel, by result
Calibration data is returned from FPGA, wherein step S2 includes following sub-step: step S2-1, is established for keeping in original image number
According to image data frame buffer, and raw image data is pre-processed to obtain preprocessing image data, image data frame
Buffer area includes frame buffer zone scheduling controller and address management module；Step S2-2, by image calibration algorithm to pre- place
Reason image data is handled and is obtained Δ X, Δ Y, Δ θ；Step S2-3 establishes result internal memory cache region, the Δ X that will be obtained,
Δ Y, Δ θ are arranged, and final calibration result data are obtained.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample, wherein the transmission medium of raw image data receiving channel and calibration result output channel is 10,000,000,000 nets
Card, Gigabit Ethernet or PCIE.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample, wherein the quantity of FPGA is one or more.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample, wherein pretreatment are as follows: preliminary decoder is carried out to raw image data first and handles to obtain pretreatment image number
According to, after to preprocessing image data carry out frame sequence processing, further for each preprocessing image data in frame sequence into
Row personalization ID label processing.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample, preliminary decoder processing use linear codec processing method, and frame sequenceization processing is to preprocessing image data
One or more fifo queue types buffer area is established, personalized ID label processing is by the timestamp of preprocessing image data
It carries out ID label or carries out ID label by the sequencing that preprocessing image data enters queue type buffer area.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample, wherein frame buffer zone scheduling controller is used to be scheduled the state of image data frame buffer zone, including
Whether detection image frame data buffer area is in the preprocessing image data that less than state and detection FPGA memory is handled
Raw image data is delivered in image data frame buffer zone, if FPGA by no completion if image data frame buffer zone is less than
Memory processing image be completed, the preprocessing image data in queue type buffer area is passed to, address management module for pair
The memory address of FPGA is managed, including carrying out the distribution of memory according to preparatory demand and according to the processing need of part of module
It asks and carries out dynamic memory application.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample, image calibration algorithm process include that preprocessing image data is passed through integrogram computing module, convolution difference
Closed operation processing, normalization, which are negated, corrodes module, coordinate set computing module, Span matching centre coordinate computing module, ROI convolution
The processes such as module and the post-processing of Cross coordinate are handled, and Δ X, Δ Y, Δ θ are finally obtained.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample, during integrogram computing module, the independent memory address etc. that can be cached using integrogram,
Track coordinate set calculates the coordinate caching generated, can be using the independent memory address etc. of coordinate caching.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample calculates the coordinate generated with Track coordinate set in the integrogram caching that integrogram computing module generates and caches,
The form of shared storage address can be used.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample, wherein normalization, which is negated, corrodes module including using primary corrosion or successive ignition etching operation.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample, wherein result internal memory cache region be set up according to preset address fixed internal memory cache region or
In real time according to the dynamic memory of the size of data application of calibration result data.
The ultrahigh resolution video image real-time calibration method based on FPGA provided according to the present invention, can also have this
The technical characteristic of sample, the process arranged to calibration result data are according to the corresponding pretreatment figure of each correction result data
The ID label generated as data in pretreatment carries out judgement arrangement,
Invention action and effect
The accelerated processing method of the ultra high-definition video image calibration provided according to the present invention, for ultrahigh resolution video
Large-scale image data collection directly carries out FPGA processing mode end to end by data transmission channel, reduces in calibration
Pass through the time overheads such as CPU transmission, the copy handled.During the operation of image calibration algorithm, deep learning has been used
Method has the advantage that compared to traditional images processing, avoids the missing of hand-designed parameter bring accuracy,
For this kind of demand relatively high to exact requirements of gene sequencing, actual effect is had more.Meanwhile in the inter-process of calibration
In, calibration process from thick to thin is combined, then obtain first rough offset as a result, detect for result whole
Reason, the final result obtained by this method have more accuracy.The present invention can be completed in the case where guaranteeing calibration accuracy
The real time calibration task of ultra high-definition video image.
Detailed description of the invention
Fig. 1 is the system architecture diagram of the ultra high-definition video image calibration in the embodiment of the present invention based on FPGA；
Fig. 2 is the flow chart of image calibration process in the embodiment of the present invention；
Fig. 3 is that the data of the embodiment of the present invention are sent and collection control logic schematic diagram；
Fig. 4 is the scheme schematic diagram of data interaction control logic in the embodiment of the present invention；And
Fig. 5 is in the embodiment of the present invention using the execution flow chart of the single-frame images data calibration of end-to-end FPGA.
Specific embodiment
Illustrate a specific embodiment of the invention below in conjunction with attached drawing and embodiment.
<embodiment>
The software systems of the present embodiment are realized on Windows server 2012, and ten thousand Broadcoms are used on hardware, are accelerated
Computing unit uses FPGA, totally 8 FPGA.
The present embodiment is process object with 4 road ultra high-definition video images, and video image single frames is 16bit image, resolution ratio
It is 5120 × 5120, frame per second is 20 frames/s.
The implementation goal of this example is, by embodiment end to end, by raw video image data directly around CPU
Processing, be fed directly to be handled in FPGA by input channels such as ten thousand Broadcoms, achieve the purpose that eliminate delay, in FPGA
In reach the processing of real-time, the offset of the centre coordinate of image is quickly calculated, is sat centered on finally obtained result
Target X-axis, the offset of Y-axis, the deflection angle of image, that is, Δ X, Δ Y, Δ θ.Then offset is calculated, and led to
The output channels such as ten thousand Broadcoms are crossed to return the result.
For video flowing, due to real-time transporting, so for whole flow process three phases can be divided into:
(1) video image input phase.Video image can not pass through CPU through but not limited to the form of ten thousand Broadcoms
It directly passes and is delivered to FPGA.
(2) FPGA processing stage.FPGA by carry out the decoding of video image, pretreatment, queue processing, status indication,
Integrogram computing module, convolution difference closed operation processing, normalization, which are negated, corrodes module, coordinate set computing module, Span matching
The processes such as centre coordinate computing module, ROI convolution module, the post-processing of Cross coordinate, obtain final calibration result Δ X, Δ Y,
Δθ。
(3) calibration result output stage.FPGA by internal final calibration result Δ X, Δ Y, Δ θ obtained by calculation,
The output of result is carried out through but not limited to forms such as ten thousand Broadcoms.
Fig. 1 is the system architecture diagram of the ultra high-definition video image calibration in the embodiment of the present invention based on FPGA.
Fig. 2 is the flow chart of image calibration process in the embodiment of the present invention.
As shown in Figures 1 and 2, the ultra high-definition video image calibration method of the present embodiment mainly includes 3 steps, i.e. step
S1~S3.
Wherein, step S1 is to establish raw image data receiving channel and by the raw image data of ultra high-definition video image
Directly FPGA is delivered to from computer.
In step sl, the process of transmission is the not time-consuming processing by CPU, is directly acquired from video capture device original
Video image data Raw Data, FPGA (i.e. data -> input -> FPGA number is conveyed directly to by the method for step S1
According to stream transmission procedure), in-between transmission channel (i.e. raw image data receiving channel) is PCIE, Gigabit Ethernet, 10,000,000,000 nets
Card etc. (such as Mellanox ConnetX-3Pro VPI network interface card, it supports 40Gbps throughput, both supports to pass by interchanger
It is defeated, also support direct interconnection mode to transmit).
In the present embodiment, the quantity of FPGA is one, is linearly handled raw image data by the FPGA.
Step S2 in Fig. 1 is the core processing part that FPGA accelerates, for carrying out calibration process to raw image data
It mainly include three sub-steps to obtain result calibration data: S2-1 image preprocessing, the processing of S2-2 image calibration, S2-
3 results arrange and output.
Step S2-1 establishes the image data frame buffer for keeping in raw image data, and to raw image data
It is pre-processed to obtain preprocessing image data.
In step S2-1, the process mainly calibrated for subsequent S2-2 step provides the preliminary place of image data
Reason.It mainly includes following several points that it, which is handled, it is necessary first to be decoded to image data, then carry out at frame sequence to it
Reason and the processing of ID label.
In the present embodiment, raw image data is decoded using linear codec processing method, frame sequenceization processing
For the queue type buffer area for establishing multiple first in first out to the preprocessing image data obtained after decoding, and by pretreatment image number
According to successively each input queue type buffer area, ID label is further carried out according to the timestamp of each preprocessing image data.
In other embodiments, queue type buffer area can also be one, and buffer to preprocessing image data according to the queue type is inputted
The sequence in area successively carries out ID label.
In step S2-1, image data frame buffer also has address management module and buffer scheduling controller.
Address management module is DMA address management module, the correct reference offer guarantee of the address FPGA is responsible for, to facilitate pair
The management of address in FPGA.Buffer scheduling controller is also used to video image data, thus utilization as much as possible
The memory source of FPGA.For example, being scheduled by state of the buffer scheduling controller to buffer area: whether detection buffer area
Less than state, if receive the Raw Data newly inputted；Whether detection frame queue has expired, if less than be delivered to image data
In frame queue；Whether the image of detection FPGA memory processing is completed, if being completed the image in queue is incoming etc..
Step S2-2 obtains Δ X, Δ Y, Δ θ in parallel through image calibration algorithm process preprocessing image data.
In step S2-2, the calibration algorithm for the video image mainly realized works.Wherein for input at
The image data of reason negates by integrogram computing module, convolution difference closed operation processing, normalization and corrodes module, coordinate set
The processes such as computing module, Span matching centre coordinate computing module, ROI convolution module, the post-processing of Cross coordinate, obtain final
Result Δ X, Δ Y, Δ θ.Wherein it should be noted that two pieces of cache units, i.e. integrogram cache and coordinate buffer area, herein
In embodiment, the design taken is to design two pieces of adjacent independent memory address in a memory as integrogram to cache and sit
Mark buffer area.
Step S2-3 establishes result internal memory cache region, and the Δ X that will be obtained, Δ Y, Δ θ carry out detection arrangement, obtains final
Calibration result data.
In step S2-3, one-time authentication is mainly carried out again to final result Δ X, Δ Y, Δ θ, because in step
During S2-2, it can not centainly guarantee that the ID number of final output result and input is correspondingly, in order to solve to compile
Number consistency, need to carry out verification process, that is, judge calibration result data ID and preprocessing image data ID whether one
It causes: when the ID of calibration result data is consistent with the ID of the preprocessing image data inputted before, continuing with subsequent place
Reason；As the ID of calibration result data and the ID of the preprocessing image data inputted before inconsistent, then by the calibration result number
Image calibration algorithm process is carried out again according to deletion, and by corresponding preprocessing image data, obtains new calibration result number
According to the further verifying of the calibration result data progress ID new to this.
In addition, the numerical value of Δ Y, Δ θ are according to preset calibration also to Δ X in calibration result data in the present embodiment
As a result threshold value does the judgement of a size, it is avoided the excessive problem of numerical value occur, causes the significantly deviation of calibration result, shadow
Ring calibration effect.
In the present embodiment, as a result internal memory cache region for store calibration result data and to the calibration result data of caching into
Row detection arranges, which is that fixed internal memory cache region is set up according to preset address.In other realities
It applies in example, as a result internal memory cache region can also be the dynamic memory of the size of data application according to calibration result data: Mei Dangbu
As soon as rapid S2-2 exports group calibration result data, address management module applies for the size of data of corresponding one group of calibration result data
Memory as a result internal memory cache region a part, whenever storage calibration result number innings arrangement finish and by step S3 data it is defeated
After out, address management module just removes the address of corresponding part from result internal memory cache region.
Step S3 establishes calibration result output channel, and result calibration data is returned to computer from FPGA.
The step S3 of the present embodiment with the effect of step S1 be it is corresponding, this step is the school that will be handled in FPGA
It is quasi- as a result, without CPU etc. processing, be directly sent at result receiving by transmission channel (i.e. calibration result output channel)
(i.e. data flow at FPGA calibration result -> output -> HOST data receiving).The carrier of its transmission channel is PCIE, kilomega network
Card, ten thousand Broadcoms etc. (such as Mellanox ConnetX-3Pro VPI network interface card).
Fig. 3 is that the HOST end data of the embodiment of the present invention is sent and collection control logic schematic diagram.
As shown in figure 3, entire data flow is considered as process end to end, but in order to accelerate the overall operation of process
Time is provided with caching in FPGA.During data transmission and data receiver, since there is when the transmission of data
Between, an input rank is preset during image input, such as, two data 1 and data 2 are continuously inputted,
It, can be by 3 input rank of image data during handling image data 1.And so on, available one is as shown in Figure 3
Input data 1, input data 2, receive result 1, input data 3, receive result 2..... input data N, receive result N-
1, receive the processing sequence of result N, reached with this and send and receive the time using FPGA calculating time obfuscated data.
Fig. 4 is the scheme schematic diagram of the data interaction control logic of the embodiment of the present invention.
As shown in figure 4, mainly using ping-pong operation for FPGA context interactive controlling logic.I.e. for inputting,
Two pieces of memory spaces are opened up on DDR4, such as I1, I2；For output, similar also opens up two pieces of output memory spaces on DDR4,
Such as O1, O2.In this way, execute the condition of calculating for FPGA calculating as I1, I2 at least block space and have data, and
At least one piece of space O1 or O2 of output is empty.Cooperate the data transmission and reception logic at the end HOST in this way, it is entire to calculate
The influence of data transmission period can effectively be lowered.Certainly, it for input, is completed every time when the corresponding input space calculates,
Just the space is discharged, for output, whenever result is read by HOST, also corresponding space is discharged.Guarantee
There is space to continue to store the result of input data and generation.
The scheme of data interaction control logic shown in Fig. 4, the step S2 corresponding to Fig. 1.Pass through Fig. 4, it can be seen that step
DMA address control unit in rapid S2-1, can complete opening for the memory space of data interaction control logic unit shown in Fig. 4
It wards off.For wherein inputting corresponding at least one non-empty of I1, I2 memory space, and output space O1 or O2 met to
Few one piece is empty condition, can be completed by the scheduling controller in step S2.It can clearly be obtained by Fig. 4 and Fig. 1,
In FPGA, by individually setting up DMA address control module, the memory address of each memory space in FPGA is controlled with this,
It is convenient for managing.On the other hand, by scheduling controller, one very easily can be done to the image sequence for being input to FPGA
Regulation.From the point of view of details, when image frame sequence is discontented with, new image sequence is inputted into queue, when queue has been expired, is prevented new
Image carries out frame sequence.At the same time, need to meet in I1, I2 memory space in Fig. 4 at least one non-empty, in O1, O2
At least one piece is sky, and the dynamic equilibrium of data transmission and reception is reached with this, so that entire calculate effectively lowers data transmission
The influence of time.
Fig. 5 is in the embodiment of the present invention using the execution flow chart of the single-frame images data calibration of end-to-end FPGA.
As shown in figure 5, single frame video image execution flow chart is broadly divided into three parts, input picture and output result portion
Divide, I/O channel part, FPGA processing part.Live video stream of the input picture from connection, the speed of 20 frame per second
Rate.The form for exporting result is offset of the image coordinate compared to the x-axis and y-axis of the centre coordinate of basis coordinates system, and deflection
Angle delta X, Δ Y, Δ θ.It is transmitted to console (i.e. computer) by these three offsets, carries out subsequent operation.
Embodiment action and effect
According to the accelerated processing method that ultra high-definition video image provided in this embodiment is calibrated, for ultrahigh resolution video
Large-scale image data collection, calibration when, FPGA processing mode end to end is directly carried out by data transmission channel, reduce
Pass through the time overheads such as transmission, the copy of CPU processing.During the operation of image calibration algorithm, deep learning has been used
Method, compared to traditional images processing, have the advantage that, avoid hand-designed parameter bring accuracy lack
It loses, for this kind of demand relatively high to exact requirements of gene sequencing, has more actual effect.Meanwhile in the inside of calibration
In processing, calibration process from thick to thin is combined, obtains being detected as a result, being then directed to result for rough offset first
It arranges, the final result obtained by this method has more accuracy.The present invention, can be complete in the case where guaranteeing calibration accuracy
At the real time calibration task of ultra high-definition video image.
In embodiment, due to passing through raw image data receiving channel and calibration result output channel directly to original graph
As data and calibration result are transmitted, therefore reduce in traditional treatment method, the time-consuming of processing is decoded by CPU
Problem, the image data for acquiring video acquisition device is before being delivered to FPGA without other extra processing.
In embodiment, the quantity of FPGA is one, and is linearly handled by the FPGA raw image data.Except this
Except, the mode (i.e. the quantity of FPGA is multiple) that the present invention can also be cooperated using more FPGA simultaneously, therefore original graph
As data receiving channel can be connected with each FPGA, so that raw image data, which is sent to each FPGA, carries out multiframe figure
The parallel processing of picture, it is a greater degree of that image calibration is accelerated to execute speed.It is handled by the image calibration acceleration based on FPGA,
Such as in the treatment process of gene sequencing, identical data volume, speed promoted decades of times, can considerably reduce calibration when
Between cost and resources costs, the cumulative delay during better Processing for removing.
In embodiment, it is asymmetric to cause information result for calibration data and raw image data in order to prevent, therefore to original
It is the setting that each image data carries out ID coding that beginning data image, which carries out pretreatment,.
In embodiment, due to being scheduled by state of the frame buffer zone scheduling controller to image data frame buffer zone,
And it is managed by memory address of the address management module to FPGA, therefore the shape during data transmission and reception of FPGA
At a kind of dynamic equilibrium, to effectively lower the influence of data transmission period in entire calculating process.
In embodiment, integrogram caching and coordinate caching use memory address independent, since video image exists
Calibration process inside FPGA is linear process process, so handling primary time speed is limited to slow integral fortune
It calculates, in order to make up this deficiency, additionally one buffer area of setting is cached for integrogram, to accomplish not delay other part just
Often processing.In addition to this, integrogram caching and coordinate caching can also use the form of shared storage address, because of calculation amount
There is the hysteresis quality of calculated result, the form of common memory can increase the utilization rate of memory in big process.
In embodiment, as a result internal memory cache region is used to cache the calibration result data for carrying out arrangement to be detected, by
In the result internal memory cache region be the fixed memory buffer area set up according to preset address, therefore can be more right in an orderly manner
Calibration result data are cached.In addition to this, result internal memory cache region of the invention can also be in real time according to calibration result number
According to the dynamic memory of size of data application avoid idle address to preferably utilize to the memory of FPGA totality
Excessively.
In embodiment, carried out by the ID of ID and the preprocessing image data inputted before to calibration result data consistent
Property detect and detection arrangement is carried out to the numerical values recited of the calibration result data, to realize further to calibration result data
Inspection, avoiding calibration result corrupt data causes data inaccurate.
Above-described embodiment is only used for the specific embodiment illustrated the present invention, and the present invention is not limited to the above embodiments
Description range.
In embodiment, calibration result data include Δ X, Δ Y, Δ θ.In other embodiments, calibration result of the invention
Data can also be Δ X, the permutation and combination both in Δ Y, Δ θ, even the 4th of greater demand end value.
Claims (10)
1. a kind of ultrahigh resolution video image real-time calibration method based on FPGA, for passing through FPGA to ultra high-definition video figure
As carrying out real time calibration, which comprises the steps of:
Step S1 establishes raw image data receiving channel, directly that the raw image data of the ultra high-definition video image is defeated
It send to the FPGA；
Step S2 carries out calibration process to the raw image data, obtains result calibration data, which includes
Offset Δ X of the centre coordinate of the ultra high-definition video image in X-axis, the offset Δ Y in Y-axis and deflection angle Δ θ；
Step S3 establishes calibration result output channel, and the result calibration data is returned from FPGA,
Wherein, the step S2 includes following sub-step:
Step S2-1 establishes the image data frame buffer for keeping in the raw image data, and to the original image
Data are pre-processed to obtain preprocessing image data, described image data frame buffer memory area include frame buffer zone scheduling controller with
And address management module；
Step S2-2 is handled by image calibration algorithm the preprocessing image data and is obtained Δ X, Δ Y, Δ θ；
Step S2-3 establishes result internal memory cache region, and the Δ X that will be obtained, Δ Y, Δ θ carry out detection arrangement, obtains final institute
State calibration result data.
2. the ultrahigh resolution video image real-time calibration method according to claim 1 based on FPGA, it is characterised in that:
Wherein, the transmission medium of the raw image data receiving channel and the calibration result output channel is 10,000,000,000 nets
Card, Gigabit Ethernet or PCIE.
3. the ultrahigh resolution video image real-time calibration method according to claim 1 based on FPGA, it is characterised in that:
Wherein, the quantity of the FPGA is one or more.
4. the ultrahigh resolution video image real-time calibration method according to claim 1 based on FPGA, it is characterised in that:
Wherein, the pretreatment are as follows:
First to the raw image data carry out preliminary decoder handle to obtain preprocessing image data, after to the pretreatment figure
It is further that each of frame sequence preprocessing image data carries out personalization ID mark as data progress frame sequence processing
Note processing.
5. the ultrahigh resolution video image real-time calibration method according to claim 4 based on FPGA, it is characterised in that:
Wherein, the preliminary decoder processing uses linear codec processing method,
The frame sequenceization processing is that one or more fifo queue types buffer area is established to the preprocessing image data,
The personalization ID label processing is by the timestamp progress ID label of the preprocessing image data or by described pre-
Handle the sequencing progress ID label that image data enters queue type buffer area.
6. the ultrahigh resolution video image real-time calibration method according to claim 5 based on FPGA, it is characterised in that:
Wherein, the frame buffer zone scheduling controller is used to be scheduled the state in described image frame data buffer area, including
Whether detection described image frame data buffer area is in the pretreatment figure of less than state and detection FPGA memory processing
As whether data are completed,
The raw image data is delivered to described image frame data buffer area if described image frame data buffer area is less than
In,
The preprocessing image data in queue type buffer area is passed to if the image of FPGA memory processing is completed,
The address management module carries out memory for being managed to the memory address of the FPGA, including according to preparatory demand
Distribution and dynamic memory application is carried out according to the process demand of part of module.
7. the ultrahigh resolution video image real-time calibration method according to claim 1 based on FPGA, it is characterised in that:
Wherein, the processing of described image calibration algorithm includes that the preprocessing image data is passed through integrogram computing module, convolution
Difference closed operation processing, normalization, which are negated, corrodes module, coordinate set computing module, Span matching centre coordinate computing module, ROI
The processes such as convolution module and the post-processing of Cross coordinate are handled, and Δ X, Δ Y, Δ θ are finally obtained.
8. the ultrahigh resolution video image real-time calibration method according to claim 7 based on FPGA, it is characterised in that:
Generated during the integrogram computing module integrogram caching and each coordinate calculate during by
Track coordinate set calculates the coordinate caching generated using memory address independent or uses shared memory address,
The normalization, which is negated, corrodes module including using primary corrosion or successive ignition etching operation.
9. the ultrahigh resolution video image real-time calibration method according to claim 1 based on FPGA, it is characterised in that:
Wherein, the result internal memory cache region is that fixed internal memory cache region or real-time root are set up according to preset address
According to the dynamic memory of the size of data application of the calibration result data.
10. the ultrahigh resolution video image real-time calibration method according to claim 9 based on FPGA, feature exist
In:
Wherein, the process for carrying out detection arrangement to the calibration result data is corresponding according to each correction result data
The ID label that the preprocessing image data is generated in pretreatment carries out judgement arrangement,
As the ID of the calibration result data and the ID of the preprocessing image data inputted before inconsistent, then the calibration is tied
Fruit data are deleted, and corresponding preprocessing image data is carried out image calibration algorithm process again, obtain new calibration knot
Fruit data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910276412.0A CN109995965B (en) | 2019-04-08 | 2019-04-08 | Ultrahigh-resolution video image real-time calibration method based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910276412.0A CN109995965B (en) | 2019-04-08 | 2019-04-08 | Ultrahigh-resolution video image real-time calibration method based on FPGA |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109995965A true CN109995965A (en) | 2019-07-09 |
| CN109995965B CN109995965B (en) | 2021-12-03 |
Family
ID=67132544
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910276412.0A Active CN109995965B (en) | 2019-04-08 | 2019-04-08 | Ultrahigh-resolution video image real-time calibration method based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109995965B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113358825A (en) * | 2021-06-02 | 2021-09-07 | 重庆大学 | Indoor air quality detector with assimilation algorithm |
| CN114979481A (en) * | 2022-05-23 | 2022-08-30 | 深圳市海创云科技有限公司 | 5G ultra-high-definition video monitoring system and method |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103714531A (en) * | 2013-12-05 | 2014-04-09 | 南京理工大学 | FPGA-based phase correlation method image registration system and method |
| US20140139541A1 (en) * | 2012-10-18 | 2014-05-22 | Barco N.V. | Display with optical microscope emulation functionality |
| CN104296876A (en) * | 2013-07-15 | 2015-01-21 | 南京理工大学 | FPGA-based scene non-uniformity correction method and device |
| CN104363383A (en) * | 2014-10-16 | 2015-02-18 | 青岛歌尔声学科技有限公司 | Image pre-distortion correction method and device |
| CN106250939A (en) * | 2016-07-30 | 2016-12-21 | 复旦大学 | System for Handwritten Character Recognition method based on FPGA+ARM multilamellar convolutional neural networks |
| CN106982356A (en) * | 2017-04-08 | 2017-07-25 | 复旦大学 | A kind of distributed extensive video flow processing system |
| CN107168782A (en) * | 2017-04-24 | 2017-09-15 | 复旦大学 | A kind of concurrent computational system based on Spark and GPU |
| CN107426466A (en) * | 2017-07-25 | 2017-12-01 | 中国科学院长春光学精密机械与物理研究所 | A kind of TDICCD imaging systems non-uniform noise Quick correction device and bearing calibration |
| US20180262744A1 (en) * | 2017-02-07 | 2018-09-13 | Mindmaze Holding Sa | Systems, methods and apparatuses for stereo vision |
-
2019
- 2019-04-08 CN CN201910276412.0A patent/CN109995965B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140139541A1 (en) * | 2012-10-18 | 2014-05-22 | Barco N.V. | Display with optical microscope emulation functionality |
| CN104296876A (en) * | 2013-07-15 | 2015-01-21 | 南京理工大学 | FPGA-based scene non-uniformity correction method and device |
| CN103714531A (en) * | 2013-12-05 | 2014-04-09 | 南京理工大学 | FPGA-based phase correlation method image registration system and method |
| CN104363383A (en) * | 2014-10-16 | 2015-02-18 | 青岛歌尔声学科技有限公司 | Image pre-distortion correction method and device |
| CN106250939A (en) * | 2016-07-30 | 2016-12-21 | 复旦大学 | System for Handwritten Character Recognition method based on FPGA+ARM multilamellar convolutional neural networks |
| US20180262744A1 (en) * | 2017-02-07 | 2018-09-13 | Mindmaze Holding Sa | Systems, methods and apparatuses for stereo vision |
| CN106982356A (en) * | 2017-04-08 | 2017-07-25 | 复旦大学 | A kind of distributed extensive video flow processing system |
| CN107168782A (en) * | 2017-04-24 | 2017-09-15 | 复旦大学 | A kind of concurrent computational system based on Spark and GPU |
| CN107426466A (en) * | 2017-07-25 | 2017-12-01 | 中国科学院长春光学精密机械与物理研究所 | A kind of TDICCD imaging systems non-uniform noise Quick correction device and bearing calibration |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113358825A (en) * | 2021-06-02 | 2021-09-07 | 重庆大学 | Indoor air quality detector with assimilation algorithm |
| CN113358825B (en) * | 2021-06-02 | 2023-03-24 | 重庆大学 | Indoor air quality detector with assimilation algorithm |
| CN114979481A (en) * | 2022-05-23 | 2022-08-30 | 深圳市海创云科技有限公司 | 5G ultra-high-definition video monitoring system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109995965B (en) | 2021-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102200906B (en) | Processing system and processing method of large-scale concurrent data stream | |
| US10241799B2 (en) | Out-of-order command execution with sliding windows to maintain completion statuses | |
| US9990412B2 (en) | Data driven parallel sorting system and method | |
| CN109995965A (en) | A kind of ultrahigh resolution video image real-time calibration method based on FPGA | |
| CN102254297A (en) | Multiple shader system and processing method thereof | |
| CN106951322A (en) | The image collaboration processing routine acquisition methods and system of a kind of CPU/GPU isomerous environments | |
| CN106454354B (en) | A kind of AVS2 parallel encoding processing system and method | |
| CN112580481B (en) | Edge node and cloud collaborative video processing method, device and server | |
| CN110851255B (en) | Method for processing video stream based on cooperation of terminal equipment and edge server | |
| CN109167979B (en) | Processing method and system for artificial intelligence analysis of multi-path monitoring videos | |
| Lin et al. | Training kinetics in 15 minutes: Large-scale distributed training on videos | |
| WO2022011841A1 (en) | Implementation method, apparatus, terminal for cluster in gpgpu, and medium | |
| CN101640795B (en) | Video decoding optimization method and device | |
| CN105205476A (en) | Face recognition hardware framework based on LBP characteristics | |
| CN112749011A (en) | FPGA (field programmable Gate array) acceleration method based on non-maximum suppression algorithm | |
| He et al. | FPGA-based high definition image processing system | |
| CN111343393A (en) | Ultrahigh-definition video picture cutting method | |
| Stone et al. | Towards scalable video analytics at the edge | |
| Lu et al. | Dynamic offloading on a hybrid edge–cloud architecture for multiobject tracking | |
| CN108845794A (en) | A kind of streaming operation frame, method, readable medium and storage control | |
| Geninatti et al. | FPGA implementation of the generalized Hough transform | |
| CN115190303A (en) | Cloud desktop image processing method and system and related equipment | |
| CN113869141A (en) | Feature extraction method and device, encoder and communication system | |
| Khalil et al. | An Enhanced System on Chip-Based Sobel Edge Detector | |
| CN107480616B (en) | Skin color detection unit analysis method and system based on image analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |