CN112616031B - High-speed target tracking method and system based on pulse array image sensor - Google Patents
High-speed target tracking method and system based on pulse array image sensor Download PDFInfo
- Publication number
- CN112616031B CN112616031B CN202011483754.9A CN202011483754A CN112616031B CN 112616031 B CN112616031 B CN 112616031B CN 202011483754 A CN202011483754 A CN 202011483754A CN 112616031 B CN112616031 B CN 112616031B
- Authority
- CN
- China
- Prior art keywords
- data
- pulse
- frame
- maximum value
- image sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/62—Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels
- H04N25/625—Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels for the control of smear
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
A high-speed target tracking method and a system based on a pulse array image sensor belong to the technical field of high-speed target tracking based on a bionic visual image sensor, and solve the problems of reducing data processing time and improving algorithm instantaneity and precision by designing a high-speed target tracking method aiming at a 1-bit special pulse data format of the pulse array image sensor; traversing rows and columns of single-frame pulse data of the pulse array image sensor by a frame, positioning an area with the most dense pulse distribution in the single-frame pulse data, receiving LVDS data of the pulse array image sensor by the algorithm, and greatly reducing data processing time and improving the real-time property of the algorithm through data processing in a pipeline form; by using the tailing compensation algorithm, the positioning accuracy of the algorithm is improved, the target can be positioned in a short time, and the high-speed target tracking under the condition of 40Kfps can be realized.
Description
Technical Field
The invention belongs to the technical field of high-speed target tracking based on a bionic visual image sensor, and particularly relates to a high-speed target tracking method and system based on a pulse array image sensor.
Background
The development of science and technology enables artificial intelligence to enter the daily life of people. High speed object tracking systems are also receiving increasing attention as an important component of machine vision. The position information of the target is quickly and accurately positioned, and is an important index of the sensitivity of the machine. However, as the frame rate and resolution of the image sensor increase, the amount of data which increases dramatically also puts a great deal of pressure on data processing. Taking a conventional image sensor as an example, the resolution is 1920 × 1080, the frame rate is 1000fps, and the output data is 8-bit gray scale data, and the sensor generates 16.59Gb data per second. The enormous data puts a great strain on the real-time processing of the data.
In recent years, more and more high-speed image sensors have been proposed. A pulse array image sensor is a high-speed image sensor that can convert an optical signal into a digital pulse signal. The image sensor adopts a traditional frame scanning reading mode to ensure the integrity of data. The continuity of the collected light intensity information is ensured by adopting an asynchronous reset working mode among the pixels. And finally, 1-bit pulse data is output, so that higher frame frequency can be realized under the condition of limited bandwidth.
In the prior art, the chinese patent application CN201811608766 with publication date of 2020, 07/10, discloses an algorithm for removing smear of reconstructed images applied to a high-speed pulse sensor, and discloses a method for removing a smear part by calculating the number of cycles occupied by target light in actual smear through a background pulse interval, a target light pulse interval and a smear pulse interval. However, in practical applications, the gray scale of the tail portion is gradually changed, and an accurate pulse interval of the tail portion cannot be obtained, so that there are some limitations in terms of practicality.
Conventional image sensors use 8-bit or 10-bit data to represent optical information, with synchronous reset between pixels. Due to the different operating modes, conventional target tracking algorithms cannot be applied to the pulse data generated by the pulse array image sensor. The integration period of the pulse array image sensor is a plurality of frame periods, and accurate gray scale information of a shooting scene cannot be obtained in a single frame period.
Disclosure of Invention
The invention aims to solve the technical problem of how to design a high-speed target tracking method aiming at a 1-bit special pulse data format of a pulse array image sensor, reduce the data processing time and improve the real-time property and the precision of an algorithm.
The invention solves the technical problems through the following technical scheme:
the high-speed target tracking method based on the pulse array image sensor comprises the following steps:
firstly, traversing rows and columns of single-frame pulse data of a pulse array image sensor by a frame according to a 1-bit pulse data format of the pulse array image sensor, inputting two paths of adjacent LVDS pulse data of the pulse array image sensor, processing the LVDS pulse data, starting accumulation, and counting to obtain a single-channel accumulation maximum value;
step two, according to the step one, two paths of adjacent LVDS pulse data of the input pulse array image sensor of 8 channels are processed in parallel to obtain the accumulated maximum value of each channel, the maximum value in the 8 channels is taken as the maximum value of single frame pulse data statistics, and the position of the maximum value of the statistics is taken as a positioning coordinate;
and step three, calculating to obtain the image trailing length, adding the image trailing length to the positioning coordinate obtained in the step two for coordinate compensation, and outputting the compensated positioning coordinate.
The invention provides a pulse positioning algorithm aiming at a 1-bit special pulse data format of a pulse array image sensor, traverses rows and columns of single-frame pulse data of the pulse array image sensor by a frame, positions an area with the most dense pulse distribution in the single-frame pulse data, receives LVDS (Low-Voltage Differential Signaling) data of the pulse array image sensor, and performs data processing in a pipeline form, thereby greatly reducing data processing time and improving the real-time property of the algorithm. By using the tailing compensation algorithm, the positioning accuracy of the algorithm is improved, the target can be positioned in a short time, and the high-speed target tracking under the condition of 40Kfps can be realized.
As a further improvement of the technical solution of the present invention, the process of processing LVDS pulse data in the step one specifically includes:
(1) Converting a differential signal of the input single-channel LVDS pulse data into a single-ended signal by utilizing a Xilinx FPGA device primitive IBUFDS;
(2) Performing serial-parallel conversion on the converted single-ended signal through data shift, outputting pulse data of X rows of pixels by using the single-channel LVDS, taking the frame effective signal as a starting node, and taking every X pulse data as one line;
(3) After LVDS data are converted, two paths of adjacent converted LVDS data are spliced together, and the spliced data are pulse data of adjacent 2X pixels on the same row, wherein X is a positive integer.
As a further improvement of the technical solution of the present invention, the obtaining of the accumulated maximum value of each channel in step two specifically includes:
after the splicing data is input, accumulating the adjacent N data: m is to be E-mailThe cumulative sum of (N + m-1) (m =1,2, …, X) is recorded as a m (m＝1，2,…,X)；
Wherein the parameter c M For storing the accumulated result of the input concatenated data, c M Is 0, where M denotes the mth row of the box;
after the single-frame data output by the pulse array image sensor is positioned, all parameters in the algorithm are reset, so that the accuracy of positioning of each frame of data is ensured; c. C M The expression of (a) is as follows:
c M ＝(a 1 ,a 2 ,...,a X ) (1)
when c is M When collecting new accumulation results, parameter c j ' (j = 2.. Multidot.M) for saving c j Data before data change, and c j ' previously saved data transfer to c j-1 (ii) a So when c is j When a change occurs, the data will move by:
c j-1 ＝c′ j (2)
wherein, c j (a m ) For indicating storage in c j In the accumulated result a m The data of (a); path1 output c 1 Path 2 output a m (m＝1，2,…,X),c 1 (a m ) A representing delay of M line periods m A value of (d);
b i ＝b i +a i -c 1 (a i )(i＝1,2,...,X) (3)
wherein, the formula (3) is used for calculating the number of pulses in the frame, and after the calculation of the first M rows is completed, the number of pulses in the X frames of the first M rows is obtained, namely b i A value of (d); in the calculation of b i (i =1,2, …, X), and then b is compared i A value in between.
As a further improvement of the technical solution of the present invention, the maximum value in the 8 channels in step three is taken as the maximum value of the single frame pulse data statistics, specifically:
(1) Presetting single-channel statistical maximum b max Is 0，b i Maximum value of (b) i_max And b max Comparing;
(2) If b is i_max Greater than b max Then b is i_max Is assigned to b max And recording the coordinate information at the moment;
(3) After M lines, each line will calculate a new b i And performing a previous comparison;
(4) Finally, when the whole frame data transmission is finished, each channel has only one b max ；
(5) By comparing b of the respective channels max And obtaining the maximum value b _ max of the single frame statistics.
As a further improvement of the technical scheme of the present invention, the image trailing length obtained by the calculation in step three is specifically:
suppose the current frame is Nth e Frame, target at N e Distance moved within a frame divided by N e Obtaining the average speed v of the target in the interval 0 (ii) a Before starting to perform high-speed target tracking, the positions of the target and the background are determined by performing a frame difference between two frames spaced by 64 frames, and the pulse interval N of the background light can be obtained b And the pulse interval N of the target light e (ii) a In the process of real-time target tracking, the speed of the current moment is represented by the position difference between two coordinate vectors with an interval of 7 frames, and then the length of image tailing is calculated according to the following formula (4):
after the image trailing length is obtained through calculation, the image trailing length is added to the output coordinate of the algorithm to perform coordinate compensation; finally, outputting the compensated positioning coordinates, wherein N 0 Is the number of pixel cells passed by the movement in unit time.
A high-speed target tracking system applying the high-speed target tracking method based on the pulse array image sensor comprises the following steps:
the single-channel maximum value calculation module is used for traversing rows and columns of single-frame pulse data of the pulse array image sensor by a frame according to a 1-bit pulse data format of the pulse array image sensor, inputting two paths of adjacent LVDS pulse data of the pulse array image sensor, processing the LVDS pulse data, starting accumulation, and counting to obtain a single-channel accumulated maximum value;
the 8-channel maximum value calculating module is used for parallelly processing two paths of adjacent LVDS pulse data of the input pulse array image sensor of 8 channels to obtain the accumulated maximum value of each channel, the maximum value in the 8 channels is taken as the maximum value of single-frame pulse data statistics, and the position where the maximum value is counted is taken as a positioning coordinate;
and the compensation positioning module is used for calculating to obtain the image trailing length, adding the image trailing length to the positioning coordinates obtained in the second step to perform coordinate compensation, and outputting the compensated positioning coordinates.
As a further improvement of the technical solution of the present invention, the process of processing the LVDS pulse data in the single-channel maximum value calculation module specifically includes:
(1) Converting a differential signal of the input single-channel LVDS pulse data into a single-ended signal by utilizing a Xilinx FPGA device primitive IBUFDS;
(2) Performing serial-parallel conversion on the converted single-ended signal through data shift, outputting pulse data of X rows of pixels by using the single-channel LVDS, taking the frame effective signal as a starting node, and taking every X pulse data as one line;
(3) After LVDS data are converted, two paths of adjacent converted LVDS data are spliced together, and the spliced data are pulse data of adjacent 2X pixels on the same row, wherein X is a positive integer.
As a further improvement of the technical solution of the present invention, the obtaining of the cumulative maximum value of each channel in the 8-channel maximum value calculation module specifically includes:
after the splicing data is input, accumulating the adjacent N data: recording the cumulative sum of m- (N + m-1) (m =1,2, …, X) as a m (m＝1，2,…,X)；
Wherein the parametersc M For storing the accumulated result of the input concatenated data, c M Is 0, where M denotes the mth row of the box;
after the single-frame data output by the pulse array image sensor is positioned, all parameters in the algorithm are reset, so that the accuracy of positioning of each frame of data is ensured; c. C M The expression of (c) is as follows:
c M ＝(a 1 ,a 2 ,...,a X ) (1)
when c is going to M When collecting new accumulation results, parameter c j ' (j = 2.. Multidot.M) for saving c j Data before data change, and c j ' previously saved data transfer to c j-1 (ii) a So when c is j When a change occurs, the data will move by:
c j-1 ＝c′ j (2)
wherein, c j (a m ) For indicating storage in c j In the accumulated result a m The data of (a); path1 output c 1 Path 2 output a m (m＝1，2,…,X),c 1 (a m ) A representing delay of M line periods m A value of (d);
b i ＝b i +a i -c 1 (a i )(i＝1,2,...,X) (3)
wherein, the formula (3) is used for calculating the number of pulses in the frame, and after the calculation of the first M rows is completed, the number of pulses in the X frames of the first M rows is obtained, namely b i A value of (d); in the calculation of b i (i =1,2, …, X), and then b is compared i A value in between.
As a further improvement of the technical solution of the present invention, the maximum value of the 8 channels in the compensation positioning module is taken as the maximum value of the single frame pulse data statistics, and specifically:
(1) Presetting single-channel statistical maximum b max Is 0,b i Maximum value of (b) i_max And b max Comparing;
(2) If it is notb i_max Greater than b max Then b is i_max Is assigned to b max And recording the coordinate information at the moment;
(3) After M lines, each line will calculate a new b i And performing a previous comparison;
(4) Finally, when the whole frame data transmission is finished, each channel has only one b max ；
(5) By comparing b of the respective channels max And obtaining the maximum value b _ max of the single frame statistics.
As a further improvement of the technical solution of the present invention, the calculating in the compensation positioning module to obtain the image trailing length specifically includes: suppose the current frame is Nth e Frame, target at N e Distance moved within a frame divided by N e Obtaining the average speed v of the target in the interval 0 (ii) a Before starting to perform high-speed target tracking, the positions of the target and the background are determined by performing a frame difference between two frames spaced by 64 frames, and the pulse interval N of the background light can be obtained b And the pulse interval N of the target light e (ii) a In the process of real-time target tracking, the speed of the current moment is represented by the position difference between two coordinate vectors with an interval of 7 frames, and then the length of image tailing is calculated according to the following formula (4):
after the image trailing length is obtained through calculation, the image trailing length is added to the output coordinate of the algorithm to perform coordinate compensation; finally, outputting the compensated positioning coordinate, wherein N 0 Is the number of pixel cells passed by the movement in unit time.
The invention has the advantages that:
the invention provides a pulse positioning algorithm aiming at a 1-bit special pulse data format of a pulse array image sensor, traverses rows and columns of single-frame pulse data of the pulse array image sensor by a frame, positions an area with the most dense pulse distribution in the single-frame pulse data, receives LVDS (Low-Voltage Differential Signaling) data of the pulse array image sensor, and performs data processing in a pipeline form, thereby greatly reducing data processing time and improving the real-time property of the algorithm. By using the tailing compensation algorithm, the positioning accuracy of the algorithm is improved, the target can be positioned in a short time, and the high-speed target tracking under the condition of 40Kfps can be realized.
Drawings
FIG. 1 is a flow chart of a high speed target tracking method based on a pulse array image sensor;
FIG. 2 is a flow chart diagram of a parallel pulse counting method;
FIG. 3 is a flow chart of a method of calculating a single frame statistical maximum;
fig. 4 is a flowchart of a smear effect elimination method.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The technical scheme of the invention is further described by combining the drawings and specific embodiments in the specification:
example one
As shown in fig. 1, after data is input, accumulation is started, pulse statistics of the first 50 blocks is performed first, and after the statistics is completed, comparison is performed to obtain a statistical maximum value of the 50 blocks. And in the process of inputting data line by line, comparing statistical data, and obtaining the maximum value of single-channel statistics when the single-frame data statistics is finished. And comparing the 8 channels to obtain the position with the maximum single-frame pulse statistics. And finally, eliminating tailing influence to obtain a positioning result of the algorithm and outputting the positioning result.
As shown in fig. 2, the input of the single-channel pulse statistics is two adjacent LVDS (Low-Voltage Differential Signaling) pulse data output by the sensor; the serial input data of LVDS needs to be converted into parallel data: firstly, converting differential signals into single-ended signals, and converting the input differential signals into the single-ended signals by utilizing Xilinx FPGA device primitive IBUFDS. And performing serial-parallel conversion on the converted single-ended signal through data shifting, and outputting pulse data of 50 columns of pixels by the single-channel LVDS. The frame valid signal is pulled high as a starting node, and data is in a row every 50 pulses. After LVDS data is converted, it is passed to the next module. Next, two adjacent converted LVDS data paths are spliced together. The data after stitching is the impulse data of the adjacent 100 pixels on the same line.
And after splicing data are input, accumulating the N adjacent data. Recording the cumulative sum of m (N + m-1) (m =1,2, …, 50) as a m (m =1,2, …, 50). Parameter c M For storing the accumulated result of the input concatenated data, c M Is 0, where M denotes the mth row of the block. After the single-frame data output by the pulse array image sensor is positioned, all parameters in the algorithm are reset, so that the accuracy of positioning of each frame of data is ensured; c. C M The expression of (c) is as follows:
c M ＝(a 1 ,a 2 ,...,a 50 ) (1)
when c is going to M When collecting new accumulation results, parameter c j ' (j = 2.. Multidot.M) for saving c j Data before data change, and c j ' previously saved data transfer to c j-1 . So when c is j When a change occurs, the data will move by:
c j-1 ＝c′ j (2)
c j (a m ) For indicating storage in c j In the accumulated result a m The data of (1). Equation (3) is used to calculate the number of pulses in the box.As shown in FIG. 2, path1 outputs c 1 Path 2 output a m (m＝1，2,…,50)。c 1 (a m ) A representing delay of M line periods m The value of (c). For example, when path 2 outputs a for row M +1 m When a value of (c) 1 (a m ) Keep a of row 1 m The value of (c). b i (i =1,2, …, 50) is used to represent the value of the pulse statistics in 50 boxes.
b i ＝b i +a i -c 1 (a i )(i＝1,2,...,50) (3)
After the calculation of the first M lines is completed, the number of pulses in the 50 boxes of the first M lines, i.e., b, can be obtained i The value of (c). In the calculation of b i (i =1,2, …, 50), and then b is compared i A value in between.
B to be calculated as shown in FIG. 3 i Are compared with each other to obtain a maximum value b therebetween i_max . Presetting single-channel statistical maximum b max Is 0,b i Maximum value of (b) i_max And b max A comparison is made. If b is i_max Greater than b max Then b is i_max Is assigned to b max And recording the coordinate information at this time. After M lines, each line will calculate a new b i And the previous comparison is made. Finally, when the whole frame data transmission is finished, each channel has only one b max . By comparing b of the respective channels max And obtaining the maximum value b _ max of the single frame statistics.
To eliminate the interference of the background impulse information, the smaller value of b _ max will be filtered out by a predefined threshold. Therefore, positioning deviation due to the influence of background information does not occur.
As shown in FIG. 4, assume that the current frame is Nth e Frame, target at N e Distance moved within a frame divided by N e Obtaining the average speed v of the target in the interval 0 . The pulse interval N of the background light can be found by determining the position of the target and the background by performing a frame difference between two frames spaced 64 frames before starting the execution of the high speed target tracking algorithm b And pulses of target lightInterval N e . In the process of real-time target tracking, the speed of the current moment is represented by the position difference between two coordinate vectors spaced by 7 frames, and then the length of the image smear is calculated according to the following formula.
And after the image trailing length is obtained through calculation, the image trailing length is added to the output coordinates of the algorithm to perform coordinate compensation. Finally, outputting the compensated positioning coordinate, wherein N 0 Is the number of pixel cells passed by the movement in unit time.
The method and the system are applied to a 40K fps pulse array image sensor, receive 8 paths of LVDS data output by the image sensor, can process the data in real time along with a sampling clock and a frame effective signal, realize the algorithm based on an FPGA platform, can realize the real-time position of a 44ns positioning target, and have smaller positioning error. The real-time motion of the table tennis ball in high-speed motion is shot, the motion trail of the table tennis ball emitted by the table tennis ball serving machine is shot in a black test box, and a high-speed target tracking algorithm is realized based on Xilinx Kintex-7 FPGA. The real-time motion trail of the table tennis is displayed on the display. Since the frame rate of the display is 60fps and the frame rate of the pulse array image sensor is 40Kfps, the positioning coordinates of the 667 frames positioned most recently are displayed on one frame of the display. In order to more intuitively display the motion trail of the table tennis, the positioning coordinates output by the algorithm are set to maintain the display of 0.3 s. In order to distinguish the positioning coordinate output by the latest algorithm from the positioning coordinate cached before, the display color of the cached positioning coordinate is gradually lightened as the caching time is longer in the display process.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (8)
1. The high-speed target tracking method based on the pulse array image sensor is characterized by comprising the following steps of:
firstly, traversing rows and columns of single-frame pulse data of a pulse array image sensor by a frame according to a 1-bit pulse data format of the pulse array image sensor, inputting two paths of adjacent LVDS pulse data of the pulse array image sensor, processing the LVDS pulse data, starting accumulation, and counting to obtain a single-channel accumulation maximum value;
step two, according to the step one, two paths of adjacent LVDS pulse data of the input pulse array image sensor of 8 channels are processed in parallel to obtain the accumulated maximum value of each channel, the maximum value in the 8 channels is taken as the maximum value of single frame pulse data statistics, and the position of the maximum value of the statistics is taken as a positioning coordinate;
step three, calculating to obtain the image trailing length, adding the image trailing length to the positioning coordinate obtained in the step two for coordinate compensation, and outputting a compensated positioning coordinate;
the calculation to obtain the image trailing length specifically comprises the following steps:
suppose that the current frame is Nth e Frame, target at N e Distance moved within a frame divided by N e Obtaining the average speed v of the target in the interval 0 (ii) a Before starting to perform high-speed target tracking, the positions of the target and the background are determined by performing a frame difference between two frames spaced by 64 frames, and the pulse interval N of the background light can be obtained b And the pulse interval N of the target light e (ii) a In the process of real-time target tracking, the speed of the current moment is represented by the position difference between two coordinate vectors with an interval of 7 frames, and then the length of image tailing is calculated according to the following formula (4):
wherein, N 0 Is the number of pixel cells passed by the movement in unit time.
2. The method according to claim 1, wherein the step one of processing LVDS pulse data specifically comprises:
(1) Converting a differential signal of the input single-channel LVDS pulse data into a single-ended signal by utilizing a Xilinx FPGA device primitive IBUFDS;
(2) Performing serial-parallel conversion on the converted single-ended signal through data shift, outputting pulse data of X rows of pixels by using the single-channel LVDS, taking the frame effective signal as a starting node, and taking every X pulse data as one line;
(3) After LVDS data are converted, two paths of adjacent converted LVDS data are spliced together, and the spliced data are pulse data of 2X adjacent pixels on the same line, wherein X is a positive integer.
3. The method according to claim 2, wherein the obtaining of the maximum accumulated value of each channel in step two comprises:
after the splicing data is input, accumulating the adjacent N data: recording the cumulative sum of m- (N + m-1) (m =1,2, …, X) as a m (m＝1，2,…,X)；
Wherein the parameter c M For storing the accumulated result of the input concatenated data, c M Is 0, where M denotes the mth line of the box;
after the single-frame data output by the pulse array image sensor are positioned, resetting all parameters in the first step and the second step to ensure the accuracy of positioning each frame of data; c. C M The expression of (c) is as follows:
c M ＝(a 1 ,a 2 ,...,a X ) (1)
when c is going to M When collecting new accumulation results, parameter c j ' (j = 2.. Multidot.M) for saving c j Data before data change, and c j ' previously saved data transfer to c j-1 (ii) a So when c is j When a change occurs, the data will move by:
c j-1 ＝c′ j (2)
wherein, c j (a m ) For indicating storage in c j In the accumulated result a m The data of (a); path1 output c 1 Path 2 output a m (m＝1，2,…,X),c 1 (a m ) A representing delay of M line periods m A value of (d);
b i ＝b i +a i -c 1 (a i )(i＝1,2,...,X) (3)
wherein, the formula (3) is used for calculating the number of pulses in the frame, and after the calculation of the first M rows is completed, the number of pulses in the X frames of the first M rows is obtained, namely b i A value of (d); in the calculation of b i (i =1,2, …, X), and then b is compared i A value in between.
4. The method according to claim 3, wherein the maximum value of the 8 channels in step three is the maximum value of the single frame pulse data statistics, specifically:
(1) Presetting single-channel statistical maximum b max Is 0,b i Maximum value of (b) i_max And b max Comparing;
(2) If b is i_max Greater than b max Then b is i_max Is assigned to b max And recording the coordinate information at the moment;
(3) After M lines, each line will calculate a new b i And performing a previous comparison;
(4) Finally, when the whole frame data transmission is completed, each channel has only one b max ；
(5) By comparing respective channelsB of road max And obtaining the maximum value b _ max of the single frame statistics.
5. A high-speed target tracking system based on a pulse array image sensor is characterized by comprising:
the single-channel maximum value calculation module is used for traversing rows and columns of single-frame pulse data of the pulse array image sensor by a frame according to a 1-bit pulse data format of the pulse array image sensor, inputting two paths of adjacent LVDS pulse data of the pulse array image sensor, processing the LVDS pulse data, then starting accumulation, and counting to obtain a single-channel accumulation maximum value;
the 8-channel maximum value calculating module is used for parallelly processing two paths of adjacent LVDS pulse data of the input pulse array image sensor of 8 channels to obtain the accumulated maximum value of each channel, the maximum value in the 8 channels is taken as the maximum value of single-frame pulse data statistics, and the position where the maximum value is counted is taken as a positioning coordinate;
the compensation positioning module is used for calculating to obtain the image trailing length, adding the image trailing length to the positioning coordinates obtained in the second step to perform coordinate compensation, and outputting compensated positioning coordinates;
the image trailing length is obtained through calculation, and the method specifically comprises the following steps: suppose the current frame is Nth e Frame, target at N e Distance moved within a frame divided by N e Obtaining the average speed v of the target in the interval 0 (ii) a Before starting to perform high-speed target tracking, the positions of the target and the background are determined by performing a frame difference between two frames spaced by 64 frames, and the pulse interval N of the background light can be obtained b And the pulse interval N of the target light e (ii) a In the process of real-time target tracking, the speed of the current moment is represented by the position difference between two coordinate vectors with an interval of 7 frames, and then the length of image tailing is calculated according to the following formula (4):
wherein the content of the first and second substances,N 0 is the number of pixel cells passed by the movement in unit time.
6. The high-speed target tracking system based on the pulse array image sensor according to claim 5, wherein the processing of the LVDS pulse data in the single-channel maximum value calculation module specifically comprises:
(1) Converting a differential signal of the input single-channel LVDS pulse data into a single-ended signal by utilizing a Xilinx FPGA device primitive IBUFDS;
(2) Performing serial-parallel conversion on the converted single-ended signal through data shift, outputting pulse data of X rows of pixels by using single-channel LVDS, taking frame effective signal pull-up as a starting node, and taking every X pulse data as one line;
(3) After LVDS data are converted, two paths of adjacent converted LVDS data are spliced together, and the spliced data are pulse data of adjacent 2X pixels on the same row, wherein X is a positive integer.
7. The high-speed target tracking system based on the pulse array image sensor as claimed in claim 6, wherein the 8-channel maximum value calculating module obtains the accumulated maximum value of each channel, specifically:
after the splicing data is input, accumulating the adjacent N data: recording the cumulative sum of m- (N + m-1) (m =1,2, …, X) as a m (m＝1，2,…,X)；
Wherein the parameter c M For storing the accumulated result of the input concatenated data, c M Is 0, where M denotes the mth row of the box;
after the single-frame data output by the pulse array image sensor are positioned, resetting all parameters in the first step and the second step to ensure the accuracy of positioning each frame of data; c. C M The expression of (a) is as follows:
c M ＝(a 1 ,a 2 ,...,a X ) (1)
when c is going to M When collecting new accumulation results, parameter c j ' (j = 2.. Multidot.M) for saving c j Data before data change, and c j ' previously saved data transfer to c j-1 (ii) a So when c is j When a change occurs, the data will move by:
c j-1 ＝c′ j (2)
wherein, c j (a m ) For indicating storage in c j In the accumulated result a m The data of (a); path1 output c 1 Path 2 output a m (m＝1，2,…,X),c 1 (a m ) A representing a delay of M line periods m A value of (d);
b i ＝b i +a i -c 1 (a i )(i＝1,2,...,X) (3)
wherein, the formula (3) is used for calculating the number of pulses in the frame, and after the calculation of the first M rows is completed, the number of pulses in the X frames of the first M rows is obtained, namely b i A value of (d); in the calculation of b i (i =1,2, …, X), and then b is compared i A value in between.
8. The high-speed target tracking system based on the pulse array image sensor as claimed in claim 7, wherein the maximum value of the 8 channels in the compensation positioning module is the maximum value of single frame pulse data statistics, specifically:
(1) Presetting single-channel statistical maximum b max Is 0,b i Maximum value of (b) i_max And b max Comparing;
(2) If b is i_max Greater than b max Then b is i_max Is assigned to b max And recording the coordinate information at the moment;
(3) After M lines, each line will calculate a new b i And performing a previous comparison;
(4) Finally, when the whole frame data transmission is finished, each channel has only one b max ；
(5) By comparing b of the respective channels max Obtaining the maximum value b of single frame statistics_max。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011483754.9A CN112616031B (en) | 2020-12-16 | 2020-12-16 | High-speed target tracking method and system based on pulse array image sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011483754.9A CN112616031B (en) | 2020-12-16 | 2020-12-16 | High-speed target tracking method and system based on pulse array image sensor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112616031A CN112616031A (en) | 2021-04-06 |
| CN112616031B true CN112616031B (en) | 2022-11-04 |
Family
ID=75239633
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011483754.9A Active CN112616031B (en) | 2020-12-16 | 2020-12-16 | High-speed target tracking method and system based on pulse array image sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112616031B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113473048B (en) * | 2021-06-16 | 2022-08-30 | 天津大学 | Non-uniformity correction method for pulse array image sensor |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101694716A (en) * | 2009-10-10 | 2010-04-14 | 北京理工大学 | Stereoscopic vision optical tracking system aiming at multipoint targets |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7184832B2 (en) * | 2003-10-07 | 2007-02-27 | Medtronic, Inc. | Refractory period tracking and arrhythmia detection |
| CN103439756B (en) * | 2013-07-31 | 2016-02-10 | 中国人民解放军理工大学 | A kind of natural precipitation particle Microphysical Characteristics measuring method based on Particle Image Velocity |
| CN103544714A (en) * | 2013-10-22 | 2014-01-29 | 中国科学院半导体研究所 | Visual tracking system and method based on high-speed image sensor |
| CN116359934A (en) * | 2017-07-20 | 2023-06-30 | 深圳市大疆创新科技有限公司 | System and method for optical distance measurement |
| US10491832B2 (en) * | 2017-08-16 | 2019-11-26 | Qualcomm Incorporated | Image capture device with stabilized exposure or white balance |
| CN107896308B (en) * | 2017-10-27 | 2020-08-18 | 天津大学 | Pulse array type retina-imitating image sensor |
| CN109274928A (en) * | 2018-06-03 | 2019-01-25 | 合肥学院 | Dynamic object compression of images sampling apparatus and reconstruct device |
| CN111405207B (en) * | 2018-12-27 | 2022-07-26 | 天津大学青岛海洋技术研究院 | Reconstructed image smear elimination method applied to high-speed pulse sensor |
-
2020
- 2020-12-16 CN CN202011483754.9A patent/CN112616031B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101694716A (en) * | 2009-10-10 | 2010-04-14 | 北京理工大学 | Stereoscopic vision optical tracking system aiming at multipoint targets |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112616031A (en) | 2021-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111770290A (en) | Noise reduction method for dynamic vision sensor output event stream | |
| CN112232356B (en) | Event camera denoising method based on group degree and boundary characteristics | |
| CN112801900B (en) | Video blurring removal method for generating countermeasure network based on bidirectional circular convolution | |
| CN112616031B (en) | High-speed target tracking method and system based on pulse array image sensor | |
| CN111064865B (en) | Background activity noise filter of dynamic vision sensor and processor | |
| CN111294520B (en) | FPGA-based real-time lucky imaging method and system | |
| CN110414558A (en) | Characteristic point matching method based on event camera | |
| CN115714855A (en) | Three-dimensional visual perception method and system based on stereoscopic vision and TOF fusion | |
| CN113160106B (en) | Infrared target detection method and device, electronic equipment and storage medium | |
| CN114596233A (en) | Attention-guiding and multi-scale feature fusion-based low-illumination image enhancement method | |
| CN117726547A (en) | Defogging implementation method and system based on improved dark channel algorithm | |
| CN116389912B (en) | Method for reconstructing high-frame-rate high-dynamic-range video by fusing pulse camera with common camera | |
| KR20110053417A (en) | Digital video filter and image processing | |
| CN111107295B (en) | Video scaling method based on FPGA and nonlinear interpolation | |
| CN113163120A (en) | Transformer-based video anti-shake method | |
| WO2018047642A1 (en) | Imaging element, operation method for imaging element, imaging device, and electronic apparatus | |
| Zuo et al. | Accurate depth estimation from a hybrid event-RGB stereo setup | |
| CN116958192A (en) | Event camera image reconstruction method based on diffusion model | |
| CN114885112A (en) | High frame rate video generation method and device based on data fusion | |
| CN211127965U (en) | Real-time lucky imaging device based on FPGA | |
| CN102254308B (en) | Method and system for computing interpolation of realistic scene | |
| CN114202564A (en) | High-speed target tracking method and system based on event camera | |
| Zhang et al. | An ultra-high-speed hardware accelerator for image reconstruction and stereo rectification on event-based camera | |
| CN106027881B (en) | Imaging sensor column delay leads to image fault restoration methods | |
| Chen et al. | Event-Based Motion Magnification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP02 | Change in the address of a patent holder |
Address after: Building 39, Sino German Cooperation and Innovation Demonstration Park, Intersection of Qingtan Road and Zipeng Road, Economic and Technological Development Zone, Hefei City, Anhui Province, 230000 Patentee after: Hefei Institute of innovation and development, Tianjin University Address before: 14 / F and 16 / F, comprehensive business building, export processing zone, Hefei Economic Development Zone, Anhui Province (south of Yungu road and east of Taozhi Road, Hefei Economic Development Zone) Patentee before: Hefei Institute of innovation and development, Tianjin University |
|
| CP02 | Change in the address of a patent holder |