CN112446236A - Conversion event linear detection method based on pulse type image sensor - Google Patents
Conversion event linear detection method based on pulse type image sensor Download PDFInfo
- Publication number
- CN112446236A CN112446236A CN201910804850.XA CN201910804850A CN112446236A CN 112446236 A CN112446236 A CN 112446236A CN 201910804850 A CN201910804850 A CN 201910804850A CN 112446236 A CN112446236 A CN 112446236A
- Authority
- CN
- China
- Prior art keywords
- event
- straight line
- line
- activity
- image sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Computational Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Operations Research (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Indicating Or Recording The Presence, Absence, Or Direction Of Movement (AREA)
Abstract
A conversion event straight line detection method based on a pulse type image sensor comprises the steps of converting high-speed pulses into event signals with directions, adopting a least square method to parameterize straight lines, calculating liveness and judging whether events belong to straight lines or not. The detection algorithm is suitable for parallel operation, can be applied in real time, and can achieve image detection, identification and tracking accurate to pixel points.
Description
Technical Field
The invention relates to the field of image processing algorithms, in particular to a dynamic object detection method aiming at a pulse type image sensor, namely a conversion event linear detection method based on the pulse type image sensor.
Background
The impulse type high-speed image sensor has great advantages when shooting complex high-speed changing scenes, and can be used for flight detection systems of robots and unmanned aerial vehicles. Extracting straight lines from man-made environments is a fundamental task of machine vision, often used for robotically recognizing paths and various pattern recognition scenarios. The commonly used detection means are classical huffman transform based methods, gradient based methods, statistical based methods. After the address-event sensor appears, an event-based line detection method also appears, a buffer is usually needed to store the whole frame of data, and when the scene changes dramatically to increase the number of events, the invalid time occupies a large amount of resources of the buffer, resulting in low calculation efficiency.
Disclosure of Invention
Aiming at the problems in the prior art, the invention discloses a conversion event straight line detection method based on a pulse type image sensor. The detection algorithm is suitable for parallel operation, can be applied in real time, and can achieve image detection, identification and tracking accurate to pixel points.
A conversion event straight line detection method based on a pulse type image sensor specifically comprises the following steps:
1. converting the high-speed pulse into an event signal with a direction: for a position of [ xk,yk]A pixel of (2), taking a small time tsCounting the number of pulses, t adjacent to itsThe number of pulses within is subtracted, a trigger event is deemed to be present above a certain threshold, and the sign of the subtraction is retained, so that the high speed pulse signal is converted into a dynamic event signal, denoted ek=[uk T,tk,pk]TWherein u isk=[xk,yk]TRepresents the location of the occurrence of the event, tkRepresenting the time of occurrence of the event, pkRepresents the polarity of the event, is 1 or-1;
2. and (3) adopting a least square method to realize straight line parameterization: calculating the visual flow of each incoming event, locally approximating a space-time surface described By the incoming event from a normal three-dimensional plane By applying, and fitting the plane By adopting a least square fitting algorithm in a parameterized standard plane Ax + By + t + C =0 to obtain the visual flow of each incoming event
The velocity vector v can be obtained by using the Clamer's rulek:
Recording the directional event as ok= [uk T,vk T, tk,pk]TRepresenting an event with a rate;
defining a straight line in two-dimensional space by using a rho-theta parameter by using an event-based iterative least squares fitting straight line method with vertical offsetL (i)(ρk (i),θk (i)),ρk (i)Is the distance of the straight line from the origin,θ k (i)is nk (i)Angle to the horizontal as shown in fig. 1. Straight lines and time being linked by k, e.g.ρ k (i)Representing the straight line i at time tkThe straight line can be expressed as:
3. calculating the liveness and judging whether the event belongs to a straight line: each possible fitting straight line calculated in the last step has corresponding activity Ak (i)At a new event ekWhen this occurs, the activity is updated with the exponential decay equation:
where Δ tk=tk-tk-1This calculation method makes the activity of the line independentAt their respective rates, the description of each line in an adaptive manner is implemented if the line has a degree of activity Ak (i)If the set threshold is exceeded, thenL (i)Is visible, indicating that a line is present at this location.
One direction event okThe criterion determined to belong to a straight line is as follows 1. distance d from the point where the event occurred to the straight linek (i)Less than threshold dmax(ii) a 2. Straight normal sum nk (i)Angle therebetweenα k (i)Must be less than a threshold valueα maxTo ensure orthogonality of the velocity flow and the straight line; these two principles can be expressed by formula (5)
If this condition is not met, a new line needs to be initialized, the initialization method is as follows:
and judging whether the newly generated line meets the requirement, if so, stopping, otherwise, continuously generating a new line, and circularly judging.
A conversion event straight line detection method based on a pulse type image sensor converts pulse data into event data with directions, so that high-speed pulse data which cannot directly restore a scene can be applied to a high-efficiency and quick straight line fitting detection algorithm, and high-speed detection of line segments is realized; the method adopts a weight exponential decay method for the early events, so that the algorithm can still keep effective under the condition of violent scene change, and the adaptability of the algorithm to high-speed calculation is improved.
Drawings
FIG. 1 is a schematic diagram of the principle of an event-based line detection algorithm.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, a detailed description of the embodiments of the present invention will be given below with reference to examples.
As shown in FIG. 1, a straight line indexed by superscript (i)L (i) (ρk (i),θk (i)) It consists of two features:ρ k (i)represents the distance of the straight line from the origin,θ k (i)representative line nk (i)And the angle to the horizontal. When a new event arrives, two conditions are used to determine which line it belongs to before: (1) point to line distance d of event occurrencek (i)(2) angle between normal phase of straight line and event directionα k (i). The actual operation steps of the line detection are as follows:
1. adjusting data format, arranging data into a striped event form, i.e. ok= [uk T,vk T, tk,pk]T(k.gtoreq.0);
2. for all direction events okLine of sumL (i)Doing the formula according to the formula (4) to calculate the activeness
3. When there is a line satisfying the constraint condition (5):
(1) let i = arg maxj Ak (j)The liveness corresponds to a straight line satisfying the constraint condition (5)
(2) Make A ak (j) =Ak (j) +1
(3) Update sinθ k (i) ,cosθ k (i)
4. Judging if the activity degree Ak (j)If the activity is larger than the activity of the previous line, the line is output, otherwise, a line is updated by the formula (6), and a new loop is developed from the step 2.
Claims (1)
1. A conversion event straight line detection method based on a pulse type image sensor is characterized by comprising the following steps: the method specifically comprises the following steps:
1. converting the high-speed pulse into an event signal with a direction: for a position of [ xk,yk]A pixel of (2), taking a small time tsCounting the number of pulses, t adjacent to itsThe number of pulses within is subtracted, a trigger event is deemed to be present above a certain threshold, and the sign of the subtraction is retained, so that the high speed pulse signal is converted into a dynamic event signal, denoted ek=[uk T,tk,pk]TWherein u isk=[xk,yk]TRepresents the location of the occurrence of the event, tkRepresenting the time of occurrence of the event, pkRepresents the polarity of the event, is 1 or-1;
2. and (3) adopting a least square method to realize straight line parameterization: calculating the visual flow of each incoming event, locally approximating a space-time surface described By the incoming event from a normal three-dimensional plane By applying, and fitting the plane By adopting a least square fitting algorithm in a parameterized standard plane Ax + By + t + C =0 to obtain the visual flow of each incoming event
The velocity vector v can be obtained by using the Clamer's rulek:
Recording the directional event as ok= [uk T,vk T, tk,pk]TRepresenting an event with a rate;
defining a straight line in two-dimensional space by using a rho-theta parameter by using an event-based iterative least squares fitting straight line method with vertical offsetL (i)(ρk (i),θk (i)),ρk (i)Is the distance of the straight line from the origin,θ k (i)is nk (i)Angle to the horizontalThe line and time are linked by k, and the line can be expressed as:
3. calculating the liveness and judging whether the event belongs to a straight line: each possible fitting straight line calculated in the last step has corresponding activity Ak (i)At a new event ekWhen this occurs, the activity is updated with the exponential decay equation:
where Δ tk=tk-tk-1The calculation method makes the activity of the lines independent of their respective speed, and realizes that each line is described by an adaptive method if the activity A of the linek (i)If the set threshold is exceeded, thenL (i)Is visible, indicating that a line is present at this location;
one direction event okThe criterion determined to belong to a straight line is as follows 1. distance d from the point where the event occurred to the straight linek (i)Less than threshold dmax(ii) a 2. Straight normal sum nk (i)Angle therebetweenα k (i)Must be less than a threshold valueα maxTo ensure orthogonality of the velocity flow and the straight line; these two principles can be expressed by formula (5)
If this condition is not met, a new line needs to be initialized, the initialization method is as follows:
and judging whether the newly generated line meets the requirement, if so, stopping, otherwise, continuously generating a new line, and circularly judging.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910804850.XA CN112446236B (en) | 2019-08-29 | 2019-08-29 | Conversion event linear detection method based on pulse type image sensor |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910804850.XA CN112446236B (en) | 2019-08-29 | 2019-08-29 | Conversion event linear detection method based on pulse type image sensor |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112446236A true CN112446236A (en) | 2021-03-05 |
CN112446236B CN112446236B (en) | 2023-04-07 |
Family
ID=74741527
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910804850.XA Active CN112446236B (en) | 2019-08-29 | 2019-08-29 | Conversion event linear detection method based on pulse type image sensor |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112446236B (en) |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020008042A1 (en) * | 2000-06-20 | 2002-01-24 | Pierro Joseph J. | Synchronous detection and remote monitoring and regulating of cell potential for cathodic protection |
CN101173954A (en) * | 2006-11-03 | 2008-05-07 | 通用电气公司 | Systems and method for locating failure events in samples under load |
CN102970692A (en) * | 2012-12-03 | 2013-03-13 | 哈尔滨工程大学 | Method for detecting boundary nodes of wireless sensor network event |
WO2017136585A1 (en) * | 2016-02-02 | 2017-08-10 | Massachusetts Institute Of Technology | Distributed nanowire sensor for single photon imaging |
CN108475144A (en) * | 2016-01-20 | 2018-08-31 | 株式会社半导体能源研究所 | Input unit, input/output device and data processing equipment |
CN109760604A (en) * | 2017-11-10 | 2019-05-17 | Lg电子株式会社 | The method of the vehicle control apparatus and control vehicle that are installed on vehicle |
CN109946638A (en) * | 2019-04-01 | 2019-06-28 | 山西省计量科学研究院 | Based on residence time is adaptive and the electric energy meter radiation immunity automatic detection device and method of flashlight glint identification |
-
2019
- 2019-08-29 CN CN201910804850.XA patent/CN112446236B/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020008042A1 (en) * | 2000-06-20 | 2002-01-24 | Pierro Joseph J. | Synchronous detection and remote monitoring and regulating of cell potential for cathodic protection |
CN101173954A (en) * | 2006-11-03 | 2008-05-07 | 通用电气公司 | Systems and method for locating failure events in samples under load |
CN102970692A (en) * | 2012-12-03 | 2013-03-13 | 哈尔滨工程大学 | Method for detecting boundary nodes of wireless sensor network event |
CN108475144A (en) * | 2016-01-20 | 2018-08-31 | 株式会社半导体能源研究所 | Input unit, input/output device and data processing equipment |
WO2017136585A1 (en) * | 2016-02-02 | 2017-08-10 | Massachusetts Institute Of Technology | Distributed nanowire sensor for single photon imaging |
CN109760604A (en) * | 2017-11-10 | 2019-05-17 | Lg电子株式会社 | The method of the vehicle control apparatus and control vehicle that are installed on vehicle |
CN109946638A (en) * | 2019-04-01 | 2019-06-28 | 山西省计量科学研究院 | Based on residence time is adaptive and the electric energy meter radiation immunity automatic detection device and method of flashlight glint identification |
Non-Patent Citations (3)
Title |
---|
DENIS GUANGYIN CHEN等: ""A low-complexity image compression algorithm for Address-Event Representation (AER) PWM image sensors"", 《2011 IEEE INTERNATIONAL SYMPOSIUM OF CIRCUITS AND SYSTEMS (ISCAS)》 * |
SHOUSHUN CHEN等: ""Efficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors"", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 * |
赵众等: ""基于图像识别技术的烟包封条缺陷检测"", 《北京工业大学学报》 * |
Also Published As
Publication number | Publication date |
---|---|
CN112446236B (en) | 2023-04-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110261870B (en) | Synchronous positioning and mapping method for vision-inertia-laser fusion | |
CN111201451B (en) | Method and device for detecting object in scene based on laser data and radar data of scene | |
CN109784333B (en) | Three-dimensional target detection method and system based on point cloud weighted channel characteristics | |
CN110097553B (en) | Semantic mapping system based on instant positioning mapping and three-dimensional semantic segmentation | |
CN111612760B (en) | Method and device for detecting obstacles | |
EP3788597A1 (en) | Associating lidar data and image data | |
CN110688905B (en) | Three-dimensional object detection and tracking method based on key frame | |
CN113223045B (en) | Vision and IMU sensor fusion positioning system based on dynamic object semantic segmentation | |
Munguia et al. | Monocular SLAM for visual odometry | |
KR20210058686A (en) | Device and method of implementing simultaneous localization and mapping | |
CN110570474B (en) | Pose estimation method and system of depth camera | |
CN112184757A (en) | Method and device for determining motion trail, storage medium and electronic device | |
CN113763427A (en) | Multi-target tracking method based on coarse-fine shielding processing | |
CN116879870A (en) | Dynamic obstacle removing method suitable for low-wire-harness 3D laser radar | |
CN111798486A (en) | Multi-view human motion capture method based on human motion prediction | |
CN115144828A (en) | Automatic online calibration method for intelligent automobile multi-sensor space-time fusion | |
CN117406234A (en) | Target ranging and tracking method based on single-line laser radar and vision fusion | |
CN108694725A (en) | A kind of the Robotic Dynamic tracking and system of view-based access control model conspicuousness | |
CN117132620A (en) | Multi-target tracking method, system, storage medium and terminal for automatic driving scene | |
CN112446236B (en) | Conversion event linear detection method based on pulse type image sensor | |
Xiao et al. | Research on uav multi-obstacle detection algorithm based on stereo vision | |
CN116740575A (en) | Unmanned ship multi-sensor situation sensing and collision prevention decision method | |
CN114648639B (en) | Target vehicle detection method, system and device | |
CN106960440B (en) | Based on subject image characteristic point pixel space position combination control method | |
Wang et al. | A moving target detection and localization strategy based on optical flow and pin-hole imaging methods using monocular vision |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |