CN117253203A - Obstacle detecting system based on visual sensor - Google Patents
Obstacle detecting system based on visual sensor Download PDFInfo
- Publication number
- CN117253203A CN117253203A CN202311347708.XA CN202311347708A CN117253203A CN 117253203 A CN117253203 A CN 117253203A CN 202311347708 A CN202311347708 A CN 202311347708A CN 117253203 A CN117253203 A CN 117253203A
- Authority
- CN
- China
- Prior art keywords
- camera
- obstacle detection
- platform door
- vision
- detection system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 24
- 238000001514 detection method Methods 0.000 claims abstract description 70
- 238000001931 thermography Methods 0.000 claims description 34
- 238000004891 communication Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 13
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000010191 image analysis Methods 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 238000013500 data storage Methods 0.000 claims description 4
- 230000001360 synchronised effect Effects 0.000 claims description 2
- 239000003550 marker Substances 0.000 claims 2
- 238000000034 method Methods 0.000 description 19
- 238000009434 installation Methods 0.000 description 7
- 238000012544 monitoring process Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 238000005259 measurement Methods 0.000 description 5
- 230000035945 sensitivity Effects 0.000 description 3
- 230000001502 supplementing effect Effects 0.000 description 3
- 239000002390 adhesive tape Substances 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000009545 invasion Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 230000008092 positive effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000033001 locomotion Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/141—Control of illumination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Train Traffic Observation, Control, And Security (AREA)
Abstract
The invention relates to the technical field of detection systems, in particular to an obstacle detection system based on a visual sensor, which is arranged at a gap between a platform door and a track, wherein an obstacle detection module is used for acquiring front image information and intelligently identifying obstacles for early warning and prompting; a background calibration object is arranged at the gap between the platform door and the track, and can be matched with the obstacle detection module and the video recording module to capture visual images and videos.
Description
Technical Field
The invention relates to the technical field of detection systems, in particular to an obstacle detection system based on a visual sensor.
Background
The subway shielding door is a high-tech product integrating the subjects of construction, machinery, materials, electronics, information and the like and is used for subway platforms. The shielding door separates the platform from the train running area, and is controlled to be automatically opened by the control system, so that the safety of the train and passengers when the train and the passengers enter and leave the station is mainly ensured. A gap of 10 cm-30 cm is generally formed between a shielding door of a subway and a train, particularly a curve platform, the gap is larger, and a driver needs to detect whether a person clamping object is in the gap of more than 120 meters before driving in order to ensure safety. The slit between the subway shield door and the train door is called a boundary region, which is necessary for ensuring safe operation of the subway train. Once the person clamping object occurs in the limiting area between the shielding door and the train door, if a train running at the moment can cause a serious safety accident, a method for detecting the person clamping object occurs in the limiting area between the subway shielding door and the train door is very necessary to be found.
The existing subway platform door anti-pinch protection device mainly adopts a physical anti-pinch baffle plate 1, an infrared/laser electronic fence 3, laser radar detection 4, camera detection 5, a camera and thermal imaging identification detection.
1. The physical anti-pinch baffle is characterized in that an anti-pinch rubber structure is arranged on the outer side of each subway platform door unit vertical frame, the structure does not have an alarm function, and once passengers are clamped, a driver can hardly find the structure; in general, the rubber material of the anti-pinch baffle should ensure the rigidity to be too hard, and if an obstacle exists between the sliding door and the train during the closing of the sliding door, the anti-pinch baffle can give out an alarm, and prompt a driver to stop departure at the same time, so that the train can be normally started along with the disappearance of the alarm.
2. The infrared/laser electronic fence adopts a laser detector, and the infrared method and the laser method are respectively provided with a transmitting end and a receiving end at two ends of each subway platform door unit, can be installed in a floor mode or on the rail side of a rail door column, and if an obstacle blocks a light beam, the infrared/laser electronic fence is judged to be abnormal, and a train cannot be driven out of a platform through audible and visual alarm and the like. Because the infrared device has a divergence angle of 2-3 degrees, the infrared device is easily influenced by external stray light and self-reflected light, and the system has false alarm and missing alarm; the laser detector has smaller laser divergence angle, but the sensitivity is easy to be interfered by dust and the like of an infrared electronic fence, so that the false alarm rate is high. And this solution is not applicable to crescent shaped platforms, while being limited to detecting larger objects.
3. Laser radar detection: the current laser radar uses a TOF ranging mode to identify obstacles, has higher cost, is also provided with a camera to carry out auxiliary image transmission under normal conditions, cannot be widely applied due to high product cost, and needs special personnel for later maintenance.
4. And (3) detecting by a camera: the monocular camera is used for ranging, a large amount of reference data is needed, and the precision is not high; most of the existing methods utilize cameras with fixed focal lengths with the same parameters to be installed on the same plane, the same target has horizontal parallax on left and right images through calibrating lenses, and then the calibration results are utilized to carry out three-dimensional matching on binocular images, so that depth images are generated. The disadvantages are: in many applications, the fixed-focus camera needs to clearly observe a long-distance object and a short-distance object without changing the position of the binocular vision system, and the fixed-focus binocular vision system is difficult to adapt to the requirement, so that the actual application scene has more possibility of false alarm due to the defect, and is still in an attempt stage at present.
5. Camera and thermal imaging detection: the zoom binocular vision system has flexibility and convenience which are not possessed by a fixed focus system, and can be widely applied to motion tracking, three-dimensional detection, man-machine interaction and the like. The zoom binocular vision system can also self-learn and adapt to the application scene of the scene, and the algorithm safety of the whole system in accurately identifying the obstacle (person or object) is improved after the thermal imaging sensor and the active light background are assisted for calibration. There is no limitation of recognition rate, because in principle, recognition is not needed before calculation, but all obstacles are directly measured; the parallax is directly utilized to calculate the distance, and the accuracy is higher than that of a single vision; there is no need to maintain a sample database, as there is no notion of samples for binocular.
Disclosure of Invention
In order to solve the technical problems, the invention provides the technical field of detection systems, and particularly relates to an obstacle detection system based on a visual sensor.
The technical scheme adopted by the invention is as follows: an obstacle detection system based on a vision sensor, comprising:
the obstacle detection unit comprises visual detection equipment, wherein the visual detection equipment is arranged at a gap position between the platform door and the track, and can be used for acquiring image information of the gap position between the platform door and the track, identifying an obstacle and carrying out early warning and prompting according to the image information;
the platform door anti-pinch unit is connected with the obstacle detection unit and can control the opening and closing of the platform door;
the video recording unit is used for recording a real-time scene of the gap position between the platform door and the track by combining the visual detection equipment, so that the original data required by accident tracing can be provided;
the communication unit is used for connecting the TCMS system to communicate;
the system also comprises an automatic detection system, and the implementation steps of the automatic detection system comprise:
the visual detection equipment captures images of the clearance environment before a train enters a station, when the train arrives at the station and starts to leave the station, transmits the images after the steps and performs image analysis to obtain an image analysis result;
if the image analysis result meets the pre-condition of subway up and down, opening a platform door;
if the image analysis result does not meet the pre-condition of the subway, the obstacle detection system starts a locking signal to tightly close the platform door, and starts the video recording unit to shoot a real-time picture.
Further, the visual detection device comprises a first camera and a second camera, wherein the first camera and the second camera are arranged in parallel at a gap position between the platform door and the track and are used for shooting pictures and video pictures, and obstacles in the area can be respectively judged through a two-dimensional algorithm; the pictures and the video pictures acquired by the first camera and the second camera respectively can be combined and compared to judge the obstacle state information of the gap position between the platform door and the track.
Further, the first camera and the second camera perform data storage and data transmission of the pictures and the video pictures through a camera data communication processing unit.
Further, the second camera further comprises a thermal imaging sensor module, the thermal imaging sensor module can acquire a thermal imaging sheet, the thermal imaging sensor module is connected with a thermal imaging data communication processing unit, and the thermal imaging sheet performs data storage and data transmission through the thermal imaging data communication processing unit.
Further, the obstacle detection unit is provided with a graphic analysis unit, the first camera and the second camera can be used for obtaining pictures, coordinates in a view field can be set through the graphic analysis unit, feature mark points can be set in the coordinates in the pictures, and data graphic marking and graphic analysis can be performed according to the feature mark points.
Further, the first camera and the second camera adopt fixed-distance fixed-focus cameras, a binocular vision system is adopted to acquire images, and the binocular vision system performs self-adaptive focusing focal length adjustment through the first camera and the second camera binocular zooming and focusing system, so that focal length synchronization and focusing clarity are maintained.
Further, the obstacle detection unit further comprises a model building system, and the steps of the model building system comprise:
collecting zoom images for training;
generating a main point coordinate map of the camera;
modeling is completed on-line positioning.
Further, including background calibration thing and light filling lamp, the background calibration thing sets up clearance department between platform door and the track, the light filling lamp position with supplementary light filling device is unanimous all to be set up in the gap department between platform door and the track.
Further, the system also comprises an alarm device, wherein the alarm device comprises a voice alarm and a light alarm.
The invention has the advantages and positive effects that: by adopting the technical scheme, the invention has the advantages and positive effects that: by adopting the technical scheme, the platform door video anti-pinch system uses the camera to collect the stereoscopic image, and is assisted by thermal imaging, so that the anti-interference capability is high, and the accuracy is high; the nanoscale dustproof lens does not have the risk of light attenuation and suspended particle interference. The platform door system adopts a top-mounted installation mode, and has low interference sensitivity to mechanical displacement. When each platform door is provided with an image recognition module, the system can judge foreign matters in the gap of the platform and can count the number of passengers getting on or off the vehicle accurately. According to the accurate statistics data, real-time data of an AFC ticketing system is combined, and statistics of platform/carriage passenger flow distribution, passenger flow route data, passenger flow detention platform time, passenger flow transfer time and passenger flow travel speed can be obtained through a big data technology. The subway platform gap foreign matter detection system combines a high-definition video with a thermal imaging recognition system to automatically collect, recognize, screen, gather and analyze the platform gap foreign matter information, so as to realize detection information distribution acquisition, centralized management and comprehensive application and provide a timely and accurate gap foreign matter alarm function. The method mainly detects the damages such as gap entrance, gap object clamping, falling off of a pedal hollow adhesive tape and the like of the platform which endanger the operation safety of the subway, and prevents the occurrence of the accident of clamping the person and the object, so as to ensure the safe and positive and efficient operation of the train.
Drawings
FIG. 1 is a schematic flow chart of an obstacle detection system based on a visual sensor provided by the invention;
fig. 2 is a system block diagram of an obstacle detection system based on a vision sensor according to the present invention.
Detailed Description
The objects, technical solutions and advantages of the present invention will become more apparent by the following detailed description of the present invention with reference to the accompanying drawings. It should be understood that the description is only illustrative and is not intended to limit the scope of the invention. In the following description, descriptions of techniques of a well-known structure are omitted so as not to unnecessarily obscure the concept of the present invention.
Embodiments of the present invention are described below with reference to the accompanying drawings.
In the description of the embodiments of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "top", "bottom", etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of description and to simplify the description, and are not indicative or implying that the apparatus or element in question must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be construed as limiting the present invention. In the description of the present invention, it should be noted that, unless explicitly stated and limited otherwise, the terms "disposed," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art in a specific case.
An obstacle detection system based on a visual sensor, as shown in fig. 1-2, comprises an obstacle detection unit, a platform door anti-pinch unit, a video recording unit and a communication unit, wherein,
the obstacle detection unit comprises a visual detection device, wherein the visual detection device is arranged at a gap position between the platform door and the track, and can be used for acquiring image information of the gap position between the platform door and the track, identifying an obstacle and carrying out early warning and prompting according to the image information;
the platform door anti-pinch unit is connected with the obstacle detection unit and can control the opening and closing of the platform door;
the video recording unit is used for recording a real-time scene of a gap position between the platform door and the track by combining visual detection equipment, and can provide original data required by accident tracing;
and the communication unit is used for connecting the TCMS system to communicate.
The platform door safety gap anti-pinch detection alarm device mainly comprises an image recognition sensor module, a thermal imaging sensor module and a data analysis processing module, and specifically comprises image recognition in artificial intelligence, and the scene is subjected to pixel segmentation, object classification, model calibration and target tracking through a deep learning neural network, so that recognition and matching of obstacles are realized.
The binocular identification and thermal imaging fusion detection of the camera brings improvement of information acquisition capability, algorithm requirements are reduced, and driving safety is guaranteed. The camera monocular has the obstacle recognition algorithm, the binocular range judges the obstacle distance state, the thermal imaging recognition distinguishes whether the obstacle is a person or an object, and the camera monocular is integrated with the active light background calibration object, so that the safe and reliable level of the system operation and maintenance is effectively ensured.
Wherein, the image recognition sensor module: the device comprises a first camera, a second camera and a camera data communication processing unit;
a first camera: and shooting a gap video picture, wherein the two-dimensional algorithm independently judges the obstacle in the region.
A second camera: and shooting a gap video picture, wherein the two-dimensional algorithm independently judges the obstacle in the region.
First camera and second camera: and shooting a gap video picture, and combining and applying video information of the gap video picture and video information of the gap video picture to obtain state information of the three-dimensional algorithm recognition obstacle.
Camera data communication processing unit: the intelligent image recognition algorithm of the camera processes and stores the communication.
The system also comprises a thermal imaging sensor module: the thermal imaging sensor and the thermal imaging data communication processing unit are used for identifying people or objects according to the on-site assistance of the data samples, so that the occurrence of person clamping events is effectively avoided, and the equipment functional module has independent alarm.
The data analysis processing module: the camera module and the thermal imaging module perform analysis processing on data and perform data interaction with the upper computer.
Active light: and an adaptive illumination intensity light supplementing device is designed according to the site and used for supplementing light and fixing the scene to resist interference.
Background calibration: the identification mark object which is beneficial to image acquisition is installed according to the field environment, and the identification mark object can also be combined with a light supplementing device to increase the reliability of field image acquisition.
After the train enters the station, the equipment and the system of the embodiment are used for controlling the door opening of the train and the platform door when the train and the platform door are not opened:
the first camera and the second camera independently acquire a gap area scene; simultaneously, according to the images acquired by the first camera and the second camera, calculating three-dimensional coordinates of all feature points in the overlapped view field, and identifying the boundary range of the limiting area; when the subway train and the shielding door start to close, three-dimensional coordinates of all feature points in the overlapped view fields are calculated according to image parallax acquired by the first camera and the second camera.
When the system gives a signal that all doors are closed and locked after the train and platform doors start to close:
comparing all the characteristic points acquired again by the front-end equipment with all the characteristic points acquired before the train is opened to acquire new characteristic points; further calculating whether the new feature points fall in the boundary range of the limit area, if so, comparing the obstacle with any camera to give an alarm, then calculating the real-time difference value between the coordinates of the feature points and the coordinates of the boundary points of the limit area, comparing the real-time difference value with a preset alarm threshold, and if the real-time difference value is larger than the preset alarm threshold, outputting an alarm signal; and if the real-time difference value is smaller than the preset alarm threshold value, outputting a safe operation signal. Specifically, a binocular vision principle is adopted to calculate three-dimensional coordinates of all feature points in the overlapped view field ECF.
Preferably, the thermal imaging sensor module is arranged at the center line position of a subway train door at the top end of the platform door, and the installation direction is parallel to the door opening direction of the subway train; the preferable installation mode of the ground platform is that the ground platform is laterally installed at a non-emergency and fixed door, and the lateral installation direction is parallel to the door opening direction of the subway train; the overlapped view fields of the camera module and the thermal imaging completely cover the limit area, and the limit area is respectively subjected to image acquisition and temperature acquisition.
Preferably, the TCMS system is connected with a train driver, and if foreign matters are found at the gap between the platform door and the track, the system can be used for contacting the driver, and the driver makes braking or corresponding judgment according to the prompt, so that the driving safety is ensured.
Preferably, as shown in fig. 2, the embodiment is provided with an anti-pinch detection alarm method, and the specific application method is as follows:
when a system receives a platform door opening signal before a train stops and stably opens a door, gap space sampling is carried out by utilizing a binocular camera, boundary ranges of boundary areas between the platform door and the platform door are calculated by extracting a platform protection rubber strip, a door frame, a carriage door frame, an anti-collision baffle, a reflective background calibration plate and the like, and when the subway platform door and the train carriage door are closed, three-dimensional coordinates of all characteristic points in the boundary ranges are calculated by utilizing a binocular vision principle. Comparing all characteristic points obtained when the subway train and the platform door start to be closed with all characteristic points obtained before the train is opened to obtain new characteristic points; calculating a real-time difference value between the coordinates of the feature points and the coordinates of the boundary points of the boundary region, comparing the real-time difference value with a preset alarm threshold, outputting an alarm result if the real-time difference value is larger than the preset alarm threshold, and judging that the risk of clamping a person or an object exists in the boundary region of the subway control system by assisting with the result of the thermal imaging sensor, so that the safety of the system is improved; and if the real-time difference value is smaller than the preset alarm threshold value, outputting an obstacle-free result.
The system can also obtain platform gap video in the installation and debugging stage, and establish a feature library through video identification and analysis as a basis for comparison in normal operation. The system judges whether the train enters the station in real time, and when the train enters the station, the whole system starts to enter a foreign matter detection mode. When the parking time is up, the train door and the shielding door are closed, the system recognizes whether foreign matters exist between the gaps of the platform in a video mode, continuously prompts the detection result, and provides the detection result for train drivers and platform staff. When the train leaves, the system stops working and returns to the mode of detecting the train entering.
Each sliding door is provided with a detection unit (a video image recognition module, a thermal imaging module, active light and a background calibration object) with independent image acquisition and calculation capability, and each side platform is provided with an independent operation monitoring host and an independent display. The detection unit is responsible for detecting the monitoring area, when the obstacle invasion of the monitoring area is identified, the alarm information and the image are uploaded to the end door alarm box, the machine room monitoring host and the central control dispatching room display end monitoring interface to display the alarm state and the image in real time, and drivers, train passengers and station personnel can judge the invasion.
Alarm uploading and indication: after the front-end camera or the thermal imaging invades the monitoring area, the alarm information and the video picture are uploaded to an end door alarm box, a machine room monitoring host and a central control dispatching room display end through an industrial communication bus. The software interface displays alarm information and video real-time pictures, and all display terminals can output audible and visual alarm signals to remind a user to pause departure of obstacles to break in the pinch, so that accidents of unsafe driving are effectively avoided.
The platform door video anti-pinch system uses a camera to collect stereoscopic images, and is assisted by thermal imaging, so that the anti-interference capability is high, and the accuracy is high; the nanoscale dustproof lens does not have the risk of light attenuation and suspended particle interference. The underground platform door system adopts a top-mounted installation mode, and has low interference sensitivity to mechanical displacement. When each platform door is provided with an image recognition module, the system can judge foreign matters in the gap of the platform and can count the number of passengers getting on or off the vehicle accurately. According to the accurate statistics data, real-time data of an AFC ticketing system is combined, and statistics of platform/carriage passenger flow distribution, passenger flow route data, passenger flow detention platform time, passenger flow transfer time and passenger flow travel speed can be obtained through a big data technology.
The subway platform gap foreign matter detection system combines a high-definition video with a thermal imaging recognition system to automatically collect, recognize, screen, gather and analyze the platform gap foreign matter information, so as to realize detection information distribution acquisition, centralized management and comprehensive application and provide a timely and accurate gap foreign matter alarm function. The method mainly detects the damages such as gap entrance, gap object clamping, falling off of a pedal hollow adhesive tape and the like of the platform which endanger the operation safety of the subway, and prevents the occurrence of the accident of clamping the person and the object, so as to ensure the safe and positive and efficient operation of the train.
The visual detection equipment adopts the fixed-distance fixed-focus double-camera in the embodiment, and the problems of high requirement on hardware, complex installation design, field adaptability and poor practical effect of the fixed-distance fixed-focus double-camera system provide a double-camera focal length synchronization method based on image matching, so that the binocular visual system of the zoom has better universality and flexibility; the binocular zoom focusing system can perform automatic focusing and keep focal length synchronization; after the system is developed, the adaptive adjustment can be effectively carried out, the focus of the two cameras is kept clear, the focal length is synchronous, and the algorithm adaptive modeling after the field application scene calibration is achieved.
Preferably, the on-site system building model flow is as follows:
collecting zoom image training: the training phase includes three modules, LFE, AKS and WFM. Firstly, giving a predicted obstacle pose, and selecting an image of the predicted obstacle pose on an active light background calibration object; next, the LFE module extracts dense features from the online image and the map image, respectively, and extracts corresponding attention heat maps from the map image; the AKS module selects points with good characteristics from the map image as key points according to the attention score of the heat map; obtaining their corresponding three-dimensional coordinates by means of a binocular camera; finally, using the three-dimensional key points and the feature descriptors as input, searching in a three-dimensional cost volume by using the WFM module, searching for the optimal pose offset, comparing the optimal pose offset with the ground real pose, and constructing a loss function; two cameras are arranged on the same side of a target to be measured to form a binocular vision measurement system, and the left camera 2a and the right camera 2b are respectively arranged according to the position relation of shooting; when the left camera 2a is calibrated, the position of a calibration template is adjusted, so that the calibration template is ensured to be in the view field of the camera when different shooting focal lengths are achieved, and the camera is focused under the maximum focal length; calibrating the template to be static, gradually reducing the focal length of the camera, and shooting 8 template images under different focal lengths; the same method is used to acquire 8 images of the right camera 2 b.
Generating a camera principal point coordinate map: after training, the map generation can be completed by using part of the subnetworks of the network; given the real pose size of the obstacle and the scanning of the camera, the global three-dimensional coordinates of the camera points can be easily obtained; the true values of the positions and the postures of the camera and the obstacle are only used for mapping; firstly, under the condition of giving the real pose size of an obstacle, projecting the real pose size of the obstacle onto an image through a binocular camera, and associating map image pixels with global three-dimensional coordinates; then, solving an attention heat map of the map image and feature maps with different resolutions by using an LFE network; next, a set of keypoints is selected for different resolutions in the pyramid of the AKS module; in general, the method of the present embodiment saves the keypoints and their feature descriptors, as well as their 3D coordinates, into a map database; when the focal length is calculated to be changed, the field of view of the camera shows the proportional expansion and contraction phenomenon, so that the intersection point of the characteristic point connecting lines is the principal point in the images shot by different focal lengths. Calculating the coordinates of main points of a camera, and fitting 8 image points corresponding to the same feature points of calibration templates on different images into m straight lines by using a linear regression method by using 8 photographed pictures, wherein the fitting equation of the ith straight line is as follows:
li:aiu+biv+ci＝0,(i＝1,2...n) (2)
taking the minimum sum of squares of the connecting distances to the image points as an objective function S, wherein main point coordinates (u 0, v 0) are calculated, and main point coordinates (ul 0, vl 0) and (ur 0, vr 0) of the left camera and the right camera are respectively obtained; where ai, bi, ci are parameters obtained by fitting a straight line, and (u, v) are pixel coordinates of points required for fitting.
Modeling is completed on-line positioning: in the localization phase, the LFE network is used to estimate again feature maps of different resolutions in the online image. The method comprises the steps of eliminating camera distortion through a graph distortion correction algorithm, correcting the position of a camera mainly, enabling binocular cameras to be strictly parallel, enabling binocular images to be convenient to carry out three-dimensional matching, and finally obtaining the internal and external parameters of the camera and the position relation of the binocular cameras according to a three-dimensional matching algorithm of a self-adaptive area, wherein key points, characteristic descriptors and global 3D coordinates of the key points are collected from the nearest map image of the predicted obstacle pose size of the given camera; then, in the WFM module, candidate poses are given in the constructed cost volume, and the key points are projected onto the online image by using the candidate poses; and the pose size positioning from thick to thin is realized through three feature matching networks with different resolutions.
The invention provides a method and a system for identifying obstacles by real-time dynamic binocular range finding based on a range finding method of binocular zoom cameras with different focal lengths, wherein the method comprises the following steps: marking a target to be measured, and establishing a functional relation between the vertical distance between the target to be measured and the two cameras and the physical parameters of the two cameras; based on the established functional relation, different distances are selected for calibration, and the fixed deviation between the real measurement distance and the real distance during calibration is determined; after time synchronization, the two cameras respectively acquire images of the target to be measured, which relatively move, in real time according to fixed frequency, and pixel coordinates of the target to be measured at a certain moment and/or vertical distances between the target to be measured and the two cameras are measured based on the established functional relation; the invention considers the influence of objective physical errors such as the length of the base line, the focal length and the like, can be applied to the distance measurement of a moving target, and has high measurement precision.
The device is characterized in that an input image of the object recognition module based on the deep learning is from an image captured by the left eye of the calibrated binocular camera, and the image is the same as a left eye image used by the three-dimensional reconstruction part; the input of the three-dimensional reconstruction module is from two images captured by the calibrated binocular camera, and the three-dimensional reconstruction module generates a parallax image and a point cloud image by using the two input images.
The data processing module is used for providing image input for the object recognition module and the three-dimensional reconstruction module. The input image is an image captured by a calibrated binocular camera. Since the internal and external parameters of the used binocular camera need to be known in the three-dimensional reconstruction process, calibration operation needs to be performed on the used binocular camera.
The object of the present embodiment is to overcome the above-mentioned drawbacks in the prior art, and to provide a method for identifying an obstacle based on a binocular zoom camera with different focal lengths, which is reliable in operation, strong in practicability and high in measurement accuracy, and can identify the far and near objects accurately by dividing the areas according to the object image information collected by the lenses of the camera with different focal lengths. Meanwhile, a thermal imaging sensor and an active light background calibration object are used for improving the obstacle identification safety performance of the system.
The foregoing describes the embodiments of the present invention in detail, but the description is only a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. All equivalent changes and modifications within the scope of the present invention are intended to be covered by the present invention.
Claims (9)
1. An obstacle detection system based on a vision sensor, comprising:
the obstacle detection unit comprises visual detection equipment, wherein the visual detection equipment is arranged at a gap position between the platform door and the track, and can be used for acquiring image information of the gap position between the platform door and the track, identifying an obstacle and carrying out early warning and prompting according to the image information;
the platform door anti-pinch unit is connected with the obstacle detection unit and can control the opening and closing of the platform door;
the video recording unit is used for recording a real-time scene of the gap position between the platform door and the track by combining the visual detection equipment, so that the original data required by accident tracing can be provided;
the communication unit is used for connecting the TCMS system to communicate;
the system also comprises an automatic detection system, and the implementation steps of the automatic detection system comprise:
the visual detection equipment captures images of the clearance environment before a train enters a station, when the train arrives at the station and starts to leave the station, transmits the images after the steps and performs image analysis to obtain an image analysis result;
if the image analysis result meets the pre-condition of subway up and down, opening a platform door;
if the image analysis result does not meet the pre-condition of the subway, the obstacle detection system starts a locking signal to tightly close the platform door, and starts the video recording unit to shoot a real-time picture.
2. The vision-sensor-based obstacle detection system according to claim 1, wherein the vision detection device comprises a first camera and a second camera, the first camera and the second camera are arranged in parallel at a gap position between the platform door and the track, and are used for shooting pictures and video pictures, and the obstacles in the area can be respectively judged through a two-dimensional algorithm; the pictures and the video pictures acquired by the first camera and the second camera respectively can be combined and compared to judge the obstacle state information of the gap position between the platform door and the track.
3. The vision-sensor-based obstacle detection system of claim 2, wherein the first and second cameras perform data storage and data transmission of the pictures and video frames through a camera data communication processing unit.
4. A vision-sensor-based obstacle detection system as in any one of claims 1-3 wherein the second camera further comprises a thermal imaging sensor module, the thermal imaging sensor module operable to acquire a thermal imaging patch, the thermal imaging sensor module coupled to a thermal imaging data communication processing unit, the thermal imaging patch operable to perform data storage and data transmission via the thermal imaging data communication processing unit.
5. A vision-sensor-based obstacle detection system according to claim 3, wherein the obstacle detection unit is provided with a graphic analysis unit, and the pictures acquired by the first camera and the second camera can be provided with coordinates in a field of view by the graphic analysis unit, and feature mark points can be provided in the coordinates in the pictures, and the graphic marking and graphic analysis can be performed according to the feature mark points.
6. The vision-sensor-based obstacle detection system according to any one of claims 1-3 and 5, wherein the first camera and the second camera use fixed-distance fixed-focus cameras, and a binocular vision system is used to acquire images, and the binocular vision system performs adaptive focus focal length adjustment through the first camera and the second camera binocular zoom focusing system, so as to keep focal length synchronous and clear.
7. The vision-sensor-based obstacle detection system of claim 1, wherein the obstacle detection unit further comprises a model building unit, the application step of the model building unit comprising:
collecting zoom images for training;
generating a main point coordinate map of the camera;
modeling is completed on-line positioning.
8. The vision-sensor-based obstacle detection system of claim 1, comprising a background marker and a light-compensating lamp, wherein the background marker is disposed at a gap between the platform door and the track, and the light-compensating lamp is disposed at a gap between the platform door and the track in correspondence with the auxiliary light-compensating device.
9. The vision-sensor-based obstacle detection system of claim 1, further comprising an alert device comprising a voice alert and a light alert.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311347708.XA CN117253203A (en) | 2023-10-18 | 2023-10-18 | Obstacle detecting system based on visual sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311347708.XA CN117253203A (en) | 2023-10-18 | 2023-10-18 | Obstacle detecting system based on visual sensor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117253203A true CN117253203A (en) | 2023-12-19 |
Family
ID=89131272
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311347708.XA Pending CN117253203A (en) | 2023-10-18 | 2023-10-18 | Obstacle detecting system based on visual sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117253203A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117935115A (en) * | 2024-01-29 | 2024-04-26 | 深圳市宇泰科技有限公司 | Method and device for detecting foreign matters in shielding door |
-
2023
- 2023-10-18 CN CN202311347708.XA patent/CN117253203A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117935115A (en) * | 2024-01-29 | 2024-04-26 | 深圳市宇泰科技有限公司 | Method and device for detecting foreign matters in shielding door |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107161141B (en) | Unmanned automobile system and automobile | |
| CN103413313B (en) | The binocular vision navigation system of electrically-based robot and method | |
| CN109878552B (en) | Foreign matter monitoring devices between track traffic platform door and train based on machine vision | |
| Semertzidis et al. | Video sensor network for real-time traffic monitoring and surveillance | |
| CN106808482B (en) | A kind of crusing robot multisensor syste and method for inspecting | |
| CN101969548B (en) | Active video acquiring method and device based on binocular camera shooting | |
| KR101999993B1 (en) | Automatic traffic enforcement system using radar and camera | |
| WO2015024407A1 (en) | Power robot based binocular vision navigation system and method based on | |
| CN112235537B (en) | Transformer substation field operation safety early warning method | |
| KR20200064873A (en) | Method for detecting a speed employing difference of distance between an object and a monitoring camera | |
| CN104378582A (en) | Intelligent video analysis system and method based on PTZ video camera cruising | |
| JPH0431200A (en) | Aircraft docking guidance device | |
| CN111753797A (en) | Vehicle speed measuring method based on video analysis | |
| CN103696652A (en) | Electric folding door and protecting method and system thereof | |
| CN117253203A (en) | Obstacle detecting system based on visual sensor | |
| CN111160220B (en) | Deep learning-based parcel detection method and device and storage medium | |
| CN114905512B (en) | Panoramic tracking and obstacle avoidance method and system for intelligent inspection robot | |
| CN106846297A (en) | Pedestrian's flow quantity detecting system and method based on laser radar | |
| CN112508893B (en) | Method and system for detecting tiny foreign matters between double rails of railway based on machine vision | |
| CN117197779A (en) | Track traffic foreign matter detection method, device and system based on binocular vision | |
| CN112001971A (en) | Object identification method and system | |
| CN111126363A (en) | Object recognition and distance measurement method and device for automatic driving vehicle | |
| Malik et al. | Smart Cars and Smart Roads. | |
| CN114596657A (en) | Gate passing system based on depth data | |
| KR20220120211A (en) | Drone image analysis system based on deep learning for traffic measurement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |