CN111160224B - High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation - Google Patents

High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation Download PDF

Info

Publication number
CN111160224B
CN111160224B CN201911369862.0A CN201911369862A CN111160224B CN 111160224 B CN111160224 B CN 111160224B CN 201911369862 A CN201911369862 A CN 201911369862A CN 111160224 B CN111160224 B CN 111160224B
Authority
CN
China
Prior art keywords
horizon
image
segmentation
target detection
fpga
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911369862.0A
Other languages
Chinese (zh)
Other versions
CN111160224A (en
Inventor
陈积明
吕嘉宜
贺诗波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN201911369862.0A priority Critical patent/CN111160224B/en
Publication of CN111160224A publication Critical patent/CN111160224A/en
Application granted granted Critical
Publication of CN111160224B publication Critical patent/CN111160224B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a high-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation. The video acquisition module is used for acquiring a monitoring video of a high-speed railway contact network in real time on board; the horizon line segmentation module uses an improved maximum inter-class variance method and an intelligent iterative algorithm to realize rapid self-adaptive horizon line segmentation, and the part above the horizon line is used as a detection area to be input into the target detection module; the target detection module detects possible foreign matters in the pictures transmitted by the horizon segmentation module by using the target detection neural network. The modules run on an on-board FPGA terminal to realize accelerated calculation. The invention can quickly and accurately detect the foreign matters on the contact net, meets the actual requirements of railway departments, and can effectively ensure the operation safety of the high-speed railway.

Description

High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation
Technical Field
The invention relates to the field of high-speed railway contact net foreign body hanging detection, in particular to an image processing method for FPGA (field programmable gate array) accelerated operation, deep learning and the like.
Background
The contact net of the high-speed railway is used as a device for supplying power to the electric locomotive, and if foreign matters (such as plastic bags, kites and other light drifts) are attached to the contact net, the power supply to the train can be influenced, so that the normal operation of the high-speed railway is threatened. At present, China mainly collects and stores contact network video images along a railway through a vehicle-mounted camera such as a motor train unit driver control information analysis system, but the contact network video images do not have the function of detecting and identifying foreign matters, so that detection personnel are required to manually observe video image data to check whether the foreign matters are attached to the contact network, the detection mode is time-consuming and labor-consuming, and the real-time performance is low.
Nowadays, image processing technology based on neural network is developed at a high speed, and has been well applied in various fields such as road video monitoring and analysis, automatic driving vehicles and the like. If can utilize advanced image processing technique to carry out the automatic analysis of video content to the contact net surveillance video to whether detect the foreign matter invasion contact net, just so can discover the foreign matter that exists on the contact net fast, ensure high-speed railway's safe operation.
Compared with target detection methods in other scenes, the method for detecting the foreign matters in the high-speed rail contact network is characterized in that the ground images are complex and have excessive interference information and are not detection target areas. Therefore, the method is characterized in that a horizon segmentation step is added before the target detection algorithm is applied, the video is automatically segmented into a sky part and a ground part according to the horizon after a monitoring video is trained by an improved maximum inter-class variance method (OTSU algorithm), and the sky part is extracted to carry out foreign matter detection by using a target detection neural network based on deep learning. Meanwhile, in consideration of the requirement of detection on real-time performance, the method makes full use of the characteristics of FPGA high-speed data processing capacity, hardware programming design and the like, and the algorithm is deployed on the FPGA, so that the foreign matter net hanging condition of the high-speed railway contact net is effectively monitored in real time.
Disclosure of Invention
In order to overcome the defect that time and labor are consumed for manually observing and monitoring videos to inspect foreign matters, the invention provides a high-speed rail contact net foreign matter detection system and method based on FPGA and horizon segmentation. The system uses an improved maximum inter-class variance method and a neural network algorithm to automatically analyze video image data, automatically divides a video into a sky part and a ground part according to a horizon line, quickly identifies foreign matters entering and exiting a contact net by using a target detection neural network in the sky part, and deploys the foreign matters on a vehicle-mounted FPGA terminal to increase real-time performance, improve the detection efficiency of the foreign matters invading the contact net and ensure the safe operation of a high-speed railway.
The technical scheme adopted by the invention for solving the technical problems is as follows: a high-speed rail contact net foreign matter detection system based on FPGA and horizon segmentation, this system includes following part:
(1) the video acquisition module: using FPGA as core, adopting SAA7113H video decoding chip, passing through I2The C bus protocol carries out initialization configuration on a CCD camera, the CCD camera is externally connected with a camera simulating a PAL/NTAL system, the camera is deployed in a train cab of the motor train unit to shoot towards the front, a running monitoring video containing a contact net is collected in real time, and format compression coding is carried out on a video signal;
(2) an image storage module: one path of the collected video data is written into DDR2 SDRAM through an FIFO buffer for storage, and the video data is read out through the FIFO buffer; the other path is encoded by an SAA7121 video encoding chip and then is output to a monitor to display a real-time image;
(3) horizon segmentation module: video data are obtained from DDR2 SDRAM, self-adaptive horizon segmentation is realized through an improved maximum inter-class variance method (OTSU algorithm) and an intelligent iterative algorithm, and a sky partial image above the horizon is taken and transmitted to a target detection module for further processing;
the improved maximum inter-class variance method specifically comprises the following steps: calculating a pixel point when the inter-class variance of a blue component (B value) is maximum in the range of each column of pixels (0.4N,0.8N) of an RGB image with the input size of M multiplied by N (M, N is the number of pixels in the horizontal and column directions of the RGB image respectively) as a horizon position point; obtaining a complete horizon position by traversing the whole image, taking a horizontal line so that 85% of horizon points are below the horizontal line, and taking the horizontal line as a final horizon estimation position; cutting partial images above a reserved horizon line as sky partial images, and inputting the sky partial images into a target detection neural network for training;
in the intelligent iterative algorithm, the updating frequency of the horizon position is controlled, and the image frame number interval of the horizon position updating is determined through an exponential function related to the change rate of the horizon position, so that the computing resources are saved, and the method specifically comprises the following steps:
initially, the horizon position is calculated every 1 second, i.e. v for recording frequency (in fps)The k frame picture calculates the horizon position h1Then, the k + v frame picture calculates the horizon position h2(ii) a Then, the rate of change of the position of the horizon is calculated twice
Figure BDA0002339385400000021
Figure BDA0002339385400000022
According to the rate of change
Figure BDA0002339385400000023
Image frame number interval u for automatic change horizon position update:
Figure BDA0002339385400000024
i.e. the local horizon change rate is
Figure BDA0002339385400000025
Then, the k + v frame calculates the horizon position h2Then, continuously calculating the position of the horizon in the k + v + u frame; iteratively updating the horizon position by the algorithm;
(4) a target detection module: and inputting the sky partial image transmitted into the module by the horizon segmentation module into a pre-trained target detection neural network for target detection, detecting possible foreign matters such as plastic bags and the like in the image, marking and early warning according to a detection result.
Further, in the adaptive horizon segmentation, the original image is compressed to 1/8 in length and width and then input to an algorithm for calculation, and the calculated corresponding estimated horizon position is mapped to the original image for segmentation, thereby speeding up calculation.
Further, the target detection neural network takes VGG-16 pre-training neural network weight as initial weight, and trains the high-speed rail contact network foreign matter data set marked in the VOC2012 format, so as to train out network weight suitable for the scene; the process specifically comprises the following steps: presetting 4 prior frames with different length-width ratios, wherein the specific size of the prior frame is determined by the size of the characteristic diagram; during training, matching each real target (ground route) in the picture with a prior frame with the largest IOU formed by the real targets; during classification, calculating the background as one of classes, namely for the detection targets of c classes, predicting c +1 confidence degrees by an algorithm, wherein the class with the highest confidence degree is a prediction class; during prediction, for each prediction frame, determining a target category according to the maximum category confidence, and filtering out the prediction frames of the background and the prediction frames with too low confidence (less than 0.5); and decoding the remaining prediction frames, obtaining the real position parameters of the prediction frames according to the prior frames, and reserving a plurality of prediction frames with the maximum confidence coefficient.
The invention also provides a high-speed rail contact net foreign matter detection method based on FPGA and horizon segmentation, which comprises the following steps:
(1) the FPGA equipment and the CCD image sensor are arranged in the locomotive of the motor train unit, and the visual angle is shot towards the front so as to be shot into a contact net structure, and the real-time acquisition is carried out on the train;
(2) the collected video data is continuously written into DDR2 SDRAM through FIFO buffer for storage, and then read out through FIFO buffer, and displayed in monitor through decoder;
(3) estimating the horizon position by traversing each row of pixels in the image acquired in the step (2) by adopting an improved maximum inter-class variance method, thereby performing self-adaptive horizon segmentation on the input image and segmenting a sky partial image; meanwhile, the updating frequency of the horizon position is controlled through an intelligent iterative algorithm, so that computing resources are saved;
(4) and inputting the segmented sky partial image into a pre-trained target detection neural network for target detection, detecting foreign matters such as plastic bags and the like possibly existing in the image, marking and early warning according to a detection result.
Further, in the step (4), the specific steps of detecting and identifying the foreign object by the target detection neural network are as follows:
(1) constructing a positive and negative sample data set according to the existing catenary monitoring video;
(2) marking a bounding box of a positive sample (foreign body part) in the data set according to a VOC2012 target detection task format;
(3) inputting the data set into a target detection neural network for training, adjusting the data set according to a training result, and training until a network weight with excellent detection performance indexes is obtained;
(4) and according to the actual operation detection result, continuously updating and training the data set, and optimizing the network weight, thereby improving the detection performance.
The invention has the beneficial effects that:
(1) the invention divides the horizon in advance through a fast self-adaptive algorithm, and only occupies small computing resources to extract the key monitoring area of the sky part, thereby reducing the detection range, eliminating the ground information interference and simultaneously increasing the foreign matter detection accuracy.
(2) The invention adopts the image processing neural network based on deep learning to accurately and efficiently detect the foreign matters in the contact network.
(3) The invention fully utilizes the high-speed and large data processing capacity of the FPGA to transfer the computational power of image acquisition and processing from the remote end of the server to the on-site edge end, thereby enhancing the effectiveness of railway monitoring.
(4) By applying the image processing technology based on the neural network, the invention can enable a computer to replace part of manual work, improve the detection efficiency of the contact net invading foreign matter, ensure the safe operation of the high-speed railway, simultaneously liberate manpower and improve the office automation level.
Drawings
FIG. 1 is a schematic diagram of the system of the present invention;
FIG. 2 is a schematic flow chart of the method of the present invention;
FIG. 3 is a flow chart of an adaptive horizon segmentation algorithm;
FIG. 4 is a schematic diagram of the effect of dividing the horizon;
fig. 5 is a schematic diagram of the target detection effect.
Detailed Description
The invention is described in further detail below with reference to the figures and specific examples.
As shown in fig. 1, the high-speed rail catenary foreign matter detection system based on FPGA and horizon segmentation provided by the invention comprises the following parts:
(1) the video acquisition module: the method comprises the steps of taking an FPGA as a core, adopting an SAA7113H video decoding chip, carrying out initialization configuration on a CCD camera through an I2C bus protocol, externally connecting a camera simulating a PAL/NTAL system, arranging the camera in a train cab of the motor train unit to shoot towards the front, collecting a driving monitoring video containing a contact net in real time, and carrying out format compression coding on a video signal.
Further, the video capture module comprises the following parts: the adopted FPGA chip is EP4CE617C8 of Cyclone IV series developed by Alter company; the adopted video decoding chip is SAA7113H, the video decoding chip completes configuration and initialization processes through an I2C bus under the control of FPGA, and 8-bit video data compatible with CCIR656 is output.
(2) An image storage module: one path of the collected video data is written into DDR2 SDRAM through an FIFO buffer for storage, and the video data is read out through the FIFO buffer; and the other path is encoded in an SAA7121 video encoding chip and then output to a monitor to display a real-time image.
Further, the image storage module includes the following parts: DDR 2800 series SDRAM of 8GB memory is used as memory; a FIFO is used as a data buffer.
(3) Horizon segmentation module: video data are obtained from DDR2 SDRAM, self-adaptive horizon segmentation is realized through an improved maximum inter-class variance method and an intelligent iterative algorithm, and a sky partial image above the horizon is taken and transmitted to a target detection module for further processing.
(4) A target detection module: and inputting the sky partial image transmitted into the module by the horizon segmentation module into a pre-trained target detection neural network for target detection calculation, detecting possible foreign matters such as plastic bags and the like in the image, and marking and early warning according to a detection result.
FIG. 2 is a schematic flow chart of the method of the present invention. The method comprises the steps of estimating the position of a horizon through a self-adaptive horizon segmentation algorithm based on an improved maximum inter-class variance method and an intelligent iterative algorithm for a frame of image obtained from a CCD monitoring video, and cutting the position to obtain a partial image above the horizon (namely a sky partial image). After the horizontal line positions are initially calculated twice, whether the horizontal line positions of the current frame image need to be calculated is determined through an intelligent iterative algorithm, and the image which does not need to be calculated is directly cut after being segmented at the calculated positions of the previous time. And then, performing Beijing target detection calculation in the sky partial image through a target detection neural network algorithm of a trained model: if the foreign body can be detected, marking a foreign body boundary frame, carrying out corresponding early warning prompt, and then continuously detecting the next image; otherwise, the next image is directly detected.
FIG. 3 is a flow chart of an adaptive horizon segmentation algorithm. The self-adaptive horizon calculation based on the improved maximum inter-class variance method is specifically calculated as follows:
for an input image I (x, y) expressed by an RGB format with the size of M multiplied by N (pixels), the vertex at the upper left corner of the image is I (x,0), the ith row of pixels is a matrix of 1 multiplied by N, wherein the sky ground segmentation threshold is marked as T, and the proportion of pixel points belonging to the sky in the whole matrix is marked as omega0The B value (blue component value) in RGB is averaged to μ0(ii) a The proportion of the pixels belonging to the ground occupying the whole matrix is recorded as omega1The average B value is mu1(ii) a The total average B value for an entire column of pixels is denoted as μ and the inter-class variance is denoted as g. In a 1 XN matrix formed by the ith row of pixels, the number of pixels with B value smaller than the threshold value T is counted as N0The number of pixels with B value greater than threshold T is recorded as N1Then, there are:
Figure BDA0002339385400000051
Figure BDA0002339385400000052
μ＝ω0μ0+ω1μ1 (3)
g＝ω0(μ0-μ)2+ω1(μ1-μ)2 (4)
substituting formula (3) for formula (4) to obtain the equivalent formula:
g＝ω0ω1(μ0-μ1)2 (5)
g obtained by the formula (5) is the inter-class variance.
According to experience, the horizon is generally positioned at the middle lower part of a picture, so that the g is maximum only by traversing the (0.4N,0.8N) interval of a 1 xN matrix formed by the ith row of pixels, namely the maximum between-class variance; the pixel position I (I, j) at this time is recorded, which is the pixel position of the horizon in the row of pixels. Traversing the whole image, recording the pixel position of the horizon in each row of pixels, recording j into a matrix, and finally forming an M multiplied by 1 matrix which is recorded as TH.
Because the input image of the neural network of the next module is rectangular, according to experience, a value H is taken as a final horizon position in the TH matrix, so that the H is smaller than 85% of the value in the TH matrix (namely, is more than 85% of horizon points), the horizon position can be well expressed, and noise interference is reduced. And (3) cutting and reserving pixels with y < H in the input image I (x, y) of M multiplied by N, namely acquiring an image above the horizon, namely a sky partial image.
Meanwhile, in order to accelerate the calculation, in the actual operation, the length and width dimensions of the original image of M multiplied by N are compressed to 1/8, then the input module performs calculation, and the calculated corresponding estimated position of the horizon line is mapped to the original image for segmentation, so that the calculation is accelerated; from experimental comparisons, such compression calculations result in negligible error.
The method for controlling the updating frequency of the horizon estimation position through the intelligent iterative algorithm specifically comprises the following steps:
initially, the horizon position is calculated every 1 second, namely for a video with a recording frequency v (the unit is fps), the horizon position h is calculated for the k frame picture1Then, the k + v frame picture calculates the horizon position h2(ii) a Then, the rate of change of the position of the horizon is calculated twice
Figure BDA0002339385400000061
Figure BDA0002339385400000062
According to the rate of change
Figure BDA0002339385400000063
Image frame number interval u for automatic change horizon position update:
Figure BDA0002339385400000064
i.e. the local horizon change rate is
Figure BDA0002339385400000065
Then, the k + v frame calculates the horizon position h2Then, continuously calculating the position of the horizon in the k + v + u frame; iteratively updating the horizon position by the algorithm; and the other frame images are directly segmented and cut by using the previously calculated horizon position.
FIG. 4 is a diagram illustrating the effect of dividing the horizon. Fig. 5 is a schematic diagram of the target detection effect. The detected target is marked out by a bounding box, and characters represent the identification type and the confidence of the target. The invention can quickly and accurately detect the foreign matters on the contact net, meets the actual requirements of railway departments, and can effectively ensure the operation safety of the high-speed railway.
The foregoing is only a preferred embodiment of the present invention, and although the present invention has been disclosed in the preferred embodiments, it is not intended to limit the present invention. Those skilled in the art can make numerous possible variations and modifications to the present teachings, or modify equivalent embodiments to equivalent variations, without departing from the scope of the present teachings, using the methods and techniques disclosed above. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are still within the scope of the protection of the technical solution of the present invention, unless the contents of the technical solution of the present invention are departed.

Claims (5)

1. The utility model provides a high-speed railway contact net foreign matter detecting system based on FPGA and horizon segmentation, its characterized in that, this system includes following part:
(1) the video acquisition module: using FPGA as core, adopting SAA7113H video decoding chip, passing through I2The C bus protocol carries out initialization configuration on a CCD camera, the CCD camera is externally connected with a camera simulating a PAL/NTAL system, the camera is deployed in a train cab of the motor train unit to shoot towards the front, a running monitoring video containing a contact net is collected in real time, and format compression coding is carried out on a video signal;
(2) an image storage module: one path of the collected video data is written into DDR2 SDRAM through an FIFO buffer for storage, and the video data is read out through the FIFO buffer; the other path is encoded by an SAA7121 video encoding chip and then is output to a monitor to display a real-time image;
(3) horizon segmentation module: video data are obtained from DDR2 SDRAM, self-adaptive horizon segmentation is realized through an improved maximum inter-class variance method and an intelligent iterative algorithm, and a sky partial image above the horizon is taken and transmitted to a target detection module;
the improved maximum inter-class variance method specifically comprises the following steps: calculating a pixel point when the inter-class variance of a blue component (B value) is maximum in the range of each row of pixels (0.4N,0.8N) of an RGB image with the input size of M multiplied by N as a horizon position point; obtaining a complete horizon position by traversing the whole image, taking a horizontal line so that 85% of horizon position points are below the horizontal line, and taking the horizontal line as a final horizon estimation position; cutting partial images above a reserved horizon line as sky partial images, and inputting the sky partial images into a target detection neural network for training;
the intelligent iterative algorithm specifically comprises the following steps: initially setting to calculate the horizon position every 1 second, namely for a video with the recording frequency v, calculating the horizon position h by the kth frame of picture1Then, the k + v frame picture calculates the horizon position h2(ii) a Then, the rate of change of the position of the horizon is calculated twice
Figure FDA0003508625100000011
Figure FDA0003508625100000012
According to the rate of change
Figure FDA0003508625100000013
Image frame number interval u for automatic change horizon position update:
Figure FDA0003508625100000014
i.e. the local horizon change rate is
Figure FDA0003508625100000015
Then, the k + v frame calculates the horizon position h2Then, continuously calculating the position of the horizon in the k + v + u frame; iteratively updating the horizon position by the algorithm;
(4) a target detection module: and inputting the sky partial image output by the horizon segmentation module into a pre-trained target detection neural network for target detection, detecting possible foreign matters in the image, marking and early warning according to a detection result.
2. The system for detecting the foreign matter in the overhead contact line of the high-speed rail based on the FPGA and the horizon segmentation as claimed in claim 1, wherein in the horizon segmentation module, the original image is compressed to 1/8 in length and width, and then the compressed original image is input into an improved maximum inter-class variance method for calculation, and the calculated corresponding estimated position of the horizon is mapped to the original image for segmentation, thereby accelerating the calculation.
3. The system for detecting the foreign matter in the high-speed rail contact net based on the FPGA and the horizon segmentation as claimed in claim 1, wherein the target detection neural network trains the foreign matter data set of the high-speed rail contact net marked in the VOC2012 format by using the VGG-16 pre-training neural network weight as an initial weight, so as to train a network weight suitable for the scene; the method specifically comprises the following steps: presetting 4 prior frames with different length-width ratios, wherein the specific size of the prior frame is determined by the size of the characteristic diagram; during training, each real target in the picture is matched with the prior frame with the largest IOU formed by the real target; during classification, calculating the background as one of classes, namely predicting c +1 confidence coefficients of detection targets of c classes, wherein the class with the highest confidence coefficient is a prediction class; during prediction, determining a target category according to the maximum category confidence for each prediction frame, and filtering out the prediction frames with backgrounds and the prediction frames with over-low confidence; and decoding the remaining prediction frames, obtaining the real position parameters of the prediction frames according to the prior frames, and reserving a plurality of prediction frames with the maximum confidence coefficient.
4. The detection method of the high-speed rail contact net foreign matter system based on the FPGA and the horizon segmentation of any one of claims 1 to 3 is characterized by comprising the following steps:
(1) the FPGA equipment and the CCD image sensor are arranged in the locomotive of the motor train unit, and the visual angle is shot towards the front so as to be shot into a contact net structure, and the real-time acquisition is carried out on the train;
(2) the collected video data is continuously written into DDR2 SDRAM through FIFO buffer for storage, and then read out through FIFO buffer, and displayed in monitor through decoder;
(3) estimating the horizon position by traversing each row of pixels in the image acquired in the step (2) by adopting an improved maximum inter-class variance method, thereby performing self-adaptive horizon segmentation on the input image and segmenting a sky partial image; meanwhile, the updating frequency of the horizon position is controlled through an intelligent iterative algorithm, so that computing resources are saved;
(4) and inputting the segmented sky partial image into a pre-trained target detection neural network for target detection, detecting possible foreign matters in the image, marking according to a detection result and early warning.
5. The detection method of the high-speed rail contact line foreign matter system based on the FPGA and the horizon segmentation as recited in claim 4, wherein in the step (4), the specific steps of detecting and identifying the foreign matter by the target detection neural network are as follows:
(1) constructing a positive and negative sample data set according to the existing catenary monitoring video;
(2) marking a bounding box of a positive sample (foreign body part) in the data set according to a VOC2012 target detection task format;
(3) inputting the data set into a target detection neural network for training, adjusting the data set according to a training result, and training until a network weight with excellent detection performance indexes is obtained;
(4) and according to the actual operation detection result, continuously updating and training the data set, and optimizing the network weight, thereby improving the detection performance.
CN201911369862.0A 2019-12-26 2019-12-26 High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation Active CN111160224B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911369862.0A CN111160224B (en) 2019-12-26 2019-12-26 High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911369862.0A CN111160224B (en) 2019-12-26 2019-12-26 High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation

Publications (2)

Publication Number Publication Date
CN111160224A CN111160224A (en) 2020-05-15
CN111160224B true CN111160224B (en) 2022-04-05

Family

ID=70558316

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911369862.0A Active CN111160224B (en) 2019-12-26 2019-12-26 High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation

Country Status (1)

Country Link
CN (1) CN111160224B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112995505B (en) * 2021-02-09 2022-06-17 西南科技大学 Image processing method, device and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109697424A (en) * 2018-12-19 2019-04-30 浙江大学 A kind of high-speed railway impurity intrusion detection device and method based on FPGA and deep learning
CN109753893A (en) * 2018-12-20 2019-05-14 广州航天海特***工程有限公司 Video detecting method, system, computer equipment and storage medium along track

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018105112A1 (en) * 2016-12-09 2018-06-14 株式会社日立国際電気 Water intrusion detection system and method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109697424A (en) * 2018-12-19 2019-04-30 浙江大学 A kind of high-speed railway impurity intrusion detection device and method based on FPGA and deep learning
CN109753893A (en) * 2018-12-20 2019-05-14 广州航天海特***工程有限公司 Video detecting method, system, computer equipment and storage medium along track

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
一种基于类间方差的地平线检测算法;程序等;《航空学报》;20101025(第10期);全文 *
区域协方差与中值校正融合的天际线检测算法研究;涂兵等;《计算机科学》;20170315(第03期);全文 *
基于残差回归网络的复杂背景下海界线检测;邱艺铭等;《舰船电子工程》;20180820(第08期);全文 *

Also Published As

Publication number Publication date
CN111160224A (en) 2020-05-15

Similar Documents

Publication Publication Date Title
CN108446630B (en) Intelligent monitoring method for airport runway, application server and computer storage medium
CN109460709B (en) RTG visual barrier detection method based on RGB and D information fusion
CN109753929B (en) High-speed rail insulator inspection image recognition method
CN111784633B (en) Insulator defect automatic detection algorithm for electric power inspection video
CN113947731B (en) Foreign matter identification method and system based on contact net safety inspection
Kulkarni et al. Real time vehicle detection, tracking and counting using Raspberry-Pi
CN104092988A (en) Method, device and system for managing passenger flow in public place
CN112800860A (en) Event camera and visual camera cooperative high-speed scattered object detection method and system
CN109460787A (en) IDS Framework method for building up, device and data processing equipment
CN110255318B (en) Method for detecting idle articles in elevator car based on image semantic segmentation
CN111667655A (en) Infrared image-based high-speed railway safety area intrusion alarm device and method
CN111967396A (en) Processing method, device and equipment for obstacle detection and storage medium
CN111626170A (en) Image identification method for railway slope rockfall invasion limit detection
CN113362374A (en) High-altitude parabolic detection method and system based on target tracking network
CN113298059A (en) Pantograph foreign matter intrusion detection method, device, computer equipment, system and storage medium
CN109191492B (en) Intelligent video black smoke vehicle detection method based on contour analysis
CN112084928A (en) Road traffic accident detection method based on visual attention mechanism and ConvLSTM network
Su et al. A new local-main-gradient-orientation HOG and contour differences based algorithm for object classification
CN111561967A (en) Real-time online detection method and system for pantograph-catenary operation state
CN111160224B (en) High-speed rail contact net foreign matter detection system and method based on FPGA and horizon line segmentation
CN115600124A (en) Subway tunnel inspection system and inspection method
CN115546738A (en) Rail foreign matter detection method
CN111009136A (en) Method, device and system for detecting vehicles with abnormal running speed on highway
CN105574511B (en) Have the adaptability object sorter and its method of parallel framework
CN110316630B (en) Deviation early warning method and system for installation angle of elevator camera

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant