CN117788475B - Railway dangerous tree detection method, system and equipment based on monocular depth estimation - Google Patents
Railway dangerous tree detection method, system and equipment based on monocular depth estimation Download PDFInfo
- Publication number
- CN117788475B CN117788475B CN202410211999.8A CN202410211999A CN117788475B CN 117788475 B CN117788475 B CN 117788475B CN 202410211999 A CN202410211999 A CN 202410211999A CN 117788475 B CN117788475 B CN 117788475B
- Authority
- CN
- China
- Prior art keywords
- depth
- image
- green plant
- coordinates
- railway
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 65
- 241001464837 Viridiplantae Species 0.000 claims abstract description 62
- 238000012549 training Methods 0.000 claims abstract description 13
- 238000000034 method Methods 0.000 claims description 44
- 238000004422 calculation algorithm Methods 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 abstract description 10
- 230000008569 process Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 239000007943 implant Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000004020 conductor Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000013213 extrapolation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 231100001261 hazardous Toxicity 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention belongs to the field of image detection, and particularly relates to a railway dangerous tree detection method, system and equipment based on monocular depth estimation, which aim to solve the problems of low detection speed and large error in dangerous tree detection in the prior art. The invention comprises the following steps: obtaining a video stream to obtain an image to be detected; acquiring a green plant area image; pre-constructing a depth acquisition model, training, combining an image to be detected to obtain a depth map of the image to be detected, and acquiring depth coordinates; acquiring coordinates of a cylinder region in an image to be detected; combining the depth coordinates and the coordinates of the cylinder area to obtain the average depth of the cylinder, and calculating a preset distance; performing grid division on the green plant area image, combining the depth coordinates to obtain the depth of the green plant area corresponding to each grid after grid division, and calculating and judging the distance; and combining the judging distance and the preset distance to obtain the dangerous tree area. The invention improves the calculation speed, reduces the power consumption, improves the detection precision and can adapt to more scenes.
Description
Technical Field
The invention belongs to the field of image detection, and particularly relates to a railway dangerous tree detection method, system and equipment based on monocular depth estimation.
Background
The railway power supply dangerous tree is a tree which is positioned along a railway line and forms potential danger for a railway power supply system. Railway power systems are commonly used by electrified railways, where wires or cables are suspended from a post or truss to provide power to the train. Dangerous trees may grow or incline at improper locations, causing them to collide or contact wires or cables of the power supply system, possibly causing safety problems such as arc, short circuit, fire, etc., and in severe cases, possibly causing derailment of trains or power system faults.
The definition of a railway power risk tree also typically includes the following elements:
position: dangerous trees are located in areas adjacent to the railway line, possibly on the ground on both sides of the railway, along the side slope of the railway, around the power supply system poles, or across the railway line, etc.
Size and morphology: dangerous trees may vary in size and morphology and may be adult trees, bushes in clusters, dead trees, and the like. They may have branches, leaves, trunks or crowns, etc. that may come into contact with the equipment or conductors of the power supply system.
Inclination degree: the inclination of dangerous trees may be an important determining factor. If the tree is too inclined, the risk of contact with the power supply system may be increased.
Distance and contact risk: the distance of dangerous trees is closely related to the distance of the power supply system and the risk of contact between the tree and the power supply system. If the tree is too close to the power supply system, there may be a risk of contact or collision.
In the prior art, when the distance is calculated, a longitudinal parallax method is adopted, the distance from a pillar to a camera is relatively accurate, and when the distance of a transverse dangerous tree is calculated through the longitudinal distance, the acceleration of pixel points of objects at different distances is usually different in continuous image frames captured by a vehicle-mounted camera. The position of the pixel point in the image of the object close to the camera will change faster, i.e. the acceleration will be greater. While objects far from the camera will move relatively slowly with less acceleration of their pixels. Conversely, when the vehicle is decelerating or stopped, objects closer to the camera move relatively slowly, while objects farther from the camera are stationary more quickly. Thus, the use of equal length distances in calculating the longitudinal distance can be subject to significant error, particularly for remote objects.
In the prior art, when the distance is measured, the image information of a plurality of frames before and after the distance measuring target is required to be obtained, the time average required by the program for calculating each frame of image is about 330 ms, so that the time for completely calculating the whole detection target exceeds 1000ms, and the frame number requirement of more than 40 frames in 1s cannot be met.
Based on the method, the system and the equipment for detecting the railway dangerous tree based on monocular depth estimation are provided by the invention.
Disclosure of Invention
The invention provides a railway dangerous tree detection method, a railway dangerous tree detection system and railway dangerous tree detection equipment based on monocular depth estimation, which aim to solve the problems in the prior art, namely the problems of low detection speed and large error in dangerous tree detection in the prior art.
One aspect of the invention provides a railway dangerous tree detection method based on monocular depth estimation, which comprises the following steps:
acquiring a video stream when a railway train runs, and preprocessing to obtain an image to be detected;
Acquiring a green plant area image based on a set element and a preset judging condition in the image to be detected; the setting elements include hue, saturation, and brightness;
Pre-constructing a depth acquisition model, performing mixed training on the pre-constructed depth acquisition model by utilizing a plurality of data sets to obtain a trained depth acquisition model, inputting the image to be detected into the trained depth acquisition model to obtain a depth map of the image to be detected, and acquiring depth coordinates;
acquiring coordinates of a cylinder region in the image to be detected based on a cylinder detection algorithm; the cylinder detection algorithm comprises a single-stage target detection algorithm;
obtaining a depth value of the coordinates of the cylinder region on the corresponding depth coordinates, obtaining the average depth of the cylinder, and calculating a preset distance based on the average depth;
Performing grid division on the green plant area image, combining depth coordinates to obtain the depth of a green plant area corresponding to each grid after grid division, and calculating to obtain a judgment distance based on the difference value between the average depth of the column and the depth of the green plant area corresponding to each grid;
And merging the green planting areas corresponding to the grids with the judging distance smaller than the preset distance to serve as dangerous tree areas.
In some preferred embodiments, the preprocessing includes GPU frame-extraction processing.
In some preferred embodiments, the preset determination condition is:
Wherein H is the color tone; s is saturation; v is brightness.
In some preferred embodiments, the depth acquisition model is constructed based on an encoder comprising ResNet-101 or ResNeXt-101 or DenseNet-161.
In some preferred embodiments, the dataset comprises a Pareto dataset.
In some preferred embodiments, the depth of the green plant area is obtained by:
And acquiring the outline of the green plant area image, dividing grids in the outline to obtain a plurality of green plant blocks, projecting the green plant blocks into the depth map, acquiring the depth coordinate of each green plant block, and further obtaining the depth of the green plant area corresponding to each grid.
In some preferred embodiments, the dangerous tree area is obtained by the following steps:
and selecting the green plant area corresponding to the grid exceeding the preset threshold value in the depth of the green plant area corresponding to each grid in a frame mode to obtain a plurality of frame selection areas, and combining the plurality of frame selection areas to obtain the dangerous tree area.
In another aspect of the present invention, a railway dangerous tree detection system based on monocular depth estimation is provided, and a railway dangerous tree detection method based on monocular depth estimation is provided, the system includes:
the image acquisition module is configured to acquire a video stream when the railway train runs, and preprocesses the video stream to obtain an image to be detected;
a green plant acquisition module configured to acquire a green plant area image based on a set element and a preset judgment condition in the image to be detected; the setting elements include hue, saturation, and brightness;
the depth acquisition module is configured to pre-construct a depth acquisition model, perform mixed training on the pre-constructed depth acquisition model by utilizing a plurality of data sets to obtain a trained depth acquisition model, input the image to be detected into the trained depth acquisition model to obtain a depth map of the image to be detected, and acquire depth coordinates;
the column acquisition module is configured to acquire coordinates of a column region in the image to be detected based on a column detection algorithm; the cylinder detection algorithm comprises a single-stage target detection algorithm;
The column depth acquisition module is configured to acquire a depth value of the coordinates of the column region on the corresponding depth coordinates, obtain the average depth of the column, and calculate a preset distance based on the average depth;
The green planting depth acquisition module is configured to grid divide the green planting area image, obtain the depth of the green planting area corresponding to each grid after grid division by combining the depth coordinates, and calculate the judgment distance based on the difference value between the average depth of the column and the depth of the green planting area corresponding to each grid;
And the dangerous tree area acquisition module is configured to combine the green planting areas corresponding to the grids with the judging distances smaller than the preset distances to serve as dangerous tree areas.
In a third aspect of the present invention, an electronic device is provided, including:
At least one processor; and
A memory communicatively coupled to at least one of the processors; wherein,
The memory stores instructions executable by the processor for execution by the processor to implement a railway hazard tree detection method based on monocular depth estimation as described above.
In a fourth aspect of the present invention, a computer readable storage medium is provided, where computer instructions are stored, where the computer instructions are configured to be executed by the computer to implement a method for detecting a railway risk tree based on monocular depth estimation as described above.
The invention has the beneficial effects that:
(1) Accuracy: the method and the device can accurately calculate the depth of each pixel point in a single image without depending on the calculation of a plurality of images or scales, can directly infer the depth information of the object from the single image, and improve the accuracy of the depth calculation.
(2) Efficiency is that: according to the invention, the calculation speed is improved through an optimization algorithm or a model, so that the depth calculation can be completed in a shorter time, the real-time performance requirement is met, and the high-speed depth calculation is realized so as to meet the requirement of processing more than 40 frames per second;
(3) Accurate depth estimation: the invention can realize highly accurate depth estimation in a 2C scene. By analyzing the single image, it can infer distance and depth information of objects, providing accurate scene perception and distance measurement. This is critical for applications in 2C scenarios, such as security monitoring, environmental awareness, etc.
(4) Real-time performance: the method has real-time performance and can quickly and efficiently perform depth estimation in a 2C scene. It is capable of processing an image in real time in a real-time application and generating a depth estimation result in a short time. This real-time performance is very important for 2C scenarios requiring immediate feedback and real-time decisions, such as intelligent driving, real-time safety monitoring, etc.
(5) Robustness and adaptability: the invention shows robustness and adaptability in 2C scene. It can accommodate various complex scenes and lighting conditions and effectively cope with challenges in 2C scenes, such as occlusion, texture changes, and lighting changes. This enables it to maintain stable depth estimation results in different 2C environments, providing reliable scene perception.
(6) Efficient calculation: the method has high-efficiency computing capability, and depth estimation is performed on equipment with limited computing resources. It may employ an optimized algorithm or model to increase computing speed and reduce power consumption. This enables it to efficiently perform depth estimation in 2C scenarios, adapting to situations where computational resources are limited.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the accompanying drawings in which:
FIG. 1 is a flow diagram of a railway dangerous tree detection method based on monocular depth estimation of the present invention;
FIG. 2 is an image to be detected in a railway dangerous tree detection method based on monocular depth estimation according to the present invention;
FIG. 3 is a depth map of a railway hazard tree detection method based on monocular depth estimation of the present invention;
FIG. 4 is a schematic diagram of a green plant area acquisition process of the railway dangerous tree detection method based on monocular depth estimation;
FIG. 5 is a schematic view of a green plant area of a railway dangerous tree detection method based on monocular depth estimation according to the present invention;
FIG. 6 is a schematic diagram of column region acquisition for a railway hazard tree detection method based on monocular depth estimation in accordance with the present invention;
FIG. 7 is a meshing schematic diagram of a railway dangerous tree detection method based on monocular depth estimation according to the present invention;
FIG. 8 is a schematic diagram of a computer system of a server for implementing embodiments of the method, system, and apparatus of the present application.
Detailed Description
The application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be noted that, for convenience of description, only the portions related to the present application are shown in the drawings.
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.
The first embodiment of the invention provides a railway dangerous tree detection method based on monocular depth estimation, which comprises the following steps:
acquiring a video stream when a railway train runs, and preprocessing to obtain an image to be detected;
Acquiring a green plant area image based on a set element and a preset judging condition in the image to be detected; the setting elements include hue, saturation, and brightness;
Pre-constructing a depth acquisition model, performing mixed training on the pre-constructed depth acquisition model by utilizing a plurality of data sets to obtain a trained depth acquisition model, inputting the image to be detected into the trained depth acquisition model to obtain a depth map of the image to be detected, and acquiring depth coordinates;
acquiring coordinates of a cylinder region in the image to be detected based on a cylinder detection algorithm; the cylinder detection algorithm comprises a single-stage target detection algorithm;
obtaining a depth value of the coordinates of the cylinder region on the corresponding depth coordinates, obtaining the average depth of the cylinder, and calculating a preset distance based on the average depth;
Performing grid division on the green plant area image, combining depth coordinates to obtain the depth of a green plant area corresponding to each grid after grid division, and calculating to obtain a judgment distance based on the difference value between the average depth of the column and the depth of the green plant area corresponding to each grid;
And merging the green planting areas corresponding to the grids with the judging distance smaller than the preset distance to serve as dangerous tree areas.
The invention provides a depth learning method for calculating depth in order to solve the problems of dynamic parallax, overtime calculation and the like, and the distance from the whole imaging image to a camera is calculated by using a monocular depth estimation technology. By analyzing visual features, texture and geometric information in the image, we can infer depth information for the object. This method no longer relies on longitudinal parallax, but rather estimates the true distance of the object by learning and extrapolation. Monocular depth estimation techniques may use a deep learning model, such as convolutional neural networks or self-encoders, for training and inference. The method can calculate the distance of the transverse object more accurately, and is particularly suitable for remote objects.
In order to solve the problem of overtime calculation, in order to meet the real-time requirement, the monocular depth estimation technology is optimized in real-time. We use high performance hardware devices, such as Graphics Processing Units (GPUs) or dedicated depth learning accelerators, to accelerate the inference process of the depth estimation algorithm. Furthermore, we optimize and lightweight the deep learning model to reduce computational effort and inference time. Through the optimization measures, the whole depth estimation process can be completed in a short time, and the real-time requirement is met.
In order to more clearly describe the railway dangerous tree detection method based on monocular depth estimation of the present invention, each step in the embodiment of the present invention is described in detail below with reference to fig. 1.
The railway dangerous tree detection method based on monocular depth estimation of the first embodiment of the invention comprises the following steps of:
referring to fig. 2, 4 and 5, a video stream is obtained when a railway train runs, and preprocessing is performed to obtain an image to be detected;
in the invention, the preprocessing comprises GPU frame extraction processing.
Acquiring a green plant area image based on a set element and a preset judging condition in the image to be detected; the setting elements include hue, saturation, and brightness;
in the invention, the preset judging conditions are as follows:
Wherein H is the color tone; s is saturation; v is brightness.
The trees of the evergreen are planted on the two sides of the railway, the trees are basically stopped from growing after withering in winter, and no new dangerous tree area is generated, so that only green trees are required to be identified, and the color becomes an obvious characteristic for identifying the tree area. The color in the traditional RGB color model can be changed under the influence of illumination, so that further tree region segmentation is inconvenient, the HSV color space consists of three elements of hue, saturation and brightness, the color of objects can be intuitively reflected, the application is wide, the region is segmented and identified by utilizing the HSV color space to become a common method, the hue of the tree region in the wide-risk tree detection process is similar, and therefore the HSV color space can be adopted to judge the region. The method comprises the steps of dividing and marking tree pictures collected by a shooting sample and a network, and selecting a minimum color interval under the condition of ensuring no missed detection.
Referring to fig. 3, a depth acquisition model is pre-built, the pre-built depth acquisition model is subjected to mixed training by utilizing a plurality of data sets to obtain a trained depth acquisition model, the image to be detected is input into the trained depth acquisition model to obtain a depth map of the image to be detected, and depth coordinates are acquired;
The depth acquisition model is constructed based on an encoder, wherein the encoder comprises ResNet-101 or ResNeXt-101 or DenseNet-161; the dataset includes a Pareto dataset.
The invention uses multiple training data sets to perform mixed training, which can improve model performance, and compared with the data sets used alone, the mixed training model performs better. The use of Pareto optimal data set blending strategies may further improve model performance and make more efficient use of additional data sets. The model represents a great advantage in terms of zero sample performance compared to other most advanced methods. Model performance can be significantly improved by using better encoders (e.g., resNet-101, resNeXt-101, and DenseNet-161). The use of different data sets in the training set may improve the generalization ability of the model. The model demonstrated good performance on the DIW test set and good results on the DA VIS video dataset.
Referring to fig. 6, coordinates of a cylinder region in the image to be detected are acquired based on a cylinder detection algorithm; the cylinder detection algorithm comprises a single-stage target detection algorithm;
obtaining a depth value of the coordinates of the cylinder region on the corresponding depth coordinates, obtaining the average depth of the cylinder, and calculating a preset distance based on the average depth;
Referring to fig. 7, the green plant area image is subjected to grid division, the depth of the green plant area corresponding to each grid after grid division is obtained by combining the depth coordinates, and the judgment distance is calculated based on the difference value between the average depth of the column and the depth of the green plant area corresponding to each grid;
The green plant area image is divided mainly to improve the precision. The reason is that when calculating a certain depth of an area, the actual values of some pixels will be averaged due to the use of the average value. The depth change gradient between adjacent pixels in the real scene is small, the region is divided, the smaller the division is, the smaller the average real pixel difference value is, and the higher the precision is.
When the method is specifically used, firstly, coarse grid division is carried out on the green plant area image, whether the depth value of each grid after the coarse grid division is in a preset dangerous range or not is judged, if yes, the grid is divided with higher precision, so that the divided area is smaller, and the precision is improved; as shown in fig. 7, the divided grid intervals are different, that is, the modulation is performed for different average values, so that the candidates of the dangerous tree are focused more.
In the invention, the depth of the green planting area is obtained by the following steps:
And acquiring the outline of the green plant area image, dividing grids in the outline to obtain a plurality of green plant blocks, projecting the green plant blocks into the depth map, acquiring the depth coordinate of each green plant block, and further obtaining the depth of the green plant area corresponding to each grid.
In the invention, the green plants have the characteristic of different depths in the image. Each green plant has a unique depth, and the depth difference has important significance for depth calculation. In order to more fully understand the depth distribution of green plants, the invention adopts a blocking strategy. By dividing the green plant image into blocks, we can calculate the depth of each block one by one. The blocking processing method can provide fine depth information, provides powerful support for subsequent analysis and research, and further reveals the inherent characteristics and dynamic change rules of the green plant ecological system.
And merging the green planting areas corresponding to the grids with the judging distance smaller than the preset distance to serve as dangerous tree areas.
In the invention, the green plant area corresponding to the grid with the judgment distance smaller than the preset distance is selected in a frame mode to obtain a plurality of frame selection areas, and the plurality of frame selection areas are combined to obtain the dangerous tree area.
In the present invention, in each green implant block, we introduce a threshold to evaluate the relationship between green implant depth and cylinder depth. When the difference between the depth of the green plants and the average depth of the pillars exceeds a preset threshold, we will reserve the region and frame it for further analysis and processing. Conversely, if the difference does not meet the threshold requirement, the region will be excluded from the final result in the hazardous region. By merging all the remaining box-like regions we finally get an accurate dangerous region boundary. The optimization method not only can accurately determine the range of the dangerous area, but also has important significance for the research of related patent technology and field, and provides reliable support and innovative solutions for the method. By the innovative method, a highly reliable and efficient solution is provided for green plant depth analysis and dangerous area identification, and a powerful guarantee is provided for further development and application of related industries.
In the present invention, the distances in the horizontal direction (left and right on the image) are also different for different depths due to the perspective principle of "farther and smaller", for example, the depth of 100 is 60, the depth of 110 is 60, and the depths are the same. Since the initial value of the depth is 0 with the camera lens and the depth area is fan-shaped, the true lateral distance of the coordinates 100 from the coordinates 110 can be precisely determined. Through statistics of a large amount of data, a formula of depth x-distance y can be obtained, and a preset distance and a judgment distance are obtained based on the formula of depth x-distance y, wherein the formula of depth x-distance y is as follows:
；
Where x is the depth value and y is the distance.
Although the steps are described in the above-described sequential order in the above-described embodiments, it will be appreciated by those skilled in the art that in order to achieve the effects of the present embodiments, the steps need not be performed in such order, and may be performed simultaneously (in parallel) or in reverse order, and such simple variations are within the scope of the present invention.
The second embodiment of the invention provides a railway dangerous tree detection system based on monocular depth estimation, which is based on a railway dangerous tree detection method based on monocular depth estimation, and comprises the following steps:
the image acquisition module is configured to acquire a video stream when the railway train runs, and preprocesses the video stream to obtain an image to be detected;
a green plant acquisition module configured to acquire a green plant area image based on a set element and a preset judgment condition in the image to be detected; the setting elements include hue, saturation, and brightness;
the depth acquisition module is configured to pre-construct a depth acquisition model, perform mixed training on the pre-constructed depth acquisition model by utilizing a plurality of data sets to obtain a trained depth acquisition model, input the image to be detected into the trained depth acquisition model to obtain a depth map of the image to be detected, and acquire depth coordinates;
the column acquisition module is configured to acquire coordinates of a column region in the image to be detected based on a column detection algorithm; the cylinder detection algorithm comprises a single-stage target detection algorithm;
The column depth acquisition module is configured to acquire a depth value of the coordinates of the column region on the corresponding depth coordinates, obtain the average depth of the column, and calculate a preset distance based on the average depth;
The green planting depth acquisition module is configured to grid divide the green planting area image, obtain the depth of the green planting area corresponding to each grid after grid division by combining the depth coordinates, and calculate the judgment distance based on the difference value between the average depth of the column and the depth of the green planting area corresponding to each grid;
And the dangerous tree area acquisition module is configured to combine the green planting areas corresponding to the grids with the judging distances smaller than the preset distances to serve as dangerous tree areas.
It will be clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the system described above and the related description may refer to the corresponding process in the foregoing method embodiment, which is not repeated here.
It should be noted that, in the railway dangerous tree detection system based on monocular depth estimation provided in the foregoing embodiment, only the division of the foregoing functional modules is illustrated, in practical application, the foregoing functional allocation may be completed by different functional modules according to needs, that is, the modules or steps in the foregoing embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into a plurality of sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps related to the embodiments of the present invention are merely for distinguishing the respective modules or steps, and are not to be construed as unduly limiting the present invention.
An electronic device of a third embodiment of the present invention includes:
At least one processor; and
A memory communicatively coupled to at least one of the processors; wherein,
The memory stores instructions executable by the processor for execution by the processor to implement a railway hazard tree detection method based on monocular depth estimation as described above.
A fourth embodiment of the present invention is a computer-readable storage medium storing computer instructions for execution by the computer to implement a railway hazard tree detection method based on monocular depth estimation as described above.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the storage device and the processing device described above and the related description may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.
Those of skill in the art will appreciate that the various illustrative modules, method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the program(s) corresponding to the software modules, method steps, may be embodied in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application, but such implementation is not intended to be limiting.
Referring now to FIG. 8, there is shown a block diagram of a computer system of a server for implementing embodiments of the methods, systems, and apparatus of the present application. The server illustrated in fig. 8 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present application.
As shown in fig. 8, the computer system includes a central processing unit (CPU, central Processing Unit) 801 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a random access Memory (RAM, random Access Memory) 803. In the RAM803, various programs and data required for system operation are also stored. The CPU 801, ROM 802, and RAM803 are connected to each other by a bus 804. An Input/Output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input section 807 including a keyboard, a mouse, and the like; an output portion 807 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like; a storage section 808 including a hard disk or the like; and a communication section 809 including a network interface card such as a LAN (local area network ) card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. The drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage section 808 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811. The above-described functions defined in the method of the present application are performed when the computer program is executed by a Central Processing Unit (CPU) 801. The computer readable medium of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terms "first," "second," and the like, are used for distinguishing between similar objects and not for describing a particular sequential or chronological order.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus/apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus/apparatus.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will be within the scope of the present invention.
Claims (9)
1. A railway dangerous tree detection method based on monocular depth estimation is characterized by comprising the following steps:
acquiring a video stream when a railway train runs, and preprocessing to obtain an image to be detected;
Acquiring a green plant area image based on a set element and a preset judging condition in the image to be detected; the setting elements include hue, saturation, and brightness;
Pre-constructing a depth acquisition model, performing mixed training on the pre-constructed depth acquisition model by utilizing a plurality of data sets to obtain a trained depth acquisition model, inputting the image to be detected into the trained depth acquisition model to obtain a depth map of the image to be detected, and acquiring depth coordinates;
acquiring coordinates of a cylinder region in the image to be detected based on a cylinder detection algorithm; the cylinder detection algorithm comprises a single-stage target detection algorithm;
obtaining a depth value of the coordinates of the cylinder region on the corresponding depth coordinates, obtaining the average depth of the cylinder, and calculating a preset distance based on the average depth;
Performing grid division on the green plant area image, combining depth coordinates to obtain the depth of a green plant area corresponding to each grid after grid division, and calculating to obtain a judgment distance based on the difference value between the average depth of the column and the depth of the green plant area corresponding to each grid;
The depth of the green planting area is obtained by the following steps:
Acquiring the outline of a green plant area image, dividing grids in the outline to obtain a plurality of green plant blocks, projecting the green plant blocks into the depth map, acquiring the depth coordinate of each green plant block, and further obtaining the depth of the green plant area corresponding to each grid;
And merging the green planting areas corresponding to the grids with the judging distance smaller than the preset distance to serve as dangerous tree areas.
2. The method for detecting railway risk trees based on monocular depth estimation according to claim 1, wherein the preprocessing includes GPU frame extraction processing.
3. The railway dangerous tree detection method based on monocular depth estimation according to claim 1, wherein the preset determination conditions are:
Wherein H is the color tone; s is saturation; v is brightness.
4. The method of claim 1, wherein the depth acquisition model is constructed based on an encoder comprising ResNet-101 or ResNeXt-101 or DenseNet-161.
5. The method of monocular depth estimation based railway hazard tree detection of claim 1, wherein the dataset comprises a Pareto dataset.
6. The railway dangerous tree detection method based on monocular depth estimation according to claim 1, wherein the dangerous tree area is obtained by the following steps:
and selecting the green plant area corresponding to the grid exceeding the preset threshold value in the depth of the green plant area corresponding to each grid in a frame mode to obtain a plurality of frame selection areas, and combining the plurality of frame selection areas to obtain the dangerous tree area.
7. A railway dangerous tree detection system based on monocular depth estimation, based on the railway dangerous tree detection method based on monocular depth estimation according to any one of claims 1 to 6, characterized in that the system comprises:
the image acquisition module is configured to acquire a video stream when the railway train runs, and preprocesses the video stream to obtain an image to be detected;
a green plant acquisition module configured to acquire a green plant area image based on a set element and a preset judgment condition in the image to be detected; the setting elements include hue, saturation, and brightness;
the depth acquisition module is configured to pre-construct a depth acquisition model, perform mixed training on the pre-constructed depth acquisition model by utilizing a plurality of data sets to obtain a trained depth acquisition model, input the image to be detected into the trained depth acquisition model to obtain a depth map of the image to be detected, and acquire depth coordinates;
the column acquisition module is configured to acquire coordinates of a column region in the image to be detected based on a column detection algorithm; the cylinder detection algorithm comprises a single-stage target detection algorithm;
The column depth acquisition module is configured to acquire a depth value of the coordinates of the column region on the corresponding depth coordinates, obtain the average depth of the column, and calculate a preset distance based on the average depth;
The green planting depth acquisition module is configured to grid divide the green planting area image, obtain the depth of the green planting area corresponding to each grid after grid division by combining the depth coordinates, and calculate the judgment distance based on the difference value between the average depth of the column and the depth of the green planting area corresponding to each grid;
The depth of the green planting area is obtained by the following steps:
Acquiring the outline of a green plant area image, dividing grids in the outline to obtain a plurality of green plant blocks, projecting the green plant blocks into the depth map, acquiring the depth coordinate of each green plant block, and further obtaining the depth of the green plant area corresponding to each grid;
And the dangerous tree area acquisition module is configured to combine the green planting areas corresponding to the grids with the judging distances smaller than the preset distances to serve as dangerous tree areas.
8. An electronic device, comprising:
At least one processor; and
A memory communicatively coupled to at least one of the processors; wherein,
The memory stores instructions executable by the processor for execution by the processor to implement a monocular depth estimation-based railway hazard tree detection method of any of claims 1-6.
9. A computer readable storage medium having stored thereon computer instructions for execution by the computer to implement the method of monocular depth estimation-based railway risk tree detection of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410211999.8A CN117788475B (en) | 2024-02-27 | 2024-02-27 | Railway dangerous tree detection method, system and equipment based on monocular depth estimation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410211999.8A CN117788475B (en) | 2024-02-27 | 2024-02-27 | Railway dangerous tree detection method, system and equipment based on monocular depth estimation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117788475A CN117788475A (en) | 2024-03-29 |
| CN117788475B true CN117788475B (en) | 2024-06-07 |
Family
ID=90402186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410211999.8A Active CN117788475B (en) | 2024-02-27 | 2024-02-27 | Railway dangerous tree detection method, system and equipment based on monocular depth estimation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117788475B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101771893A (en) * | 2010-01-05 | 2010-07-07 | 浙江大学 | Video frequency sequence background modeling based virtual viewpoint rendering method |
| CN106504294A (en) * | 2016-10-17 | 2017-03-15 | 浙江工业大学 | RGBD image vector methods based on diffusion profile |
| CN111368607A (en) * | 2018-12-26 | 2020-07-03 | 北京欣奕华科技有限公司 | Robot, obstacle detection method and detection device |
| CN111428859A (en) * | 2020-03-05 | 2020-07-17 | 北京三快在线科技有限公司 | Depth estimation network training method and device for automatic driving scene and autonomous vehicle |
| CN113763350A (en) * | 2021-09-03 | 2021-12-07 | 苏州凌云光工业智能技术有限公司 | Glue line detection method and device, glue line detection equipment and storage medium |
| CN113822919A (en) * | 2021-11-24 | 2021-12-21 | 中国海洋大学 | Underwater image relative depth estimation method based on semantic information constraint |
| CN113870334A (en) * | 2021-09-29 | 2021-12-31 | 北京百度网讯科技有限公司 | Depth detection method, device, equipment and storage medium |
-
2024
- 2024-02-27 CN CN202410211999.8A patent/CN117788475B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101771893A (en) * | 2010-01-05 | 2010-07-07 | 浙江大学 | Video frequency sequence background modeling based virtual viewpoint rendering method |
| CN106504294A (en) * | 2016-10-17 | 2017-03-15 | 浙江工业大学 | RGBD image vector methods based on diffusion profile |
| CN111368607A (en) * | 2018-12-26 | 2020-07-03 | 北京欣奕华科技有限公司 | Robot, obstacle detection method and detection device |
| CN111428859A (en) * | 2020-03-05 | 2020-07-17 | 北京三快在线科技有限公司 | Depth estimation network training method and device for automatic driving scene and autonomous vehicle |
| CN113763350A (en) * | 2021-09-03 | 2021-12-07 | 苏州凌云光工业智能技术有限公司 | Glue line detection method and device, glue line detection equipment and storage medium |
| CN113870334A (en) * | 2021-09-29 | 2021-12-31 | 北京百度网讯科技有限公司 | Depth detection method, device, equipment and storage medium |
| CN113822919A (en) * | 2021-11-24 | 2021-12-21 | 中国海洋大学 | Underwater image relative depth estimation method based on semantic information constraint |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117788475A (en) | 2024-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yan et al. | Real-time vehicle detection using histograms of oriented gradients and AdaBoost classification | |
| CN109325935B (en) | Power transmission line detection method based on unmanned aerial vehicle image | |
| CN109284674A (en) | A kind of method and device of determining lane line | |
| CN103069434A (en) | Multi-mode video event indexing | |
| CN110910443B (en) | Contact net geometric parameter real-time measuring method and device based on single monitoring camera | |
| CN111753757B (en) | Image recognition processing method and device | |
| Chiu et al. | Automatic Traffic Surveillance System for Vision-Based Vehicle Recognition and Tracking. | |
| CN113238578B (en) | Routing planning method and system for power tower unmanned aerial vehicle inspection route | |
| CN104881879A (en) | Remote sensing image haze simulation method based on dark-channel priori knowledge | |
| CN109935080A (en) | The monitoring system and method that a kind of vehicle flowrate on traffic route calculates in real time | |
| Liu et al. | An efficient method for high-speed railway dropper fault detection based on depthwise separable convolution | |
| CN111964763B (en) | Method for detecting intermittent driving behavior of automobile in weighing area of dynamic flat-plate scale | |
| CN113095441A (en) | Pig herd bundling detection method, device, equipment and readable storage medium | |
| CN113240906A (en) | Vehicle guiding method and system based on real-time monitoring of road congestion in logistics park | |
| CN113435801B (en) | Method, device and equipment for evaluating potential safety hazard of power transmission line based on artificial intelligence | |
| CN115825984A (en) | Power transmission line inspection system and inspection method based on laser point cloud | |
| CN111242052A (en) | Method and device for automatically distinguishing rigid and flexible wires of contact network | |
| CN117788475B (en) | Railway dangerous tree detection method, system and equipment based on monocular depth estimation | |
| CN109580653A (en) | A kind of rail fastener image-pickup method | |
| Müller et al. | Robustness evaluation and improvement for vision-based advanced driver assistance systems | |
| Kini | Real time moving vehicle congestion detection and tracking using OpenCV | |
| CN106446832A (en) | Pedestrian real-time detection method based on video | |
| CN112818836B (en) | Method and system for detecting personnel target of transformer substation scene | |
| CN111723792B (en) | Real-time positioning point identification method suitable for rigid-flexible contact network | |
| CN114820931A (en) | Virtual reality-based CIM (common information model) visual real-time imaging method for smart city |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |