CN113344964A - Image processing-based mine robot rockfall monitoring and early warning method - Google Patents
Image processing-based mine robot rockfall monitoring and early warning method Download PDFInfo
- Publication number
- CN113344964A CN113344964A CN202110697062.2A CN202110697062A CN113344964A CN 113344964 A CN113344964 A CN 113344964A CN 202110697062 A CN202110697062 A CN 202110697062A CN 113344964 A CN113344964 A CN 113344964A
- Authority
- CN
- China
- Prior art keywords
- image
- moving
- moving target
- frame
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000012545 processing Methods 0.000 title claims abstract description 46
- 238000012544 monitoring process Methods 0.000 title claims abstract description 19
- 230000002159 abnormal effect Effects 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 8
- 230000003287 optical effect Effects 0.000 claims abstract description 6
- 238000001914 filtration Methods 0.000 claims abstract description 5
- 238000007689 inspection Methods 0.000 claims abstract description 5
- 238000013024 troubleshooting Methods 0.000 claims abstract description 5
- 238000004364 calculation method Methods 0.000 claims description 18
- 230000000877 morphologic effect Effects 0.000 claims description 13
- 239000011435 rock Substances 0.000 claims description 13
- 230000008859 change Effects 0.000 claims description 10
- 238000012216 screening Methods 0.000 claims description 10
- 230000007797 corrosion Effects 0.000 claims description 6
- 238000005260 corrosion Methods 0.000 claims description 6
- 230000001186 cumulative effect Effects 0.000 claims description 3
- 238000005315 distribution function Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000000717 retained effect Effects 0.000 claims description 3
- 239000003245 coal Substances 0.000 description 6
- 238000011160 research Methods 0.000 description 4
- 239000002817 coal dust Substances 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011410 subtraction method Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a mine robot rockfall monitoring and early warning method based on image processing, which comprises the following steps: s1, scanning an underground environment through an optical camera on the inspection robot, and acquiring video images of the underground roadway environment; step S2, image preprocessing is carried out on the collected video stream, firstly abnormal image frame troubleshooting is carried out on the collected video image by calculating the average value of the gray value of the image, the abnormal image frame is equalized by adopting a histogram equalization mode, and the image frame after troubleshooting is denoised by adopting a Gaussian filtering method. The invention provides a mine robot rockfall monitoring and early warning method based on image processing.
Description
Technical Field
The invention relates to a mine robot rockfall monitoring and early warning method based on image processing.
Background
At present, the collapse and falling rocks are a great threat to the safety of the underground environment of a coal mine. For decades, the occurrence of rockfall and collapse disasters causes accidents such as casualties and the like to be uncommon. Therefore, in order to improve the safety index of underground coal mine personnel, timely monitoring the rock fall event and early warning are important tasks in the field of coal mine safety.
Current research methods for rockfall monitoring fall into two categories: sensor-based and image processing-based. Most sensors are pressure sensors, optical sensors and the like, and the research method for detecting falling rocks based on image processing comprises the following steps: the method comprises the steps of identifying falling rocks by constructing a classifier, identifying falling rocks by matching features such as motion vectors and circularity, identifying falling rocks by matching form features and vibration feature confidence coefficients, and the like. Most of the researches are carried out aiming at environments such as landslide and mountain track tunnels, and almost no research is carried out on rock falling detection under coal mines.
The underground environment of the coal mine is complex, the phenomena of insufficient illumination, coal dust diffusion and the like exist, and the identification through a machine learning method is high in identification difficulty, large in calculated amount and low in accuracy rate.
Disclosure of Invention
The invention aims to solve the technical problem of overcoming the defects of the prior art and providing the mine robot rockfall monitoring and early warning method based on image processing.
In order to solve the technical problems, the technical scheme of the invention is as follows:
a mine robot rockfall monitoring and early warning method based on image processing comprises the following steps:
s1, scanning an underground environment through an optical camera on the inspection robot, and acquiring video images of the underground roadway environment;
step S2, carrying out image preprocessing on the collected video stream, firstly carrying out abnormal image frame troubleshooting on the collected video image by calculating an image gray value average value, carrying out equalization processing on the abnormal image frame by adopting a histogram equalization mode, and carrying out denoising processing on the debugged image frame by adopting a Gaussian filtering method;
step S3, extracting a moving target of the falling object in the video image by adopting a three-frame difference method, performing morphological denoising processing on the extracted binary image of the moving target, then calculating the area of a moving target area, and screening out the moving target which meets the area threshold value characteristic;
step S4, similarity calculation is carried out on the moving target which accords with the area threshold value characteristic in the two frames of video images, when the moving targets in the two frames of video images are matched with the same target, the moving direction of the moving target is judged, and when the moving target appears in more than three continuous frames of video images and the moving direction of the moving target accords with the downward moving characteristic, the central coordinate of the moving target is reserved;
step S5, judging the track of the moving target, calculating the inclination angle and the distance of the centroid point of the moving target in the two frames of images according to the central coordinates of the moving target, and when the change of the angle and the distance meets the threshold value requirement, considering the track of the moving target as a rockfall event;
and step S6, carrying out alarm reminding according to the generated rock falling event.
Further, in step S2, performing abnormal image frame examination on the acquired video image by calculating an image gray value average value, and performing equalization processing on the abnormal image frame by using a histogram equalization method, including the following steps:
step S21, carrying out gray average calculation on the collected video image, firstly converting the video image into a gray image, then calculating the gray average of the gray image, and judging whether the current image frame is normal or not according to the size of the gray average;
step S22, performing equalization processing on the abnormal image frame in a histogram equalization manner, where the equalization processing uses a cumulative distribution function, and the mapping method is as follows:
wherein n is the sum of pixels in the image;
Njthe number of pixels of the current gray level;
l is the total number of possible gray levels in the image.
Further, the step S3 of extracting a moving object of the falling object in the video image by using a three-frame difference method includes the following steps:
step S31, performing an and operation on the two binary images obtained by subtracting the current frame image from the previous frame image and the next frame image, respectively, and implementing the formula as follows:
wherein D is1(x, y) is a binary image obtained by subtracting the current frame from the previous frame;
D2(x, y) is a binary image obtained by subtracting the current frame from the next frame;
d (x, y) is D1(x, y) and D2(x, y) the result after the AND operation;
x is the abscissa of the position of the pixel point of the original image;
y is the vertical coordinate of the position of the pixel point of the original image;
k is the current image frame number;
t is a threshold value of binarization processing;
i is the abscissa of the pixel point position of the binary image;
j is the ordinate of the pixel point position of the binary image.
Further, the step S3 of performing morphological denoising processing on the extracted binary image of the moving object, then calculating the area of the moving object region, and screening out the moving object that meets the area threshold feature includes the following steps:
and step S32, firstly, performing morphological denoising on the extracted binary image of the moving target through twice corrosion and twice expansion processing, removing small moving noise, then calculating the area of the moving target area, and screening out the moving target which meets the area threshold characteristic.
Further, the step S4 of calculating the similarity of the moving objects in the two frames of video images, which meet the area threshold feature, includes the following steps:
step S41, calculating the similarity between the two targets by using a square error calculation method, wherein the calculation formula is as follows:
Rsq_diff＝∑x′,y′[T(x′,y′)-I(x+x′,y+y′)]2；
the sum of the squared differences is smaller when the similarity of two moving objects is higher, and is 0 if there is a perfect match. Wherein, the two matched moving objects must satisfy that the moving object of the next frame image is in the extended area of the moving object of the previous frame image, namely xi-d<xi+1<xi+d；
Wherein x is the abscissa of the pixel point corresponding to the moving target;
y is the vertical coordinate of the pixel point corresponding to the moving target;
t is one of the moving targets;
i is another moving object;
d is the length of the outward expansion.
Further, the step S4 of determining the moving direction of the moving object includes the steps of:
step S42, when the moving objects in the two frame images are matched with the same object, judging whether the mass center point of the moving object in the two frame images meets yi+1>yiIf the answer is satisfied, the data is retained, and if the answer is not satisfied, the data is discarded.
Further, in the step S5, the inclination angle and the distance of the centroid point of the moving object in the two frames of images are calculated according to the central coordinates of the moving object, and when the change of the angle and the distance meets the threshold requirement, the trajectory of the moving object is considered as a rockfall event, which includes the following steps:
step S51, judging whether the angle change of the centroid point of the moving object in the two frames of images is less than +/-10 degrees, wherein the specific calculation formula is as follows:
wherein x isendAbscissa of the end point, xstartAs the abscissa of the origin, yendIs the ordinate of the end point, ystartOrdinate as starting point;
step S52, the distance between the centroid point of the moving object in the last frame image and the centroid point of the moving object in the previous frame image is yend-yend-1Judgment of yend-yend-1Whether the following formula is satisfied:
daverage<yend-yend-1<2*daverage；
wherein d isaverageIs the average distance between centroid points.
By adopting the technical scheme, the underground roadway is scanned in real time by the optical camera, the video image of the roadway is acquired, the acquired video image is subjected to preprocessing such as filtering, the rockfall target is preliminarily acquired by extracting and screening the moving target, then the rockfall target is tracked and track acquired, and whether the rockfall target track is a free falling body event or not is judged; and finally, giving an alarm to the rockfall event with collapse hidden danger. The method monitors the rock falling event of the underground roadway in real time, early warns collapse hidden danger in advance, has small identification difficulty, small calculated amount and high accuracy, reduces the probability of collapse and improves the safe operation efficiency of the coal mine.
Drawings
FIG. 1 is a flow chart of a mine robot rockfall monitoring and early warning method based on image processing according to the present invention;
FIG. 2 is a flow chart of convolution for morphological denoising in accordance with the present invention;
fig. 3 is a flowchart of calculating the area of the moving target region according to the present invention.
Detailed Description
In order that the present invention may be more readily and clearly understood, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments that are illustrated in the appended drawings.
As shown in fig. 1, a mine robot rockfall monitoring and early warning method based on image processing includes:
s1, scanning an underground environment through an optical camera on the inspection robot, and acquiring video images of the underground roadway environment;
step S2, carrying out image preprocessing on the collected video stream, firstly carrying out abnormal image frame troubleshooting on the collected video image by calculating an image gray value average value, carrying out equalization processing on the abnormal image frame by adopting a histogram equalization mode, and carrying out denoising processing on the debugged image frame by adopting a Gaussian filtering method; the image preprocessing is preprocessing performed for improving the image analysis accuracy, and the acquired images often have a large amount of noise because the camera itself may have jitter interference, complex underground environment, insufficient illumination and interference of coal dust and the like, and the purpose of the image preprocessing is to remove the noise;
step S3, extracting a moving target of the falling object in the video image by adopting a three-frame difference method, performing morphological denoising processing on the extracted binary image of the moving target, then calculating the area of a moving target area, and screening out the moving target which meets the area threshold value characteristic;
step S4, similarity calculation is carried out on the moving target which accords with the area threshold value characteristic in the two frames of video images, when the moving targets in the two frames of video images are matched with the same target, the moving direction of the moving target is judged, and when the moving target appears in more than three continuous frames of video images and the moving direction of the moving target accords with the downward moving characteristic, the central coordinate of the moving target is reserved;
step S5, judging the track of the moving target, calculating the inclination angle and the distance of the centroid point of the moving target in the two frames of images according to the central coordinates of the moving target, and when the change of the angle and the distance meets the threshold value requirement, considering the track of the moving target as a rockfall event;
and step S6, carrying out alarm reminding according to the generated rock falling event.
As shown in fig. 1, in step S2, the method performs abnormal image frame elimination on the collected video image by calculating an image gray value average value, and performs equalization processing on the abnormal image frame by using a histogram equalization method, including the following steps:
step S21, carrying out gray average calculation on the collected video image, firstly converting the video image into a gray image, then calculating the gray average of the gray image, and judging whether the current image frame is normal or not according to the size of the gray average;
step S22, performing equalization processing on the abnormal image frame in a histogram equalization manner, and using the histogram equalization processing to effectively solve the problem of too dark or too bright image, where the equalization processing uses a cumulative distribution function, and the mapping method is as follows:
wherein n is the sum of pixels in the image;
Njthe number of pixels of the current gray level;
l is the total number of possible gray levels in the image.
As shown in fig. 1, the step S3 of extracting a moving object of a falling object in a video image by using a three-frame difference method includes the following steps:
step S31, the method for extracting a moving target includes a hybrid gaussian method, a three-frame difference method, and a background subtraction method, where the hybrid gaussian method is analyzed from the perspective of analysis efficiency and accuracy with high accuracy but low speed, and the background subtraction method has a high requirement on the background environment, so in this embodiment, the three-frame difference method is selected to extract the moving target, and the current frame image is respectively subtracted from the previous frame image and the next frame image to obtain two binary images, and the two binary images are subjected to an and operation, and the implementation formula is as follows:
wherein D is1(x, y) is a binary image obtained by subtracting the current frame from the previous frame;
D2(x, y) is a binary image obtained by subtracting the current frame from the next frame;
d (x, y) is D1(x, y) and D2(x, y) the result after the AND operation;
x is the abscissa of the position of the pixel point of the original image;
y is the vertical coordinate of the position of the pixel point of the original image;
k is the current image frame number;
t is a threshold value of binarization processing;
i is the abscissa of the pixel point position of the binary image;
j is the ordinate of the pixel point position of the binary image.
As shown in fig. 1, the morphological denoising processing is performed on the extracted binary image of the moving object in step S3, then the area of the moving object region is calculated, and the moving object meeting the area threshold feature is screened out, including the following steps:
and step S32, firstly, performing morphological denoising on the extracted binary image of the moving target through twice corrosion and twice expansion processing, removing small moving noise, then calculating the area of the moving target area, and screening out the moving target which meets the area threshold characteristic. In the binary image of the moving target, besides the moving target object, there are some other noise interferences, for example, some slight shakes which cannot be detected by naked eyes may appear when the camera shoots, or the movement of some tiny dust particles in the roadway is subjected to the first-step screening through morphological denoising, so as to filter the noise interferences. And calculating the area of the moving target area, and screening out the moving target according with the area threshold characteristic as a second step, wherein only rockfall is reserved, so that the monitoring is more accurate.
In the morphological denoising in the embodiment, the main role of mathematical morphology in image processing is to transform geometric parameters or gray values of an object in an image, and common methods include corrosion, expansion, opening operation, closing operation, morphological reconstruction and the like. The expansion has the function of expanding the image, namely, the edge of the image is expanded to realize the filling of a target edge or an inner edge gap, the corrosion has the function of contracting the image, the noise in the image can be removed by using the operation, and the burr of the target edge is removed.
As shown in fig. 2, in this embodiment, the expansion is analyzed from a mathematical perspective, that is, the operation of finding the local maximum, and the convolution of a and B is to find the maximum of the pixel points in the area covered by B, and assign the maximum to the pixel specified by the reference point, thereby increasing the highlight area. Its operator is [ ] [:
dst(x,y)＝max(src(x+x',y+y'))；
wherein (x ', y'): element (x ', y') > is not equal to 0;
as shown in fig. 2, the corrosion in this embodiment is that the value is 1 only when the values of the elements corresponding to the convolution kernels are all 1, otherwise, the value is modified to 0. In other words, when a certain position is traversed, the surrounding is all white, white is reserved, otherwise, the image becomes black, and image erosion is reduced. The process is exactly the reverse of the expansion. The operator is "-":
dst(x,y)＝min(src(x+x',y+y'))；
wherein (x ', y'): element (x ', y'). noteq.0.
In this embodiment, the area of the moving target region is calculated, the moving target conforming to the area threshold characteristic is screened out, and the area of the moving target region is calculated by using a counterraa method in opencv, the method is calculated according to Green formula, a specific calculation mode is shown in fig. 3, and the calculation is 2.5 for actually 7 pixels.
As shown in fig. 1, the step S4 of calculating the similarity of the moving objects in the two frames of video images, which meet the area threshold feature, includes the following steps:
step S41, in this embodiment, the degree of similarity between two targets is calculated by using a square error calculation method, and the calculation formula is:
Rsq_diff＝∑x′,y′[T(x′,y′)-I(x+x′,y+y′)]2；
the sum of the squared differences is smaller when the similarity of two moving objects is higher, and is 0 if there is a perfect match. Wherein, the two matched moving objects must satisfy that the moving object of the next frame image is in the extended area of the moving object of the previous frame image, namely xi-d<xi+1<xi+d；
Wherein x is the abscissa of the pixel point corresponding to the moving target;
y is the vertical coordinate of the pixel point corresponding to the moving target;
t is one of the moving targets;
i is another moving object;
d is the length of the outward expansion.
As shown in fig. 1, the step S4 of determining the moving direction of the moving object includes the following steps:
step S42, when the moving objects in the two frame images are matched with the same object, judging whether the mass center point of the moving object in the two frame images meets yi+1>yiIf the answer is satisfied, the data is retained, and if the answer is not satisfied, the data is discarded.
As shown in fig. 1, in step S5, the inclination angle and the distance of the centroid point of the moving object in the two frames of images are calculated according to the central coordinates of the moving object, and when the change of the angle and the distance meets the threshold requirement, the trajectory of the moving object is considered as a rockfall event, which includes the following steps:
step S51, judging whether the angle change of the centroid point of the moving object in the two frames of images is less than +/-10 degrees, wherein the specific calculation formula is as follows:
wherein x isendAbscissa of the end point, xstartAs the abscissa of the origin, yendIs the ordinate of the end point, ystartOrdinate as starting point;
step S52, the distance between the centroid point of the moving object in the last frame image and the centroid point of the moving object in the previous frame image is yend-yend-1Judgment of yend-yend-1Whether the following formula is satisfied:
daverage<yend-yend-1<2*daverage；
wherein d isaverageIs the average distance between centroid points.
When the inclination angle and the distance change of the centroid point of the moving target meet the conditions, the moving target can be judged to be a complete free falling process, the track of the moving target is considered to be a rock falling event, and alarm reminding is carried out according to the generated rock falling event.
The technical problems, technical solutions and advantages of the present invention have been described in detail with reference to the above embodiments, and it should be understood that the above embodiments are merely exemplary and not intended to limit the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (7)
1. A mine robot rockfall monitoring and early warning method based on image processing is characterized by comprising the following steps:
s1, scanning an underground environment through an optical camera on the inspection robot, and acquiring video images of the underground roadway environment;
step S2, carrying out image preprocessing on the collected video stream, firstly carrying out abnormal image frame troubleshooting on the collected video image by calculating an image gray value average value, carrying out equalization processing on the abnormal image frame by adopting a histogram equalization mode, and carrying out denoising processing on the debugged image frame by adopting a Gaussian filtering method;
step S3, extracting a moving target of the falling object in the video image by adopting a three-frame difference method, performing morphological denoising processing on the extracted binary image of the moving target, then calculating the area of a moving target area, and screening out the moving target which meets the area threshold value characteristic;
step S4, similarity calculation is carried out on the moving target which accords with the area threshold value characteristic in the two frames of video images, when the moving targets in the two frames of video images are matched with the same target, the moving direction of the moving target is judged, and when the moving target appears in more than three continuous frames of video images and the moving direction of the moving target accords with the downward moving characteristic, the central coordinate of the moving target is reserved;
step S5, judging the track of the moving target, calculating the inclination angle and the distance of the centroid point of the moving target in the two frames of images according to the central coordinates of the moving target, and when the change of the angle and the distance meets the threshold value requirement, considering the track of the moving target as a rockfall event;
and step S6, carrying out alarm reminding according to the generated rock falling event.
2. The mine robot rockfall monitoring and early warning method based on image processing as claimed in claim 1, wherein in step S2, abnormal image frame inspection is performed on the collected video image by calculating an image gray value average value, and the abnormal image frame is equalized by means of histogram equalization, comprising the steps of:
step S21, carrying out gray average calculation on the collected video image, firstly converting the video image into a gray image, then calculating the gray average of the gray image, and judging whether the current image frame is normal or not according to the size of the gray average;
step S22, performing equalization processing on the abnormal image frame in a histogram equalization manner, where the equalization processing uses a cumulative distribution function, and the mapping method is as follows:
wherein n is the sum of pixels in the image;
Njthe number of pixels of the current gray level;
l is the total number of possible gray levels in the image.
3. The mine robot rockfall monitoring and early warning method based on image processing as claimed in claim 1, wherein in step S3, a three-frame difference method is adopted to extract a moving object of a falling object in a video image, comprising the following steps:
step S31, performing an and operation on the two binary images obtained by subtracting the current frame image from the previous frame image and the next frame image, respectively, and implementing the formula as follows:
wherein D is1(x, y) is a binary image obtained by subtracting the current frame from the previous frame;
D2(x, y) is a binary image obtained by subtracting the current frame from the next frame;
d (x, y) is D1(x, y) and D2(x, y) the result after the AND operation;
x is the abscissa of the position of the pixel point of the original image;
y is the vertical coordinate of the position of the pixel point of the original image;
k is the current image frame number;
t is a threshold value of binarization processing;
i is the abscissa of the pixel point position of the binary image;
j is the ordinate of the pixel point position of the binary image.
4. The mine robot rockfall monitoring and early warning method based on image processing as claimed in claim 3, wherein in step S3, morphological denoising processing is performed on the extracted binary image of the moving target, then the area of the moving target region is calculated, and the moving target meeting the area threshold feature is screened out, including the following steps:
and step S32, firstly, performing morphological denoising on the extracted binary image of the moving target through twice corrosion and twice expansion processing, removing small moving noise, then calculating the area of the moving target area, and screening out the moving target which meets the area threshold characteristic.
5. The mine robot rockfall monitoring and early warning method based on image processing as claimed in claim 1, wherein the step S4 of performing similarity calculation on the moving objects meeting the area threshold feature in the two frames of video images includes the following steps:
step S41, calculating the similarity between the two targets by using a square error calculation method, wherein the calculation formula is as follows:
Rsq_diff＝∑x′,y′[T(x′,y′)-I(x+x′,y+y′)]2；
the sum of the squared differences is smaller when the similarity of two moving objects is higher, and is 0 if there is a perfect match. Wherein, the two matched moving objects must satisfy that the moving object of the next frame image is in the extended area of the moving object of the previous frame image, namely xi-d<xi+1<xi+d；
Wherein x is the abscissa of the pixel point corresponding to the moving target;
y is the vertical coordinate of the pixel point corresponding to the moving target;
t is one of the moving targets;
i is another moving object;
d is the length of the outward expansion.
6. The mine robot rockfall monitoring and early warning method based on image processing as claimed in claim 5, wherein the step S4 of distinguishing the moving direction of the moving object includes the following steps:
step S42, when the moving objects in the two frame images are matched with the same object, judging whether the mass center point of the moving object in the two frame images meets yi+1>yiIf the answer is satisfied, the data is retained, and if the answer is not satisfied, the data is discarded.
7. The mine robot rockfall monitoring and early warning method based on image processing as claimed in claim 1, wherein in step S5, the inclination angle and distance of the centroid point of the moving target in the two frames of images are calculated according to the central coordinates of the moving target, and when the change of the angle and distance meets the threshold requirement, the moving target trajectory is considered as a rockfall event, comprising the following steps:
step S51, judging whether the angle change of the centroid point of the moving object in the two frames of images is less than +/-10 degrees, wherein the specific calculation formula is as follows:
wherein x isendAbscissa of the end point, xstartAs the abscissa of the origin, yendIs the ordinate of the end point, ystartOrdinate as starting point;
step S52, the distance between the centroid point of the moving object in the last frame image and the centroid point of the moving object in the previous frame image is yend-yend-1Judgment of yend-yend-1Whether the following formula is satisfied:
daverage<yend-yend-1<2*daverage；
wherein d isaverageIs the average distance between centroid points.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110697062.2A CN113344964B (en) | 2021-06-23 | 2021-06-23 | Mine robot falling stone monitoring and early warning method based on image processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110697062.2A CN113344964B (en) | 2021-06-23 | 2021-06-23 | Mine robot falling stone monitoring and early warning method based on image processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113344964A true CN113344964A (en) | 2021-09-03 |
| CN113344964B CN113344964B (en) | 2024-02-23 |
Family
ID=77478055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110697062.2A Active CN113344964B (en) | 2021-06-23 | 2021-06-23 | Mine robot falling stone monitoring and early warning method based on image processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113344964B (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20120076494A (en) * | 2010-11-26 | 2012-07-09 | 주식회사 휴텍이일 | Mine safety surveilance system |
| CN103726879A (en) * | 2013-12-26 | 2014-04-16 | 辽宁石油化工大学 | Method for utilizing camera to automatically capture mine earthquakes and collapses of mine and timely record and alarm |
| CN104992139A (en) * | 2015-04-23 | 2015-10-21 | 北京宇航时代科技发展有限公司 | Falling rock detection method based on matching between motion vector and circularity |
| CN104992428A (en) * | 2015-04-23 | 2015-10-21 | 北京宇航时代科技发展有限公司 | Image falling rook volume determination method based on K mean value cluster analysis |
| CN107886495A (en) * | 2017-09-30 | 2018-04-06 | 北京得华机器人技术研究院有限公司 | A kind of auto-parts defect identification method based on similarity mode |
| CN108765470A (en) * | 2018-06-17 | 2018-11-06 | 天津理工大学 | One kind being directed to the improved KCF track algorithms of target occlusion |
| CN109102523A (en) * | 2018-07-13 | 2018-12-28 | 南京理工大学 | A kind of moving object detection and tracking |
| CN109685084A (en) * | 2019-03-07 | 2019-04-26 | 无锡众创未来科技应用有限公司 | A kind of image processing apparatus based on the description of image object feature |
| CN110163119A (en) * | 2019-04-30 | 2019-08-23 | 中国地质大学（武汉） | A kind of finger vein identification method and system |
| CN110211182A (en) * | 2019-05-31 | 2019-09-06 | 东北大学 | A kind of LCD backlight vision positioning method based on Gray-scale Matching and objective contour |
| CN111062293A (en) * | 2019-12-10 | 2020-04-24 | 太原理工大学 | Unmanned aerial vehicle forest flame identification method based on deep learning |
| CN111160305A (en) * | 2019-12-31 | 2020-05-15 | 武汉工程大学 | Transformer bushing abnormity monitoring method and computer readable medium |
| CN111502763A (en) * | 2020-04-29 | 2020-08-07 | 山东黄金集团蓬莱矿业有限公司 | Mine safety dynamic video monitoring management system |
| CN111951178A (en) * | 2020-07-07 | 2020-11-17 | 中国人民解放军93114部队 | Image processing method and device for remarkably improving image quality and electronic equipment |
| KR102225146B1 (en) * | 2019-12-13 | 2021-03-10 | 주식회사 이에스피 | Mine Vehicle V2X System and Method for Management of Mine Safety |
| CN112627899A (en) * | 2020-12-31 | 2021-04-09 | 兰州资源环境职业技术学院 | Mine safety dynamic monitoring management system |
-
2021
- 2021-06-23 CN CN202110697062.2A patent/CN113344964B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20120076494A (en) * | 2010-11-26 | 2012-07-09 | 주식회사 휴텍이일 | Mine safety surveilance system |
| CN103726879A (en) * | 2013-12-26 | 2014-04-16 | 辽宁石油化工大学 | Method for utilizing camera to automatically capture mine earthquakes and collapses of mine and timely record and alarm |
| CN104992139A (en) * | 2015-04-23 | 2015-10-21 | 北京宇航时代科技发展有限公司 | Falling rock detection method based on matching between motion vector and circularity |
| CN104992428A (en) * | 2015-04-23 | 2015-10-21 | 北京宇航时代科技发展有限公司 | Image falling rook volume determination method based on K mean value cluster analysis |
| CN107886495A (en) * | 2017-09-30 | 2018-04-06 | 北京得华机器人技术研究院有限公司 | A kind of auto-parts defect identification method based on similarity mode |
| CN108765470A (en) * | 2018-06-17 | 2018-11-06 | 天津理工大学 | One kind being directed to the improved KCF track algorithms of target occlusion |
| CN109102523A (en) * | 2018-07-13 | 2018-12-28 | 南京理工大学 | A kind of moving object detection and tracking |
| CN109685084A (en) * | 2019-03-07 | 2019-04-26 | 无锡众创未来科技应用有限公司 | A kind of image processing apparatus based on the description of image object feature |
| CN110163119A (en) * | 2019-04-30 | 2019-08-23 | 中国地质大学（武汉） | A kind of finger vein identification method and system |
| CN110211182A (en) * | 2019-05-31 | 2019-09-06 | 东北大学 | A kind of LCD backlight vision positioning method based on Gray-scale Matching and objective contour |
| CN111062293A (en) * | 2019-12-10 | 2020-04-24 | 太原理工大学 | Unmanned aerial vehicle forest flame identification method based on deep learning |
| KR102225146B1 (en) * | 2019-12-13 | 2021-03-10 | 주식회사 이에스피 | Mine Vehicle V2X System and Method for Management of Mine Safety |
| CN111160305A (en) * | 2019-12-31 | 2020-05-15 | 武汉工程大学 | Transformer bushing abnormity monitoring method and computer readable medium |
| CN111502763A (en) * | 2020-04-29 | 2020-08-07 | 山东黄金集团蓬莱矿业有限公司 | Mine safety dynamic video monitoring management system |
| CN111951178A (en) * | 2020-07-07 | 2020-11-17 | 中国人民解放军93114部队 | Image processing method and device for remarkably improving image quality and electronic equipment |
| CN112627899A (en) * | 2020-12-31 | 2021-04-09 | 兰州资源环境职业技术学院 | Mine safety dynamic monitoring management system |
Non-Patent Citations (2)
| Title |
|---|
| 卢嘉宁， ***，栗飞，杨瑞: "隧道口异物入侵与塌方监测预警***", 《机电一体化》, 30 April 2019 (2019-04-30), pages 32 - 46 * |
| 徐成伟，张德强，张 昊， 王宝田，黎悦韬: "基于雷达与视频分析技术的铁路崩塌落石自动监测报警***设计与实现", 《铁路计算机应用》, vol. 26, no. 11, 30 November 2017 (2017-11-30), pages 22 - 27 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113344964B (en) | 2024-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111951237B (en) | Visual appearance detection method | |
| US10217229B2 (en) | Method and system for tracking moving objects based on optical flow method | |
| US10127448B2 (en) | Method and system for dismount detection in low-resolution UAV imagery | |
| CN101739686B (en) | Moving object tracking method and system thereof | |
| CN110211101A (en) | A kind of rail surface defect rapid detection system and method | |
| KR101735365B1 (en) | The robust object tracking method for environment change and detecting an object of interest in images based on learning | |
| CN108198206A (en) | The multi-object tracking method combined based on multiple features combining and Camshift algorithms | |
| CN110532875B (en) | Night mode lens attachment detection system, terminal and storage medium | |
| CN108765453B (en) | Expressway agglomerate fog identification method based on video stream data | |
| Li et al. | Detection algorithm of defects on polyethylene gas pipe using image recognition | |
| CN111815556A (en) | Vehicle-mounted fisheye camera self-diagnosis method based on texture extraction and wavelet transformation | |
| CN115631191A (en) | Coal blockage detection algorithm based on gray level features and edge detection | |
| Wilson et al. | A comprehensive study on fire detection | |
| CN108446581B (en) | Unmanned aerial vehicle detection method in severe environment | |
| Wei et al. | Real-time moving object detection for video monitoring systems | |
| CN113344964B (en) | Mine robot falling stone monitoring and early warning method based on image processing | |
| CN117671799A (en) | Human body falling detection method, device, equipment and medium combining depth measurement | |
| Raikar et al. | Automatic building detection from satellite images using internal gray variance and digital surface model | |
| CN116452976A (en) | Underground coal mine safety detection method | |
| JP4302801B2 (en) | Object tracking method and object tracking apparatus | |
| CN106778675B (en) | A kind of recognition methods of target in video image object and device | |
| Zaharin et al. | Comparison of human detection using background subtraction and frame difference | |
| CN112801963B (en) | Video image occlusion detection method and system | |
| KR102585146B1 (en) | Pothole detection device and method based on deep learning | |
| CN110210447B (en) | Method and device for detecting moving target in underground dangerous area |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |