CN110335255B - Dam slope crack detection method and bionic gecko crawling detection device - Google Patents

Dam slope crack detection method and bionic gecko crawling detection device Download PDF

Info

Publication number
CN110335255B
CN110335255B CN201910514001.0A CN201910514001A CN110335255B CN 110335255 B CN110335255 B CN 110335255B CN 201910514001 A CN201910514001 A CN 201910514001A CN 110335255 B CN110335255 B CN 110335255B
Authority
CN
China
Prior art keywords
crack
camera
detection device
bionic
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910514001.0A
Other languages
Chinese (zh)
Other versions
CN110335255A (en
Inventor
邹湘军
雷子毅
唐昀超
黄矿裕
陈明猷
黄钊丰
朱惠贤
张晋豪
徐婉冬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China Agricultural University
Zhongkai University of Agriculture and Engineering
Original Assignee
South China Agricultural University
Zhongkai University of Agriculture and Engineering
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China Agricultural University, Zhongkai University of Agriculture and Engineering filed Critical South China Agricultural University
Priority to CN201910514001.0A priority Critical patent/CN110335255B/en
Publication of CN110335255A publication Critical patent/CN110335255A/en
Application granted granted Critical
Publication of CN110335255B publication Critical patent/CN110335255B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62D—MOTOR VEHICLES; TRAILERS
    • B62D57/00—Vehicles characterised by having other propulsion or other ground- engaging means than wheels or endless track, alone or in addition to wheels or endless track
    • B62D57/02—Vehicles characterised by having other propulsion or other ground- engaging means than wheels or endless track, alone or in addition to wheels or endless track with ground-engaging propulsion means, e.g. walking members
    • B62D57/024—Vehicles characterised by having other propulsion or other ground- engaging means than wheels or endless track, alone or in addition to wheels or endless track with ground-engaging propulsion means, e.g. walking members specially adapted for moving on inclined or vertical surfaces
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20228—Disparity calculation for image-based rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30132—Masonry; Concrete

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Mechanical Engineering (AREA)
  • Geometry (AREA)
  • Transportation (AREA)
  • Combustion & Propulsion (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Quality & Reliability (AREA)
  • Signal Processing (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a dam slope crack detection method and a bionic gecko crawling detection device. The method comprises the steps of firstly training a neural network, then detecting dam crack images acquired in real time by adopting the trained u-net neural network, and accurately segmenting cracks and background sundries in the images. When the crack detection is carried out on the inclined dam, a bionic gecko crawling detection device is adopted, and the detection device comprises a machine body, a walking bionic mechanism, a camera clamping mechanism and a light source illumination system. The invention can well detect the inclined dam cracks including information such as crack width, crack depth and the like through the neural network and the binocular vision detection system, has high detection precision, and enables the detection device to walk and detect on the inclined dam stably through the walking bionic mechanism.

Description

Dam slope crack detection method and bionic gecko crawling detection device
Technical Field
The invention relates to the field of detection, in particular to a dam slope crack detection method based on deep learning and a bionic gecko crawling detection device.
Background
The dam is a water retaining structure for intercepting river channel water flow to raise the water level or adjust the flow, and if cracks occur, the stability of the inclined plane of the dam and the safety of the dam are affected. At present, dam cracks are detected generally by a professional regularly removing water from a dam by using a crack depth measuring instrument and a crack width measuring instrument, and then the professional judges the risk level of the cracks according to experience. However, dams generally have gradient, manual detection is dangerous, time-consuming and labor-consuming, real-time detection is difficult, and phenomena of missing judgment and misjudgment easily occur in manual detection. At present, the deep learning method is applied to the field of image processing, so that the deep learning can be used for detecting dam slope cracks to improve the detection accuracy.
Disclosure of Invention
The invention aims to overcome the defects in the prior art, and provides a dam slope crack detection method based on deep learning, which has high accuracy, real-time detection and low cost.
The invention also aims to provide a bionic gecko crawling detection device adopting the dam slope crack detection method.
The purpose of the invention is realized by the following technical scheme:
a dam slope crack detection method based on deep learning comprises the following steps:
(1) Training a neural network: because the image of the dam crack comprises the crack and background sundries, and the proportion of the crack in the image is generally only 2-6%, the U-net neural network cannot be directly used for detection, and a loss function needs to be improved firstly, and then the U-net neural network is trained:
1-1, preparing a training set and a testing set of the neural network: deep learning training requires a large number of samples, with two solutions: one is to obtain new data, but the method is troublesome, needs a large amount of cost, and has less sample data of dam cracks; the other method is to enhance the existing data, namely to turn, translate or rotate the existing data to create more data, so that the neural network has better generalization effect; the invention adopts a second method, firstly, a CCD industrial camera is used for collecting real-time images of dam slopes to obtain more than 1000 crack images, and the collected images are preprocessed; then using 60-80% of the images as a training set for network training, and using the other 20-40% of the images as a test set for network testing;
1-2, adjusting parameters of the neural network: the U-net neural network can adapt to the crack image by adjusting the parameters of the U-net neural network, so that cracks and background impurities are accurately segmented;
1-3, testing: testing the training result of the U-net neural network by using the test set obtained in the step 1-1, and evaluating by using the average cross-over ratio MIOU and the Dice coefficient;
(2) Detecting the dam crack image acquired in real time by adopting the trained u-net neural network, and accurately segmenting cracks and background sundries in the image:
2-1, calibrating a camera: aiming a camera at a known structure with a plurality of independent identifiable points to obtain a calibration image, and then solving internal parameters and external parameters of the camera through an internal function of OpenCV, wherein the internal parameters comprise a rotation matrix R and a translational vector T;
2-2, correcting the image: shooting by a left camera and a right camera to obtain real-time left and right images of the dam crack; then, correcting the real-time left and right images according to the rotation matrix R and the translational vector T obtained in the step 2-1 to obtain corrected left and right images;
2-3, accurately segmenting the corrected cracks and background sundries in the left and right images by adopting the trained u-net neural network to obtain crack characteristic information of the left and right images;
2-4, stereo matching: matching crack characteristic information of the left image and the right image, and analyzing the deviation between the characteristics, namely parallax error to obtain a parallax error image;
2-5, solving the depth of the crack: and converting the disparity map into a depth map, wherein the depth is as follows:
Figure GDA0004065116380000021
wherein depth represents depth information of the crack; f represents a normalized focal length; b is the distance between the optical centers of the two cameras, also known as the baseline distance; disp is the disparity value;
2-6. Solving the width of the crack: the width H (a, B) is the maximum width of the crack in the image, and the formula is shown below:
H(A,B)＝max[h(A,B),h(B,A)]
Figure GDA0004065116380000022
Figure GDA0004065116380000023
wherein A is the upper edge point of the crack, and B is the lower edge point of the crack; h (A, B) is the one-way distance from point set A to point set B, and h (B, A) is the one-way distance from point set B to point set A. The distance measurement adopts the squared distance of Euclidean space, the width information of the crack in the image is solved, and actually, the surface distance between the upper edge area and the lower edge area of the crack is solved.
In the step 1-1, the acquired image is preprocessed because the acquired image under natural conditions may have large variation of illumination parameters, and the acquired original image needs to be uniformly illuminated to eliminate the influence of different illuminations on crack detection.
The algorithm for uniform illumination processing is as follows:
f(x,y)＝g(x,y)-h(x,y)+a
wherein f (x, y) is an image pixel point subjected to the dodging, g (x, y) is an image pixel point which is not subjected to the dodging, h (x, y) is a background pixel point reflecting the brightness distribution, and a is the gray offset. The gray level offset has the function of enabling the gray level of the image after the dodging processing to be distributed in a reasonable value range, so that the image brightness mean value is consistent with the original brightness, and the problems of large brightness in some places and small brightness in some places are solved.
In the step 1-2, the training set obtained in the step 1-1 is adopted, the U-net network is trained by using TensorFlow, the learning rate is 0.01-0.001, the iteration is carried out for 2000-3000 times, and the loss rate is iterated to be below 0.4, so that the training of the network model is completed.
In step 1-2, the adjusted parameters include setting a loss function and adjusting an L2 regularization coefficient.
The loss function is set because the cracks only occupy a small proportion in the image, but the proportion of the complex background is large, under the condition, overfitting of a neural network model is easily caused, and through weighting the loss function, the proportion of the region where the cracks are located can be improved, so that the neural network can be more concentrated on learning of crack characteristics. The loss function L is shown below:
Figure GDA0004065116380000031
wherein n is represented by the number of categories (n takes the value of 2 and represents two categories of complex background and crack); w is a weight of the image,
Figure GDA0004065116380000032
respectively representing the probability of the class i category and the probability of model prediction.
The weight w in the loss function is updated for each pixel in the image so that the neural network can learn the edge information of the crack specifically. The weight w (x) after update, is as follows:
Figure GDA0004065116380000033
wherein b represents a deviation, and parameters of the deviation need to be given artificially; d 1 Representing the nearest distance from the pixel point to the boundary on the crack image; d 2 And representing the nearest distance from the pixel point to the lower boundary of the crack image. Based on experience, we can weight the initial weight w 0 Set to 10, the standard deviation σ is about 5 pixels, and b is about 1. The method compensates different frequencies of each type of pixel in training data by pre-calculating the weight to obtain the weight of each pixel in the loss function, and enables a neural network to pay more attention to learning the information of the crack in the graph.
In training the neural network, the L2 regularization coefficient is set to 0.0005 to prevent overfitting of the neural network model.
In the steps 1-3, dice is a judgment standard of fracture image segmentation:
Figure GDA0004065116380000041
the method comprises the following steps that I, G and P respectively represent a real value and a prospect in a prediction result, and for a crack image, a crack in an expert calibration image and a crack in a segmentation result correspond to each other; | G ≦ P | represents an overlapping area of the two; 0 means no overlap and no similarity at all; 1 represents complete overlap; higher values indicate better network segmentation.
In step 1-3, MIOU is also a criterion for crack image segmentation, and calculates the ratio of the intersection and union of two sets, namely an expert calibration image and a predicted value:
Figure GDA0004065116380000042
wherein K is represented by the categoryNumber (K takes the value 2, representing two categories of complex background and fracture); p is a radical of ij Representing the number of objects of which the ith is predicted to be the jth type; p is a radical of ji Indicating the number of the jth predicted as the ith type of articles; p is a radical of ii Indicating that the ith is predicted to be the number of the ith type of articles.
In step 2-4, the parallax refers to a difference value of the same features observed by the left camera and the right camera on an x coordinate, and the parallax value disp = x l -x r Wherein x is l Is the distance from a certain characteristic point of the crack in the left camera picture to the center of the optical axis, x r Is the distance from the same characteristic point of the right camera to the center of the optical axis of the right camera.
A bionic gecko crawling detection device comprises a machine body 2-1, a walking bionic mechanism 2-2, a camera clamping mechanism 2-3 and a light source illumination system 2-4, and is used for detecting by adopting the dam slope crack detection method based on deep learning; the walking bionic mechanism 2-2 is arranged on the periphery of the lower edge of the machine body and used for driving the bionic gecko crawling detection device to walk on the inclined dam; the camera clamping device 2-3 is arranged on an L-shaped section sliding rail 2-1-2 of the machine body and used for erecting a camera and adjusting the height of the camera from a detection surface; the light source illumination system 2-4 is mounted on top of the body for providing sufficient and stable light source illumination to the monitored area.
The machine body 2-1 comprises a machine cover 2-1-1, four L-shaped section sliding rails 2-1-2 and two side ear buckles 2-1-3. The hollow part of the hood is a detection area; four sliding rails with L-shaped sections are vertically arranged on the upper part of the hood; two side ear buckles are respectively installed in aircraft bonnet middle part both sides for connect the cable. A large square hole and eight small rectangular holes are formed in the top of the hood, so that the range of adjustment from the lower portion to the upper portion of the top surface of the hood is provided for the camera clamping mechanisms 2-3, and meanwhile, the set screws can be conveniently screwed by detection personnel.
The walking bionic mechanism 2-2 comprises four bionic gecko feet which are respectively arranged on the periphery of the lower edge of the machine body, and each bionic gecko foot comprises a gecko foot main body 2-2-2, a gecko foot limb 2-2-3 and a gecko foot toe 2-2-1. Steering gears are arranged on the gecko foot limbs 2-2-3 to steer. The bionic gecko foot is of a multi-fiber surface structure, more than one row of gaskets is arranged below each toe of the bionic gecko foot, more than one setae is arranged on each gasket, and more than one hairy antler small brush is arranged at the top end of each setae. The small brushes form very large adsorption force, so that the bionic gecko crawling detection device can stably detect and walk on the inclined dam. When the bionic gecko crawling detection device is used for detecting, feet of the bionic gecko can be firmly attached to the inclined dam and cannot slide down due to gravity and dam inclination. When the bionic gecko crawling detection device walks, part of bristles on the toes of the gecko are contracted, the inclination angle of the bristles is reduced, so that the friction force of the detection device on a dam is reduced, and then the gecko crawling detection device walks on the dam through the gecko feet.
The camera clamping mechanism 2-3 comprises a machine cover 2-3-1, a sliding block 2-3-2 and a clamp 2-3-4; four corners of the machine cover 2-3-1 are provided with slide blocks 2-3-2 which are used for moving on an L-shaped section slide rail 2-1-2 of the machine body, and the L-shaped section slide rail is provided with scales which are used for calculating the distance from the optical center of the camera to the detection surface; the camera clamping mechanism can be fixed at a specific height from the detection surface by fixing the camera through the fastening screw, and the height of the camera from the detection surface can also be adjusted; a camera clamp 2-3-4 is arranged above the cover 2-3-1 and used for clamping a monocular, binocular or multiocular camera 2-3-3, and the angle between the optical axis of the camera and the detection surface can be adjusted.
The light source lighting system 2-4 comprises four light sources, each light source inclines to 5-100 degrees towards the axis of the machine body, and the light sources are used for providing sufficient and uniform illumination for the detection area.
The bionic gecko crawling detection device further comprises a pulling and descending mechanism 1 used for pulling a bionic gecko crawling detection device main body 2, and the detection device main body 2 comprises a machine body 2-1, a walking bionic mechanism 2-2, a camera clamping mechanism 2-3 and a light source lighting system 2-4. The detection device main body 2 is stabilized on a certain height of the inclined dam in the transverse detection process without sliding down, and the pull-down mechanism 1 can enable the detection device main body 2 to be lifted to any height of the inclined dam. The pull-down mechanism 1 is installed on the upper edge of the inclined dam and pulls the detection device main body 2 by a cable.
The pulling and descending mechanism 1 comprises a long rail 1-1, a fixed pulley 1-2, a sliding seat 1-3 and a cable 1-4; the long rail 1-1 is arranged on the upper edge of the inclined dam, and the sliding seat 1-3 is arranged on the long rail 1-1 and can move on the long rail; the two fixed pulleys 1-2 are arranged on the sliding seat 1-3, and the rotating angular speeds of the two fixed pulleys are kept consistent; one end of the cable 1-4 is fixed at the bottom of the wheel groove of the fixed pulley 1-2 and pulled by the fixed pulley, and the other end of the cable 1-4 is tied at the side ear buckle 2-1-3 of the machine body 2-1. When the detection device is used, the cable is pulled by rotating the fixed pulley, so that the detection device main body is stabilized at any height of the surface of the inclined dam. The sliding seat can also be provided with electromechanical components such as a computer host, a motor, an air pump, an electromagnetic valve and the like so as to simplify the structure of the detection device main body.
Compared with the prior art, the invention has the following advantages and effects:
(1) The method can well detect the inclined dam cracks through the neural network and the binocular vision detection system, and has high detection precision, wherein the information comprises crack width, crack depth and the like.
(2) The device has high safety, and can enable detection personnel to be far away from dangerous inclined dam wall surfaces, and crack detection can be carried out on the whole inclined dam wall surface by operating the device at a safe place; can reduce the cost of labor, improve detection efficiency, simple structure, convenient guide is carried.
(3) The invention has wide detection range, and can switch the traveling wheel group in a lifting mode to detect the whole inclined dam wall surface; and the mesh number and the height of the distance detection surface of the cameras can be adjusted, so that the system can be suitable for various camera systems.
(4) The detection result of the invention has high stability, and the detection device can stably walk and detect on the inclined dam through the walking bionic mechanism.
Drawings
Fig. 1 is a schematic diagram of the operation of the present invention.
Fig. 2 is a schematic view of the pull down mechanism of the present invention.
FIG. 3 is a front view of the main body of the detecting unit of the present invention.
FIG. 4 is a plan view of the detecting unit main body of the present invention.
Fig. 5 is a schematic view of the walking bionic mechanism of the invention.
Fig. 6 is a top view of the fuselage of the present invention.
Fig. 7 is a block diagram of a u-net network of the present invention.
In the figure, 1, a pull-down mechanism; 1-1, long track; 1-2, a fixed pulley; 1-3, a slide seat; 1-4, cable;
2. a subject detection device;
2-1, a fuselage; 2-1-1, a hood; 2-1-2, an L-shaped section slide rail; 2-1-3, side ear buckles;
2-2, a walking bionic mechanism; 2-2-1, gecko toes; 2-2-2, gecko feet main body; 2-2-3, gecko feet and limbs;
2-3, a camera clamping mechanism; 2-3-1, a machine cover; 2-3-2, a slide block; 2-3-3, a camera; 2-3-4, clamping;
2-4, a light source lighting system.
Detailed Description
The present invention will be further described with reference to the accompanying drawings, but the embodiments of the invention are not limited thereto. In the description of the present invention, it is to be understood that, if there are terms indicating directions or positional relationships such as "up", "down", "left", "right", "horizontal", "vertical", etc., they are based on the orientations or positional relationships shown in the drawings and are only for convenience of describing the present invention or simplifying the description, but do not indicate or imply that the device or element referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and therefore, the terms describing the positional relationships in the drawings are only for illustrative purposes and are not to be construed as limiting the present patent.
Examples
As shown in fig. 7, the neural network is first trained. Firstly, preparing a training set and a testing set of a neural network, then training a U-net network by using TensorFlow, adjusting the initial learning rate to be 0.001, iterating for about 5000 times to reduce the loss rate to be below 0.4, and then testing the model accuracy by using the MIOU and the Dice to obtain the MIOU and Dice coefficients of 0.89 and 0.91 respectively, which indicates that the segmentation effect is good.
And detecting the dam crack image acquired in real time by adopting the trained u-net neural network, accurately segmenting the crack and background sundries in the image, calculating the depth and width of the crack in real time, and feeding the depth and width back to detection personnel.
The bionic gecko crawling detection device comprises a machine body 2-1, a walking bionic mechanism 2-2, a camera clamping mechanism 2-3 and a light source illumination system 2-4, and is used for detecting by adopting the dam slope crack detection method based on deep learning, as shown in figures 1-6; the walking bionic mechanism 2-2 is arranged on the periphery of the lower edge of the machine body and used for driving the bionic gecko crawling detection device to walk on the inclined dam; the camera clamping device 2-3 is arranged on an L-shaped section sliding rail 2-1-2 of the machine body and used for erecting a camera and adjusting the height of the camera from a detection surface; light source illumination systems 2-4 are mounted on top of the fuselage for providing sufficiently stable light source illumination to the monitored area.
When detecting, the method comprises the following steps:
(1) The long rail 1-1 of the pull-down mechanism 1 is arranged on the upper edge of the wall surface of the inclined dam, and the detection device main body 2 is pulled through the cable 1-4, so that the detection device main body 2 does not slide downwards. Adjusting the industrial camera 2-3-3 of the detection device main body 2 to a proper height from the detection surface;
(2) The detection device main body 2 walks on the inclined dam wall surface, the industrial camera 2-3-3 collects dam images in real time, when cracks are detected on the dam, the walking bionic mechanism 2-2 is started, the detection device is fixed on the inclined dam wall surface, information such as specific width and depth of the cracks is detected, and then the information is transmitted back to the computer in real time;
(3) When the detection device main body 2 needs to move to the other end of the wall surface of the inclined dam, setae on the feet 2-2-1 of the gecko are contracted to reduce viscosity, a steering engine on the feet 2-2-3 of the gecko in the walking bionic mechanism 2-2 turns to 90 degrees, then the gecko walks for a certain distance, and the steering engine is turned back through the feet 2-2-3 of the gecko, so that the whole inclined dam can be detected;
(4) In the process that the detection device body 2 traverses the wall surface of the inclined dam, the light source lighting system 2-4 is started to provide uniform and sufficient light sources for a detection area; and the camera 2-3-3 is also started to work to detect the cracks on the wall surface of the inclined dam.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (10)

1. A dam slope crack detection method is characterized by comprising the following steps:
(1) Training a neural network:
1-1, preparing a training set and a testing set of the neural network: acquiring a real-time image of a dam slope by using a CCD industrial camera, preprocessing the image, and then enhancing the image, including turning, translating or rotating, to obtain more than 1000 crack images; then using 60-80% of the images as a training set for network training, and using 20-40% of the images as a test set for network testing;
1-2, adjusting parameters of the neural network: the U-net neural network can adapt to the crack image by adjusting the parameters of the U-net neural network, so that cracks and background impurities are accurately segmented;
1-3, testing: testing the training result of the U-net neural network by using the test set obtained in the step 1-1, and evaluating by using a uniform cross-over ratio MIOU and a Dice coefficient;
(2) Detecting the dam crack image acquired in real time by adopting the trained u-net neural network, and accurately segmenting cracks and background sundries in the image:
2-1, calibrating a camera: aiming a camera at a known structure with a plurality of independent identifiable points to obtain a calibration image, and then solving internal parameters and external parameters of the camera through an internal function of OpenCV, wherein the internal parameters comprise a rotation matrix R and a translational vector T;
2-2, correcting the image: shooting by a left camera and a right camera to obtain real-time left and right images of the dam crack; then, correcting the real-time left and right images according to the rotation matrix R and the translational vector T obtained in the step 2-1 to obtain corrected left and right images;
2-3, accurately segmenting the corrected cracks and background sundries in the left and right images by adopting the trained u-net neural network to obtain crack characteristic information of the left and right images;
2-4, stereo matching: matching crack characteristic information of the left image and the right image, and analyzing the deviation between the characteristics, namely parallax error to obtain a parallax error image;
2-5, solving the depth of the crack: and converting the disparity map into a depth map, wherein the depth is as follows:
Figure FDA0004065116360000011
wherein depth represents depth information of the crack; f represents the normalized focal length; b is the distance between the optical centers of the two cameras, also known as the baseline distance; disp is the disparity value;
2-6, solving the width of the crack: the width H (a, B) is the maximum width of the crack in the image, and the formula is shown below:
H(A,B)＝max[h(A,B),h(B,A)]
Figure FDA0004065116360000012
Figure FDA0004065116360000021
wherein A is the upper edge point of the crack, and B is the lower edge point of the crack; h (A, B) is the one-way distance from point set A to point set B, and h (B, A) is the one-way distance from point set B to point set A.
2. The dam slope crack detection method of claim 1, wherein: in the step 1-2, the u-net network is trained by using the training set obtained in the step 1-1 and using TensorFlow, the learning rate is 0.01-0.001, the iteration is 2000-3000 times, the loss rate is iterated to be below 0.4, and the training of the network model is completed; the adjusted parameters comprise setting a loss function and adjusting an L2 regularization coefficient; the loss function L is as follows:
Figure FDA0004065116360000022
/>
wherein n represents the number of categories, and the value of n is 2, which represents two categories of complex background and crack; w is a weight of the object,
Figure FDA0004065116360000023
respectively representing the probability of the ith class and the probability of model prediction;
the weight w in the loss function is to update the weight of each pixel in the image, so that the neural network can specially learn the edge information of the crack; the weight w (x) after update, is as follows:
Figure FDA0004065116360000024
wherein b represents a deviation; d 1 Representing the nearest distance from the pixel point to the boundary on the crack image; d 2 And representing the nearest distance from the pixel point to the lower boundary of the crack image.
3. The dam slope crack detection method of claim 1, wherein: in step 2-4, the parallax refers to a difference value of the same features observed by the left and right cameras on the x coordinate, and the parallax value disp = x l -x r Wherein x is l Is the distance from a certain characteristic point of the crack in the left camera picture to the center of the optical axis, x r Is the distance from the same characteristic point of the right camera to the center of the optical axis of the right camera.
4. The utility model provides a bionical gecko detection device that crawls which characterized in that: the method comprises a machine body, a walking bionic mechanism, a camera clamping mechanism and a light source lighting system, and is used for detecting by adopting the dam slope crack detection method of any one of claims 1-3; the walking bionic mechanism is arranged on the periphery of the lower edge of the machine body and used for driving the bionic gecko crawling detection device to walk on the inclined dam; the camera clamping device is arranged on an L-shaped section slide rail of the machine body and used for erecting a camera and adjusting the height of the camera from a detection surface; the light source illumination system is mounted on the top of the body for providing sufficient and stable light source illumination to the monitored area.
5. The bionic gecko crawling detection device according to claim 4, wherein: the machine body comprises a machine cover, four sliding rails with L-shaped sections and two side ear buckles; the hollow part of the hood is a detection area; four sliding rails with L-shaped sections are vertically arranged on the upper part of the hood; two side ear buckles are respectively installed in aircraft bonnet middle part both sides for connect the cable.
6. The bionic gecko crawling detection device according to claim 4, wherein: the walking bionic mechanism comprises four bionic gecko feet which are respectively arranged on the periphery of the lower edge of the machine body; each bionic gecko foot comprises a gecko foot main body, gecko foot limbs and gecko toes; the gecko foot limb is provided with a steering engine which can steer; the bionic gecko foot is of a multi-fiber surface structure, more than one row of liners are arranged below each toe of the bionic gecko foot, more than one seta is arranged on each liner, and more than one seta small brush is arranged at the top end of each seta.
7. The bionic gecko crawling detection device according to claim 4, wherein: the camera clamping mechanism comprises a machine cover, a sliding block and a clamp; the four corners of the machine cover are provided with slide blocks which are used for moving on an L-shaped section slide rail of the machine body, and the L-shaped section slide rail is provided with scales which are used for calculating the distance from the optical center of the camera to the detection surface; the camera clamping mechanism can be fixed at a specific height from the detection surface by fixing the camera through the fastening screw, and the height of the camera from the detection surface can also be adjusted; the camera clamp is arranged above the cover and used for clamping a monocular camera, a binocular camera or a multi-eye camera, and the angle between the optical axis of the camera and the detection surface is adjustable.
8. The bionic gecko crawling detection device according to claim 4, wherein: the light source lighting system comprises four light sources, and each light source inclines towards the axis of the machine body by 5-10 degrees and is used for providing sufficient and uniform illumination for the detection area.
9. The bionic gecko crawling detection device according to claim 4, wherein: the bionic gecko crawling detection device comprises a pulling and descending mechanism and is used for pulling a bionic gecko crawling detection device main body, wherein the detection device main body comprises a machine body, a walking bionic mechanism, a camera clamping mechanism and a light source lighting system.
10. The bionic gecko crawling detection device according to claim 9, wherein: the pulling and descending mechanism comprises a long rail, a fixed pulley, a sliding seat and a cable; the long rail is arranged on the upper edge of the inclined dam, and the sliding seat is arranged on the long rail and can move on the long rail; the two fixed pulleys are arranged on the sliding seat, and the rotating angular speeds of the two fixed pulleys are kept consistent; one end of the cable is fixed at the bottom of the wheel groove of the fixed pulley and pulled by the fixed pulley, and the other end of the cable is tied on the side ear buckle of the machine body.
CN201910514001.0A 2019-06-14 2019-06-14 Dam slope crack detection method and bionic gecko crawling detection device Active CN110335255B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910514001.0A CN110335255B (en) 2019-06-14 2019-06-14 Dam slope crack detection method and bionic gecko crawling detection device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910514001.0A CN110335255B (en) 2019-06-14 2019-06-14 Dam slope crack detection method and bionic gecko crawling detection device

Publications (2)

Publication Number Publication Date
CN110335255A CN110335255A (en) 2019-10-15
CN110335255B true CN110335255B (en) 2023-04-18

Family

ID=68142065

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910514001.0A Active CN110335255B (en) 2019-06-14 2019-06-14 Dam slope crack detection method and bionic gecko crawling detection device

Country Status (1)

Country Link
CN (1) CN110335255B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111105411B (en) * 2019-12-30 2023-06-23 创新奇智(青岛)科技有限公司 Magnetic shoe surface defect detection method
CN111294514A (en) * 2020-02-11 2020-06-16 天津大学 Data set acquisition system for image deep learning
CN111751380B (en) * 2020-07-08 2021-08-31 中国水利水电科学研究院 Concrete dam crack inspection method based on light and small unmanned aerial vehicle
CN115082450A (en) * 2022-07-26 2022-09-20 北京中科慧眼科技有限公司 Pavement crack detection method and system based on deep learning network
CN116309447B (en) * 2023-03-17 2024-01-05 水利部交通运输部国家能源局南京水利科学研究院 Dam slope crack detection method based on deep learning

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106548182A (en) * 2016-11-02 2017-03-29 武汉理工大学 Based on deep learning and the causal analytic pavement distress survey method and device of master
WO2018000731A1 (en) * 2016-06-28 2018-01-04 华南理工大学 Method for automatically detecting curved surface defect and device thereof

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018000731A1 (en) * 2016-06-28 2018-01-04 华南理工大学 Method for automatically detecting curved surface defect and device thereof
CN106548182A (en) * 2016-11-02 2017-03-29 武汉理工大学 Based on deep learning and the causal analytic pavement distress survey method and device of master

Also Published As

Publication number Publication date
CN110335255A (en) 2019-10-15

Similar Documents

Publication Publication Date Title
CN110335255B (en) Dam slope crack detection method and bionic gecko crawling detection device
CN105225482B (en) Vehicle detecting system and method based on binocular stereo vision
CN103077386B (en) A kind of video flowing iris image quality detection method of cascade
CN104112370B (en) Parking lot based on monitoring image intelligent car position recognition methods and system
CN106248686A (en) Glass surface defects based on machine vision detection device and method
Martinez-de Dios et al. Computer vision and robotics techniques in fish farms
US20070071311A1 (en) Method for processing stereo vision data using image density
CN105346706A (en) Flight device, and flight control system and method
CN103702015B (en) Exposure control method for human face image acquisition system under near-infrared condition
CN110458157B (en) Intelligent monitoring system for power cable production process
CN112733950A (en) Power equipment fault diagnosis method based on combination of image fusion and target detection
CN103630091B (en) Leaf area measurement method based on laser and image processing techniques
CN106485751B (en) Unmanned aerial vehicle photographic imaging and data processing method and system applied to foundation pile detection
CN110246151B (en) Underwater robot target tracking method based on deep learning and monocular vision
CN106778713A (en) The iris identification device and method of a kind of dynamic human eye tracking
CN111354077B (en) Binocular vision-based three-dimensional face reconstruction method
CN102750584B (en) On-corncob corn niblet counting method
CN106228569A (en) A kind of fish speed of moving body detection method being applicable to water quality monitoring
CN103220852A (en) Control system and control method of light-emitting diode (LED) illumination
CN204039886U (en) A kind of pavement damage crack detection system based on multiple stage camera stereoscopic shooting
CN109028232A (en) A kind of band moves the kitchen ventilator and oil smoke concentration detection method of vision detection system
CN106709529A (en) Visual detection method for color difference classification of photovoltaic cells
CN105043351A (en) Biological robot-based miniature wireless active omni-directional vision sensor
CN107144755A (en) A kind of charge coupling device charge transfer effciency universal testing method based on hot spot
CN108288273B (en) Automatic detection method for abnormal targets of railway contact network based on multi-scale coupling convolution network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant