CN113700978A

CN113700978A - Pipeline foreign matter detection device and detection method

Info

Publication number: CN113700978A
Application number: CN202111078516.4A
Authority: CN
Inventors: 郎立国; 康涛; 李旭; 杨天兵; 孙丹
Original assignee: Avic East China Photoelectric Shanghai Co ltd
Current assignee: Avic East China Photoelectric Shanghai Co ltd
Priority date: 2021-09-15
Filing date: 2021-09-15
Publication date: 2021-11-26
Also published as: WO2023040104A1

Abstract

The invention relates to the technical field of pipeline detection, in particular to a pipeline foreign matter detection device and a detection method, wherein the pipeline foreign matter detection device comprises: detecting a vehicle; a control member located within the inspection vehicle; one end of the mechanical arm is connected with the detection vehicle; the pipeline foreign matter detection device and the detection method can effectively and quickly detect the metal foreign matters in the pipeline, reduce the influence of poor environment during pipeline detection and the inevitable problems of negligence, danger, irretraceability and the like of manual detection, and provide convenience for workers.

Description

Pipeline foreign matter detection device and detection method

Technical Field

The invention relates to the technical field of pipeline detection, in particular to a pipeline foreign matter detection device and a detection method.

Background

The pipeline foreign matter refers to certain foreign matters, debris or objects which are left in the pipeline and affect the pipeline, such as metal tools, scattered screws, nuts, gaskets, fuses and the like, and the detection of the pipeline foreign matter is always a great problem as to how to effectively and quickly detect the pipeline foreign matter due to the fact that metal materials are more.

At present, the detection of pipelines at home and abroad mainly depends on manpower, and the problems of poor pipeline detection environment, inevitable negligence, dangerousness, irretraceability and the like exist, so that aiming at the current situation, the development of a pipeline foreign matter detection device and a detection method is urgently needed to overcome the defects in the current practical application.

Disclosure of Invention

The present invention is directed to a device and a method for detecting foreign matters in a pipeline, so as to solve the problems of the background art.

In order to achieve the purpose, the invention provides the following technical scheme:

a pipe foreign matter detection device, comprising:

detecting a vehicle;

a control member located within the inspection vehicle;

one end of the mechanical arm is connected with the detection vehicle;

and the image acquisition part is connected with the other end of the mechanical arm.

A method for detecting foreign matters in a pipeline, applied to the device for detecting foreign matters in a pipeline, comprising the steps of:

1) the detection vehicle sets a distance D according to the set operation;

2) the mechanical arm runs to a certain appointed waypoint;

3) a camera collects images;

4) after the image is collected, identifying foreign matters;

5) if the foreign body is detected, carrying out foreign body prompt;

6) continuing to step 2) until the foreign matter of the current section is identified, and continuing to execute step 1);

7) after whole pipeline detection accomplished, the detection car withdraws from the pipeline, accomplishes and detects, can bring the foreign matter into in order to prevent to detect the car, omits in the pipeline, can carry out the foreign matter once more when detecting the car and moving back and detect.

Compared with the prior art, the invention has the beneficial effects that:

when the foreign matter detection work is required, the foreign matter detection device can move in the pipeline through the arranged detection vehicle. Wherein the control part is the main control of the whole system, controls the detection vehicle to carry the mechanical arm to move back and forth in the pipeline, circularly moves the mechanical arm to different pre-calculated pose points, then the image acquisition part acquires images, then the control part identifies the foreign matters of the acquired images by an image processing method, after identification, the foreign matters are prompted to be detected in the forms of voice, images and the like, wherein, the foreign body detection algorithm adopts a deep learning method based on YoloV3 to complete the detection of the foreign bodies in the pipeline, has small influence by illumination and good identification effect, and by the reciprocating motion of the detection vehicle in the pipeline, the metal foreign bodies in the pipeline can be effectively and quickly detected, the problems of poor environment influence and inevitable negligence, danger, incapability of being traced and the like in manual detection during pipeline detection are reduced, convenience is provided for workers, and the pipeline detection device is worthy of popularization.

Drawings

Fig. 1 is a schematic overall structure diagram in the embodiment of the present invention.

Fig. 2 is a schematic flow chart of foreign object detection according to an embodiment of the present invention.

Fig. 3 is a schematic diagram illustrating the identification and location principle of YoloV3 in the embodiment of the present invention.

Fig. 4 is a schematic structural diagram of a YoloV3 foreign object detection model in the embodiment of the present invention.

Fig. 5 is a schematic diagram of a structural diagram of a conditional layer in the embodiment of the present invention.

Fig. 6 is a schematic structural diagram of a Residual layer in the embodiment of the present invention.

FIG. 7 is a diagram of a target bounding box in an embodiment of the invention.

Fig. 8 is a flowchart illustrating the foreign object identification and location according to an embodiment of the present invention.

Fig. 9 is a schematic diagram of foreign object identification and location in an embodiment of the invention.

Fig. 10 is a diagram illustrating an actual foreign object detection result according to an embodiment of the present invention.

In the figure: the method comprises the following steps of 1-detecting vehicle, 2-mechanical arm, 3-image acquisition part, 4-control part, 5-pipeline, 6-driving wheel and 7-control module.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Specific implementations of the present invention are described in detail below with reference to specific embodiments.

Referring to fig. 1, a pipeline foreign matter detection device according to an embodiment of the present invention includes:

a detection vehicle 1;

a control part 4, wherein the control part 4 is positioned in the detection vehicle 1;

one end of the mechanical arm 2 is connected with the detection vehicle 1;

and the image acquisition part 3 is connected with the other end of the mechanical arm 2.

When foreign matter detection work is required, the detection vehicle 1 can move in the pipeline 5, the control piece 4 is the master control of the whole system, the detection vehicle 1 is controlled to carry the mechanical arm 2 to move back and forth in the pipeline 5, the mechanical arm 2 circularly moves to different pre-calculated pose points, then the image acquisition piece 3 carries out image acquisition, then the control piece 4 identifies the foreign matter in the acquired image by an image processing method, and after identification, the foreign matter is prompted to be detected in the forms of sound, images and the like, wherein a foreign matter detection algorithm adopts a deep learning method based on Yolov3 to complete detection of the foreign matter in the pipeline, the influence of illumination is small, the identification effect is good, the detection vehicle 1 can effectively and quickly detect the metal foreign matter in the pipeline 5 by reciprocating in the pipeline 5, the influence of environment difference and inevitable manual detection during detection of the pipeline 5 are reduced, and the influence of negligence of environment difference and the inevitable manual detection are reduced, The problems of danger, irreproducibility and the like are solved, convenience is provided for workers, and the method is worthy of popularization.

In an embodiment of the present invention, referring to fig. 1, the running speed of the detection vehicle 1 is less than 50mm/s, the self weight is less than 60Kg, and the load is less than 50Kg, a driving wheel 6 is fixedly installed on the detection vehicle 1, a low-voltage hub motor is installed in the driving wheel 6, and a control module 7 is also installed on the detection vehicle 1.

During detection, in order to ensure complete detection, the running speed of the detection vehicle cannot be too high, the detection vehicle cannot slide during running, the steering control is simple, the volume of the whole vehicle is as small as possible, the detection vehicle 1 is driven in a four-wheel differential drive mode, the driving wheel 6 is independently driven by a low-voltage hub motor, the low-speed high-torque motor has the characteristics of low speed and capability of saving mechanical installation space, the motor adopts a four-way incremental grating encoder as position and speed feedback, the control module 7 comprises a main controller, a laser radar and an IMU (Inertial Measurement Unit), the robot platform control system adopts a high-performance industrial personal computer as the main controller, motion control is carried out through a four-way CAN interface, the laser radar acquires environmental information, the IMU measures the posture of the robot mobile platform in real time, the IMU information is fused with the laser radar and a laser ranging module, and judges the state and the environment of the robot in real time, ensure the stability and safety of the platform.

In one embodiment of the present invention, referring to fig. 1, the robot arm 2 is a six-axis cooperative robot arm, and the running arm of the robot arm 2 has a span of 615mm, a dead weight of 15kg, a load of 3kg, and a 10-step force protection function.

In the detection process, the mechanical arm 2 is required to cover the working range and be as short as possible because the longer the arm span is, the more easily the mechanical arm can touch the inner wall of the pipeline, the working range of the mechanical arm 2 is simulated by calculating the load of the mechanical arm 2, the arm span is adopted to be 615mm, the dead weight is 15Kg, the load is 3Kg, and the six-axis cooperative mechanical arm with 10-level force protection function can timely stop when the mechanical arm collides against the pipe wall, so that the pipe wall can be prevented from being damaged.

In an embodiment of the present invention, referring to fig. 1, the image capturing element 3 is an industrial camera, the industrial camera is a surface scanning camera, a lens is mounted on the surface scanning camera, and a polarizer is mounted on the lens.

During detection, the quality of the acquired image has direct influence on the design of the algorithm and whether the final algorithm is effective or not, and even the success or failure of the invention project is determined, so that the quality of image acquisition is improved to the greatest extent through the model selection of the camera and the light source design. Considering that the depth image processing algorithm requires a color camera, the final selected camera parameters are as follows:

a) resolution ratio: 2448 (H). times.2048 (V)

b) And the size of the sensor: 2/3"

c) And pixel size: 3.45 μm x 3.45.45 μm

Frame rate: 20fps

A lens interface: c port

The main parameters of the camera lens are as follows:

focal length: 6mm

Aperture range: F1.4-F16

Interface: c-mount

Induction size: 1"

Because the inner wall of the pipeline is made of metal and the surfaces of the inner wall of the pipeline are provided with radians, in order to avoid reflected light generated by the inner wall of the pipeline from directly entering a camera, a square light source is finally adopted for side projection, and the light source is projected onto the wall surface of the pipeline at a certain angle; and a polarizing plate is installed on the lens to further prevent reflection.

In an embodiment of the present invention, referring to fig. 1, the control unit 4 is a computer, a CPU of the computer adopts an i 76 core processor, a memory is 24G, a hard disk is 2TB, and a graphics card adopts NVIDIA RTX 2080.

In the detection process, the detection and identification of foreign matters are completed on the acquired image by an image processing method, so that the computer needs to perform deep learning image processing and store the pipeline foreign matter detection process image, and a CPU (central processing unit) is selected as an i 76 core and above processor, a memory is selected to be more than 24G, a hard disk is selected to be more than 2TB, and a video card is a high-performance computer with NVIDIA RTX 2080 and above.

Based on the structure given in the above embodiment, a detection method of a pipeline foreign matter detection device is given herein, the method including the steps of:

1) the detection vehicle sets a distance D according to the set operation;

2) the mechanical arm runs to a certain appointed waypoint;

3) a camera collects images;

4) after the image is collected, identifying foreign matters;

5) if the foreign body is detected, carrying out foreign body prompt;

In an embodiment of the present invention, referring to fig. 2-10, in step 4, the foreign object detection may be completed under different illumination conditions, the foreign objects are mainly some metal objects, and are completed through yoolov 3 foreign object detection, and the yoolov 3 algorithm uses a single CNN model to achieve end-to-end target detection, and directly predicts the type and position of the target for the input image.

The yoloV3 directly predicts the types and positions of different targets by using only one CNN network, has simple structure and high speed, the processing speed can reach 29 frames/second, the real-time processing can be realized, and the yoloV3 supports the detection of 3 different scales, and is excellent for the detection of small targets.

The CNN network of YoloV3 divides the input picture into S × S meshes (actually, S × S meshes are obtained by convolution downsampling, and for convenience of explanation, the input picture is divided into S × S meshes), and each cell is responsible for detecting the objects whose center points fall within the cell, as shown in FIG. 6, it can be seen that the center of the hexagon falls in the figureWithin the orange cell, then the cell is responsible for predicting this hexagon. Each cell predicts the predicted values (t) of B (YoloV3 obtains 3 groups of prior frames through a clustering algorithm, namely 3 groups of default preset boundary frames obtained through pre-training are called anchor points, so that the value of B is 3) preset boundary frames_x,t_y,t_w,t_h,p_O,p₁,p₂,-…-,p_c) Wherein (t)_x,t_y,t_w,t_h) To predict the size and position of the bounding box (actually the center offset and the aspect scaling ratio), p_OTo predict the probability that an object is contained within the object bounding box, (p)₁,p₂,…,p_c) The probability of c object classes corresponding to the bounding box is predicted.

Each cell needs to predict a (B × (5+ C)) value. If the input picture is divided into an S × S grid, the final prediction value is a tensor of size S × S × (B × (5+ C)).

Because the size of the target object is large or small, in order to better identify and position the large-size and small-size imaging targets, YoloV3 predicts on three scale levels by respectively down-sampling the size of the

input image

32, 16, 8, i.e. dividing the picture into three grids, if the size of the input image is 416 × 416, three kinds of feature maps, namely three kinds of grids of S × S, respectively 13 × 13 grids, 26 × 26 grids, and 52 × 52 grids, wherein the 13 × 13 grids can identify the targets with larger imaging sizes, the 26 × 26 grids can identify the targets with medium imaging sizes, and the 52 × 52 grids can identify the targets with smaller imaging sizes.

In the detection work, it is assumed that the target to be measured is 4 types of scattered screws, nuts, gaskets and fuses and a wall surface background type, so the target type is 5 types (4+1), that is, c is 5, the tensor with the 13 × 13 grid predicted value of 13 × 13 × (3 × (5+5)) size, the tensor with the 26 × 26 grid predicted value of 13 × 13 × (3 × (5+5)) size, and the tensor with the 52 × 52 grid predicted value of 52 × 52 × (3 × (5+5)) size, and finally yolo v3 analyzes all the predicted value probabilities to obtain the position information and the size information of the final target.

YoloV3 adopts Darknet-53 network structure, the model structure is shown in figure 4, the network is mainly composed of a series of Convolitional layers of 1x1 and 3x3 and Residual layers. Wherein, the conditional layer is a Darknet-53 network basic unit, and is composed of a conv convolution layer, a BN layer and a LeakyReLU layer, and the structure of the conditional layer is shown in FIG. 5. The Residual layer is a dark net-53 Residual module, the structure of which is shown in fig. 6, and here, the benefit of the structure using the Residual is as follows: (1) a key point of the depth model is whether the network structure can be normally converged, and the use of a residual structure can ensure that the network structure can still be converged under the condition of very deep. (2) The deeper the network, the better the expressed characteristics, and the better the target identification and positioning effect can be improved.

As can be seen from fig. 7, at the 86 th layer, the 61 st layer and the 85 th layer are tensor-spliced, and at the 98 th layer, the 36 th layer and the 97 th layer are tensor-spliced, wherein the 61 st layer and the 36 th layer are shallow features, and the 85 th layer and the 97 th layer are deep features, and the deep features and the shallow features are simultaneously utilized, so that the effect of the network is further improved. The network finally predicts three kinds of feature maps, namely a 13 × 13 grid feature map, a 26 × 26 grid feature map and a 52 × 52 grid feature map, and only one target needs to be identified here, so that the prediction tensor size of the 13 × 13 grid feature map is 13 × 13 × (3 × (5+5)), the prediction tensor size of the 26 × 26 grid feature map is 26 × 26 × (3 × (5+5)), and the prediction tensor size of the 52 × 52 grid feature map is 52 × 52 × (3 × (5+ 5)). And then obtaining a final predicted value according to the maximum probability value.

YoloV3 to obtain the final predicted value (t)_x,t_y,t_w,t_h,p_O,p,p,-…-,-p_c) Then, because of the (t) obtained_x,t_y,t_w,t_h) The actual predicted bounding box center offset for the net and the aspect scaling ratio, the target bounding box also needs to be calculated. The calculation principle is as shown in fig. 7, in which a dotted-line rectangular frame is a preset boundary frame, i.e., an anchor point, and a solid-line rectangular frame is a predicted target boundary frame obtained by calculating the offset of the network prediction. Wherein- (p)_w,p_h) To preset the width and height of the bounding box on the feature map, (t)_x,t_y,t_w,t_h) Predicted bounding box center offsets for the network, respectivelyAnd aspect ratio, (b)_x，b_y,b_w,b_h) Is the final predicted target bounding box.

The foreign object identification and positioning process of YoloV3 is divided into two parts, namely a training model and target identification and positioning, and the process is shown in fig. 8.

Training a model: the method comprises the steps of collecting target images to be detected in advance (in order to improve the recognition effect, the target images with different illumination conditions, different background conditions, different angles and different distances are collected as much as possible, and the number of the collected images is preferably not less than 10000), and then training to obtain trained model characteristics.

Target identification and positioning: firstly, reading in a corrected image, converting the image resolution into 416x416, then reading the model characteristics, identifying and positioning the target, and finally obtaining the type and the position of the target.

The position information and the width and height information of the target image in the original image can be obtained through the algorithm by utilizing the YoloV3 positioning and identifying algorithm to identify and position the target and inputting the original image. Positioning effect map As shown in FIG. 9, the target of INPUT image INPUT _ IMG is identified and positioned, and the position (x) of the target TAG _ IMG in the original image INPUT _ IMG is identified and positioned_lta y_lta) Width of w_ltaHeight of h_ltaThe actual foreign object recognition result is shown in fig. 10.

In an embodiment of the present invention, in step 5, after the foreign object is detected and identified, the position of the foreign object in the pipeline is determined by combining the position of the detection vehicle, the foreign object information is prompted to the operation user, and the original image and the identification result are stored, so as to facilitate the subsequent backtracking and provide convenience for the user.

It should be noted that, in the present invention, unless otherwise specifically stated or limited, the terms "sliding", "rotating", "fixing", "providing", and the like are to be understood in a broad sense, and may be, for example, a welded connection, a bolted connection, or an integral body; can be mechanically or electrically connected; they may be directly connected or indirectly connected through intervening media, or they may be connected internally or in any other suitable relationship, unless expressly stated otherwise. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.

Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.

Claims

1. A pipeline foreign matter detection device, characterized in that, pipeline foreign matter detection device includes:

detecting a vehicle;

a control member located within the inspection vehicle;

one end of the mechanical arm is connected with the detection vehicle;

2. The pipeline foreign matter detection device according to claim 1, wherein the running speed of the detection vehicle is less than 50mm/s, the self weight is less than 60Kg, the load is less than 50Kg, a driving wheel is fixedly mounted on the detection vehicle, a low-voltage hub motor is arranged in the driving wheel, and a control module is further mounted on the detection vehicle.

3. The pipeline foreign matter detection device according to claim 2, wherein the robot arm is a six-axis cooperative robot arm, and a running arm of the robot arm has a spread of 615mm, a dead weight of 15kg, a load of 3kg, and has a 10-stage force protection function.

4. The pipeline foreign matter detection device according to any one of claims 1 to 3, wherein the image capturing member is an industrial camera, the industrial camera is a face scan camera, a lens is mounted on the face scan camera, and a polarizing plate is mounted on the lens.

5. The pipeline foreign matter detection device according to claim 4, wherein the control part is a computer, a CPU of the computer adopts an i 76 core processor, a memory is 24G, a hard disk is 2TB, and a graphics card adopts NVIDIA RTX 2080.

6. A method for detecting foreign matters in a pipeline, which is applied to the apparatus according to any one of claims 1 to 5, comprising the steps of:

1) the detection vehicle sets a distance D according to the set operation;

2) the mechanical arm runs to a certain appointed waypoint;

3) a camera collects images;

4) after the image is collected, identifying foreign matters;

5) if the foreign body is detected, carrying out foreign body prompt;

7. The pipeline foreign matter detection method according to claim 6, wherein in step 4, the foreign matter detection can be completed under different illumination conditions, the foreign matter is some metal objects and is completed through yoolov 3 foreign matter detection, and the yoolov 3 algorithm adopts a single CNN model to realize end-to-end target detection, and directly predicts the type and position of the target on the input image.

8. The method for detecting foreign matters in pipelines according to claim 7, wherein in the step 5, after the foreign matters are detected and identified, the position of the foreign matters in the pipelines is determined by combining the position of the detection vehicle, foreign matter information is prompted to an operation user, and an original image and an identification result are saved.