LU500120B1

LU500120B1 - Multi-arm robot for automatic tunnel maintenance and control method thereof

Info

Publication number: LU500120B1
Application number: LU500120A
Authority: LU
Inventors: Qi Jiang
Original assignee: Univ Shandong
Priority date: 2020-09-11
Filing date: 2021-04-30
Publication date: 2021-11-09
Also published as: CN112282787A; CN112282787B

Abstract

The present disclosure proposes a multi-arm robot for automatic tunnel maintenance, and a control method thereof. The robot includes a walking mechanism, a fixed chassis arranged on the walking mechanism, and a plurality of multi-degree-of-freedom mechanical arms and a main control platform arranged on the fixed chassis, wherein the tail end of the multi-degree-of-freedom mechanical arm is equipped with an image detection device and a maintenance tool fixing groove, and the main control platform is respectively in communication connection with the multi-degree-of-freedom mechanical arm, the image detection device and a maintenance tool arranged in the maintenance tool fixing groove. In the present disclosure, the plurality of multi-degree-of-freedom mechanical arms can be arranged to realize multi-directional simultaneous operation of the robot in the present embodiment, thereby improving the maintenance efficiency of tunnel maintenance equipment. The present disclosure uses big data processing and machine learning algorithms to realize automatic task planning and mechanical arm trajectory planning, and can intelligently realize two functions of automatic tunnel detection and maintenance.

Description

MULTI-ARM ROBOT FOR AUTOMATIC TUNNEL MAINTENANCE AND CONTROL

METHOD THEREOF Field of the Invention The present disclosure relates to the related technical field of tunnel robots, in particular to a multi-arm robot for automatic tunnel maintenance and a control method thereof. Background of the Invention The statements in this section merely provide background art information related to the present disclosure, and do not necessarily constitute the prior art. At present, a large number of rails in China have entered the operation and maintenance stage. However, the narrow space of rail transit tunnels, dense mechanical and electrical equipment and short skylight maintenance time have resulted in low tunnel maintenance efficiency, poor effect and difficult disease treatment, resulting in shortened service life and even threatening traffic operation safety. In view of the complex tunnel environment and the treatment of multiple types of diseases, the comprehensive maintenance operation equipment needs to have the functions of drilling, grouting, grooving and manned operations, which have the difficult problems in development, integration and testing. The inventor finds that, in response to the needs of fast, intelligent and precise maintenance, in the treatment of water leakage disease in tunnel maintenance, general equipment is generally used in China for punching and grouting, which has a single function and lacks the research and development of corresponding special equipment. The existing tunnel maintenance equipment in China has a low degree of automation, most tasks require manual leadership, and each kind of equipment responds to a relatively single task scenario, so that the degree of integration 1s low, and the common drilling, grooving, grouting and manned multi-task operation requirements in maintenance tasks cannot be achieved. Summary of the Invention In order to solve the above problems, the present disclosure proposes a multi-arm robot for automatic tunnel maintenance and a control method thereof, so as to realize the automation of tunnel maintenance work.

In order to achieve the above purpose, the present disclosure adopts the following technical solutions: One or more embodiments provide a multi-arm robot for automatic tunnel maintenance, including a walking mechanism, a fixed chassis arranged on the walking mechanism, and a plurality of multi-degree-of-freedom mechanical arms and a main control platform arranged on the fixed chassis, wherein the tail end of the multi-degree-of-freedom mechanical arm is equipped with an image detection device and a maintenance tool fixing groove, and the main control platform 1s respectively in communication connection with the multi-degree-of-freedom mechanical arm, the image detection device and a maintenance tool arranged in the maintenance tool fixing groove.

One or more embodiments provide a control method of the multi-arm robot for automatic tunnel maintenance, including an automatic task planning method, a mechanical arm trajectory planning method, and a man-machine interaction working method; the automatic task planning method includes the following steps: constructing a corresponding relationship model of tunnel defect conditions and maintenance operation procedures by adopting the Apriori correlation analysis algorithm; obtaining defect condition data of a tunnel; and inputting the defect condition data into the corresponding relationship model of tunnel defect conditions and maintenance operation procedures, and outputting a tunnel maintenance operation scheme.

Compared with the prior art, the present disclosure has the following beneficial effects: (1) In the present disclosure, the plurality of multi-degree-of-freedom mechanical arms can be arranged to realize multi-directional simultaneous operation of the robot in the present embodiment, thereby improving the maintenance efficiency of tunnel maintenance equipment.

(2) The present disclosure uses big data processing and machine learning algorithms to realize automatic task planning and mechanical arm trajectory planning, and can intelligently realize two functions of automatic tunnel detection and maintenance.

(3) The present disclosure is equipped with a man-machine interaction device on a fixed bottom plate for the onsite complex tunnel maintenance work scenario, and a worker can conveniently perform manual control and can observe the location of the tunnel disease at a close range for treatment.

Brief Description of the Drawings The drawings of the specification, forming a part of the present disclosure, are used to provide a further understanding of the present disclosure.

The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure, and do not constitute an improper limitation of the present disclosure.

Fig. 1 is a front view of a multi-arm robot according to an embodiment of the present disclosure; Fig. 2 is a top view of the multi-arm robot according to the embodiment of the present disclosure; Fig. 3 is a flow diagram of an automatic task planning method according to an embodiment of the present disclosure; Fig. 4 is a flow diagram of a mechanical arm trajectory planning method according to an embodiment of the present disclosure.

Detailed Description of Embodiments The present disclosure will be further described below in conjunction with the drawings and embodiments.

It should be pointed out that the following detailed descriptions are all illustrative and are intended to provide further explanations for the present disclosure.

Unless otherwise specified, all technical and scientific terms used herein have the same meanings as commonly understood by those of ordinary skill in the technical field to which the present disclosure belongs.

It should be noted that the terms used herein are only used for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present disclosure.

As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form.

In addition, it should also be understood that when the terms "comprising" and/or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and/or combinations thereof.

It should be noted that various embodiments in the present disclosure and the features in the embodiments can be combined with each other if there is no conflict.

The embodiments will be described in detail below in conjunction with the drawings.

Inthe technical solutions disclosed in one or more embodiments, as shown in Figs. 1-2, a multi-arm robot for automatic tunnel maintenance includes a walking mechanism, a fixed chassis arranged on the walking mechanism, and a plurality of multi-degree-of-freedom mechanical arms and a main control platform arranged on the fixed chassis, wherein the tail end of the multi-degree-of-freedom mechanical arm is equipped with an image detection device and a maintenance tool fixing groove, and the main control platform is respectively in communication connection with the multi-degree-of-freedom mechanical arm, the image detection device and a maintenance tool arranged in the maintenance tool fixing groove.

In the present embodiment, by arranging the plurality of multi-degree-of-freedom mechanical arms, the multi-directional simultaneous operation of the robot in the present embodiment can be realized, and the maintenance efficiency of tunnel maintenance equipment can be improved.

Optionally, the image detection device can be arranged on each multi-degree-of-freedom mechanical arm, and the image detection device can adopt a non-contact image detection device, which is convenient for disease detection in the automatic maintenance process and the own location positioning of the multi-degree-of-freedom mechanical arm.

Optionally, the image detection device includes a CCD area-array camera, an infrared imager, a laser scanner, and an information transmission system, and the CCD area-array camera, the infrared imager and the laser scanner respectively transmit collected information to the control platform through the information transmission system.

The CCD area-array camera has high sensitivity, good information quality, high resolution and stable camera, which ensures the complete collection of information on tunnel lining cracks and reduces the difficulty of image stitching. It can adapt to the complex environment, insufficient illumination, uneven tunnel lining background in the tunnel, and adapt to poor shooting environment and other use environments.

The infrared imager, by detecting the temperature distribution of the tunnel, analyzes the information of water leakage on the surface of the tunnel. Since the temperature of an object is related to its infrared radiation energy, the infrared imager can convert infrared radiation and measure the temperature of the object in a non-contact manner.

Point data on the surface of the tunnel collected by laser scanning must be converted to solve an inverse kinematics model. In order to facilitate the conversion between a radar coordinate system and a mechanical arm coordinate system, a radar is calibrated before use. When in use, the angle of the radar is fine tuned by rotating, and trajectory planning is performed on the tail end of the mechanical arm based on the main body of tunnel point cloud to obtain a series of trajectory points running at the tail end, including the position and orientation to be reached.

In some embodiments, the maintenance tool includes a cutter, a drilling machine or/and a grouting pipe, and the like, different maintenance tools are disposed, and a fixing device that matches the 5 maintenance tool is selected to fix the maintenance tool in the maintenance tool fixing groove at the tail end of the mechanical arm.

The multi-degree-of-freedom mechanical arms at least include a first mechanical arm, a second mechanical arm and a third mechanical arm, which are connected in sequence, and an axial rotation joint and a pitching joint are arranged at the joint between the mechanical arms. The axial rotation Joint is used for realizing a 360-degree axial rotation function of the tail end of the mechanical arm, which 1s convenient for the maintenance work.

It can be realized that the multi-degree-of-freedom mechanical arms are arranged in pairs on the fixed chassis, and the control platform controls the coordinated work of the multi-degree-of-freedom mechanical arms arranged in pairs. Multi-directional simultaneous construction can be realized. Two mechanical arms in a single direction can perform different tasks independently, and multiple arms can also work together, thereby improving the efficiency and flexibility of the maintenance work.

In some embodiments, the fixed chassis includes an upper supporting surface and a lower supporting surface, a spiral lifting platform is arranged between the upper supporting surface and the lower supporting surface, the control platform and the multi-degree-of-freedom mechanical arms are arranged on the upper supporting surface, and the spiral lifting platform can provide axial orientation rotation to achieve height adjustment.

Optionally, a supporting bracket is arranged on the lower surface of the lower supporting surface, which can be used for supporting the system to work and can also be used for fixing on the walking mechanism.

The control platform can realize two working modes: automatic maintenance and man-machine interaction auxiliary operation.

In order to realize manual operation, it can be realized that the control platform can also be equipped with a man-machine interaction device, the man-machine interaction device includes a display device and a force feedback teleoperation master manipulator, and the display device and the force feedback teleoperation master manipulator are respectively in communication connection with the control platform.

When the disease condition is too complicated to make independent decision and maintenance, a constructor performs operation through the man-machine interaction device.

The main control platform is equipped with a force feedback teleoperation master manipulator for sensing a contact state of the mechanical arm during maintenance operations, so that the worker can adjust the working strategy in time.

Based on the above robot, the present embodiment further provides a control method of the multi-arm robot for automatic tunnel maintenance.

The method can be implemented in a control platform, and includes an automatic task planning method, a mechanical arm trajectory planning method, and a man-machine interaction working method.

The automatic task planning method can include the following steps: modelling the working procedures of skilled operators during an automatic maintenance process by using big data processing and machine learning algorithms, and providing professional AI decision-making to perform the automatic maintenance task planning method, specifically: step 101, constructing a corresponding relationship model of tunnel defect conditions and maintenance operation procedures by adopting the Apriori correlation analysis algorithm; step 102, obtaining defect condition data of a tunnel; specifically, tunnel image collection can be performed by an image collection device, and tunnel defects are obtained through image processing; and step 103, inputting the defect condition data into the corresponding relationship model of tunnel defect conditions and maintenance operation procedures, and outputting a tunnel maintenance operation scheme.

In step 101, the method of constructing the corresponding relationship model of tunnel defect conditions and maintenance operation procedures by adopting the Apriori correlation analysis algorithm specifically includes: step 1011, obtaining historical data of tunnel maintenance, wherein the historical data include historical defect data of the tunnel and a historical maintenance scheme corresponding to the historical defect data; optionally, the historical data can be obtained from various expert knowledge bases, forums,

technical reports and other databases; and step 1012, calculating the similarity and correlation between the disease condition and related knowledge of the maintenance scheme by using the Apriori correlation analysis algorithm, removing redundant knowledge, and finally forming a knowledge network, that is, the corresponding relationship model of tunnel defect conditions and maintenance operation procedures.

Specifically, the redundant knowledge is judged by the magnitude of the calculated correlation, a correlation threshold can be set, and when the calculated correlation is lower than the set threshold, it is judged as the redundant knowledge.

The corresponding relationship model of tunnel defect conditions and maintenance operation procedures in the present embodiment can perform self-learning based on the tunnel maintenance data, can closely follow the development of the modern tunnel disease diagnosis and maintenance technology, and strengthen the ability to diagnose and deal with sudden diseases.

In step 102, the obtaining defect condition data of the tunnel specifically includes: the defect condition data can be collected by performing tunnel image collection via the image collection device and obtaining tunnel defect and evaluation result data through image processing. Specifically, it can be as follows: step 1021, performing image processing on images collected by the CCD area-array camera to obtain tunnel disease and crack information; the image processing process can be specifically as follows: step 1021-1: obtaining the images collected by the area-array camera, and performing sampling according to set frequency; step 1021-2: based on the convolutional neural network algorithm, setting a sliding window to perform feature extraction on the images; and optionally, the convolutional neural network can adopt the yolov4 network.

For the images collected by the area-array camera, sampling is performed according to certain frequency, then the sampled images are input into a pre-trained target detection network for detection, and the pre-trained target detection network adopts the yolov4 network to extract candidate frames from the images.

Specifically, the candidate frame is extracted by using a sliding window method, the size of the sliding window is set, and feature extraction is performed on the local information in each window. Specifically, the extracted features include image color, image texture feature, image shape feature, and some middle-level or high-level semantic features. Step 1021-3: performing classification according to the extracted image features to obtain classification information of diseases, cracks and defects of the tunnel.

After feature extraction, the features extracted from the candidate areas are classified and judged, a decision making tree model or a naive Bayes model can be used to construct a classifier, and the classifier can be obtained through prior learning and training. In this process, for single-category target detection, it is only necessary to distinguish whether the object contained in the current window is the background or the target. For multi-classification problems, it is necessary to further distinguish the category of the object in the current window. After judging a check box, a series of candidate boxes that may be the detection target will be obtained, these candidate boxes may have some overlaps. At this time, an NMS is needed to merge the candidate boxes to finally obtain the target to be detected, that is, the result finally output by the algorithm.

Step 1022. performing image processing on the image collected by the infrared imager to obtain information on the surface of the tunnel; specifically, the image processing process is specifically as follows: step 1022-1: dividing the collected image into a set size, and inputting the same into a GAN semantic segmentation network; step 1022-2: performing iterative calculation in the GAN semantic segmentation network according to set features to merge small blocks to obtain pixel blocks; and step 1022-3: inputting the obtained pixel blocks into a target detection network to judge the target disease category. The target detection network can be specifically R-CNN, Fast R-CNN, or Faster R-CNN.

Step 1023: obtaining point cloud data of a laser radar to determine the current distance and relative position of the mechanical arm and the surface of the tunnel, step 1024: obtaining three-dimensional reconstruction data of the surface of the tunnel by combining the diseases and cracks of the tunnel, the water leakage information on the surface of the tunnel and the point cloud data of the laser radar; and step 1025. displaying different diseases of the lining concrete in different colors, generating a heat map to realize automatic evaluation, and realizing real-time information sharing between a construction site and a remote terminal by using cloud computing, which can be guided by experts. In some embodiments, the mechanical arm trajectory planning method includes the following steps: step 201, obtaining tunnel three-dimensional environmental information of the laser radar, a tunnel surface disease position, obstacle distribution and mechanical arm position information; step 202, analyzing and processing the tunnel three-dimensional environmental information collected by the laser radar and the mechanical arm position information by using a trained deep reinforcement learning network through a scene analysis network; and step 203, predicting and outputting an operating state of the mechanical arm by using the Lstm network to serve as a real-time planning trajectory of the mechanical arm, until an actuator at the tail end of the mechanical arm reaches the tunnel surface disease position. step 205, In order to facilitate the conversion between the radar coordinate system and an engine body base coordinate system, the radar is calibrated before being used.

When in use, the angle of the radar is fine tuned by rotating, and trajectory planning is performed on the tail end of the mechanical arm based on the main body of tunnel point cloud to obtain a series of trajectory points running at the tail end, including the position and orientation to be reached.

Point data on the surface of the tunnel collected by the laser radar are converted to solve an inverse kinematics model.

In this embodiment, in response to the requirements of maintenance operation safety and the dexterity of the mechanical arm, a multi-target planning algorithm of the mechanical arm based on collaborative space and dexterity is used to construct a real-time path planning scheme.

In a training process, training data information obtained from a simulation experiment is used as a current state and is substituted into the network to obtain a pre-training model. The tunnel three-dimensional environmental information collected by the laser radar and the mechanical arm position information are analyzed and processed by using the trained deep reinforcement learning network through the scene analysis network to serve as the input of a subsequent network, and the next operating state of the mechanical arm is predicted by using the Lstm network, until the actuator at the tail end of the mechanical arm reaches the tunnel surface disease position.

Under the influence of multi-source errors, the capture movement of the mechanical arm randomly deviates from a predefined trajectory, and the uncertainty of the mechanical arm and a vision system is described by using Gaussian distribution; when the mechanical arm captures the target, a feasible capture trajectory is generated by using a path planning algorithm at first, priori probability evaluation is performed on the feasible trajectory by using the probability theory combined with Kalman filtering or nonlinear optimization method and combined with the modern control theory, the trajectory with the highest capture probability is used as a working trajectory, and relying on the construction of a visual servo closed-loop control system, the mechanical arm can move along a planned feasible trajectory.

The man-machine interaction working method specifically includes the following steps: step 301, obtaining a movement direction and a movement force input by the feedback teleoperation master manipulator of the man-machine interaction device in real time; and step 302, moving the mechanical arm according to the moving direction, calculating the moving speed by sensing the moving force through the master manipulator, and moving according to the moving speed.

The above descriptions are only preferred embodiments of the present disclosure and are not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and changes. Any modifications, equivalent replacements, improvement and the like, made within the spirit and principle of the present disclosure, shall all be included in the protection scope of the present disclosure.

Although the specific embodiments of the present disclosure are described above in conjunction with the drawings, the protection scope of the present disclosure is not limited thereto. Those skilled in the art should understand that, on the basis of the technical solutions of the present disclosure, various modifications or deformations made by those skilled in the art without creative effect are still within the protection scope of the present disclosure.

Claims

1. À multi-arm robot for automatic tunnel maintenance, comprising a walking mechanism, a fixed chassis arranged on the walking mechanism, and a plurality of multi-degree-of-freedom mechanical arms and a main control platform arranged on the fixed chassis, wherein the tail end of the multi-degree-of-freedom mechanical arm is equipped with an image detection device and a maintenance tool fixing groove, and the main control platform is respectively in communication connection with the multi-degree-of-freedom mechanical arm, the image detection device and a maintenance tool arranged in the maintenance tool fixing groove.

2. The multi-arm robot for automatic tunnel maintenance of claim 1, wherein the image detection device comprises a CCD area-array camera, an infrared imager, a laser scanner, and an information transmission system, and the CCD area-array camera, the infrared imager and the laser scanner are respectively in transmission connection with the control platform through the information transmission system.

3. The multi-arm robot for automatic tunnel maintenance of claim 1, wherein the multi-degree-of-freedom mechanical arms at least comprise a first mechanical arm, a second mechanical arm and a third mechanical arm, which are connected in sequence, and an axial rotation joint and a pitching joint are arranged at the joint between the mechanical arms.

4 The multi-arm robot for automatic tunnel maintenance of claim 1, wherein the multi-degree-of-freedom mechanical arms are arranged in pairs on the fixed chassis, and the control platform controls the coordinated work of the multi-degree-of-freedom mechanical arms arranged in pairs.

5. The multi-arm robot for automatic tunnel maintenance of claim 1, wherein the fixed chassis comprises an upper supporting surface and a lower supporting surface, a spiral lifting platform 1s arranged between the upper supporting surface and the lower supporting surface, the control platform and the multi-degree-of-freedom mechanical arms are arranged on the upper supporting surface, and the spiral lifting platform provides axial orientation rotation to achieve height adjustment.

6. The multi-arm robot for automatic tunnel maintenance of claim 1, wherein the maintenance tool comprises a cutter, a drilling machine or/and a grouting pipe.

7. The multi-arm robot for automatic tunnel maintenance of claim 1, wherein the control platform is further equipped with a man-machine interaction device, the man-machine interaction device comprises a display device and a force feedback teleoperation master manipulator, and the display device and the force feedback teleoperation master manipulator are respectively in communication connection with the control platform.

8. A control method of the multi-arm robot for automatic tunnel maintenance, comprising an automatic task planning method, a mechanical arm trajectory planning method, and a man-machine interaction working method; the automatic task planning method comprises the following steps: constructing a corresponding relationship model of tunnel defect conditions and maintenance operation procedures by adopting the Apriori correlation analysis algorithm; obtaining defect condition data of a tunnel; and inputting the defect condition data into the corresponding relationship model of tunnel defect conditions and maintenance operation procedures, and outputting a tunnel maintenance operation scheme.

9. The control method of the multi-arm robot for automatic tunnel maintenance of claim 8, wherein the mechanical arm trajectory planning method comprises the following steps: obtaining tunnel three-dimensional environmental information of the laser radar, a tunnel surface disease position, obstacle distribution and mechanical arm position information; analyzing and processing the tunnel three-dimensional environmental information collected by the laser radar and the mechanical arm position information by using a trained deep reinforcement learning network through a scene analysis network; and predicting and outputting an operating state of the mechanical arm by using the Lstm network to serve as a real-time planning trajectory of the mechanical arm, until an actuator at the tail end of the mechanical arm reaches the tunnel surface disease position.

10. The control method of the multi-arm robot for automatic tunnel maintenance of claim 8, wherein the man-machine interaction working method comprises the following steps: obtaining a movement direction and a movement force input by the feedback teleoperation master manipulator of the man-machine interaction device in real time; and moving the mechanical arm according to the moving direction, calculating the moving speed by sensing the moving force through the master manipulator, and moving according to the moving speed.