CN113733101A

CN113733101A - Inspection robot, inspection robot operation method, and storage medium

Info

Publication number: CN113733101A
Application number: CN202111154858.XA
Authority: CN
Inventors: 高永昊; 王雪云; 刘魏; 刘伟
Original assignee: Zhuhai Unitech Power Technology Co Ltd
Current assignee: Zhuhai Unitech Power Technology Co Ltd
Priority date: 2021-09-29
Filing date: 2021-09-29
Publication date: 2021-12-03
Anticipated expiration: 2041-09-29
Also published as: CN113733101B

Abstract

The invention relates to the technical field of electric power safety, and provides an inspection operation robot, an operation method of the inspection operation robot and a storage medium. An inspection robot (1000) comprises: the robot comprises a robot body (1100), a three-dimensional camera (1200), a two-dimensional camera (1300), a mechanical arm (1400) and a controller, wherein the controller is electrically connected with the three-dimensional camera (1200), the two-dimensional camera (1300) and the mechanical arm (1400); the three-dimensional camera (1200) is mounted on the robot body (1100); the two-dimensional camera (1300) is mounted on the mechanical arm (1400); the mechanical arm (1400) is provided with a connecting end and an operating end, the connecting end of the mechanical arm (1400) is connected with the robot body (1100), the operating end of the mechanical arm (1400) is provided with an operating mechanism (1500), and the controller controls the target operating mechanism of the mechanical arm (1400) to execute an operating task on a target operating point.

Description

Inspection robot, inspection robot operation method, and storage medium

Technical Field

The invention relates to the technical field of electric power safety, in particular to an inspection operation robot, an operation method of the inspection operation robot and a storage medium.

Background

With the development of the robot technology, the application of the robot in the electric power field is more and more extensive, and the inspection operation robot replaces the manual inspection task to be executed, and becomes a relatively common phenomenon in the power transformation field. Currently, corresponding operations are performed by operating robots.

However, the existing operation robot has a simple structure, cannot perform a relatively complicated operation when performing an inspection task, lacks safety and reliability, and results in a lack of a robot that can perform a complicated operation, and a safe and reliable robot operation method.

Disclosure of Invention

The invention provides an inspection operation robot, an operation method of the inspection operation robot and a storage medium, which are used for providing a robot capable of being operated in a complex way and a safe and reliable operation method of the robot.

The invention provides an inspection operation robot in a first aspect, which is characterized in that the inspection operation robot (1000) comprises a robot body (1100), a three-dimensional camera (1200), a two-dimensional camera (1300), a mechanical arm (1400) and a controller electrically connected with the three-dimensional camera (1200), the two-dimensional camera (1300) and the mechanical arm (1400); wherein the content of the first and second substances,

the three-dimensional camera (1200) is mounted to the robot body (1100);

the two-dimensional camera (1300) is mounted to the robotic arm (1400);

the robot comprises a robot body (1100), a mechanical arm (1400) and a controller, wherein the mechanical arm (1400) is provided with a connecting end and an operating end, the connecting end of the mechanical arm (1400) is connected with the robot body (1100), the operating end of the mechanical arm (1400) is provided with an operating mechanism (1500), the controller reads position information of a target operating mechanism corresponding to a target operating point and a target three-dimensional coordinate of the target operating point, and the target operating mechanism of the mechanical arm (1400) is controlled to execute an operating task on the target operating point based on the position information of the target operating mechanism and the target three-dimensional coordinate.

In one possible embodiment, the robotic arm (1400) comprises a first robotic arm (1401) and a second robotic arm (1402), the first robotic arm (1401) being a six-axis robotic arm and the second robotic arm (1402) being a four-axis robotic arm.

In one possible embodiment, the first robot arm (1401) is used to change the operating mechanism on the second robot arm (1402); the operating mechanism for disassembly on the first mechanical arm (1401) is used for disassembling and replacing the operating mechanism on the second mechanical arm (1402).

In one possible embodiment, the first robot arm (1401) is configured to move to a target position of the target actuator and mount the target actuator to the actuator when it is detected that the actuator is not mounted with the current actuator;

the first mechanical arm (1401) is further used for moving to a storage station corresponding to the current operating mechanism when the fact that the current operating mechanism is installed on the operating end is detected, placing the current operating mechanism into the corresponding storage station, moving to the position of the target operating mechanism, and installing the target operating mechanism on the operating end.

In one possible embodiment, the mechanical arm (1400) is configured to perform the mounting operation of the target operating mechanism or the switching operation or non-switching operation and mounting operation of the target operating mechanism, move the target operating mechanism to a position corresponding to the target three-dimensional coordinate, and control the target operating mechanism to perform an operation task on the target operating point.

In a possible implementation manner, the three-dimensional camera (1200) is used for shooting a target mark point part of a target object to obtain a three-dimensional mark point position image;

the two-dimensional camera (1300) is used for shooting a target operation point at the position adjusted based on the three-dimensional mark point position image shot by the three-dimensional camera (1200);

the two-dimensional camera (1300) is also used for shooting an operation target corresponding to the target operation point after the operation task is completed.

In a possible embodiment, the inspection operation robot (1000) further comprises a walking module (1600) electrically connected with the controller, wherein the walking module (1600) is mounted on the robot body (1100) and is used for driving the robot body (1100) to travel or rotate on the ground.

In a possible implementation manner, the inspection operation robot (1000) further comprises a tool library (1700), wherein the tool library (1700) is installed on the robot body (1100), the tool library (1700) is provided with a plurality of storage stations (1701), and the plurality of storage stations (1701) are all used for storing operation mechanisms (1500);

the operating end of the mechanical arm (1400) is detachably connected with the operating mechanism (1500), and the controller controls the operating end of the mechanical arm (1400) to be connected with the operating mechanism on one of the storage stations (1701) of the tool library (1700).

In one possible embodiment, the controller includes a first control module, a second control module, a third control module, and a fourth control module;

the first control module is used for acquiring a target three-dimensional coordinate obtained by shooting a target mark point part of a target object by the three-dimensional camera and/or the two-dimensional camera;

the second control module is used for controlling the first mechanical arm (1401) to move according to the action instruction of the first control module;

the third control module is used for controlling the second mechanical arm (1402) to move according to the action command of the first control module;

the fourth control module is used for controlling the walking module (1600) to move according to the action instruction of the first control module.

In a possible implementation mode, the inspection operation robot (1000) further comprises a binocular camera (1800) electrically connected with the controller, wherein the binocular camera (1800) is used for performing instrument reading identification on an inspection target; the binocular camera (1800) is also used for detecting the temperature of the routing inspection target.

In one possible embodiment, the controller further comprises a fifth control module;

the fifth control module is used for receiving the shooting result of the binocular camera (1800) and analyzing the shooting result.

A second aspect of the present invention provides an operation method of an inspection operation robot, including:

acquiring a target three-dimensional coordinate obtained by shooting a target mark point part of a target object by the three-dimensional camera and/or the two-dimensional camera, wherein the target operation point is a task operation point of the target object;

and reading position information of a target operating mechanism corresponding to the target operating point, and controlling the target operating mechanism of the mechanical arm to execute an operating task on the target operating point based on the position information of the target operating mechanism and the target three-dimensional coordinate.

In a possible implementation manner, the reading of the position information of the target operating mechanism corresponding to the target operating point, and controlling the mechanical arm to execute the operation task on the target operating point based on the position information of the target operating mechanism and the target three-dimensional coordinates includes:

reading position information of a target operating mechanism corresponding to the target operating point;

controlling the mechanical arm to execute installation operation of the target operating mechanism or switching operation of the target operating mechanism or not to execute switching operation and installation operation based on the position information of the target operating mechanism;

and controlling a target operation mechanism on the operated mechanical arm to move to a position corresponding to the target three-dimensional coordinate, and executing an operation task on the target operation point.

In one possible embodiment, the controlling the robot arm to perform the mounting operation of the target operating mechanism or the switching operation of the target operating mechanism or not to perform the switching operation or the mounting operation based on the position information of the target operating mechanism includes:

when the position information of the target operating mechanism indicates that the target operating mechanism is arranged in a corresponding storage station in the tool library, acquiring an operating mechanism detection result, wherein the operating mechanism detection result is used for indicating and detecting whether a current operating mechanism is arranged at an operating end of the mechanical arm;

controlling the mechanical arm to execute the installation operation or switching operation of the target operating mechanism according to the detection result of the operating mechanism;

and when the position information of the target operating mechanism is that the target operating mechanism is positioned on the operating end of the mechanical arm, controlling the mechanical arm not to execute switching operation and installation operation.

In a possible embodiment, the controlling the robot arm to perform the mounting operation or the switching operation of the target operating mechanism according to the operating mechanism detection result includes:

if the detection result of the operating mechanism indicates that the operating end of a first mechanical arm in the mechanical arms is not provided with the current operating mechanism, controlling the first mechanical arm to move to the target position of the target operating mechanism, and installing the target operating mechanism to the operating end of the first mechanical arm;

if the detection result of the operating mechanism is that the current operating mechanism is installed at the operating end of the first mechanical arm, obtaining a storage station detection result, wherein the storage station detection result is used for indicating whether a storage station corresponding to the current operating mechanism in the tool library is occupied or not;

if the storage station corresponding to the current operating mechanism in the tool library is not occupied, controlling the first mechanical arm to move to the storage station corresponding to the current operating mechanism, and placing the current operating mechanism into the corresponding storage station;

and controlling the first mechanical arm to move to the position of the target operating mechanism, and mounting the target operating mechanism on the operating end of the first mechanical arm.

if the detection result of the operating mechanism indicates that the operating end of the second mechanical arm is not provided with the current operating mechanism, controlling the first mechanical arm to move to the target position of the target operating mechanism, and installing the target operating mechanism to the operating end of the second mechanical arm;

if the detection result of the operating mechanism is that the current operating mechanism is installed at the operating end of the second mechanical arm, obtaining a storage station detection result, wherein the storage station detection result is used for indicating whether a storage station corresponding to the current operating mechanism in the tool library is occupied or not;

if the storage station corresponding to the current operating mechanism in the tool library is not occupied, controlling the first mechanical arm to detach the current operating mechanism on the second mechanical arm through the operating mechanism for detaching on the first mechanical arm, moving the current operating mechanism to the storage station corresponding to the current operating mechanism, and placing the current operating mechanism into the corresponding storage station;

and controlling the first mechanical arm to move to the position of the target operating mechanism, installing the target operating mechanism to the operating end of the second mechanical arm through the operating mechanism used for disassembly on the first mechanical arm, and controlling the second mechanical arm to perform rechecking of installation of the target operating mechanism.

In a possible implementation manner, after the reading of the position information of the target operating mechanism corresponding to the target operating point and the controlling of the mechanical arm to perform the operation task on the target operating point based on the position information of the target operating mechanism and the target three-dimensional coordinates, the method further includes:

when the operation action is detected to be completed, acquiring an operation target image, performing state analysis on the operation target image and confirming that the operation target is completed, wherein the operation target image is used for indicating the two-dimensional camera to shoot an image obtained by the operation target corresponding to the target operation point after the operation task is completed.

In one possible embodiment, the method for operating the inspection operation robot further includes:

and acquiring the polling target shooting result of the binocular camera, and performing instrument reading identification or temperature detection on the polling target according to the polling target shooting result.

A third aspect of the present invention provides a computer-readable storage medium having stored therein a computer program which, when run on a computer, causes the computer to execute the above-described inspection operation robot operation method.

According to the technical scheme, the inspection operation robot comprises a robot body, a three-dimensional camera, a two-dimensional camera, a mechanical arm and a controller electrically connected with the two-dimensional camera and the mechanical arm of the three-dimensional camera, the controller of the inspection operation robot reads position information of a target operation mechanism corresponding to a target operation point and a target three-dimensional coordinate of the target operation point, and the target operation mechanism of the mechanical arm is controlled to execute an operation task on the target operation point based on the position information and the target three-dimensional coordinate of the target operation mechanism, so that inspection and operation are combined into a whole, an operation method for determining a target operation structure and operation is realized, the robot capable of being operated in a complex mode is provided, and a safe and reliable robot operation method is provided.

Drawings

FIG. 1 is a schematic structural diagram of an inspection robot according to an embodiment of the present invention;

fig. 2 is a schematic diagram of an embodiment of an operation method of the inspection operation robot in the embodiment of the invention;

FIG. 3 is a schematic diagram of a three-dimensional camera with ideal fiducial locations on the body and/or a two-dimensional camera with ideal fiducial locations on the camera according to an embodiment of the present invention;

fig. 4 is a schematic diagram of another embodiment of the operation method of the inspection operation robot in the embodiment of the invention;

FIG. 5 is a schematic diagram of a position where the target operating mechanism moves to a position corresponding to the target three-dimensional coordinates and performs an operation task on the target operating point according to an embodiment of the present invention;

fig. 6 is another schematic position diagram of the target operating mechanism moving to the position corresponding to the target three-dimensional coordinate and executing the operating task on the target operating point in the embodiment of the present invention.

The reference numbers illustrate:

reference numerals	Name (R)	Reference numerals	Name (R)
				1000	Inspection operation robot	1500b	Operating mechanism on second mechanical arm
1100	Robot body	1600	Walking module
				1200	Three-dimensional camera	1601	Movable chassis
1300	Two-dimensional camera	1602	Traveling mechanism
				1301	First two-dimensional camera	1700	Tool library
1302	Second two-dimensional camera	1701	Storage station
				1400	Mechanical arm	1800	Binocular camera
1401	First mechanical arm	1801	Visible light camera
				1402	Second mechanical arm	1802	Infrared thermal imaging instrument
1500	Operating mechanism	1900	Cloud platform
				1500a	Operating mechanism for disassembly	1010	Lifting platform

Detailed Description

The embodiment of the invention provides an inspection operation robot, an operation method of the inspection operation robot and a storage medium, and provides a robot capable of being operated in a complex way and a safe and reliable robot operation method.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

For convenience of understanding, a specific flow of the embodiment of the present invention is described below, and referring to fig. 1, fig. 1 is a schematic structural diagram of an inspection robot 1000 according to the embodiment of the present invention, where the inspection robot 1000 includes: a robot body 1100, a three-dimensional camera 1200, a two-dimensional camera 1300, a robot arm 1400, and a controller (not shown) electrically connected to the three-dimensional camera 1200, the two-dimensional camera 1300, and the robot arm 1400; wherein, the three-dimensional camera 1200 is mounted on the robot body 1100; the two-dimensional camera 1300 is mounted on the robotic arm 1400; the mechanical arm 1400 is provided with a connecting end and an operating end, the connecting end of the mechanical arm 1400 is connected with the robot body 1100, and the operating end of the mechanical arm 1400 is provided with an operating mechanism 1500; the controller is used for controlling the three-dimensional camera 1200, the two-dimensional camera 1300 and the mechanical arm 1400 to execute corresponding operation actions.

Further, the mechanical arm (1400) is used for executing installation operation of the target operating mechanism or switching operation of the target operating mechanism or not executing switching operation and installation operation, moving the target operating mechanism to a position corresponding to the target three-dimensional coordinate, and controlling the target operating mechanism to execute an operating task on the target operating point.

The three-dimensional camera (1200) is used for shooting a target mark point part of a target object to obtain a three-dimensional mark point position image; the two-dimensional camera (1300) is used for shooting a target operation point at the position adjusted based on the three-dimensional mark point position image shot by the three-dimensional camera (1200); the two-dimensional camera (1300) is also used for shooting the operation target corresponding to the target operation point after the operation task is completed.

The robot arm 1400 includes a first robot arm 1401 and a second robot arm 1402, and the two-dimensional camera 1300 includes a first two-dimensional camera 1301 and a second two-dimensional camera 1302; the first two-dimensional camera 1301 is mounted on the operation end of the first robot 1401; a second two-dimensional camera 1302 is mounted at the operating end of the second robotic arm 1402. The first mechanical arm 1401 is used for operating an operation target with high requirement on the operation freedom degree and is used for installing different operation mechanisms 1500, and the first mechanical arm 1401 can be a six-axis mechanical arm; the second robot 1402 is configured to operate an operation target with a high operation torque requirement, and the second robot 1402 may be a four-axis robot. The first mechanical arm (1401) is also used for replacing the operating mechanism 1500b on the second mechanical arm (1402); the operation mechanism 1500a for disassembly on the first robot arm 1401 is used for disassembling and replacing the operation mechanism 1500b on the second robot arm 1402.

Optionally, the first mechanical arm (1401) is configured to, when it is detected that the current operating mechanism is not mounted on the operating end, move to a target position of the target operating mechanism, and mount the target operating mechanism on the operating end; the first mechanical arm (1401) is also used for moving to a storage station corresponding to the current operating mechanism when the fact that the current operating mechanism is installed on the operating end is detected, placing the current operating mechanism into the corresponding storage station, moving to the position of the target operating mechanism, and installing the target operating mechanism on the operating end.

The inspection operation robot 1000 further comprises a tool magazine 1700, the tool magazine 1700 is mounted on the robot body 1100, the tool magazine 1700 is provided with a plurality of storage stations 1701, and the plurality of storage stations 1701 are all used for storing the operation mechanism 1500; the manipulator 1400 is detachably connected to the manipulator 1500, and the controller controls the manipulator 1400 to be connected to the manipulator 1500 at one of the plurality of storage sites 1701 in the tool library 1700. The manipulator 1500 at one of the plurality of storage stations 1701 of the tool library 1700 is mounted to the first robot 1401. An identification device is arranged in each storage station 1701, an identification code is arranged on each operating mechanism 1500, and an identification device is arranged at the operating end of the mechanical arm 1400 and is used for identifying the identification code on each operating mechanism 1500.

The operative end of the robotic arm 1400 may be coupled to the operating mechanism 1500 at one of the plurality of storage stations 1701 of the tool library 1700 via a magnet-type coupling. The operation end of the robot 1400 may also be connected to the operation mechanism 1500 at one of the plurality of storage stations 1701 of the tool magazine 1700 via a hydraulic jaw, and when the operation end of the robot 1400 performs the mounting operation of the operation mechanism 1500 (mounting the operation mechanism 1500 to the operation end of the robot 1400), the operation mechanism 1500 is clamped by controlling the hydraulic jaw to be pressurized; when the operation end of the robot arm 1400 performs the detaching operation of the operation mechanism 1500 (detaches the operation mechanism 1500 from the operation end of the robot arm 1400), the operation mechanism 1500 is released by controlling the pressure reduction of the hydraulic jaws.

The operating end of the mechanical arm 1400 may be provided with a turntable, the two-dimensional camera 1300 is mounted on the turntable, and the turntable is used for driving the two-dimensional camera 1300 to rotate; and the two-dimensional camera 1300 is used for moving to the target position of the operation target on the target object under the driving of the mechanical arm 1400 and shooting.

The inspection operation robot 1000 further includes a walking module 1600 electrically connected to the controller, and the walking module 1600 is installed in the robot body 1100 and is used to drive the robot body 1100 to travel or rotate on the ground. The walking module 1600 comprises a mobile chassis 1601 and a walking mechanism 1602; the robot body 1100 is mounted on a mobile chassis 1601, and the mobile chassis 1601 is used for driving the robot body 1100 to travel or rotate on the ground; the traveling mechanism 1602 is mounted to the mobile chassis 1601 for traveling or rotating on the ground. The mobile chassis 1601 is provided with one or more than one laser radar for routing inspection of the navigation and obstacle avoidance (i.e., obstacle avoidance) of the operating robot 1000.

The running gear 1602 may be a wheel-type running gear, such as: a drive wheel and a steering wheel; the running gear 1602 may also be a crawler running gear; the walking mechanism 1602 can also be a walking mechanism, such as: a two-foot walking mechanism, a four-foot walking mechanism and a six-foot walking mechanism; the traveling mechanism 1602 may also be a suction cup type traveling mechanism; the traveling mechanism 1602 may also be a magnet type traveling mechanism; the running mechanism 1602 may also be a hybrid running mechanism, such as: the walking mechanism is composed of wheels and feet, and the spherical rotating body is arranged at the end of the foot and is not limited herein.

The inspection operation robot 1000 further comprises a binocular camera 1800 electrically connected with the controller, wherein the binocular camera 1800 is used for performing instrument reading identification on an inspection target; binocular camera 1800 is also used for temperature detection of the inspection target. The binocular camera 1800 comprises a visible light camera 1801 and an infrared thermal imager 1802; the visible light camera 1801 is used for photographing an instrument of the inspection target and performing instrument reading identification through image analysis; the infrared thermal imager 1802 is used to detect the temperature of the inspection target.

The inspection operation robot 1000 further comprises a holder 1900 and a lifting platform 1010, wherein the holder 1900 is used for placing a camera; the three-dimensional camera 1200 is mounted on the pan/tilt head 1900; the binocular camera 1800 is mounted on the pan/tilt head 1900; the first robot 1401 is mounted on a lifting platform 1010, and the lifting platform 1010 is used for controlling the first robot 1401 to extend or retract along the height direction of the inspection robot 1000. The pan/tilt head 1900 can perform a telescopic operation. The connection positions of the pan tilt 1900 and the three-dimensional camera 1200 and the binocular camera 1800 are respectively provided with a connection mechanism, the connection mechanism can be a turntable, and the turntable is used for supporting the rotation of the three-dimensional camera 1200 and the binocular camera 1800.

The controller comprises a first control module, a second control module, a third control module, a fourth control module and a fifth control module. The first control module is used for receiving and analyzing the shooting results of the three-dimensional camera 1200 and the two-dimensional camera 1300, and issuing action instructions to the second control module, the third control module and the fourth control module, and specifically, the first control module is used for acquiring a target three-dimensional coordinate obtained by shooting a target mark point part of a target object by the three-dimensional camera and/or the two-dimensional camera; the second control module is used for controlling the first mechanical arm 1401 to move according to the action instruction of the first control module; the third control module is used for controlling the second mechanical arm 1402 to move according to the action command of the first control module; the fourth control module is used for controlling the walking module 1600 to move according to the action instruction of the first control module, and further, the fourth control module is used for controlling the mobile chassis 1601 to move according to the action instruction of the first control module; the fifth control module is configured to receive the shooting result of the binocular camera 1800, analyze the shooting result, and detect and identify the inspection target.

According to the embodiment of the invention, the inspection operation robot comprises a robot body, a three-dimensional camera, a two-dimensional camera, a mechanical arm and a controller electrically connected with the three-dimensional camera, the two-dimensional camera and the mechanical arm, wherein the three-dimensional camera is arranged on the robot body; the two-dimensional camera is arranged on the mechanical arm; the mechanical arm is provided with a connecting end and an operating end, the connecting end of the mechanical arm is connected with the robot body, and the operating end of the mechanical arm is provided with an operating mechanism; the controller is used for controlling the three-dimensional camera, the two-dimensional camera and the mechanical arm to execute corresponding operation actions, so that the inspection and the operation are combined into a whole, the target operation structure determination and the operation determination operation method are realized, and the robot capable of performing complex operation is provided.

Referring to fig. 2, an embodiment of an operation method of an inspection operation robot according to an embodiment of the present invention includes:

s101, acquiring a target three-dimensional coordinate obtained by shooting a target mark point part of a target object by a three-dimensional camera and/or a two-dimensional camera, wherein a target operation point is a task operation point of the target object.

The inspection operation robot and the operation method of the inspection operation robot can be suitable for industries, rail traffic, power plants and the like, and are not particularly limited, for example, in the rail traffic, the inspection operation robot can be a rail intelligent inspection operation robot, and accurate identification, accurate positioning and corresponding accurate operation can be carried out on each rail device through the rail intelligent inspection operation robot by combining a three-dimensional camera and a two-dimensional camera. The target object correspondingly operated by the inspection operation robot can be electrical equipment and non-electrical equipment, and the electrical equipment comprises the following components: buttons, switches, etc. on a switch cabinet at a certain location in the substation are not specifically limited herein. For convenience of explanation, the target object in the present embodiment is explained by taking an electrical device as an example, but not limited thereto.

The present invention is described with reference to an inspection robot as an execution subject of an operation method of the inspection robot by way of example and not limitation.

Wherein, one or more than one mark points are arranged on the surface (one or more than one surface) of the preset electrical equipment of the operation task. The inspection operation robot 1000 controls the mobile chassis 1601 through laser navigation according to the content of the operation task through the controller, and navigates the inspection operation robot 1000 to a preset position of the electrical device according to a given walking path, so that the mark point appears in a visual range of the three-dimensional (3D) camera. The electrical device is a device in which an operation target in the operation task is located, that is, the target object is an object in which the operation target in the operation task is located.

The inspection operation robot 1000 sends a first camera shooting instruction to the three-dimensional camera 1200 through the controller, after receiving the first camera shooting instruction, the three-dimensional camera 1200 shoots a target marking point part on the surface of the electrical equipment based on the first camera shooting instruction to obtain a three-dimensional marking point position image, and returns the three-dimensional marking point position image to the controller, wherein the target marking point part is used for indicating the surface of part or all cabinets of the electrical equipment including the marking point, and the three-dimensional marking point position image includes the pixel coordinates (three-dimensional camera coordinates) of the target marking point part. And the controller extracts the three-dimensional camera coordinates in the three-dimensional mark point position image and takes the three-dimensional camera coordinates as target three-dimensional coordinates.

Or, the target three-dimensional coordinate is a corrected three-dimensional coordinate, and the execution process of acquiring the corrected three-dimensional coordinate includes: the controller processes a three-dimensional mark point position image obtained by shooting a target mark point part of a target object according to the three-dimensional camera, calculates compensation position information and extracts the height coordinate of a target operation point, processes a target operation point image shot by the two-dimensional camera after being adjusted based on the compensation position information, calculates the two-dimensional coordinate of the target operation point, and generates a corrected three-dimensional coordinate of the target operation point based on the two-dimensional coordinate and the height coordinate. Specifically, after receiving the three-dimensional mark point position image sent by the three-dimensional camera 1200, the controller calculates a compensation angle and a compensation distance from the three-dimensional mark point position image to obtain compensation position information. The controller acquires a target operation point image which is shot by the two-dimensional camera after being adjusted based on the compensation position information, and calculates a two-dimensional coordinate of the target operation point based on the target operation point image; target three-dimensional coordinates (i.e., modified three-dimensional coordinates) of the target operation point are generated based on the two-dimensional coordinates and the height coordinates.

It should be noted that the target three-dimensional coordinates may also be three-dimensional coordinates obtained by analyzing images captured by the three-dimensional camera 1200 at the ideal reference point position of the robot body 1100 and/or the two-dimensional camera 1300 at the ideal reference point position of the camera, as shown in fig. 3, and fig. 3 is a schematic structural diagram of the three-dimensional camera at the ideal reference point position of the robot body and/or the two-dimensional camera at the ideal reference point position of the camera.

And S102, reading position information of a target operating mechanism corresponding to the target operating point, and controlling the target operating mechanism of the mechanical arm to execute an operating task on the target operating point based on the position information of the target operating mechanism and the target three-dimensional coordinate.

Specifically, the inspection operation robot 1000 reads position information of a target operation mechanism corresponding to the target operation point; based on the position information of the target operating mechanism, the robot arm 1400 is controlled to perform the mounting operation of the target operating mechanism or the switching operation of the target operating mechanism or not to perform the switching operation or the mounting operation; and controlling the operated target operation mechanism on the mechanical arm 1400 to move to a position corresponding to the target three-dimensional coordinate, and executing an operation task on the target operation point.

The execution process of the inspection operation robot 1000 for reading the position information of the target operation mechanism corresponding to the target operation point includes: the inspection operation robot 1000 sends an identification instruction to an identification device at the operation end of the robot arm 1400 and an identification device in the storage station 1701 through the controller, the identification device at the operation end of the robot arm 1400 identifies the operation end of the robot arm 1400 based on the identification instruction to obtain first identification information, the first identification information includes position information of the identification device and an identification result of the operation end of the robot arm 1400 and sends the first identification information to the controller, the identification device in the storage station 1701 identifies each storage station 1701 based on the identification instruction to obtain second identification information, the second identification information includes position information of the identification device and an identification result in the storage station 1701 and sends the second identification information to the controller, and the controller receives the first identification information and the second identification information to obtain target information (including structure information and identification code information of a target operation mechanism) of the target operation mechanism, and matching the first identification information and the second identification information through the target information of the target operating mechanism to obtain target identification information, and determining the position information of the identification device in the target identification information as the position information of the target operating mechanism.

Further, in order to improve the accuracy of the position information of the target operating mechanism, the inspection robot 1000 controls the three-dimensional camera 1200 or the two-dimensional camera 1300 to move to the position corresponding to the position information of the recognition device in the target recognition information for photographing to obtain a first position image, invokes a preset image processing algorithm, image preprocessing and target operating mechanism identification are carried out on the first position image, if the target operating mechanism is identified as the target operating mechanism in the first position image, the position information of the recognition device in the target recognition information is determined as the final position information of the target operating mechanism, if the target operating mechanism is not present in the position image as a result of the recognition of the target operating mechanism, the identification device is called to identify the target operation structure again, and the corresponding position information is acquired, so that the position information of the target operation mechanism corresponding to the target operation point (operation target) is obtained.

The inspection robot 1000 controls the robot arm 1400 to perform the mounting operation of the target operating mechanism or the switching operation of the target operating mechanism or not to perform the switching operation, based on the position information of the target operating mechanism, and the performing process of the mounting operation includes: the inspection operation robot 1000 issues corresponding (different) action instructions to the mechanical arm 1400 through the controller according to the position information of the target operation mechanism, so that the mechanical arm 1400 performs the target operation mechanism installation operation or the target operation mechanism switching operation or does not perform the switching operation or the installation operation. Specifically, if the position information of the target operating mechanism indicates that the target operating mechanism is located on the mechanical arm 1400, the inspection operating robot 1000 sends a target item non-execution instruction to the mechanical arm 1400 through the controller, and after receiving the target item non-execution instruction, the mechanical arm 1400 does not execute the switching operation and the installation operation, and executes the content corresponding to step S106; if the position information of the target operating mechanism indicates that the target operating mechanism is not located on the robot 1400, an installation instruction or a switching instruction is sent to the robot 1400 according to the position information of the target operating mechanism, and after receiving the installation instruction or the switching instruction, the robot 1400 executes an operation of installing the target operating mechanism based on the installation instruction or executes an operation of switching the target operating mechanism based on the switching instruction.

The execution process that the target operation mechanism on the mechanical arm controlled and operated by the inspection operation robot 1000 moves to the position corresponding to the target three-dimensional coordinate and executes the operation task on the target operation point includes: the inspection operation robot 1000 may send a second motion instruction to the mechanical arm 1400 after the control operation through the controller, after the mechanical arm 1400 after the control operation receives the second motion instruction, the second motion instruction is moved to a position corresponding to the target three-dimensional coordinate based on the second motion instruction, so as to move the target operation mechanism on the mechanical arm 1400 to the position corresponding to the target three-dimensional coordinate, after the mechanical arm 1400 performs an operation action corresponding to the second motion instruction, a completion instruction is returned to the controller, after the controller receives the completion instruction, a task operation instruction sequence is sent to the mechanical arm 1400 based on the operation task, after the mechanical arm 1400 receives the task operation instruction sequence, the operation action is performed on the target operation point (of the operation target) according to the task operation instruction sequence.

In the embodiment of the invention, the target three-dimensional coordinate obtained by shooting the target mark point part of the target object by the three-dimensional camera and/or the two-dimensional camera is obtained, the mechanical arm is controlled to execute the operation task on the target operation point based on the position information of the target operation mechanism and the target three-dimensional coordinate, the accuracy of the operation is ensured, and the safe and reliable robot operation method is provided by combining the routing inspection and the operation into a whole and realizing the operation method of determining the target operation structure and the operation.

Referring to fig. 4, another embodiment of the operation method of the inspection operation robot according to the embodiment of the present invention includes:

s201, acquiring a target three-dimensional coordinate obtained by shooting a target mark point part of a target object by a three-dimensional camera and/or a two-dimensional camera, wherein a target operation point is a task operation point of the target object.

The inspection operation robot and the operation method of the inspection operation robot can be suitable for oil and gas transportation industry, rail transit, power plants and the like, and are not particularly limited, for example, in the oil and gas transportation industry, the inspection operation robot can be an oil well intelligent inspection operation robot, and various instrument devices in an oil well are accurately identified, accurately positioned and accurately operated by the oil well intelligent inspection operation robot in combination with a binocular camera, a three-dimensional camera and a two-dimensional camera. The target object correspondingly operated by the inspection operation robot can be electrical equipment and non-electrical equipment, and the electrical equipment comprises the following components: buttons, switches and the like on the electrical cabinet at a certain position in the substation are not specifically limited herein.

For convenience of explanation, the target object in the present embodiment is described by taking an electrical cabinet in electrical equipment and a surface of the target object as a cabinet surface by way of example and not limitation. The electrical cabinet is a device where an operation target in the operation task is located, that is, the target object is an object where the operation target in the operation task is located, and the surface of the target object may be a cabinet surface where the operation target is located, that is, a cabinet surface.

The process of step S201 is similar to the process of step S101, and is not described herein again.

Wherein, it is flexible along the direction of perpendicular robot body 1100 that the operation robot 1000 patrols and examines passes through controller control cloud platform to adjust three-dimensional camera 1200 and binocular camera 1800's height. When the operating height of the first robot 1401 reaches a preset threshold, the inspection operating robot 1000 may control the lifting platform 1010 to extend and retract along a direction perpendicular to the robot body 1100 through the controller, so as to adjust the height of the first robot 1401.

S202, reading position information of a target operating mechanism corresponding to the target operating point, and controlling the target operating mechanism of the mechanical arm to execute an operating task on the target operating point based on the position information of the target operating mechanism and the target three-dimensional coordinates.

Specifically, the inspection operation robot 1000 reads position information of a target operation mechanism corresponding to the target operation point; controlling the mechanical arm to execute the installation operation of the target operating mechanism or the switching operation of the target operating mechanism or not to execute the switching operation and the installation operation based on the position information of the target operating mechanism; and controlling the target operation mechanism on the operated mechanical arm to move to a position corresponding to the target three-dimensional coordinate, and executing an operation task on the target operation point.

As shown in fig. 5 and 6, fig. 5 is a schematic view of a position where the target operating mechanism moves to a position corresponding to the target three-dimensional coordinates and performs an operation task on the target operating point, and fig. 6 is a schematic view of another position where the target operating mechanism moves to a position corresponding to the target three-dimensional coordinates and performs an operation task on the target operating point.

The execution process of the inspection operation robot 1000 for reading the position information of the target operation mechanism corresponding to the target operation point includes: the inspection operation robot 1000 controls the recognition device at the operation end of the mechanical arm 1400 and the recognition device in the storage station 1701 to recognize the recognition code on the target operation mechanism corresponding to the target operation point (operation target) in real time and send the information recognized in real time to the controller, the controller sends the received information recognized in real time to the cloud (which can be a private cloud), and the cloud storage is performed on the information recognized in real time through the cloud (which can be a private cloud), so that the management convenience and the safety of the information recognized in real time are improved. When the inspection operation robot 1000 needs to read the current-time position information of the target operation mechanism, a reading request is sent to the cloud (private cloud) through the first control module, the reading request includes the target operation mechanism, key information corresponding to the current time and the position, and the current-time position information of the target operation mechanism sent by the cloud (private cloud) is received, so that the position information of the target operation mechanism is obtained.

Further, in order to improve the accuracy of the position information of the target operating mechanism, the inspection operating robot 1000 sends a third shooting instruction to the three-dimensional camera 1200 or the two-dimensional camera 1300 through the first control module, the three-dimensional camera 1200 or the two-dimensional camera 1300 shoots a position corresponding to the current position information of the target operating mechanism sent by the cloud (private cloud) based on the third shooting instruction to obtain a second position image, and a preset image processing algorithm is called to perform image preprocessing and target operating mechanism identification on the second position image; if the target operating mechanism exists in the second position image as a result of the identification of the target operating mechanism, determining the current-time position information of the target operating mechanism sent by the cloud (private cloud) as the final position information of the target operating mechanism; and if the target operating mechanism is not identified in the position image as a result of identification of the target operating mechanism, sending a re-identification instruction to the identification device, identifying the identification code corresponding to the target operating mechanism again by the identification device based on the re-identification instruction, and acquiring corresponding position information so as to obtain the position information of the target operating mechanism corresponding to the target operating point (operating target).

The inspection operation robot 1000 controls the robot arm to perform the mounting operation of the target operation mechanism or the switching operation of the target operation mechanism or not to perform the switching operation based on the position information of the target operation mechanism, and the performing process of the mounting operation includes: specifically, when the position information of the target operating mechanism is that the target operating mechanism is placed in the corresponding storage station 1701 in the tool library 1700, an operating mechanism detection result is obtained, and the operating mechanism detection result is used for indicating whether the current operating mechanism is installed at the operating end of the detection mechanical arm 1400; controlling the mechanical arm 1400 to perform the mounting operation or switching operation of the target operating mechanism according to the operating mechanism detection result; when the position information of the target operating mechanism is that the target operating mechanism is on the operating end of the robot 1400, the robot 1400 is controlled not to perform the switching operation or the mounting operation.

Specifically, if the detection result of the operating mechanism is that the operating end of the first robot 1401 in the robot 1400 is not equipped with the current operating mechanism, the first robot 1401 is controlled to move to the target position of the target operating mechanism, and the target operating mechanism is installed on the operating end of the first robot 1401, and the robot 1400 comprises the first robot 1401 and the second robot 1402; if the detection result of the operating mechanism is that the current operating mechanism is installed at the operating end of the first mechanical arm 1410, a storage station detection result is obtained, and the storage station detection result is used for indicating whether the storage station 1701 corresponding to the current operating mechanism in the detection tool library 1700 is occupied or not; if the storage station 1701 corresponding to the current operating mechanism in the tool library 1700 is not occupied, controlling the first mechanical arm 1401 to move to the storage station 1701 corresponding to the current operating mechanism, and placing the current operating mechanism into the corresponding storage station 1701; controls the first robot arm 1401 to move to the position of the target actuator and mounts the target actuator to the operating end of the first robot arm 1401.

When the position information of the target operating mechanism is that the target operating mechanism is placed in the corresponding storage station 1701 in the tool library 1700, the inspection operating robot 1000 sends a first detection instruction to the identification device of the first mechanical arm 1401 through the first control module, the identification device of the first mechanical arm 1401 identifies the operating end of the first mechanical arm 1401 based on the first detection instruction to obtain an operating mechanism detection result, and sends the operating mechanism detection result to the first control module, and the first control module analyzes the operating mechanism detection result after receiving the operating mechanism detection result.

If the operating mechanism detection result indicates that the current operating mechanism is not installed at the operating end of the first robot 1401, the first control module sends an installation instruction to the second control module, and the second control module controls the first robot 1401 to move to the target position of the target operating mechanism based on the installation instruction and installs the target operating mechanism to the operating end of the first robot 1401, and further, the first control module receives the installed identification information sent by the identification device of the operating end of the first robot 1401 and analyzes the identification information to detect and confirm whether the target operating mechanism is installed at the operating end of the first robot 1401.

If the detection result of the operating mechanism is that the current operating mechanism is installed at the operating end of the first mechanical arm 1401, the first control module sends a second detection instruction to an identification device in the storage station 1701 corresponding to the current operating mechanism, the identification device identifies the storage station 1701 corresponding to the current operating mechanism in the tool library 1700 based on the second detection instruction to obtain a storage station detection result, and sends the storage station detection result to the first control module, the storage station detection result is used for indicating whether the storage station 1701 corresponding to the current operating mechanism in the tool library 1700 is occupied, the first control module analyzes the storage station detection result after receiving the storage station detection result, if the storage station 1701 corresponding to the current operating mechanism in the tool library 1700 is not occupied, the first control module sends a hole position moving instruction to the second control module, and the second control module controls the first mechanical arm 1401 to move to the storage station 1701 corresponding to the current operating mechanism according to the hole position moving instruction Placing the current operating mechanism into the corresponding storage station 1701 at the storage station 1701, moving the current operating mechanism into the corresponding storage station 1701, and mounting the target operating mechanism to the operating end of the first robot 1401; if the storage station 1701 corresponding to the current operating mechanism in the tool library 1700 is (illegally) occupied, the operation process is ended and the first control module sends a warning message.

When the position information of the target operating mechanism is that the target operating mechanism is located at the operating end of the robot arm 1400, the inspection operating robot 1000 issues corresponding (different) action instructions to the robot arm 1400 through the first control module according to the position information of the target operating mechanism, so that the robot arm 1400 performs the installation operation of the target operating mechanism or the switching operation of the target operating mechanism or does not perform the switching operation or the installation operation.

The execution process that the target operation mechanism on the mechanical arm controlled and operated by the inspection operation robot 1000 moves to the position corresponding to the target three-dimensional coordinate and executes the operation task on the target operation point includes: the inspection operation robot 1000 may also send an action instruction and a task operation instruction to the operated mechanical arm 1400 through the controller, and the operated mechanical arm 1400 receives the action instruction and the task operation instruction, moves to a position corresponding to the target three-dimensional coordinate based on the action instruction, and then performs an operation action on a target operation point (operation target) according to the operation task based on the task operation instruction, where the action instruction and the task operation instruction have an execution sequence, that is, the priority of the action instruction is greater than that of the task operation instruction, the action instruction is preferentially executed, and the task operation instruction is executed after the action instruction is executed.

Optionally, controlling the mechanical arm to perform an installation operation or a switching operation of the target operating mechanism according to the detection result of the operating mechanism, includes: if the detection result of the operating mechanism is that the operating end of the second mechanical arm is not provided with the current operating mechanism, controlling the first mechanical arm to move to the target position of the target operating mechanism, and installing the target operating mechanism to the operating end of the second mechanical arm; if the detection result of the operating mechanism is that the current operating mechanism is installed at the operating end of the second mechanical arm, a storage station detection result is obtained and used for indicating whether a storage station corresponding to the current operating mechanism in the detection tool library is occupied or not; if the storage station corresponding to the current operating mechanism in the tool library is not occupied, controlling the first mechanical arm to disassemble the current operating mechanism on the second mechanical arm through the operating mechanism for disassembly on the first mechanical arm, moving the current operating mechanism to the storage station corresponding to the current operating mechanism, and placing the current operating mechanism into the corresponding storage station; and controlling the first mechanical arm to move to the position of the target operating mechanism, installing the target operating mechanism to the operating end of the second mechanical arm through the operating mechanism for disassembly on the first mechanical arm, and controlling the second mechanical arm to perform rechecking of installation of the target operating mechanism.

If the detection result of the operating mechanism is that the operating end of the second mechanical arm is not provided with the current operating mechanism, the controller is used for detecting whether the operating structure for disassembly is arranged on the first mechanical arm 1401 or not, if so, the controller is used for controlling the first mechanical arm 1401 to move to the target position of the target operating mechanism, the target operating mechanism is arranged on the operating end of the second mechanical arm based on the operating structure for disassembly on the first mechanical arm 1401, if not, the controller is used for reading the storage station for storing the operating structure for disassembly on the tool library 1700, controlling the first mechanical arm 1401 to move to the position of the storage station for storing the operating structure for disassembly on the tool library 1700, the operating structure for disassembly is arranged on the operating end of the first mechanical arm 1401 through the first mechanical arm 1401, and the first mechanical arm 1401 after the operating structure for disassembly is arranged is controlled to move to the target position of the target operating mechanism through the controller, the target manipulation mechanism is attached to the manipulation end of the second robot arm based on the manipulation structure for detachment on the first robot arm 1401.

If the detection result of the operating mechanism is that the current operating mechanism is installed at the operating end of the second mechanical arm, detecting whether an operating structure for disassembly is installed on the first mechanical arm 1401 or not through the controller, and if the controller detects that the operating structure for disassembly is installed on the first mechanical arm 1401, obtaining a storage station detection result which is used for indicating whether a storage station corresponding to the current operating mechanism in the detection tool library is occupied or not; if the storage station corresponding to the current operating mechanism in the tool library is not occupied, controlling the first mechanical arm to disassemble the current operating mechanism on the second mechanical arm through the operating mechanism for disassembly on the first mechanical arm, moving the current operating mechanism to the storage station corresponding to the current operating mechanism, and placing the current operating mechanism into the corresponding storage station; and controlling the first mechanical arm to move to the position of the target operating mechanism, installing the target operating mechanism to the operating end of the second mechanical arm through the operating mechanism for disassembly on the first mechanical arm, and controlling the second mechanical arm to perform rechecking of installation of the target operating mechanism so as to determine that the installation of the target operating mechanism on the second mechanical arm is completed. If the controller detects that the operating structure for disassembly is not mounted on the first mechanical arm 1401, reading a storage station on the tool library 1700, where the operating structure for disassembly is stored, by the controller, controlling the first mechanical arm 1401 to move to a position on the tool library 1700, where the storage station is stored, where the operating structure for disassembly is stored, mounting the operating structure for disassembly on the operating end of the first mechanical arm 1401 by the first mechanical arm 1401, and acquiring a storage station detection result, where the storage station detection result is used for indicating whether the storage station corresponding to the current operating mechanism in the tool library is occupied; if the storage station corresponding to the current operating mechanism in the tool library is not occupied, controlling the first mechanical arm to disassemble the current operating mechanism on the second mechanical arm through the operating mechanism for disassembly on the first mechanical arm, moving the current operating mechanism to the storage station corresponding to the current operating mechanism, and placing the current operating mechanism into the corresponding storage station; and controlling the first mechanical arm to move to the position of the target operating mechanism, installing the target operating mechanism to the operating end of the second mechanical arm through the operating mechanism for disassembly on the first mechanical arm, and controlling the second mechanical arm to perform rechecking of installation of the target operating mechanism so as to determine that the installation of the target operating mechanism on the second mechanical arm is completed. If the storage station 1701 corresponding to the current operating mechanism in the tool library 1700 is (illegally) occupied, the operation process is ended and the first control module sends a warning message.

The executing process of controlling the second mechanical arm to perform the rechecking of the installation of the target operating mechanism may include: the installation position of the target operation mechanism on the second mechanical arm can be shot through the three-dimensional camera 1200 and/or the two-dimensional camera 1300, and the installation condition of the target operation mechanism is detected through the shot image, so that the second mechanical arm is controlled to perform recheck of the installation of the target operation mechanism.

And S203, when the completion of the operation action is detected, acquiring an operation target image, performing state analysis on the operation target image and confirming that the operation target is operated completely, wherein the operation target image is used for indicating the two-dimensional camera to shoot an image obtained by the operation target corresponding to the target operation point after the operation action is completed.

Specifically, when it is detected that the operation motion is completed, the robot arm 1400 is controlled to move so that the two-dimensional camera moves to an ideal reference point of the camera; acquiring an operation target image shot by a two-dimensional camera positioned at an ideal reference point of the camera, wherein the operation target image is used for indicating an image of an operation target corresponding to a target operation point after an operation action is finished; and performing state analysis on the operation target image to obtain a state analysis result, and confirming that the operation target completes the operation according to the state analysis result.

When the first control module detects that the operation action is completed, a third motion instruction is sent to the second control module or the third control module, the second control module controls the first mechanical arm 1401 to move according to the third motion instruction, so that the first two-dimensional camera 1301 moves to an ideal camera reference point, or the third control module controls the second mechanical arm 1402 to move according to the third motion instruction, so that the second two-dimensional camera 1302 moves to the ideal camera reference point; the first control module sends a fourth shooting instruction to the first two-dimensional camera 1301 or the second two-dimensional camera 1302 at the ideal reference point of the camera, the first two-dimensional camera 1301 or the second two-dimensional camera 1302 at the ideal reference point of the camera shoots an operation target after the operation action is finished based on the fourth shooting instruction to obtain an operation target image, the operation target image is sent to the first control module, the first control module receives the first control module, performs operation target state recognition on the first control module to obtain a state recognition result, compares the state recognition result with preset operation finishing state information and analyzes, if the state recognition result is consistent with the preset operation finishing state information, the first control module confirms that the operation target is finished, if the state recognition result is inconsistent with the preset operation finishing state information, the first control module sends warning information or obtains the current operation step of the operation target, and obtaining the non-operated operation steps according to the current operation steps of the operation target, and sending the continuous execution instruction comprising the non-operated operation steps to the second control module or the third control module, wherein the second control module controls the first mechanical arm 1401 to continuously execute according to the continuous execution instruction, or the third control module controls the second mechanical arm 1402 to continuously execute according to the continuous execution instruction.

Specifically, the inspection robot 1000 may further perform step S206: and acquiring a polling target shooting result of the binocular camera 1800, and performing instrument reading identification or temperature detection on the polling target according to the polling target shooting result.

When the inspection target is an operation target in the operation task, the step S206 may be executed between the steps S203 to S204: the inspection operation robot 1000 sends a fifth shooting instruction to a visible light camera 1801 in the binocular camera 1800 through a fifth control module, the visible light camera 1801 shoots a current operation target based on the fifth shooting instruction to obtain an operation target shooting result, a preset image processing algorithm is called to perform image preprocessing, feature extraction, instrument identification and instrument reading extraction on the operation target shooting result to obtain an instrument reading, so that the instrument reading identification is performed on the operation target through the operation target shooting result, whether the instrument reading meets a preset operation condition or not is judged, if yes, step S204 is executed, and if not, warning information is sent; and/or the inspection operation robot 1000 sends a sixth shooting instruction to the infrared thermal imager 1802 in the binocular camera 1800 through the fifth control module, the infrared thermal imager 1802 performs infrared thermal imaging on the current operation target based on the sixth shooting instruction to obtain an operation target shooting result, a preset image processing algorithm is called to perform image preprocessing, feature extraction and temperature identification on the operation target shooting result to obtain temperature data, whether the temperature data meet preset operation conditions or not is judged, if yes, the step S204 is executed, and if not, warning information is sent.

The step S206 may be executed after the step S205: the inspection operation robot 1000 sends a seventh shooting instruction to the visible light camera 1801 in the binocular camera 1800 through the fifth control module, the visible light camera 1801 shoots a current operation target based on the fifth shooting instruction to obtain an operation target shooting result, a preset image processing algorithm is called to perform image preprocessing, feature extraction, instrument identification and instrument reading extraction on the operation target shooting result to obtain an instrument reading, so that the operation target is subjected to instrument reading identification through the operation target shooting result, whether the instrument reading meets an operation result in an operation task or not is judged, if yes, execution is stopped, if no, warning information is sent, or the current operation step of the operation target is obtained, an unoperated operation step is obtained according to the current operation step of the operation target, a continuous execution instruction including the unoperated operation step is sent to the second control module or the third control module, the second control module controls the first mechanical arm 1401 to continue executing according to the continuous execution instruction, or the third control module controls the second mechanical arm 1402 to continue executing according to the continuous execution instruction; and/or the inspection operation robot 1000 sends a seventh shooting instruction to the infrared thermal imager 1802 in the binocular camera 1800 through the fifth control module, the infrared thermal imager 1802 performs infrared thermal imaging on the current operation target based on the sixth shooting instruction to obtain an operation target shooting result, a preset image processing algorithm is called to perform image preprocessing, feature extraction and temperature identification on the operation target shooting result to obtain temperature data, whether the temperature data is greater than a preset temperature value or not is judged, if yes, warning information is sent and operation is stopped, and if not, any operation is not executed.

When the inspection target is not the operation target in the above operation task, this step S206 is to perform real-time monitoring on the inspection target. The inspection operation robot 1000 controls the visible light camera 1801 in the binocular camera 1800 through the fifth control module, performs real-time shooting on the current operation target to obtain a real-time shooting result, calls a preset image processing algorithm to perform image preprocessing, feature extraction, instrument identification and instrument reading extraction on the real-time shooting result to obtain an instrument reading, and thus, the instrument reading identification is performed on the operation target through the real-time shooting result; and/or the inspection operation robot 1000 controls the infrared thermal imager 1802 in the binocular camera 1800 to perform real-time infrared thermal imaging on the current operation target through the fifth control module to obtain a real-time shooting result, and invokes a preset image processing algorithm to perform image preprocessing, feature extraction and temperature identification on the real-time shooting result to obtain temperature data, so as to realize temperature detection on the operation target through the real-time shooting result; and carrying out real-time private cloud storage on the meter reading and/or the temperature data or sending the meter reading and/or the temperature data to a target terminal (or a target server).

In the embodiment of the invention, the three-dimensional coordinate of the target obtained by shooting the target mark point part of the target object by the three-dimensional camera and/or the two-dimensional camera is obtained, and the mechanical arm is controlled to execute the operation task on the target operation point based on the position information of the target operation mechanism and the three-dimensional coordinate of the target, so that the accuracy of the operation is ensured, the routing inspection and the operation are combined into a whole, the replacement and the operation determination of the operation structure are realized, the three-dimensional coordinate target and the operation target are confirmed before operation by two position target methods of adjusting the mechanical arm and adjusting the body, the operation target state is confirmed after operation to recheck, and a main mechanical arm and an auxiliary mechanical arm (namely a first mechanical arm and a second mechanical arm) are adopted, flexibility and strength can be considered, one or both of the operation tasks can be selected to cooperate to complete complex operation actions according to different operation tasks, and a safe and reliable robot operation method is provided.

The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and which may also be a volatile computer-readable storage medium, in which a computer program is stored, which, when run on a computer, causes the computer to perform the steps of the method of operating an inspection robot.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several computer programs to enable a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. The inspection operation robot is characterized in that the inspection operation robot (1000) comprises a robot body (1100), a three-dimensional camera (1200), a two-dimensional camera (1300), a mechanical arm (1400) and a controller which is electrically connected with the three-dimensional camera (1200), the two-dimensional camera (1300) and the mechanical arm (1400); wherein the content of the first and second substances,

the three-dimensional camera (1200) is mounted to the robot body (1100);

the two-dimensional camera (1300) is mounted to the robotic arm (1400);

2. The inspection robot of claim 1, wherein the robotic arm (1400) includes a first robotic arm (1401) and a second robotic arm (1402), the first robotic arm (1401) being a six-axis robotic arm and the second robotic arm (1402) being a four-axis robotic arm.

3. The inspection robot of claim 2, wherein the first robotic arm (1401) is configured to exchange operating mechanisms on the second robotic arm (1402); the operating mechanism for disassembly on the first mechanical arm (1401) is used for disassembling and replacing the operating mechanism on the second mechanical arm (1402).

4. The inspection robot according to claim 2, wherein the first robot arm (1401) is configured to move to a target position of the target actuator and mount the target actuator to the manipulation end when it is detected that the manipulation end is not mounted with the current actuator;

5. The inspection operation robot according to claim 1, wherein the robotic arm (1400) is configured to perform a mounting operation of the target operating mechanism or move the target operating mechanism to a position corresponding to the target three-dimensional coordinates and control the target operating mechanism to perform an operation task on the target operating point when the target operating mechanism is switched or not switched or mounted.

6. The inspection operation robot according to claim 1, wherein the three-dimensional camera (1200) is configured to capture a target marker location of a target object to obtain a three-dimensional marker location image;

7. The inspection operation robot according to claim 2, wherein the inspection operation robot (1000) further comprises a walking module (1600) electrically connected with the controller, the walking module (1600) being mounted to the robot body (1100) and used for driving the robot body (1100) to travel or rotate on the ground.

8. The inspection operation robot according to claim 1, wherein the inspection operation robot (1000) further comprises a tool magazine (1700), the tool magazine (1700) is mounted on the robot body (1100), the tool magazine (1700) is provided with a plurality of storage stations (1701), and the plurality of storage stations (1701) are used for storing an operation mechanism (1500);

9. The inspection operation robot according to claim 7, wherein the controller includes a first control module, a second control module, a third control module, and a fourth control module;

10. The inspection robot according to any one of claims 1-9, wherein the inspection robot (1000) further includes a binocular camera (1800) electrically connected to the controller, the binocular camera (1800) being used for meter reading identification of an inspection target;

the binocular camera (1800) is also used for detecting the temperature of the routing inspection target.

11. The inspection operation robot according to claim 10, wherein the controller further includes a fifth control module;

12. An inspection operation robot operating method according to any one of claims 1 to 11, the inspection operation robot comprising:

13. The inspection operation robot operation method according to claim 12, wherein the reading of the position information of the target operation mechanism corresponding to the target operation point and the controlling of the manipulator to perform the operation task on the target operation point based on the position information of the target operation mechanism and the target three-dimensional coordinates include:

14. The inspection operation robot operating method according to claim 13, wherein the controlling the robot arm to perform the mounting operation of the target operating mechanism or the switching operation of the target operating mechanism or not to perform the switching operation or the mounting operation based on the position information of the target operating mechanism includes:

15. The inspection operation robot operating method according to claim 14, wherein the controlling the robot arm to perform the mounting operation or the switching operation of the target operating mechanism according to the operating mechanism detection result includes:

16. The inspection operation robot operating method according to claim 14, wherein the controlling the robot arm to perform the mounting operation or the switching operation of the target operating mechanism according to the operating mechanism detection result includes:

17. The inspection operation robot operating method according to claim 12, wherein after reading the position information of the target operating mechanism corresponding to the target operating point and controlling the manipulator to perform the operation task on the target operating point based on the position information of the target operating mechanism and the target three-dimensional coordinates, the method further comprises:

18. The inspection operation robot operating method according to any one of claims 12 to 17, further comprising:

19. A computer-readable storage medium, having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a method of operating an inspection robot according to any one of claims 12-18.