CN111906788B - Bathroom intelligent polishing system based on machine vision and polishing method thereof - Google Patents
Bathroom intelligent polishing system based on machine vision and polishing method thereof Download PDFInfo
- Publication number
- CN111906788B CN111906788B CN202010805350.0A CN202010805350A CN111906788B CN 111906788 B CN111906788 B CN 111906788B CN 202010805350 A CN202010805350 A CN 202010805350A CN 111906788 B CN111906788 B CN 111906788B
- Authority
- CN
- China
- Prior art keywords
- polishing
- bathroom
- intelligent
- robot
- control module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
- B25J11/005—Manipulators for mechanical processing tasks
- B25J11/0065—Polishing or grinding
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B24—GRINDING; POLISHING
- B24B—MACHINES, DEVICES, OR PROCESSES FOR GRINDING OR POLISHING; DRESSING OR CONDITIONING OF ABRADING SURFACES; FEEDING OF GRINDING, POLISHING, OR LAPPING AGENTS
- B24B1/00—Processes of grinding or polishing; Use of auxiliary equipment in connection with such processes
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B24—GRINDING; POLISHING
- B24B—MACHINES, DEVICES, OR PROCESSES FOR GRINDING OR POLISHING; DRESSING OR CONDITIONING OF ABRADING SURFACES; FEEDING OF GRINDING, POLISHING, OR LAPPING AGENTS
- B24B51/00—Arrangements for automatic control of a series of individual steps in grinding a workpiece
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B24—GRINDING; POLISHING
- B24B—MACHINES, DEVICES, OR PROCESSES FOR GRINDING OR POLISHING; DRESSING OR CONDITIONING OF ABRADING SURFACES; FEEDING OF GRINDING, POLISHING, OR LAPPING AGENTS
- B24B9/00—Machines or devices designed for grinding edges or bevels on work or for removing burrs; Accessories therefor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Landscapes
- Engineering & Computer Science (AREA)
- Mechanical Engineering (AREA)
- Robotics (AREA)
- Automation & Control Theory (AREA)
- Constituent Portions Of Griding Lathes, Driving, Sensing And Control (AREA)
Abstract
The invention belongs to the field of industrial robots, and relates to a bathroom intelligent polishing system based on machine vision, which comprises a polishing machine fixing device, a polishing control device and a bathroom fixing device, wherein the polishing control device and the bathroom fixing device are fixedly arranged on the polishing machine fixing device, the polishing control device comprises a polishing robot fixedly arranged on the polishing machine fixing device, an optical camera and a pneumatic polishing device, the optical camera and the pneumatic polishing device are connected to the free end of the polishing robot, an intelligent polishing control module is fixedly arranged at the top of the longitudinal axis of the polishing robot, and the control device is arranged on the polishing machine fixing device on one side of the polishing robot.
Description
Technical Field
The invention belongs to the field of industrial robots, and relates to a bathroom intelligent polishing system based on machine vision and a polishing method thereof.
Background
The research level of the robot is directly closely related to the economic and technological level of a country, and the comprehensive strength of the country is reflected to a certain extent. At present, polishing is mainly performed manually, and due to high harm to human bodies, the polishing industry faces serious labor waste; the universal six-degree-of-freedom industrial robot can realize automatic grinding and polishing, but the equipment is expensive, and the universal six-degree-of-freedom industrial robot is difficult to be applied to the thin and good grinding and polishing industry on a large scale. In view of this, this patent develops the research and the development of burnishing and polishing robot intelligence control system, promotes the metal polishing industry equipment level of china. Therefore, the intelligent polishing system of the polishing and grinding robot has high academic value and has great practical significance.
The research on the polishing and grinding robot and the drawing system for the polishing and grinding robot by relevant colleges and scientific research institutions at home and abroad is greatly advanced, and a large amount of research results are obtained, and the research results mainly develop towards an intelligent robot polishing system which is adaptive, multi-task, capable of programming repeatedly, and capable of programming a complex mould in a polishing off-line manner and simulating movement. Practice proves that the robot has the advantages of high precision, good stability, good quality, 3-4 times of manual efficiency and the like in the grinding process, the robot can provide a higher, more reliable and wider control means, and a plurality of technical breakthroughs are still needed in the aspects of polishing and grinding control systems, three-dimensional scene construction, path planning and the like.
Disclosure of Invention
In view of the above, the invention provides a bathroom intelligent polishing system based on machine vision and a polishing method thereof, in order to solve the problems that an existing polishing robot cannot construct a three-dimensional scene and plan a polishing path in real time, so that the adaptability is not strong, and the production requirements of various, small-batch and nonstandard polishing workpieces cannot be met. The intelligent bathroom polishing system utilizes a machine vision technology, utilizes SLAM to achieve real-time three-dimensional reconstruction of a polishing workbench scene, then utilizes a computer vision technology to achieve bathroom object identification in the reconstructed three-dimensional scene, identifies three-dimensional coordinates of a bathroom region needing polishing, then maps the three-dimensional coordinates of the polishing region to a mechanical arm coordinate space under a relative coordinate system, and finally drives a mechanical arm to complete polishing of the bathroom region to be polished based on the type and the spatial position of the polishing region.
In order to achieve the purpose, the invention provides a machine vision-based bathroom intelligent polishing system which comprises a polishing machine fixing device, a polishing control device and a bathroom fixing device, wherein the polishing control device and the bathroom fixing device are fixedly arranged on the polishing machine fixing device, the polishing control device comprises a polishing robot fixedly arranged on the polishing machine fixing device, an optical camera and a pneumatic polishing device, the optical camera and the pneumatic polishing device are connected to the free end of the polishing robot, an intelligent polishing control module is fixedly arranged at the top of the longitudinal axis of the polishing robot, and a control device and a PLC control cabinet are arranged on the polishing machine fixing device on one side of the polishing robot.
The beneficial effect of this basic scheme lies in: the polishing machine fixing device is used for supporting the polishing control device and the bathroom fixing device. The polishing robot drives the optical camera and the pneumatic polishing device to scan and realize polishing tracks of bathroom workpieces to be polished through movement of the spatial position of the polishing robot, three-dimensional reconstruction is carried out on a working scene of the bathroom workpieces to be polished in real time, three-dimensional coordinates of bathroom areas to be polished are identified, then the three-dimensional coordinates of the polishing areas are mapped to a mechanical arm coordinate space of the polishing robot under a relative coordinate system, and finally the mechanical arm is driven to complete polishing of the bathroom areas to be polished based on the types and spatial positions of the polishing areas. The intelligent polishing control module sends the three-dimensional coordinates of the bathroom workpiece to be polished and the three-dimensional coordinates of the polishing area of the bathroom workpiece to be polished to the control device, and the control device controls the polishing robot to drive the pneumatic polishing device to polish the polishing area of the bathroom workpiece to be polished after receiving the three-dimensional position coordinates of the polishing area.
Furthermore, intelligence control module of polishing is NVIDIA Jetson module, and intelligence control module of polishing communicates through USB with optical camera. Has the advantages that: the polishing robot drives the optical camera to perform non-contact rotary scanning on a bathroom workpiece to be polished, the intelligent polishing control module receives image data of the bathroom workpiece to be polished, which are shot by the optical camera, three-dimensional reconstruction of a polishing scene and three-dimensional identification of the bathroom workpiece to be polished are completed, and on the basis of the three-dimensional identification of the bathroom workpiece to be polished, the intelligent polishing control module also completes positioning of a polishing area of the bathroom workpiece to be polished.
Further, the optical camera comprises a high-definition camera, a laser generator and a filter. Has the advantages that: the high-definition camera is responsible for collecting 550nm wavelength red line laser projected on the surface of the bathroom workpiece by the laser generator, and the collected line laser images are overlapped to generate a complete image. The optical filter is responsible for filtering light waves with wavelength of 550nm, and is beneficial to improving the quality of pictures acquired by the high-definition camera.
Further, the pneumatic polishing device comprises a pneumatic support frame and a polishing head which are connected with a mechanical arm of the polishing robot. Has the advantages that: the pneumatic support frame is used for providing a supporting device for the polishing head, external compressed air acts on the pneumatic support frame, and the stress of the pneumatic support frame is transmitted to the polishing head so as to act on the polishing head.
Further, the polishing machine fixing device comprises a cuboid-shaped base and a plurality of supporting legs which are uniformly and fixedly installed on the periphery of the base. Has the advantages that: the landing leg is used for supporting the polishing machine fixing device, the polishing control device and the bathroom fixing device, and has the function of adjusting the levelness of the base.
A polishing method of a bathroom intelligent polishing system based on machine vision comprises the following steps:
A. 360-degree monocular multi-view image acquisition of fixed operation scene
Fixing the relative position coordinates of the bathroom fixing device and the polishing robot, controlling the polishing robot to perform 360-degree circumambulation around the bathroom fixing device by the control device, and transmitting the shot image data to the intelligent polishing control module by the optical camera in real time for data processing;
B. three-dimensional reconstruction and identification based on intelligent polishing control module
After the intelligent polishing control module receives 360-degree image data of a group of bathroom fixing devices, based on GPU resources on the intelligent polishing control module, performing dense three-dimensional reconstruction on an operation scene where the bathroom fixing devices are located by using an LSD-SLAM algorithm; identifying the bathroom of the three-dimensional scene model by using a PointCNN network based on GPU resources on the intelligent polishing control module on the three-dimensional reconstruction result; finally, on the basis of identifying the three-dimensional bathroom model, using a fine PointCNN network to realize identification of bathroom burrs, thereby determining a spatial position coordinate of bathroom polishing;
C. force control-based vision-guided bathroom polishing control
The intelligent polishing control module transmits three-dimensional spatial coordinates of polished bathroom burrs to the control device, the control device determines polishing parameters by utilizing the recognized bathroom types and sizes of the burrs, and then the control device drives the polishing robot to control the pneumatic polishing device to polish the bathroom burrs based on the polishing parameters and the spatial position coordinates of the burrs;
D. bathroom quality of polishing judges based on visual identification
After the control device drives the polishing robot to polish all identification burrs, the control device drives the polishing robot to control the optical camera to complete 360-degree image shooting of an operation scene where the bathroom fixing device is located, the control optical camera transmits acquired image data to the intelligent polishing control module, the intelligent polishing control module completes secondary three-dimensional scene reconstruction of a polishing operation area, secondary identification is carried out on the burrs of the bathroom, and therefore evaluation of bathroom polishing quality is achieved.
Further, in the step a, the fixed coordinates of the bathroom fixing device are input into the control device, and the position coordinates of the center point of the bathroom fixing device are used as the reference point.
And further, step C, converting the recognized bathroom grinding space position coordinates into space coordinates under a robot coordinate system, and transmitting the space coordinates to the control device.
The invention has the beneficial effects that:
1. the intelligent bathroom polishing system based on the machine vision can realize self-adaptive calibration of six-axis industrial robots based on three-dimensional vision, so that the same robot can self-adaptively identify a polishing object on the same production line. And through the automatic positioning in the region of polishing, change original strategy of polishing based on fixed route, this intelligent industrial robot can realize polishing based on region to improve efficiency of polishing, prolonged the live time of the consumptive material of polishing, saved the supplementary loaded down with trivial details work of accomplishing the calibration of polishing of manual work. This bathroom intelligence system of polishing based on machine vision can carry out the three-dimensional coordinate of real-time adjustment region of polishing according to actual scene, and traditional robot polishing technique can't accomplish the design of polishing the tactics according to specific bathroom type, and the parameter of all arms all belongs to empirical data.
2. According to the intelligent bathroom polishing system based on the machine vision, when a bathroom to be polished is placed on a robot operation table, the robot automatically controls a high-definition camera of a front-end mechanical arm to shoot the operation table on a bathroom fixing device at 360 degrees, then the intelligent control system with GPU computing power is used for achieving three-dimensional reconstruction of the operation table, and a multilayer convolutional neural network is used for achieving recognition of the bathroom on the operation table on the intelligent control system. On the basis, the bathroom polishing area is identified and spatially positioned by utilizing a fine adjustment technology. The intelligent control system converts the spatial coordinates of the polishing area which is positioned by the intelligent control system into the spatial position coordinates of a mechanical arm coordinate system, then automatically controls the mechanical arm force control system according to the type of the bathroom, and finally realizes the self-adaptive feedback of polishing force control according to visual data to realize the automatic adjustment of the polishing area.
3. The bathroom intelligent polishing system based on the machine vision can realize real-time evaluation of the service operation quality of the working robot based on the machine vision, and can judge the quality of polishing service and improve the final polishing yield by directly comparing the geometric characteristics of three-dimensional point clouds before and after polishing.
4. The intelligent bathroom polishing system based on the machine vision disclosed by the invention is accelerated based on an NVIDIA Jetson algorithm, a complex LSD-SLAM algorithm is accelerated on the existing working robot body by utilizing an NVIDIA Jetson platform, and a PointCNN network is deployed on the NVIDIA Jetson platform, so that the real-time performance of the provided intelligent industrial robot is ensured.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.
Drawings
For the purposes of promoting a better understanding of the objects, aspects and advantages of the invention, reference will now be made to the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 is a schematic structural view of a machine vision-based intelligent bathroom polishing system according to the present invention;
FIG. 2 is a flow chart of a training algorithm of bath identification PointCNN network in the machine vision-based intelligent bathroom polishing system of the present invention;
FIG. 3 is a flow chart of a training algorithm for identifying PointCNN network for burr recognition of bath in the intelligent bathroom polishing system based on machine vision according to the present invention;
FIG. 4 is a flow chart of an identification algorithm of the intelligent bathroom polishing system based on machine vision according to the present invention;
fig. 5 is a polishing flow chart of the bathroom intelligent polishing system based on machine vision according to the invention.
Reference numerals: polishing machine fixing device 1, bathroom fixing device 2, optical camera 3, pneumatic grinding device 4, polishing robot 5, intelligent polishing control module 6, controlling means 7, bathroom work piece 8, PLC switch board 9.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention in a schematic way, and the features in the following embodiments and examples may be combined with each other without conflict.
Wherein the showings are for the purpose of illustrating the invention only and not for the purpose of limiting the same, and in which there is shown by way of illustration only and not in the drawings in which there is no intention to limit the invention thereto; to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there is an orientation or positional relationship indicated by terms such as "upper", "lower", "left", "right", "front", "rear", etc., based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not an indication or suggestion that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes, and are not to be construed as limiting the present invention, and the specific meaning of the terms may be understood by those skilled in the art according to specific situations.
In the patent, the SLAM (simultaneous Localization and mapping) technology refers to synchronous positioning and mapping of a robot, namely the robot starts to move from an unknown position in an unknown environment, self-positioning is carried out according to position estimation and a map in the moving process, and meanwhile, an incremental map is built on the basis of self-positioning, so that the autonomous positioning and navigation of the robot are realized.
As shown in fig. 1, the intelligent bathroom polishing system based on machine vision includes a polishing machine fixing device 1, and a polishing control device and a bathroom fixing device 2 which are fixedly installed on the polishing machine fixing device, wherein the polishing machine fixing device is used for supporting the polishing control device and the bathroom fixing device 2. The polishing machine fixing device comprises a cuboid-shaped base and a plurality of supporting legs which are uniformly and fixedly installed on the periphery of the base, and the supporting legs are used for supporting the polishing machine fixing device, the polishing control device and the bathroom fixing device and have the function of adjusting the levelness of the base.
The bathroom fixing device is provided with a bathroom workpiece to be polished. The bathroom fixing device is used for supporting and fixing a bathroom workpiece to be polished at a corresponding position. The bathroom fixing device comprises a bottom fixing frame fixed on a polishing machine fixing device base, supporting columns fixed at four corners of the base fixing frame and a top fixing frame fixed on the supporting columns, wherein a clamping rod with a clamping head at the upper part is fixedly installed on the top fixing frame. After the polishing of the bathroom workpiece to be polished is completed, the clamp heads on the clamping rods are opened to take away the polished bathroom workpiece for replacement, so that the bathroom workpiece to be polished is simple and efficient to circulate.
The polishing control device comprises a polishing robot 5 fixedly installed on a base of a polishing machine fixing device, an optical camera 3 and a pneumatic polishing device 4 which are connected to the free end of the polishing robot, the polishing robot is an industrial six-joint robot, the optical camera and the pneumatic polishing device are driven to scan a bathroom workpiece 8 to be polished and realize polishing tracks mainly through movement of the spatial position of the optical camera and the pneumatic polishing device, three-dimensional reconstruction is carried out on a working scene of the bathroom workpiece to be polished in real time, three-dimensional coordinates of a bathroom region to be polished are identified, then the three-dimensional coordinates of the polishing region are mapped to the space of a mechanical arm coordinate of the polishing robot under a relative coordinate system, and finally the mechanical arm is driven to complete polishing of the bathroom region to be polished based on the type of the polishing region and the spatial position of the polishing region.
The top fixed mounting of 5 axis of ordinates of robot that polish has intelligence control module 6 of polishing, and intelligence control module of polishing is NVIDIA Jetson module, and intelligence control module 6 of polishing communicates through USB with optical camera. The polishing robot 5 drives the optical camera to perform non-contact rotary scanning on the bathroom workpiece to be polished, the intelligent polishing control module 6 receives image data of the bathroom workpiece to be polished of a plurality of groups shot by the optical camera 3, three-dimensional reconstruction of a polishing scene and three-dimensional identification of the bathroom workpiece to be polished are completed, and on the basis of the three-dimensional identification of the bathroom workpiece to be polished, the intelligent polishing control module also completes positioning of a polishing area of the bathroom workpiece to be polished. The intelligent polishing control module sends the three-dimensional coordinates of the bathroom workpiece to be polished and the three-dimensional coordinates of the polishing area of the bathroom workpiece to be polished to the control device 7, and the control device controls the polishing robot to drive the pneumatic polishing device to polish the polishing area of the bathroom workpiece after receiving the three-dimensional position coordinates of the polishing area.
The pneumatic grinding device 4 comprises a pneumatic support frame and a grinding head, the pneumatic support frame is connected with a mechanical arm of the grinding robot, the pneumatic support frame is used for providing a support device for the grinding head, external compressed air acts on the pneumatic support frame, and the pneumatic support frame is stressed and conducted to the grinding head so as to act on the grinding head. The gas pressure regulating valve is used for regulating the flow of the compressed gas and outputting pressures with different sizes, so that the purposes of supporting the polishing head and regulating the pressure are achieved. The polishing head is a tail end executing mechanism, and polishing sheets with different specifications can be conveniently selected according to different polishing requirements, so that different polishing processes are realized.
This bathroom intelligence system of polishing based on machine vision utilizes computer vision and artificial intelligence technique to realize the bathroom discernment under the scene of polishing, the regional location of polishing to realize the intelligence of robot arm and polish, like the planning of the route of polishing, the self-adaptation feedback of the dynamics of polishing. The core problem that solves is: (1) identifying and positioning a bathroom polishing area based on computer vision; (2) intelligent driving of the robot arm based on the type of the polishing area and the spatial position of the polishing area.
A polishing method of a bathroom intelligent polishing system based on machine vision comprises the following steps:
A. 360-degree monocular multi-view image acquisition of fixed operation scene
Fixing the relative position coordinates of the bathroom fixing device and the polishing robot, inputting the fixed coordinates of the bathroom fixing device into the control device, taking the position coordinate of the central point of the bathroom fixing device as a reference point, controlling the polishing robot to detour around the bathroom fixing device by 360 degrees by the control device, simultaneously driving the optical camera to photograph the bathroom fixing device according to a set photographing rate by the control device, and transmitting photographed image data to the intelligent polishing control module by the optical camera in real time for data processing;
B. three-dimensional reconstruction and identification based on intelligent polishing control module
After the intelligent polishing control module receives 360-degree image data of a group of bathroom fixing devices, based on GPU resources on the intelligent polishing control module, dense three-dimensional reconstruction is carried out on an operation scene where the bathroom fixing devices are located by using a Large-Scale Direct cellular SLAM algorithm (LSD-SLAM algorithm); identifying the bathroom of the three-dimensional scene model by using a PointCNN network based on GPU resources on the intelligent polishing control module on the three-dimensional reconstruction result; finally, on the basis of identifying the three-dimensional bathroom model, using a fine PointCNN network to realize identification of bathroom burrs, thereby determining a spatial position coordinate of bathroom polishing;
C. force control-based vision-guided bathroom polishing control
The intelligent polishing control module transmits three-dimensional spatial coordinates of polished bathroom burrs to the control device, the control device determines polishing parameters by utilizing the recognized bathroom types and sizes of the burrs, and then the control device drives the polishing robot to control the pneumatic polishing device to polish the bathroom burrs based on the polishing parameters and the spatial position coordinates of the burrs;
D. bathroom quality of polishing judges based on visual identification
After the control device drives the polishing robot to polish all identification burrs, the control device drives the polishing robot to control the optical camera to complete 360-degree image shooting of an operation scene where the bathroom fixing device is located, the control optical camera transmits acquired image data to the intelligent polishing control module, the intelligent polishing control module completes secondary three-dimensional scene reconstruction of a polishing operation area, secondary identification is carried out on the burrs of the bathroom, and therefore evaluation of bathroom polishing quality is achieved.
This patent bathroom intelligence system of polishing based on machine vision's algorithm divides into two aspects: training algorithms and recognition algorithms.
The training algorithm utilizes the existing data set to realize the identification of the bathroom and the identification of the burr, wherein the flow of the training algorithm of the bathroom identification PointCNN network is shown in figure 2, and the flow of the training algorithm of the bathroom burr identification PointCNN network is shown in figure 3.
The identification algorithm utilizes an LSD-SLAM algorithm, a bathroom PointCNN network and a bathroom burr PointCNN network to realize three-dimensional scene reconstruction, bathroom identification and burr identification of a polishing operation area, and the specific algorithm flow is shown in figure 4.
As shown in fig. 5, the working principle of the bathroom intelligent polishing system based on machine vision is as follows:
(1) rotation control is carried out on JT1 and JT6 motors of a six-axis industrial Robot through a grinding Robot software platform (ROS), and 360-degree monocular multi-view images of the grinding Robot based on a fixed working scene are acquired. And then carrying out dense three-dimensional reconstruction on the obtained scene image by using a Large-Scale Direct simple spatial SLAM (LSD-SLAM), thereby constructing a globally consistent industrial robot grinding three-dimensional scene. In the specific implementation process, ROS is used for completing relevant control of three-dimensional scene reconstruction, and acceleration of an LSD-SLAM algorithm based on NVIDIA Jetson is carried out, so that the requirement of the six-axis industrial robot on real-time performance is met.
(2) After a grinding three-dimensional scene of the industrial robot is built, a PointCNN model is used for carrying out three-dimensional grinding object recognition on the three-dimensional scene model. In order to better exert the advantages of a PointCNN deep learning model, a three-dimensional scene data set facing the polishing field is constructed, polishing objects in the three-dimensional scene are labeled with attributes such as categories and shapes through manual labeling and automatic tools, and input features are weighted and rearranged simultaneously by using a multilayer perceptron in the training process, so that the characteristics of point cloud irregularity and disorder are overcome, and point cloud target identification based on a convolutional neural network is realized. In order to realize the automatic identification of the polishing area, on the basis of realizing the identification of the polishing object, the polishing area of the polishing object is further positioned by using PointCNN through constructing a burr area data set of the polishing object. In a specific implementation process, the NVIDIA Jetson is adopted to accelerate the network, so that the network can be deployed on the robot body, and the provided intelligent robot has stronger practicability.
(3) The recognized polishing area space coordinate is converted into a space coordinate under a robot coordinate system, then the converted space coordinate is sent to a robot control system, and the polishing force control parameter of the robot is determined according to the recognized polishing bathroom type, so that the robot can be automatically polished based on visual recognition.
(4) According to the identification and the positioning of the polishing area, after the intelligent polishing robot finishes polishing based on the area, the technology of vision intelligent perception three-dimensional scene construction based on the LSD-SLAM is used for carrying out three-dimensional reconstruction on the polished scene, and the identification and the point cloud acquisition of the polished object are realized by using the PointCNN network. And then, carrying out three-dimensional space geometric characteristic matching on the polishing object polishing area before polishing and the polishing object polishing area after polishing, and judging the completeness of the to-be-polished area. If the polishing area is low in finish degree, the system drives the industrial robot to perform secondary polishing.
Finally, the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all of them should be covered by the claims of the present invention.
Claims (8)
1. A polishing method of a bathroom intelligent polishing system based on machine vision is characterized by comprising the following steps:
A. 360-degree monocular multi-view image acquisition of fixed operation scene
Fixing the relative position coordinates of the bathroom fixing device and the polishing robot, controlling the polishing robot to perform 360-degree circumambulation around the bathroom fixing device by the control device, and transmitting the shot image data to the intelligent polishing control module by the optical camera in real time for data processing;
B. three-dimensional reconstruction and identification based on intelligent polishing control module
After the intelligent polishing control module receives 360-degree image data of a group of bathroom fixing devices, based on GPU resources on the intelligent polishing control module, performing dense three-dimensional reconstruction on an operation scene where the bathroom fixing devices are located by using an LSD-SLAM algorithm; identifying the bathroom of the three-dimensional scene model by using a PointCNN network based on GPU resources on the intelligent polishing control module on the three-dimensional reconstruction result; finally, on the basis of identifying the three-dimensional bathroom model, using a fine PointCNN network to realize identification of bathroom burrs, thereby determining a spatial position coordinate of bathroom polishing;
C. force control-based vision-guided bathroom polishing control
The intelligent polishing control module transmits three-dimensional spatial coordinates of polished bathroom burrs to the control device, the control device determines polishing parameters by utilizing the recognized bathroom types and sizes of the burrs, and then the control device drives the polishing robot to control the pneumatic polishing device to polish the bathroom burrs based on the polishing parameters and the spatial position coordinates of the burrs;
D. bathroom quality of polishing judges based on visual identification
After the control device drives the polishing robot to polish all identification burrs, the control device drives the polishing robot to control the optical camera to complete 360-degree image shooting of an operation scene where the bathroom fixing device is located, the control optical camera transmits acquired image data to the intelligent polishing control module, the intelligent polishing control module completes secondary three-dimensional scene reconstruction of a polishing operation area, secondary identification is carried out on the burrs of the bathroom, and therefore evaluation of bathroom polishing quality is achieved.
2. The polishing method of claim 1, wherein the fixed coordinates of the bathroom fixture are inputted into the controller in step a, and the coordinates of the center point of the bathroom fixture are used as the reference point.
3. The polishing method of the machine vision based intelligent polishing system for bathrooms as claimed in claim 2, wherein the step C converts the recognized coordinates of the spatial polishing position of the bathrooms into the coordinates of the space in the robot coordinate system and transmits the coordinates to the control device.
4. The intelligent bathroom polishing system used in the polishing method of the intelligent bathroom polishing system based on machine vision according to claim 1, comprising a polishing machine fixing device, and a polishing control device and a bathroom fixing device fixedly installed on the polishing machine fixing device, wherein the polishing control device comprises a polishing robot fixedly installed on the polishing machine fixing device, an optical camera and a pneumatic polishing device connected to a free end of the polishing robot, the top of a longitudinal axis of the polishing robot is fixedly installed with an intelligent polishing control module, and the polishing machine fixing device on one side of the polishing robot is installed with a control device and a PLC control cabinet.
5. The machine vision based bathroom intelligent polishing system of claim 4, wherein the intelligent polishing control module is a NVIDIA Jetson module, and the intelligent polishing control module communicates with the optical camera through USB.
6. The machine vision based bathroom intelligent finishing system of claim 5, wherein optical camera comprises a high definition camera, a laser generator and a filter.
7. The machine vision based bathroom intelligent finishing system of claim 6, wherein the pneumatic finishing device comprises a pneumatic support frame connected with a finishing robot mechanical arm, a finishing head.
8. The machine vision based bathroom intelligent finishing system of claim 7, wherein the finishing machine fixture comprises a rectangular parallelepiped base and a plurality of legs uniformly and fixedly mounted around the base.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010805350.0A CN111906788B (en) | 2020-08-12 | 2020-08-12 | Bathroom intelligent polishing system based on machine vision and polishing method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010805350.0A CN111906788B (en) | 2020-08-12 | 2020-08-12 | Bathroom intelligent polishing system based on machine vision and polishing method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111906788A CN111906788A (en) | 2020-11-10 |
| CN111906788B true CN111906788B (en) | 2021-09-14 |
Family
ID=73284274
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010805350.0A Active CN111906788B (en) | 2020-08-12 | 2020-08-12 | Bathroom intelligent polishing system based on machine vision and polishing method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111906788B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113001381A (en) * | 2021-03-23 | 2021-06-22 | 合肥中科重明科技有限公司 | Surface polishing device for 3D printing blank and operation method |
| CN113510702B (en) * | 2021-06-16 | 2022-08-02 | 扬州鹏顺散热器有限公司 | Industrial robot based on laser vision guide |
| CN114367880A (en) * | 2021-12-30 | 2022-04-19 | 浙江庆元万美文具有限公司 | Production equipment for surface finish of pencil and manufacturing method thereof |
| CN114580846B (en) * | 2022-01-28 | 2023-03-24 | 广东开放大学（广东理工职业学院） | Real-time processing path correction method combining vision and digital twinning |
| CN116442249B (en) * | 2023-06-19 | 2023-08-18 | 广东隆崎机器人有限公司 | Assembly control method, assembly device, and computer-readable storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102007009950A1 (en) * | 2007-03-01 | 2008-09-04 | Maschinen- Und Stahlbau Julius Lippert Gmbh & Co. Kg | Finishing assembly for cast ceramic bathroom products has robotic arm fitted with polishing tool |
| CN103995934A (en) * | 2014-05-22 | 2014-08-20 | 北京铭隆世纪科技有限公司 | Automatic polishing method and device |
| CN107378780A (en) * | 2017-07-19 | 2017-11-24 | 江苏大学 | A kind of robot casting grinding adaptive approach of view-based access control model system |
| CN109483369A (en) * | 2018-12-13 | 2019-03-19 | 中国船舶重工集团公司第七六研究所 | A kind of robot polishing system and its control method with 3D vision |
| CN110355757A (en) * | 2019-07-05 | 2019-10-22 | 佛山智能装备技术研究院 | Body feeling interaction system based on 3D vision |
| CN111230862A (en) * | 2020-01-10 | 2020-06-05 | 上海发那科机器人有限公司 | Handheld workpiece deburring method and system based on visual recognition function |
| CN210939628U (en) * | 2019-11-06 | 2020-07-07 | 珠海汉迪自动化设备有限公司 | Automatic processing device for bathtub |
| CN111376111A (en) * | 2018-12-30 | 2020-07-07 | 广州智信科技有限公司 | Polishing method and polishing device |
-
2020
- 2020-08-12 CN CN202010805350.0A patent/CN111906788B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102007009950A1 (en) * | 2007-03-01 | 2008-09-04 | Maschinen- Und Stahlbau Julius Lippert Gmbh & Co. Kg | Finishing assembly for cast ceramic bathroom products has robotic arm fitted with polishing tool |
| CN103995934A (en) * | 2014-05-22 | 2014-08-20 | 北京铭隆世纪科技有限公司 | Automatic polishing method and device |
| CN107378780A (en) * | 2017-07-19 | 2017-11-24 | 江苏大学 | A kind of robot casting grinding adaptive approach of view-based access control model system |
| CN109483369A (en) * | 2018-12-13 | 2019-03-19 | 中国船舶重工集团公司第七六研究所 | A kind of robot polishing system and its control method with 3D vision |
| CN111376111A (en) * | 2018-12-30 | 2020-07-07 | 广州智信科技有限公司 | Polishing method and polishing device |
| CN110355757A (en) * | 2019-07-05 | 2019-10-22 | 佛山智能装备技术研究院 | Body feeling interaction system based on 3D vision |
| CN210939628U (en) * | 2019-11-06 | 2020-07-07 | 珠海汉迪自动化设备有限公司 | Automatic processing device for bathtub |
| CN111230862A (en) * | 2020-01-10 | 2020-06-05 | 上海发那科机器人有限公司 | Handheld workpiece deburring method and system based on visual recognition function |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111906788A (en) | 2020-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111906788B (en) | Bathroom intelligent polishing system based on machine vision and polishing method thereof | |
| CN109483369B (en) | Robot polishing system with three-dimensional vision and control method thereof | |
| CN104369188B (en) | Based on workpiece gripper device and the method for machine vision and ultrasonic sensor | |
| CN103831695B (en) | Large-scale free form surface robot polishing system | |
| CN106041946B (en) | Image-processing-based robot polishing production method and production system applying same | |
| CN109623656A (en) | Mobile dual robot collaboration grinding device and method based on thickness on-line checking | |
| CN109702290B (en) | Steel plate groove cutting method based on visual identification | |
| CN105499953A (en) | Automobile engine piston and cylinder block assembly system based on industrial robot and method thereof | |
| CN210452170U (en) | Flexible intelligent polishing system of robot based on six-dimensional force sensor | |
| CN110281108A (en) | A kind of robot flexibility intelligence polishing system based on six-dimension force sensor | |
| CN111055293A (en) | High-precision constant-force grinding method for industrial robot based on curved surface self-adaption | |
| CN109202912A (en) | A method of objective contour point cloud is registrated based on monocular depth sensor and mechanical arm | |
| CN111067197A (en) | Robot sole dynamic gluing system and method based on 3D scanning | |
| CN110293559B (en) | Installation method for automatically identifying, positioning and aligning | |
| CN106144524B (en) | CCD visual positioning method and device used in high-speed motion | |
| CN111923053A (en) | Industrial robot object grabbing teaching system and method based on depth vision | |
| WO2021050646A1 (en) | Robot-mounted moving device, system, and machine tool | |
| CN100361792C (en) | Controlling system of movable manipulator | |
| CN113276106A (en) | Climbing robot space positioning method and space positioning system | |
| CN112139860A (en) | Flexible track output method based on power-controlled robot polishing and implementation device thereof | |
| CN115026683B (en) | Aviation blade grinding and polishing device based on multi-robot cooperation and control method | |
| CN113246142A (en) | Measuring path planning method based on laser guidance | |
| CN113601333A (en) | Intelligent flexible polishing method, device and equipment | |
| CN107745123B (en) | 3D printing system and method for equipment part imitation | |
| CN114454060A (en) | Robot self-adaptive curved surface tracking constant-force grinding and polishing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |