CN114147706A - Cooperative robot remote monitoring system and method based on digital twin - Google Patents

Cooperative robot remote monitoring system and method based on digital twin Download PDF

Info

Publication number
CN114147706A
CN114147706A CN202111415248.0A CN202111415248A CN114147706A CN 114147706 A CN114147706 A CN 114147706A CN 202111415248 A CN202111415248 A CN 202111415248A CN 114147706 A CN114147706 A CN 114147706A
Authority
CN
China
Prior art keywords
data
real
time
cooperative robot
virtual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111415248.0A
Other languages
Chinese (zh)
Inventor
褚明
王宁
张歆悦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Posts and Telecommunications
Original Assignee
Beijing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Posts and Telecommunications filed Critical Beijing University of Posts and Telecommunications
Priority to CN202111415248.0A priority Critical patent/CN114147706A/en
Publication of CN114147706A publication Critical patent/CN114147706A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1602Programme controls characterised by the control system, structure, architecture
    • B25J9/1605Simulation of manipulator lay-out, design, modelling of manipulator
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1628Programme controls characterised by the control loop
    • B25J9/1653Programme controls characterised by the control loop parameters identification, estimation, stiffness, accuracy, error analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

The invention provides a digital twin-based cooperative robot remote monitoring system and a method. Compared with the traditional remote monitoring system, the invention has the following characteristics: the real-time performance is realized, the data come from the sensor, and the state of the monitored object can be more accurately reflected; the data is persistent, and the data can be persisted into a database finally, so that the data can be analyzed and processed conveniently; the time delay is low, the virtual model is simplified, the resource and time consumption of real-time rendering of the system is reduced, and the real-time performance is enhanced; the invention adopts a fuzzy comprehensive evaluation method to evaluate the health degree of the robot in real time, does not need a large amount of historical data training models, and has quick evaluation response. The invention can be applied to the remote monitoring of the man-machine co-fusion scene of complex equipment, can master the equipment state in real time, can early warn in time and can reduce the occurrence of safety accidents.

Description

Cooperative robot remote monitoring system and method based on digital twin
Technical Field
The invention relates to the technical field of digital twin and remote monitoring, in particular to a cooperative robot remote monitoring system and method based on digital twin.
Background
The traditional robot completes operation tasks in a specific working space according to pre-programmed program instructions under the condition of unmanned operation, and has the defects of large size, inconvenient installation and transportation, long running and debugging time, high price and the like. Moreover, for safety reasons, such robots are usually placed in independent working spaces, and safety protection facilities need to be set up again when working scenes are switched, which all cause obvious restrictions on the application of the robot technology in enterprises. Compared with the traditional industrial robot, the cooperative robot has the advantages of small occupied space, convenience in installation, good safety, capability of dragging teaching and the like. In the actual production process, the cooperative robot and the human can complete high-difficulty operation in the same working space, and the working mode of man-machine integration is realized. Compared with the traditional robot, the cooperative robot is more complex equipment, and the internal structure of the cooperative robot is more diverse and has higher intelligent degree.
The cooperative robot needs higher requirements in the aspects of design, test, maintenance and the like due to the working attribute of cooperation with people, so that the cost of the cooperative robot in the whole life cycle of design, development, test, operation, maintenance and the like is greatly increased, meanwhile, the complexity of equipment greatly increases the probability of occurrence of faults, performance degradation and functional failure, the working efficiency of the robot is seriously influenced, unpredictable dangers can be even caused to people due to the working attribute of cooperation with people, and the monitoring and evaluation of the state of the equipment and the like gradually become a difficult point.
The emergence of the digital twin technology provides a new idea for solving the problem of equipment monitoring, and the digital twin is to establish and simulate a physical entity, a process or a system in an information platform. By means of the digital twin, the state of the physical entity can be known on the information platform, and the predefined interface element inside the physical entity can be controlled. The digital twin is a concept in the Internet of things, and a digital simulation is established in the information platform by integrating feedback data of physical entities. The simulation automatically makes corresponding changes as the physical entity changes based on the feedback. Ideally, the digital twin can learn itself from the multiple feedback source data, presenting the true status of the physical entity in the digital world in near real time. Therefore, aiming at the analysis, a robot remote monitoring system based on the digital twin is designed.
Disclosure of Invention
The invention provides a digital twin-based remote monitoring system aiming at the defects that the existing remote monitoring system is poor in real-time performance and needs to be judged by relying on a large amount of expert experience, the digital twin system is based on the concept of a five-dimensional model, and the five-dimensional model is divided into five layers: physical entities, virtual models, functional services, data, connections. The physical entity is an object to be researched, the virtual model is a digital model established aiming at the physical entity, the functional service is an extended function brought by the system, such as state monitoring and visual display of virtual reality and the like, the data is data transmitted in the whole system and collected from the physical entity, and the connection is the connection among all layers. Aiming at a data protection layer for data acquisition, storage and transmission of the entity mechanical arm, a data acquisition method and a transmission flow are designed, and a data acquisition interface is compiled, so that a data source guarantee is provided for the whole digital twin system; modeling the solid mechanical arm by using a three-dimensional development tool (such as unity) and simplifying and quickly rendering the model by using a model simplification algorithm and a model simplification algorithm fusion, so that the digital model can truly and quickly reflect the solid mechanical arm; and establishing real-time contact with a database based on a three-dimensional development tool (such as unity) platform, completing real-time transmission of data, and realizing establishment of the whole system. And finally, realizing the production of a digital model scene based on a three-dimensional development tool (such as unity) platform fusion virtual reality technology, and presenting the whole digital twin system in a virtual reality mode.
Data acquisition and transmission: the corner data is obtained by reading the data record of the encoder of the motor through the programming interface, the data of voltage, current, temperature, rotating speed and the like are collected by the sensor of the cooperative robot, and then the data of the servo motor controller is read through the programming interface.
Data storage: the method comprises the steps of designing a database table, wherein data in the database need to be consistent with data needed by a virtual model, an object in the virtual model is determined by coordinate axis position data of x, y and z relative to a world coordinate system and rotation angle data of the x, y and z axes and the world coordinate system, therefore, when the database stores data, the database table needs to be designed by the same field as that in the virtual model, and each record stores state information of a mechanical arm at a certain moment in the data storage process.
Data transmission: the transfer of data from the database to the digital mockup is one of the most critical steps of the overall digital twin system. The method comprises the steps of utilizing scripts written in a three-dimensional development tool (such as unity) to be mounted on a corresponding object to achieve a corresponding data driving function, writing a database connection program through C # and other programming languages, adding database connection dependency, further opening a connection channel between a database and the three-dimensional development tool (such as unity) and achieving data transmission.
Establishing and simplifying a digital model of the cooperative mechanical arm: the method comprises the steps of establishing a three-dimensional digital model in a three-dimensional development tool (such as unity) and simplifying a virtual three-dimensional model by adopting a simplified edge folding algorithm based on a quadratic error matrix, so that the consumption of system resources in a real-time rendering process can be reduced, the display speed is accelerated and the monitoring efficiency is improved in a scene consisting of objects with multi-layer structures on the premise of keeping entity characteristics and description details.
Making a virtual reality scene of the digital model: the virtual reality scene is manufactured based on a three-dimensional development tool (such as unity) tool, the virtual reality scene can be built by utilizing the tool with a plug-in, the virtual scene is presented by utilizing a virtual reality technology, and full-scene immersive experience is provided.
Data visualization and state evaluation: the electrocardio-line graph, the pie graph, the instrument panel and other graphs are adopted to display the index data in real time, the characteristics of intuition and clarity are achieved, and each graph is realized through a written program script (C # and other languages); the overall health state of the mechanical arm is evaluated by adopting a multi-stage fuzzy comprehensive evaluation method, a large amount of historical data and a model training process are not needed, the evaluation time of the system state is greatly reduced, and the real-time performance is good.
Drawings
FIG. 1 is an overall architecture of a digital twin remote monitoring system implemented by the present invention;
FIGS. 2 and 3 are the optimization process and the optimization result of the three-dimensional model;
FIG. 4 is a real-time data visualization chart;
FIG. 5 is an evaluation process of the fuzzy comprehensive evaluation method.
Detailed description of the preferred embodiments
In order to make the present invention clear in design concept, technical features and application range, the present invention is further described in detail below with reference to the accompanying drawings and embodiments:
as shown in fig. 1, the overall architecture of a digital twin monitoring system based on a digital twin five-dimensional model is shown. The correlation among the dimensions and the function of each dimension are shown.
The physical entity layer is a monitoring object of the monitoring system, is a source of real-time data and a virtual model, and the monitoring state and result presented by the whole monitoring system are based on the physical entity. The data layer is a data transfer and storage layer, is connected with the physical entity layer through a sensor and an interface, and is a data source of the virtual entity layer. The virtual entity layer is a receiving layer of data, the data stored in the data layer needs to be presented in the virtual entity layer, and the virtual entity is a real mapping of a physical entity. The functional service layer is a data processing layer, the presentation of the monitoring system depends on the analysis and processing of data, and the state of the physical entity is evaluated and displayed by utilizing a fuzzy comprehensive evaluation method.
As shown in fig. 2 and 3, the optimization process of the three-dimensional model is performed, the texture mapping shown in the mapping is obtained through a real object, it is ensured that the virtual model after mapping is consistent with the physical entity, and the material of the virtual model is added according to the real material. The simplification of the three-dimensional model is realized by a simplified edge folding algorithm based on a quadratic error matrix, and the algorithm can be used for manufacturing the three-dimensional model with different detail levels, which is the key for accelerating the rendering speed. It can be seen that the difference in detail is hardly visible to the naked eye when reduced to 50% of the original model, whereas significant distortion of edge sharpening is visible when reduced to 20% of the original model.
As shown in fig. 4, the data is visualized, and an electrocardiogram, pie chart, instrument panel and the like are made for the data characteristics to visualize and present the real-time data. Data visualization aims at clearly and effectively conveying and communicating information by means of graphical means. Data visualization is closely related to information graphs, information visualizations, and statistical graphs. The data visualization can be utilized to facilitate data development and data analysis.
As shown in fig. 5, the evaluation process of the multi-stage fuzzy comprehensive evaluation method is performed to evaluate the state of the whole physical entity by using the multi-stage fuzzy comprehensive evaluation method. Firstly, establishing a factor set U of an evaluation object, wherein a primary evaluation index is selected as a control system host state and a driving motor state, a secondary evaluation index is selected as data of a control system host such as CPU utilization rate, memory utilization rate and disk utilization rate, and driving motor rotation angle, voltage, current, temperature and the like, and the motor state is evaluated through the factors. A panel V is then established where four states, healthy, good, poor, and faulty, are defined. And thirdly, determining the weight of the factor set U, wherein the weight is the proportion of each factor in the factor set in the evaluation process, a variation coefficient method and an entropy method are adopted to jointly endow the weight, and the final weight W is obtained by calculating the weight respectively by the variation coefficient method and the entropy method and then weighting the weight by the weight average. And finally, constructing a membership matrix, wherein the membership matrix is obtained by calculating a Gaussian membership function, the difference between the membership matrix and the traditional Gaussian membership function is different, the dynamic Gaussian membership function is adopted, 10 groups of historical data before the current moment are obtained for calculation, the mean value and the variance of the Gaussian membership function are obtained, the Gaussian membership function is further obtained, the current data is substituted to obtain a membership matrix M, the evaluation result is obtained by M.W (representing a weighted average operator) to obtain a state grade membership matrix, and the state with the largest value in the matrix is the final evaluation state.

Claims (7)

1. A remote monitoring system and method of a cooperative robot based on digital twin comprises a physical entity, a virtual model layer, a data acquisition and transmission layer, a virtual-real connection layer and a functional service layer, wherein the physical entity is a cooperative robot, the virtual model layer comprises drawing of a virtual model of 1:1 with the physical entity, simplification of the virtual model and the like, the model simplification adopts a simplified edge folding algorithm based on a quadratic error matrix, and the virtual model is simplified by utilizing the algorithm on the premise of keeping the characteristics of the physical entity, so that rendering consumption is reduced, and system time delay is reduced; the data acquisition and transmission layer comprises original data acquisition, data processing and transmission and data storage; the virtual-real connection layer comprises fusion of data and a virtual model and a data driving model; the functional service layer comprises real-time visualization of data such as corners, voltage and current of the cooperative robot, evaluation of health state of the cooperative robot by using a multi-stage fuzzy comprehensive evaluation method, real-time presentation and the like; by constructing the five-layer framework, the virtual model can move in real time along with the entity in the movement process of the cooperative robot entity and display real-time data in the movement process of the cooperative robot, so that the purpose of state monitoring is achieved.
2. The method as claimed in claim 1, wherein the physical entity is a six-degree-of-freedom cooperative robot, and is a main body for constructing a remote monitoring system of the cooperative robot, and is a data source of the whole monitoring system.
3. The method as claimed in claim 1, wherein the virtual model rendering and simplification layer is a rendering main body of the monitoring system, real-time status data of the physical entity is required to be rendered in the virtual model, and the virtual 3D model is simplified by using a simplified edge folding algorithm based on a quadratic error matrix, so as to reduce consumption of system resources in a real-time rendering process, increase display speed, and improve monitoring efficiency in a scene composed of objects with multi-level structures on the premise of keeping entity characteristics and description details.
4. The data acquisition and transmission layer of claim 1, which is a data processing center of a monitoring system and is a basis of a functional service layer, wherein the data support is provided for a virtual-real connection layer and the functional service layer by acquiring information of real-time rotation angle, voltage, current, temperature and the like of a driving motor of a cooperative robot, the rotation angle data is obtained by writing an interface to read data records of a coder carried by the motor, the data of voltage, current, temperature, rotation speed and the like are acquired by a sensor carried by the cooperative robot, and the data of a servo motor controller is read by the interface. All the read data can be persisted into a database (such as mysql database), so that the functional service layer can analyze and present the data conveniently.
5. The virtual-real connection layer according to claim 1 is a process of mapping physical entity data to a virtual model, for a motion state of a cooperative robot, first obtaining corner data persisted in a database in claim 4, then dividing the virtual model by robot degrees of freedom, and assigning the corner data of each degree of freedom to the corresponding degree of freedom of the virtual model on the basis of satisfying the kinematics of the robot, so as to achieve the purpose of presenting the motion state of the cooperative robot in real time. The same is true for voltage, current, temperature, speed, etc. data acquisition.
6. The functional service layer is the final presentation layer of the whole remote monitoring system according to claim 1, and firstly obtains the historical data persisted in the database in claim 4, and visually presents each index data, and visually presents the data according to different selection charts of data characteristics, such as a broken line chart, a pie chart, a dashboard and the like; the overall health state of the cooperative robot is evaluated by a multi-stage fuzzy comprehensive evaluation method, and the evaluation result is divided into four grades of health, good, moderate and fault, so that the state of the cooperative robot at any moment is detected and evaluated.
7. The digital twin based collaborative robotic remote monitoring system of claim 1, wherein: presenting the virtual scene by using a virtual reality technology, and providing full-scene immersive experience; the electrocardiogram line graph and other charts are adopted to display the index data in real time, and the characteristics of intuition and clarity are achieved; the method has the advantages that a large amount of historical data and a training process of a complex model are not needed, evaluation time of the system state is greatly reduced, and good real-time performance and accuracy are achieved.
CN202111415248.0A 2021-11-25 2021-11-25 Cooperative robot remote monitoring system and method based on digital twin Pending CN114147706A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111415248.0A CN114147706A (en) 2021-11-25 2021-11-25 Cooperative robot remote monitoring system and method based on digital twin

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111415248.0A CN114147706A (en) 2021-11-25 2021-11-25 Cooperative robot remote monitoring system and method based on digital twin

Publications (1)

Publication Number Publication Date
CN114147706A true CN114147706A (en) 2022-03-08

Family

ID=80457709

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111415248.0A Pending CN114147706A (en) 2021-11-25 2021-11-25 Cooperative robot remote monitoring system and method based on digital twin

Country Status (1)

Country Link
CN (1) CN114147706A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115268646A (en) * 2022-08-02 2022-11-01 清华大学 Man-machine collaborative construction process sensing system, device, analysis method and medium
CN115741682A (en) * 2022-11-08 2023-03-07 安徽信息工程学院 Intelligent mechanical arm remote control system based on digital twin technology

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107832973A (en) * 2017-11-29 2018-03-23 国网山东省电力公司电力科学研究院 A kind of method of the equipment quality management and control based on polymorphism information Comprehensive Evaluation
CN111161410A (en) * 2019-12-30 2020-05-15 中国矿业大学(北京) Mine digital twinning model and construction method thereof
CN111300381A (en) * 2020-01-20 2020-06-19 兰州理工大学 Software robot or driver system based on digital twin three-dimensional model and modeling method thereof
CN111496781A (en) * 2020-03-17 2020-08-07 浙江大学 Mechanical arm modeling, controlling and monitoring integrated system driven by digital twin
CN111633644A (en) * 2020-05-15 2020-09-08 哈尔滨工程大学 Industrial robot digital twin system combined with intelligent vision and operation method thereof
CN112171669A (en) * 2020-09-21 2021-01-05 西安交通大学 Brain-computer cooperation digital twin reinforcement learning control method and system
WO2021169598A1 (en) * 2020-02-27 2021-09-02 广东工业大学 Digital twinning-based physical simulation method for production line and system thereof

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107832973A (en) * 2017-11-29 2018-03-23 国网山东省电力公司电力科学研究院 A kind of method of the equipment quality management and control based on polymorphism information Comprehensive Evaluation
CN111161410A (en) * 2019-12-30 2020-05-15 中国矿业大学(北京) Mine digital twinning model and construction method thereof
CN111300381A (en) * 2020-01-20 2020-06-19 兰州理工大学 Software robot or driver system based on digital twin three-dimensional model and modeling method thereof
WO2021169598A1 (en) * 2020-02-27 2021-09-02 广东工业大学 Digital twinning-based physical simulation method for production line and system thereof
CN111496781A (en) * 2020-03-17 2020-08-07 浙江大学 Mechanical arm modeling, controlling and monitoring integrated system driven by digital twin
CN111633644A (en) * 2020-05-15 2020-09-08 哈尔滨工程大学 Industrial robot digital twin system combined with intelligent vision and operation method thereof
CN112171669A (en) * 2020-09-21 2021-01-05 西安交通大学 Brain-computer cooperation digital twin reinforcement learning control method and system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115268646A (en) * 2022-08-02 2022-11-01 清华大学 Man-machine collaborative construction process sensing system, device, analysis method and medium
CN115741682A (en) * 2022-11-08 2023-03-07 安徽信息工程学院 Intelligent mechanical arm remote control system based on digital twin technology

Similar Documents

Publication Publication Date Title
CN112818446B (en) Construction method of intelligent workshop digital twin system
Qiu et al. Digital assembly technology based on augmented reality and digital twins: a review
CN111208759B (en) Digital twin intelligent monitoring system for unmanned fully mechanized coal mining face of mine
CN112528525B (en) Visual industrial process management and control platform based on modeling technology
CN111159793B (en) Modeling method and model system based on digital twin five-dimensional model 3D printer
CN111210359A (en) Intelligent mine scene oriented digital twin evolution mechanism and method
CN104484522B (en) A kind of construction method of robot simulation's drilling system based on reality scene
CN114147706A (en) Cooperative robot remote monitoring system and method based on digital twin
CN113110283A (en) MPS system intelligent control method and system based on digital twin
CN115562158A (en) Digital twin driven numerical control machine tool intelligent diagnosis method, system and terminal
CN115222792B (en) Digital twin modeling method for railway bridge
CN112085261A (en) Enterprise production status diagnosis method based on cloud fusion and digital twin technology
CN114357732A (en) Digital twin model of electronic equipment and construction method and application thereof
CN115099075A (en) Digital twinning method based on intelligent rod, wire and vehicle
CN113326570B (en) Physical and data driving-based cluster system reliability digital twin modeling method
CN114035515A (en) Digital twin system for discrete workshop production process and construction method thereof
Guo et al. Design and research of digital twin machine tool simulation and monitoring system
Guo et al. Art product design and vr user experience based on iot technology and visualization system
CN113496548A (en) Transparent factory oriented production field data mapping method
Du et al. Industrial robot digital twin system motion simulation and collision detection
CN109829174A (en) The implementation method of the synchronous verification platform of actual situation
CN115034147A (en) Intelligent manufacturing system based on digital twins
CN113221461B (en) Tunneling and anchoring all-in-one machine monitoring system based on digital twin technology
Ji Design of ship visual communication teleoperation system based on virtual reality
Wang et al. Workshop Management and Control System Based on Digital Twin

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20220308