CN117409625A - Substation equipment training system and method based on mixed reality - Google Patents

Substation equipment training system and method based on mixed reality Download PDF

Info

Publication number
CN117409625A
CN117409625A CN202311326758.XA CN202311326758A CN117409625A CN 117409625 A CN117409625 A CN 117409625A CN 202311326758 A CN202311326758 A CN 202311326758A CN 117409625 A CN117409625 A CN 117409625A
Authority
CN
China
Prior art keywords
training
mixed reality
module
equipment
learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311326758.XA
Other languages
Chinese (zh)
Inventor
许路广
周刚
朱凯元
叶润潮
张思远
李锐锋
王佳培
周冰
王聃
卢思瑶
盛鹏飞
魏泽民
徐东辉
叶丽雅
张媛
黄辰昊
张斌
周一飞
王舒清
郑思源
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiaxing Power Supply Co of State Grid Zhejiang Electric Power Co Ltd
Original Assignee
Jiaxing Power Supply Co of State Grid Zhejiang Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiaxing Power Supply Co of State Grid Zhejiang Electric Power Co Ltd filed Critical Jiaxing Power Supply Co of State Grid Zhejiang Electric Power Co Ltd
Priority to CN202311326758.XA priority Critical patent/CN117409625A/en
Publication of CN117409625A publication Critical patent/CN117409625A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B9/00Simulators for teaching or training purposes

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a substation equipment training system based on mixed reality, which comprises the following components: the mixed reality module is used for constructing a three-dimensional model and a corresponding scene model of the power transformation equipment and fusing the virtual model with the acquired real environment space; the holographic interaction module projects the virtual model constructed by the mixed reality module to the real world to perform interaction operation; the man-machine interaction module is used for inputting interaction operation instructions; the control center controls the work of the training system and collects data information in the working process; also discloses a substation equipment training method based on mixed reality. According to the invention, the virtual three-dimensional model of the power transformation equipment is fused with a real environment space based on a mixed reality technology, new staff is trained through the assembly and disassembly in the superposition virtual model and the simulation of an accident scene in the real environment, so that the new staff can perform immersive training learning, and the structure and operation of the power transformation equipment are better understood, thereby improving the training effect and efficiency.

Description

Substation equipment training system and method based on mixed reality
Technical Field
The invention relates to the technical field of power equipment, in particular to a power transformation equipment training system and method based on mixed reality.
Background
With the continuous development of society and the continuous growth of economy, electric power is an indispensable energy supply mode of modern society, and has become an important infrastructure for supporting various activities and industrial operations of society. However, with the ever-increasing scale of power systems and the increasing complexity of grid structures, the operation and maintenance management tasks of power equipment become increasingly heavy and complex. Meanwhile, in order to ensure the stable operation of the power system and prevent the occurrence of large-area power failure accidents, the maintenance and management of the power equipment are particularly critical. In the traditional power equipment maintenance and management, a scheduled maintenance system is mainly adopted. However, this conventional system has shown serious drawbacks in coping with the operation and maintenance of increasingly complex power equipment. The traditional method is highly dependent on the technical level of field maintenance personnel, so that the operation efficiency is low, and the reliability of the power equipment is affected because the problems of insufficient maintenance or excessive and blind maintenance and the like can occur due to the technical differences of different technical personnel; meanwhile, along with the continuous expansion of the scale number of the power equipment, the traditional first-line staff number is difficult to meet the requirement of rapid development of the power equipment industry, a large number of new generation staff are needed to be adopted in expansion, but the expertise and experience of the new staff are lacking, the new staff cannot be directly put on duty to meet the maintenance and management requirements of the complex power equipment, and the first-line work can be carried out after a large amount of pre-duty training and examination. The existing training mainly enables new staff to be familiar with operation contents through basic knowledge explanation of text data and picture videos, and the lack of experience of actual manual operation cannot give visual feeling to the new staff, and cannot deeply understand the internal structure of the power transformation equipment, so that the actual training effect is not ideal, and the training efficiency is not high.
The power transformation equipment overhaul simulation training method disclosed in China patent literature has a publication number of CN106504607A and a publication date of 2017-03-15, and comprises the following steps of: (1) The student logs in the system (2) to select the type and model of the power transformation equipment; (3) selecting an overhaul tool; (4) selecting an entered system mode; (5) And the trainee completes the operation according to the requirements of different modes in the simulation scene. However, the technology is still training through a computer, the training and the learning are indirectly reflected to the virtual character in the display screen through the operation of a mouse or a keyboard, and different operation steps are clicked to control the virtual character to operate, so that compared with the learning materials of text materials and picture videos, the visual feeling of new staff on the power transformation equipment can be enhanced to a certain extent, but the training effect of actual manual operation on the new staff can not be replaced, the new staff is not beneficial to familiarity with the internal structure of the power transformation equipment, the actual training effect still needs to be improved, and the training efficiency is not high.
Disclosure of Invention
The invention provides a transformer equipment training system and a method based on mixed reality, which are used for fusing a virtual transformer equipment three-dimensional model with a real environment space to realize real and virtual interaction, realizing the training of actual operation of new staff by overlapping equipment parts in the virtual model in the real environment and simulating an accident scene, and simultaneously, being provided with a holographic interaction module, the new staff can perform immersive training learning to better understand the structure and operation of the transformer equipment, thereby improving the training effect and efficiency.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
a mixed reality based substation equipment training system, comprising:
the mixed reality module is used for constructing a three-dimensional model and a corresponding scene model of the power transformation equipment, fusing the virtual model with the acquired real environment space and then projecting the fused virtual model into the mixed reality equipment;
the holographic interaction module is used for projecting the virtual model constructed by the mixed reality module to the real world and carrying out interaction operation through the man-machine interaction module;
the man-machine interaction module comprises mixed reality equipment and is used for inputting interaction operation instructions;
the control center controls the work of the training system and collects data information in the working process, and the control center comprises a data analysis feedback module.
According to the invention, the mixed reality module is used for realizing functions of virtual display, operation maintenance guidance, training and the like by using mixed reality equipment such as a head-mounted display or intelligent glasses, wherein a user can see a virtual equipment model in a real environment; the holographic interaction module projects a virtual image into the real world by utilizing a holographic projection technology, and realizes immersive training experience through projection of the virtual model and real-time interaction between a user and the virtual model; the man-machine interaction module collects interaction operation instruction information sent by a user through the mixed reality equipment and inputs the interaction operation instruction information into the system, and specific working states of each module are indirectly controlled, such as selection, scaling and rotation operation of a three-dimensional model of the power transformation equipment; the control center receives data information of other modules and controls the work of the system according to the interactive operation instruction, and the data analysis feedback module can analyze the collected data and feed back corresponding results; the method provides an innovative mode for training, operation and teaching of the transformer equipment, and the virtual model is projected into the real world, so that fusion of the virtual model and the real model is realized, visual feeling of a training user is enhanced, and training effect is improved.
Preferably, the mixed reality module includes:
the three-dimensional modeling unit is used for establishing a three-dimensional model comprising the appearance, the structure and the internal composition of the power transformation equipment and a corresponding scene model;
the real environment acquisition unit scans the real environment to form real environment space data and establishes a space coordinate system;
and the scene construction unit is used for fusing the virtual model and the real environment space to obtain a mixed reality scene and adjusting the mixed reality scene in real time according to the interactive operation instruction.
The mixed reality module further comprises a space positioning unit for positioning the coordinate position of the three-dimensional model in the real environment space and the coordinate position of the user corresponding to the mixed reality equipment in the real environment space, so that the mixed reality module is beneficial to being fed back to the holographic interaction module subsequently and immersive interaction experience is realized in the real environment; the scene construction unit determines the fusion position of the virtual model in the real environment space according to the real environment space and the space coordinate system of the current real environment, and can directly match the real space environment with the current real environment in a mutually overlapped manner when the holographic projection is carried out subsequently, so that the virtual model is accurately projected at the corresponding position of the real environment.
Preferably, the man-machine interaction module comprises at least one of a gesture recognition unit for controlling an operation instruction through gestures, a voice recognition unit for controlling the operation instruction through voices and an external device unit for controlling by operating the external device;
the man-machine interaction module can be simultaneously connected with a plurality of mixed reality devices for team interaction.
According to the invention, the man-machine interaction module can be connected with a plurality of mixed reality devices, so that teaching training for multiple persons can be realized at the same time, multiple users can see virtual models of the multiple users in the same space through the holographic interaction module, real-time interaction, discussion and cooperation can be performed, and the interactivity of training and team cooperation capability are improved; any one or more of gestures, voices or external devices can be selected for inputting the instruction operation, and the selection is performed according to actual conditions and needs.
Preferably, the control hub further comprises:
the simulation learning module is used for controlling the mixed reality module to construct a virtual learning scene and learning training knowledge of the power transformation equipment;
the simulation training module is used for carrying out daily operation maintenance, inspection and fault processing training on the power transformation equipment after carrying out space fusion projection on the virtual model and the real environment to the real world;
And the assessment module is used for carrying out assessment and evaluation on the results of the simulation learning and the simulation training.
The control center comprises a data analysis feedback module, a simulation learning module, a simulation training module and an assessment and evaluation module, wherein the simulation learning module can only utilize the mixed reality module to project a virtual learning scene and a virtual model of the power transformation equipment into the mixed reality equipment for a user to learn and watch, and can also utilize the holographic interaction module to project the virtual model into the real world for learning; the simulation training module is used for completing training operation by combining the interaction between a user and the virtual model through the human-computer interaction module after the virtual model is projected to the real world through the holographic interaction module; the assessment module can assess the knowledge mastering degree of the simulation learning and the operation proficiency and accuracy degree of the simulation training.
Preferably, the data analysis feedback module analyzes the learning, training and checking conditions of each training person according to the data in the working process of the training system, and generates a corresponding learning and training scheme.
In the invention, a large number of examination questions are stored in the examination evaluation module, and the content of the question bank can be updated in real time; the data feedback analysis module analyzes the question type with the highest error rate of each training person, pertinently improves the proportion of the questions of the corresponding type and the operation training of the corresponding type, so that the training person can better master the training content of the power transformation equipment, and the training effect is improved.
Preferably, the simulation learning module includes:
the equipment foundation learning unit is used for carrying out integral display explanation and split display explanation on the three-dimensional model of the power transformation equipment;
the device structure learning unit is used for explaining the internal structure of the three-dimensional model of the power transformation device and the relation between the structures;
and the standard demonstration learning unit is used for demonstrating standard operation processes of all training items in the simulation training module.
The simulation learning module comprises at least three learning units, an equipment foundation learning unit, an equipment structure learning unit and a standard demonstration learning unit, the knowledge learning level is gradually increased, subsequent equipment structure knowledge learning can be performed after equipment foundation knowledge learning and assessment are completed, the standard demonstration learning unit can be entered after equipment structure knowledge learning and assessment are completed, the standard operation process is learned, the knowledge grasping degree of training students is tamped, and the training content of the power transformation equipment is easily grasped from shallow to deep.
Preferably, the system further comprises a user management module for managing information and rights of different users, registering, logging in and authenticating the users, and establishing a team comprising a plurality of users.
In the invention, the user management module is responsible for the registration, login and authentication of the user, and the user can log in the system through the personal account number to manage personal information and operation records; team mode can also be established, and specific data information is set so that only users belonging to the team can watch the operation.
Preferably, when the user management module performs user authentication, the mixed reality device collects face information of an object to be logged in;
judging whether the object to be logged in is legal or not according to the matching of the collected face information of the object to be logged in and the face information stored during the registration; and if the object to be logged in is legal, the control center sends login control information to the mixed reality equipment login training system.
When the user is registered, face information, login password, fingerprint information and the like can be adopted as the identity verification data for login, the login condition of the object to be logged in is judged by matching the identity verification data input by the mixed reality equipment with the identity verification data stored during the registration, the account number registered before the registered user logs in is registered, and the registration prompt is jumped out for the user registered in position.
A substation equipment training method based on mixed reality comprises the following steps:
Constructing a three-dimensional model and a corresponding scene model of the power transformation equipment and storing the three-dimensional model and the corresponding scene model in a virtual model library;
responding to the selection of a login user on a simulation learning module or a simulation training module or an assessment module, and calling a corresponding virtual model from a virtual model library for simulation display;
and responding to an interactive operation instruction input by a login user through the man-machine interaction module, and displaying the state change of the virtual model under the interactive operation instruction.
After logging in the substation equipment training system, a training person can input a selection instruction through the man-machine interaction module, and select to enter the simulation learning module, the simulation training module or the assessment module to perform corresponding learning training or assessment operation; meanwhile, for the virtual model projected into the mixed reality equipment or the real space, the states of splitting, zooming, rotating and the like of the virtual model can be controlled through the input interactive operation instruction, so that training staff can learn the relation of the internal result of the power transformation equipment in detail from different angles; the three-dimensional model of the power transformation equipment is built in advance and stored in a virtual model library, and the corresponding scene model and the three-dimensional model are fused to simulate the power transformation equipment conditions under different environments and different fault conditions, so that training contents are more fit with reality.
Preferably, the displaying the state change of the virtual model under the interactive operation instruction includes:
simulating and displaying the state change of the virtual model under the interactive operation instruction according to the visual angle of the current operation user; and/or the number of the groups of groups,
simulating and displaying the state change of the virtual model under the interactive operation instruction according to the visual angle of any user in the team.
When the multi-person learning training is carried out in the team mode, the training operation condition of one training person can be simultaneously transmitted to other training persons for watching, and the current situation of the simulation operation is watched at multiple angles, so that the training person can learn and find problems conveniently; in addition, the method can help the training staff to solve the problems together and finish tasks cooperatively, so that the interactivity of training and team cooperation capability are enhanced; meanwhile, the training guide can participate in interaction or demonstration through remote connection, and guidance and support are provided for training staff.
The invention has the following beneficial effects: the virtual three-dimensional model of the power transformation equipment is fused with the real environment space based on the mixed reality technology, so that real and virtual interaction is realized, the new staff can be trained in actual operation by overlapping equipment parts in the virtual model in the real environment, assembling and disassembling equipment parts and simulating an accident scene, and meanwhile, the holographic interaction module is also arranged, so that the new staff can perform immersive training learning, the structure and operation of the power transformation equipment are better understood, and the training effect and efficiency are improved; the data analysis and feedback module can collect the operation data of the learning user, analyze the operation habit and error of the user, feed back the operation instruction and training advice personalized to the user according to the data, conduct more targeted instruction training and improve training effect.
Drawings
FIG. 1 is a schematic diagram of a substation equipment training system in accordance with the present invention;
FIG. 2 is a schematic diagram of the unit modules included in the control hub of the present invention;
FIG. 3 is another schematic diagram of a substation equipment training system of the present invention;
fig. 4 is a flow chart of a substation equipment training method in the present invention;
in the figure: 1. a control hub; 2. a holographic interaction module; 3. a mixed reality module; 4. a man-machine interaction module; 5. a user management module; 11. a data analysis feedback module; 12. an assessment and evaluation module; 13. a simulation training module; 14. a simulation learning module; 31. a three-dimensional modeling unit; 32. a real environment acquisition unit; 33. a scene construction unit; 34. a space positioning unit; 41. a gesture recognition unit; 42. a voice recognition unit; 43. an external device unit; 131. an operation maintenance unit; 132. a patrol inspection unit; 133. a fault handling unit; 141. an equipment foundation learning unit; 142. an equipment structure learning unit; 143. and a standard demonstration learning unit.
Detailed Description
The invention is further described below with reference to the drawings and detailed description.
As shown in fig. 1, a mixed reality-based substation equipment training system includes:
The control center 1 can control the training system and collect data information in the working process and comprises a data analysis feedback module 11;
the control center 1 is respectively connected with the mixed reality module 3, the holographic interaction module 2 and the man-machine interaction module 4;
the mixed reality module 3 constructs and stores a three-dimensional model and a corresponding scene model of the power transformation equipment in a virtual model library, and simultaneously fuses and projects the corresponding virtual model and the acquired current real environment space into the mixed reality equipment according to the selection of a user, so that the user can wear the mixed reality equipment to watch and learn;
the holographic interaction module 2 projects the virtual model constructed by the mixed reality module to the real world, and performs interaction operation through the man-machine interaction module 4;
the man-machine interaction module 4 is used for inputting interaction operation instructions and can be provided with a plurality of mixed reality devices.
The mixed reality module includes:
a three-dimensional modeling unit 31 that establishes a three-dimensional model including the appearance, structure, and internal composition of the power transformation device, and a corresponding scene model; by means of the three-dimensional modeling technology, the appearance and the internal structure of the power transformation device can be presented in a highly real form. For example, for complex transformer devices, the internal windings, insulation, cooling systems, etc. may be carefully modeled to provide an accurate data basis for subsequent mixed reality projections.
A real environment acquisition unit 32 that scans a real environment to form real environment space data and establishes a space coordinate system; a real environment space corresponding to the real environment in which the current user is located is formed.
The scene construction unit 33 fuses the virtual model and the real environment space to obtain a mixed reality scene, and adjusts in real time according to the interactive operation instruction. The working principle of the transformer equipment is displayed by combining the animation of the three-dimensional model, for example, basic principles of the current transformation process of the transformer, electromagnetic induction of windings and the like are demonstrated through the animation, and training staff are helped to understand the working mechanism of the equipment deeply. The operation condition of the equipment under different fault modes can be simulated, the training personnel can be helped to identify the fault characteristics, and the early diagnosis and prevention of faults can be realized. For example, for transformers, animation may simulate faults such as winding shorts, insulation breaks, etc.; the method can also guide training staff to carry out correct maintenance operation through the disassembly, assembly and maintenance processes of the animation demonstration equipment, reduce operation risks and improve the efficiency of the device.
The man-machine interaction module 4 comprises at least one of a gesture recognition unit 41 for controlling an operation instruction through a gesture, a voice recognition unit 42 for controlling the operation instruction through voice and an external device unit 43 for controlling by operating an external device; the man-machine interaction module can be simultaneously connected into a plurality of mixed reality devices to perform team interaction.
The gesture recognition unit 41 collects gesture information through a camera of the mixed reality device, and performs matching and corresponds to preset defined gesture recognition objects, setting gesture types needing to be monitored, setting operations needing to be performed after the gesture recognition is performed, controlling dragging, zooming, rotating, disassembling and the like of the virtual model, and simultaneously controlling selection of the objects. The voice recognition unit 42 presets voice keywords corresponding to different operations, and performs corresponding operations through the keywords in the recognized voice information. The external device unit 43 may correspond to different operations with different keys, similar to a mouse, a keyboard, or an operation handle.
As shown in fig. 2, the control center 1 includes, in addition to the data analysis feedback module 11:
the simulation learning module 14 controls the mixed reality module to construct a virtual learning scene and performs learning of training knowledge of the power transformation equipment; comprises the following steps:
the device base learning unit 141 performs overall display explanation and split display explanation on the three-dimensional model of the power transformation device;
an equipment structure learning unit 142 that explains the internal structure of the three-dimensional model of the power transformation equipment and the relationship between the respective structures;
a standard demonstration learning unit 143 for demonstrating standard operation processes of the respective training items in the simulation training module; the training teacher stores the pre-recorded standard operation processes of various daily operation maintenance, inspection and fault treatment in the standard demonstration learning unit, and the training staff can autonomously select the corresponding project learning standard operation process.
The simulation training module 13 performs the training of daily operation maintenance, inspection and fault treatment of the power transformation equipment after the virtual model and the real environment are spatially fused and projected to the real world; comprises the following steps:
the operation maintenance unit 131 covers typical operation and maintenance simulation of the power transformation equipment, and includes manual simulation operations corresponding to training contents such as equipment introduction, installation step explanation, disassembly step explanation, single component explanation, installation notice, disassembly notice, equipment overhaul process and the like.
The patrol inspection unit 132 performs operations of the primary and secondary devices by using the mixed reality system according to the project and the patrol period specified by the operation procedure issued by the department or established in the field. All equipment of the power transformation equipment is displayed in a three-dimensional mode, including all scenes of sites such as normal installation and disassembly of the power transformation equipment, the normal and disassembly states of the equipment are simulated extremely realistically in time and space, the effect of being on the spot is achieved, and all operable equipment and observable dynamic quantity belong to patrol training content; after the training personnel completes the simulated inspection of the substation equipment, inspection records can be obtained, and compared with the standard inspection records of the standard demonstration learning unit, the inspection personnel can check for leakage and repair defects.
The fault handling unit 133 simulates an accident in two cases, one is a set accident such as: line faults, transformer faults, etc.; another is a malfunction accident, which naturally occurs in the model. Various accident exercises and replay can be performed by setting accidents, and the capability of training personnel for accident handling is improved.
The accident is divided into the aspects of power network, line, transformer, bus, breaker, mutual inductor, reactor, capacitor, station AC/DC system, protection, automatic device, secondary circuit, etc. In the set accident, the types (such as interphase short circuit, ground short circuit, three-phase short circuit and two-phase short circuit ground), properties (permanent or instantaneous) and places (any point from the head end to the tail end of all lines) can be arranged and combined at will.
The system can manually set single or composite faults to manufacture accident sites, so that students can judge the accident properties, types, places and the like and process the faults, find errors and omission in accident processing, and keep in mind the correct processing method, thereby improving the technical level.
The abnormal simulation of the equipment is divided into two cases, wherein one is naturally occurring in operation, such as: overload of the transformer, exceeding of transformer oil temperature by an allowable value, disappearance of a power supply and the like; another is an exception that needs to be set, such as: oil leakage of the transformer, slight short circuit among turns of the transformer, and the like. When the equipment is abnormal, corresponding alarm signals are triggered, and corresponding pictures and monitoring instrument reactions are consistent with the scene.
Equipment defects are often found in field operation, maintenance and tests, and some equipment anomalies and accidents which endanger safety but do not yet form are treated. In the training process, a coach can insert equipment defects timely to check whether the learner can find out and process corresponding equipment in time in the equipment inspection process.
The assessment module 12 performs assessment on the results of the simulation learning and the simulation training.
The data analysis feedback module analyzes the learning, training and checking conditions of each training person according to the data in the working process of the training system, and generates a corresponding learning and training scheme.
The system performs intelligent analysis and feedback by collecting operation data of the user. The system can analyze the indexes of the operation behavior, speed, precision and the like of the user and identify the advantages and the disadvantages of the user. Based on these analysis results, the system generates personalized operational guidelines, exercise programs, and training advice that help the user improve the operational skills. The user can perform self-evaluation and self-lifting according to the feedback of the system, so that progressive and continuous improvement of training is realized. For example, for training personnel with more skill in operation, the system may provide more difficult tasks and challenges; for training personnel who operate slowly or frequently in error, the system can provide repeated exercises and operation guidance in a targeted manner. The requirements of different users are met to the greatest extent, and the training effect is improved.
As shown in fig. 3, the transformation training system further includes a user management module 5 connected to the control center, for managing information and rights of different users, performing registration, login and authentication of the users, and establishment of a team including a plurality of users.
When the user management module 5 performs user authentication through face information:
the mixed reality equipment collects face information of an object to be logged in; extracting face characteristic data of an object to be logged on based on face information;
according to the collected face information of the object to be logged in and the face information stored during registration, judging whether the face characteristic data of the object to be logged in is matched with the face characteristic data stored in advance, if so, judging whether the object to be logged in is legal;
if the login request is not matched with the login request, judging that the object to be logged in is illegal, and outputting a prompt or prompt that login is impossible for registering;
and if the object to be logged in is legal, the control center sends login control information to the mixed reality equipment login training system.
When the user management module 5 performs user authentication through fingerprint information:
the mixed reality equipment collects fingerprint information of an object to be logged in; extracting fingerprint characteristic data of an object to be logged on the basis of the fingerprint information;
According to the collected fingerprint information of the object to be logged in and the fingerprint information stored during registration, judging whether the fingerprint characteristic data of the object to be logged in is matched with the pre-stored fingerprint characteristic data, if so, judging whether the object to be logged in is legal;
if the login request is not matched with the login request, judging that the object to be logged in is illegal, and outputting a prompt or prompt that login is impossible for registering;
and if the object to be logged in is legal, the control center sends login control information to the mixed reality equipment login training system.
In addition, the mixed reality module 3 includes a space positioning unit 34 in addition to the three-dimensional modeling unit 31, the real environment acquisition unit 32, and the scene construction unit 33, to position the coordinate position of the three-dimensional model of the power transformation device in the real environment space, and to position the coordinate position of the user corresponding to the mixed reality device in the real environment space, so as to facilitate the fusion of the three-dimensional model of the power transformation device and the real environment space, and in the real world of the subsequent projection, the real environment space and the current real space can be directly overlapped and matched, so that the virtual model of the power transformation device is projected to the accurate position in the real environment.
Spatial localization based on trigonometric relations, assuming known coordinates (x 1 ,y 1 ,z 1 )、(x 2 ,y 2 ,z 2 ) And (x) 3 ,y 3 ,z 3 ) The coordinates of the device are the coordinates (x i ,y i ,z i ) The device coordinates to be solved can be obtained according to the geometrical relationship between the device coordinates and the distances from the three base stations:
the method comprises the steps of placing a known marker (which can be a marker with a specific shape or a mixed reality device) in a real environment in advance, determining the mapping marker position, performing image recognition and gesture evaluation on the marker by using a camera sensor arranged on a mixed reality module, and marking the spatial position of the marker; taking one point of the marker as a mapped coordinate origin, and establishing a mapped coordinate system; and establishing a mapping relation between a coordinate system in a display screen of the mixed reality equipment and a mapping coordinate system through mapping transformation, and mapping the displayed virtual model into a real environment space calibrated by the marker based on the mapping relation, so as to realize superposition interaction of the virtual model and the real environment. The calculation of the mapping coordinate system in the spatial localization can be determined by the existing conversion relation including the camera internal reference matrix and the external reference matrix, and thus will not be described in detail.
According to the invention, the mixed reality module is used for realizing functions of virtual display, operation maintenance guidance, training and the like by using mixed reality equipment such as a head-mounted display or intelligent glasses, wherein a user can see a virtual equipment model in a real environment; the holographic interaction module projects a virtual image into the real world by utilizing a holographic projection technology, and realizes immersive training experience through projection of the virtual model and real-time interaction between a user and the virtual model; the man-machine interaction module collects interaction operation instruction information sent by a user through the mixed reality equipment and inputs the interaction operation instruction information into the system, and specific working states of each module are indirectly controlled, such as selection, scaling and rotation operation of a three-dimensional model of the power transformation equipment; the control center receives data information of other modules and controls the work of the system according to the interactive operation instruction, and the data analysis feedback module can analyze the collected data and feed back corresponding results; the method provides an innovative mode for training, operation and teaching of the transformer equipment, and the virtual model is projected into the real world, so that fusion of the virtual model and the real model is realized, visual feeling of a training user is enhanced, and training effect is improved.
The mixed reality module further comprises a space positioning unit for positioning the coordinate position of the three-dimensional model in the real environment space and the coordinate position of the user corresponding to the mixed reality equipment in the real environment space, so that the mixed reality module is beneficial to being fed back to the holographic interaction module subsequently and immersive interaction experience is realized in the real environment; the scene construction unit determines the fusion position of the virtual model in the real environment space according to the real environment space and the space coordinate system of the current real environment, and can directly match the real space environment with the current real environment in a mutually overlapped manner when the holographic projection is carried out subsequently, so that the virtual model is accurately projected at the corresponding position of the real environment.
According to the invention, the man-machine interaction module can be connected with a plurality of mixed reality devices, so that teaching training for multiple persons can be realized at the same time, multiple users can see virtual models of the multiple users in the same space through the holographic interaction module, real-time interaction, discussion and cooperation can be performed, and the interactivity of training and team cooperation capability are improved; any one or more of gestures, voices or external devices can be selected for inputting the instruction operation, and the selection is performed according to actual conditions and needs.
The control center comprises a data analysis feedback module, a simulation learning module, a simulation training module and an assessment and evaluation module, wherein the simulation learning module can only utilize the mixed reality module to project a virtual learning scene and a virtual model of the power transformation equipment into the mixed reality equipment for a user to learn and watch, and can also utilize the holographic interaction module to project the virtual model into the real world for learning; the simulation training module is used for completing training operation by combining the interaction between a user and the virtual model through the human-computer interaction module after the virtual model is projected to the real world through the holographic interaction module; the assessment module can assess the knowledge mastering degree of the simulation learning and the operation proficiency and accuracy degree of the simulation training.
In the invention, a large number of examination questions are stored in the examination evaluation module, and the content of the question bank can be updated in real time; the data feedback analysis module analyzes the question type with the highest error rate of each training person, pertinently improves the proportion of the questions of the corresponding type and the operation training of the corresponding type, so that the training person can better master the training content of the power transformation equipment, and the training effect is improved.
The simulation learning module comprises at least three learning units, an equipment foundation learning unit, an equipment structure learning unit and a standard demonstration learning unit, the knowledge learning level is gradually increased, subsequent equipment structure knowledge learning can be performed after equipment foundation knowledge learning and assessment are completed, the standard demonstration learning unit can be entered after equipment structure knowledge learning and assessment are completed, the standard operation process is learned, the knowledge grasping degree of training students is tamped, and the training content of the power transformation equipment is easily grasped from shallow to deep.
In the invention, the user management module is responsible for the registration, login and authentication of the user, and the user can log in the system through the personal account number to manage personal information and operation records; team mode can also be established, and specific data information is set so that only users belonging to the team can watch the operation.
When the user is registered, face information, login password, fingerprint information and the like can be adopted as the identity verification data for login, the login condition of the object to be logged in is judged by matching the identity verification data input by the mixed reality equipment with the identity verification data stored during the registration, the account number registered before the registered user logs in is registered, and the registration prompt is jumped out for the user registered in position.
As shown in fig. 4, a power transformation equipment training method based on mixed reality includes:
constructing a three-dimensional model and a corresponding scene model of the power transformation equipment and storing the three-dimensional model and the corresponding scene model in a virtual model library;
responding to the selection of a login user on a simulation learning module or a simulation training module or an assessment module, and calling a corresponding virtual model from a virtual model library for simulation display;
and responding to an interactive operation instruction input by a login user through the man-machine interaction module, and displaying the state change of the virtual model under the interactive operation instruction.
Displaying the state change of the virtual model under the interactive operation instruction comprises the following steps:
simulating and displaying the state change of the virtual model under the interactive operation instruction according to the visual angle of the current operation user; and/or the number of the groups of groups,
simulating and displaying the state change of the virtual model under the interactive operation instruction according to the visual angle of any user in the team.
The man-machine interaction module receives interaction commands input by a plurality of training personnel through the mixed reality equipment in a team mode, shares the commands, and controls a series of operations such as disassembly, assembly, rotation, scaling and the like of a virtual model in the mixed reality module according to the shared multi-source commands so as to facilitate team cooperation; because multiple users can see the virtual models in the mutual view angles in the same space, real-time communication, discussion and cooperation can be performed; meanwhile, a training teacher can be included in the team, and the training teacher selects any visual angle of the training student to observe the simulation operation condition of the corresponding student and conduct real-time guidance on the training student.
After logging in the substation equipment training system, a training person can input a selection instruction through the man-machine interaction module, and select to enter the simulation learning module, the simulation training module or the assessment module to perform corresponding learning training or assessment operation; meanwhile, for the virtual model projected into the mixed reality equipment or the real space, the states of splitting, zooming, rotating and the like of the virtual model can be controlled through the input interactive operation instruction, so that training staff can learn the relation of the internal result of the power transformation equipment in detail from different angles; the three-dimensional model of the power transformation equipment is built in advance and stored in a virtual model library, and the corresponding scene model and the three-dimensional model are fused to simulate the power transformation equipment conditions under different environments and different fault conditions, so that training contents are more fit with reality.
When the multi-person learning training is carried out in the team mode, the training operation condition of one training person can be simultaneously transmitted to other training persons for watching, and the current situation of the simulation operation is watched at multiple angles, so that the training person can learn and find problems conveniently; in addition, the method can help the training staff to solve the problems together and finish tasks cooperatively, so that the interactivity of training and team cooperation capability are enhanced; meanwhile, the training guide can participate in interaction or demonstration through remote connection, and guidance and support are provided for training staff.
A complete hybrid training system mainly comprises three parts, namely equipment modeling, scene modeling and scene interaction. Specialized three-dimensional modeling software such as 3DS MAX, maya and the like can be used for modeling equipment and scenes. Since the power simulation needs to simulate the state and defect of the device and needs to complete the functions of scene switching, device operation and the like, the mixed reality training implemented by the system of the embodiment provides a device editor to assist in device modeling. The three-dimensional engine of the three-dimensional modeling unit in this embodiment includes a base frame, an application interaction frame, and an LOD (level of detail) management frame. The basic framework is a core computing framework of the intelligent interaction engine and is mainly an operation framework of modules such as basic mathematical algorithm, basic graphic units, basic operation framework and the like. Basic mathematical algorithm: the basic operation algorithm is subjected to operations such as definition packaging and the like. Basic graphics unit: the base graphic is an elemental base that makes up the three-dimensional world. Basic operation framework: almost every interaction scene has operations of initialization, operation, destruction and the like, and preliminary development and encapsulation are needed.
Level of detail management is divided into two types, continuous LOD (CLOD) and discontinuous LOD (OLOD), which are processed at appropriate times in order to reduce system operating pressure. The system operation efficiency can be improved. And the running smoothness of the system is improved. The DLOD of the model is critical in that the model base should be the highest progress descriptive model. And has higher requirements on the face reduction threshold value. The main technical scheme is that different levels of face reduction operations are performed on the model in advance, and the face reduction operations are cached so as to facilitate the calling of different levels.
The basic requirements of the mixed practical training platform of the power transformation equipment are met. The three-dimensional scene implementation content comprises: 1. basic object representation: basic geometric bodies including cubes, spheres, cylinders, cones, and the like; 2. model introduction: model import supporting formats of 3DS, OBJ, MD2 and the like at present; 3. and (3) picture importing: the introduction of the TGA format picture is added on the basis of the picture format supported by Qt; 4. collision detection: the collision detection among the lines, cubes, balls and ellipsoids is realized, and the functions of equipment operation, role walking, ascending and descending stairs, wall sliding and the like can be conveniently realized; 5. light: point light sources, parallel light sources, spotlights and the like can be conveniently created; 6. texture, texture: the material and texture of the object can be edited; 7. particle system: providing a simulation of fire, smoke, steam, explosion, rain, snow phenomena; 8. bulletin board (billboards): providing a function of inexpensively simulating a large number of three-dimensional objects in a planar manner, such as trees, etc.; 9. mist: the OpenGL mist effect is simply packaged; 10. sky body and sky box; 11. BSP and visual cutting are carried out, so that the rendering efficiency is improved; 12. the dynamic image sequence can realize the effects of lightning, flowing water and the like.
The device editor builds a real device by importing an existing three-dimensional model and combining and setting properties of the various models. Mainly comprises the following steps:
the build settings, the user can create a new model by combining different sub-models and can enhance its realism by texture and texture.
In fault setting, in training, it is often necessary to manually set a fault of a certain device. Such as cracking of porcelain bottles, rust of ground wires, steam emission, etc. This is achieved by alternating sub-models, replacing textures, and adding special effects.
The submodels alternately display: for faults such as cracking of the porcelain bottle, reddening of the porcelain bottle and the like, a plurality of sub-models with the same size can be arranged at the same position in advance, and the sub-models correspond to the cracked porcelain bottle, the reddened porcelain bottle and the normal porcelain bottle respectively. Faults are simulated by displaying a certain sub-model and hiding other sub-models.
Texture replacement: for faults such as ground wire rust, equipment dirt and the like, the display cost can be reduced by simply replacing textures.
Special effects: for steam emission, smoke emission, explosion, etc., special effects of the particle system can be realized.
In order to enhance the realism, in a three-dimensional scene, an operation device such as a switch door, a pull switch, a turning handle, or the like is often required. This can be achieved by a key frame supporting model, which is simple in structure and provides key frame functions. The FBX model may be used to display the effect of the operation by interpolating between key frames during a particular display.
And the scene editor uses the device model edited by the device editor to deploy the whole corresponding scene model. The implementation method comprises the following steps: 1. the topography is divided into grid shapes, the texture of each grid can be set independently (used for creating roads, curved roads can be created by changing the size of the grid, concave-convex effects are further added, effects of soil hills, curbstones and the like are formed), and objects can be added at the grid. 2. The size of the object is adjusted by inputting the operation instruction, and the position and the size of the object can be accurately defined by manually inputting data. 3. Providing a conversion of common viewing angles including top, low, front, back, left, right, perspective, and the like; the same editing operation can be performed in each view, in which case the relative position between the objects can be better controlled.
In the system of the embodiment, the mixed reality module is one of core modules of the system, and projects a virtual power equipment model into a real environment through a mixed reality technology, wherein the module is responsible for virtual display, operation guidance and training functions of equipment. The man-machine interaction module recognizes gesture actions of a user through equipment such as a camera or collects voice information or converts operation ideas of the user into operation instructions through external equipment, and in the gesture recognition unit, the user can simulate actual operation actions through gestures to realize operation of virtual equipment. The holographic interactive mode is another core module of the system, virtual images are projected into the real world through a holographic projection technology, and a user can perform operation demonstration, demonstration and training through holographic projection, so that immersive experience is realized. The data analysis and feedback module is responsible for collecting operation data of the user, performing intelligent analysis and feedback, and the system can analyze operation behaviors of the user and generate personalized operation guidance and training advice. The user management module manages information and authority of different users, including registration, login, identity verification and other functions, and is the basis for ensuring the safety and individuation of the system.
The modules of the system have close component relation and data flow, and form a cooperative whole. A user interacts with the mixed reality module through a mixed reality device or smart glasses access system. The mixed reality module projects the virtual power transformation device model into the field of view of the user, who can freely observe the external and internal structures of the device. The gesture action of the user is recognized by the gesture recognition unit and is converted into a corresponding operation instruction. The user can detach, assemble, adjust etc. through the gesture, and the system can be discerned in real time and give feedback. The holographic interaction module projects the virtual image into the real world through a holographic projection technology, and a user can perform operation demonstration, demonstration and training through the virtual image, so that the fusion of the virtual and the reality is realized. The operation data of the user is collected to a data analysis and feedback module for intelligent analysis, and the system can generate personalized operation guidance, exercise plans and training suggestions for the user according to the data analysis result. The user management module is responsible for registration, login and identity verification of the user, and the user can log in the system through the personal account number to manage personal information and operation records.
The foregoing embodiments are further illustrative and explanatory of the invention, as is not restrictive of the invention, and any modifications, equivalents, and improvements made within the spirit and principles of the invention are intended to be included within the scope of the invention.

Claims (10)

1. A mixed reality-based substation equipment training system, comprising:
the mixed reality module is used for constructing a three-dimensional model and a corresponding scene model of the power transformation equipment, fusing the virtual model with the acquired real environment space and then projecting the fused virtual model into the mixed reality equipment;
the holographic interaction module is used for projecting the virtual model constructed by the mixed reality module to the real world and carrying out interaction operation through the man-machine interaction module;
the man-machine interaction module comprises mixed reality equipment and is used for inputting interaction operation instructions;
the control center controls the work of the training system and collects data information in the working process, and the control center comprises a data analysis feedback module.
2. The mixed reality based substation equipment training system of claim 1, wherein the mixed reality module comprises:
the three-dimensional modeling unit is used for establishing a three-dimensional model comprising the appearance, the structure and the internal composition of the power transformation equipment and a corresponding scene model; the real environment acquisition unit scans the real environment to form real environment space data and establishes a space coordinate system;
And the scene construction unit is used for fusing the virtual model and the real environment space to obtain a mixed reality scene and adjusting the mixed reality scene in real time according to the interactive operation instruction.
3. The power transformation equipment training system based on mixed reality according to claim 1 or 2, wherein the man-machine interaction module comprises at least one of a gesture recognition unit for controlling an operation instruction by a gesture, a voice recognition unit for controlling an operation instruction by voice, and an external equipment unit for controlling by operating an external equipment;
the man-machine interaction module can be simultaneously connected with a plurality of mixed reality devices for team interaction.
4. The mixed reality based substation equipment training system of claim 1, wherein the control hub further comprises:
the simulation learning module is used for controlling the mixed reality module to construct a virtual learning scene and learning training knowledge of the power transformation equipment;
the simulation training module is used for carrying out daily operation maintenance, inspection and fault processing training on the power transformation equipment after carrying out space fusion projection on the virtual model and the real environment to the real world;
and the assessment module is used for carrying out assessment and evaluation on the results of the simulation learning and the simulation training.
5. The power transformation equipment training system based on mixed reality according to claim 1 or 4, wherein the data analysis feedback module analyzes the learning, training and assessment conditions of each training person according to the data in the working process of the training system, and generates a corresponding learning and training scheme.
6. The mixed reality based substation equipment training system of claim 4, wherein the simulation learning module comprises:
the equipment foundation learning unit is used for carrying out integral display explanation and split display explanation on the three-dimensional model of the power transformation equipment;
the device structure learning unit is used for explaining the internal structure of the three-dimensional model of the power transformation device and the relation between the structures;
and the standard demonstration learning unit is used for demonstrating standard operation processes of all training items in the simulation training module.
7. A mixed reality based substation equipment training system according to claim 1 or 2 or 4 or 6, characterized in that the system further comprises a user management module managing information and rights of different users, performing registration, login and authentication of the users, and establishment of a team comprising a plurality of users.
8. The mixed reality-based substation equipment training system according to claim 7, wherein the mixed reality equipment collects face information of an object to be logged in when the user is authenticated in the user management module;
judging whether the object to be logged in is legal or not according to the matching of the collected face information of the object to be logged in and the face information stored during the registration; and if the object to be logged in is legal, the control center sends login control information to the mixed reality equipment login training system.
9. A mixed reality-based substation equipment training method suitable for the substation equipment training system according to any one of claims 1 to 8, comprising:
constructing a three-dimensional model and a corresponding scene model of the power transformation equipment and storing the three-dimensional model and the corresponding scene model in a virtual model library;
responding to the selection of a login user on a simulation learning module or a simulation training module or an assessment module, and calling a corresponding virtual model from a virtual model library for simulation display;
and responding to an interactive operation instruction input by a login user through the man-machine interaction module, and displaying the state change of the virtual model under the interactive operation instruction.
10. The mixed reality-based substation equipment training method of claim 9, wherein displaying the state change of the virtual model under the interactive operation instruction comprises:
Simulating and displaying the state change of the virtual model under the interactive operation instruction according to the visual angle of the current operation user; and/or the number of the groups of groups,
simulating and displaying the state change of the virtual model under the interactive operation instruction according to the visual angle of any user in the team.
CN202311326758.XA 2023-10-13 2023-10-13 Substation equipment training system and method based on mixed reality Pending CN117409625A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311326758.XA CN117409625A (en) 2023-10-13 2023-10-13 Substation equipment training system and method based on mixed reality

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311326758.XA CN117409625A (en) 2023-10-13 2023-10-13 Substation equipment training system and method based on mixed reality

Publications (1)

Publication Number Publication Date
CN117409625A true CN117409625A (en) 2024-01-16

Family

ID=89497300

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311326758.XA Pending CN117409625A (en) 2023-10-13 2023-10-13 Substation equipment training system and method based on mixed reality

Country Status (1)

Country Link
CN (1) CN117409625A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117745496A (en) * 2024-02-19 2024-03-22 成都运达科技股份有限公司 Intelligent evaluation method, system and storage medium based on mixed reality technology

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117745496A (en) * 2024-02-19 2024-03-22 成都运达科技股份有限公司 Intelligent evaluation method, system and storage medium based on mixed reality technology
CN117745496B (en) * 2024-02-19 2024-05-31 成都运达科技股份有限公司 Intelligent evaluation method, system and storage medium based on mixed reality technology

Similar Documents

Publication Publication Date Title
CN107871414A (en) Maintenance electrician training tutoring system and method based on VR and BIM technology
RU2617972C1 (en) Simulator for operational and maintenance staff on the basis of virtual reality models of transformer substation
CN105139714A (en) Visualized simulation training system and method for electrified railway traction substation
CN110928419A (en) Transformer substation construction bottom crossing method based on virtual reality technology
CN108038910A (en) The implementation method of substation's main equipment Virtual Reality Demonstration
CN207096933U (en) Depot of urban railway transit vehicles operation maintenance management system based on BIM and VR technologies
CN109102731A (en) A kind of substation simulation platform and method based on VR
CN104090816B (en) A kind of extra-high voltage steel tube tower simulation training system and method
CN110322564A (en) A kind of 3 D model construction method of the substation operation environment suitable for VR/AR
KR20190124045A (en) System for training and education using augmented reality and method for training and education using therof
CN111738459A (en) Holographic expert system is examined in communication dispatch operation based on AR technique
CN110349467A (en) A kind of navigation training on maintenance system and method based on augmented reality
CN117409625A (en) Substation equipment training system and method based on mixed reality
CN102881202A (en) Virtual reality based on-load tap switch maintenance training method
CN113299139A (en) Nuclear power station main pump maintenance virtual simulation platform and construction method thereof
CN209183033U (en) A kind of power safety study safe simulation training system
Arendarski et al. Maintenance of complex machines in electric power systems using virtual reality techniques
Chang et al. A training simulation system for substation equipments maintenance
CN110764623A (en) Virtual reality simulation system for pipe gallery space
CN113128716A (en) Operation guidance interaction method and system
CN116580610A (en) Virtual transformer station operation and maintenance VR training and real training system
CN115631668A (en) Virtual real scene comprehensive training system for railway electric service
Dong et al. Research on overhead crane training system and its construction based on virtual reality
Zhang et al. Design and application of electric power skill training platform based on virtual reality technology
CN112233491A (en) Railway electric service construction simulation system based on virtual reality technology

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination