CN115407874B - VR maintenance training operation proficiency prediction method based on neural network - Google Patents
VR maintenance training operation proficiency prediction method based on neural network Download PDFInfo
- Publication number
- CN115407874B CN115407874B CN202210993556.XA CN202210993556A CN115407874B CN 115407874 B CN115407874 B CN 115407874B CN 202210993556 A CN202210993556 A CN 202210993556A CN 115407874 B CN115407874 B CN 115407874B
- Authority
- CN
- China
- Prior art keywords
- sequence
- maintenance
- training
- decoder
- name
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012423 maintenance Methods 0.000 title claims abstract description 90
- 238000012549 training Methods 0.000 title claims abstract description 57
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 17
- 230000009471 action Effects 0.000 claims abstract description 24
- 230000008569 process Effects 0.000 claims abstract description 16
- 238000010606 normalization Methods 0.000 claims description 24
- 230000007246 mechanism Effects 0.000 claims description 11
- 239000011521 glass Substances 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000000087 stabilizing effect Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000004148 unit process Methods 0.000 claims description 3
- 238000000638 solvent extraction Methods 0.000 claims description 2
- 230000008439 repair process Effects 0.000 claims 2
- 238000005516 engineering process Methods 0.000 abstract description 14
- 230000000694 effects Effects 0.000 abstract description 4
- 238000004088 simulation Methods 0.000 description 5
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000013209 evaluation strategy Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Tourism & Hospitality (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Human Computer Interaction (AREA)
- Molecular Biology (AREA)
- Educational Technology (AREA)
- Educational Administration (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a VR maintenance training operation proficiency prediction method based on a neural network, and belongs to the technical field of equipment virtual maintenance training. The invention uses a transducer and combines IETM flow and VR equipment to predict the skill of VR maintenance training operation. According to the prediction method, the IETM provides a VR maintenance training process, information, such as actions of a person receiving VR maintenance training, collected by a VR equipment sensor is used as input, the neural network is utilized to predict the operation proficiency of the person, the problem of a traditional knowledge tracking technology is solved, and the prediction effect is more stable.
Description
Technical Field
The invention belongs to the technical field of equipment virtual maintenance training, and particularly relates to a VR maintenance training operation proficiency prediction method based on a neural network.
Background
Knowledge tracking is an important research direction of artificial intelligence technology, and aims to judge the knowledge grasping degree of related personnel by establishing a model of the change of knowledge state along with time, thereby providing personalized guidance and achieving the aim of artificial intelligence auxiliary teaching.
The VR technology is a computer simulation system capable of creating and experiencing a virtual world, and utilizes a computer to generate a simulation environment, is a system simulation of interactive three-dimensional dynamic view and entity behaviors fused by multi-source information, enables a user to be immersed in the environment, and is a collection of simulation technology and multiple technologies such as computer graphics, man-machine interface technology, multimedia technology, sensing technology, network technology and the like. The method mainly comprises the aspects of simulation environment, perception, natural skill, sensing equipment and the like, wherein the simulation environment is a three-dimensional realistic image which is generated by a computer and is dynamic in real time. With the great progress of VR technology, there are more and more systems for performing equipment maintenance training by VR technology, however, the evaluation of personnel receiving maintenance training is generally performed manually, and there is a lack of effective proficiency objective evaluation strategies and methods. Incorporating knowledge tracking into the VR maintenance training system in conjunction with maintenance data in the IETM enables effective, objective predictions of the operational proficiency of the person receiving the maintenance training using artificial intelligence techniques.
The traditional knowledge tracking technology can only make a prejudgment on the knowledge mastering condition of a person through a single input, and generally utilizes an RNN structure, so that knowledge mastering records of the person cannot be well combined, and the prediction result of a single user has larger fluctuation.
Disclosure of Invention
First, the technical problem to be solved
The invention aims to solve the technical problems that: how to design a VR maintenance training operation proficiency prediction method, the prediction effect is more stable.
(II) technical scheme
In order to solve the technical problems, the invention provides a VR maintenance training operation proficiency prediction method based on a neural network, which comprises the following steps:
the first stage: reading IETM database and data stream of a certain device by VR glasses, creating IETM maintenance training flow, dividing key points of corresponding training task to obtain step name of maintenance operation, time of operation and number of operation attempts,
and a second stage: the person who receives VR maintenance training wears VR glasses and an operating handle;
and a third stage: the VR glasses construct a VR scene of the maintained equipment according to the IETM maintenance training process;
fourth stage: the personnel receiving VR maintenance training carries out maintenance operation according to the operation standard given by the IETM maintenance training flow, acquires action information by using an operation handle, and stores the action information of key points in the maintenance process;
fifth stage: after the VR maintenance task is finished, splitting action information of an operation handle according to key points, and coding the action information into an action sequence conforming to the network input format specification by combining the next information of maintenance operation, namely operation time and operation attempt times, wherein in the stage, the corresponding relation among operation step names, operation time and operation attempt times is considered to be maintained, a position code is added to the action sequence in the coding process, and the position information of the action sequence is provided for a subsequent multi-head attention mechanism;
sixth stage: and inputting the input variable, namely the action sequence information, into the network model to predict maintenance proficiency.
Preferably, in the sixth stage, the network model is designed to include the following parts:
a gating unit introduced in the input part of the network, the gating unit processes the input average proficiency result sequence of the maintenance operation, and then utilizes the processed result to perform gating constraint on the fusion result of the step name sequence, the operation time sequence, the position code and the operation try number sequence of the maintenance operation, and the output result E of the gating unit gate Expressed as:
wherein E is position ,E name ,E time ,E times ,E results Respectively representing a position sequence, a name sequence, a time sequence, a trial number sequence and an average proficiency result sequence of the maintenance operation;
secondly, a trans-former-based codec performs parallel computation by using a trans-former structure; the coding part of the coder-decoder consists of three cascaded encoders, and a single encoder comprises a multi-head attention module, a first layer normalization module, a feedforward network and a second layer normalization module;
let Q denote vectors obtained by processing the maintenance operation name sequence, K denote vectors obtained by processing the maintenance operation time sequence and the number of times sequence, V denote results obtained by gating the maintenance operation name sequence, the time sequence, the number of attempts sequence, and the position sequence with the maintenance proficiency result sequence, and the output results of the three encoders are expressed as:
wherein W is q ，W k ，W v Representing a linear transformation matrix, E, which is the operation process of the head in the multi-head attention mechanism position ，E time ，E name ，E times ，E gate The position information sequence, the operation time sequence, the operation step name sequence, the operation try number sequence and the output result of the gate control unit are respectively represented by the maintenance operation sequence; the feed forward network is used to further extract the characteristics of the input variables. The first layer normalization module and the second layer normalization module are used for stabilizing the gradient of the whole network;
the decoder of the decoding part is also composed of three cascaded decoders, and a single decoder comprises a second multi-head attention module, a third layer normalization module, a third multi-head attention module and a fourth layer normalization module, wherein the input of the third multi-head attention module of the first layer decoder is the output of the last layer of encoder, and the difference from the encoder is that the output of the second layer normalization module in the encoder of the corresponding layer is introduced into the second multi-head attention module of the decoder;
in the decoder stage of the network model, the output of the second multi-head attention module of the decoder is the Q end of the third multi-head attention module, the output of the encoder of the corresponding level is added in the input of the second multi-head attention module in the decoder, and the output is set as the K end and the V end of the second multi-head attention module in the decoder;
and thirdly, the prediction unit inputs the output information of the decoder into a feedforward neural network of the prediction unit, performs normalization operation in the prediction unit, sends the output information into a full-connection layer of the prediction unit, and finally constrains the output of the full-connection layer to be within a range of [0,1] through a Sigmoid function to serve as predicted maintenance proficiency information.
Preferably, parameters such as the step name of the equipment servicing operation, the time of the operation, the number of operation attempts, may be adaptively adjusted according to the specific training steps.
Preferably, the results of the keypoint partitioning are shown in table 1:
table 1 data parameters for training
| Name of the name | Value range | Unit (B) |
| Name of operation procedure | [0,6] | - |
| Time of operation | [0,3600] | Second of |
| Number of operation attempts | [0,10] | - |
Preferably, the network model optimizes different feature portions in the input variables by introducing a multi-headed attentiveness mechanism in the network model.
Preferably, the position coding is implemented in a sinusoidal position coding manner.
Preferably, the codec uses the Transfomer structure to perform parallel computation while also taking into account global information of the input features.
The invention also provides a prediction system for realizing the method.
The invention also provides application of the method in the technical field of equipment virtual maintenance training.
The invention also provides application of the system in the technical field of equipment virtual maintenance training.
(III) beneficial effects
The invention uses a transducer and combines IETM flow and VR equipment to predict the skill of VR maintenance training operation. According to the prediction method, the IETM provides a VR maintenance training process, information, such as actions of a person receiving VR maintenance training, collected by a VR equipment sensor is used as input, the neural network is utilized to predict the operation proficiency of the person, the problem of a traditional knowledge tracking technology is solved, and the prediction effect is more stable.
Drawings
FIG. 1 is a schematic diagram of a knowledge tracking model designed in accordance with the present invention;
FIG. 2 is a schematic diagram of a VR maintenance training operation proficiency prediction neural network in accordance with the present invention.
Detailed Description
To make the objects, contents and advantages of the present invention more apparent, the following detailed description of the present invention will be given with reference to the accompanying drawings and examples.
The invention provides a VR maintenance training operation proficiency prediction method based on a neural network, which is a knowledge tracking method for predicting the operation proficiency of a receiving virtual maintenance training person by using the neural network based on a transducer. According to the prediction method, an IETM provides a VR maintenance training process, information, such as actions of a person receiving VR maintenance training, collected by a VR equipment sensor is used as input, and the prediction of the operation proficiency of the person is realized by using a neural network. The range of the input parameters of the neural network can be adaptively adjusted according to a specific VR maintenance training task.
Referring to fig. 1, the VR maintenance training operation proficiency prediction method provided by the present invention includes the following steps:
the first stage: and reading an IETM database and a data stream of a certain device by using VR glasses, creating an IETM maintenance training process, and dividing key points of corresponding training tasks. Common key point division is shown in table 1, and the table includes step names of equipment maintenance operations, operation time and operation attempt times, and the parameters can be adaptively adjusted according to specific training steps.
Table 1 data parameters for training
| Name of the name | Value range | Unit (B) |
| Name of operation procedure | [0,6] | - |
| Time of operation | [0,3600] | Second(s) |
| Number of operation attempts | [0,10] | - |
And a second stage: the person who receives VR maintenance training wears VR glasses, operating handle.
And a third stage: and the VR glasses construct a VR scene of the maintained equipment according to the IETM maintenance training process.
Fourth stage: and the personnel receiving VR maintenance training performs maintenance operation according to the operation specification given by the IETM maintenance training flow, acquires action information by using an operation handle, and stores the action information of key points in the maintenance process.
Fifth stage: after the VR maintenance task is finished, the action information of the operation handle is split according to key points, and the action sequence which accords with the network input format specification is encoded by combining the next information (operation time and operation attempt times) of maintenance operation. In this stage, in consideration of maintaining the correspondence between the operation step name, the operation time and the operation attempt number, a position code is added to the motion sequence in the above-mentioned coding process, and the coding mode is implemented in a sinusoidal position coding mode, so as to provide the position information of the motion sequence for the subsequent multi-head attention mechanism.
Sixth stage: input variables (action sequence information) are input to a network model designed to predict maintenance proficiency, wherein the network model is designed to include the following major components:
and a gating unit introduced in the input part of the network, wherein the gating unit processes the input average proficiency result sequence of the maintenance operation, and then utilizes the processed result to carry out gating constraint on the fusion result of the step name sequence, the operation time sequence, the position code and the operation try number sequence of the maintenance operation, thereby further improving the prediction accuracy. Output result E of gating unit gate Expressed as:
wherein E is position ,E name ,E time ,E times ,E results The sequence of positions, the sequence of names, the sequence of time, the sequence of attempts and the sequence of average proficiency results for the maintenance operation are shown, respectively.
And secondly, the encoder and decoder based on the Transformer can use the Transformer structure to perform parallel calculation and simultaneously consider the global information of the input characteristics, thereby further enhancing the accuracy of the network on the proficiency prediction. The coding part of the coder consists of three cascaded encoders, and the structure of a single encoder is shown in a broken line frame on the left side of fig. 1, and the coding part comprises a multi-head attention module, a first layer normalization module, a feedforward network and a second layer normalization module.
The Query (Query), key (Key) and Value (Value) in the self-attention operation of the network are generally indicated by Q, K, V, Q in the present invention represents a vector obtained by processing a maintenance operation name sequence, K represents a vector obtained by processing a maintenance operation time sequence and a number of times sequence, V represents a result obtained by gating a maintenance operation name sequence, a time sequence, a number of attempts sequence and a position sequence with a maintenance proficiency result sequence, and the output results of the three encoders can be expressed as follows:
wherein W is q ，W k ，W v Representing a linear transformation matrix, E, which is the operation process of the head in the multi-head attention mechanism position ，E time ，E name ，E times ，E gate The position information sequence, the operation time sequence, the operation step name sequence, the operation try number sequence and the output result of the gate control unit respectively represent the maintenance operation sequence. By introducing a multi-head attention mechanism in the network, the network model can be enabled to optimize different characteristic parts in the input variables, so that the deviation problem caused by using a single attention mechanism is avoided. The feed forward network functions to further extract the characteristics of the input variables. The first layer normalization module and the second layer normalization module are used for stabilizing the gradient of the whole network and preventing the gradient explosion phenomenon.
The decoder is also composed of three cascaded decoders, and the frame of a single decoder is shown as a dotted line box on the right side in fig. 1, and similar to the encoder, the decoder also comprises a second multi-head attention module, a third layer normalization module, a third multi-head attention module and a fourth layer normalization module. Wherein the input of the third multi-headed attention module (the upper "multi-headed attention" in the right dashed box in fig. 1) of the first layer decoder is the output of the last layer encoder, unlike the encoder, the output of the second layer normalization module in the encoder of the corresponding level is introduced in the second multi-headed attention module of the decoder (the lower "multi-headed attention" in the right dashed box in fig. 1).
It should be noted that, in the decoder stage of the network model, the output of the second multi-head attention module of the decoder is the Q end of the third multi-head attention module, and in order to introduce low-level feature information, the output of the encoder of the corresponding level is added to the input of the second multi-head attention module in the decoder, and is set as the K end and the V end of the second multi-head attention module in the decoder.
And fourthly, the prediction unit inputs the output information of the decoder into a feedforward neural network of the prediction unit, performs normalization operation in the prediction unit, sends the output information into a full-connection layer of the prediction unit, and finally constrains the output of the full-connection layer to be within a range of [0,1] through a Sigmoid function to serve as predicted maintenance proficiency information.
In the actual reasoning stage of the network, only the operation step name sequence, the operation time sequence and the operation attempt number sequence fed back by the handle of the operator are input as the input of the network model, and the computing equipment can automatically perform reasoning operation on the operation information of the operator by combining the network model and output the final predicted maintenance proficiency result.
It can be seen that the invention provides a VR maintenance training operation proficiency prediction method based on a neural network, in which a knowledge tracking model combining an IETM and VR equipment is designed, the IETM provides a maintenance training flow, maintenance training operation information acquired by the VR equipment is used as input, and a transducer model combining an attention mechanism is used to replace an RNN structure to predict the maintenance proficiency of personnel, so that the problem of the traditional knowledge tracking technology is solved, and the prediction effect is more stable.
The foregoing is merely a preferred embodiment of the present invention, and it should be noted that modifications and variations could be made by those skilled in the art without departing from the technical principles of the present invention, and such modifications and variations should also be regarded as being within the scope of the invention.
Claims (9)
1. The VR maintenance training operation proficiency prediction method based on the neural network is characterized by comprising the following steps of:
the first stage: the VR glasses are utilized to read the IETM database and the data flow of a certain device, an IETM maintenance training flow is established, the key points of the corresponding training tasks are divided, the step name of the maintenance operation of the device, the time of the operation and the number of operation attempts are obtained,
and a second stage: the person who receives VR maintenance training wears VR glasses and an operating handle;
and a third stage: the VR glasses construct a VR scene of the maintained equipment according to the IETM maintenance training process;
fourth stage: the personnel receiving VR maintenance training carries out maintenance operation according to the operation standard given by the IETM maintenance training flow, acquires action information by using an operation handle, and stores the action information of key points in the maintenance process;
fifth stage: after the VR maintenance task is finished, splitting action information of an operation handle according to key points, and coding the action information into an action sequence conforming to the network input format specification by combining the next information of maintenance operation, namely operation time and operation attempt times, wherein in the stage, the corresponding relation among operation step names, operation time and operation attempt times is considered to be maintained, a position code is added to the action sequence in the coding process, and the position information of the action sequence is provided for a subsequent multi-head attention mechanism;
sixth stage: inputting the input variable, namely the action sequence information, into a network model to predict maintenance proficiency;
in the sixth stage, the network model is designed to include the following parts:
a gating unit introduced in the input part of the network, the gating unit processes the input average proficiency result sequence of the maintenance operation, and then utilizes the processed result to perform gating constraint on the fusion result of the step name sequence, the operation time sequence, the position code and the operation try number sequence of the maintenance operation, and the output result E of the gating unit gate Expressed as:
wherein E is position ,E name ,E time ,E times ,E results Respectively representing a position sequence, a name sequence, a time sequence, an operation try number sequence and an average proficiency result sequence of the maintenance operation;
secondly, a trans-former-based codec performs parallel computation by using a trans-former structure; the coding part of the coder-decoder consists of three cascaded encoders, and a single encoder comprises a multi-head attention module, a first layer normalization module, a feedforward network and a second layer normalization module;
let Q denote vectors obtained by processing the maintenance operation name sequence, K denote vectors obtained by processing the maintenance operation time sequence and the operation trial number sequence, V denote results obtained by gating the maintenance operation name sequence, the time sequence, the operation trial number sequence, and the position sequence with the maintenance proficiency result sequence, and the output results of the three encoders are expressed as:
wherein W is q ，W k ，W v Representing a linear transformation matrix, E, which is the operation process of the head in the multi-head attention mechanism position ，E time ，E name ，E times ，E gate Respectively are provided withA position information sequence representing a maintenance operation sequence, an operation time sequence, an operation step name sequence, an operation attempt number sequence and an output result of the gating unit; the feedforward network is used for further extracting the characteristics of the input variables; the first layer normalization module and the second layer normalization module are used for stabilizing the gradient of the whole network;
the decoder of the decoding part is also composed of three cascaded decoders, and a single decoder comprises a second multi-head attention module, a third layer normalization module, a third multi-head attention module and a fourth layer normalization module, wherein the input of the third multi-head attention module of the first layer decoder is the output of the last layer of encoder, and the difference from the encoder is that the output of the second layer normalization module in the encoder of the corresponding layer is introduced into the second multi-head attention module of the decoder;
in the decoder stage of the network model, the output of the second multi-head attention module of the decoder is the Q end of the third multi-head attention module, the output of the encoder of the corresponding level is added in the input of the second multi-head attention module in the decoder, and the output is set as the K end and the V end of the second multi-head attention module in the decoder;
and thirdly, the prediction unit inputs the output information of the decoder into a feedforward neural network of the prediction unit, performs normalization operation in the prediction unit, sends the output information into a full-connection layer of the prediction unit, and finally constrains the output of the full-connection layer to be within a range of [0,1] through a Sigmoid function to serve as predicted maintenance proficiency information.
2. The method of claim 1, wherein parameters of step name of equipment maintenance operation, time of operation, number of operation attempts are adaptively adjusted according to specific training steps.
3. The method of claim 1, wherein the results of the keypoint partitioning are shown in table 1:
table 1 data parameters for training
。
4. The method of claim 1, wherein the network model optimizes different feature portions in the input variables by introducing a multi-headed attention mechanism in the network model.
5. The method of claim 1, wherein the position encoding is implemented using sinusoidal position encoding.
6. The method of claim 1, wherein the codec uses a fransfomer structure to perform parallel computation while also taking into account global information of input features.
7. A predictive system for implementing the method of any one of claims 1 to 6.
8. Use of a method according to any one of claims 1 to 6 in the field of equipment virtual repair training techniques.
9. Use of the system according to claim 7 in the field of equipment virtual repair training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210993556.XA CN115407874B (en) | 2022-08-18 | 2022-08-18 | VR maintenance training operation proficiency prediction method based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210993556.XA CN115407874B (en) | 2022-08-18 | 2022-08-18 | VR maintenance training operation proficiency prediction method based on neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115407874A CN115407874A (en) | 2022-11-29 |
| CN115407874B true CN115407874B (en) | 2023-07-28 |
Family
ID=84158893
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210993556.XA Active CN115407874B (en) | 2022-08-18 | 2022-08-18 | VR maintenance training operation proficiency prediction method based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115407874B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116542290B (en) * | 2023-06-25 | 2023-09-08 | 城云科技（中国）有限公司 | Information prediction model construction method, device and application based on multi-source multi-dimensional data |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111860785A (en) * | 2020-07-24 | 2020-10-30 | 中山大学 | Time sequence prediction method and system based on attention mechanism cyclic neural network |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114625077A (en) * | 2016-05-09 | 2022-06-14 | 强力物联网投资组合2016有限公司 | Method and system for industrial internet of things |
| CN108492661A (en) * | 2018-04-02 | 2018-09-04 | 四川中烟工业有限责任公司 | A kind of virtual experiment training system |
| US10499081B1 (en) * | 2018-06-19 | 2019-12-03 | Sony Interactive Entertainment Inc. | Neural network powered codec |
| US11376500B2 (en) * | 2019-02-27 | 2022-07-05 | Nvidia Corporation | Gamer training using neural networks |
| CN110154024B (en) * | 2019-05-22 | 2020-09-15 | 清华大学 | Assembly control method based on long-term and short-term memory neural network incremental model |
| CN110599040A (en) * | 2019-09-16 | 2019-12-20 | 中国人民解放军陆军工程大学 | Maintenance training evaluation method and system and terminal equipment |
| KR20210107250A (en) * | 2020-02-24 | 2021-09-01 | 주식회사 한화 | Apparatus and method for evaluating maintenance trainling results based on virtual reality |
| US20210287071A1 (en) * | 2020-03-12 | 2021-09-16 | Morgan State University | Method and Apparatus for Augmented Data Anomaly Detection |
| CN111507488B (en) * | 2020-04-14 | 2021-01-01 | 李松 | VR-based vehicle maintenance auxiliary system |
| CN111695703A (en) * | 2020-05-29 | 2020-09-22 | 甘肃畜牧工程职业技术学院 | Internet-based intelligent automobile maintenance control system and control method |
| CN112289434A (en) * | 2020-11-03 | 2021-01-29 | 珠海虎江科技有限公司 | Medical training simulation method, device, equipment and storage medium based on VR |
| CN113642379B (en) * | 2021-05-18 | 2024-03-01 | 北京航空航天大学 | Human body posture prediction method and system based on attention mechanism fusion multi-flow diagram |
| CN114023132A (en) * | 2021-11-03 | 2022-02-08 | 中国人民解放***箭军工程大学 | Maintenance training system and method based on digital twins |
| CN114863353A (en) * | 2022-04-19 | 2022-08-05 | 华南理工大学 | Method and device for detecting relation between person and object and storage medium |
-
2022
- 2022-08-18 CN CN202210993556.XA patent/CN115407874B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111860785A (en) * | 2020-07-24 | 2020-10-30 | 中山大学 | Time sequence prediction method and system based on attention mechanism cyclic neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115407874A (en) | 2022-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ghosh et al. | Synthesis of compositional animations from textual descriptions | |
| CN110490946B (en) | Text image generation method based on cross-modal similarity and antagonism network generation | |
| CN111695779A (en) | Knowledge tracking method, knowledge tracking device and storage medium | |
| CN111274398A (en) | Method and system for analyzing comment emotion of aspect-level user product | |
| CN115797606B (en) | 3D virtual digital human interaction action generation method and system based on deep learning | |
| CN111930887A (en) | Multi-document multi-answer machine reading understanding system based on joint training mode | |
| CN113204633B (en) | Semantic matching distillation method and device | |
| CN115407874B (en) | VR maintenance training operation proficiency prediction method based on neural network | |
| CN113516133B (en) | Multi-modal image classification method and system | |
| CN117218498B (en) | Multi-modal large language model training method and system based on multi-modal encoder | |
| CN113934887A (en) | No-proposal time sequence language positioning method based on semantic decoupling | |
| Xu et al. | Bridging video and text: A two-step polishing transformer for video captioning | |
| CN114970517A (en) | Visual question and answer oriented method based on multi-modal interaction context perception | |
| CN114660934A (en) | Mechanical arm autonomous operation strategy learning method based on vision-touch fusion | |
| CN116580278A (en) | Lip language identification method, equipment and storage medium based on multi-attention mechanism | |
| CN114020900B (en) | Chart English abstract generating method based on fusion space position attention mechanism | |
| CN116910294A (en) | Image filter generation method based on emotion analysis | |
| CN115292533B (en) | Cross-modal pedestrian retrieval method driven by visual positioning | |
| CN111552817A (en) | Electric power scientific and technological achievement knowledge map completion method | |
| WO2023168818A1 (en) | Method and apparatus for determining similarity between video and text, electronic device, and storage medium | |
| CN116070025A (en) | Interpretable recommendation method based on joint score prediction and reason generation | |
| CN116362247A (en) | Entity extraction method based on MRC framework | |
| CN115422329A (en) | Knowledge-driven multi-channel screening fusion dialogue generation method | |
| CN114840680A (en) | Entity relationship joint extraction method, device, storage medium and terminal | |
| CN114255300A (en) | Flow chart extraction model training method, flow chart extraction model acquisition equipment and flow chart extraction model acquisition medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |