CN114881179B - Intelligent experiment method based on intention understanding - Google Patents

Intelligent experiment method based on intention understanding Download PDF

Info

Publication number
CN114881179B
CN114881179B CN202210796763.6A CN202210796763A CN114881179B CN 114881179 B CN114881179 B CN 114881179B CN 202210796763 A CN202210796763 A CN 202210796763A CN 114881179 B CN114881179 B CN 114881179B
Authority
CN
China
Prior art keywords
intention
probability
experiment
experimenter
channel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN202210796763.6A
Other languages
Chinese (zh)
Other versions
CN114881179A (en
Inventor
冯志全
杨璐榕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Jinan
Original Assignee
University of Jinan
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Jinan filed Critical University of Jinan
Priority to CN202210796763.6A priority Critical patent/CN114881179B/en
Publication of CN114881179A publication Critical patent/CN114881179A/en
Application granted granted Critical
Publication of CN114881179B publication Critical patent/CN114881179B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/279Recognition of textual entities
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/40Processing or translation of natural language
    • G06F40/42Data-driven translation
    • G06F40/44Statistical methods, e.g. probability models
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Human Computer Interaction (AREA)
  • Electrically Operated Instructional Devices (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention relates to the technical field of data processing, and provides an intelligent experiment method based on intention understanding. The method has the advantages of accurately understanding the user intention and greatly improving the learning concentration degree and efficiency of learners.

Description

Intelligent experiment method based on intention understanding
Technical Field
The invention relates to the technical field of data processing, in particular to an intelligent experiment method based on intention understanding.
Background
With the development of virtual reality technology, the virtual reality technology is increasingly applied to production, life and scientific research and education activities of people. In the teaching, the virtual reality technology has interactive characteristics and an intelligent three-dimensional dynamic vision and virtual behavior simulation system, so that learners can be introduced into different situations more quickly, and the limitation of many real teaching can be overcome.
For example, many chemical experimental designs are currently performed on a virtual-real fused experimental platform, and in a virtual platform, a real operation process and a virtual experimental phenomenon display can be performed by using a fused real-object interaction suite.
According to the heart flow theory, the heart flow experience is a completely immersed feeling of forgetting fatigue, time and other things except the current things when doing something, if students can reach such a state in the experimental process, the students with different bases are guided to find confidence and places of whereabouts of learning by creating reasonable scenes, the learning targets of the students are clear and accord with the abilities of the students, the learning interests of the students are stimulated to the maximum extent, the students can gradually obtain the heart flow experience, and the experience becomes the power of the students for continuously meeting challenges and realizing self-growth.
In practical use, the interactive system collects various static information and dynamic information of an operator by using multiple channels, but how to fuse various multi-channel information data and more accurately predict the intention of the operator and apply the result of the intention prediction to the interesting learning mode design, designs an interesting learning strategy by fusing the heart flow theory, improves the learning interest and the learning efficiency of students, and is an application hotspot of the current virtual reality in chemical experiment application.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides an intelligent experiment method based on intention understanding according to a model for obtaining the heart fluid experiment learning.
The invention is realized through the following technical scheme, and provides an intelligent experiment method based on intention understanding, which is characterized in that multi-mode fusion intention understanding is firstly carried out, and then a weak point capturing feedback mechanism, an individualized dynamic experiment mode matching strategy and a strategy of a man-machine competition answering strategy are adopted;
the multimodal fusion is intended to be understood to include the following steps:
(1) processing multi-channel data;
firstly, setting initial probability, and establishing a state transition probability model based on the operation behavior of an experimenter
Figure 775433DEST_PATH_IMAGE001
The transition probability from the j-state to the i-state is recorded as
Figure 630256DEST_PATH_IMAGE002
Establishing a behavior intention library, collecting information of a scene channel and a voice channel, processing the information and then carrying out quantitative matching;
for the calculation of the position information, the intention probability of the current position channel is obtained through a formula (1) according to the hand coordinates captured in real time and the coordinates of each apparatus in the scene,
Figure 206731DEST_PATH_IMAGE003
(1)
wherein,
Figure 788891DEST_PATH_IMAGE004
r is an operation intention probability surge region set according to a large number of experiments as the probability of the ith intention of the position lane, S1, S2, S3 represent distances,
Figure 540946DEST_PATH_IMAGE005
is the probability weight in the R region;
for the calculation of the motion direction, the moving track of the hand is captured in real time, the direction relation between the motion direction of the hand and each apparatus in the scene is calculated to obtain a probability value, the specific calculation method is as formula (2),
Figure 51824DEST_PATH_IMAGE006
(2)
wherein,
Figure 990962DEST_PATH_IMAGE007
the probability of the ith intention of a directional channel,
Figure 642392DEST_PATH_IMAGE008
the distance between the ith equipment and the hand movement track;
for voice information, firstly, a voice database is established according to intentions of different experiment intention sets
Figure 983374DEST_PATH_IMAGE009
Wherein, the voice database is in one-to-one correspondence with the intention set of the chemical experiment,
Figure 662224DEST_PATH_IMAGE010
which indicates the type of the experiment to be performed,
Figure 354236DEST_PATH_IMAGE011
shows the second experiment
Figure 12620DEST_PATH_IMAGE012
The voice corresponding to the individual intention is presented,
Figure 473688DEST_PATH_IMAGE013
representing the number of intentions of the experiment, sorting out keywords with different intentions in an intention set, acquiring voice text information by adopting a Baidu API (application programming interface), extracting the keywords, and calculating the matching degree of the keywords captured by a computing system and each intention keyword in an intention library to calculate the probability of each intention calculated according to the voice information in real time
Figure 326369DEST_PATH_IMAGE014
(2) Multi-channel information fusion;
firstly, fusing the obtained dynamic information of each channel, and calculating the intention probability variance of each channel to obtain the weight of each channel, wherein the specific formula is as follows,
Figure 364732DEST_PATH_IMAGE015
(3)
wherein,
Figure 577538DEST_PATH_IMAGE016
representing three channels of position, direction and voice,
Figure 142381DEST_PATH_IMAGE017
for the weight of each channel it is desirable to,
Figure 415230DEST_PATH_IMAGE018
is a first
Figure 829638DEST_PATH_IMAGE019
The probability of the ith intention of an individual channel,
Figure 580556DEST_PATH_IMAGE020
is a first
Figure 265484DEST_PATH_IMAGE021
Average probability of n intents in each channel;
final dynamic information prediction probability of each intention
Figure 974814DEST_PATH_IMAGE022
Is composed of
Figure 348289DEST_PATH_IMAGE023
(4)
Combining the prediction probability obtained by the static information with the probability obtained by the dynamic information to obtain the intention probability after fusion, wherein the formula is as follows
Figure 434057DEST_PATH_IMAGE024
(5)
Wherein,
Figure 707912DEST_PATH_IMAGE025
as the probability of the predicted ith intention,
Figure 447198DEST_PATH_IMAGE026
to state transition probabilities from the jth intent in the jth intent library to the ith intent,
Figure 822816DEST_PATH_IMAGE027
probability of the ith intention obtained by fusing the dynamic information;
selecting
Figure 952356DEST_PATH_IMAGE028
The largest one corresponds to the predicted current intent.
Preferably, a vulnerability capturing and feedback mechanism is set in the steps of monitoring fluency and operation of user experiment operation, and the vulnerability capturing and feedback relearning process comprises the following steps:
(1) recording the knowledge of the suspicious weaknesses: monitoring the operation process of an experimenter in real time, and recording the points of the experimenter which are guided and warned;
(2) relearning the vulnerability knowledge: testing the question matched with the question with the highest conformity with the captured weak point knowledge in the question removing library, and if the guide and the warning are finished and the experimenter already masters the knowledge point, continuing to perform the next experiment; otherwise, the computer explains the knowledge point by voice or video, extracts the question from the question bank again after the completion of the explanation and tests the question until the answer is correct, which means that the experimenter has completely mastered the knowledge point, and after the experiment is finished, the computer returns a summary of the weak point knowledge point to help the students to more intuitively see the weak points and to review the weak points.
Preferably, the individual experiment mode is set according to different experiment states of each experimenter by monitoring the operation time and the accuracy of the experimenter, and comprises the following strategies,
(1) timely excitation: when an experimenter well completes a certain step, the system can issue corresponding rewards according to the completion time and the operation specification, wherein the rewards adopt a virtual gold coin form;
(2) intelligent navigation: the system monitors the operation process of an experimenter in real time, and when the fact that the experimenter stays in a certain step for too long time and cannot quickly perform the next step is detected, the system can conduct timely guidance; meanwhile, through intention understanding, the computer predicts that an experimenter is about to perform an error operation, and the system also warns;
(3) the barrier is established to intelligence: the hidden level is set, when the experimenter operates smoothly, when the whole experiment process reaches a certain condition, the corresponding hidden level can be unlocked, the setting of the hidden level is the expansion and the pull-out of the current basic experiment knowledge point, and the experimenter can also obtain extra gold coin rewards when learning more knowledge.
In conclusion, the invention can evaluate multi-channel dynamic information, automatically endow different weights to each channel according to the value content of effective information, can acquire static information according to the relationship between human behaviors and motivations, and fuse the static information and the dynamic information by adopting a fusion strategy, thereby improving the accuracy of intention prediction. The method has the advantages of accurately understanding the user intention and greatly improving the learning concentration degree and efficiency of learners.
Drawings
FIG. 1 is a schematic diagram of the overall architecture of the intelligent experimental method based on the intention understanding of the invention;
fig. 2 is a schematic view of a schematic flow structure of a vulnerability capture feedback mechanism in an intelligent experimental method based on intention understanding.
Detailed Description
In order to clearly illustrate the technical features of the present invention, the present invention is further illustrated by the following detailed description with reference to the accompanying drawings.
As shown in fig. 1 to 2, the invention provides an intelligent experiment method based on intention understanding, which first performs multi-modal fusion intention understanding, and then adopts a vulnerability capture feedback mechanism, a personalized dynamic experiment mode matching strategy and a man-machine competition question answering strategy;
the multimodal fusion is intended to be understood to include the following steps:
(1) processing multi-channel data;
firstly setting initial probability, inviting a plurality of classmates of different grades to operate concentrated sulfuric acid dilution experiments in sequence, recording the operation sequence and selected instruments of the experiments, analyzing and establishing a state transition probability model based on the operation behaviors of experimenters, wherein the model is one
Figure 97030DEST_PATH_IMAGE029
The transition probability from the j-state to the i-state is recorded as
Figure 131851DEST_PATH_IMAGE030
The dynamic information comprises a plurality of channels, so that information supplement can be better realized, and the precision of a prediction result is improved, so that the robustness of a prediction model is improved;
for the calculation of the position information, the intention probability of the current position channel is obtained through a formula (1) according to the hand coordinates captured in real time and the coordinates of each equipment in the scene,
Figure 994764DEST_PATH_IMAGE003
(1)
wherein,
Figure 641909DEST_PATH_IMAGE004
r is an operation intention probability surge region set according to a large number of experiments as the probability of the ith intention of the position lane, S1, S2, S3 represent distances,
Figure 641089DEST_PATH_IMAGE005
is the probability weight in the R region;
for the calculation of the motion direction, the moving track of the hand is captured in real time, the direction relation between the motion direction of the hand and each apparatus in the scene is calculated to obtain a probability value, the specific calculation method is as formula (2),
Figure 722177DEST_PATH_IMAGE006
(2)
wherein,
Figure 56075DEST_PATH_IMAGE007
the probability of the ith intention of a directional channel,
Figure 21757DEST_PATH_IMAGE008
the distance between the ith equipment and the hand movement track;
for voice information, firstly, a voice database is established according to intentions of different experiment intention sets
Figure 623247DEST_PATH_IMAGE009
Wherein, the voice database is in one-to-one correspondence with the intention set of the chemical experiment,
Figure 485023DEST_PATH_IMAGE010
which indicates the type of the experiment to be performed,
Figure 837376DEST_PATH_IMAGE011
shows the second experiment
Figure 75591DEST_PATH_IMAGE012
The voice corresponding to the individual intention is presented,
Figure 65674DEST_PATH_IMAGE013
representing the number of intentions of the experiment, sorting out keywords with different intentions in an intention set, acquiring voice text information by adopting a Baidu API (application programming interface), extracting the keywords, and calculating the matching degree of the keywords captured by a computing system and each intention keyword in an intention library to calculate the probability of each intention calculated according to the voice information in real time
Figure 222986DEST_PATH_IMAGE014
(2) Multi-channel information fusion;
firstly, fusing the obtained dynamic information of each channel, and calculating the intention probability variance of each channel to obtain the weight of each channel, wherein the specific formula is as follows,
Figure 547788DEST_PATH_IMAGE015
(3)
wherein,
Figure 838961DEST_PATH_IMAGE016
representing position, squareThree channels of direction and voice are provided,
Figure 667240DEST_PATH_IMAGE017
for the weight of each channel it is desirable to,
Figure 149781DEST_PATH_IMAGE018
is as follows
Figure 430720DEST_PATH_IMAGE019
The probability of the ith intention of an individual channel,
Figure 525584DEST_PATH_IMAGE020
is as follows
Figure 739528DEST_PATH_IMAGE021
Average probability of n intents in each channel;
final per-intent dynamic information prediction probability
Figure 395899DEST_PATH_IMAGE022
Is composed of
Figure 164135DEST_PATH_IMAGE023
(4)
Combining the prediction probability obtained by the static information with the probability obtained by the dynamic information to obtain the intention probability after fusion, wherein the formula is as follows
Figure 62690DEST_PATH_IMAGE024
(5)
Wherein,
Figure 865561DEST_PATH_IMAGE025
as the probability of the predicted ith intention,
Figure 801156DEST_PATH_IMAGE026
the state of the jth intention to the ith intention from the jth intention in the ith intention libraryThe probability of the transition is,
Figure 70070DEST_PATH_IMAGE027
the probability of the ith intention obtained by fusing the dynamic information;
selecting
Figure 257469DEST_PATH_IMAGE028
The largest one corresponds to the predicted current intent.
Based on the intention understanding fusion strategy, the next operation intention of the user can be obtained, the operation of the current user is interfered according to the correct experiment process, meanwhile, interesting learning elements are added in the experiment process, interesting learning strategies are set, different interesting learning strategies are matched with students of different levels, the user is better assisted to efficiently learn, finally, the system gives learning evaluation feedback, the user can learn after thinking resistance and summarization conveniently, in the embodiment, the fluency and the operation steps of the experiment operation of the user are monitored, a weakness capturing feedback mechanism is set, and the weakness capturing and feedback relearning process is as follows:
(1) recording the knowledge of the suspicious weaknesses: monitoring the operation process of an experimenter in real time, and recording the points of the experimenter which are guided and warned;
(2) relearning the vulnerability knowledge: testing the question matched with the question with the highest conformity with the captured weak point knowledge in the question removing library, and if the guide and the warning are finished and the experimenter already masters the knowledge point, continuing to perform the next experiment; otherwise, the computer explains the knowledge point by voice or video, extracts the question from the question bank again after the completion of the explanation and tests the question until the answer is correct, which means that the experimenter has completely mastered the knowledge point, and after the experiment is finished, the computer returns a summary of the weak point knowledge point to help the students to more intuitively see the weak points and to review the weak points.
In order to adapt to students with different bases and enable a computer to effectively and efficiently cooperate with the students to perform correct and ordered experiments, in the interesting learning mode in the embodiment, by monitoring the operation time and the accuracy of experimenters and according to different experimental states of each experimenter, a personalized experiment mode is set, which comprises the following strategies,
(1) timely excitation: when an experimenter well completes a certain step, the system can issue corresponding rewards according to the completion time and the operation specification, wherein the rewards adopt a virtual gold coin form;
(2) intelligent navigation: the system monitors the operation process of an experimenter in real time, and when the fact that the experimenter stays in a certain step for too long time and cannot quickly perform the next step is detected, the system can conduct timely guidance; meanwhile, through intention understanding, the computer predicts that an experimenter is about to perform an error operation, and the system also warns;
(3) the barrier is established to intelligence: the hidden level is set, when the experimenter operates smoothly, when the whole experiment process reaches a certain condition, the corresponding hidden level can be unlocked, the setting of the hidden level is the expansion and the pull-out of the current basic experiment knowledge point, and the experimenter can also obtain extra gold coin rewards when learning more knowledge.
In the learning process, it is very important to provide timely feedback to learners, and for experimenters, positive feedback can show the effectiveness of the current learning behaviors of the learners, so that the control feeling of the learners on the application can be ensured, and meanwhile, the learners can have a more definite target in the learning process. Therefore, after the experiment is finished in the embodiment, a knowledge answering module is arranged, the computer and the experimenter perform questions at the same time, the computer is provided with reaction time, the experimenter and the machine perform answering in a rush mode, the correct answering can be added with points, the wrong answering can be deducted, if the answering is abandoned, the points are not added or deducted, and if the experimenter finally outputs the data to the computer, all previous gold coins are rewarded and cleared; on the contrary, if the experimenter defeats the computer, different titles are awarded according to the situation of the gold coins obtained by the experimenter in the whole experiment process, including 'bad feeling', 'good feeling', 'surplus' and 'pure green' of the fire.
In this implementation, an algorithm description is also given as follows:
the algorithm name is as follows: multimode fusion intention probability acquisition algorithm (VCSIG algorithm) based on static information and dynamic information fusion
Inputting: voice information (V), scene position information (C) captured by a camera, state transition probability (S), knowledge base information (D)
And (3) outputting: user's current operation intention (I), interesting learning strategy (G)
1.While(C):
2.Computer
Figure 429693DEST_PATH_IMAGE031
using the formula (1);
3.Computer
Figure 411555DEST_PATH_IMAGE032
using the formula (2);
4.
Figure 905116DEST_PATH_IMAGE033
Figure 630626DEST_PATH_IMAGE034
Figure 75700DEST_PATH_IMAGE036
/*Compare keywords to get intention probability*/
Figure 164879DEST_PATH_IMAGE037
Figure 707462DEST_PATH_IMAGE038
/* Each intention probability of voice channel is 0*/
5.Computer
Figure 73853DEST_PATH_IMAGE039
using the formula (3);
Figure 381206DEST_PATH_IMAGE040
Figure 364206DEST_PATH_IMAGE041
)/*The weight is calculated according to the information value content of each channel*/
6.Computer
Figure 447830DEST_PATH_IMAGE042
using the formula (4);
Figure 793361DEST_PATH_IMAGE043
/* The information of each channel is weighted and summed*/
7.
Figure 553507DEST_PATH_IMAGE044
was obtained through a large number of experimental statistics
8.ComputerP using the formula (5);
Figure 878495DEST_PATH_IMAGE046
/*Fusion of static information and dynamic information*/
9.
Figure 967280DEST_PATH_IMAGE047
/*Takethe intention corresponding to
Figure 163906DEST_PATH_IMAGE048
as the final result*/
10.
Figure 839607DEST_PATH_IMAGE049
11.Recording time t;
12.
Figure 373357DEST_PATH_IMAGE050
13.Record current operation process
Figure 834425DEST_PATH_IMAGE051
,Executegame strategy
Figure 687106DEST_PATH_IMAGE052
/*warning*/
14.
Figure 600835DEST_PATH_IMAGE053
15.Record current operation process
Figure 62909DEST_PATH_IMAGE054
,Executegame strategy
Figure 378484DEST_PATH_IMAGE055
/*guide*/
16.
Figure 510388DEST_PATH_IMAGE056
17.Unlock hidden level/*Hide levels to improve current knowledge*/
18.ReturnO
End
The main difficulty of multi-mode fusion intention understanding lies in that for the processing problem of multi-channel information, some channel information is incomplete, and the information of some channels has redundancy, which all affect the accuracy of intention understanding, and effective information needs to be correctly distinguished and utilized to obtain more accurate intention. Meanwhile, the design of many current virtual simulation experiment systems is too streamlined and not flexible enough, and the design of interesting learning modes is to enable users to be concentrated in the learning process and improve the learning efficiency of the users.
In the embodiment, the multi-modal fusion intention probability obtaining algorithm based on static information and dynamic information fusion has the advantages that (1) multi-channel dynamic information can be evaluated, and different weights are automatically given to each channel according to the value content of effective information; (2) acquiring static information according to the relationship between the behavior of a person and the motivation, and fusing the static information and the dynamic information by adopting a fusion strategy to improve the accuracy rate of intention prediction; (3) interesting learning elements are added, and targeted guiding warning, barrier setting and the like can be performed according to the experimental conditions of different students, so that the students can more easily reach the state of mind.
Finally, it should be further noted that the above examples and descriptions are not limited to the above embodiments, and technical features of the present invention that are not described may be implemented by or using the prior art, and are not described herein again; the above embodiments and drawings are only for illustrating the technical solutions of the present invention and not for limiting the present invention, and the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that changes, modifications, additions or substitutions within the spirit and scope of the present invention may be made by those skilled in the art without departing from the spirit of the present invention, and shall also fall within the scope of the claims of the present invention.

Claims (3)

1. An intelligent experiment method based on intention understanding is characterized in that multi-mode fusion intention understanding is firstly carried out, and then a weak point capturing feedback mechanism, a personalized dynamic experiment mode matching strategy and a man-machine competition answering strategy are adopted;
the multimodal fusion is intended to be understood to include the following steps:
(1) processing multi-channel data;
firstly, setting initial probability, and establishing a state transition probability model based on the operation behavior of an experimenter
Figure 492999DEST_PATH_IMAGE001
Matrix, transition probability from j state to i state is noted
Figure 364003DEST_PATH_IMAGE002
Establishing a behavior intention library, collecting information of a scene channel and a voice channel, processing the information and then carrying out quantitative matching;
for the calculation of the position information, the intention probability of the current position channel is obtained through a formula (1) according to the hand coordinates captured in real time and the coordinates of each apparatus in the scene,
Figure 954253DEST_PATH_IMAGE003
(1)
wherein,
Figure 150879DEST_PATH_IMAGE004
r is an operation intention probability surge region set according to a large number of experiments as the probability of the ith intention of the position lane, S1, S2, S3 represent distances,
Figure 328045DEST_PATH_IMAGE005
probability weight in the R region;
for the calculation of the motion direction, the moving track of the hand is captured in real time, the direction relation between the motion direction of the hand and each apparatus in the scene is calculated to obtain a probability value, the specific calculation method is as formula (2),
Figure 268319DEST_PATH_IMAGE006
(2)
wherein,
Figure 775392DEST_PATH_IMAGE007
the probability of the ith intention of a directional channel,
Figure 611761DEST_PATH_IMAGE008
the distance between the ith equipment and the hand movement track;
for voice information, firstly, a voice database is established according to intentions of different experiment intention sets
Figure 587808DEST_PATH_IMAGE009
Wherein, the voice database is in one-to-one correspondence with the intention set of the chemical experiment,
Figure 813996DEST_PATH_IMAGE010
which indicates the type of the experiment to be performed,
Figure 129571DEST_PATH_IMAGE011
shows the first experiment
Figure 120530DEST_PATH_IMAGE012
The voice corresponding to the individual intention is presented,
Figure 318293DEST_PATH_IMAGE013
representing the number of intentions of the experiment, sorting out keywords with different intentions in an intention set, acquiring voice text information by adopting a Baidu API (application programming interface), extracting the keywords, and calculating the matching degree of the keywords captured by a computing system and each intention keyword in an intention library to calculate the probability of each intention calculated according to the voice information in real time
Figure 85523DEST_PATH_IMAGE014
(2) Multi-channel information fusion;
firstly, the dynamic information of each channel is fused, the intention probability variance of each channel is calculated to obtain the weight of each channel, the concrete formula is as follows,
Figure 521183DEST_PATH_IMAGE015
(3)
wherein,
Figure 10939DEST_PATH_IMAGE016
representing three channels of position, direction and voice,
Figure 164840DEST_PATH_IMAGE017
for the weight of each channel it is desirable to,
Figure 732831DEST_PATH_IMAGE018
is as follows
Figure 554157DEST_PATH_IMAGE019
The probability of the ith intention of an individual channel,
Figure 168809DEST_PATH_IMAGE020
is as follows
Figure 528115DEST_PATH_IMAGE021
Average probability of n intents in each channel;
final per-intent dynamic information prediction probability
Figure 151994DEST_PATH_IMAGE022
Is composed of
Figure 312980DEST_PATH_IMAGE023
(4)
Combining the prediction probability obtained by the static information with the probability obtained by the dynamic information to obtain the intention probability after fusion, wherein the formula is as follows
Figure 98533DEST_PATH_IMAGE024
(5)
Wherein,
Figure 210714DEST_PATH_IMAGE025
as the probability of the predicted ith intention,
Figure 372705DEST_PATH_IMAGE026
from the jth intention to the ith intention in the intention libraryThe probability of a state transition of (a),
Figure 182005DEST_PATH_IMAGE027
probability of the ith intention obtained by fusing the dynamic information;
selecting
Figure 404039DEST_PATH_IMAGE028
The largest one corresponds to the predicted current intent.
2. The intelligent experiment method based on intention understanding, as claimed in claim 1, is characterized in that a vulnerability capturing and feedback mechanism is provided in the steps of monitoring fluency and operation of the user experiment, and the vulnerability capturing and feedback relearning process is as follows:
(1) recording the knowledge of the suspicious weaknesses: monitoring the operation process of an experimenter in real time, and recording the points of the experimenter which are guided and warned;
(2) relearning the vulnerability knowledge: testing the question matched with the question with the highest conformity with the captured weak point knowledge in the question removing library, and if the guide and the warning are finished and the experimenter already masters the knowledge point, continuing to perform the next experiment; otherwise, the computer explains the knowledge point by voice or video, extracts the question from the question bank again after the completion of the explanation and tests the question until the answer is correct, which means that the experimenter has completely mastered the knowledge point, and after the experiment is finished, the computer returns a summary of the weak point knowledge point to help the students to more intuitively see the weak points and to review the weak points.
3. The intelligent experiment method based on intention understanding, according to claim 1, is characterized in that, by monitoring the operation time and accuracy of experimenters, personalized experiment modes are set according to different experiment states of each experimenter, including the following strategies,
(1) timely excitation: when an experimenter well completes a certain step, the system can issue corresponding rewards according to the completion time and the operation specification, wherein the rewards adopt a virtual gold coin form;
(2) intelligent navigation: the system monitors the operation process of an experimenter in real time, and when the fact that the experimenter stays in a certain step for too long time and cannot quickly perform the next step is detected, the system can conduct timely guidance; meanwhile, through intention understanding, the computer predicts that an experimenter is about to perform an error operation, and the system also warns;
(3) the barrier is established to intelligence: the hidden level is set, when the experimenter operates smoothly, when the whole experiment process reaches a certain condition, the corresponding hidden level can be unlocked, the setting of the hidden level is the expansion and the pull-out of the current basic experiment knowledge point, and the experimenter can also obtain extra gold coin rewards when learning more knowledge.
CN202210796763.6A 2022-07-08 2022-07-08 Intelligent experiment method based on intention understanding Expired - Fee Related CN114881179B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210796763.6A CN114881179B (en) 2022-07-08 2022-07-08 Intelligent experiment method based on intention understanding

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210796763.6A CN114881179B (en) 2022-07-08 2022-07-08 Intelligent experiment method based on intention understanding

Publications (2)

Publication Number Publication Date
CN114881179A CN114881179A (en) 2022-08-09
CN114881179B true CN114881179B (en) 2022-09-06

Family

ID=82683653

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210796763.6A Expired - Fee Related CN114881179B (en) 2022-07-08 2022-07-08 Intelligent experiment method based on intention understanding

Country Status (1)

Country Link
CN (1) CN114881179B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110286763A (en) * 2019-06-21 2019-09-27 济南大学 A kind of navigation-type experiment interactive device with cognitive function
CN110554774A (en) * 2019-07-22 2019-12-10 济南大学 AR-oriented navigation type interactive normal form system
CN114582030A (en) * 2022-05-06 2022-06-03 湖北工业大学 Behavior recognition method based on service robot

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106997236B (en) * 2016-01-25 2018-07-13 亮风台(上海)信息科技有限公司 Based on the multi-modal method and apparatus for inputting and interacting

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110286763A (en) * 2019-06-21 2019-09-27 济南大学 A kind of navigation-type experiment interactive device with cognitive function
CN110554774A (en) * 2019-07-22 2019-12-10 济南大学 AR-oriented navigation type interactive normal form system
CN114582030A (en) * 2022-05-06 2022-06-03 湖北工业大学 Behavior recognition method based on service robot

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
多通道人机交互信息融合的智能方法;杨明浩等;《中国科学:信息科学》;20180420(第04期);79-94页 *

Also Published As

Publication number Publication date
CN114881179A (en) 2022-08-09

Similar Documents

Publication Publication Date Title
Jalal et al. Students’ behavior mining in e-learning environment using cognitive processes with information technologies
CN112069970B (en) Classroom teaching event analysis method and device
CN110069707A (en) Artificial intelligence self-adaptation interactive teaching system
CN111665941B (en) Virtual experiment-oriented multi-mode semantic fusion human-computer interaction system and method
CN114372137B (en) Depth knowledge tracking-fused dynamic perception test question recommendation method and system
Lin et al. Is it a good move? Mining effective tutoring strategies from human–human tutorial dialogues
Adnan et al. Earliest possible global and local interpretation of students’ performance in virtual learning environment by leveraging explainable AI
Vishnumolakala et al. In-class student emotion and engagement detection system (iSEEDS): an AI-based approach for responsive teaching
Byrne et al. Leveraging eye tracking in digital classrooms: A step towards multimodal model for learning assistance
CN116611969B (en) Intelligent learning and scoring system for traditional martial arts
CN114881179B (en) Intelligent experiment method based on intention understanding
Rozaliev et al. Methods and applications for controlling the correctness of physical exercises performance
Peng et al. Single shot state detection in simulation-based laparoscopy training
Chan et al. Grounding commands for autonomous vehicles via layer fusion with region-specific dynamic layer attention
Ou et al. Predicting visual focus of attention from intention in remote collaborative tasks
Hung et al. Building an online learning model through a dance recognition video based on deep learning
Sharma et al. Surya Namaskar: real-time advanced yoga pose recognition and correction for smart healthcare
CN114119932A (en) VR teaching method, apparatus, electronic device, storage medium and program product
CN113409635A (en) Interactive teaching method and system based on virtual reality scene
Huang et al. Pull together: Option-weighting-enhanced mixture-of-experts knowledge tracing
Barnes et al. Data-driven domain models for problem solving
Cao et al. Mamba4KT: An Efficient and Effective Mamba-based Knowledge Tracing Model
Li Improving Inspector Training and Understanding Through Human-Centric Intelligence Methods for Drone-Assisted Bridge Inspection
Kelly Analysis and Control of Temporal Biases in Surgical Skill Evaluation
US11990059B1 (en) Systems and methods for extended reality educational assessment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20220906

CF01 Termination of patent right due to non-payment of annual fee