Disclosure of Invention
The invention aims to provide a VR teaching system, so that a user can actively participate in a VR scene to achieve a better learning effect.
The VR teaching system provided by the embodiment of the invention comprises:
the teacher terminal is used for receiving courses recorded or compiled by the teacher;
the server is in communication connection with the teacher terminal and is used for receiving courses uploaded by the teacher terminal;
the student terminal is in communication connection with the server side and is used for receiving and playing courses of the server side and receiving interaction actions of students when the courses are played;
wherein, student terminal includes:
at least one VR handle for receiving student interactions while the lesson is playing.
Preferably, the teacher terminal includes:
the audio acquisition equipment is used for acquiring audio information of teachers;
the image acquisition equipment is used for acquiring image information of a teacher;
the first control device is electrically connected with the audio acquisition device and the image acquisition device; the first control device is used for acquiring the audio information and the image information acquired by the audio acquisition device and the image acquisition device, and receiving the editing operation of a teacher on the audio information and the image information so as to generate courses;
the first control device includes:
the first communication module is in communication connection with the server side;
the display module is used for displaying the audio information and the image information;
the human input module is used for receiving editing operation of a teacher;
the processing module is electrically connected with the first communication module, the display module, the human input module, the audio acquisition equipment and the image acquisition equipment.
Preferably, the image acquisition apparatus includes:
and the at least four cameras shoot images of the teacher when recording courses from four directions of the front, the back, the left and the right of the teacher respectively.
Preferably, the audio collection device includes:
the microphones are arranged around the teacher in an array mode when the teacher records courses.
Preferably, the student terminal includes:
the wearable VR equipment is electrically connected with the VR handle and used for playing courses sent by the server side;
the first movement sensing module is arranged in the wearable VR equipment and is used for sensing movement of students;
the server side performs the following operations:
constructing a virtual course space based on the image information acquired by the image acquisition equipment and the audio information acquired by the audio acquisition equipment;
storing the virtual course space and the course correspondingly;
when receiving the state of playing courses by the student terminals, calling a corresponding virtual course space;
mapping the students to an initial point of a virtual course space, and playing images in a sight range of the students in the virtual course space;
sensing movement of the student through the first movement sensing module;
based on the movement of the students, determining images in the sight range of the students in real time and playing the images;
based on the movement and initial point of the student, determining the position of the student in the virtual course space, and playing the audio information acquired by the audio acquisition equipment corresponding to the position.
Preferably, based on the movement of the student, determining and playing the image in the sight line of the student in real time includes:
acquiring a central shooting vector of each camera constructing a virtual space, wherein the central shooting vector is a shooting direction which is a central point of a camera lens and perpendicular to the camera lens facing the camera;
obtaining a central vision vector of the student, wherein the central vision vector takes the pupil of the student as a starting point to outwards;
determining a position vector based on the student position and the teacher's position in the virtual lesson space;
when the included angle between the position vector and the central view vector is smaller than a preset first threshold, calculating the included angle between the central view vector and the central camera vector of each camera, wherein the calculation formula is as follows:
wherein θ i An included angle between the central view vector and the ith central shooting vector; x is x j A j-th dimension parameter value that is a center view vector; y is ij A j-th dimensional parameter value for the i-th center imaging vector; n is the data dimension;
comparing the angles, and selecting the image shot by the camera with the smallest angle as the image in the sight range of the student and playing the image;
based on the movement and initial point of the student, determining the position of the student in the virtual course space, playing the audio information collected by the audio collection device corresponding to the position, including:
the distance between the position of the student and the set position of each audio acquisition device is calculated by the following formula:
wherein L is k Distance between student position and k audio acquisition device setting position; a is that 1k 、A 2k 、A 3k Coordinate values of an X axis, a Y axis and a Z axis of the setting position of the kth audio acquisition device are respectively set; b (B) 1 、B 1 、B 1 Coordinate values of an X axis, a Y axis and a Z axis of the position of the student respectively;
comparing the distance between the position of the student and the set position of each audio acquisition device, and taking the audio information of the audio acquisition device with the shortest distance as the base audio;
determining a second distance between the position of the student and the position of the teacher in the virtual course space, and a third distance between the audio acquisition equipment with the shortest distance and the position of the teacher in the virtual course space;
inquiring a preset association coefficient table based on the first distance, the second distance and the third distance, and determining a relationship coefficient between the final playing audio and the basic audio;
based on the relationship coefficients and the base audio, a final playback audio is determined.
Preferably, the server side performs the following operations:
when receiving a first action of the students, representing the recorded notes, through the VR handle, acquiring a play point of a current played course, which is advanced by a preset time, as a recording starting point;
when receiving a second action of the student, which indicates that the note is finished, through the VR handle, acquiring a second position of the currently played course as a recording termination point;
the audio data between the recording starting point and the recording ending point are converted into text data, and the text data are output to students when the course is finished;
receiving the editing operation of students on the text data to form course notes; a step of
Wherein the first action comprises: pressing a key for pressing the VR handle; the second action includes: releasing the pressed key.
Preferably, the VR handle comprises:
the shell is cylindrical, and the periphery of the shell is provided with anti-skid grains;
a key set, which is arranged on one end face of the shell;
a pressure sensor array provided on the outer periphery of the housing;
the second movement sensing module is arranged in the shell and is used for sensing movement of the VR handle;
the processor is arranged in the shell and is electrically connected with the key, the second mobile sensing module and the pressure sensor array respectively;
the second communication module is electrically connected with the processor and the server side;
the processor detects the grip strength of the student through the pressure sensor array; when the grip strength is larger than a preset first pressure threshold value, the corresponding virtual image hand is gripped to grasp an object on the virtual image;
the processor senses the movement of the VR handle through the second sensing module so as to synchronously change the virtual images corresponding to the students to obtain the hand movement.
The invention also provides a multimedia classroom, which is applied to any VR teaching system, comprising:
and the plurality of VR handle connection interfaces are respectively arranged on each student seat and used for connecting VR handles.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
An embodiment of the present invention provides a VR teaching system, as shown in fig. 1, including:
a teacher terminal 1, configured to receive a course recorded or compiled by a teacher;
the server end 2 is in communication connection with the teacher terminal 1 and is used for receiving courses uploaded by the teacher terminal 1;
the student terminal 3 is in communication connection with the server side 2 and is used for receiving and playing courses of the server side 2 and receiving interaction actions of students when the courses are played;
wherein, student terminal 3 includes:
at least one VR handle 31 for receiving student interactions as the lesson is being played.
The working principle and the beneficial effects of the technical scheme are as follows:
the teacher terminal 1 is used by a teacher, and the teacher can use the teacher terminal 1 to compile or record courses; recording courses, namely recording the course of a teacher by adopting a video recording function; the course is complicated to build, and the interaction possibility of students needs to be considered when the course is played. For example: when the concentrated sulfuric acid dilution experiment is compiled, the correct step is to pour the concentrated sulfuric acid into water along the cup wall; there may be a variety of situations in the operation of students: 1. directly and instantaneously pouring water into concentrated sulfuric acid; 3. pouring concentrated sulfuric acid into water along the wall of the cup; when the video or phenomenon is compiled, various interactive operations of students need to be compiled respectively.
The student terminal 3 is used by students, and mainly the students log in the server side 2 for use; when the student uses the student terminal 3 to watch the course sent by the server side 2, interaction actions are realized through the VR handle 31, for example: video pause, playback, fast forward, etc.; in addition, the experiment operation can be performed during the playing of the experiment. For example, in the dilution experiment of the concentrated sulfuric acid, firstly, the concentrated sulfuric acid is directly poured into water instantly, and secondly, the water is poured into the concentrated sulfuric acid; 3. pouring concentrated sulfuric acid into water along the wall of the cup, and the like.
The VR teaching system provided by the invention enables a user to actively participate in a VR scene so as to achieve a better learning effect, and during dangerous experiments, accidents caused by errors of operation steps are avoided, and students can clearly and intuitively feel the results caused by the operation errors.
In one embodiment, as shown in fig. 2, the teacher terminal 1 includes:
an audio collection device 11 for collecting audio information of a teacher;
an image pickup device 12 for picking up image information of a teacher;
a first control device 13 electrically connected to the audio collection device 11 and the image collection device 12; the first control device 13 is configured to acquire audio information and image information acquired by the audio acquisition device 11 and the image acquisition device 12, and receive editing operations of teachers on the audio information and the image information to generate a lesson;
the first control device 13 includes:
the first communication module 132 is in communication connection with the server side 2;
a display module 133 for displaying audio information and image information;
a human input module 134 for receiving an edit operation of a teacher;
the processing module 131 is electrically connected with the first communication module 132, the display module 133, the human input module 134, the audio acquisition device 11 and the image acquisition device 12.
The working principle and the beneficial effects of the technical scheme are as follows:
the teacher terminal 1 respectively acquires video information and audio information of a course recorded by a teacher through the image acquisition device 12 and the audio acquisition device 11; and the course recording is realized. Editing the recorded courses; editing operations include editing, setting triggering interaction actions and the like; wherein the display module 133 is a display; the human input module 134 includes: a keyboard and a mouse; the first communication module 132 includes a network card; the processing module 131 is a computer host.
In one embodiment, image capture device 12 includes:
and the at least four cameras shoot images of the teacher when recording courses from four directions of the front, the back, the left and the right of the teacher respectively.
The working principle and the beneficial effects of the technical scheme are as follows:
panoramic recording is realized through omnibearing recording; when the student terminal 3 plays, the playing angle can be switched according to the interaction of the student, so that the student can feel as if the student is on the scene, and the learning experience of the student is improved.
In one embodiment, the audio acquisition device 11 comprises:
the microphones are arranged around the teacher in an array mode when the teacher records courses.
The working principle and the beneficial effects of the technical scheme are as follows:
through the microphone that the array set up, can realize all-round sound capture, realize synchronous switching sound's play when the student switches the broadcast angle. For example, four microphones can be adopted to correspond to the cameras one by one; and synchronously switching to play sound collected by the corresponding microphone when switching to the picture recorded by the corresponding camera.
In one embodiment, the student terminal 3 includes:
the wearable VR device is electrically connected with the VR handle 31 and used for playing courses sent by the server side 2;
the first movement sensing module is arranged in the wearable VR equipment and is used for sensing movement of students;
the server side 2 performs the following operations:
constructing a virtual lesson space based on the image information acquired by the image acquisition device 12 and the audio information acquired by the audio acquisition device 11;
storing the virtual course space and the course correspondingly;
when receiving the state of playing courses by the student terminals 3, calling a corresponding virtual course space;
mapping the students to an initial point of a virtual course space, and playing images in a sight range of the students in the virtual course space;
sensing movement of the student through the first movement sensing module;
based on the movement of the students, determining images in the sight range of the students in real time and playing the images;
based on the movement and initial point of the student, the position of the student in the virtual course space is determined, and the audio information collected by the audio collection device 11 corresponding to the position is played.
The working principle and the beneficial effects of the technical scheme are as follows:
based on the wearable VR equipment and the first mobile perception module, mapping between students and a virtual course space is achieved, images are displayed according to the visual angles of the students, corresponding visual angles and sounds can be switched according to movement of the students, and immersive learning experience is achieved.
In one embodiment, determining and playing images in the student's line of sight in real time based on the movement of the student includes:
acquiring a central shooting vector of each camera constructing a virtual space, wherein the central shooting vector is a shooting direction which is a central point of a camera lens and perpendicular to the camera lens facing the camera;
obtaining a central vision vector of the student, wherein the central vision vector takes the pupil of the student as a starting point to outwards;
determining a position vector based on the student position and the teacher's position in the virtual lesson space;
when the included angle between the position vector and the central view vector is smaller than a preset first threshold, calculating the included angle between the central view vector and the central camera vector of each camera, wherein the calculation formula is as follows:
wherein θ i An included angle between the central view vector and the ith central shooting vector; x is x j A j-th dimension parameter value that is a center view vector; y is ij A j-th dimensional parameter value for the i-th center imaging vector; n is the data dimension;
comparing the angles, and selecting the image shot by the camera with the smallest angle as the image in the sight range of the student and playing the image;
based on the movement and initial point of the student, determining the position of the student in the virtual course space, playing the audio information collected by the audio collection device 11 corresponding to the position, including:
the distance between the position of the student and the set position of each audio collection device 11 is calculated as follows:
wherein L is k Distance between the position of the student and the set position of the kth audio acquisition device 11; a is that 1k 、A 2k 、A 3k Coordinate values of an X axis, a Y axis and a Z axis of the setting position of the kth audio acquisition device 11, respectively; b (B) 1 、B 1 、B 1 Coordinate values of an X axis, a Y axis and a Z axis of the position of the student respectively;
comparing the distance between the position of the student and the setting position of each audio acquisition device 11, and taking the audio information of the audio acquisition device 11 with the shortest distance as the base audio;
determining a second distance between the position of the student and the position of the teacher in the virtual course space and a third distance between the audio acquisition equipment 11 with the shortest distance and the position of the teacher in the virtual course space;
inquiring a preset association coefficient table based on the first distance, the second distance and the third distance, and determining a relationship coefficient between the final playing audio and the basic audio;
based on the relationship coefficients and the base audio, a final playback audio is determined.
The working principle and the beneficial effects of the technical scheme are as follows:
the central view vector is used as a characterization parameter of the sight line, and whether the student views the lecture of the teacher can be determined through the included angle between the central view vector and the position vector; the first threshold may be set to 60 degrees only when the included angle thereof is smaller than the first threshold; when the included angle is larger than the first threshold, the background of the preset virtual course space can be played, and the direction of sight line adjustment is indicated by an arrow, so that students can conveniently and quickly adjust the sight line. When students speak class for watching teachers, the best display image is determined through the included angle between the central view vector and the central shooting vector so as to be displayed to the students. And the final playing audio is determined through the position of the student, so that accurate audio determination is realized. And then realized that audio and video changes according to student's removal, improved student's immersion, improved student's VR experience.
In one embodiment, the server side 2 performs the following operations:
when receiving a first action of the student representing a note taking through the VR handle 31, acquiring a play point of a currently played course, which advances forward by a preset time, as a record start point;
when receiving a second action of the student indicating the end of the note through the VR handle 31, acquiring a second position of the currently played course as a recording termination point;
the audio data between the recording starting point and the recording ending point are converted into text data, and the text data are output to students when the course is finished;
receiving the editing operation of students on the text data to form course notes; a step of
Wherein the first action comprises: pressing a key 313 holding the VR handle 31; the second action includes: the pressed key 313 is released.
The working principle and the beneficial effects of the technical scheme are as follows:
by pressing the key 313, the note recording function of the teaching system is started, the problem that a user cannot take notes when watching VR courses is solved, and the accuracy and efficiency of notes are improved by directly carrying out text conversion on audio data. Pushing the record starting point forward from the trigger point of the first action, and considering a response time of the student and a corresponding time of a command of the system, so as to ensure the integrity of note record; finally, the text data is edited according to the demands of students, and the editing includes deleting redundant data, so that the note is simplified. Further, the preset time is associated with the force path pressed by the key 313, that is, the larger the pressing force path is, the longer the preset time is, a corresponding table may be set in advance, and the recording start point is determined according to the preset time extracted from the pressing force path.
In one embodiment, as shown in fig. 3, VR handle 31 includes:
the shell is cylindrical, and the periphery of the shell is provided with anti-skid grains;
a key 313 provided on one end surface of the housing;
a pressure sensor array 314 disposed on the outer periphery of the housing;
a second movement sensing module 315 disposed within the housing for sensing movement of the VR handle 31;
the processor 311 is arranged in the shell and is electrically connected with the key 313, the second mobile sensing module 315 and the pressure sensor array 314 respectively;
the second communication module 312 is electrically connected with the processor 311 and the server side 2;
the processor 311 detects the grip of the student through the pressure sensor array 314; when the grip strength is larger than a preset first pressure threshold value, the corresponding virtual image hand is gripped to grasp an object on the virtual image;
the processor 311 senses the movement of the VR handle 31 through the second sensing module to synchronously change the hand movement of the avatar corresponding to the student.
The working principle and the beneficial effects of the technical scheme are as follows:
based on the second motion sensing module 315, the pressure sensor array 314, and the keys 313; input of interaction is realized; synchronizing the avatar with the student's operation of VR handle 31 facilitates the parameter manipulation during the student's immersive experience. Wherein the pressure sensor array 314 is a plurality of pressure sensors arranged in an array; the second movement awareness module 315 includes: three-axis gyroscopes and three-axis accelerometers.
The invention also provides a multimedia classroom, which is applied to any VR teaching system, comprising:
and a plurality of VR handle connection interfaces are respectively arranged on each student seat and used for connecting the VR handles 31.
The working principle and the beneficial effects of the technical scheme are as follows:
the VR handle connection interface is used for students to connect to enter the VR teaching system; and VR course learning is performed by multiple persons at the same time.
One specific application scenario: a large screen is arranged at the platform position of the teacher to display the course of the teacher; when the course enters the interactive operation end, the operation interfaces are respectively projected onto the small screens on the seats of the students, and the students perform interactive operation through the VR handle 31.
Yet another specific application scenario: the teacher carries out course explanation at the podium position, and when the course gets into interactive operation end, throw the operation interface respectively on the little screen on the student's seat, the student carries out the interactive operation through VR handle 31, and the interactive operation is accomplished the back and is summarized the teacher terminal 1 of operation process result to the teacher position.
Furthermore, holographic projection equipment is arranged at the teacher position to realize holographic projection of courses.
In addition, the multimedia teacher also provides access to wearable VR devices.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.