CN109474648B - Virtual reality interaction compensation method and server-side equipment - Google Patents
Virtual reality interaction compensation method and server-side equipment Download PDFInfo
- Publication number
- CN109474648B CN109474648B CN201710801198.7A CN201710801198A CN109474648B CN 109474648 B CN109474648 B CN 109474648B CN 201710801198 A CN201710801198 A CN 201710801198A CN 109474648 B CN109474648 B CN 109474648B
- Authority
- CN
- China
- Prior art keywords
- virtual reality
- user
- information
- picture
- displayed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/131—Protocols for games, networked simulations or virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Information Transfer Between Computers (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention provides a compensation method for virtual reality interaction and server-side equipment, wherein the method comprises the following steps: acquiring historical display picture information of first virtual reality equipment with an interaction failure event, head movement history information of a user of the first virtual reality equipment and limb movement history information of the user of the first virtual reality equipment; determining a picture to be displayed of the first virtual reality device according to the historical display picture information, the user head movement historical information and the user limb movement historical information; and transmitting the determined picture to be displayed to the first virtual reality equipment. The embodiment of the invention ensures the visual continuity of the virtual reality interaction service; meanwhile, the accuracy of picture prediction can be improved based on historical display picture information, user head movement historical information and user limb movement historical information, and user experience is further optimized.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a compensation method for virtual reality interaction and a server device.
Background
Interactive information in an interactive VR (Virtual Reality) based on a wired manner is transmitted through a physical line, and a server determines content to be displayed in a next screen according to the interactive information. The information interaction in the wireless-based interactive VR can only be transmitted through a wireless network, and the reliability and timeliness of the transmission are inferior to those of the wired method. And because the interactive information is crucial for the interactive VR, once the transmission of the interactive information fails (is lost or lags seriously), the preparation and subsequent transmission of the next frame will be seriously affected, and the VR experience of the user is poor.
Disclosure of Invention
The invention aims to provide a compensation method for virtual reality interaction and server equipment, so as to solve the problem that the next frame of picture cannot be normally transmitted due to the failure of the transmission of the interaction information of the virtual reality in the prior art.
In order to solve the above problem, an embodiment of the present invention provides a compensation method for virtual reality interaction, which is applied to a server and includes:
acquiring historical display picture information of first virtual reality equipment with an interaction failure event, head movement history information of a user of the first virtual reality equipment and limb movement history information of the user of the first virtual reality equipment; the method comprises the steps that a server side does not receive interaction information reported by first virtual reality equipment interacting with the server side within a preset time period, and then the first virtual reality equipment is determined to have an interaction failure event;
determining a picture to be displayed of the first virtual reality device according to the historical display picture information, the user head movement historical information and the user limb movement historical information;
and transmitting the determined picture to be displayed to the first virtual reality equipment.
Preferably, the step of determining the to-be-displayed picture of the first virtual reality device according to the history display picture information, the user head movement history information, and the user limb movement history information includes:
predicting a panoramic picture to be displayed of the first virtual reality device according to the historical display picture information;
predicting the current position and the current orientation angle of the head of the user of the first virtual reality device according to the historical motion information of the head of the user;
determining the trend of the user limb motion trail of the first virtual reality equipment according to the user limb motion history information, and predicting the response result of the user limb to preset operation according to the trend of the user limb motion trail;
and determining the to-be-displayed picture of the first virtual reality device according to the to-be-displayed panoramic picture, the current position of the head of the user, the current orientation angle of the head of the user and the response result of the limbs of the user to preset operation.
Preferably, after determining the to-be-displayed frame of the virtual reality device, the method further includes:
acquiring sound information corresponding to the picture to be displayed;
coding the sound information and transmitting the sound information to the virtual reality equipment; and the sound information is matched with the picture to be displayed.
Preferably, if N virtual reality devices interact with the server side simultaneously in a preset application, N is an integer greater than or equal to 2; the N virtual reality devices include: at least one first virtual reality device;
the step of transmitting the determined picture to be displayed to the first virtual reality device includes:
and distributing the determined picture to be displayed to each virtual reality device in the preset application.
Preferably, the method further comprises:
if the number of times of the interaction failure event of the first virtual reality device in the preset application is equal to a preset number of times or the time of the interaction failure event of the first virtual reality device exceeds a preset threshold, generating virtual operation and substitution of the first virtual reality device according to historical display picture information of the N virtual reality devices and reported interaction information of the second virtual reality device which does not have the interaction failure event;
pushing the substitute of the first virtual reality device into the preset application, and executing the virtual operation in the preset application by the substitute of the first virtual reality device.
Preferably, after the step of pushing the substitute of the first virtual reality device into the preset application and executing the virtual operation in the preset application, the method further comprises:
and if the first virtual reality equipment and the server side resume interaction at a preset moment, sending historical information of the preset application between the moment when the interaction failure event occurs and the preset moment to the first virtual reality equipment, and stopping pushing the substitute of the first virtual reality equipment to the preset application.
The embodiment of the invention also provides a server device, which is applied to the server and comprises a processor and a transceiver, wherein the processor is used for executing the following processes:
acquiring historical display picture information of first virtual reality equipment with an interaction failure event, head movement history information of a user of the first virtual reality equipment and limb movement history information of the user of the first virtual reality equipment; the method comprises the steps that a server side does not receive interaction information reported by first virtual reality equipment interacting with the server side within a preset time period, and then the first virtual reality equipment is determined to have an interaction failure event;
determining a picture to be displayed of the first virtual reality device according to the historical display picture information, the user head movement historical information and the user limb movement historical information;
and transmitting the determined picture to be displayed to the first virtual reality equipment.
Preferably, the processor is further configured to perform the following process:
predicting a panoramic picture to be displayed of the first virtual reality device according to the historical display picture information;
predicting the current position and the current orientation angle of the head of the user of the first virtual reality device according to the historical motion information of the head of the user;
determining the trend of the user limb motion trail of the first virtual reality equipment according to the user limb motion history information, and predicting the response result of the user limb to preset operation according to the trend of the user limb motion trail;
and determining the to-be-displayed picture of the first virtual reality device according to the to-be-displayed panoramic picture, the current position of the head of the user, the current orientation angle of the head of the user and the response result of the limbs of the user to preset operation.
Preferably, the processor is further configured to perform the following process:
acquiring sound information corresponding to the picture to be displayed;
coding the sound information and transmitting the sound information to the virtual reality equipment; and the sound information is matched with the picture to be displayed.
Preferably, if N virtual reality devices interact with the server side simultaneously in a preset application, N is an integer greater than or equal to 2; the N virtual reality devices include: at least one first virtual reality device;
the processor is further configured to perform the following process:
and distributing the determined picture to be displayed to each virtual reality device in the preset application.
Preferably, the processor is further configured to perform the following process:
if the number of times of the interaction failure event of the first virtual reality device in the preset application is equal to a preset number of times or the time of the interaction failure event of the first virtual reality device exceeds a preset threshold, generating virtual operation and substitution of the first virtual reality device according to historical display picture information of the N virtual reality devices and reported interaction information of the second virtual reality device which does not have the interaction failure event;
pushing the substitute of the first virtual reality device into the preset application, and executing the virtual operation in the preset application by the substitute of the first virtual reality device.
Preferably, the processor is further configured to perform the following process:
and if the first virtual reality equipment and the server side resume interaction at a preset moment, sending historical information of the preset application between the moment when the interaction failure event occurs and the preset moment to the first virtual reality equipment, and stopping pushing the substitute of the first virtual reality equipment to the preset application.
The embodiment of the present invention further provides a server device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the program, the compensation method for virtual reality interaction as described above is implemented.
Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps in the compensation method for virtual reality interaction as described above.
The technical scheme of the invention at least has the following beneficial effects:
in the technical scheme of the embodiment of the invention, when an interaction failure event occurs between the first virtual reality device and the server, the server predicts the picture to be displayed of the next frame of the first virtual reality device by comprehensively analyzing the historical display picture information of the first virtual reality device, the historical movement information of the head of the user and the historical movement information of the limbs of the user, and transmits the predicted picture to be displayed to the first virtual reality device, so that the visual continuity of the virtual reality interaction service is ensured; meanwhile, the accuracy of picture prediction can be improved based on historical display picture information, user head movement historical information and user limb movement historical information, and user experience is further optimized.
Drawings
Fig. 1 is a flowchart illustrating steps of a virtual reality interaction compensation method according to an embodiment of the present invention;
FIG. 2 is an interaction diagram illustrating a compensation method for virtual reality interaction according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a server device according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments.
As shown in fig. 1, an embodiment of the present invention provides a compensation method for virtual reality interaction, which is applied to a server and includes:
and step 13, transmitting the determined picture to be displayed to the first virtual reality device.
In the embodiment of the invention, for the interactive virtual reality VR service, when the interactive information between the VR device and the server is lost or does not arrive within the specified time, the server performs prediction of a next display picture by comprehensively analyzing the history display picture information of the VR device in which the interaction fails, the user head movement history information, and the user limb movement history information, so as to determine a next frame of picture to be displayed of the first virtual reality device.
Preferably, the compensation method provided by the above embodiment of the present invention can continue to prepare the visual display content to be transmitted in the next frame within a specified time according to the history information, and complete content distribution within a specified time, thereby ensuring the visual continuity of the virtual reality interactive service. Further, in order to ensure the visual continuity of the user of the first virtual reality device, the server needs to complete the prediction, encoding and transmission of the picture to be displayed before the display time of the next frame of picture arrives.
It should be noted that, in the foregoing embodiment of the present invention, when determining whether the interaction failure event occurs in the virtual reality device, the preset time period may be set to be less than or equal to the interval between adjacent frames.
Further, in the foregoing embodiment of the present invention, step 12 includes:
step 121, predicting a panoramic picture to be displayed of the first virtual reality device according to the historical display picture information; the history display screen information may also be referred to as content perception of the user of the first virtual display device. The history display screen information mainly includes the relevance of the front and back contents (for example, a 360-degree screen scene which needs to be displayed next is predicted), and the switching of the core content (such as event trigger, switching speed and the like).
Step 122, predicting the current position and the current orientation angle of the head of the user of the first virtual reality device according to the historical motion information of the head of the user; the historical information of the head movement of the user mainly determines the current position of the head of the user and the current orientation angle of the head of the user by analyzing the track of the head movement and the speed of the head movement. The current perspective of the user can be determined from its orientation angle, which is typically in the range of about 110 degrees to 120 degrees.
Step 123, determining the trend of the user limb motion trail of the first virtual reality device according to the user limb motion history information, and predicting the response result of the user limb to preset operation according to the trend of the user limb motion trail; the compensation method provided by the embodiment of the invention is applied to a scene that the VR equipment and the server have interaction, and in order to more accurately predict the picture to be displayed, the embodiment of the invention also collects the limb movement history information of the user, so that the response result of the limb of the user to the preset operation of the server is predicted. For example, the preset operation sent by the server to the user in the previous frame is shooting the abdomen of the user, and since the interactive information of the previous frame is not transmitted to the server within the specified time, the server cannot know whether the user is shooting the abdomen or the user successfully avoids the shooting operation, and at this time, the possibility of successfully avoiding the user and the possibility of shooting the user are estimated according to the limb movement history information of the user, so that whether the user is shooting or successfully avoiding the shooting operation is predicted.
And step 124, determining the to-be-displayed picture of the first virtual reality device according to the to-be-displayed panoramic picture, the current position of the head of the user, the current orientation angle of the head of the user and the response result of the limbs of the user to preset operation.
In the embodiment of the invention, when the interactive information does not arrive on time, the picture prediction at the next moment (or called as the next frame) is carried out, and the historical display picture information, the user head movement historical information and the user limb movement historical information are analyzed and combined to carry out the picture prediction, so that the prediction accuracy can be greatly improved. According to the embodiment of the invention, not only the historical display picture information of the first virtual reality device and the user head movement historical information are considered, but also the response result of the preset operation with the server (also called as the interaction result with the server) is preset according to the user limb movement historical information, so that the accuracy of prediction of the next frame picture in the interactive VR service is further improved, and the user experience is improved.
It should be noted that, in the above embodiment of the present invention, the server may perform big data training and establish a prediction model through the history display picture information, the user head movement history information, and the user limb movement history information, so that when the interactive information does not arrive, the prediction model is directly used to perform picture prediction, thereby further improving the prediction efficiency; meanwhile, the preset model can be further corrected through all newly arrived interaction information, so that the prediction model is further improved, and the prediction accuracy is improved.
As shown in fig. 2, after the server finishes the transmission of the previous frame of data, and exceeds a predetermined time, if the server does not receive the interactive information of the previous frame fed back by the first VR device, the server performs comprehensive prediction and coding of the picture display of the current frame based on the analysis of the history display picture information, the history information of the head movement of the user, and the history information of the body movement of the user, and after the coding is finished, the server performs the transmission of the picture data of the current frame.
Further, after step 12 in the above embodiment of the present invention, the method further includes:
acquiring sound information corresponding to the picture to be displayed;
coding the sound information and transmitting the sound information to the virtual reality equipment; and the sound information is matched with the picture to be displayed.
In the embodiment of the invention, when the server predicts the picture to be displayed, the server also needs to determine the sound information corresponding to the picture to be displayed, so that the accuracy of the hearing of the user is ensured. Since sound travels much slower than light, sound has a relatively low latency requirement than pictures, and slight dyssynchrony does not affect human resolution. Meanwhile, the accuracy of the voice for interaction (especially the direct participation of people) is rather higher. And the voice information needs to be processed and transmitted after the next frame of picture to be displayed is determined, and then the coding transmission of the voice information according to the preset instruction also needs to be sequentially delayed.
It should be noted that the compensation method provided by the embodiment of the present invention can be used in an interaction scenario between a single VR device and a server, and is also applicable to a scenario in which multiple VR devices interact with the server at the same time, such as a multi-player real-time online VR game scenario. Specifically, N virtual reality devices interact with the server side simultaneously in a preset application, where N is an integer greater than or equal to 2; the N virtual reality devices include: at least one first virtual reality device; namely, an interaction failure event occurs between at least one VR device of the N VR devices and the server.
The corresponding step 13 comprises:
and 131, distributing the determined picture to be displayed to each virtual reality device in the preset application.
Although not all the virtual reality devices have interaction failure events, since the N virtual reality devices interact with the server in the same scene, the interaction failure of one virtual reality device also affects the visual experience of other VR devices, and therefore, the picture to be displayed is encoded and then distributed to each virtual reality device in the preset application. Furthermore, the content distributed to each virtual reality device may indicate which VR device has an interaction failure event, for example, VR device A, VR device B and VR device C interact with the server in a preset application, and when an interaction failure event occurs between VR device a and the server, the server predicts and encodes a to-be-displayed picture of VR device a, and then distributes encoded data to VR device A, VR device B and VR device C; and when the data is distributed, the distributed data is indicated to be data of a picture to be displayed in the next frame of the VR device A predicted by the server.
Further, in the above embodiment of the present invention, the method further includes:
if the number of times of the interaction failure event of the first virtual reality device in the preset application is equal to a preset number of times or the time of the interaction failure event of the first virtual reality device exceeds a preset threshold, generating virtual operation and substitution of the first virtual reality device according to historical display picture information of the N virtual reality devices and reported interaction information of the second virtual reality device which does not have the interaction failure event;
pushing the substitute of the first virtual reality device into the preset application, and executing the virtual operation in the preset application by the substitute of the first virtual reality device.
Specifically, once a certain VR device or multiple VR devices in a preset application have failed in interaction for a long time or the number of times of failure in interaction exceeds a preset number of times, the server needs to generate virtual operation and substitution of the first virtual reality device according to historical display picture information of the N virtual reality devices and interaction information reported by a second virtual reality device where no interaction failure event occurs; and pushing the substitute of the first virtual reality device to the preset application, wherein the substitute of the first virtual reality device executes the virtual operation in the preset application, so that the application experience of a second virtual reality device without an interaction failure event is maintained.
For example, the VR device A, VR device B and the VR device C interact with the server in a preset application at the same time, when an interaction failure event occurs between the VR device a and the server, and the number of times of the interaction failure event exceeds a preset number of times or the duration of the interaction failure exceeds a preset time, the server generates a virtual operation and a substitute (the substitute is VR device a ') of the VR device a according to respective history display screen information of the VR device A, VR device B and the VR device C and interaction information reported by the VR device B and the VR device C, and the VR device a' executes the virtual operation in the preset application, thereby maintaining the application experience of the VR device B and the VR device C.
Further, in the above embodiment of the present invention, after the pushing the substitute of the first virtual reality device into the preset application and the substitute of the first virtual reality device performs the virtual operation in the preset application, the method further includes:
and if the first virtual reality equipment and the server side resume interaction at a preset moment, sending historical information of the preset application between the moment when the interaction failure event occurs and the preset moment to the first virtual reality equipment, and stopping pushing the substitute of the first virtual reality equipment to the preset application.
For VR equipment with interaction failure times exceeding preset times or interaction failure duration time exceeding preset time, if the VR equipment is recovered in the execution process of the preset application (namely the preset time is any time in the execution process of the preset application), historical information of the preset application between the moment of the interaction failure event and the preset time is sent to the first virtual reality equipment by the server side, so that the first virtual reality equipment is merged into the preset application as soon as possible, and after the interaction failure event is merged into the preset application, the replacement of the first virtual reality equipment is stopped being pushed to the preset application.
In summary, the compensation method provided by the above embodiment of the present invention is not only applicable to the interaction between a single user and a server, but also applicable to the interaction between multiple users and the server; specifically, no matter single-user interaction with a server or multi-user interaction with the server, when one or more users and the server have interaction failure events, the server predicts a to-be-displayed picture and encodes and issues the to-be-displayed picture based on historical display picture information, user head movement historical information and user limb movement historical information, so that the visual fluency of VR (virtual reality) services is guaranteed; meanwhile, the accuracy of VR service hearing is ensured according to the transmission characteristics of sound; furthermore, if multiple users interact with the server, predicted pictures to be displayed need to be distributed to each user in the application, and substitution and virtual operation are generated when interaction failure exceeds a preset time or interaction failure times exceed preset times, so that application experience of other users without interaction failure is maintained, and visual fluency of the VR service is further guaranteed.
As shown in fig. 3, an embodiment of the present invention further provides a server device, which is applied to a server, and includes a processor 300 and a transceiver 310, where the processor 300 is configured to perform the following processes:
acquiring historical display picture information of first virtual reality equipment with an interaction failure event, head movement history information of a user of the first virtual reality equipment and limb movement history information of the user of the first virtual reality equipment; the method comprises the steps that a server side does not receive interaction information reported by first virtual reality equipment interacting with the server side within a preset time period, and then the first virtual reality equipment is determined to have an interaction failure event;
determining a picture to be displayed of the first virtual reality device according to the historical display picture information, the user head movement historical information and the user limb movement historical information;
and transmitting the determined picture to be displayed to the first virtual reality equipment.
Preferably, the processor 300 in the above embodiment of the present invention is further configured to perform the following processes:
predicting a panoramic picture to be displayed of the first virtual reality device according to the historical display picture information;
predicting the current position and the current orientation angle of the head of the user of the first virtual reality device according to the historical motion information of the head of the user;
determining the trend of the user limb motion trail of the first virtual reality equipment according to the user limb motion history information, and predicting the response result of the user limb to preset operation according to the trend of the user limb motion trail;
and determining the to-be-displayed picture of the first virtual reality device according to the to-be-displayed panoramic picture, the current position of the head of the user, the current orientation angle of the head of the user and the response result of the limbs of the user to preset operation.
Preferably, the processor 300 in the above embodiment of the present invention is further configured to perform the following processes:
acquiring sound information corresponding to the picture to be displayed;
coding the sound information and transmitting the sound information to the virtual reality equipment; and the sound information is matched with the picture to be displayed.
Preferably, in the embodiment of the present invention, if N virtual reality devices interact with the server side simultaneously in a preset application, N is an integer greater than or equal to 2; the N virtual reality devices include: at least one first virtual reality device;
the processor 300 is further configured to perform the following process:
and distributing the determined picture to be displayed to each virtual reality device in the preset application.
Preferably, the processor 300 in the above embodiment of the present invention is further configured to perform the following processes:
if the number of times of the interaction failure event of the first virtual reality device in the preset application is equal to a preset number of times or the time of the interaction failure event of the first virtual reality device exceeds a preset threshold, generating virtual operation and substitution of the first virtual reality device according to historical display picture information of the N virtual reality devices and reported interaction information of the second virtual reality device which does not have the interaction failure event;
pushing the substitute of the first virtual reality device into the preset application, and executing the virtual operation in the preset application by the substitute of the first virtual reality device.
Preferably, in the above embodiment of the present invention, the processor 300 is further configured to perform the following processes:
and if the first virtual reality equipment and the server side resume interaction at a preset moment, sending historical information of the preset application between the moment when the interaction failure event occurs and the preset moment to the first virtual reality equipment, and stopping pushing the substitute of the first virtual reality equipment to the preset application.
In summary, the server device provided in the above embodiments of the present invention is not only suitable for interaction between a single user and a server, but also suitable for interaction between multiple users and a server; specifically, no matter single-user interaction with a server or multi-user interaction with the server, when one or more users and the server have interaction failure events, the server predicts a to-be-displayed picture and encodes and issues the to-be-displayed picture based on historical display picture information, user head movement historical information and user limb movement historical information, so that the visual fluency of VR (virtual reality) services is guaranteed; meanwhile, the accuracy of VR service hearing is ensured according to the transmission characteristics of sound; furthermore, if multiple users interact with the server, predicted pictures to be displayed need to be distributed to each user in the application, and substitution and virtual operation are generated when interaction failure exceeds a preset time or interaction failure times exceed preset times, so that application experience of other users without interaction failure is maintained, and visual fluency of the VR service is further guaranteed.
It should be noted that the server device provided in the above embodiment of the present invention is a server device capable of executing the virtual reality interaction compensation method, and all embodiments of the virtual reality interaction compensation method are applicable to the server device and can achieve the same or similar beneficial effects.
The embodiment of the present invention further provides a server device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements each process in the above-described embodiment of the compensation method for virtual reality interaction when executing the program, and can achieve the same technical effect, and details are not repeated here to avoid repetition.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements each process in the above-mentioned embodiment of the compensation method for virtual reality interaction, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block or blocks.
These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A compensation method for virtual reality interaction is applied to a server side, and is characterized by comprising the following steps:
acquiring historical display picture information of first virtual reality equipment with an interaction failure event, head movement history information of a user of the first virtual reality equipment and limb movement history information of the user of the first virtual reality equipment; the method comprises the steps that a server side does not receive interaction information reported by first virtual reality equipment interacting with the server side within a preset time period, and then the first virtual reality equipment is determined to have an interaction failure event;
determining a picture to be displayed of the first virtual reality device according to the historical display picture information, the user head movement historical information and the user limb movement historical information;
transmitting the determined picture to be displayed to the first virtual reality equipment;
if N virtual reality devices interact with the server side simultaneously in a preset application, N is an integer greater than or equal to 2; the N virtual reality devices include: at least one first virtual reality device;
the step of transmitting the determined picture to be displayed to the first virtual reality device includes:
distributing the determined picture to be displayed to each virtual reality device in the preset application;
the method further comprises the following steps:
if the number of times of the interaction failure event of the first virtual reality device in the preset application is equal to a preset number of times or the time of the interaction failure event of the first virtual reality device exceeds a preset threshold, generating virtual operation and substitution of the first virtual reality device according to historical display picture information of the N virtual reality devices and reported interaction information of the second virtual reality device which does not have the interaction failure event;
pushing the substitute of the first virtual reality device into the preset application, and executing the virtual operation in the preset application by the substitute of the first virtual reality device.
2. The method according to claim 1, wherein the step of determining the picture to be displayed of the first virtual reality device according to the historical display picture information, the user head movement history information and the user limb movement history information comprises:
predicting a panoramic picture to be displayed of the first virtual reality device according to the historical display picture information;
predicting the current position and the current orientation angle of the head of the user of the first virtual reality device according to the historical motion information of the head of the user;
determining the trend of the user limb motion trail of the first virtual reality equipment according to the user limb motion history information, and predicting the response result of the user limb to preset operation according to the trend of the user limb motion trail;
and determining the to-be-displayed picture of the first virtual reality device according to the to-be-displayed panoramic picture, the current position of the head of the user, the current orientation angle of the head of the user and the response result of the limbs of the user to preset operation.
3. The method according to claim 1, wherein after determining the screen to be displayed of the virtual reality device, the method further comprises:
acquiring sound information corresponding to the picture to be displayed;
coding the sound information and transmitting the sound information to the virtual reality equipment; and the sound information is matched with the picture to be displayed.
4. The method of claim 1, wherein the step of pushing the substitute for the first virtual reality device into the preset application and the substitute for the first virtual reality device performing the virtual operation in the preset application is followed by the method further comprising:
and if the first virtual reality equipment and the server side resume interaction at a preset moment, sending historical information of the preset application between the moment when the interaction failure event occurs and the preset moment to the first virtual reality equipment, and stopping pushing the substitute of the first virtual reality equipment to the preset application.
5. A server-side device, applied to a server side, comprising a processor and a transceiver, wherein the processor is configured to perform the following processes:
acquiring historical display picture information of first virtual reality equipment with an interaction failure event, head movement history information of a user of the first virtual reality equipment and limb movement history information of the user of the first virtual reality equipment; the method comprises the steps that a server side does not receive interaction information reported by first virtual reality equipment interacting with the server side within a preset time period, and then the first virtual reality equipment is determined to have an interaction failure event;
determining a picture to be displayed of the first virtual reality device according to the historical display picture information, the user head movement historical information and the user limb movement historical information;
transmitting the determined picture to be displayed to the first virtual reality equipment;
if N virtual reality devices interact with the server side simultaneously in a preset application, N is an integer greater than or equal to 2; the N virtual reality devices include: at least one first virtual reality device;
the processor is further configured to perform the following process:
distributing the determined picture to be displayed to each virtual reality device in the preset application;
the processor is further configured to perform the following process:
if the number of times of the interaction failure event of the first virtual reality device in the preset application is equal to a preset number of times or the time of the interaction failure event of the first virtual reality device exceeds a preset threshold, generating virtual operation and substitution of the first virtual reality device according to historical display picture information of the N virtual reality devices and reported interaction information of the second virtual reality device which does not have the interaction failure event;
pushing the substitute of the first virtual reality device into the preset application, and executing the virtual operation in the preset application by the substitute of the first virtual reality device.
6. The server device of claim 5, wherein the processor is further configured to perform the following process:
predicting a panoramic picture to be displayed of the first virtual reality device according to the historical display picture information;
predicting the current position and the current orientation angle of the head of the user of the first virtual reality device according to the historical motion information of the head of the user;
determining the trend of the user limb motion trail of the first virtual reality equipment according to the user limb motion history information, and predicting the response result of the user limb to preset operation according to the trend of the user limb motion trail;
and determining the to-be-displayed picture of the first virtual reality device according to the to-be-displayed panoramic picture, the current position of the head of the user, the current orientation angle of the head of the user and the response result of the limbs of the user to preset operation.
7. The server device of claim 5, wherein the processor is further configured to perform the following process:
acquiring sound information corresponding to the picture to be displayed;
coding the sound information and transmitting the sound information to the virtual reality equipment; and the sound information is matched with the picture to be displayed.
8. The server device of claim 5, wherein the processor is further configured to perform the following process:
and if the first virtual reality equipment and the server side resume interaction at a preset moment, sending historical information of the preset application between the moment when the interaction failure event occurs and the preset moment to the first virtual reality equipment, and stopping pushing the substitute of the first virtual reality equipment to the preset application.
9. A server device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor; characterized in that the processor, when executing the program, implements a compensation method for virtual reality interaction according to any one of claims 1 to 4.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of compensation of virtual reality interactions according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710801198.7A CN109474648B (en) | 2017-09-07 | 2017-09-07 | Virtual reality interaction compensation method and server-side equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710801198.7A CN109474648B (en) | 2017-09-07 | 2017-09-07 | Virtual reality interaction compensation method and server-side equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109474648A CN109474648A (en) | 2019-03-15 |
| CN109474648B true CN109474648B (en) | 2021-09-14 |
Family
ID=65657714
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710801198.7A Active CN109474648B (en) | 2017-09-07 | 2017-09-07 | Virtual reality interaction compensation method and server-side equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109474648B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111984111A (en) * | 2019-05-22 | 2020-11-24 | ***通信有限公司研究院 | Multimedia processing method, device and communication equipment |
| CN110941342B (en) * | 2019-12-04 | 2023-07-18 | 中国联合网络通信集团有限公司 | Data processing method, VR terminal and server |
| CN112651783B (en) * | 2020-12-30 | 2024-05-07 | 咪咕文化科技有限公司 | VR-based media file delivery method, device, equipment and storage medium |
| CN115079881B (en) * | 2022-06-16 | 2024-07-09 | 广州国威文化科技有限公司 | Picture correction method and system based on virtual reality |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105929950A (en) * | 2016-04-15 | 2016-09-07 | 惠州Tcl移动通信有限公司 | Mobile terminal and time-limited use method and apparatus for VR glasses |
| CN106200957A (en) * | 2016-07-07 | 2016-12-07 | 北京行云时空科技有限公司 | Virtual reality interactive system and method towards motion |
| CN106200953A (en) * | 2016-07-05 | 2016-12-07 | 乐视控股（北京）有限公司 | The combination of wireless ring, ring and virtual reality device control method thereof |
| CN106325506A (en) * | 2016-08-17 | 2017-01-11 | 捷开通讯(深圳)有限公司 | Interaction method for virtual reality device, virtual reality device and virtual reality system |
| CN106598253A (en) * | 2016-12-23 | 2017-04-26 | 北京搜狐新媒体信息技术有限公司 | Data prediction method and apparatus |
| GB2544458A (en) * | 2015-10-08 | 2017-05-24 | Facebook Inc | Binaural synthesis |
-
2017
- 2017-09-07 CN CN201710801198.7A patent/CN109474648B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2544458A (en) * | 2015-10-08 | 2017-05-24 | Facebook Inc | Binaural synthesis |
| CN105929950A (en) * | 2016-04-15 | 2016-09-07 | 惠州Tcl移动通信有限公司 | Mobile terminal and time-limited use method and apparatus for VR glasses |
| CN106200953A (en) * | 2016-07-05 | 2016-12-07 | 乐视控股（北京）有限公司 | The combination of wireless ring, ring and virtual reality device control method thereof |
| CN106200957A (en) * | 2016-07-07 | 2016-12-07 | 北京行云时空科技有限公司 | Virtual reality interactive system and method towards motion |
| CN106325506A (en) * | 2016-08-17 | 2017-01-11 | 捷开通讯(深圳)有限公司 | Interaction method for virtual reality device, virtual reality device and virtual reality system |
| CN106598253A (en) * | 2016-12-23 | 2017-04-26 | 北京搜狐新媒体信息技术有限公司 | Data prediction method and apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109474648A (en) | 2019-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109474648B (en) | Virtual reality interaction compensation method and server-side equipment | |
| JP7072677B2 (en) | Methods and devices for transmitting scene images of virtual scenes, computer devices and computer readable storage media. | |
| CN111526927B (en) | Temporary game control via user simulation after loss of active control | |
| US10477179B2 (en) | Immersive video | |
| US11701584B2 (en) | Dynamic client buffering and usage of received video frames for cloud gaming | |
| CN105791977B (en) | Virtual reality data processing method, equipment and system based on cloud service | |
| US10034036B2 (en) | Media synchronization for real-time streaming | |
| EP3807766B1 (en) | Shadow tracking of real-time interactive simulations for complex system analysis | |
| US20160366228A1 (en) | Sending application input commands over a network | |
| CN104954349B (en) | The synchronous method of client, device and system in a kind of 2D game | |
| JP6379107B2 (en) | Information processing apparatus, control method therefor, and program | |
| CN104998412A (en) | Single-player game realization method and apparatus | |
| US10530824B2 (en) | Real-time end-to-end latency monitoring | |
| CN113079416B (en) | Multimedia intelligent control method, client and system | |
| Ma et al. | Research on end-to-end cloud asynchronous rendering based on virtual reality scenes | |
| CN116489414A (en) | Live interaction method, device, system, computing equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |