CN110114746A - The method and virtual reality device of display virtual real picture - Google Patents
The method and virtual reality device of display virtual real picture Download PDFInfo
- Publication number
- CN110114746A CN110114746A CN201780079800.9A CN201780079800A CN110114746A CN 110114746 A CN110114746 A CN 110114746A CN 201780079800 A CN201780079800 A CN 201780079800A CN 110114746 A CN110114746 A CN 110114746A
- Authority
- CN
- China
- Prior art keywords
- task
- window
- application program
- stack
- front stage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Digital Computer Display Output (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
A kind of method and virtual reality device (800) of display virtual real picture.Method includes: that at least two windows are established in virtual reality interface, and corresponding at least two application programs of at least two windows, at least two windows are used to show the display picture (110) of at least two application programs；Establish at least two task stacks, at least two task stacks are corresponding at least two windows, at least one task of first task stack management the first application program corresponding with first window at least two task stacks, any one task at least one task includes the component for being in front stage operation state, first task stack is corresponding with first window, and first task stack is any task stack (120) at least two task stacks；By the component in front stage operation state managed at least two task stacks, at least two corresponding display pictures (130) of component for being in front stage operation state are shown at least two windows.Multiple utility program foreground in the interface VR can be made to run parallel.
Description
This application claims Patent Office of the People's Republic of China was submitted on 01 04th, 2017, application No. is 201710005621.2, a kind of priority of the Chinese patent application of application entitled " method and apparatus that combination mobile terminal realizes virtual reality ", entire contents are hereby incorporated by reference in the application.
This application involves technical field of virtual reality, and more particularly, to the method and virtual reality device of a kind of display virtual real picture.
Virtual reality (Virtual Reality, VR) technology is a kind of technology for being capable of providing and experiencing to 360 degree of immersions of user, allows what user can be on the spot in person to be in some environment.Specifically, VR technology can construct a virtual three-dimensional scenic, such as hospital, forest, family etc., be then divided into this scene and being observed respectively by the right and left eyes of people in a manner of the picture of left and right, allow people that can experience the immersion experience of 3 D stereo.
According to system platform differentiation is realized, VR be can be generally divided into conjunction with mobile terminal VR personal computer (PC, Personal Computer) end the PC VR realized and mobile terminal is combined to realize.Current most of VR belong to the end PC VR, mobile terminal VR is in the starting stage, and many application programs need developer to develop again, is originally that the magnanimity application of mobile terminal exploitation cannot easily be shown in VR, and in existing mobile terminal VR system, foreground can only run an application program.
Summary of the invention
The application provides the method and virtual reality device of a kind of display virtual real picture, existing application easily can be transplanted to Virtual Reality and show mode, and multiple utility program foreground is run parallel, improve user experience.
First aspect, provide a kind of method of display virtual real picture, applied to virtual reality device, this method comprises: establishing at least two windows in Virtual Reality interface, at least two windows correspond at least two application programs, and at least two windows are used to show the display picture of at least two application programs；Establish at least two task stacks, at least two task stacks and at least two windows correspond, at least one task of first task stack management the first application program corresponding with first window at least two task stacks, any one task at least one task includes the component for being in front stage operation state, wherein, first task stack is corresponding with first window, and first task stack is any one task stack at least two task stacks；By the component in front stage operation state managed at least two task stacks, to show the corresponding display picture of component for being in front stage operation state at least two windows.
The method of display virtual real picture according to an embodiment of the present invention, by establishing at least two windows, each window is used to show the display picture of application program, and the corresponding task stack of each window, there is the component for being in front stage operation state in task stack, to not only make existing application be transplanted to Virtual Reality display module, but also it is also possible that multiple application programs are in front stage operation state simultaneously.
Optionally, the first task at least two tasks includes the first assembly for being in front stage operation state, wherein first task is any one task at least two tasks；When first assembly exits, determine the corresponding first task of first assembly whether be first task stack a last task；If it is determined that first task is a last task for first task stack, first task stack and first window corresponding with first task stack are destroyed；Or, if it is determined that first task is not a last task for first task stack, and the next task of first task stack is shown on the first window corresponding to first task stack.
By destroying task stack and the corresponding window of task stack, the system resource occupied by task stack can be discharged.
It optionally, is that the display picture of the second application program creates image texture, the second application program is any one application program at least two application programs when detecting that the second application program is activated；By binding corresponding with the second application program the second window of image texture, image texture is placed on the object model where the second window；The display picture of the second application program is shown on the object model where the second window.
By the way that texture and window are bound, it will be created that the texture come is attached on specific object model, it when so opening application program is placed into default screen, and it is placed into the present invention and is created that on the window come, the picture of application program in this way is not just to directly display in default screen, and be shown on the object model of patch texture.
Optionally, when the display frame updating image of the second application program, by the image buffer storage of the second application program update into the second window, and it is very that renewable flag bit is used to determine whether to refresh the texture that renewable flag bit, which is arranged,；When renewable flag bit is true, refresh the texture, and it is false for updating the renewable flag bit.
Optionally it is determined that activating third window, and interact operation with the third application program for being in front stage operation state on third window, wherein third window is any one window at least two windows when the blinkpunkt of user is fallen on third window.
Optionally, third window is activated, and interacts operation with the third application program for being in front stage operation state on third window, comprising: focus variable is set for third window；The window list opened is traversed, the third window with focus variable is searched from window list, and third window is set and obtains focus；According to visibility and focus, alternative events are distributed to the third application program that front stage operation state is on third window, to activate interactivity to operate.
Second aspect provides virtual reality device, the module of each step in the method including each implementation for executing above-mentioned first aspect and first aspect.
The third aspect provides virtual reality device, including memory and processor, and the memory is for storing computer program；The processor from memory for calling and running the computer program, so that virtual reality device executes the method in any possible implementation of first aspect and first aspect.
Fourth aspect, provide a kind of computer program product, the computer program product includes: computer program code, when the computer program code is run by the processor of virtual reality device, so that virtual reality device executes the method in any possible implementation of first aspect or first aspect.
5th aspect, provides a kind of computer readable storage medium, including computer instruction, when the computer instruction is run on computers, so that computer is executed such as the method in any possible implementation of first aspect or first aspect.
The application is by creating the window in virtual reality interface for application program, the display picture of the application program is shown on window, and pass through the hierarchical relationship for establishing window, the display picture of application program can be not only made all to show in the interface VR, it is also possible that multiple utility program foreground is run parallel.
Fig. 1 is the schematic flow chart of the method for display virtual real picture according to an embodiment of the present invention；
Fig. 2 is the display relational graph of native system；
Fig. 3 is display relational graph according to an embodiment of the present invention；
Fig. 4 is the schematic diagram of the activity Booting sequence under activity Booting sequence according to an embodiment of the present invention and native system mode；
Fig. 5 is schematic diagram when exiting when activity according to an embodiment of the present invention is exited with the activity under native system mode；
Fig. 6 is the schematic diagram of the corresponding relationship of application screen window and virtual monitor according to an embodiment of the present invention；
Fig. 7 is the schematic diagram of the interactive process under interactive process according to an embodiment of the present invention and native system mode；
Fig. 8 is the schematic block diagram of virtual reality device according to an embodiment of the present invention；
Fig. 9 is the schematic block diagram of virtual reality system according to an embodiment of the present invention；
Figure 10 is the specific flow chart of display module 910；
Figure 11 is the schematic block diagram of the device of display virtual real picture according to an embodiment of the present invention.
Below in conjunction with attached drawing, the technical solution in the application is described.
The application program that the application refers to can be the application program in mobile terminal, and wherein the type of mobile terminal is not especially limited.Optionally, in some embodiments, which can be the terminal device with online and/or call function.Such as, the mobile terminal can be mobile phone, tablet computer, helmet-mounted display (head-mounted displays, HMD), laptop, Ultra-Mobile PC (Ultra-mobile Personal Computer, UMPC), net book, personal digital assistant (Personal Digital Assistant, PDA) etc..
It should be noted that the application program that the application refers to is without being limited thereto, any application program for wanting VRization to show is suitable for the application, and the embodiment of the present application is illustrated by taking the application program in mobile terminal as an example.
Virtual reality (Virtual Reality, VR) technology is a kind of computer simulation system that can be created with the experiencing virtual world, it generates a kind of simulated environment using computer, is that a kind of Multi-source Information Fusion, interactive Three-Dimensional Dynamic what comes into a driver's and the system emulation of entity behavior are immersed to user in the environment.The Virtual Reality interface that the application refers to is then the scene generated by VR technology, object and personage and the three-dimensional immersion experience established, e.g., can be formed by 3D software development.
VR can be generally divided into conjunction with mobile terminal VR personal computer (Personal Computer, PC) end the PC VR realized and mobile terminal is combined to realize.Currently, mobile terminal VR is in the starting stage, the mobile terminal field VR application program is deficient, and the magnanimity application of originally mobile terminal exploitation cannot show that many application programs need developer to develop or modify again in VR, considerably increases cost.Furthermore in existing VR system, an application program can only be opened on foreground, for example, user can not accomplish the application scenarios such as watch movie and chat simultaneously.Therefore, the embodiment of the present invention, which is concerned with, is shown the application program of mobile terminal directly in the interface VR, and can open simultaneously at least two application programs on foreground.
In embodiments of the present invention, it is illustrated by taking the application program of mobile terminal as an example.Mobile terminal is connected with VR equipment, e.g., can be at least one of eyeball tracking equipment, the VR helmet, VR box and mobile intelligent terminal by bluetooth connection, VR equipment, but not limited to this, any equipment that can show VR picture can be VR equipment.
A set of software program is developed in the embodiment of the present invention (can be the system level program of mobile device, or application layer program) carry out the system (hereinafter referred to as VR system) that VRization is handled, corresponding user's display interface is known as VR system desktop, and the program of starting VR system desktop is known as VR system desktop application program.The promotion of the hardware performance of existing mobile device, and the restriction of display screen size, the limitation that the application shown in mobile device is subject to is increasing, therefore, by the way that the system of mobile device is carried out VRization, it can to open simultaneously multiple application programs inside VR system, and open application program is all placed in the same scene, such as, video display video is played in the scene of one family, in virtual TV, hangs calendar information on wall, it is vacantly placed social chat window etc. in the air, so that user has better VR to experience.
Next combine Fig. 1 that the embodiment of the present invention is described in detail.Fig. 1 is a kind of schematic flow chart of method for realizing virtual reality according to an embodiment of the present invention.This method 100 is applied to virtual reality device, can be executed by VR system.Method 100 includes step 110-130, describes each step in detail below.
110, at least two windows are established in Virtual Reality interface, corresponding at least two application programs of at least two window, at least two window is used to show the display picture of at least two application program.
Embodiment through the invention can make at least two application programs while be in front stage operation state.
In embodiments of the present invention, it is illustrated so that at least two application programs are the application programs in mobile terminal as an example.Mobile terminal includes multiple (more than one) application programs, for example has the application program of magnanimity on mobile phone.First turn on VR system, such as, it can be based on user's operation, connect VR equipment and mobile terminal, a VR system desktop is taken in VR equipment, the icon of application program in mobile terminal can show in VR system desktop, when the user clicks the icon of corresponding application programs, i.e., openable corresponding application program.When user opens the application program on mobile phone, system is at least one virtual monitor (Virtual Display in each application program creation interface VR, VD) (i.e., an example of window), the VD is used to show the display picture of application program, specifically, the display picture of application program is the interface of application program, and the picture on VD be free to be placed on from anywhere in the interface VR.
At least two VD correspond at least two application programs, at least two VD are used to show the display picture of at least two application programs, specifically, illustrate by taking two VD as an example, the display picture of two different applications can be shown on two VD, such as, a VD shows that the chat interface of wechat, a VD show the broadcast interface of video；Alternatively, it is also possible to the different display pictures that two VD show same application, and e.g., the chat interface of wechat and the circle of friends interface of wechat.Likewise, more than two VD are also such.The display picture of each application program is shown to user on its corresponding VD, and the state between application program is independent, that is to say, that multiple application programs may be at different operating statuses.Wherein, these states include but is not limited to run, suspend, stop etc..
It should be noted that the embodiment of the present invention is illustrated so that window is VD as an example, however, the present invention is not limited thereto, for example, window is also possible to the virtual screen object that the included SurfaceComposerClinet class of Android is created that.
The method for creating VD for application program is similar to the prior art, and this is not limited by the present invention.
After creating VD for application program, needs to show the display picture of application program on VD, will be illustrated next and how to show application program on VD.
As one embodiment, when detecting that the second application program in mobile terminal is activated, image texture is created for the display picture of second application program, which is any one application program at least two application programs in mobile terminal；By binding corresponding with the second application program the second window of image texture, image texture is placed on the object model where the second window；The display picture of the second application program is shown on the object model where the second window.
Specifically, although can also grab concrete application picture by the end PC, and the embodiment of the present invention is realized in mobile device, and due to the difference of operating system, the implementation of the two is completely different.Optionally, it detects that the second application program in mobile terminal is activated, specifically, can be the operation process etc. of detection central processing unit, similarly to the prior art, the present invention is not construed as limiting specific detection mode.When detecting that the second application program in mobile terminal is activated, pass through open graphic interface (Open Graphics Library, OpenGL) texture object is created for the picture of application program, and the texture object and the embodiment of the present invention are created that the VD come is bound for the application program, it will be created that the texture object come is attached on specific object model by VR system desktop application program, be no longer placed into when opening application program on default screen (DefaultDisplay) and be placed into the embodiment of the present invention and be created that on the VD come.The picture of application program in this way is not just to directly display on DefaultDisplay, and be shown on the object model of patch texture.
Wherein, the dummy object in object model VR scene, which can be is formed by 3D software development.
It should be understood that the image texture present invention of the display picture of creation application program is not construed as limiting, for example, it is also possible to come to create texture object for application program by Multimedia Programming interface (Direct eXtension, DX).
As one embodiment, when the display frame updating image of the second application program, by the image buffer storage of the second application program update into the second window, and it is that very, renewable flag bit is used to determine whether to refresh texture that renewable flag bit, which is arranged,；When renewable flag bit is true, refresh texture, and it is false for updating renewable flag bit.
120, establish at least two task stacks, at least two task stack and at least two windows correspond, at least one task of first task stack management the first application program corresponding with first window at least two task stacks, any one task at least one task includes the component for being in front stage operation state, wherein, the first task stack is corresponding with first window, and first task stack is any one task stack at least two task stacks.
Through the embodiment of the present invention, multiple application programs can be made while is in the state of front stage operation.
For a better understanding, introducing the screen (Display) in native system first, Display has following hierarchical relationship in primary Android (Android) system:
DefaultDisplay- > Stack (multiple) -> Task (multiple) -> activity (multiple), that is, in native system default framework, one DefaultDisplay one or more (more than one) stacks (Stack) included below, it again include one or more (more than one) task (Task) under one Stack, an a minimum of activity (activity) (that is, an example of component) inside one Task.Wherein, it is also most commonly seen one of four big components that activity, which is most basic in Android component, it is an application component, provides a screen, and user can be used to interact to complete a certain task.
Fig. 2 shows the screen relationships of native system.It can be seen that, there are two Stack for lower of native system default situations, one of them is system stack (HomeStack), the other is using stack (AppStack), there are two Task inside HomeStack, one is that system desktop applies (Launcher Task), the other is applying (Recent Task) recently.Launcher Task just includes the display of desktop.In addition to Launcher Task and Recent Task, other Task in system are included in AppStack.Only one activity is in front stage operation state inside each Stack.In this case, multiple programs cannot be achieved while running.
The embodiment of the invention provides a kind of methods, and multiple application programs can be made to run parallel on foreground simultaneously, as shown in Figure 3.Fig. 3 shows the schematic flow chart of display process according to an embodiment of the present invention, in embodiments of the present invention, multiple application programs while the state of operation is in allow, as described above, establish multiple VD, one application program is corresponding at least one VD, and VD can be a conventional modules of android system, rebuilds a set of new hierarchical relationship according to VD:
VD- > Stack (single) -> Task (multiple) -> activity (multiple), that is, a Stack is established on each VD (i.e., an example of task stack), the Stack is not influenced by primary Stack (such as HomeStack, AppStack), the state that each Stack has an activity to be in front stage operation.
Specifically, it is application program _ 1 corresponding stack of creation VD_1, VD_1, includes task 1 ... ..., task n in stack；VD_n is created for application program _ n, VD_n corresponds to a stack, it include task 1 ... ..., task n in stack, i.e., each application program corresponds at least one VD, the corresponding Stack of each VD, the state that each Stack has an activity to be in front stage operation, to which multiple application programs can be in front stage operation state simultaneously.
That is, managing at least one task of application program corresponding with the task stack in each task stack, there are multiple activity in each task, only one activity is the state in front stage operation.Task stack is a kind of data structure, and the mode of management role can be the related data for storing application program corresponding with the task stack about the task.
130, by the component in front stage operation state managed at least two task stacks, at least two corresponding display pictures of component for being in front stage operation state are shown at least two windows.
So that activity is (i.e., an example of component) it is in front stage operation state, it is meant that and the corresponding application program of activity is also at front stage operation state, the corresponding display picture of activity, that is, the display picture of a task in the corresponding application program of activity.The Booting sequence of activity in embodiments of the present invention and the Booting sequence under native system mode are different, see Fig. 4.
From fig. 4, it can be seen that in native system, comprising:
401, start activity from desktop；
402, create new task；
403, task is added to using in stack；
404, activity is to returning to (Resume) state for starting.
As previously described, in native system, in addition to Launcher Task and Recent Task, other Task in system are included in using in stack, only one activity is in front stage operation state inside each application stack, and then only one application program is in front stage operation state.
In embodiments of the present invention, comprising:
410, start activity from desktop.The step is identical under native system mode.
420, obtain screen number to be started.There are multiple display screens in VR scene, each display screen has corresponding number, and user can determine that application program is shown on which specific display screen.
430, create window.That is, creation VD.
440, stack is established under new window.The hierarchical relationship of VD as described previously is created that on the VD come in step 430 and establishes a Stack, constructs the Stack of a non-protogenous on VD, which is not influenced by primary Stack (for example, AppStack, HomeStack).
450, create new task.There can be one or more tasks inside one Stack.
460, task is added in corresponding stack.The difference of this step and the step 403 under above-mentioned native system mode is, when task is added to corresponding Stack, that is, the Stack established on VD, rather than AppStack.That is a Task can reside on any one VD, ensure that multiple Task can be simultaneously in the state run in this way.470, activity is to returning to state for starting.
Wherein, step 420-440 is the step not having in native system, by creating the VD in VR system desktop for application program, each application program can be made to correspond at least one VD, then multiple application programs can be in front stage operation state simultaneously.
It should be understood that magnitude of the sequence numbers of the above procedures are not meant that the order of the execution order, the execution sequence of each process be should be determined by its function and internal logic, and the implementation process of the embodiments of the invention shall not be constituted with any limitation.
The case where describeing how to show in the display picture of application program when on VD and the case where activity starts, next combining Fig. 5 introduction closing task of above example.The flow chart that the task that Fig. 5 shows exits.
Firstly, simply introducing the situation in native system mode, under native system mode, task is closed, and passes directly to desk, what desk was constantly present.
In embodiments of the present invention, optionally, the first task at least two tasks includes the first activity (that is, an example of first assembly) for being in front stage operation state, wherein first task is any one task at least two tasks；When the first activity is exited, determine the corresponding first task of the first activity whether be first task stack a last task；When determining first task is a last task for first task stack, first task stack and first window corresponding with first task stack are destroyed；Alternatively, showing the next task of first task stack on the first window corresponding to first task stack when determining first task not is a last task for first task stack.
By removing stack and corresponding window, the system resource occupied by task stack can be discharged.
In order to provide good user experience, its application program for wanting to operate very easily is chosen and operated to the embodiment of the present invention by providing following interactive modes support users.
The interactivity of VR refer to user in simulated environment object can operational degree and the natural degree fed back from environment.
Optionally, by the output of the virtual reality device or interactive device of detection user come so that user and application program interact.
Optionally, virtual reality device is at least one of eyeball tracking equipment, virtual implementing helmet and mobile terminal, interactive device is at least one of the helmet, mouse, keyboard, handle, such as, when user can operate the application program using interactive device or other interactive devices by user when watching that perhaps the rotation helmet makes some application program be in interaction mode attentively.
As one embodiment, when determining that the blinkpunkt of user is fallen on third window, third window is activated, and interact operation with the third application program for being in front stage operation state on third window, wherein, third window is any one window at least two windows.
Specifically, when the interface VR is opened, there are the application screens of multiple openings in interface (i.e., an example of window), the display picture an of application program is shown in each application screen, when user is fallen in by way of watching attentively or rotating the helmet in some application screen, user can activate the application screen, and interact operation to the application program of front stage operation in the application screen.
As one embodiment, third window is activated, and interacts operation with the third application program for being in front stage operation state on third window, comprising: focus variable is set for third window；The window list opened is traversed, the third window with focus variable is searched from window list, and third window is set and obtains focus；According to visibility and focus, alternative events are distributed to the third application program that front stage operation state is on third window, to activate interactivity to operate.
Specifically, by taking window is application screen window as an example, there are multiple application screen windows in the interface VR, application screen window and VD are corresponded, as shown in Figure 6.It is assumed to be an application program in application layer and creates n VD, wherein n is the integer not less than 1, each VD corresponds to a stack, and the task of at least one application program, in system layer, there are multiple application screen windows, respectively correspond a VD, the VD that user interacts is the application screen window for being assigned focus variable.
Specific interaction is realized as shown in fig. 7, Fig. 7 shows interaction realization when multiple utility program foreground according to an embodiment of the present invention is run parallel and the interaction under native system mode.
Firstly, simply introducing the interaction scenario under native system mode.
Application program 1 is opened, open application program 1 is placed on virtual monitor 1, sets the focal point to application program 1；Similarly, application program 2 is opened, open application program 2 is placed on virtual monitor 2, sets the focal point to application program 2；……；Application program n is opened, open application program n is placed on virtual monitor n, sets the focal point to application program n.Then judge the opening sequence of application program, default Focus Club corresponds to application screen window corresponding to the last one application program.
In embodiments of the present invention, switch focus, by VD belonging to focus setting to the application screen for needing to switch, numbered according to VD, traverse open application screen window, find the window for operating in the application screen of stack top, according to visibility and focus, distribute alternative events.
Specifically, after normal starting application, when calling the application interface of setting focus, the focus variable in the service of application screen window management can be set, and then the service of application screen window management can traverse open application screen window list.The application screen window for having focus variable is found from list, and this application screen window is set as to obtain focus, then when distributing alternative events in interactive distribution process, it can be according to visibility and focus, alternative events are distributed to the application program for needing to be arranged focus, to activate interactivity to operate.
As can be seen that in embodiments of the present invention, under the parallel operational mode of multiple utility program, user can quickly and accurately be interacted.
Optionally it is determined that the number of the VD where the viewpoint of user；The corresponding application program of VD is searched according to the number, and carries out interactive operation.
It should be noted that, the embodiment of the present invention is illustrated with the virtual monitor module that android system carries, however, the present invention is not limited thereto, such as, also the virtual monitor module creation that the included SurfaceComposerClient class of android system or non-android system can be used goes out a virtual monitor, then reaches identical purpose.
It should be understood that, in embodiments of the present invention, the succinct convenience for illustration only such as " first ", " second ", " third ", but the range of the embodiment of the present invention is not construed as limiting, such as, " first window, the second window, third window " is all any one for indicating multiple windows in the interface VR, and for another example, " the first application program, the second application program, third application program " is all any one indicated in multiple application programs.
In embodiments of the present invention, by creating window for application program, application program is shown on window, and makes multiple application programs while being in front stage operation state, existing mobile terminal application program can be made all to show in VR system desktop in the form of VRization；And technically may be implemented that user is supported easily to choose and operate its application program for wanting operation, so as to provide good user experience.
The embodiment of the invention also provides a kind of virtual reality devices, as shown in figure 8, equipment 800 includes:
810, creation module, for establishing at least two windows, corresponding at least two application programs of at least two windows in Virtual Reality interface, at least two windows are used to show the display picture of at least two application programs；
Creation module 810 is also used to establish at least two task stacks, at least two task stacks and at least two windows correspond, at least one task of first task stack management the first application program corresponding with first window at least two task stacks, any one task at least one task includes the component for being in front stage operation state, wherein, first task stack is corresponding with first window, and first task stack is any one task stack at least two task stacks；
820, display module shows at least two corresponding display pictures of component for being in front stage operation state for the component in front stage operation state by managing at least two task stacks at least two windows.
Embodiment through the invention is modified by the operating system to mobile terminal, and VR is transplanted in existing application shows mode with can be convenient.
Optionally, the first task at least two tasks includes the first activity (that is, an example of first assembly) for being in front stage operation state, wherein first task is any one task at least two tasks；Equipment 800 further includes determining module 830, and determining module 830 is used for: when the first activity is exited, determine the corresponding first task of the first activity whether be first task stack a last task；If it is determined that first task is a last task for first task stack, first task stack and first window corresponding with first task stack are destroyed；Or, if it is determined that first task is not a last task for first task stack, and the next task of first task stack is shown on the first window corresponding to first task stack.
Optionally, when creation module 810 is also used to detect that the second application program is activated, image texture is created for the display picture of second application program, which is any one application program at least two application programs in mobile terminal；Display module 820 is specifically used for: by binding image texture the second window corresponding with the second application program, image texture is placed on the object model where the second window, the display picture of the second application program is shown on the object model where the second window.
Optionally, display module 820 is also used to: when the interface more new images of the second application program, by the image buffer storage of the second application program update into the second window, and it is very that renewable flag bit is used to determine whether to refresh the texture that renewable flag bit, which is arranged,；When renewable flag bit is true, refresh the texture, and it is false for updating the renewable flag bit.
Optionally, equipment 800 further includes interactive module 840, for: when determining that the blinkpunkt of user is fallen on third window, activate third window, and operation is interacted with the third application program for being in front stage operation state on third window, wherein, third window is any one window at least two windows.
Optionally, interactive module 840 is specifically used for: focus variable is arranged for third window；The window list opened is traversed, the third window with focus variable is searched from window list, and third window is set and obtains focus；According to visibility and focus, touch event is distributed to the third application program that front stage operation state is on third window, to activate interactivity to operate.
Each module or unit are respectively used to execute the movement in the above method 100 in the equipment 800, and here, in order to avoid repeating, description is omitted.
In embodiments of the present invention, by creating window for application program, the display picture of application program is shown on window, and makes multiple application programs while being in front stage operation state, existing mobile terminal application program can be made all to show in VR system desktop in the form of VRization；And technically may be implemented that user is supported easily to choose and operate its application program for wanting operation, so as to provide better user experience.
Fig. 9 provides a kind of schematic diagram of virtual reality system 900.System 900 includes display module 910, parallel to run module 920, interactive module 930.Next modules are specifically described.
Display module 910 carries out VRization processing for the operating system desktop to mobile device end, and on the surface for the dummy object that the display picture of application program is attached in the VR scene for needing to place, forms an application screen.
As one embodiment, display module 910 divides for two parts, produce module and consumption module, produce module creation VD, it opens application program to be simultaneously placed on VD, such application program update picture when will not be updated directly on physical display screen, but update into the caching of VD, after having updated picture, it is true that renewable flag bit, which can be arranged, in production module.The texture mapping is attached on the correspondence object model in scene (that is, the interface VR) by consumption module by VR system desktop application creation texture mapping.Texture is updated by detecting renewable flag bit, it will be in the texture mapping of the buffer update on VD to object model if renewable flag bit is very, and updating renewable flag bit is vacation, then refreshes the picture in the interface VR on object model using updated texture mapping.Shown in detailed process Figure 10.
In generation module, 1005, creation VD, that is, create VD for application program；
1010, it opens application program and application program is placed on VD；
1015, application program update picture, and picture is cached to VD；
1020, updating flag bit is that very, then flag bit may be updated in set, detects renewable flag bit；
1025, a texture mapping is created using VR system desktop application program；
1030, it will be created that on some object model that is attached in scene of texture come；
1035, detect renewable flag bit；
1040, flag bit may be updated in judgement, if renewable flag bit is vacation, returns to 1035, continues to test renewable flag bit, if renewable flag bit is that very, next step 1045 will be cached on the frame updating to texture on VD, and it is false for updating flag bit；
1050, using texture refreshed image, 1035 are then returned, detects renewable flag bit.
It should be understood that magnitude of the sequence numbers of the above procedures are not meant that the order of the execution order, the execution sequence of each process be should be determined by its function and internal logic, and the implementation process of the embodiments of the invention shall not be constituted with any limitation.
Parallel operation module 920, also referred to as multiple utility program foreground run module parallel, are in the technology of front stage operation, the i.e. parallel running technology in multiple utility program foreground simultaneously for realizing multiple utility program.
Specifically, the realization of module 920 is run parallel as shown in figure 3, i.e., a Stack is established for each VD, at least one activity of each VD is in the state of front stage operation, and an application program corresponds at least one VD, and application programs multiple in this way can be in the state of front stage operation simultaneously.The method that parallel operation module 920 can execute Fig. 3 to Fig. 5, details are not described herein again.
Interactive module 930 for flexible choice application program and is at interaction mode.Such as: user can make some application program be in interaction mode by eyeball fixes or the rotation helmet, and above-mentioned technology can assist realizing by devices such as eyeball tracking or sensors.The method that interactive module 930 can execute Fig. 6 to Fig. 7, details are not described herein again.
Figure 11 is a kind of structural schematic diagram of the device 1100 of display virtual real picture according to an embodiment of the present invention.
As shown in figure 11, which includes: processor 1110 and memory 1120.Wherein, processor 1110 is connected with memory 1120 by bus system, and for storing instruction, which is used to execute the instruction of the memory 1120 storage to memory 1120, so that device 1100 executes the function of above-mentioned each method or equipment.
It will also be understood that, in embodiments of the present invention, the processor 1110 can be comprising central processing unit (Central Processing Unit,) and graphics processing unit (Graphics Processing Unit CPU, GPU integrated processor or isolated processor), the processor 1110 can also be other general processors, digital signal processor (DSP), specific integrated circuit (ASIC), ready-made programmable gate array (FPGA) either other programmable logic device, discrete gate or transistor logic, discrete hardware components etc..General processor can be microprocessor or the processor is also possible to any conventional processor etc..
The memory 1120 may include read-only memory and random access memory, and provide instruction and data to processor.The a part of of memory can also include nonvolatile RAM.For example, memory can be with the information of storage device type.
The bus system can also include power bus, control bus and status signal bus in addition etc. in addition to including data/address bus.But for the sake of clear explanation, various buses are all designated as bus system in figure.
Those of ordinary skill in the art may be aware that unit described in conjunction with the examples disclosed in the embodiments of the present disclosure and algorithm steps, can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are implemented in hardware or software actually, the specific application and design constraint depending on technical solution.Professional technician can use different methods to achieve the described function each specific application, but this realization is it is not considered that exceed scope of the present application.
It is apparent to those skilled in the art that for convenience and simplicity of description, system, the specific work process of device and unit of foregoing description can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods may be implemented in other ways.Such as, the apparatus embodiments described above are merely exemplary, such as, the division of the unit, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed mutual coupling, direct-coupling or communication connection can be through some interfaces, the indirect coupling or communication connection of device or unit, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, and component shown as a unit may or may not be physical unit, it can and it is in one place, or may be distributed over multiple network units.It can some or all of the units may be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
In addition, each functional unit in each embodiment of the application can integrate in one processing unit, it is also possible to each unit and physically exists alone, can also be integrated in one unit with two or more units.
If the function is realized in the form of SFU software functional unit and when sold or used as an independent product, can store in a computer readable storage medium.Based on this understanding, substantially the part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products the technical solution of the application in other words, the computer software product is stored in a storage medium, it uses including some instructions so that a computer equipment (can be personal computer, server or the network equipment etc.) execute each embodiment the method for the application all or part of the steps.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), the various media that can store program code such as random access memory (Random Access Memory, RAM), magnetic or disk.
It is described above; the only specific embodiment of the application, but the protection scope of the application is not limited thereto, and anyone skilled in the art is within the technical scope of the present application; it can easily think of the change or the replacement, should all cover within the scope of protection of this application.Therefore, the protection scope of the application should be based on the protection scope of the described claims.
Claims (13)
- A kind of method of display virtual real picture, which is characterized in that be applied to virtual reality device, which comprisesAt least two windows, corresponding at least two application programs of at least two window are established in Virtual Reality interface, at least two window is used to show the display picture of at least two application program；Establish at least two task stacks, at least two task stack and at least two window correspond, first task stack at least two task stack is used to manage at least one task of the first application program corresponding with first window, any one task at least one described task includes the component for being in front stage operation state, wherein, the first task stack is corresponding with the first window, and the first task stack is any one task stack at least two task stack；By the component in front stage operation state managed at least two task stack, to be in the corresponding display picture of component of front stage operation state described in the display at least two at least two window.
- The method according to claim 1, wherein the method also includes:First task at least two task includes the first assembly for being in front stage operation state, wherein the first task is any one task at least two task；When the first assembly exits, determine the corresponding first task of the first assembly whether be the first task stack a last task；If it is determined that the first task is a last task for the first task stack, the first task stack and the first window corresponding with the first task stack are destroyed；OrIf it is determined that the first task is not a last task for the first task stack, the next task of the first task stack is shown on the corresponding first window of the first task stack.
- Method according to claim 1 or 2, which is characterized in that the corresponding display picture of component for being in front stage operation state described in display at least two at least two window, comprising:It is that the display picture of second application program creates image texture, second application program is any one application program at least two application program when detecting that the second application program is activated；By binding corresponding with second application program the second window of described image texture, described image texture is placed on the object model where second window；The display picture of second application program is shown on the object model where second window.
- According to the method described in claim 3, it is characterized in that, the method also includes:When the display frame updating image of second application program, by the image buffer storage of second application program update into second window, and it is very that the renewable flag bit is used to determine whether to refresh the texture that renewable flag bit, which is arranged,；When the renewable flag bit is true, refresh the texture, and it is false for updating the renewable flag bit.
- Method according to claim 1 to 4, which is characterized in that the method also includes:When determining that the blinkpunkt of user is fallen on third window, activate the third window, and operation is interacted with the third application program for being in front stage operation state on the third window, wherein the third window is any one window at least two window.
- According to the method described in claim 5, it is characterized in that, the activation third window, and operation is interacted with the third application program for being in front stage operation state on the third window, comprising:For the third window, focus variable is set；The window list opened is traversed, the third window with the focus variable is searched from the window list, and the third window is set and obtains focus；According to visibility and focus, alternative events are distributed to the third application program that front stage operation state is on the third window, to activate interactivity to operate.
- A kind of virtual reality device, which is characterized in that the virtual reality device includes:Creation module, for establishing at least two windows, corresponding at least two application programs of at least two window in Virtual Reality interface, at least two window is used to show the display picture of at least two application program；The creation module is also used to establish at least two task stacks, at least two task stack and at least two window correspond, first task stack at least two task stack is used to manage at least one task of the first application program corresponding with first window, any one task at least one described task includes the component for being in front stage operation state, wherein, the first task stack is corresponding with the first window, and the first task stack is any one task stack at least two task stack；Display module, for the component in front stage operation state by being managed at least two task stack, to be in the corresponding display picture of component of front stage operation state described in the display at least two at least two window.
- Virtual reality device according to claim 7, which is characterized in that the first task at least two task includes the first assembly for being in front stage operation state, wherein the first task is any one task at least two task；The virtual reality device further includes determining module, and the determining module is used for:When the first assembly exits, determine the corresponding first task of the first assembly whether be the first task stack a last task；If it is determined that the first task is a last task for the first task stack, the first task stack and the first window corresponding with the first task stack are destroyed；OrIf it is determined that the first task is not a last task for the first task stack, the next task of the first task stack is shown on the first window corresponding to the first task stack.
- Virtual reality device according to claim 7 or 8, it is characterized in that, the creation module is also used to: when detecting that the second application program is activated, image texture is created for the display picture of second application program, second application program is any one application program at least two application program；The display module is specifically used for:By the way that described image texture the second window corresponding with second application program is bound, described image texture is placed on the object model where second window, the display picture of second application program is shown on the object model where second window.
- Virtual reality device according to claim 9, which is characterized in that the display module is also used to:When the display frame updating image of second application program, by the image buffer storage of second application program update into second window, and it is very that the renewable flag bit is used to determine whether to refresh the texture that renewable flag bit, which is arranged,；When the renewable flag bit is true, refresh the texture, and it is false for updating the renewable flag bit.
- Virtual reality device according to any one of claims 7 to 10, which is characterized in that the virtual reality device further includes interactive module, is used for:When determining that the blinkpunkt of user is fallen on third window, activate the third window, and operation is interacted with the third application program for being in front stage operation state on the third window, wherein the third window is any one window at least two window.
- Virtual reality device according to claim 11, which is characterized in that the interactive module is specifically used for:For the third window, focus variable is set；The window list opened is traversed, the third window with the focus variable is searched from the window list, and the third window is set and obtains focus；According to visibility and focus, alternative events are distributed to the third application program that front stage operation state is on the third window, to activate interactivity to operate.
- A kind of computer readable storage medium, which is characterized in that the computer-readable recording medium storage has program, and described program makes computer execute such as method of any of claims 1-6.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2017100056212 | 2017-01-04 | ||
| CN201710005621 | 2017-01-04 | ||
| PCT/CN2017/118907 WO2018126957A1 (en) | 2017-01-04 | 2017-12-27 | Method for displaying virtual reality screen and virtual reality device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110114746A true CN110114746A (en) | 2019-08-09 |
Family
ID=62789202
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201780079800.9A Pending CN110114746A (en) | 2017-01-04 | 2017-12-27 | The method and virtual reality device of display virtual real picture |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110114746A (en) |
| WO (1) | WO2018126957A1 (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110519628A (en) * | 2019-09-20 | 2019-11-29 | 青岛海信移动通信技术股份有限公司 | A kind of picture-in-picture display methods and display equipment |
| CN111061549A (en) * | 2019-12-03 | 2020-04-24 | 西安闻泰电子科技有限公司 | Multitask management method based on android, intelligent terminal and storage medium |
| CN111766945A (en) * | 2020-06-05 | 2020-10-13 | 维沃移动通信有限公司 | Interface display method and device |
| CN114115618A (en) * | 2020-08-28 | 2022-03-01 | 荣耀终端有限公司 | Application window display method and electronic equipment |
| WO2022135195A1 (en) * | 2020-12-25 | 2022-06-30 | 华为技术有限公司 | Method and apparatus for displaying virtual reality interface, device, and readable storage medium |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110347305A (en) * | 2019-05-30 | 2019-10-18 | 华为技术有限公司 | A kind of VR multi-display method and electronic equipment |
| CN110377260B (en) * | 2019-08-29 | 2024-01-05 | 亿咖通(湖北)技术有限公司 | Multi-screen display system and method |
| CN110716780A (en) * | 2019-09-23 | 2020-01-21 | 广州视源电子科技股份有限公司 | Interface display method and device |
| CN110851226A (en) * | 2019-11-13 | 2020-02-28 | 联想(北京)有限公司 | Control method and electronic equipment |
| CN114257852A (en) * | 2020-09-25 | 2022-03-29 | 华为技术有限公司 | Video preview method based on VR scene, electronic equipment and storage medium |
| CN112379808B (en) * | 2020-12-01 | 2023-08-22 | 腾讯科技（深圳）有限公司 | Window control method, device, equipment and medium |
| CN113689231A (en) * | 2021-07-23 | 2021-11-23 | 泰康保险集团股份有限公司 | Service integration method and device, electronic equipment and readable storage medium |
| CN115129444A (en) * | 2022-06-10 | 2022-09-30 | 北京凌宇智控科技有限公司 | Application program display method and device and computer readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105472441A (en) * | 2015-12-31 | 2016-04-06 | 中科创达软件股份有限公司 | Image processing method and device of virtual reality device |
| US20160343315A1 (en) * | 2015-05-18 | 2016-11-24 | Oculus Vr, Llc | Stacked display panels for image enhancement |
| CN106201222A (en) * | 2016-08-09 | 2016-12-07 | 深圳市金立通信设备有限公司 | The display packing of a kind of virtual reality interface and terminal |
| CN106250017A (en) * | 2016-07-26 | 2016-12-21 | 努比亚技术有限公司 | A kind of mobile terminal and multitask management process |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105892630A (en) * | 2015-11-02 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | List content display method and device |
-
2017
- 2017-12-27 WO PCT/CN2017/118907 patent/WO2018126957A1/en active Application Filing
- 2017-12-27 CN CN201780079800.9A patent/CN110114746A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160343315A1 (en) * | 2015-05-18 | 2016-11-24 | Oculus Vr, Llc | Stacked display panels for image enhancement |
| CN105472441A (en) * | 2015-12-31 | 2016-04-06 | 中科创达软件股份有限公司 | Image processing method and device of virtual reality device |
| CN106250017A (en) * | 2016-07-26 | 2016-12-21 | 努比亚技术有限公司 | A kind of mobile terminal and multitask management process |
| CN106201222A (en) * | 2016-08-09 | 2016-12-07 | 深圳市金立通信设备有限公司 | The display packing of a kind of virtual reality interface and terminal |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110519628A (en) * | 2019-09-20 | 2019-11-29 | 青岛海信移动通信技术股份有限公司 | A kind of picture-in-picture display methods and display equipment |
| CN111061549A (en) * | 2019-12-03 | 2020-04-24 | 西安闻泰电子科技有限公司 | Multitask management method based on android, intelligent terminal and storage medium |
| CN111766945A (en) * | 2020-06-05 | 2020-10-13 | 维沃移动通信有限公司 | Interface display method and device |
| CN114115618A (en) * | 2020-08-28 | 2022-03-01 | 荣耀终端有限公司 | Application window display method and electronic equipment |
| WO2022135195A1 (en) * | 2020-12-25 | 2022-06-30 | 华为技术有限公司 | Method and apparatus for displaying virtual reality interface, device, and readable storage medium |
| CN114691248A (en) * | 2020-12-25 | 2022-07-01 | 华为技术有限公司 | Method, device and equipment for displaying virtual reality interface and readable storage medium |
| CN114691248B (en) * | 2020-12-25 | 2024-04-12 | 华为技术有限公司 | Method, device, equipment and readable storage medium for displaying virtual reality interface |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018126957A1 (en) | 2018-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110114746A (en) | The method and virtual reality device of display virtual real picture | |
| JP7476292B2 (en) | Method and system for managing and displaying virtual content in a mixed reality system - Patents.com | |
| US20220318035A1 (en) | Computer-generated reality platform for generating computer-generated reality environments | |
| US10957107B2 (en) | Snapping, virtual inking, and accessibility in augmented reality | |
| US20220165036A1 (en) | Extended reality iot device management | |
| US9594603B2 (en) | Application-to-application launch windowing | |
| US20240143350A1 (en) | Rules Based User Interface Generation | |
| US20230221830A1 (en) | User interface modes for three-dimensional display | |
| US20230102820A1 (en) | Parallel renderers for electronic devices | |
| KR20230137936A (en) | Adaptable personal user interface in cross-application virtual reality settings | |
| CN114816597A (en) | Content display method and device and computer readable storage medium | |
| CN112684965A (en) | Dynamic wallpaper state changing method and device, electronic equipment and storage medium | |
| US11972088B2 (en) | Scene information access for electronic device applications | |
| US20220101002A1 (en) | Real-world object inclusion in a virtual reality experience | |
| KR20230171147A (en) | Method, device and system for creating presenter-participating 3d content | |
| CN117632391A (en) | Application control method, device, equipment and medium based on virtual reality space | |
| CN116347144A (en) | Special effect rendering method, special effect rendering device, storage medium and equipment | |
| CN115994850A (en) | Special effect processing method and equipment | |
| WO2024085996A1 (en) | Overlay on an artificial reality environment | |
| US9159160B1 (en) | Texture sharing between application modules | |
| CN116467535A (en) | Page display method, page display device, storage medium, computer device, and program product | |
| CN117999532A (en) | Parallel renderer for electronic device | |
| CN117762280A (en) | Man-machine interaction method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190809 |
|
| WD01 | Invention patent application deemed withdrawn after publication |