WO2020020102A1 - 生成虚拟内容的方法、终端设备及存储介质 - Google Patents
生成虚拟内容的方法、终端设备及存储介质 Download PDFInfo
- Publication number
- WO2020020102A1 WO2020020102A1 PCT/CN2019/097128 CN2019097128W WO2020020102A1 WO 2020020102 A1 WO2020020102 A1 WO 2020020102A1 CN 2019097128 W CN2019097128 W CN 2019097128W WO 2020020102 A1 WO2020020102 A1 WO 2020020102A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- markers
- marker
- spatial position
- image
- position relationship
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/163—Wearable computers, e.g. on a belt
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
- G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
- G06F1/1686—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being an integrated camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/0304—Detection arrangements using opto-electronic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/0304—Detection arrangements using opto-electronic means
- G06F3/0325—Detection arrangements using opto-electronic means using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Definitions
- the present application relates to the field of augmented reality technology, and more particularly, to a method, a terminal device, and a storage medium for generating virtual content.
- Augmented Reality is a technology that increases the user's perception of the real world through information provided by a computer system. It superimposes content objects such as computer-generated virtual objects, scenes, or system prompt information into real scenes to enhance or modify them. Perception of the real world environment or data representing the real world environment.
- the terminal device collects an image of the marker through the camera, recognizes the image, and displays the virtual content according to the recognition result of the marker.
- the identification and tracking of the marker will be affected by the visual range of the camera, which will cause the terminal device to easily lose track of the marker, and the display of the terminal device will be affected.
- the embodiments of the present application provide a method for generating virtual content, a terminal device, and a storage medium, which can enhance the display stability of the terminal device.
- an embodiment of the present application provides a method for generating virtual content, which is applied to a terminal device.
- the method includes: collecting an image including a target marker, where the target marker is in a plurality of markers. At least one marker within the field of view of the terminal device, and the plurality of markers are separately disposed at different positions; obtaining a target spatial position of the target marker relative to the terminal device according to the image; and reading the stored A first spatial position relationship between the plurality of markers, and obtaining a second spatial position relationship of the virtual object relative to the plurality of markers; based on the first spatial position relationship, the second spatial position relationship, and The target space position determines a display position of the virtual object; and generates a virtual object according to the display position.
- an embodiment of the present application provides a data update method, including: collecting an image including a marker; identifying the marker in the image, and determining identity information of the marker; and acquiring the identity information according to the identity information A binding data replacement instruction corresponding to the marker; replacing the first data bound by the marker with second data based on the binding data replacement instruction; and generating a virtual corresponding to the second data image.
- an embodiment of the present application provides an image acquisition method applied to a terminal device.
- the method includes: acquiring posture information of the terminal device; generating an acquisition control instruction based on the posture information, and A control instruction is sent to an image acquisition device on the vehicle, wherein the acquisition control instruction is used to instruct the image acquisition device to acquire a scene image of the surrounding scene of the vehicle with an orientation angle that matches the posture information; and Receiving a scene image collected by the image acquisition device, and generating display content according to the scene image.
- an embodiment of the present application provides a terminal device including a memory and a processor.
- the memory stores a computer program.
- the processor is implemented as described above. The method described.
- an embodiment of the present application provides a computer storage medium on which a computer program is stored.
- the processor When the computer program is run by a processor, the processor enables the processor to implement the method described above.
- FIG. 1 is a scene diagram of an augmented reality system provided by an embodiment of the present application.
- FIG. 2 is a schematic framework diagram of a terminal device according to an embodiment of the present application.
- FIG. 3 is a flowchart of a method for generating virtual content according to an embodiment of the present application.
- FIG. 4 is a flowchart of a method for generating virtual content according to another embodiment of the present application.
- FIG. 5 is a scene diagram of an embodiment provided by the embodiment of the present application.
- FIGS 6a-6d are scene diagrams of another implementation manner provided by an embodiment of the present application.
- FIGS. 7a-7b are schematic diagrams of a usage scenario in an optional manner in another implementation manner provided by an embodiment of the present application.
- FIG. 8 is a flowchart of determining a display position of a virtual object according to an embodiment.
- 9a and 9b are schematic diagrams of another augmented reality system provided by the present application.
- FIG. 10 is a flowchart of a data updating method according to an embodiment of the present application.
- FIG. 11 is a schematic diagram of another augmented reality system provided by the present application.
- FIG. 12 is a flowchart of an image acquisition method according to an embodiment of the present application.
- an augmented reality system 10 provided by an embodiment of the present application includes a terminal device 100 and a plurality of markers 200.
- the plurality of markers 200 may be separately disposed at different positions in the scene.
- the plurality of markers 200 may be separately disposed on a desktop or in an area on the ground.
- the actual setting manner of the plurality of markers is not limited.
- the terminal device 100 may be a head-mounted display device, or may be a mobile device such as a mobile phone or a tablet.
- the terminal device 100 is a head-mounted display device, the head-mounted display device may be an integrated head-mounted display device.
- the terminal device 100 may also be a head-mounted display device to which an external electronic device is connected.
- the terminal device 100 may also be a smart terminal such as a mobile phone connected to an external / plug-in head-mounted display device, that is, the terminal device 100 may serve as a processing and storage device of the head-mounted display device, and plug in or access the external head-mounted display device. To display a virtual object in a head-mounted display device.
- the terminal device 100 is a head-mounted display device, and includes a processor 110, a memory 120, and one or more application programs.
- One or more application programs may be stored in the memory 120 and stored in the memory 120. It is configured to be executed by one or more processors 110, and one or more programs are used to perform the methods described in this application.
- the processor 110 may include one or more processing cores.
- the processor 110 uses various interfaces and lines to connect various parts in the entire electronic device 100, and executes or executes instructions, programs, code sets, or instruction sets stored in the memory 120 by calling or executing data stored in the memory 120 to execute Various functions and processing data of the electronic device 100.
- the processor 110 may use at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). achieve.
- DSP Digital Signal Processing
- FPGA Field-Programmable Gate Array
- PDA Programmable Logic Array
- the processor 110 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem.
- the CPU mainly handles the operating system, user interface, and application programs; the GPU is responsible for rendering and rendering of the displayed content; the modem is used for wireless communication.
- the modem may not be integrated into the processor 110, and may be implemented by a communication chip alone.
- the memory 120 may include Random Access Memory (RAM), and may also include Read-Only Memory.
- RAM Random Access Memory
- the memory 120 may be used to store instructions, programs, codes, code sets, or instruction sets.
- the memory 120 may include a storage program area and a storage data area, where the storage program area may store instructions for implementing an operating system and instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , Instructions for implementing the following method embodiments, and the like.
- the storage data area may also store data and the like created by the terminal device 100 in use.
- the terminal device 100 may further include a camera 130 for capturing an image of a real object and a scene image of a target scene.
- the camera 130 may be an infrared camera or a visible light camera, and the specific type is not limited.
- the terminal device may further include one or more of the following components: a display module, an optical module, a communication module, and a power source.
- the display module may include a display control unit.
- the display control unit is configured to receive a display image of the virtual content rendered by the processor, display the display image on the optical module, and enable the user to view the virtual content through the optical module.
- the display module may be a display screen or a projection device, and is used to display an image.
- the optical module can use an off-axis optical system or a waveguide optical system. After the display module displays the display image, the image can be projected to the user's eyes. The user can see the display image projected by the display module through the optical module.
- the user can also observe the real environment through the optical module, and experience the visual effect of the virtual content superimposed on the real environment.
- the communication module can be a module such as Bluetooth, WiFi (Wireless-Fidelity, Wireless Fidelity), ZigBee (Zifeng Technology), and the terminal device can communicate with the interactive device through the communication module to perform information and instruction interaction.
- the power supply can supply power to the entire terminal equipment to ensure the normal operation of each component of the terminal equipment.
- the camera 130 may acquire an image of one of the markers within the visual range.
- the terminal device 100 can identify the image, and obtain the identification results such as the position and orientation of the marker included in the image with respect to the terminal device, and the identity information of the marker.
- the terminal device may display the virtual object based on the spatial position information of the identified marker relative to the terminal device and the spatial position relationship between the plurality of markers 200 described above.
- the marker 200 may have a pattern of a topological structure, and the topological structure refers to a connection relationship between a sub-marker and a feature point in the marker.
- the specific marker is not limited in this application, and can be identified and tracked by the terminal device.
- an embodiment of the present application provides a method for generating virtual content, which is applied to the foregoing terminal device 100.
- the method may include:
- Step S310 Acquire an image including a target marker, where the target marker is at least one marker within a visual range of the camera among the plurality of markers, and the plurality of markers are separately disposed at different positions.
- the positions of each marker can be fixed or can be changed at any time according to requirements.
- the camera may acquire images of at least one marker in the visual range.
- Step S320 Obtain a target spatial position of the target marker relative to the terminal device according to the image.
- the terminal device can identify the marker contained in the collected image to obtain the target spatial position of the marker in the image relative to the terminal device, including position and attitude information, and the identity information of the marker.
- the posture information is the orientation and rotation angle of the target marker relative to the terminal device, and different markers correspond to different identity information.
- Step S330 Read a first spatial position relationship between the plurality of stored markers, and obtain a second spatial position relationship of the virtual object relative to the plurality of markers.
- the terminal device can collect one or more images in advance through the camera.
- the image contains various markers separately located at different positions. Each marker is identified based on the image, and the relative spatial position relationship between different markers is established. And stored in the terminal device.
- the positions of multiple markers are relatively fixed, and the terminal device only needs to obtain the relative spatial position relationship between each marker once and store it in the memory, without the need to repeatedly obtain the relative spatial position between each marker relationship.
- the terminal device only needs to obtain the relative spatial position relationship between each marker once and store it in the memory, without the need to repeatedly obtain the relative spatial position between each marker relationship.
- different markers are fixedly set at different positions, and the positions of the markers can be stored in the terminal device in advance.
- Information such as the position and posture of other markers can be obtained according to the relative spatial position relationship between the markers stored in advance, and corresponding virtual objects are displayed.
- the terminal device can obtain the spatial position relationship of the virtual object with respect to multiple markers, and the spatial position relationship can be set according to actual requirements.
- the virtual object can be located at the center of an area composed of multiple markers, or the virtual object is located in the virtual object.
- the location of a marker, etc. is not limited thereto.
- the spatial position relationship of the position that the virtual object needs to display relative to the plurality of markers can be stored in the terminal device in advance, or can be selected by the user in real time.
- Step S340 Determine a display position of the virtual object based on the first spatial position relationship, the second spatial position relationship, and the target spatial position.
- the terminal device can determine the positional relationship of the virtual object with respect to the target marker according to the positional relationship of the virtual object with respect to multiple markers and the relative spatial positional relationship between the markers, and then based on the target spatial position of the target marker with respect to the terminal device. Using the target marker as a reference, obtain the spatial position of the virtual object relative to the terminal device. The terminal device can perform coordinate conversion on the spatial position of the virtual object relative to the terminal device to obtain the rendering coordinates of the virtual object in the virtual space of the terminal device, and use it as a display position of the virtual object to display the virtual object.
- Step S350 Display the virtual object according to the display position.
- the terminal device can obtain the data of the virtual object, and render the virtual object at the corresponding rendering coordinates according to the data of the virtual object, generate a virtual screen, and display the virtual object.
- the data of the virtual object may be model data of the virtual object, and the user may observe that the virtual object is superimposed and displayed in the real space through the terminal device.
- a plurality of different markers are separately set at a plurality of different positions.
- the terminal device recognizes any one of the markers, it can determine the position of other markers that are not in the visual range and obtain the display of the virtual object. Position, to realize the identification and tracking of a wide range of markers, and improve the stability of the display of virtual objects by terminal devices.
- This method does not require multiple markers to exist in the visual range at the same time, and can accurately obtain the display position of the virtual object, and solves the problem of the range limitation of marker recognition and tracking.
- another embodiment of the present application provides a method for generating virtual content, which is applied to a terminal device.
- the method may include:
- Step S410 Obtain a first spatial position relationship between a plurality of markers.
- the terminal device may perform image acquisition on the markers to obtain an image including all the markers.
- the terminal device can identify and track each marker contained in the acquired image, and obtain the spatial position and identity information of each marker relative to the terminal device to establish a relative spatial position relationship between different markers.
- the terminal device can sequentially obtain the spatial position of each marker relative to other markers according to the spatial position of each marker relative to the terminal device and the identity information of each marker, so as to obtain each Relative spatial position relationship between markers. For example, as shown in FIG. 5, a total of markers A, B, C, D, and E are set in the scene, and the terminal device can acquire an image including markers A, B, C, D, and E at the same time, and then obtain each marker. Spatial position relative to the terminal device; according to the spatial position of each marker relative to the terminal device, the relative spatial positions between the marker A, marker B, marker C, marker D, and marker E can be obtained separately relationship.
- the terminal device may also sequentially obtain the spatial position relationship between the markers disposed at adjacent positions according to the spatial position of each marker relative to the terminal device and the identity information of each marker, Thereby, the relative spatial position relationship between each marker is obtained.
- the marker includes a marker A, a marker B, a marker C, a marker D, and a marker E.
- the marker A is disposed adjacent to the marker B
- the marker B is disposed adjacent to the marker C
- the marker C Set adjacent to the marker D
- the marker D and the marker E are adjacent to the device, and can obtain the marker A and the marker B, the marker B and the marker C, the marker C and the marker D, and the marker D, respectively.
- the spatial positional relationship with the marker E is used to obtain the relative positional relationship between the markers.
- the adjacent relationship between the markers is not limited to the above example, it only needs to satisfy the space position relationship between the obtained at least four adjacent relationships, and the space between the obtained adjacent labels. It is sufficient that no island exists in the position relationship, that is, there is no separate spatial position relationship, that is, there is no existence between adjacent markers corresponding to the separate spatial position relationship and adjacent markers corresponding to other spatial position relationships. The same markers.
- the terminal device may collect images of the markers multiple times, and separately identify and track the images acquired multiple times, to obtain a spatial position relationship between different markers contained in each image, and then obtain each marker.
- Relative spatial position relationship between When the terminal device collects each marker separated in different positions multiple times, the image acquired each time includes at least two markers, and each marker appears in at least two acquired images to ensure each image obtained.
- There are no islands in the spatial position relationship of the markers that is, there is no separate spatial position relationship in the multiple spatial position relationships, that is, between the marker corresponding to the separate spatial position relationship and the markers corresponding to other spatial position relationships, There are no identical markers. That is, at least one identical marker can be included in the images acquired two adjacent times.
- the marker includes a marker A, a marker B, a marker C, a marker D, and a marker E.
- the terminal device may acquire an image containing the markers A and B.
- the first The secondary terminal device can collect images containing markers B and C
- the third terminal device can capture images containing markers C and D
- the fourth terminal device can capture images containing markers D and E, of which the first The image acquired in the second acquisition contains the same marker B as the image acquired in the second acquisition
- the image acquired in the second acquisition and the image acquired in the third includes the same marker C
- the image acquired in the third acquisition is the same as the fourth acquisition.
- the image contains the same marker D.
- the terminal device may select the same marker included in different images as a reference based on the spatial position relationship between the markers in each image, so as to obtain a first spatial position relationship between the markers.
- the markers include markers A, B, and C. Recognizing the first image containing markers A and B, the spatial positions of markers A and B relative to the terminal device can be obtained respectively, and then obtained The spatial position relationship between the marker A and the marker B; the second acquisition of the image containing the markers B and C can obtain the spatial position of the marker B and the marker C relative to the terminal device, and then obtain The spatial positional relationship between the marker B and the marker C; according to the spatial positional relationship between the marker A and the marker B, and the spatial positional relationship between the marker B and the marker C, the marker A, Relative spatial position relationship between the marker B and the marker C.
- the distribution range of the plurality of markers may be large. Due to the limited visual range of the camera, it is not possible to acquire images containing all the markers at one time. The images are collected for the multiple markers multiple times, and the relative spatial position relationship between the markers is established based on the multiple captured marker images. For example, a plurality of markers may be divided into a first portion of markers and a second portion of markers, wherein the first portion of markers includes at least two markers and the second portion of markers includes at least one marker in the first portion of markers And all other markers except the first part.
- the terminal device collects an image containing the first part of the marker, and can identify the image to obtain the spatial position and identity information of each marker in the first part of the marker relative to the terminal device, and then obtain the information between each marker in the first part of the marker. Spatial location relationship; the terminal device collects the image containing the second part of the marker, and can identify the image to obtain the spatial position and identity information of each marker in the second part of the marker relative to the terminal device, and then obtain the second part of the marker. The spatial relationship between the various markers.
- the first part of the marker is the same as the second part of the marker as a reference, according to the spatial position relationship between the first part of the marker And the spatial position relationship between the markers in the second part, the relative spatial position relationship between the markers can be obtained.
- markers A, B, C, D, and E are set in the scene.
- the terminal device 100 can collect an image of the first part of the marker including the markers A, B, and C, and identify Track the image of the first part of the marker to obtain the spatial position relationship between the markers A, B, and C; the terminal device 100 can also collect the image of the second part of the markers of the markers B, D, and E, and identify and track the second Partial marker images are used to obtain the spatial positional relationship between markers B, D, and E; using marker B as a reference, the spatial positional relationship between markers A, B, C, D, and E is obtained.
- the spatial position relationship between the first part of the markers according to the image of the first part of the markers, and the spatial position relationship between the second part of the markers according to the image of the second part of the markers can be referred to in the previous embodiment
- the solution of this method obtains the spatial position relationship between the markers in the image, which is not described in detail here.
- the terminal device may perform image acquisition on only one marker at a time, and obtain a spatial position relationship between multiple markers according to the posture change information of the terminal device. Specifically, an image containing the marker Xn is collected for the nth time, and the position and attitude information of the marker Xn relative to the terminal device is obtained by identifying and tracking the image containing the marker Xn. The n + 1th time is collected for the image containing the marker Xn + 1.
- the 6DOF information may include information such as the position and attitude of the terminal device, and then the position and attitude change information of the n + 1th time when the terminal device acquires the image relative to the nth time when the image is acquired; according to the terminal
- the position and posture change information of the device when acquiring the image for the n + 1th time, and the position of Xn for the terminal device, and the position of Xn + 1 for the terminal device can obtain the marker Xn and the marker Xn +
- the spatial position relationship between 1 is obtained by performing the above operations on all the marker images to obtain the first spatial position relationship between multiple markers.
- the relative spatial position relationship may be stored in a memory and used to determine a display position of the virtual object.
- Step S420 Collect an image containing a target marker, where the target marker is at least one marker within a field of view of the terminal device among the plurality of markers, and the plurality of markers are separately set at different positions.
- Step S430 Obtain a target spatial position of the target marker relative to the terminal device according to the image.
- Step S440 Read a first spatial position relationship between the plurality of stored markers, and obtain a second spatial position relationship of the virtual object relative to the plurality of markers.
- the terminal device can acquire a spatial position relationship between a position where the virtual object needs to be displayed in a superimposed manner in the real space and a setting position of a plurality of markers.
- the position of the virtual object is related to a spatial area composed of multiple markers, which may be a part of the spatial area, or may coincide with the spatial area, and may also cover the spatial area, etc .
- the space region formed by the plurality of markers may be the largest area surrounded by the plurality of markers, and the space region may include the plurality of markers.
- the virtual object is a virtual chessboard, which coincides with a space area composed of a plurality of markers. The user can observe through the terminal device that the virtual chessboard is superimposed and displayed on the space area composed of the plurality of markers.
- the virtual object may also have a spatial position relationship with the center position of a spatial region composed of multiple markers.
- the position of the virtual object may coincide with the center position of the space region, or the virtual object may be at a specified position corresponding to the center position.
- the virtual object is a virtual decoration, and the corresponding display position coincides with the center position of the space area composed of multiple markers. The user can observe through the terminal device that the virtual decoration is superimposed and displayed on the center of the space area composed of multiple markers. Location.
- the virtual object may also have a spatial position relationship with a preset marker in the plurality of markers, and the preset marker is a selected one of the plurality of markers.
- the display position of the virtual object may coincide with the position of the preset marker, or may be at a designated position corresponding to the preset marker.
- the virtual object may have a spatial position relationship with the target marker within the visual range of the camera. For example, the display position of the virtual object may coincide with the position of the target marker, or it may be at a specified position corresponding to the target marker.
- Step S450 Determine a display position of the virtual object based on the first spatial position relationship, the second spatial position relationship, and the target spatial position.
- the display position of the virtual object coincides with the position of the target marker within the visual range of the camera, and is directly determined by the spatial position of the target marker relative to the terminal device.
- the spatial position of the target marker relative to the terminal device may be determined.
- the coordinate conversion obtains the position coordinates of the target marker in the virtual space relative to the terminal device, and the position coordinates are used as the display position of the virtual object in the virtual space.
- Step S450 may include:
- Step S851 Based on the first spatial position relationship, obtain the center position of the area formed by the plurality of markers.
- the preset marker is a marker close to a center position of a space region formed by the plurality of markers
- the terminal device may obtain a plurality of markers based on a relative spatial position relationship between the plurality of markers. The center position of the spatial area. Identify the marker near the center.
- Step S852 Obtain a preset marker from a plurality of markers based on the center position, and the distance of the preset marker from the center position is smaller than the distance of other markers from the center position.
- the terminal device may separately calculate the distance of each marker from the center position of the spatial area, and obtain the marker with the shortest distance from the center position as a preset marker.
- Step S853 Determine the display position of the virtual object according to the target spatial position, the second spatial position relationship, and the spatial position relationship between the preset marker and the target marker.
- the terminal device can obtain the spatial position relationship between the preset marker and the target marker according to the spatial position relationship between multiple markers; according to the spatial position relationship between the virtual object and the preset marker, and the preset marker The spatial position relationship between the object and the target marker to obtain the spatial position relationship between the virtual object and the target marker; and according to the target spatial position of the target marker relative to the terminal device, and the space between the virtual object and the target marker For the position relationship, using the target marker as a reference, the spatial position of the virtual object relative to the terminal device is obtained, that is, the display position of the virtual object in the virtual space is obtained.
- Step S460 Display the virtual object according to the display position.
- the terminal device may display a plurality of virtual objects, and the positions of the plurality of virtual objects are one-to-one corresponding to the plurality of markers and overlap with each other.
- the terminal device can obtain the spatial position of each marker relative to the terminal device according to the spatial position of the target marker relative to the terminal device within the field of view of the camera and the relative spatial position relationship between the markers, and compare each marker with each other. Coordinate conversion is performed on the spatial position of the terminal device to obtain the position coordinates of each marker in the virtual space relative to the terminal device, and the virtual object corresponding to each marker can be displayed at the corresponding position.
- the virtual object in an augmented reality scene of a virtual dining table, includes a plurality of different types of virtual dishes, and each marker corresponds to a type of virtual dish.
- the terminal device recognizes the marker in the visual range, and according to the marker and The spatial position relationship of other markers can superimpose and display each virtual dish on the corresponding marker.
- Step S470 retrieve the first spatial position relationship between the plurality of markers based on the position update instruction, and store the newly acquired first spatial position relationship.
- the relative spatial position relationship between multiple markers may change, such as when the markers are moved, and the terminal device may obtain a position update instruction according to the user's operation to update the relative position between each marker. Spatial location relationship.
- re-acquiring the first spatial position relationship between the plurality of markers and storing the re-acquired first spatial position relationship may include steps (a) and (b).
- the terminal device can perform image acquisition on at least one marker in which the position changes and other markers under the operation of the user, and determine the space between the marker in which the position changes and the position in which the relative position does not change according to the acquired image. Positional relationship.
- the terminal device can identify the acquired image, and obtain the spatial position and identity of the marker whose position has changed relative to the terminal device, and the spatial position and identity of the marker whose position has not changed in the image relative to the terminal device;
- the positions of the markers in the unchanged position are respectively relative to the spatial position of the terminal device, and the spatial position relationship between the markers in the changed position and the markers in the unchanged position is obtained; using the markers in the unchanged position as a reference, a plurality of previously stored
- the spatial position relationship between the markers is used to obtain the relative spatial position relationship between the markers after the position changes.
- the terminal device may perform image acquisition on multiple markers in multiple times to detect whether the positions of the markers have changed.
- detecting whether the position has changed may be performed automatically when the user is performing an augmented reality experience, and the user does not need to intentionally detect.
- a virtual object can be superimposed and displayed at a corresponding position, thereby improving the interaction of augmented reality.
- An augmented reality system 90 provided in another embodiment of the present application includes a terminal device 91 and a marker 92.
- the terminal device 91 obtains the relative value of the terminal device 91 by tracking the collected marker image. The position and rotation relationship of the marker 92, and the corresponding virtual object is displayed.
- the terminal device 91 may also communicate with the server 93 through a network.
- the terminal device 91 runs an AR / VR application client, and the server 93 runs an AR / VR application server corresponding to the client. As one way, multiple terminal devices 91 can perform data synchronization through the server 93.
- a data updating method provided by an embodiment of the present application. This method can update data bound by a marker without replacing a new marker, and displaying different virtual images through the same marker.
- the data updating method which is applied to a terminal device 91 of an augmented reality system, includes the following steps:
- Step S1001 Acquire an image containing a marker.
- Step S1002 identify the marker in the image, and determine the identity information of the marker.
- the marker may be any graphic or object having a recognizable characteristic mark
- the identity information corresponding to the marker may be an identity code represented by one or more of numbers, letters, and symbols, etc.
- the topological structure of the pattern in the marker is set, and different markers correspond to different identity information.
- Step S1003 Obtain a replacement instruction of the binding data corresponding to the marker according to the identity information.
- the identity information of the marker corresponds to the bound virtual image data. According to the determined identity information of the marker, a replacement instruction matching the marker can be obtained, and the instruction is used to change the virtual image data bound to the marker.
- the virtual image data bound by the marker may be set through a client of the terminal device.
- the terminal device collects and recognizes the code of the marker as "No.7”, and the user can change the binding data of the marker of "No.7” on the client (APP application) to generate a "No.7”
- the replacement instruction corresponding to the “7” marker replaces the virtual image data bound to the “No.7” marker stored locally.
- the virtual image data bound by the marker can also be set through the server, and the client of the terminal device can obtain it from the server to achieve data synchronization.
- the terminal device collects and recognizes the code of the marker as "No.7”, and can send the code "No.7” to the server, and the server judges whether the binding data needs to be changed.
- the server feedbacks that it needs to be replaced At this time, a replacement instruction corresponding to the marker can be generated, and the virtual image data bound to the “No. 7” marker stored locally can be replaced.
- the server may store identity information of multiple markers and binding data corresponding to each marker.
- the client of the terminal device can first confirm with the server whether the binding data corresponding to the identity information of the marker has been replaced in the server, and then decide whether to generate a replacement instruction for the marker according to the result fed back by the server.
- the client can send a binding information search request corresponding to the identity information of the marker to the server. After receiving the binding information search request, the server feeds back the binding data replacement result corresponding to the identity information to the client.
- the binding information corresponding to the marker may include the virtual image data to which the identity information is currently bound, and the binding data replacement result corresponding to the identity information.
- the binding data replacement result is used to indicate Whether the binding data corresponding to the identity information was changed when the client last accessed the server. After receiving the replacement result of the binding data corresponding to the identity information issued by the server, the client can determine whether the binding data corresponding to the identity information has been changed according to the binding data replacement result.
- the terminal device may obtain the data identifier bound to the identity information corresponding to the marker in the server, compare it with the data identifier corresponding to the marker in the terminal device, and determine whether they are consistent to generate the corresponding binding.
- Fixed data replacement results may be an indicator used to indicate whether the binding data corresponding to the marker has been replaced on the server side. For example, when the binding data is replaced, the binding data replacement result is "1. ", If the binding data has not been replaced, the binding data replacement result is" 0 ".
- a replacement instruction for the marker may be generated, which is used to instruct the terminal device to update the data bound to the marker.
- the terminal device may change the binding data of the marker through the client and synchronize the data to the server.
- the user can enter the identity information of the marker whose binding data needs to be replaced in the terminal device, and transmit the data that needs to be rebind.
- the transmission method can be transmitted through other terminals, and searched from the Internet.
- the terminal device can obtain the identity information of the marker that needs to be replaced with the binding data and the data that needs to be replaced, and sends the identity information and the replaced data to the server, and the server replaces the binding data of the corresponding marker according to the identity information. .
- Step S1004 Replace the data bound by the marker based on the replacement instruction of the bound data.
- the terminal device can obtain the data that needs to be bound again according to the replacement instruction, and will replace the originally bound data.
- the "No.2" tag corresponds to the virtual image data of the "vase”
- the terminal device obtains the replacement instruction
- the "No.2" tag can correspond to the virtual image of the "vase” that is bound to The data is replaced with "cow” virtual image data.
- the first data bound before the marker replaces the corresponding data may be stored locally on the client (for example, the memory of the terminal device), and the second data to be changed and bound may also be stored locally on the client.
- the second data that needs to be replaced can also be obtained by the client from the server.
- the terminal device can find the corresponding second data in the server and download the second data to the local storage to update the tag.
- the corresponding bound data can be first search for the corresponding second data locally, and if the corresponding second data is not stored locally, then obtain it through the server.
- the second data that needs to be replaced can also be sent by a terminal to the server and stored by the server, and finally synchronized to each client.
- the identity information of the marker is changed on the server
- the corresponding binding data is changed.
- Clients accessing the server can synchronize and update the binding data stored locally on the marker.
- all clients (terminals) accessing the server can synchronize data.
- Step S1005 Display the virtual content corresponding to the changed bound data.
- the terminal device After the terminal device binds the marker to the updated data, a corresponding virtual image can be generated for display.
- the terminal device may display the process of replacing the bound data, first generating and displaying the first virtual image according to the originally bound data, and generating and displaying the second virtual image after the marker is bound to the updated data, In order to present the transformation process from the first virtual image to the second virtual image.
- the "No.3" tag originally corresponds to the virtual image data of the "bull"
- the replacement of the "poster” virtual image data is completed
- the binding data corresponding to the "No.3” tag is completed
- the user can observe that the display content is changed from the "virtual" image to the "poster” virtual image through the terminal device, giving the user a visual sense of dynamic transformation.
- the dynamic transformation can be gradual, abrupt, etc. Set according to actual needs.
- the replacement of the binding data may also be performed in a state where the screen is turned off. After the replacement of the binding data is completed, a virtual image corresponding to the replacement data is displayed.
- the data updating method provided in this embodiment can replace the virtual image data bound to the same marker without replacing the marker, and can be applied to many virtual reality or augmented reality scenes.
- the number and location of markers are usually fixed.
- the virtual image data bound to the same marker on the booth can be replaced to
- the virtual images of various exhibits are displayed at different time periods to adapt to various changing exhibition themes (such as exhibitions of ancient cultural relics, science and technology, art exhibitions, etc.).
- the teacher's lecture content is different every day. At this time, the virtual image data of the relevant lecture notes are bound to the markers. When the lectures are replaced, the corresponding markers need not be replaced.
- the virtual image data of the lecture notes bound by the client or server can be realized, improving the practicality of the VR / AR multimedia classroom and the efficiency of lectures.
- the data updating method of the present application simply replacing the old advertising content bound by the marker with new advertising content each time, different advertising content can be displayed at different time periods, reducing frequent The cost of replacing the markers, and the long-term uninterrupted continuous display of different advertising content, improving the commercial efficiency of advertising.
- the above examples are just some practical applications of the data update method of this application. With the further development and popularization of VR / AR technology, the data update method of this application can play a role in more practical application scenarios.
- the terminal device may also provide the user with an option to set a binding data replacement period, such as displaying an option for setting a binding data replacement period on a human-computer interaction interface.
- the binding data replacement period may be sent to the server, and the server may periodically change the binding data of the marker according to the binding data replacement period.
- the binding data replacement cycle can be set for multiple markers at the same time, or it can be set for only one marker; the changed binding data can be stored in advance in a group of virtual image data in the server, according to each virtual image The data numbers are replaced sequentially, or randomly selected for replacement.
- the terminal device can also obtain the access duration of the marker within a predetermined time period from the server.
- the access time of the marker within the preset time period, but the length of time during which the marker is collected by the terminal device (which may be multiple different terminal devices) and display the corresponding virtual image within the preset time period.
- the length of time during which the marker is captured and the corresponding virtual image is less than a preset value is indicated, the total time that the virtual image corresponding to the marker is viewed in the preset time period is less. To some extent, the virtual image bound by the marker does not attract viewers.
- a replacement request may be sent to the server, so that the server can replace the virtual image data bound by the marker.
- each time the client collects a marker and displays the corresponding virtual image it generates a log associated with the visit duration of the marker and uploads the log to the server.
- the server can record the data sent by each terminal. The sum of the visit duration of the marker in the log and correlates the date of the visit to obtain the total visit duration of the marker within a preset time period.
- an augmented reality system 1100 provided by another embodiment of the present application includes a terminal device 1110, an image acquisition device 1120, and a carrier 1130.
- the image acquisition device 1120 is disposed on the carrier 1130.
- the image acquisition device 1120 can be installed on the top position of the carrier 1130 through a rotating mechanical structure, which can be rotated under control to drive the image acquisition device 1120 to rotate relative to the carrier 1130, and change the image acquisition device 1120 attitude relative to vehicle 1130.
- the position of the image acquisition device 1120 on the carrier 1130 is not limited.
- the position of the image acquisition device 1120 on the carrier 1130 may be provided.
- the terminal device 1110 and the image acquisition device 1120 may establish a communication connection.
- the communication connection is established through wifi, GPRS, 4G, 5G, and the like, which is not limited in particular.
- the image acquisition device 1120 is configured to collect an image of the scene around the vehicle and send it to the terminal device 1110.
- the image acquisition device 1120 may be a color camera, an infrared camera, and the like, and the specific type is not limited.
- an image acquisition method provided by an embodiment of the present application is applied to a terminal device 1110.
- the method includes:
- Step S1210 Acquire attitude information of the terminal device.
- a posture sensor may be provided in the terminal device to detect posture information of the terminal device.
- the attitude sensor is a three-dimensional motion attitude measurement system based on MEMS technology; the attitude sensor may include motion sensors such as a three-axis gyroscope, a three-axis accelerometer, and a three-axis electronic compass, and the attitude information of the terminal device is obtained through an embedded low-power processor. .
- the attitude information of the terminal device includes the orientation and rotation angle of the terminal device.
- the terminal device may also obtain the attitude information of the terminal device through a built-in IMU (Inertial Measurement Unit), which is not limited in particular.
- IMU Inertial Measurement Unit
- Step S1220 Generate an acquisition control instruction based on the attitude information, and send the acquisition control instruction to the image acquisition device on the vehicle, where the acquisition control instruction is used to instruct the image acquisition device to acquire the surroundings of the vehicle with the orientation angle matching the posture information Scene image of the scene.
- the image acquisition device is controlled by the posture information of the terminal device, so that the image acquisition device acquires an image in a required viewing direction.
- the terminal device may obtain the angle and direction that the image acquisition device needs to rotate according to the acquired orientation and rotation angle.
- the image acquisition device has an initial lens orientation state, and a preset posture information may be set in advance, where the preset posture information of the terminal device corresponds to the initial lens orientation state of the image acquisition device.
- the posture information acquired by the terminal device is preset posture information
- the image acquisition device does not need to adjust the lens direction, and the acquisition control instruction generated will instruct the image acquisition device not to perform rotation adjustment; the posture information acquired at the terminal device does not match the preset posture information.
- the terminal device can determine the direction and angle that the image acquisition device needs to rotate according to the difference between the acquired posture information and the preset posture information, and the generated acquisition control instruction instructs the image acquisition device to rotate according to the above-mentioned direction and angle of rotation. Orient the lens of the image acquisition device to correspond to the posture information of the terminal device.
- the image acquisition device adjusts the lens orientation according to the acquisition control instruction, so that the lens orientation matches the posture information of the terminal device, so that the image acquisition device performs image acquisition of the scene around the vehicle
- the viewing direction of the device matches the attitude information of the terminal device, and the viewing direction of the image acquisition device is controlled by the attitude information of the terminal device.
- Step S1230 Receive the scene image collected by the image acquisition device, and display the display content according to the scene image.
- the image acquisition device acquires scene images around the vehicle using a perspective that matches the attitude information of the terminal device, and then transmits the acquired scene images to the terminal device.
- the terminal device After receiving the scene image collected by the image acquisition device, the terminal device can display the corresponding content according to the scene image.
- the terminal device may directly display the scene image as display content, or may generate a virtual image based on the scene image, and display the generated virtual object.
- the perspective of the image acquisition device is adjusted accordingly, so that the perspective of the image acquisition device matches the posture information of the terminal device after the change.
- the terminal device may compare the posture information acquired two times in a row, compare the currently acquired posture information with the previously acquired posture information, and when the currently acquired posture information is compared with the previously acquired posture information,
- the image acquisition device may be controlled to change the orientation angle of the image acquisition device according to the change.
- the lens direction of the image acquisition device can be controlled to adjust based on the initial lens orientation state, so as to avoid the user's misoperation such as the shake of the terminal device and the image The orientation of the acquisition device changes.
- the terminal device is a head-mounted display device that can be worn by a user.
- the posture information detected by the head-mounted display device changes according to the movement of the user's head.
- the image is collected according to the change.
- the angle of view of the device is adjusted, that is, a control instruction is sent to the image capture device.
- the image capture device adjusts the angle of view according to the control instruction, and captures corresponding scene images, so that the user can adjust the angle of view of the image capture device according to the movement of the head.
- the terminal device After receiving the scene image collected by the image acquisition device, the terminal device can display the corresponding content and present it to the user in an AR / VR visual manner.
- an image acquisition device on a vehicle acquires an image of a scene around the vehicle according to an angle of view matched by the posture information of the terminal device, and transmits the image to the terminal device for display, so that a user can acquire a scene in a required viewing direction image.
- one or more markers may be set in the active scene of the vehicle.
- the image capture device may capture a marker image containing the markers, and
- the marker image is transmitted to the terminal device, and the terminal device receives the marker image and recognizes and tracks the marker in the marker image to obtain information such as the position, posture, and identity of the image acquisition device relative to the marker.
- the posture information is the orientation and rotation angle of the image acquisition device relative to the marker.
- the terminal device can identify the identity information of the marker, obtain corresponding virtual object data, and display it, thereby improving the immersion and reality of the augmented reality environment.
- the terminal device may use the position and posture of the image acquisition device relative to the marker as the position and posture of the terminal device relative to the marker, and determine the position of the virtual object in the virtual space according to the position and posture of the terminal device relative to the marker. To generate and display virtual objects.
- the virtual object may be a virtual road
- data of the virtual road is stored in the terminal device, and the virtual road is displayed according to the position, posture and identity of the collected marker image to achieve the effect of simulated driving.
- the terminal device may superimpose and display the generated virtual object and the captured scene image to achieve a display effect of augmented reality.
- the above-mentioned virtual object may be a virtual road sign, and the virtual road sign may be superimposed and displayed with scene images of the collected real scene.
- the above virtual object may also be a virtual obstacle, such as a virtual car, a virtual character, etc.
- the virtual obstacle is superimposed and displayed on the marker, and the user can control the vehicle according to the virtual obstacle he sees. Dodge to improve the realism of simulated driving.
- the terminal device can remotely control the vehicle, and after receiving the scene image collected by the image device, directly display the scene image, so that the user knows the scene around the vehicle.
- the terminal device may determine whether its posture information changes within a first preset time. When there is no change, it can indicate that the angle of view of the current image acquisition device does not change. At this time, multiple frames of images collected by the image acquisition device can be synthesized into the same image for display.
- the visual range of the image acquisition device is usually smaller than the visual range of the human eye, so that when the scene image collected by the image acquisition device is displayed, the user's realism is insufficient.
- the image acquisition device is controlled to rotate within a certain range to collect multiple frames of images, and the multiple frames of images collected by the image acquisition device are combined into one image for display.
- the image capture device can be controlled to rotate at a preset speed and capture images at a preset frequency, so that the angles of the collected multi-frame images are not large. .
- the image acquisition device can be rotated within a range of 15 ° to 30 °, and scene images of multiple perspective directions can be collected, and the scene images of multiple perspective directions collected by the image acquisition device can be combined into one image display, which can be viewed by the user. To a scene image with a large field of view.
- the angle of view of the image acquisition device can also be controlled by tracking changes in the focus of the eyeball.
- the terminal device acquires the image of the user's eye, captures the image data of the user's retina and cornea, and builds a 3D model of the eye based on these data.
- the focus of the eyeball is tracked in three-dimensional space to obtain the information of the focus change of the user's eyeball.
- the change information generates a control instruction and sends it to the image acquisition device to control the perspective of the image acquisition device. For example, after detecting that the focus of the user's eyeball moves to the left, a control instruction is generated to instruct the image acquisition device to rotate a certain angle counterclockwise in the horizontal plane.
- the shooting parameters of the image acquisition device may also be adjusted through the terminal device.
- a control operation made by a user on a terminal device may be used to generate a control instruction to instruct the image acquisition device to adjust shooting parameters such as focus, exposure time, and aperture.
- the vehicle may be communicatively connected with a remote controller, and the vehicle may adjust its driving direction according to the vehicle control instruction of the remote controller.
- the vehicle control instruction is a control instruction to change the driving direction
- the vehicle adjusts the driving direction according to the vehicle control instruction.
- the content of the surrounding scenes collected by the image acquisition device also changes, and the image acquisition device can send the collected scene images to the terminal device for display.
- the method of the above embodiment enables a user to see an image with a larger field of view, which improves the reality of watching.
- the present application provides a computer-readable storage medium.
- the computer-readable medium stores program code, and the program code can be called by a processor to execute the method described in the foregoing embodiment.
- the computer-readable storage medium may be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM.
- the computer-readable storage medium includes a non-transitory computer-readable storage medium.
- the computer-readable storage medium has a storage space of a program code for performing any of the method steps in the above method. These program codes can be read from or written into one or more computer program products.
- the program code may be compressed, for example, in a suitable form.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Hardware Design (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Optics & Photonics (AREA)
- Architecture (AREA)
- Processing Or Creating Images (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
一种生成虚拟对象的方法,所述方法包括:采集包含目标标记物的图像,其中,所述目标标记物为多个标记物中处于所述终端设备视野范围内的至少一个标记物,所述多个标记物分离设置在不同的位置;根据所述图像获取所述目标标记物相对所述终端设备的目标空间位置;读取存储的所述多个标记物之间的第一空间位置关系,并获取虚拟对象相对所述多个标记物的第二空间位置关系;基于所述第一空间位置关系、所述第二空间位置关系以及所述目标空间位置,确定所述虚拟对象的显示位置;及根据所述显示位置生成虚拟对象。
Description
本申请涉及增强现实技术领域,更具体地,涉及一种生成虚拟内容的方法、终端设备及存储介质。
近年来,随着科技的进步,增强现实(Augmented Reality,AR)和虚拟现实(Virtual Reality,VR)等技术已逐渐成为国内外研究的热点。以增强现实为例,增强现实是通过计算机***提供的信息增加用户对现实世界感知的技术,其将计算机生成的虚拟对象、场景或***提示信息等内容对象叠加到真实场景中,来增强或修改对现实世界环境或表示现实世界环境的数据的感知。
在多数情况下利用终端设备显示虚拟内容时,终端设备会通过相机对标记物采集图像,并对图像进行识别,根据对标记物的识别结果对虚拟内容进行显示。但是对于标记物的识别追踪会受到相机的视觉范围的影响,导致终端设备易丢失对标记物的追踪,使终端设备的显示受到影响。
发明内容
本申请实施例提出了一种生成虚拟内容的方法、终端设备及存储介质,能够增强终端设备显示的稳定性。
第一方面,本申请实施例提供了一种生成虚拟内容的方法,应用于终端设备,所述方法包括:采集包含目标标记物的图像,其中,所述目标标记物为多个标记物中处于所述终端设备视野范围内的至少一个标记物,所述多个标记物分离设置在不同的位置;根据所述图像获取所述目标标记物相对所述终端设备的目标空间位置;读取存储的所述多个标记物之间的第一空间位置关系,并获取虚拟对象相对所述多个标记物的第二空间位置关系;基于所述第一空间位置关系、所述第二空间位置关系以及所述目标空间位置,确定所述虚拟对象的显示位置;及根据所述显示位置生成虚拟对象。
第二方面,本申请实施例提供了一种数据更新方法,包括:采集包含标记物的图像;识别所述图像中的标记物,并确定所述标记物的身份信息;根据所述身份信息获取与所述标记物对应的绑定数据更换指令;基于所述绑定数据更换指令,将所述标记物绑定的第一数据更换为第二数据;及生成与所述第二数据对应的虚拟图像。
第三方面,本申请实施例提供了一种图像采集方法,应用于终端设备,所述方法包括:获取所述终端设备的姿态信息;基于所述姿态信息生成采集控制指令,并将所述采集控制指令发送至载具上的图像采集设备,其中,所述采集控制指令用于指示所述图像采集设备采用与所述姿态信息匹配的朝向角度采集所述载具的周围场景的场景图像;及接收所述图像采集设备采集的场景图像,并根据所述场景图像生成显示内容。
第四方面,本申请实施例提供了一种终端设备,包括存储器及处理器,所述存储器中存储有计算机程序,所述计算机程序被所述处理器执行时,使得所述处理器实现如上所述的方法。
第五方面,本申请实施例提供了一种计算机存储介质,其上存储有计算机程序,所述计算机程序被处理器运行时,使得所述处理器实现如上所述的方法。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的一种增强现实***的场景图。
图2为本申请实施例提供的终端设备的框架示意图。
图3根据本申请一实施例的生成虚拟内容的方法流程图。
图4为根据本申请另一实施例的生成虚拟内容的方法流程图。
图5为本申请实施例提供的一实施例的场景图。
图6a-6d为本申请实施例提供的另一种实施方式的场景图。
图7a-7b为本申请实施例提供的另一种实施方式中的可选方式的一种使用场景示意图。
图8为一个实施例中确定虚拟对象的显示位置的流程图。
图9a、9b为本申请提供的另一种增强现实***的示意图。
图10为本申请一个实施例提供的数据更新方法的流程图。
图11为本申请提供的又一种增强现实***的示意图。
图12为本申请一实施例的图像采集方法流程图。
为了使本技术领域的人员更好地理解本申请方案,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述。
下面对本申请实施例提供的生成虚拟内容的方法的应用场景进行介绍。
请参图1,本申请实施例提供的增强现实***10,包括终端设备100以及多个标记物200。其中,上述多个标记物200可分离设置在场景中不同的位置。例如,多个标记物200可分离设置于桌面,也可分离设置于地面的一块区域内,上述多个标记物的实际设置方式并不限定。终端设备100可以是头戴显示装置,也可以是手机、平板等移动设备。终端设备100为头戴显示装置时,头戴显示装置可以为一体式头戴显示装置。终端设备100也可以是连接有外置电子装置的头戴显示装置。终端设备100还可以是与外接式/***式头戴显示装置连接的手机等智能终端,即终端设备100可作为头戴显示装置的处理和存储设备,***或者接入外接式头戴显示装置,以在头戴显示装置中显示虚拟对象。
请参图2,在一个实施例中,终端设备100为头戴显示装置,包括处理器110、存储器120以及一个或多个应用程序,其中一个或多个应用程序可以被存储在存储器120中并被配置为由一个或多个处理器110执行,一个或多个程序用于执行本申请所描述的方法。
处理器110可包括一个或者多个处理核。处理器110利用各种接口和线路连接整个电子设备100内的各个部分,通过运行或执行存储在存储器120内的指令、程序、代码集或指令集,以及调用存储在存储器120内的数据,执行电子设备100的各种功能和处理数据。处理器110可采用数字信号处理(Digital Signal Processing,DSP)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)、可编程逻辑阵列(Programmable Logic Array,PLA)中的至少一种硬件形式来实现。处理器110可集成中央处理器(Central Processing Unit, CPU)、图像处理器(Graphics Processing Unit,GPU)和调制解调器等中的一种或几种的组合。其中,CPU主要处理操作***、用户界面和应用程序等;GPU用于负责显示内容的渲染和绘制;调制解调器用于处理无线通信。其中,上述调制解调器也可不集成到处理器110中,单独通过一块通信芯片进行实现。
存储器120可包括随机存储器(Random Access Memory,RAM),也可包括只读存储器(Read-Only Memory)。存储器120可用于存储指令、程序、代码、代码集或指令集。存储器120可包括存储程序区和存储数据区,其中,存储程序区可存储用于实现操作***的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的指令等。存储数据区还可存储终端设备100在使用中所创建的数据等。
在一些实施例中,终端设备100还可包括相机130,用于采集现实物体的图像以及采集目标场景的场景图像。相机130可为红外相机,也可为可见光相机,具体类型并不限定。
在一个实施例中,终端设备还可包括如下一个或多个部件:显示模组、光学模组、通信模块以及电源。显示模组可包括显示控制单元,显示控制单元用于接收处理器渲染后的虚拟内容的显示图像,将该显示图像显示并投射至光学模组上,使用户能够通过光学模组观看到虚拟内容。其中,显示模组可以是显示屏或投射装置等,用于显示图像。光学模组可采用离轴光学***或波导光学***,显示模组显示的显示图像经光学模组后,能够被投射至用户的眼睛。用户通过光学模组可看到显示模组投射的显示图像。在一些实施方式中,用户还能够透过光学模组观察到现实环境,感受虚拟内容与现实环境叠加后的视觉效果。通信模块可是蓝牙、WiFi(Wireless-Fidelity,无线保真)、ZigBee(紫峰技术)等模块,终端设备可通过通信模块与交互设备通信连接,以进行信息以及指令的交互。电源可为整个终端设备进行供电,保证终端设备各个部件的正常运行。
当多个标记物200中的任意一个或多个标记物处于终端设备100的相机130的视觉范围内时,相机130可采集处于视觉范围内的其中一个标记物的图像。终端设备100可识别该图像,得到图像中包含的标记物相对终端设备的位置、朝向等空间位置信息,以及该标记物的身份信息等识别结果。终端设备可基于识别到的标记物相对终端设备的空间位置信息,以及上述多个标记物200之间的空间位置关系显示虚拟对象。在一些实施方式中,标记物200可具有拓扑结构的图案,拓扑结构是指标记物中的子标记物和特征点等之间连通关系。具体的标记物在本申请并不限定,能够被终端设备识别追踪即可。
请参图3,本申请实施例提供了一种生成虚拟内容的方法,应用于上述终端设备100,该方法可以包括:
步骤S310:采集包含目标标记物的图像,其中,目标标记物为多个标记物中处于相机的视觉范围内的至少一个标记物,多个标记物分离设置在不同的位置。
多个标记物分离设置于实际场景中不同的位置,各个标记物设置的位置可以是固定的,也可以是根据需求可随时进行变动的。相机可对在视觉范围内的至少一个标记物进行图像采集。
步骤S320:根据图像获取目标标记物相对终端设备的目标空间位置。
终端设备可对采集的图像中包含的标记物进行识别,以得到图像中的标记物相对终端设备的目标空间位置,包括位置及姿态信息等,以及该标记物的身份信息等。其中,姿态信息为目标标记物相对终端设备的朝向及旋转角度,不同标记物分别对应不同的身份信息。
步骤S330:读取存储的多个标记物之间的第一空间位置关系,并获取虚拟对象相对多个标记物的第二空间位置关系。
终端设备可通过相机预先采集一张或多张图像,该图像包含有分离设置在不同位置的各个标记物,根据该图像对各个标记物进行识别,建立不同标记物之间的相对空间位置关系,并存储于终端设备中。
在一些实施例中,多个标记物的位置相对固定,终端设备仅需获取一次各个标记物之 间的相对空间位置关系,并存储于存储器中,无需重复获取各个标记物之间的相对空间位置关系。例如,在AR博物馆中,不同标记物分别固定设置在不同的位置,标记物的位置可预先存储在终端设备中,当用户佩戴终端设备在博物馆内时,采集到任意一个标记物的图像,即可根据预先存储的各个标记物之间的相对空间位置关系得到其他标记物的位置、姿态等信息,并显示相应的虚拟对象。
终端设备可获取虚拟对象相对多个标记物的空间位置关系,该空间位置关系可根据实际需求进行设定,例如,虚拟对象可处于多个标记物构成的区域的中心位置,或虚拟对象处于其中一个标记物所在的位置等,但不限于此。虚拟对象需要显示的位置相对多个标记物的空间位置关系可预先存储于终端设备中,也可由用户进行实时选择。
步骤S340:基于第一空间位置关系、第二空间位置关系以及目标空间位置,确定虚拟对象的显示位置。
终端设备可根据虚拟对象相对多个标记物的位置关系,以及各个标记物之间的相对空间位置关系确定出虚拟对象相对目标标记物的位置关系,再基于目标标记物相对终端设备的目标空间位置,以目标标记物作为参照,获得虚拟对象相对终端设备的空间位置。终端设备可将虚拟对象相对终端设备的空间位置,进行坐标转换,得到虚拟对象在终端设备的虚拟空间中的渲染坐标,作为虚拟对象的显示位置,用于对虚拟对象进行显示。
步骤S350:根据显示位置对虚拟对象进行显示。
终端设备可获取虚拟对象的数据,并根据虚拟对象的数据在相应的渲染坐标处渲染虚拟对象,生成虚拟画面,并对虚拟对象进行显示。其中,虚拟对象的数据可为虚拟对象的模型数据,用户可通过终端设备观察到虚拟对象叠加显示在现实空间中。
上述实施例提供的方法,多个不同的标记物分离设置在多个不同的位置,终端设备识别到任意一个标记物,即可确定其他不在视觉范围内标记物的位置,获得到虚拟对象的显示位置,实现大范围的标记物的识别和追踪,提升终端设备对虚拟对象显示的稳定性。本方法无需多个标记物同时存在于视觉范围内,即可准确得到虚拟对象的显示位置,解决了标记物识别、跟踪的范围限制的问题。
请参阅图4,本申请另一实施例提供了一种生成虚拟内容的方法,应用于终端设备,该方法可以包括:
步骤S410:获取多个标记物之间的第一空间位置关系。
在一些实施方式中,当多个标记物全部处于终端设备的图像采集装置的视觉范围内时,终端设备可对标记物进行图像采集,得到包含上述所有标记物的图像。终端设备可对采集得到的图像中包含的每个标记物进行识别追踪,可获得每个标记物相对终端设备的空间位置以及身份信息,以建立不同标记物之间的相对空间位置关系。
在一具体的实施例中,终端设备可根据每个标记物相对终端设备的空间位置以及每个标记物的身份信息,依次获得每个标记物相对其他标记物之间的空间位置,从而得到各个标记物之间的相对空间位置关系。例如,如图5所示,场景中共设置有标记物A、B、C、D及E,终端设备可获取同时包含标记物A、B、C、D及E的图像,进而获得每个标记物相对终端设备的空间位置;根据每个标记物相对终端设备的空间位置,可分别获得标记物A、标记物B、标记物C、标记物D以及标记物E中两两之间的相对空间位置关系。
在另一具体的实施例中,终端设备也可根据每个标记物相对终端设备的空间位置以及每个标记物的身份信息,依次获得设置在相邻位置的标记物之间的空间位置关系,从而得到各个标记物之间的相对空间位置关系。例如,标记物包括标记物A、标记物B、标记物C、标记物D以及标记物E,标记物A与标记物B相邻设置,标记物B与标记物C相邻设置,标记物C与标记物D相邻设置,标记物D与标记物E相邻设备,可分别获取标记物A与标记物B、标记物B与标记物C、标记物C与标记物D、以及标记物D与标记物E的空间位置关系,得到各个标记物之间的相对位置关系。其中,标记物之间的相邻关系并不仅限于上述示例,仅需满足获得至少四个相邻关系的标记物之间的空间位置关系,并获得的具 备相邻关系的标记物之间的空间位置关系中不存在孤岛即可,即不存在单独的空间位置关系,即该单独的空间位置关系所对应的相邻的标记物与其他空间位置关系对应的相邻的标记物之间,不存在相同的标记物。
在一些实施方式中,终端设备可多次采集标记物的图像,分别对多次采集的图像进行识别跟踪,得到每张图像中包含的不同标记物之间的空间位置关系,进而获得各个标记物之间的相对空间位置关系。终端设备多次采集分离设置在不同位置的各个标记物时,每次采集的图像中至少包括两个标记物,每个标记物至少在两次采集的图像中出现,以保证获得的每张图像中标记物的空间位置关系不存在孤岛,即多个空间位置关系中不存在单独的空间位置关系,即该单独的空间位置关系所对应的标记物与其他空间位置关系对应的标记物之间,不存在相同的标记物。也即是,可使相邻两次采集的图像中包含至少一个相同的标记物。例如,标记物包括标记物A、标记物B、标记物C、标记物D以及标记物E,如图6a-6d所示,第一次终端设备可采集包含标记物A、B的图像,第二次终端设备可采集包含标记物B、C的图像,第三次终端设备可采集包含标记物C、D的图像,第四次终端设备可采集包含标记物D、E的图像,其中第一次采集的图像与第二采集的图像中包含有相同的标记物B,第二次采集的图像与第三采集的图像中包含有相同的标记物C,第三次采集的图像与第四采集的图像中包含有相同的标记物D。
终端设备可基于每张图像中的标记物之间的空间位置关系,选择不同图像中包含的相同标记物作为参照,从而得到各个标记物之间的第一空间位置关系。例如,标记物包括标记物A、B、C,对第一次采集的包含标记物A、B的图像进行识别,可分别获得标记物A和标记物B相对终端设备的空间位置,,进而得到标记物A与标记物B之间的空间位置关系;对第二次采集的包含标记物B、C的图像进行识别,可分别获得标记物B和标记物C相对终端设备的空间位置,进而得到标记物B与标记物C之间的空间位置关系;根据标记物A与标记物B之间的空间位置关系,标记物B与标记物C之间的空间位置关系,即可建立标记物A、标记物B及标记物C之间的相对空间位置关系。
在一些实施方式中,当多个标记物分离设置于不同位置时,多个标记物分布的范围可能较大,由于相机的视觉范围有限,无法一次采集包含所有标记物的图像,因此,可分为多次对上述多个标记物进行采集图像,并根据多次采集的标记物图像建立各个标记物之间的相对空间位置关系。例如,可将多个标记物划分为第一部分标记物以及第二部分标记物,其中,第一部分标记物包括至少两个标记物,第二部分标记物包括第一部分标记物中的至少一个标记物以及所有标记物中除第一部分标记物以外的其他标记物。
终端设备采集包含第一部分标记物的图像,可以识别该图像,得到第一部分标记物中每个标记物相对终端设备的空间位置以及身份信息,进而获得第一部分标记物中的各个标记物之间的空间位置关系;终端设备采集包含第二部分标记物的图像,可以识别该图像,得到第二部分标记物中每个标记物相对终端设备的空间位置以及身份信息,进而获得第二部分标记物中的各个标记物之间的空间位置关系。由于第二部分标记物中包含有第一部分标记物中的至少一个标记物,以第一部分标记物与第二部分标记物中相同的标记物作为参照,根据第一部分标记物之间的空间位置关系以及第二部分标记物之间的空间位置关系,即可得到各个标记物之间的相对空间位置关系。
请参7a-7b,在一个场景中,场景内设置有标记物A、B、C、D及E,终端设备100可采集包含标记物A、B和C的第一部分标记物的图像,并识别跟踪第一部分标记物的图像,得到标记物A、B和C之间的空间位置关系;终端设备100还可采集标记物B、D和E的第二部分标记物的图像,并识别跟踪第二部分标记物的图像,得到标记物B、D和E之间的空间位置关系;以标记物B为参照,可获得标记物A、B、C、D及E之间的空间位置关系。其中,根据第一部分标记物的图像得到第一部分标记物之间的空间位置关系,以及根据第二部分标记物的图像得到第二部分标记物之间的空间位置关系,可以参照上个实施方式中的方案获取图像中标记物之间的空间位置关系,在此不再细述。
作为又一种实施方式,终端设备可每次仅对一个标记物进行图像采集,并根据终端设备的姿态变化信息,获得多个标记物之间的空间位置关系。具体的,第n次采集包含标记物Xn的图像,对包含标记物Xn的图像进行识别跟踪得到标记物Xn相对终端设备的位置及姿态信息,第n+1次采集包含标记物Xn+1的图像,对包含标记物Xn+1的图像进行识别跟踪得到标记物Xn+1相对终端设备的位置,并且通过视觉惯性里程计(Visual-Inertial Odometry,VIO)实时获取终端设备的六自由度(degree of freedom,DOF)信息,该6DOF信息可包含终端设备的位置和姿态等信息,进而可得到终端设备第n+1次采集图像时相对第n次采集图像时的位置及姿态变化信息;根据终端设备在第n+1次采集图像时相对第n次采集图像时的位置及姿态变化信息、Xn相对终端设备的位置以及Xn+1相对终端设备的位置,可得到标记物Xn与标记物Xn+1之间的空间位置关系,对所有的标记物图像进行上述操作即可获取多个标记物之间的第一空间位置关系。
终端设备在得到上述多个标记物之间的相对空间位置关系后,可将该相对空间位置关系存储于存储器,用于确定虚拟对象的显示位置。
步骤S420:采集包含目标标记物的图像,其中,目标标记物为多个标记物中处于终端设备视野范围内的至少一个标记物,多个标记物分离设置在不同的位置。
当多个标记物中仅有一个标记物处于相机的视觉范围内时,将该标记物作为目标标记物;当有两个及以上的标记物处于相机的视觉范围内时,可选取其中的一个或多个标记物作为目标标记物。
步骤S430:根据图像获取目标标记物相对终端设备的目标空间位置。
步骤S440:读取存储的多个标记物之间的第一空间位置关系,并获取虚拟对象相对多个标记物的第二空间位置关系。
终端设备可获取虚拟对象需要在现实空间中叠加显示的位置与多个标记物的设置位置之间的空间位置关系。在一些实施方式中,虚拟对象的位置与多个标记物所构成的空间区域相关,可处于该空间区域中的部分区域,也可是与该空间区域重合,还可覆盖于该空间区域等;其中,多个标记物所构成的空间区域可为多个标记物所围成的最大区域,该空间区域可包括该多个标记物。例如,虚拟对象为虚拟棋盘,其与多个标记物所构成的空间区域重合,用户通过终端设备可观察到虚拟棋盘叠加显示在多个标记物所构成的空间区域上。
在一些实施方式中,虚拟对象也可与多个标记物构成的空间区域的中心位置存在空间位置关系。其中,虚拟对象的位置可与该空间区域的中心位置重合,也可是虚拟对象处于与该中心位置对应的指定位置。例如,虚拟对象为虚拟装饰物,对应的显示位置与多个标记物构成的空间区域的中心位置重合,用户通过终端设备可观察到虚拟装饰物叠加显示在多个标记物构成的空间区域的中心位置处。
在一些实施方式中,虚拟对象也可与多个标记物中的预设标记物存在空间位置关系,预设标记物为多个标记物中选定的一个标记物。其中,虚拟对象的显示位置可与预设标记物的位置重合,也可处于与预设标记物对应的指定位置。虚拟对象可与处于相机视觉范围内的目标标记物存在空间位置关系,例如,虚拟对象的显示位置可与目标标记物的位置重合,也可处于与目标标记物对应的指定位置。
步骤S450:基于第一空间位置关系、第二空间位置关系以及目标空间位置,确定虚拟对象的显示位置。
在一些实施方式中,虚拟对象的显示位置与处于相机视觉范围内的目标标记物的位置重合,直接由目标标记物相对终端设备的空间位置确定,可将目标标记物相对终端设备的空间位置进行坐标转换,得到虚拟空间中目标标记物相对终端设备的位置坐标,该位置坐标即作为虚拟对象在虚拟空间中的显示位置。
请参图8,在一些实施方式中,虚拟对象与多个标记物中的预设标记物存在空间位置关系,步骤S450可以包括:
步骤S851:基于第一空间位置关系,获取多个标记物构成的区域的中心位置。
在一些实施方式中,预设标记物是靠近多个标记物所构成的空间区域中心位置的标记物,终端设备可基于多个标记物之间的相对空间位置关系,获取多个标记物构成的空间区域的中心位置,确定靠近中心位置的标记物。
步骤S852:基于中心位置,从多个标记物中获取预设标记物,预设标记物相对中心位置的距离小于其他标记物相对中心位置的距离。
终端设备可分别计算每个标记物相对空间区域的中心位置的距离,并获取相对该中心位置的距离最短的标记物,作为预设标记物。
步骤S853:根据目标空间位置、第二空间位置关系以及预设标记物相对目标标记物之间的空间位置关系,确定虚拟对象的显示位置。
终端设备可根据多个标记物之间的空间位置关系,得到预设标记物与目标标记物之间的空间位置关系;根据虚拟对象与预设标记物之间的空间位置关系,以及预设标记物与目标标记物之间的空间位置关系,得到虚拟对象与目标标记物之间的空间位置关系;并根据目标标记物相对终端设备的目标空间位置,及虚拟对象与目标标记物之间的空间位置关系,以目标标记物作为参照,得到虚拟对象相对终端设备的空间位置,即得到虚拟对象在虚拟空间中的显示位置。
步骤S460:根据显示位置对虚拟对象进行显示。
在一些实施例中,终端设备可显示多个虚拟对象,多个虚拟对象的位置与多个标记物一一对应且分别重合。终端设备可根据处于相机视野范围内的目标标记物相对终端设备的空间位置,以及各标记物之间的相对空间位置关系,得到每个标记物相对终端设备的空间位置,将每个标记物相对终端设备的空间位置进行坐标转换,得到虚拟空间中每个标记物相对终端设备的位置坐标,即可将每个标记物分别对应的虚拟对象显示在相应的位置。
例如,在虚拟餐桌的增强现实场景,虚拟对象包括多种不同类型的虚拟菜品,每个标记物与一种类型的虚拟菜品对应,终端设备识别在视觉范围内的标记物,根据该标记物与其他标记物的空间位置关系,可将各个虚拟菜品在对应的标记物上叠加显示。
步骤S470:基于位置更新指令,重新获取多个标记物之间的第一空间位置关系,并将重新获取的第一空间位置关系进行存储。
在一些实施例,多个标记物之间的相对空间位置关系可能会发生变化,例如标记物被移动等情况,终端设备可根据用户的操作,得到位置更新指令,更新各个标记物之间的相对空间位置关系。
在一些实施例中,上述重新获取多个标记物之间的第一空间位置关系,并将重新获取的第一空间位置关系进行存储,可包括步骤(a)及步骤(b)。
步骤(a):获取对位置变动的标记物与其他标记物中至少一个标记物进行图像采集而得到的部分标记物图像,其中,其他标记物为多个标记物中除位置变动的标记物以外的标记物。
在标记物的位置发生变动时,由于变动的仅是少数标记物,因此仅需确定发生位置变化的标记物相对其他标记物的位置关系。终端设备可在用户操作下,对位置变动的标记物及其他标记物中的至少一个标记物进行图像采集,根据该采集的图像确定位置变动的标记物相对位置未变动的标记物之间的空间位置关系。
步骤(b):基于部分标记物图像以及此前多个标记物之间的第一空间位置关系,获取更新的第一空间位置关系。
终端设备可对采集的图像进行识别,得到位置变动的标记物相对终端设备的空间位置及身份,以及图像中位置未变动的标记物相对终端设备的空间位置及身份;根据位置变动的标记物及未变动位置的标记物分别相对终端设备的空间位置,得到位置变动的标记物与未变动位置的标记物之间的空间位置关系;以未变动位置的标记物为参照,利用此前存储的多个标记物之间的空间位置关系,得到位置变动后的各个标记物之间的相对空间位置关系。
在一些实施例中,终端设备可分多次对多个标记物进行图像采集,以检测标记物的位置是否发生变化。当然,检测位置是否发生变化可以是在用户进行增强现实体验时自动进行的,无需用户刻意进行检测。
上述实施例提供的方法,通过识别跟踪多个标记物中的至少一个标记物,即可将虚拟对象叠加显示在相应的位置,提高增强现实的互动性。
请参图9a、9b,本申请另一个实施例中提供的增强现实***90,包括终端设备91以及标记物92,终端设备91通过对采集的标记物图像进行识别跟踪,获取到终端设备91相对标记物92的位置及旋转关系,并显示相应的虚拟对象。终端设备91还可通过网络与服务器93进行通信连接,其中,终端设备91上运行AR/VR应用的客户端,服务器93上运行与客户端对应的AR/VR应用的服务端。作为一种方式,多个终端设备91可通过服务器93进行数据同步。
请参图10,本申请一实施例提供的数据更新方法,该方法可对标记物绑定的数据进行更新,无需更换新的标记物,通过同一标记物实现不同虚拟图像的显示。该数据更新方法,应用于增强现实***的终端设备91,包括以下步骤:
步骤S1001:采集包含标记物的图像。
步骤S1002:识别图像中的标记物,并确定标记物的身份信息。
在一些实施方式中,标记物可以是任意具有可识别特征标记的图形或物体,标记物对应的身份信息可以是用数字、字母及符号等中的一种或多种表示的身份编码,可根据标记物中图案的拓扑结构设定,不同的标记物对应不同的身份信息。
步骤S1003:根据身份信息获取与标记物对应的绑定数据的更换指令。
标记物的身份信息对应有绑定的虚拟图像数据,根据确定的标记物的身份信息,可获取与该标记物匹配的更换指令,该指令用于改变标记物绑定的虚拟图像数据。
在一些实施方式中,可通过终端设备的客户端对标记物绑定的虚拟图像数据进行设置。例如,终端设备采集并识别到该标记物的编码为“No.7”,用户可在客户端(APP应用)对“No.7”的标记物的绑定数据进行更换,生成与“No.7”标记物对应的更换指令,对本地存储的“No.7”标记物绑定的虚拟图像数据进行更换。
在一些实施方式中,也可通过服务器端对标记物绑定的虚拟图像数据进行设置,终端设备的客户端可从服务器端获取,实现数据同步。例如,终端设备采集并识别到该标记物的编码为“No.7”,可将编码“No.7”发送至服务器,由服务器判断是否需要更换绑定数据,当服务器反馈的结果为需要更换时,即可生成与该标记物对应的更换指令,对本地存储的“No.7”标记物绑定的虚拟图像数据进行更换。其中,服务器上可存储有多个标记物的身份信息,以及与每个标记物对应的绑定数据。
在一个实施例中,终端设备的客户端可先向服务器确认,标记物的身份信息对应的绑定数据在服务器中是否已经发生更换,再根据服务器反馈的结果决定是否生成该标记物的更换指令。客户端可向服务器端发送与标记物的身份信息对应的绑定信息查找请求,服务器在接收到绑定信息查找请求后,向客户端反馈与该身份信息对应的绑定数据更换结果。作为一种方式,标记物对应的绑定信息,可包含该身份信息当前绑定的虚拟图像数据,以及该身份信息对应的绑定数据更换结果,该绑定数据更换结果用于表示相对于客户端上次访问服务器时,该身份信息对应的绑定数据是否发生了更换。客户端在接收由服务器下发的与身份信息对应的绑定数据更换结果后,可根据该绑定数据更换结果判断该身份信息对应的绑定数据是否发生更换。
在一些实施方式中,终端设备可获取服务器中与标记物对应的身份信息绑定的数据标识,与终端设备中对应该标记物绑定的数据标识进行对比,判断是否一致,以生成对应的绑定数据更换结果。作为一种方式,绑定数据更换结果可为表示符,用于表示标记物对应的绑定数据在服务器端是否发生了更换,例如,绑定数据发生更换时,绑定数据更换结果为“1”,绑定数据没有发生更换时,绑定数据更换结果为“0”。
当标记物的身份信息对应绑定的数据发生更换时,可生成该标记物的更换指令,用于指示终端设备更新与该标记物绑定的数据。
在一个实施例中,终端设备可通过客户端对标记物的绑定数据进行更换,并同步到服务器端。用户可在终端设备中输入需要更换绑定数据的标记物的身份信息,并传输需要重新绑定的数据,传输方式可通过其它终端传送,从互联网上查找等。终端设备可获取需要更换绑定数据的标记物的身份信息及需要更换的数据,并将该身份信息及更换的数据发送至服务器,服务器根据该身份信息将对应标记物的绑定的数据进行更换。
步骤S1004:基于绑定数据的更换指令,更换标记物绑定的数据。
终端设备可根据更换指令获取需要重新绑定的数据,并将对原先绑定的数据进行更换。例如,编码为“No.2”的标记物对应绑定的是“花瓶”的虚拟图像数据,终端设备获取更换指令,可将“No.2”标记物对应绑定的“花瓶”的虚拟图像数据更换为“牛”的虚拟图像数据。
作为一种方式,标记物更换对应数据前所绑定的第一数据可存储在客户端本地(例如,终端设备的存储器),需更换绑定的第二数据也可存储在客户端本地。此外,需更换绑定的第二数据也可由客户端向服务器端获取的,终端设备可在服务器中查找对应的第二数据,并将该第二数据下载至本地存储,以更新与该标记物对应的绑定数据。作为一种实施方式,终端设备可先在本地进行查找对应的第二数据,若在本地未存储对应的第二数据,再通过服务器进行获取。需更换绑定的第二数据也可由某个终端发送给服务器端并由服务器端存储,最后同步到各个客户端,当标记物的身份信息在服务器端对应的绑定数据被更换后,每个访问服务器端的客户端,均可对标记物在本地存储的绑定数据进行同步更新,通过在服务器端进行一次更新,即可让所有访问服务器端的客户端(终端)进行数据同步。
步骤S1005:显示与更换绑定的数据对应的虚拟内容。
终端设备将标记物与更新的数据绑定后,可生成相应的虚拟图像进行显示。作为一种方式,终端设备可显示绑定数据更换的过程,先根据原先绑定的数据生成并显示第一虚拟图像,在标记物与更新的数据绑定后,生成并显示第二虚拟图像,以呈现从第一虚拟图像到第二虚拟图像的变换过程。例如,“No.3”标记物原先对应绑定的是“牛”的虚拟图像数据,更换绑定的为“海报”的虚拟图像数据,在“No.3”标记物对应的绑定数据完成更换后,用户可通过终端设备观察到显示内容由“牛”的虚拟图像变换到“海报”的虚拟图像,给用户视觉上呈现动态变换的感觉,该动态变换可是渐变、突变等方式,具体可根据实际需求设定。在另一方式中,更换绑定数据也可是在熄屏状态下进行的,在将完成绑定数据的更换后,再显示与更换数据对应的虚拟图像。
本实施例提供的数据更新方法,可在不更换标记物的前提下,对同一标记物绑定的虚拟图像数据进行更换,能够应用在诸多虚拟现实或增强现实场景中。例如,在VR/AR博物馆中,设置的标记物的数量以及位置通常较为固定,在采用本申请的数据更新方法后,可对展台上的同一标记物所绑定的虚拟图像数据进行更换,以使在不同的时间段分别显示多种不同展品的虚拟图像,适应多种变换的展览主题(例如古文物展、科技展、画展等)。又例如,在VR/AR多媒体教室中,老师每天讲课的内容不同,此时将课堂相关讲义的虚拟图像数据与标记物进行绑定,在更换讲义时无需对相应的标记物进行更换,通过在客户端或服务器端对标记物绑定的讲义的虚拟图像数据即可实现,提升VR/AR多媒体教室的实用性和讲课的效率。再例如,在商场内,在使用本申请的数据更新方法后,只需每次用新的广告内容替换标记物绑定的旧广告内容,即可在不同时间段显示不同的广告内容,降低频繁更换标记物的成本,且实现不同广告内容的长时间不间断连续展示,提升广告的商业效益。上述的举例只是本申请的数据更新方法的一些实际应用,随着VR/AR技术的进一步发展与普及,本申请的数据更新方法可在更多的实际应用场景中发挥作用。
在一些实施方式中,终端设备还可向用户提供设置绑定数据更换周期的选项,比如在人机交互界面上显示用于设定绑定数据更换周期的选项,在客户端设置完成后,即可将该绑定数据更换周期发送至服务器,服务器可根据该绑定数据更换周期定期对标记物的绑定 数据进行更换。绑定数据更换周期,可对多个标记物同时设置的,也可仅针对一个标记物进行设置的;更换的绑定数据可预先存储在服务器中的一组虚拟图像数据,按照每个虚拟图像数据的编号依次进行更换,或者随机抽选进行更换。
在一些实施方式中,终端设备在采集到标记物并确认其身份信息后,还可从服务器获取该标记物在预定时间段内的访问时长。其中,标记物在预设时间段内的访问时长,可是标记物在预设时间段内被终端设备(可以是多个不同的终端设备)采集并显示相应虚拟图像的时长。当标记物在预设时间段内被采集并显示相应虚拟图像的时长小于某个预设值时,说明该标记物对应的虚拟图像在预设时间段内被观看的总时间较少,可在一定程度上反映该标记物绑定的虚拟图像并不吸引观看者,此时可向服务器发送更换请求,以使服务器对该标记物绑定的虚拟图像数据进行更换。作为一种方式,客户端在每次采集到标记物并显示相应的虚拟图像后,都会生成该标记物与访问时长关联的日志,并将日志上传至服务器,服务器端可记录由各个终端发送的日志中关于该标记物的访问时长的总和,并关联访问的日期,以获取该标记物在预设时间段内的总访问时长。
请参见图11,本申请又一个实施例提供的增强现实***1100,包括终端设备1110、图像采集设备1120以及载具1130。其中,图像采集设备1120设置于载具1130。作为一种方式,图像采集设备1120可通过旋转机械结构装设于载具1130的顶部位置,该旋转机械结构在控制下可转动,以带动图像采集设备1120相对载具1130旋转,改变图像采集设备1120相对载具1130的姿态。图像采集设备1120在载具1130上的位置并不限定,例如,还可设置于载具的尾部等。终端设备1110与图像采集设备1120可建立通信连接,例如,通过wifi、GPRS、4G、5G等方式建立通信连接,具体不限定。图像采集设备1120用于采集载具周围的场景图像发送至终端设备1110,其可为彩色摄像头、红外摄像头等,具体类型并不限定。
请参阅图12,本申请实施例提供的一种图像采集方法,应用于终端设备1110,该方法包括:
步骤S1210:获取终端设备的姿态信息。
在一些实施例中,终端设备中可设置有姿态传感器,用于检测终端设备的姿态信息。姿态传感器是基于MEMS技术的三维运动姿态测量***;姿态传感器可包括三轴陀螺仪、三轴加速度计、三轴电子罗盘等运动传感器,通过内嵌的低功耗处理器得到终端设备的姿态信息。终端设备的姿态信息包括终端设备的朝向及旋转角度。终端设备也可通过内置的IMU(Inertial measurement unit,惯性测量单元)获取终端设备的姿态信息,具体不限定。
步骤S1220:基于姿态信息生成采集控制指令,并将采集控制指令发送至载具上的图像采集设备,其中,采集控制指令用于指示图像采集设备采用与姿态信息匹配的朝向角度采集载具的周围场景的场景图像。
通过终端设备的姿态信息对图像采集设备进行控制,使图像采集设备以需要的视角方向采集图像。在一些实施例中,终端设备可根据获取的朝向及旋转角度,获得图像采集设备需要旋转的角度及方向。具体地,图像采集设备具有初始的镜头朝向状态,可预先设定一个预设姿态信息,其中终端设备的预设姿态信息与图像采集设备的初始的镜头朝向状态对应。在终端设备获取的姿态信息为预设姿态信息时,图像采集设备无需调整镜头方向,生成的采集控制指令将指示图像采集设备不作转动调整;在终端设备获取的姿态信息与该预设姿势信息不同时,终端设备可根据的获取的姿态信息相对预设姿态信息的区别,确定图像采集设备需要旋转的方向以及角度,生成的采集控制指令指示图像采集设备按照上述需要旋转的方向及角度进行旋转,使图像采集设备的镜头朝向与终端设备的姿态信息对应。
在生成的采集控制指令发送至图像采集设备后,图像采集设备根据采集控制指令将镜头朝向进行调整,使镜头朝向与终端设备的姿态信息匹配,使图像采集设备对载具周围的场景进行图像采集设备的视角方向与终端设备的姿态信息匹配,实现通过终端设备的姿态信息控制图像采集设备的视角方向。
步骤S1230:接收图像采集设备采集的场景图像,并根据场景图像将显示内容进行显 示。
图像采集设备在采用与终端设备的姿态信息匹配的视角采集载具周围的场景图像后,将采集的场景图像传输至终端设备。终端设备在接收到图像采集设备采集的场景图像后,可根据场景图像显示相应内容。在一些实施例中,终端设备可直接将场景图像作为显示内容显示,也可根据场景图像生成虚拟图像,并显示生成的虚拟对象。
在终端设备的姿态信息发生变化时,相应调整图像采集设备的视角,使图像采集设备的视角与终端设备变化后的姿态信息匹配。
在一些实施例中,终端设备可以比较相邻两次采集的姿态信息,将当前获取的姿态信息与前次获取的姿态信息进行比较,当当前获取的姿态信息相比前次获取的姿态信息的变化大于预设变化时,可以控制图像采集设备根据变化改变图像采集设备的朝向角度。可以在终端设备的姿态信息相对预设姿态信息有较大差别时,再控制图像采集设备的镜头方向在初始的镜头朝向状态基础上进行调整,避免终端设备的抖动等用户的误操作而使图像采集设备的朝向角度发生变化。
在一些实施例中,终端设备为可被用户佩戴的头戴显示装置,头戴显示装置所检测的姿态信息会根据用户头部的运动而变化,当检测到变化时,根据该变化对图像采集设备的视角进行调整,即发送控制指令至图像采集设备,图像采集设备根据控制指令调整视角,采集相应的场景图像,使得用户根据头部的运动改变对图像采集设备的视角进行调整。终端设备在接收到图像采集设备采集的场景图像后,可显示相应的内容,以AR/VR的视觉方式呈现给用户。
本申请实施例提供的方法,载具上的图像采集设备根据终端设备的姿态信息匹配的视角采集载具周围的场景图像,并传输至终端设备进行显示,使得用户能够以需要的视角方向采集场景图像。
在一些实施例中,载具的活动场景中可能设置有一个或多个标记物,当标记物处于图像采集设备的视觉范围内,图像采集设备可采集到包含标记物的标记物图像,并将标记物图像传输至终端设备,终端设备接收该标记物图像,并识别跟踪该标记物图像中的标记物,以获得图像采集设备相对标记物的位置、姿态,及身份等信息。其中,姿态信息为图像采集设备相对标记物的朝向及旋转角度。终端设备可识别标记物的身份信息,获取对应的虚拟对象数据,并进行显示,提高增强现实环境的沉浸感和真实感。
在一些实施例中,终端设备可将图像采集设备相对标记物的位置姿态,作为终端设备相对标记物的位置姿态,并根据终端设备相对标记物的位置姿态,确定虚拟对象在虚拟空间中的位置,生成并显示虚拟对象。
在模拟驾驶的场景中,上述虚拟对象可以是虚拟道路,终端设备中存储有虚拟道路的数据,根据采集的标记物图像的位置姿态及身份,显示虚拟道路,达到模拟驾驶的效果。
在一个实施例中,终端设备可将生成的虚拟对象与采集的场景图像叠加进行显示,实现增强现实的显示效果。例如,在一些场景中,上述虚拟对象可以是虚拟路标,可将虚拟路标与采集的真实场景的场景图像叠加显示。
在模拟驾驶的场景中,上述虚拟对象也可以是虚拟障碍物,例如虚拟汽车,虚拟人物等,将虚拟障碍物于叠加显示在标记物上,用户可根据看到的虚拟障碍物,控制载具进行躲避,提高模拟驾驶的真实感。
在一些实施例中,终端设备可远程控制载具,在接收到图像设备采集的场景图像后,直接将场景图像进行显示,以便用户获知载具周围的场景。
在一些实施例中,终端设备可判断其姿态信息在第一预设时间内是否发生变化。当未发生变化时,可表示当前图像采集设备的视角不发生变化,此时可将图像采集设备采集的多帧图像合成为同一张图像进行显示。
图像采集设备的视觉范围通常小于人眼的视觉范围,使得图像采集设备采集的场景图像在显示时,用户的真实感不足。通过终端设备的姿态信息在第一预设时间内未发生变化 时,控制图像采集设备在一定范围内进行转动采集多帧图像,并将图像采集设备采集的多帧图像合成为一张图像进行显示。作为一种具体实施方式,为保证采集的多帧图像的时间相差不大,可控制图像采集设备按照预设速度转动并按照预设频率采集图像,以使采集的多帧图像的角度相差不大。例如,可以使图像采集设备在15°~30°范围内转动,采集到多个视角方向的场景图像,将图像采集设备采集的多个视角方向的场景图像合成为一张图像显示,用户可看到视野范围较大的场景图像。
在一些实施方式中,还可通过跟踪眼球的焦点的变化,对图像采集设备的视角进行控制。终端设备获取用户的眼睛图像,捕捉到用户的视网膜及角膜的图像数据,并根据这些数据构建眼睛的3D模型,通过三维空间跟踪眼球的焦点,得到用户眼球的焦点变化信息,最后根据用户眼球的变化信息生成控制指令并发送至图像采集设备,以控制图像采集设备的视角。例如,在检测到用户的眼球的焦点往左移动,生成控制指令以指示图像采集设备在水平面内逆时针旋转一定角度。
在一些实施例中,还可通过终端设备对图像采集设备的拍摄参数进行调整。例如,可通过用户于终端设备做出的控制操作,生成控制指令指示图像采集设备调整焦距、曝光时间、光圈等拍摄参数。
在一个场景中,该载具可与远程控制器通信连接,载具根据远程控制器的载具控制指令,调整其行驶的方向。当载具控制指令为改变行驶方向的控制指令时,载具会根据载具控制指令调整行驶方向。此时,由于载具行驶方向发生改变,图像采集设备所采集的周围场景的内容也会发生变化,图像采集设备可将采集的场景图像发送至终端设备,进行显示。
上述实施例的方法,可使用户可以看到视野范围较大的图像,提升观看的真实感。
在一个实施例中,本申请提供一种计算机可读存储介质,该计算机可读介质中存储有程序代码,程序代码可被处理器调用执行上述实施例中所描述的方法。
计算机可读存储介质可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。可选地,计算机可读存储介质包括非易失性计算机可读介质(non-transitory computer-readable storage medium)。计算机可读存储介质具有执行上述方法中的任何方法步骤的程序代码的存储空间。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。程序代码可以例如以适当形式进行压缩。
最后应说明的是:以上实施例仅用以说明本申请的技术方案,而非对其限制;尽管参照前述实施例对本申请进行了详细的说明,本领域的普通技术人员当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不驱使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。
Claims (22)
- 一种生成虚拟内容的方法,应用于终端设备,其特征在于,所述方法包括:采集包含目标标记物的图像,其中,所述目标标记物为多个标记物中处于所述终端设备视野范围内的至少一个标记物,所述多个标记物分离设置在不同的位置;根据所述图像获取所述目标标记物相对所述终端设备的目标空间位置;读取存储的所述多个标记物之间的第一空间位置关系,并获取虚拟对象相对所述多个标记物的第二空间位置关系;基于所述第一空间位置关系、所述第二空间位置关系以及所述目标空间位置,确定所述虚拟对象的显示位置;及根据所述显示位置生成虚拟对象。
- 根据权利要求1所述的方法,其特征在于,所述虚拟对象相对所述多个标记物的第二空间位置关系包括以下中的至少一种:所述虚拟对象与所述多个标记物构成的空间区域的空间位置关系;所述虚拟对象与所述多个标记物构成的区域的中心位置的空间位置关系;所述虚拟对象与所述多个标记物中的预设标记物的空间位置关系;及所述虚拟对象与所述目标标记物的空间位置关系。
- 根据权利要求1所述的方法,其特征在于,所述虚拟对象相对所述多个标记物的第二空间位置关系为所述虚拟对象与所述多个标记物中的预设标记物的空间位置关系;所述基于所述第一空间位置关系、第二空间位置关系以及所述目标空间位置,确定所述虚拟对象的显示位置,包括:基于所述第一空间位置关系,获取所述多个标记物构成的区域的中心位置;基于所述中心位置,从所述多个标记物中获取预设标记物,所述预设标记物相对所述中心位置的距离小于其他标记物相对所述中心位置的距离;及根据所述目标空间位置、第二空间位置关系以及所述预设标记物相对所述目标标记物之间的空间位置关系,确定所述虚拟对象的显示位置。
- 根据权利要求1所述的方法,其特征在于,在所述采集包含目标标记物的图像之前,所述方法还包括:对分离设置的多个标记物进行图像采集,得到包含所述多个标记物的图像;及基于所述包含所述多个标记物的图像,获得所述多个标记物之间的第一空间位置关系。
- 根据权利要求1所述的方法,其特征在于,在所述采集包含目标标记物的图像之前,所述方法还包括:获取对分离设置的多个标记物进行多次图像采集而得到的多张图像,其中,每张图像至少包含两个标记物,相邻两次图像采集的图像中包含至少一个相同标记物;基于所述多次图像采集的多张图像,获得每张图像中的标记物之间的位置关系;及基于所述每张图像中的标记物之间的位置关系,得到所述多个标记物之间的第一空间位置关系。
- 根据权利要求1任一项所述的方法,其特征在于,所述方法还包括:基于位置更新指令,重新获取多个标记物之间的第一空间位置关系,并将重新获取的所述第一空间位置关系进行存储。
- 根据权利要求6所述的方法,其特征在于,所述重新获取所述多个标记物之间的第一空间位置关系,并将重新获取的所述第一空间位置关系进行存储,包括:获取对位置变动的标记物与其他标记物中至少一个标记物进行图像采集而得到的部分标记物图像,其中,所述其他标记物为所述多个标记物中除所述位置变动的标记物以外的标记物;及基于所述部分标记物图像以及此前所述多个标记物之间的第一空间位置关系,得到当前所述多个标记物之间的第一空间位置关系,并将第一空间位置关系进行存储。
- 一种终端设备,其特征在于,包括存储器及处理器,所述存储器中存储有计算机程序,所述计算机程序被所述处理器执行时,使得所述处理器执行如下步骤:采集包含目标标记物的图像,其中,所述目标标记物为多个标记物中处于所述终端设备视野范围内的至少一个标记物,所述多个标记物分离设置在不同的位置;根据所述图像获取所述目标标记物相对所述终端设备的目标空间位置;读取存储的所述多个标记物之间的第一空间位置关系,并获取虚拟对象相对所述多个标记物的第二空间位置关系;基于所述第一空间位置关系、所述第二空间位置关系以及所述目标空间位置,确定所述虚拟对象的显示位置;及根据所述显示位置生成虚拟对象。
- 根据权利要求8所述的终端设备,其特征在于,所述虚拟对象相对所述多个标记物的第二空间位置关系包括以下中的至少一种:所述虚拟对象与所述多个标记物构成的空间区域的空间位置关系;所述虚拟对象与所述多个标记物构成的区域的中心位置的空间位置关系;所述虚拟对象与所述多个标记物中的预设标记物的空间位置关系;及所述虚拟对象与所述目标标记物的空间位置关系。
- 根据权利要求8所述的终端设备,其特征在于,所述虚拟对象相对所述多个标记物的第二空间位置关系为所述虚拟对象与所述多个标记物中的预设标记物的空间位置关系;所述基于所述第一空间位置关系、第二空间位置关系以及所述目标空间位置,确定所述虚拟对象的显示位置,包括:基于所述第一空间位置关系,获取所述多个标记物构成的区域的中心位置;基于所述中心位置,从所述多个标记物中获取预设标记物,所述预设标记物相对所述中心位置的距离小于其他标记物相对所述中心位置的距离;及根据所述目标空间位置、第二空间位置关系以及所述预设标记物相对所述目标标记物之间的空间位置关系,确定所述虚拟对象的显示位置。
- 根据权利要求8所述的终端设备,其特征在于,所述处理器在执行所述采集包含目标标记物的图像的步骤之前,还执行以下步骤:对分离设置的多个标记物进行图像采集,得到包含所述多个标记物的图像;及基于所述包含所述多个标记物的图像,获得所述多个标记物之间的第一空间位置关系。
- 根据权利要求8所述的终端设备,其特征在于,所述处理器在执行所述采集包含目标标记物的图像的步骤之前,还执行以下步骤:获取对分离设置的多个标记物进行多次图像采集而得到的多张图像,其中,每张图像至少包含两个标记物,相邻两次图像采集的图像中包含至少一个相同标记物;基于所述多次图像采集的多张图像,获得每张图像中的标记物之间的位置关系;及基于所述每张图像中的标记物之间的位置关系,得到所述多个标记物之间的第一空间位置关系。
- 根据权利要求8所述的终端设备,其特征在于,所述处理器还执行以下步骤:基于位置更新指令,重新获取多个标记物之间的第一空间位置关系,并将重新获取的所述第一空间位置关系进行存储。
- 根据权利要求13所述的终端设备,其特征在于,所述重新获取所述多个标记物之间的第一空间位置关系,并将重新获取的所述第一空间位置关系进行存储,包括:获取对位置变动的标记物与其他标记物中至少一个标记物进行图像采集而得到的部分标记物图像,其中,所述其他标记物为所述多个标记物中除所述位置变动的标记物以外的标记物;及基于所述部分标记物图像以及此前所述多个标记物之间的第一空间位置关系,得到当前所述多个标记物之间的第一空间位置关系,并将第一空间位置关系进行存储。
- 一种计算机存储介质,其上存储有计算机程序,其特征在于,所述计算机程序被处理器运行时,使得所述处理器执行如下步骤:采集包含目标标记物的图像,其中,所述目标标记物为多个标记物中处于所述终端设备视野范围内的至少一个标记物,所述多个标记物分离设置在不同的位置;根据所述图像获取所述目标标记物相对所述终端设备的目标空间位置;读取存储的所述多个标记物之间的第一空间位置关系,并获取虚拟对象相对所述多个标记物的第二空间位置关系;基于所述第一空间位置关系、所述第二空间位置关系以及所述目标空间位置,确定所述虚拟对象的显示位置;及根据所述显示位置生成虚拟对象。
- 根据权利要求15所述的计算机存储介质,其特征在于,所述虚拟对象相对所述多个标记物的第二空间位置关系包括以下中的至少一种:所述虚拟对象与所述多个标记物构成的空间区域的空间位置关系;所述虚拟对象与所述多个标记物构成的区域的中心位置的空间位置关系;所述虚拟对象与所述多个标记物中的预设标记物的空间位置关系;及所述虚拟对象与所述目标标记物的空间位置关系。
- 根据权利要求15所述的计算机存储介质,其特征在于,所述虚拟对象相对所述多个标记物的第二空间位置关系为所述虚拟对象与所述多个标记物中的预设标记物的空间位置关系;所述基于所述第一空间位置关系、第二空间位置关系以及所述目标空间位置,确定所述虚拟对象的显示位置,包括:基于所述第一空间位置关系,获取所述多个标记物构成的区域的中心位置;基于所述中心位置,从所述多个标记物中获取预设标记物,所述预设标记物相对所述中心位置的距离小于其他标记物相对所述中心位置的距离;及根据所述目标空间位置、第二空间位置关系以及所述预设标记物相对所述目标标记物之间的空间位置关系,确定所述虚拟对象的显示位置。
- 根据权利要求15所述的计算机存储介质,其特征在于,所述计算机程序被处理器运行时,使得所述处理器在执行所述采集包含目标标记物的图像的步骤之前,还执行以下步骤:对分离设置的多个标记物进行图像采集,得到包含所述多个标记物的图像;及基于所述包含所述多个标记物的图像,获得所述多个标记物之间的第一空间位置关系。
- 根据权利要求15所述的计算机存储介质,其特征在于,所述计算机程序被处理器运行时,使得所述处理器在执行所述采集包含目标标记物的图像的步骤之前,还执行以下步骤:获取对分离设置的多个标记物进行多次图像采集而得到的多张图像,其中,每张图像至少包含两个标记物,相邻两次图像采集的图像中包含至少一个相同标记物;基于所述多次图像采集的多张图像,获得每张图像中的标记物之间的位置关系;及基于所述每张图像中的标记物之间的位置关系,得到所述多个标记物之间的第一空间位置关系。
- 根据权利要求15所述的计算机存储介质,其特征在于,所述计算机程序被处理器运行时,还使得所述处理器执行以下步骤:基于位置更新指令,获取对位置变动的标记物与其他标记物中至少一个标记物进行图像采集而得到的部分标记物图像,其中,所述其他标记物为所述多个标记物中除所述位置变动的标记物以外的标记物;及基于所述部分标记物图像以及此前所述多个标记物之间的第一空间位置关系,得到当前所述多个标记物之间的第一空间位置关系,并将第一空间位置关系进行存储。
- 一种数据更新方法,其特征在于,包括:采集包含标记物的图像;识别所述图像中的标记物,并确定所述标记物的身份信息;根据所述身份信息获取与所述标记物对应的绑定数据更换指令;基于所述绑定数据更换指令,将所述标记物绑定的第一数据更换为第二数据;及生成与所述第二数据对应的虚拟图像。
- 一种图像采集方法,其特征在于,应用于终端设备,所述方法包括:获取所述终端设备的姿态信息;基于所述姿态信息生成采集控制指令,并将所述采集控制指令发送至载具上的图像采集设备,其中,所述采集控制指令用于指示所述图像采集设备采用与所述姿态信息匹配的朝向角度采集所述载具的周围场景的场景图像;及接收所述图像采集设备采集的场景图像,并根据所述场景图像生成显示内容。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/666,429 US11049324B2 (en) | 2018-07-23 | 2019-10-29 | Method of displaying virtual content based on markers |
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810814635.3A CN110750225A (zh) | 2018-07-23 | 2018-07-23 | 数据更新方法、装置、电子设备及计算机可读取存储介质 |
CN201810814635.3 | 2018-07-23 | ||
CN201810910950.6A CN110826375B (zh) | 2018-08-10 | 2018-08-10 | 显示方法、装置、终端设备及存储介质 |
CN201810910950.6 | 2018-08-10 | ||
CN201810924523.3 | 2018-08-14 | ||
CN201810924523.3A CN110825333B (zh) | 2018-08-14 | 2018-08-14 | 显示方法、装置、终端设备及存储介质 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/666,429 Continuation US11049324B2 (en) | 2018-07-23 | 2019-10-29 | Method of displaying virtual content based on markers |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020020102A1 true WO2020020102A1 (zh) | 2020-01-30 |
Family
ID=69180384
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/097128 WO2020020102A1 (zh) | 2018-07-23 | 2019-07-22 | 生成虚拟内容的方法、终端设备及存储介质 |
Country Status (2)
Country | Link |
---|---|
US (1) | US11049324B2 (zh) |
WO (1) | WO2020020102A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113393516A (zh) * | 2021-06-17 | 2021-09-14 | 北京房江湖科技有限公司 | 用于打散ar场景中的虚拟物体的方法和装置 |
CN113407030A (zh) * | 2021-06-25 | 2021-09-17 | 浙江商汤科技开发有限公司 | 视觉定位方法及相关装置、设备和存储介质 |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11095947B2 (en) * | 2018-06-12 | 2021-08-17 | Number 9, LLC | System for sharing user-generated content |
CN110908504B (zh) * | 2019-10-10 | 2021-03-23 | 浙江大学 | 一种增强现实博物馆协作交互方法与*** |
US11756225B2 (en) * | 2020-09-16 | 2023-09-12 | Campfire 3D, Inc. | Augmented reality collaboration system with physical device |
US11176756B1 (en) | 2020-09-16 | 2021-11-16 | Meta View, Inc. | Augmented reality collaboration system |
CN112100180B (zh) * | 2020-09-21 | 2022-03-04 | 北京嘀嘀无限科技发展有限公司 | 位置范围的确定方法、装置、存储介质和电子设备 |
USD1024198S1 (en) | 2022-03-10 | 2024-04-23 | Campfire 3D, Inc. | Augmented reality console |
USD1029076S1 (en) | 2022-03-10 | 2024-05-28 | Campfire 3D, Inc. | Augmented reality pack |
USD1014499S1 (en) | 2022-03-10 | 2024-02-13 | Campfire 3D, Inc. | Augmented reality headset |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2002159019A (ja) * | 2000-11-16 | 2002-05-31 | Canon Inc | 表示制御装置、撮影位置推測装置、表示システム、撮影システム、画像位置決め方法、撮影位置推測方法、及び処理プログラムを記録した記録媒体 |
CN105931272A (zh) * | 2016-05-06 | 2016-09-07 | 上海乐相科技有限公司 | 一种运动对象追踪方法及*** |
CN106875493A (zh) * | 2017-02-24 | 2017-06-20 | 广东电网有限责任公司教育培训评价中心 | Ar眼镜中虚拟目标物的叠加方法 |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9524436B2 (en) * | 2011-12-06 | 2016-12-20 | Microsoft Technology Licensing, Llc | Augmented reality camera registration |
US9685005B2 (en) * | 2015-01-02 | 2017-06-20 | Eon Reality, Inc. | Virtual lasers for interacting with augmented reality environments |
JP6328579B2 (ja) * | 2015-03-13 | 2018-05-23 | 富士フイルム株式会社 | 仮想オブジェクト表示システムおよびその表示制御方法並びに表示制御プログラム |
JP2017187667A (ja) * | 2016-04-07 | 2017-10-12 | セイコーエプソン株式会社 | 頭部装着型表示装置およびコンピュータープログラム |
-
2019
- 2019-07-22 WO PCT/CN2019/097128 patent/WO2020020102A1/zh active Application Filing
- 2019-10-29 US US16/666,429 patent/US11049324B2/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2002159019A (ja) * | 2000-11-16 | 2002-05-31 | Canon Inc | 表示制御装置、撮影位置推測装置、表示システム、撮影システム、画像位置決め方法、撮影位置推測方法、及び処理プログラムを記録した記録媒体 |
CN105931272A (zh) * | 2016-05-06 | 2016-09-07 | 上海乐相科技有限公司 | 一种运动对象追踪方法及*** |
CN106875493A (zh) * | 2017-02-24 | 2017-06-20 | 广东电网有限责任公司教育培训评价中心 | Ar眼镜中虚拟目标物的叠加方法 |
Non-Patent Citations (2)
Title |
---|
CHUHANG_ZHQR: "Analysis of multiple identification AR routines (2)", CSDN BLOG, 17 November 2015 (2015-11-17), XP055681690, Retrieved from the Internet <URL:https://blog.csdn.net/chuhang_zhqr/article/details/49889115> * |
FLYYUFENFEI: "ARToolKit source code study notes 1: SimpleLite.c", CSDN BLOG, 15 March 2017 (2017-03-15), XP055681684, Retrieved from the Internet <URL:https://blog.csdn.net/flyyufenfei/article/details/62238549> * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113393516A (zh) * | 2021-06-17 | 2021-09-14 | 北京房江湖科技有限公司 | 用于打散ar场景中的虚拟物体的方法和装置 |
CN113393516B (zh) * | 2021-06-17 | 2022-05-24 | 贝壳找房(北京)科技有限公司 | 用于打散ar场景中的虚拟物体的方法和装置 |
CN113407030A (zh) * | 2021-06-25 | 2021-09-17 | 浙江商汤科技开发有限公司 | 视觉定位方法及相关装置、设备和存储介质 |
CN113407030B (zh) * | 2021-06-25 | 2023-08-25 | 浙江商汤科技开发有限公司 | 视觉定位方法及相关装置、设备和存储介质 |
Also Published As
Publication number | Publication date |
---|---|
US20200066054A1 (en) | 2020-02-27 |
US11049324B2 (en) | 2021-06-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020020102A1 (zh) | 生成虚拟内容的方法、终端设备及存储介质 | |
US11043031B2 (en) | Content display property management | |
US10055888B2 (en) | Producing and consuming metadata within multi-dimensional data | |
US11257294B2 (en) | Cross reality system supporting multiple device types | |
CN115151947B (zh) | 具有基于wifi/gps的地图合并的交叉现实*** | |
CN107771342B (zh) | 一种增强现实显示方法及头戴式显示设备 | |
CN102959616B (zh) | 自然交互的交互真实性增强 | |
CN105981076B (zh) | 合成增强现实环境的构造 | |
US20130120365A1 (en) | Content playback apparatus and method for providing interactive augmented space | |
WO2018113759A1 (zh) | 基于定位***和ar/mr的探测***及探测方法 | |
US11532138B2 (en) | Augmented reality (AR) imprinting methods and systems | |
CN106980378B (zh) | 虚拟显示方法和*** | |
JP6656382B2 (ja) | マルチメディア情報を処理する方法及び装置 | |
CN108830944B (zh) | 光学透视式三维近眼显示***及显示方法 | |
Samini et al. | A review of current, complete augmented reality solutions | |
KR102176805B1 (ko) | 뷰 방향이 표시되는 vr 컨텐츠 제공 시스템 및 방법 | |
CN105894581B (zh) | 一种呈现多媒体信息的方法及装置 | |
JP6680886B2 (ja) | マルチメディア情報を表示する方法及び装置 | |
JP7526169B2 (ja) | クロスリアリティシステム | |
WO2020244576A1 (zh) | 基于光通信装置叠加虚拟对象的方法和相应的电子设备 | |
CN112053444B (zh) | 基于光通信装置叠加虚拟对象的方法和相应的电子设备 | |
Hamadouche | Augmented reality X-ray vision on optical see-through head mounted displays | |
KR20220078850A (ko) | Vr 컨텐츠를 제공하는 방법 및 그 장치 | |
Blanco Pons | Analysis and development of augmented reality applications for the dissemination of cultural heritage | |
JP2024103610A (ja) | クロスリアリティシステム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19840666 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 01.06.2021) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19840666 Country of ref document: EP Kind code of ref document: A1 |