CN117635717A

CN117635717A - Information processing method, information processing device, electronic equipment and storage medium

Info

Publication number: CN117635717A
Application number: CN202311787249.7A
Authority: CN
Inventors: 袁东
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2023-12-22
Filing date: 2023-12-22
Publication date: 2024-03-01

Abstract

The application discloses an information processing method, an information processing device, electronic equipment and a storage medium, and belongs to the technical field of augmented reality. The information processing method is applied to an XR device and comprises the following steps: acquiring a picture displayed by the electronic equipment through a camera of the XR equipment to obtain a first image, wherein the picture comprises a positioning mark; determining first position information of the electronic equipment in a first coordinate system based on the positioning identification in the first image, wherein the first coordinate system is a world coordinate system of the XR equipment; and sending the first position information to the electronic equipment so that the electronic equipment can determine the pose information of the electronic equipment in the first coordinate system according to the first position information and the first pose information of the electronic equipment.

Description

Information processing method, information processing device, electronic equipment and storage medium

Technical Field

The application belongs to the technical field of augmented reality, and particularly relates to an information processing method, an information processing device, electronic equipment and a storage medium.

Background

As technology advances, extended Reality (XR) devices are becoming more mature. Mixed reality shooting (Mixed Reality Capturing, MRC) can be currently implemented through XR devices, and objects shot by electronic devices are fused with virtual scenes. Before shooting, pose information of the electronic device in a coordinate system of the XR device needs to be determined so as to fuse an object shot by the electronic device with a virtual scene.

However, in the related art, in order to determine pose information of the electronic device in the coordinate system of the XR device, a user needs to perform a plurality of manual operations, and thus, there is a problem of complicated operation and low efficiency.

Disclosure of Invention

An object of an embodiment of the present application is to provide an information processing method, an information processing device, an electronic device, and a storage medium, so as to solve the problems of complicated operation and low efficiency in the related art.

In a first aspect, an embodiment of the present application provides an information processing method, applied to an augmented reality XR device, including:

acquiring a picture displayed by the electronic equipment through a camera of the XR equipment to obtain a first image, wherein the picture comprises a positioning mark;

determining first position information of the electronic equipment in a first coordinate system based on the positioning identification in the first image, wherein the first coordinate system is a world coordinate system of the XR equipment;

and sending the first position information to the electronic equipment so that the electronic equipment can determine the pose information of the electronic equipment in the first coordinate system according to the first position information and the first pose information of the electronic equipment.

In a second aspect, an embodiment of the present application provides an information processing method, which is applied to an electronic device, and the method includes:

Displaying a picture comprising a positioning mark, so that a camera of the XR equipment acquires the picture to obtain a first image, and determining first position information of the electronic equipment in a first coordinate system based on the positioning mark in the first image, wherein the first coordinate system is a world coordinate system in which the XR equipment is located;

receiving first position information sent by XR equipment;

and determining pose information of the electronic equipment in a first coordinate system according to the first position information and the first pose information of the electronic equipment.

In a third aspect, embodiments of the present application provide an information processing apparatus for use with an augmented reality XR device, the apparatus comprising:

the first acquisition module is used for acquiring a picture displayed by the electronic equipment through a camera of the XR equipment to obtain a first image, wherein the picture comprises a positioning mark;

the first determining module is used for determining first position information of the electronic equipment in a first coordinate system based on the positioning identification in the first image, wherein the first coordinate system is a world coordinate system of the XR equipment;

and the sending module is used for sending the first position information to the electronic equipment so that the electronic equipment can determine the pose information of the electronic equipment in the first coordinate system according to the first position information and the first pose information of the electronic equipment.

In a fourth aspect, an embodiment of the present application provides an information processing apparatus, which is applied to an electronic device, including:

the display module is used for displaying a picture comprising a positioning mark, so that a camera of the XR equipment acquires the picture to obtain a first image, and determining first position information of the electronic equipment in a first coordinate system based on the positioning mark in the first image, wherein the first coordinate system is a world coordinate system in which the XR equipment is located;

the receiving module is used for receiving the first position information sent by the XR equipment;

and the second determining module is used for determining pose information of the electronic equipment in the first coordinate system according to the first position information and the first pose information of the electronic equipment.

In a fifth aspect, embodiments of the present application provide an electronic device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method according to the first and/or second aspects.

In a sixth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which, when executed by a processor, implement the steps of the method according to the first and/or second aspects.

In a seventh aspect, embodiments of the present application provide a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and where the processor is configured to execute a program or instructions to implement a method according to the first aspect and/or the second aspect.

In an eighth aspect, embodiments of the present application provide a computer program product stored in a storage medium, the program product being executable by at least one processor to implement the method according to the first and/or second aspects.

In the embodiment of the application, the electronic device may display a picture including a positioning identifier, the camera of the XR device may acquire the picture displayed by the electronic device to obtain a first image, then may determine first position information of the electronic device in a world coordinate system of the XR device based on the positioning identifier in the first image, send the first position information to the electronic device, and the electronic device may determine pose information of the electronic device in the world coordinate system of the XR device according to the first position information and the first pose information thereof. Therefore, a user only needs to aim the camera of the XR equipment at the screen of the electronic equipment, other operations are not needed, pose information of the electronic equipment in the world coordinate system of the XR equipment can be determined, and the method is simple to operate and high in efficiency.

Drawings

FIG. 1 is an architecture diagram of an information handling system shown in accordance with an exemplary embodiment;

FIG. 2 is one of the flow diagrams of a method of information processing shown in accordance with an exemplary embodiment;

FIG. 3 is a schematic diagram of a screen displayed by an electronic device, according to an example embodiment;

FIG. 4 is a second flow chart of a method of information processing according to an exemplary embodiment;

FIG. 5 is a third flow chart of a method of information processing according to an exemplary embodiment;

fig. 6 is one of block diagrams of a structure of an information processing apparatus according to an exemplary embodiment;

fig. 7 is a second block diagram showing a structure of an information processing apparatus according to an exemplary embodiment;

FIG. 8 is a block diagram of an electronic device, according to an example embodiment;

fig. 9 is a schematic hardware structure of an electronic device implementing an embodiment of the present application.

Detailed Description

Technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application are within the scope of the protection of the present application.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, as appropriate, such that embodiments of the present application may be implemented in sequences other than those illustrated or described herein, and that the objects identified by "first," "second," etc. are generally of a type and not limited to the number of objects, e.g., the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.

The information processing method, the device, the electronic equipment and the storage medium provided by the embodiment of the application are described in detail below through specific embodiments and application scenes thereof with reference to the accompanying drawings.

The information processing method provided by the embodiment of the application can be applied to an application scene for generating the mixed reality video.

The information processing method provided by the application can be applied to the architecture shown in fig. 1, and specifically described in detail with reference to fig. 1.

FIG. 1 is an architecture diagram of an information handling system, according to an example embodiment.

As shown in fig. 1, the information processing system may include: XR device 11 and electronic device 12. Wherein the XR device 11 and the electronic device 12 may establish a communication connection.

In one example, XR device 11 may be a VR device. The electronic device 12 may be, but is not limited to, a personal computer (Personal Computer, PC), a smart phone, a tablet computer, or a personal digital assistant (Personal Digital Assistant, PDA), or the like.

In one example, XR device 11 may acquire a picture displayed by electronic device 12 via a camera, resulting in a first image, which may include a location identifier; determining first location information of the electronic device 12 in a first coordinate system based on the location identity in the first image, the first coordinate system being a world coordinate system of the XR device 11; the first location information is transmitted to the electronic device 12 such that the electronic device 12 determines pose information of the electronic device 12 in the first coordinate system based on the first location information and the first pose information of the electronic device 12.

Therefore, when a user wants to determine pose information of the electronic device in the world coordinate system of the XR device so as to generate the mixed reality video, the camera of the XR device is only required to be aligned to the screen of the electronic device, other operations are not required, and the method is simple to operate and high in efficiency.

The information processing method provided in the embodiment of the present application will be described in detail below.

Fig. 2 is a flowchart illustrating an information processing method, the execution subject of which may be an XR device, according to an example embodiment. The execution body is not limited to the present application.

As shown in fig. 2, the information processing method may include the steps of:

in step 210, a first image is obtained by capturing, with a camera of the XR device, a picture displayed by the electronic device.

Here, the XR device and the electronic device may be pre-established with a communication connection. The XR device may include a head display and a handle, and the camera may be located on the head display.

The XR device may have a LiDAR (Light Detection And Ranging, liDAR) sensor. The LiDAR sensor may emit tens of thousands of tiny laser pulses that, after hitting an object, are reflected back and captured by a receiver on the LiDAR sensor, and the LiDAR system may measure the time that the laser pulse takes from emission to return, from which time the distance between the object and the LiDAR sensor can be calculated, since the speed of the laser is known (speed of light). By firing laser pulses at multiple angles and directions, liDAR can create one depth map for the entire scene. This depth map provides a distance value for each pixel, forming a 3D representation. The XR device can establish world coordinates, i.e., a first coordinate system, through LiDAR-constructed synchronized localization and mapping (Simultaneous localization and mapping, SLAM).

The camera of the XR device may obtain depth information, such as a TOF camera, but may be other cameras, without limitation.

The electronic device may be placed in any plane within the field of view of the SLAM. The electronic device may remain stationary and display a picture, and the picture displayed by the electronic device may include a positioning identifier. The XR device can observe a picture displayed by the electronic device through a perspective function (Video See Through), and specifically, the XR device can acquire the picture displayed by the electronic device through a camera to obtain a first image containing a positioning mark.

The positioning identifier may be any identifier with an area greater than a preset threshold. Illustratively, the positioning identifier may be a two-dimensional code. The two-dimensional code can comprise version information, format information, data and fault-tolerant keys, a data demand module, a positioning mark, a correction mark, a timing mark and a static area.

Step 220, determining first location information of the electronic device in the first coordinate system based on the location identifier in the first image.

Here, the first coordinate system may be a world coordinate system of the XR device. The first location information may be three-dimensional coordinates of the electronic device in a first coordinate system.

Based on a positioning identifier (e.g., a two-dimensional code of a contracted size), the XR device may identify and calculate positional information of the electronic device in a first coordinate system.

In an alternative embodiment, the screen may include at least three positioning identifiers, and step 210 may include:

acquiring pictures displayed by the electronic equipment through at least two cameras of the XR equipment to obtain at least two first images;

step 220 may include:

determining second position information corresponding to at least three positioning identifiers respectively based on at least two first images;

first location information of the electronic device in a first coordinate system is determined based on the at least three second location information.

Here, the XR device may have multiple cameras, in particular, the head display of the XR device may have multiple cameras, the XR device may optionally have two cameras located at the same level from among the multiple cameras, and a fixed distance between the two cameras, referred to as a "baseline", may be provided, which may simulate binocular stereoscopic vision of the human eye. The two cameras may constitute a binocular camera, which may capture two slightly different images, the difference being referred to as "parallax".

The XR equipment can acquire pictures displayed by the electronic equipment through two cameras to obtain two slightly different first images, and each first image can contain at least three positioning marks.

The positioning identifier may be a two-dimensional code, a layout and a pitch of at least three two-dimensional codes may be preset, and the electronic device may display a two-dimensional code array formed by the at least three two-dimensional codes.

For example, as shown in fig. 3, the screen of the electronic device may display three two-dimensional codes: the first two-dimensional code 310, the second two-dimensional code 320 and the third two-dimensional code 330 are used for enabling a user to wear the head display of the XR device and look at the screen of the electronic device, so that two cameras of the XR device can acquire pictures displayed by the electronic device to obtain two first images, the two first images are slightly different, and each first image can comprise the first two-dimensional code 310, the second two-dimensional code 320 and the third two-dimensional code 330.

The second location information may be three-dimensional coordinates of the positioning identifier in the first coordinate system.

Specifically, for any positioning identifier, the second position information of the positioning identifier can be determined by using a binocular camera ranging algorithm based on slightly different first images acquired by two cameras.

The second position information may be, for example, three-dimensional coordinates of a center point of the two-dimensional code.

The first location information may be three-dimensional coordinates of a screen center point of the electronic device.

As shown in fig. 3, three-dimensional coordinates (x 1, y1, z) of a center point 311 of the first two-dimensional code 310, three-dimensional coordinates (x 2, y2, z 2) of a center point 321 of the second two-dimensional code 320, and three-dimensional coordinates (x 3, y3, z 3) of a center point 331 of the third two-dimensional code 330 are determined based on slightly different first images acquired by two cameras by using a binocular camera ranging algorithm, and three-dimensional coordinates (x 1, y2, z 2) and (x 3, y3, z 3) of a screen center point 340 of the electronic device, that is, first position information can be determined.

In this way, by determining the second position information corresponding to the at least three positioning identifiers respectively and determining the first position information of the electronic device in the first coordinate system based on the at least three second position information, more accurate first position information can be obtained.

Step 230, the first position information is sent to the electronic device, so that the electronic device determines pose information of the electronic device in the first coordinate system according to the first position information and the first pose information of the electronic device.

Here, the first posture information may be an euler angle of the electronic device, and specifically may include a Roll angle (Roll), a Yaw angle (Yaw), and a Pitch angle (Pitch).

For example, the electronic device may have an inertial measurement unit (Inertial Measurement Unit, IMU), and the electronic device may calculate its euler angle (Roll, yaw, pitch) through its IMU.

The pose information may be 6 degrees of freedom (6degree of freedom,6DoF) information, and the 6DoF information may include position information and pose information, that is, three-dimensional coordinates and euler angles.

Here, the XR device may send the first location information to the electronic device, and the electronic device may obtain pose information (x, y, z, roll, yaw, pitch) of the electronic device by integrating the first location information and the first pose information.

Therefore, the electronic device can display a picture comprising the positioning mark, the camera of the XR device can acquire the picture displayed by the electronic device to obtain a first image, then the first position information of the electronic device in the world coordinate system of the XR device can be determined based on the positioning mark in the first image, the first position information is sent to the electronic device, and the electronic device can determine the pose information of the electronic device in the world coordinate system of the XR device according to the first position information and the first pose information of the electronic device. Therefore, a user only needs to aim the camera of the XR equipment at the screen of the electronic equipment, other operations are not needed, pose information of the electronic equipment in the world coordinate system of the XR equipment can be determined, and the method is simple to operate and high in efficiency.

Fig. 4 is a flowchart illustrating another information processing method, the execution subject of which may be an electronic device, according to an exemplary embodiment. The execution body is not limited to the present application.

As shown in fig. 4, the information processing method may include the steps of:

step 410 displays a frame including a positioning identifier to enable a camera of the XR device to acquire the frame, obtain a first image, and determine first location information of the electronic device in a first coordinate system based on the positioning identifier in the first image.

The first coordinate system may be a world coordinate system in which the XR device is located.

Step 420 receives first location information sent by an XR device.

Step 430, determining pose information of the electronic device in the first coordinate system according to the first position information and the first pose information of the electronic device.

The specific process of steps 410-430 may be referred to the above embodiments, and will not be described herein.

In an alternative embodiment, after step 430, the method may further include:

acquiring second posture information of the electronic equipment and first displacement information of the electronic equipment relative to the first position information;

determining third position information of the electronic device in the first coordinate system based on the first displacement information and the first position information;

and updating pose information of the electronic equipment in the first coordinate system according to the second pose information and the third position information.

Here, after determining pose information of the electronic device in the first coordinate system based on the first position information and the first pose information of the electronic device, the electronic device may be moved, and thus the pose information of the electronic device in the first coordinate system may also be updated in real time or periodically.

The second gesture information may be acquired after the first gesture information, and the meaning and the acquiring manner of the second gesture information and the first gesture information may be the same, which are not described herein.

The first displacement information may include a movement distance and a movement direction of the electronic device with respect to the first position information. Based on the first displacement information and the first position information, third position information of the electronic device in the first coordinate system may be determined.

The third position information may be three-dimensional coordinates in the first coordinate system after the electronic device moves the first displacement information on the basis of the first position information.

And the electronic equipment synthesizes the third position information and the second gesture information to obtain new gesture information of the electronic equipment.

In this way, the pose information of the electronic device in the first coordinate system can be updated through the above process, so that the electronic device can be moved at will, rather than being fixed only after the pose information is first determined, and the flexibility of mixed reality shooting (Mixed Reality Capturing, MRC) is improved.

In an alternative embodiment, acquiring the first displacement information of the electronic device compared to the first position information may include:

acquiring fourth position information and fifth position information of the electronic equipment in a second coordinate system;

determining second displacement information of the fifth position information relative to the fourth position information;

the second displacement information is determined as the first displacement information.

Here, the second coordinate system may be a world coordinate system of the electronic device. The electronic device may have a LiDAR sensor, and the process of establishing the second coordinate system by the electronic device may be the same as the process of establishing the first coordinate system by the XR device, which will not be described in detail herein.

The electronic device may determine its own location information in the second coordinate system via the LiDAR sensor.

The fourth location information may correspond to the same time as the first location information, that is, when the location information of the electronic device in the second coordinate system is the fourth location information, the location information of the electronic device in the first coordinate system is the first location information.

Further, the fifth position information may correspond to the same time as the third position information, that is, when the position information of the electronic device in the second coordinate system is the fifth position information, the position information of the electronic device in the first coordinate system is the third position information.

Since the displacement information includes the moving distance and the moving direction, the displacement information is not different from the coordinate system. Thus, the second displacement information of the electronic device in the second coordinate system may be determined as the first displacement information of the electronic device in the first coordinate system.

Therefore, the electronic equipment can accurately determine the displacement information of the electronic equipment in the first coordinate system based on the displacement information of the electronic equipment in the second coordinate system, so that the electronic equipment can accurately update the position information of the electronic equipment.

In an alternative embodiment, the method may further comprise:

Acquiring a first video of a target object through electronic equipment;

and fusing the target object in the first video with the virtual scene video by taking pose information of the electronic equipment in the first coordinate system as pose information of the virtual camera in the virtual scene, so as to obtain the target video.

Here, the target object may include a person and/or an object.

Specifically, the electronic device may collect a first video of the target object through its own camera, and then fuse the target object in the first video with the virtual scene video by using pose information of its own camera in the first coordinate system as pose information of the virtual camera in the virtual scene, to obtain the target video.

The pose information of the electronic device in the first coordinate system may be determined according to the first position information and the first pose information, or may be updated according to the second pose information and the third position information.

The process of acquiring the first video of the target object through the electronic equipment and fusing the target object in the first video with the virtual scene video can be MRC shooting.

Thus, the electronic equipment can be moved at will, so that richer first videos can be obtained, and the target videos can be more wonderful.

In addition, the electronic device can also send the first video and the pose information to the XR device or other electronic devices, and video fusion is carried out by the XR device or other electronic devices to obtain the target video.

By way of example, the other electronic device may be a computer.

To better describe the whole scheme, based on the above embodiments, as a specific example, as shown in fig. 5, the information processing method may include steps 501 to 514, which will be explained in detail below.

In step 501, the electronic device displays a screen including at least three location identifiers.

In step 502, the xr device acquires pictures displayed by the electronic device via at least two cameras, and obtains at least two first images.

At step 503, the xr device determines second location information corresponding to each of the at least three location identifiers based on the at least two first images.

At step 504, the XR device determines first location information of the electronic device in a first coordinate system based on the at least three second location information.

At step 505, the XR device sends the first location information to the electronic device.

In step 506, the electronic device determines pose information of itself in the first coordinate system according to the first position information and the first pose information of itself.

In step 507, the electronic device obtains its second gesture information.

In step 508, the electronic device obtains fourth location information and fifth location information of itself in the second coordinate system.

In step 509, the electronic device determines second displacement information of the fifth location information relative to the fourth location information.

At step 510, the electronic device determines the second displacement information as the first displacement information.

Step 511, the electronic device determines third position information of itself in the first coordinate system based on the first displacement information and the first position information;

in step 512, the electronic device updates its own pose information in the first coordinate system according to the second pose information and the third position information.

In step 513, the electronic device captures a first video of the target object.

In step 514, the electronic device uses the pose information of itself in the first coordinate system as the pose information of the virtual camera in the virtual scene, and fuses the target object in the first video with the virtual scene video to obtain the target video.

It should be noted that, in the embodiment of the present application, the sequence of step 507 and step 508-step 510 is not limited. The specific process from step 501 to step 514 can be referred to the above embodiments, and will not be described herein.

It should be noted that, the application scenario described in the foregoing embodiments of the present application is for more clearly describing the technical solution of the embodiments of the present application, and does not constitute a limitation on the technical solution provided in the embodiments of the present application, and as a person of ordinary skill in the art can know, with the appearance of a new application scenario, the technical solution provided in the embodiments of the present application is also applicable to similar technical problems.

In the information processing method provided in the embodiment of the present application, the execution subject may also be an information processing apparatus. In the embodiments of the present application, an information processing apparatus provided in the embodiments of the present application will be described by taking an example in which the information processing apparatus executes an information processing method.

Based on the same inventive concept as the above information processing method mainly performed by the XR device, the present application also provides an information processing apparatus which can be applied to the XR device. The information processing apparatus provided in the embodiment of the present application is described in detail below with reference to fig. 6.

Fig. 6 is a block diagram showing a structure of an information processing apparatus according to an exemplary embodiment.

As shown in fig. 6, the information processing apparatus 600 may include:

the first acquisition module 601 is configured to acquire, through a camera of the XR device, a picture displayed by the electronic device, to obtain a first image, where the picture includes a positioning identifier;

a first determining module 602, configured to determine first location information of the electronic device in a first coordinate system based on the location identifier in the first image, where the first coordinate system is a world coordinate system of the XR device;

the sending module 603 is configured to send the first position information to the electronic device, so that the electronic device determines pose information of the electronic device in the first coordinate system according to the first position information and the first pose information of the electronic device.

The information processing apparatus 600 will be described in detail below, specifically as follows:

in one embodiment, the frame includes at least three positioning identifiers, and the first acquisition module 601 may include:

The acquisition sub-module is used for acquiring pictures displayed by the electronic equipment through at least two cameras of the XR equipment to obtain at least two first images;

the first determination module 602 may include:

the first determining sub-module is used for determining second position information corresponding to at least three positioning identifiers respectively based on at least two first images;

and the second determining submodule is used for determining first position information of the electronic equipment in the first coordinate system based on at least three pieces of second position information.

Based on the same inventive concept as the above-described information processing method mainly performed by an electronic device, the present application also provides an information processing apparatus that can be applied to an electronic device. The information processing apparatus provided in the embodiment of the present application is described in detail below with reference to fig. 7.

Fig. 7 is a block diagram showing a structure of an information processing apparatus according to an exemplary embodiment.

As shown in fig. 7, the information processing apparatus 700 may include:

the display module 701 is configured to display a frame including a positioning identifier, so that a camera of the XR device acquires the frame to obtain a first image, and determine first position information of the electronic device in a first coordinate system based on the positioning identifier in the first image, where the first coordinate system is a world coordinate system in which the XR device is located;

a receiving module 702, configured to receive first location information sent by an XR device;

the second determining module 703 is configured to determine pose information of the electronic device in the first coordinate system according to the first position information and the first pose information of the electronic device.

In one embodiment, the information processing apparatus 700 may further include:

the electronic equipment comprises an acquisition module, a first position information acquisition module and a second position information acquisition module, wherein the acquisition module is used for acquiring second position information of the electronic equipment and first displacement information of the electronic equipment relative to the first position information after determining the position information of the electronic equipment in a first coordinate system according to the first position information and the first position information of the electronic equipment;

A third determining module, configured to determine third position information of the electronic device in the first coordinate system based on the first displacement information and the first position information;

and the updating module is used for updating the pose information of the electronic equipment in the first coordinate system according to the second pose information and the third position information.

In one embodiment, the acquiring module may include:

the acquisition sub-module is used for acquiring fourth position information and fifth position information of the electronic equipment in a second coordinate system, wherein the second coordinate system is a world coordinate system of the electronic equipment, and the fourth position information corresponds to the same time as the first position information;

a third determination sub-module for determining second displacement information of the fifth position information with respect to the fourth position information;

and a fourth determination sub-module for determining the second displacement information as the first displacement information.

the second acquisition module is used for acquiring a first video of the target object through the electronic equipment;

and the fusion module is used for fusing the target object in the first video with the virtual scene video by taking the pose information of the electronic equipment in the first coordinate system as the pose information of the virtual camera in the virtual scene, so as to obtain the target video.

The information processing apparatus in the embodiment of the present application may be an electronic device, or may be a component in an electronic device, for example, an integrated circuit or a chip. The electronic device may be a terminal, or may be other devices than a terminal. By way of example, the electronic device may be a mobile phone, tablet computer, notebook computer, palm computer, vehicle-mounted electronic device, mobile internet appliance (Mobile Internet Device, MID), augmented reality (augmented reality, AR)/Virtual Reality (VR) device, robot, wearable device, ultra-mobile personal computer, UMPC, netbook or personal digital assistant (personal digital assistant, PDA), etc., but may also be a server, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (TV), teller machine or self-service machine, etc., and the embodiments of the present application are not limited in particular.

The information processing apparatus in the embodiment of the present application may be an apparatus having an operating system. The operating system may be an Android operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

The information processing apparatus provided in this embodiment of the present application can implement each process implemented by the embodiments of the methods of fig. 1 to 5, and achieve the same technical effects, so that repetition is avoided, and no further description is given here.

Optionally, as shown in fig. 8, the embodiment of the present application further provides an electronic device 800, including a processor 801 and a memory 802, where a program or an instruction capable of running on the processor 801 is stored in the memory 802, and the program or the instruction implements each step of the embodiment of the information processing method when executed by the processor 801, and the steps achieve the same technical effects, so that repetition is avoided, and no further description is given here.

The electronic device in the embodiment of the application includes the mobile electronic device and the non-mobile electronic device described above.

The electronic device 900 includes, but is not limited to: radio frequency unit 901, network module 902, audio output unit 903, input unit 904, sensor 905, display unit 906, user input unit 907, interface unit 908, memory 909, and processor 910.

Those skilled in the art will appreciate that the electronic device 900 may also include a power source (e.g., a battery) for powering the various components, which may be logically connected to the processor 910 by a power management system to perform functions such as managing charge, discharge, and power consumption by the power management system. The electronic device structure shown in fig. 9 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than shown, or may combine certain components, or may be arranged in different components, which are not described in detail herein.

In one case, when electronic device 900 is XR device 11, the components may perform the following functions:

wherein, the processor 910 is configured to acquire, through a camera of the XR device, a picture displayed by the electronic device 12, to obtain a first image, where the picture includes a positioning identifier;

determining first location information of the electronic device 12 in a first coordinate system based on the location identity in the first image, the first coordinate system being a world coordinate system of the XR device;

the first location information is sent to the electronic device 12 such that the electronic device 12 determines pose information of the electronic device in a first coordinate system based on the first location information and the first pose information of the electronic device.

Optionally, the frame includes at least three positioning identifiers, and the processor 910 is further configured to acquire, through at least two cameras of the XR device, frames displayed by the electronic device, to obtain at least two first images;

In one case, when electronic device 900 is electronic device 12, the components may perform the following functions:

the display unit 906 is configured to display a frame including a positioning identifier, so that a camera of the XR device acquires the frame to obtain a first image, and determine first position information of the electronic device in a first coordinate system based on the positioning identifier in the first image, where the first coordinate system is a world coordinate system where the XR device is located;

a processor 910 configured to receive first location information sent by the XR device;

Optionally, the processor 910 is further configured to obtain second pose information of the electronic device and first displacement information of the electronic device relative to the first position information after determining pose information of the electronic device in the first coordinate system according to the first position information and the first pose information of the electronic device;

Optionally, the processor 910 is further configured to obtain fourth position information and fifth position information of the electronic device in a second coordinate system, where the second coordinate system is a world coordinate system of the electronic device, and the fourth position information corresponds to the same time as the first position information;

Therefore, the electronic equipment can only accurately determine the displacement information of the electronic equipment in the first coordinate system based on the displacement information of the electronic equipment in the second coordinate system, so that the electronic equipment can accurately update the position information of the electronic equipment.

Optionally, the processor 910 is further configured to acquire, by the electronic device, a first video of the target object;

It should be appreciated that in embodiments of the present application, the input unit 904 may include a graphics processor (Graphics Processing Unit, GPU) 9041 and a microphone 9042, with the graphics processor 9041 processing image data of still pictures or video obtained by an image capture device (e.g., a camera) in a video capture mode or an image capture mode. The display unit 906 may include a display panel 9061, and the display panel 9061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. Touch panel 9071, also referred to as a touch screen. The touch panel 9071 may include two parts, a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and so forth, which are not described in detail herein.

The memory 909 may be used to store software programs as well as various data. The memory 909 may mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area may store an operating system, application programs or instructions (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like. Further, the memory 909 may include a volatile memory or a nonvolatile memory, or the memory 909 may include both volatile and nonvolatile memories. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM), static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (ddr SDRAM), enhanced SDRAM (Enhanced SDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DRRAM). Memory 909 in embodiments of the present application includes, but is not limited to, these and any other suitable types of memory.

Processor 910 may include one or more processing units; optionally, the processor 910 integrates an application processor that primarily processes operations involving an operating system, user interface, application programs, and the like, and a modem processor that primarily processes wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into the processor 910.

The embodiment of the present application further provides a readable storage medium, where a program or an instruction is stored, and when the program or the instruction is executed by a processor, the processes of the embodiment of the information processing method are implemented, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here.

Wherein the processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes computer readable storage medium such as read-only memory, random access memory, magnetic disk or optical disk.

The embodiment of the application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled with the processor, and the processor is used for running a program or an instruction, so as to implement each process of the embodiment of the information processing method, and achieve the same technical effect, so that repetition is avoided, and no redundant description is provided here.

It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.

The embodiments of the present application provide a computer program product stored in a storage medium, where the program product is executed by at least one processor to implement the respective processes of the embodiments of the information processing method described above, and achieve the same technical effects, and for avoiding repetition, a detailed description is omitted herein.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solutions of the present application may be embodied essentially or in a part contributing to the prior art in the form of a computer software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the methods described in the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those of ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are also within the protection of the present application.

Claims

1. An information processing method for use with an augmented reality XR device, the method comprising:

and sending the first position information to the electronic equipment so that the electronic equipment can determine pose information of the electronic equipment in the first coordinate system according to the first position information and the first pose information of the electronic equipment.

2. The method of claim 1, wherein the frame includes at least three positioning marks, and wherein the capturing, by the camera of the XR device, the frame displayed by the electronic device to obtain the first image comprises:

the determining, based on the positioning identifier in the first image, first location information of the electronic device in a first coordinate system includes:

Determining second position information corresponding to the at least three positioning identifiers respectively based on the at least two first images;

first position information of the electronic device in the first coordinate system is determined based on at least three pieces of the second position information.

3. An information processing method, characterized by being applied to an electronic device, comprising:

displaying a picture comprising a positioning mark, so that a camera of an XR device acquires the picture to obtain a first image, and determining first position information of the electronic device in a first coordinate system based on the positioning mark in the first image, wherein the first coordinate system is a world coordinate system in which the XR device is located;

receiving first position information sent by the XR equipment;

and determining pose information of the electronic equipment in the first coordinate system according to the first position information and the first pose information of the electronic equipment.

4. A method according to claim 3, wherein after said determining pose information of said electronic device in said first coordinate system based on said first position information and first pose information of said electronic device, said method further comprises:

5. The method of claim 4, wherein obtaining first displacement information of the electronic device as compared to the first position information comprises:

acquiring fourth position information and fifth position information of the electronic equipment in a second coordinate system, wherein the second coordinate system is a world coordinate system of the electronic equipment, and the fourth position information corresponds to the same time as the first position information;

6. The method according to any one of claims 3-5, further comprising:

acquiring a first video of a target object through the electronic equipment;

And taking pose information of the electronic equipment in the first coordinate system as pose information of a virtual camera in a virtual scene, and fusing the target object in the first video with the virtual scene video to obtain a target video.

7. An information processing apparatus for use with an augmented reality XR device, the apparatus comprising:

a first determining module, configured to determine, based on the positioning identifier in the first image, first location information of the electronic device in a first coordinate system, where the first coordinate system is a world coordinate system of the XR device;

and the sending module is used for sending the first position information to the electronic equipment so that the electronic equipment can determine pose information of the electronic equipment in the first coordinate system according to the first position information and the first pose information of the electronic equipment.

8. The apparatus of claim 7, wherein the frame includes at least three positioning identifiers, and wherein the first acquisition module includes:

the first determining module includes:

the first determining submodule is used for determining second position information corresponding to the at least three positioning identifiers respectively based on the at least two first images;

9. An information processing apparatus, characterized by being applied to an electronic device, comprising:

10. The apparatus of claim 9, wherein the apparatus further comprises:

the electronic equipment comprises an acquisition module, a first position information acquisition module and a second position information acquisition module, wherein the acquisition module is used for acquiring second position information of the electronic equipment and first displacement information of the electronic equipment relative to the first position information after determining the position information of the electronic equipment in the first coordinate system according to the first position information and the first position information of the electronic equipment;

11. The apparatus of claim 10, wherein the acquisition module comprises:

the electronic equipment comprises an acquisition sub-module, a first position information acquisition sub-module and a second position information acquisition sub-module, wherein the acquisition sub-module is used for acquiring fourth position information and fifth position information of the electronic equipment in a second coordinate system, the second coordinate system is a world coordinate system of the electronic equipment, and the fourth position information corresponds to the same time as the first position information;

A third determination sub-module for determining second displacement information of the fifth position information relative to the fourth position information;

and a fourth determination sub-module configured to determine the second displacement information as the first displacement information.

12. The apparatus according to any one of claims 9-11, wherein the apparatus further comprises:

and the fusion module is used for fusing the target object in the first video with the virtual scene video by taking the pose information of the electronic equipment in the first coordinate system as the pose information of the virtual camera in the virtual scene, so as to obtain a target video.

13. An electronic device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the information processing method of any of claims 1-6.