CN113034697A - Method, device and system for augmented reality display of target equipment - Google Patents

Method, device and system for augmented reality display of target equipment Download PDF

Info

Publication number
CN113034697A
CN113034697A CN201911350202.8A CN201911350202A CN113034697A CN 113034697 A CN113034697 A CN 113034697A CN 201911350202 A CN201911350202 A CN 201911350202A CN 113034697 A CN113034697 A CN 113034697A
Authority
CN
China
Prior art keywords
information
augmented reality
distance
target device
display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911350202.8A
Other languages
Chinese (zh)
Inventor
李剑华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing CHJ Automobile Technology Co Ltd
Original Assignee
Beijing CHJ Automobile Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing CHJ Automobile Technology Co Ltd filed Critical Beijing CHJ Automobile Technology Co Ltd
Priority to CN201911350202.8A priority Critical patent/CN113034697A/en
Publication of CN113034697A publication Critical patent/CN113034697A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The embodiment of the disclosure discloses a method, a device and a system for augmented reality display of target equipment, relates to the technical field of automobiles, and can solve the problem that in the prior art, the effect of displaying an automobile to a client by adopting a brochure is poor. The method of the embodiment of the disclosure mainly comprises: acquiring the azimuth information of a camera of the terminal equipment relative to the target equipment; acquiring virtual display models corresponding to the orientation information, wherein the virtual display models corresponding to at least two different orientation information are respectively an internal display model and an external display model of the target equipment; and displaying the virtual display model on a display screen of the terminal equipment according to the orientation information. In the embodiment of the application, the user can check the internal display model and the external display model of the target equipment through the terminal equipment according to the position information of the camera of the terminal equipment relative to the target equipment, and in the checking process, the client checks the internal and external structures of the target equipment more intuitively.

Description

Method, device and system for augmented reality display of target equipment
Technical Field
The embodiment of the disclosure relates to the technical field of automobiles, in particular to a method, a device and a system for augmented reality display of target equipment.
Background
Automobiles have been widely popularized to various households, and the requirements of customers on the functions of automobiles, such as entertainment, automation, comfort and the like, are more and more strong without being limited to the simple function of riding instead of walking.
In order to meet the requirements of the customers for the automobiles, automobile manufacturers increasingly make automobiles with rich functions. In order to promote various functions of the own brand of automobile, the functions of the automobile are usually printed in an introduction brochure, and the introduction brochure is issued to the customer so as to show the functions of the automobile to the customer.
However, the introduction brochure can only show various functions of the automobile in a two-dimensional picture and character expression mode, and for a customer, the mode of knowing the automobile through the brochure is poor in understanding effect and not visual enough.
Disclosure of Invention
The method, the device and the system for augmented reality display of the target equipment are provided for solving the problem that the effect of displaying the automobile to a customer by adopting a brochure is poor in the prior art, so that the display effect of the automobile is improved.
The embodiment of the disclosure mainly provides the following technical scheme:
in a first aspect, an embodiment of the present disclosure provides a method for performing augmented reality display on a target device, including:
acquiring the azimuth information of a camera of the terminal equipment relative to the target equipment;
acquiring virtual display models corresponding to the orientation information, wherein the virtual display models corresponding to at least two different orientation information are respectively an internal display model and an external display model of the target equipment;
and displaying the virtual display model on a display screen of the terminal equipment according to the orientation information.
In some embodiments, the orientation information includes distance information, and the obtaining of the virtual representation model corresponding to the orientation information includes:
when the distance information is a first distance, acquiring an external display model corresponding to the first distance;
when the distance information is a second distance, acquiring an internal display model corresponding to the second distance; the first distance is greater than the second distance.
In some embodiments, the obtaining of the virtual presentation model corresponding to the orientation information includes:
and downloading the virtual display model corresponding to the orientation information from a server.
In some embodiments, before the obtaining the position information of the camera of the terminal device relative to the target device, the method includes:
receiving an identification request of a user;
if the identification request is an information display request, executing the acquisition of the orientation information of the camera of the terminal equipment relative to the target equipment;
if the identification request is a fault identification request, acquiring depth image information of the target equipment, analyzing current three-dimensional physical structure state information of a component of the target equipment contained in the depth image information, acquiring normal three-dimensional physical structure state information corresponding to the component, and carrying out fault identification on the component according to the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information.
In some embodiments, the acquiring the normal three-dimensional physical structure state information corresponding to the component includes:
and downloading the normal three-dimensional physical structure state information corresponding to the member from a server.
In some embodiments, said identifying the failure of the component based on the current three-dimensional physical structure status information and the normal three-dimensional physical structure status information comprises:
comparing the current three-dimensional physical structure state information with the normal three-dimensional physical structure state information;
and if the current three-dimensional physical structure state information is inconsistent with the normal three-dimensional physical structure state information by comparison, judging that the component has a fault.
In some embodiments, the component is a plurality of components, and the identifying the fault of the component according to the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information further includes:
and marking the component which is judged to be in failure.
In some embodiments, the identifying the component for the fault based on the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information further comprises:
uploading the component marked with the failure to a server.
In some embodiments, further comprising:
receiving a rotation instruction of a user;
and adjusting the direction of the virtual display model displayed on the display screen of the terminal equipment according to the rotation instruction.
In a second aspect, an embodiment of the present disclosure provides an apparatus for performing augmented reality display on a target device, including:
the orientation information acquisition unit is used for acquiring orientation information of the camera of the terminal equipment relative to the target equipment;
a virtual display model obtaining unit, configured to obtain a virtual display model corresponding to the orientation information, where the virtual display models corresponding to at least two different orientation information are an internal display model and an external display model of the target device, respectively;
and the display unit is used for displaying the virtual display model on a display screen of the terminal equipment according to the orientation information.
In some embodiments, the orientation information includes distance information, and the virtual presentation model obtaining unit includes:
the first obtaining module is used for obtaining an external display model corresponding to a first distance when the distance information is the first distance;
the second obtaining module is used for obtaining an internal display model corresponding to a second distance when the distance information is the second distance; the first distance is greater than the second distance.
In some embodiments, the virtual presentation model obtaining unit includes:
and the first downloading module is used for downloading the virtual display model corresponding to the direction information from a server.
In some embodiments, further comprising:
a first receiving unit for receiving an identification request of a user;
the orientation information acquisition unit is used for executing the acquisition of the orientation information of the camera of the terminal equipment relative to the target equipment if the identification request is an information display request;
and the fault identification unit is used for acquiring the depth image information of the target equipment if the identification request is a fault identification request, analyzing the current three-dimensional physical structure state information of the member of the target equipment contained in the depth image information, acquiring the normal three-dimensional physical structure state information corresponding to the member, and identifying the fault of the member according to the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information.
In some embodiments, the fault identification unit comprises:
and the second downloading module is used for downloading the normal three-dimensional physical structure state information corresponding to the component from the server.
In some embodiments, the fault identification unit comprises:
the comparison module is used for comparing the current three-dimensional physical structure state information with the normal three-dimensional physical structure state information;
and the judging module is used for judging that the component has a fault if the current three-dimensional physical structure state information is inconsistent with the normal three-dimensional physical structure state information by comparison.
In some embodiments, the member is a plurality of members, and the fault identification unit further includes:
and the marking module is used for marking the component which is judged to have the fault.
In some embodiments, the fault identification unit further comprises:
and the uploading module is used for uploading the fault marked component to the server.
In some embodiments, further comprising:
the second receiving unit is used for receiving a rotation instruction of a user;
and the adjusting unit is used for adjusting the direction of the virtual display model displayed on the display screen of the terminal equipment according to the rotating instruction.
In a third aspect, an embodiment of the present disclosure provides a storage medium, where the storage medium includes a stored program, and when the program runs, a device in which the storage medium is located is controlled to execute the method for performing augmented reality display on a target device according to the first aspect.
In a fourth aspect, an embodiment of the present disclosure provides an apparatus for performing augmented reality display on a target device, where the apparatus includes a storage medium; and one or more processors, the storage medium coupled with the processors, the processors configured to execute program instructions stored in the storage medium; the program instructions, when executed, perform the method for augmented reality display of a target device of the first aspect.
In a fifth aspect, an embodiment of the present disclosure provides a system for performing augmented reality display on a target device, including:
the device for augmented reality display of the target equipment and the server in the fourth aspect;
the device for performing augmented reality display on the target equipment is used for sending an acquisition request of a virtual display model corresponding to the azimuth information to the server;
and the server is used for sending a virtual display model corresponding to the orientation information to the device for performing augmented reality display on the target equipment according to the acquisition request.
By the technical scheme, the method, the device and the system for performing augmented reality display on the target equipment, which are provided by the technical scheme of the invention, have the following advantages at least:
in the technical scheme provided by the embodiment of the disclosure, in displaying the target equipment to the client, the virtual display model of the target equipment can be used for displaying to the client, and the internal display model and the external display model of the target equipment are selected according to the orientation information of the camera of the terminal equipment relative to the target equipment and displayed on the display screen of the terminal equipment. Compared with the mode of adopting pictures and texts in the prior art, in the embodiment of the application, a user can check the internal display model and the external display model of the target equipment through the terminal equipment according to the change of the orientation information of the camera of the terminal equipment relative to the target equipment, and in the checking process, the user can check the internal and external structures of the target equipment more intuitively.
The foregoing description is only an overview of the embodiments of the present disclosure, and in order to make the technical means of the embodiments of the present disclosure more clearly understood, the embodiments of the present disclosure may be implemented in accordance with the content of the description, and in order to make the foregoing and other objects, features, and advantages of the embodiments of the present disclosure more clearly understood, the following detailed description of the embodiments of the present disclosure is given.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the embodiments of the present disclosure. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 shows a flowchart of a method for augmented reality exhibition of a target device according to an embodiment of the present disclosure;
fig. 2 shows a flowchart of a specific method for augmented reality exhibition of a target device according to an embodiment of the present disclosure;
fig. 3 is a flowchart illustrating a further specific method for augmented reality presentation of a target device according to an embodiment of the present disclosure;
fig. 4 illustrates an example diagram of an apparatus for augmented reality presentation of a target device according to an embodiment of the present disclosure;
fig. 5 is an exemplary diagram illustrating a specific apparatus for augmented reality presentation of a target device according to an embodiment of the present disclosure;
fig. 6 shows an example diagram of a specific system for augmented reality presentation of a target device according to an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In a first aspect, as shown in fig. 1, an embodiment of the present disclosure provides a method for performing augmented reality display on a target device, including:
101, acquiring orientation information of a camera of a terminal device relative to a target device;
the terminal equipment can be a mobile phone, a tablet computer, intelligent glasses, head-mounted display equipment and the like. The terminal equipment's equipment body is provided with camera and display screen.
The target device may be a vehicle, a household appliance, or the like, which occupies a commodity in a three-dimensional physical space.
When a client needs to know the target equipment, the client can hold or wear the terminal equipment, the camera of the terminal equipment is used for shooting the target equipment, the image information of the target equipment is collected, and the azimuth information of the camera of the equipment relative to the target equipment is analyzed according to the image information, wherein the azimuth information can comprise a direction and an angle. It is to be understood that the method of analyzing the orientation information is not limited to the above-described embodiment, and any other acquisition means may be used.
For example, application software ARKit for developing augmented reality in the IOS terminal equipment can be adopted, wherein the ARKit provides a world tracking function, the world tracking function uses a visual inertial odometer and camera images and motion data, the rotation degree, the relative position and the real world scale of the terminal equipment can be provided, and the motion data of the terminal equipment can calculate the physical movement distance. The acquisition of the orientation information can be realized through application software ARKit.
102, acquiring virtual display models corresponding to the orientation information, wherein the virtual display models corresponding to at least two different orientation information are an internal display model and an external display model of the target device respectively;
the virtual display model may correspond to a preset three-dimensional model for the target device, the target device may be a commodity to be sold, and an internal display model and an external display model for displaying to the customer may be set for the target device.
It is easily understood that, among others, the number of external presentation models is not limited to one, and for a target device having a variety of external styles, a plurality of external presentation models may be provided, for example, different external presentation models correspond to models having different orientation information of the target device. Taking the target device as a vehicle as an example, the exterior display model of the vehicle may include a first color vehicle exterior display model, a second color vehicle exterior display model, a first configuration vehicle exterior display model, a second configuration vehicle exterior display model, and the like.
In the implementation of obtaining the virtual display model corresponding to the orientation information, the virtual display model corresponding to the direction may be obtained, and the virtual display models corresponding to different directions are different, for example, the first direction corresponds to the first color vehicle exterior display model, and the second direction corresponds to the second color vehicle exterior display model. For another example, the virtual display model corresponding to the angle may be obtained, and the virtual display models corresponding to different angles are different, for example, the first angle corresponds to the first configuration vehicle exterior display model, and the second angle corresponds to the second configuration vehicle exterior display model.
In some embodiments, the orientation information includes distance information, and in the implementation of obtaining the virtual display model corresponding to the orientation information, a virtual display model corresponding to the distance information may be obtained, where the virtual display models corresponding to different distance information are different (where, distance information may be understood as one section, and the virtual display models corresponding to distances of different sections are different), for example, when the distance information is a first distance, an external display model corresponding to the first distance is obtained; when the distance information is a second distance, acquiring an internal display model corresponding to the second distance; the first distance is greater than the second distance.
It is easy to understand that, the number of internal presentation models is not limited to one, and for a target device with a complicated internal presentation, a plurality of internal presentation models may be provided, for example, different internal presentation models correspond to components with different orientation information of the target device. Taking the target device as a vehicle as an example, the internal display model of the vehicle may include a vehicle engine display model, a vehicle seat display model, a vehicle chassis display model, a vehicle trunk display model, and the like.
For example, when the distance information is a third distance, obtaining an internal display model corresponding to the third distance; the second distance is greater than the third distance. That is, in the process that the user approaches the target device, the information of the target device from the outside to the inside can be gradually displayed on the display screen of the terminal device.
In some embodiments, the virtual presentation model may be pre-stored in the terminal device. In implementation, on one hand, the virtual display model occupies a large storage space, and on the other hand, the virtual display model is updated at a high frequency, so that a large amount of update loss (time, power consumption, flow and the like) of the terminal device needs to be consumed, the virtual display model is stored in the server, the terminal device sends a request for obtaining the virtual display model corresponding to the orientation information to the server, and the server sends the virtual display model corresponding to the orientation information to the terminal device according to the request, so that the terminal device downloads the virtual display model corresponding to the orientation information from the server, wherein the downloaded data volume is small due to limited content of target devices watched by a user, and the server can update latest virtual display model data periodically, so that the client can view the latest virtual display model.
103, displaying the virtual display model on a display screen of the terminal device according to the orientation information.
Along with the adjustment of the orientation information, the proportion, the display orientation and the position of the virtual display model are correspondingly adjusted on the display screen.
In implementation, the properties or characteristics of the environment surrounding the terminal device may be determined by the application software ARKit. An intersection with the real world topology is obtained for placing the virtual presentation model in the physical world. And meanwhile, light ray estimation is carried out for correctly illuminating the virtual display model, so that the virtual display model is matched with the physical world, and finally the virtual display model is integrated into the physical environment to be displayed on a display screen.
In the technical scheme provided by the embodiment of the disclosure, in displaying the target equipment to the client, the virtual display model of the target equipment can be used for displaying to the client, and the internal display model and the external display model of the target equipment are selected according to the orientation information of the camera of the terminal equipment relative to the target equipment and displayed on the display screen of the terminal equipment. Compared with the mode of adopting pictures and texts in the prior art, in the embodiment of the application, a user can check the internal display model and the external display model of the target equipment through the terminal equipment according to the change of the orientation information of the camera of the terminal equipment relative to the target equipment, and in the checking process, the user can check the internal and external structures of the target equipment more intuitively.
Further, after the virtual display model is displayed on the display screen according to the orientation information, a user can input a rotation instruction to the terminal device, the rotation instruction can be input in a sliding operation mode, a rotation button is clicked, a voice input mode and the like, and the terminal device receives the rotation instruction of the user; and adjusting the direction of the virtual display model displayed on the display screen of the terminal equipment according to the rotation instruction. For example, if the rotation command is a left turn, the virtual display model displayed on the display screen is left turned, and if the rotation command is a right turn, the virtual display model displayed on the display screen is right turned. The client can observe the multi-directional information of the displayed virtual display model conveniently at a fixed-point position.
In a second aspect, as shown in fig. 2, an embodiment of the present disclosure provides a method for performing augmented reality display on a target device, where the method further includes a function of identifying a fault of the target device in addition to a function of enabling virtual model display of the target device in the first aspect, and the method includes:
201 receiving an identification request of a user;
the identification request may be an information display request input by a user, or the identification request may be a fault identification request input by the user, and the input mode of the identification request may be an input mode such as a sliding operation, a click rotation button, a voice input, and the like, which is not limited in this embodiment.
202, if the identification request is an information display request, acquiring the orientation information of the camera of the terminal equipment relative to the target equipment; acquiring virtual display models corresponding to the orientation information, wherein the virtual display models corresponding to at least two different orientation information are respectively an internal display model and an external display model of the target equipment; and displaying the virtual display model on a display screen of the terminal equipment according to the orientation information.
In implementation, for a specific implementation manner of step 202, reference may be made to the specific description of the method for performing augmented reality display on the target device in the first aspect, which is not described in detail in this embodiment.
And 203, if the identification request is a fault identification request, acquiring depth image information of the target device, analyzing current three-dimensional physical structure state information of a member of the target device, which is contained in the depth image information, acquiring normal three-dimensional physical structure state information corresponding to the member, and performing fault identification on the member according to the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information.
The depth image information is a depth image (depth image), which is also called range image (range image), and may be an image having distances (depths) from a camera (depth camera) to respective points of the target device as pixel values, which directly reflects the geometric shape of the visible surface of the target device. The depth image information can be calculated into point cloud data through coordinate conversion, and the point cloud data can also be inversely calculated into the depth image information.
In an implementation, the depth image information of the target device is acquired through the camera, and the acquired depth image information is not limited to the information of the external position of the target device, but may also be the information of the internal position of the target device. Taking the target device as an example of a vehicle, when the vehicle has a fault, a driver can directly use the terminal device to shoot a suspected problem position, such as an engine, to obtain depth image information of the engine position, and components of the engine can be analyzed through the depth image information of the position. For another example, after the driver has detached the inner wire of the vehicle by himself or herself, in order to verify whether the installation of the wire is correct or when the vehicle cannot be started normally after the wire is installed, the terminal device may be used to acquire a newly installed wire portion, and the current three-dimensional physical structure state information of the wire member (which may reflect the connection relationship of the wire) may be recognized by the acquired depth image.
The normal three-dimensional physical structure state information is state information of the component in a normal state, and can be preset and stored for being compared with the current three-dimensional physical structure state information.
In some embodiments, the normal three-dimensional physical structure state information may be stored in the terminal device in advance. The method comprises the steps that the number of normal three-dimensional physical structure state information is large, a large storage space is occupied, in implementation, the normal three-dimensional physical structure state information can be stored in a server, a terminal device sends a request for obtaining the normal three-dimensional physical structure state information corresponding to a component to the server, the server sends the normal three-dimensional physical structure state information corresponding to the component to the terminal device according to the request, and therefore the terminal device downloads the normal three-dimensional physical structure state information corresponding to the component from the server.
In some embodiments, as shown in fig. 3, the identifying the component failure according to the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information includes:
2031 comparing the current three-dimensional physical structure state information with the normal three-dimensional physical structure state information;
2032 if the current three-dimensional physical structure state information is not consistent with the normal three-dimensional physical structure state information, determining that the component has a fault.
The number of the components is multiple, the current three-dimensional physical structure state information of each component can be compared with the normal three-dimensional physical structure state information, and 2033 the component which is judged to have a fault is marked.
The marking mode may be a table mode, a mode of marking on a picture, or the like, and the embodiment of the marking of the present invention is not limited.
2034 uploads the component marked for failure to the server.
The engineer at the remote end of the server can analyze the failure solution and feed the failure solution back to the failure uploader after receiving the feedback mark, so that the failure uploader can know the solution,
and if the current three-dimensional physical structure state information is consistent with the normal three-dimensional physical structure state information by comparison, judging that the component does not have a fault.
According to the technical scheme provided by the embodiment of the disclosure, besides the function of displaying the virtual model of the target device, the fault recognition function of the target device is also provided, and the corresponding function can be realized according to the function selection of the user. For example, in presenting the target device to the client, a virtual presentation model of the target device may be used to present the target device to the client, and an internal presentation model and an external presentation model of the target device may be selected according to the orientation information of the camera of the terminal device relative to the target device and presented on the display screen of the terminal device. Compared with the mode of adopting pictures and texts in the prior art, in the embodiment of the application, a user can check the internal display model and the external display model of the target equipment through the terminal equipment according to the change of the orientation information of the camera of the terminal equipment relative to the target equipment, and in the checking process, the user can check the internal and external structures of the target equipment more intuitively. For example, after the target device has failed, the client can use the terminal device to photograph the suspected location of the failure, thereby enabling automatic troubleshooting.
In a third aspect, according to the methods shown in fig. 1 to fig. 3, another embodiment of the present disclosure further provides an apparatus for performing augmented reality display on a target device, as shown in fig. 4, the apparatus mainly includes:
an orientation information acquiring unit 10, configured to acquire orientation information of a camera of a terminal device with respect to a target device;
a virtual display model obtaining unit 20, configured to obtain a virtual display model corresponding to the orientation information, where the virtual display models corresponding to at least two different orientation information are an internal display model and an external display model of the target device, respectively;
and the display unit 30 is configured to display the virtual display model on a display screen of the terminal device according to the orientation information.
In some embodiments, the orientation information includes distance information, and as shown in fig. 5, the virtual representation model obtaining unit 20 includes:
a first obtaining module 21, configured to obtain, when the distance information is a first distance, an external display model corresponding to the first distance;
a second obtaining module 22, configured to obtain, when the distance information is a second distance, an internal display model corresponding to the second distance; the first distance is greater than the second distance.
In some embodiments, as shown in fig. 5, the virtual representation model obtaining unit 20 includes:
and a first downloading module 23, configured to download, from the server, the virtual display model corresponding to the orientation information.
In some embodiments, further comprising:
a first receiving unit 40 for receiving an identification request of a user;
the orientation information obtaining unit 10 is configured to, if the identification request is an information display request, perform obtaining of orientation information of the terminal device camera with respect to the target device;
a fault identifying unit 50, configured to, if the identification request is a fault identification request, obtain depth image information of the target device, analyze current three-dimensional physical structure state information of a component of the target device, which is included in the depth image information, obtain normal three-dimensional physical structure state information corresponding to the component, and perform fault identification on the component according to the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information.
In some embodiments, as shown in fig. 5, the fault identifying unit 50 includes:
and the second downloading module 51 is used for downloading the normal three-dimensional physical structure state information corresponding to the component from the server.
In some embodiments, as shown in fig. 5, the fault identifying unit 50 includes:
a comparing module 52, configured to compare the current three-dimensional physical structure state information with the normal three-dimensional physical structure state information;
and the judging module 53 is configured to judge that the component has a fault if the current three-dimensional physical structure state information is inconsistent with the normal three-dimensional physical structure state information by comparison.
In some embodiments, the component is a plurality of components, and the fault identification unit 50 further includes:
and a marking module 54 for marking the component determined to be faulty.
In some embodiments, the fault identification unit further comprises:
and an uploading module 55, configured to upload the component marked with the failure to the server.
In some embodiments, as shown in fig. 5, further comprising:
a second receiving unit 60 for receiving a rotation instruction of a user;
and an adjusting unit 70, configured to adjust, according to the rotation instruction, a direction in which the virtual display model is displayed on the display screen of the terminal device.
The device comprises a processor and a storage medium, wherein the azimuth information acquisition unit, the virtual display model acquisition unit, the display unit, the first receiving unit, the fault identification unit, the second receiving unit, the adjustment unit and the like are stored in the storage medium as program units, and the processor executes the program units stored in the storage medium to realize corresponding functions.
The processor comprises a kernel, and the kernel calls a corresponding program unit from a storage medium. One or more cores may be provided.
In the technical scheme provided by the embodiment of the disclosure, in displaying the target equipment to the client, the virtual display model of the target equipment can be used for displaying to the client, and the internal display model and the external display model of the target equipment are selected according to the orientation information of the camera of the terminal equipment relative to the target equipment and displayed on the display screen of the terminal equipment. Compared with the mode of adopting pictures and texts in the prior art, in the embodiment of the application, a user can check the internal display model and the external display model of the target equipment through the terminal equipment according to the change of the orientation information of the camera of the terminal equipment relative to the target equipment, and in the checking process, the user can check the internal and external structures of the target equipment more intuitively.
The apparatus for performing augmented reality display on a target device provided in an embodiment of the third aspect may be configured to perform the method for performing augmented reality display on a target device provided in an embodiment of the first aspect or the second aspect, and the related meanings and specific implementations of the related apparatus may refer to the related descriptions in the embodiment of the first aspect or the second aspect, and are not described in detail here.
In a fourth aspect, an embodiment of the present disclosure provides a storage medium, where the storage medium includes a stored program, and when the program runs, the device where the storage medium is located is controlled to execute the method for performing augmented reality display on a target device according to the first aspect or the second aspect.
The storage medium may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
In a fifth aspect, an embodiment of the present disclosure provides an apparatus for performing augmented reality display on a target device, where the apparatus includes a storage medium; and one or more processors, the storage medium coupled with the processors, the processors configured to execute program instructions stored in the storage medium; the program instructions, when executed, perform the method for augmented reality presentation of a target device according to the first aspect or the second aspect.
In a sixth aspect, an embodiment of the present disclosure provides a system for performing augmented reality display on a target device, as shown in fig. 6, including: the apparatus 1000 and the server 2000 for augmented reality display of a target device according to the fifth aspect are provided. The apparatus 1000 for performing augmented reality display on a target device is configured to send an acquisition request of a virtual display model corresponding to the orientation information to the server 2000, and the server 2000 is configured to send the virtual display model corresponding to the orientation information to the apparatus 1000 for performing augmented reality display on the target device according to the acquisition request.
As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, embodiments of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, embodiments of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (15)

1. A method for augmented reality display of a target device, comprising:
acquiring the azimuth information of a camera of the terminal equipment relative to the target equipment;
acquiring virtual display models corresponding to the orientation information, wherein the virtual display models corresponding to at least two different orientation information are respectively an internal display model and an external display model of the target equipment;
and displaying the virtual display model on a display screen of the terminal equipment according to the orientation information.
2. The method of claim 1, wherein the orientation information comprises distance information, and the obtaining a virtual representation model corresponding to the orientation information comprises:
when the distance information is a first distance, acquiring an external display model corresponding to the first distance;
when the distance information is a second distance, acquiring an internal display model corresponding to the second distance; the first distance is greater than the second distance.
3. The method for augmented reality presentation of a target device according to claim 1, wherein before the obtaining of the orientation information of the camera of the terminal device relative to the target device, the method comprises:
receiving an identification request of a user;
if the identification request is an information display request, executing the acquisition of the orientation information of the camera of the terminal equipment relative to the target equipment;
if the identification request is a fault identification request, acquiring depth image information of the target equipment, analyzing current three-dimensional physical structure state information of a component of the target equipment contained in the depth image information, acquiring normal three-dimensional physical structure state information corresponding to the component, and carrying out fault identification on the component according to the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information.
4. The method according to claim 3, wherein the number of the members is multiple, and the identifying the failure of the member according to the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information further comprises:
and marking the component which is judged to be in failure.
5. The method for augmented reality presentation of a target device according to claim 4, wherein the identifying the component as a fault according to the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information further comprises:
uploading the component marked with the failure to a server.
6. The method for augmented reality presentation of a target device according to claim 1, further comprising:
receiving a rotation instruction of a user;
and adjusting the direction of the virtual display model displayed on the display screen of the terminal equipment according to the rotation instruction.
7. An apparatus for augmented reality display of a target device, comprising:
the orientation information acquisition unit is used for acquiring orientation information of the camera of the terminal equipment relative to the target equipment;
a virtual display model obtaining unit, configured to obtain a virtual display model corresponding to the orientation information, where the virtual display models corresponding to at least two different orientation information are an internal display model and an external display model of the target device, respectively;
and the display unit is used for displaying the virtual display model on a display screen of the terminal equipment according to the orientation information.
8. The apparatus for augmented reality presentation of a target device according to claim 7, wherein the orientation information includes distance information, and the virtual presentation model obtaining unit includes:
the first obtaining module is used for obtaining an external display model corresponding to a first distance when the distance information is the first distance;
the second obtaining module is used for obtaining an internal display model corresponding to a second distance when the distance information is the second distance; the first distance is greater than the second distance.
9. The apparatus for augmented reality presentation of a target device according to claim 7, further comprising:
a first receiving unit for receiving an identification request of a user;
the orientation information acquisition unit is used for executing the acquisition of the orientation information of the camera of the terminal equipment relative to the target equipment if the identification request is an information display request;
and the fault identification unit is used for acquiring the depth image information of the target equipment if the identification request is a fault identification request, analyzing the current three-dimensional physical structure state information of the member of the target equipment contained in the depth image information, acquiring the normal three-dimensional physical structure state information corresponding to the member, and identifying the fault of the member according to the current three-dimensional physical structure state information and the normal three-dimensional physical structure state information.
10. The apparatus for augmented reality presentation of a target device according to claim 9, wherein the member is a plurality of members, and the fault identification unit further comprises:
and the marking module is used for marking the component which is judged to have the fault.
11. The apparatus for augmented reality presentation of a target device according to claim 10, wherein the fault recognition unit further comprises:
and the uploading module is used for uploading the fault marked component to the server.
12. The apparatus for augmented reality presentation of a target device according to claim 7, further comprising:
the second receiving unit is used for receiving a rotation instruction of a user;
and the adjusting unit is used for adjusting the direction of the virtual display model displayed on the display screen of the terminal equipment according to the rotating instruction.
13. A storage medium, characterized in that the storage medium includes a stored program, and when the program runs, the storage medium is controlled by a device to execute the method for augmented reality presentation of a target device according to any one of claims 1 to 6.
14. An apparatus for augmented reality presentation of a target device, the apparatus comprising a storage medium; and one or more processors, the storage medium coupled with the processors, the processors configured to execute program instructions stored in the storage medium; the program instructions when executed perform the method for augmented reality presentation of a target device of any one of claims 1 to 6.
15. A system for augmented reality display of a target device, comprising:
the apparatus for augmented reality presentation of a target device and the server of claim 14;
the device for performing augmented reality display on the target equipment is used for sending an acquisition request of a virtual display model corresponding to the azimuth information to the server;
and the server is used for sending a virtual display model corresponding to the orientation information to the device for performing augmented reality display on the target equipment according to the acquisition request.
CN201911350202.8A 2019-12-24 2019-12-24 Method, device and system for augmented reality display of target equipment Pending CN113034697A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911350202.8A CN113034697A (en) 2019-12-24 2019-12-24 Method, device and system for augmented reality display of target equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911350202.8A CN113034697A (en) 2019-12-24 2019-12-24 Method, device and system for augmented reality display of target equipment

Publications (1)

Publication Number Publication Date
CN113034697A true CN113034697A (en) 2021-06-25

Family

ID=76452029

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911350202.8A Pending CN113034697A (en) 2019-12-24 2019-12-24 Method, device and system for augmented reality display of target equipment

Country Status (1)

Country Link
CN (1) CN113034697A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102568012A (en) * 2010-10-13 2012-07-11 株式会社泛泰 User equipment and method for providing augmented reality (ar) service
CN107564108A (en) * 2017-07-27 2018-01-09 湖北历拓网络科技有限公司 The vehicle method of commerce and device of a kind of virtual reality
CN109213885A (en) * 2017-06-29 2019-01-15 深圳市掌网科技股份有限公司 Car show method and system based on augmented reality
US20190019335A1 (en) * 2017-07-12 2019-01-17 NextEv USA, Inc. Camera Pose Estimation Method for Augmented Reality Manual For Cars
CN109785294A (en) * 2018-12-24 2019-05-21 苏州江奥光电科技有限公司 A kind of pcb board defective locations detection system and method
CN110148222A (en) * 2019-05-27 2019-08-20 重庆爱车天下科技有限公司 It is a kind of that vehicle method and system are seen based on AR technology
CN110490339A (en) * 2019-08-23 2019-11-22 北京枭龙科技有限公司 A kind of auto repair auxiliary system and method based on augmented reality

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102568012A (en) * 2010-10-13 2012-07-11 株式会社泛泰 User equipment and method for providing augmented reality (ar) service
CN109213885A (en) * 2017-06-29 2019-01-15 深圳市掌网科技股份有限公司 Car show method and system based on augmented reality
US20190019335A1 (en) * 2017-07-12 2019-01-17 NextEv USA, Inc. Camera Pose Estimation Method for Augmented Reality Manual For Cars
CN107564108A (en) * 2017-07-27 2018-01-09 湖北历拓网络科技有限公司 The vehicle method of commerce and device of a kind of virtual reality
CN109785294A (en) * 2018-12-24 2019-05-21 苏州江奥光电科技有限公司 A kind of pcb board defective locations detection system and method
CN110148222A (en) * 2019-05-27 2019-08-20 重庆爱车天下科技有限公司 It is a kind of that vehicle method and system are seen based on AR technology
CN110490339A (en) * 2019-08-23 2019-11-22 北京枭龙科技有限公司 A kind of auto repair auxiliary system and method based on augmented reality

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
ARVR大本营: "AR汽车展厅应用-增强现实汽车展示(ARVRinChina-AR VR MR)", Retrieved from the Internet <URL:https://www.bilibili.com/video/BV1gx411Z7Z5/?spm_id_from=333.1007.top_right_bar_window_history.content.click&vd_source=3d0781b795cb46020fa92284fa5bccee> *
ARVR大本营: "最佳汽车增强现实应用程序演示使用ARkit(ARVRinChina-A R VR MR)", pages 1 - 56, Retrieved from the Internet <URL:https://www.bilibili.com/video/BV1m4411u7uo/?spm_id_from=333.788.top_right_bar_window_history.content.click&vd_source=3d0781b795cb46020fa92284fa5bccee> *
ARVR大本营: "最佳汽车增强现实应用程序演示使用ARkit(ARVRinChina-AR VR MR)", pages 1 - 56, Retrieved from the Internet <URL:https://www.bilibili.com/video/BV1m4411u7uo/?spm_id_from=333.788.top_right_bar_window_history.content.click&vd_source=3d0781b795cb46020fa92284fa5bccee> *
刘思源;曾传华;夏茂栩;张一寒;: "AR技术在汽车工业领域的应用", 科技创新导报, no. 19, pages 110 - 111 *
季钰;周蓓;: "基于增强现实技术的用户体验设计", 常熟理工学院学报, no. 05, pages 39 - 44 *

Similar Documents

Publication Publication Date Title
CN110352446B (en) Method and apparatus for obtaining image and recording medium thereof
US11854149B2 (en) Techniques for capturing and displaying partial motion in virtual or augmented reality scenes
CN106471548B (en) Use the method and apparatus of the acceleration template matches of peripheral information
US9077891B1 (en) Depth determination using camera focus
CN106846497B (en) Method and device for presenting three-dimensional map applied to terminal
CN111737518B (en) Image display method and device based on three-dimensional scene model and electronic equipment
KR20190118531A (en) Image processing for augmented reality
US9756260B1 (en) Synthetic camera lenses
EP3314581B1 (en) Augmented reality device for visualizing luminaire fixtures
TWI783472B (en) Ar scene content generation method, display method, electronic equipment and computer readable storage medium
KR20170125618A (en) Method for generating content to be displayed at virtual area via augmented reality platform and electronic device supporting the same
US20150346915A1 (en) Method and system for automating data processing in satellite photogrammetry systems
US11651560B2 (en) Method and device of displaying comment information, and mobile terminal
CN111815785A (en) Method and device for presenting reality model, electronic equipment and storage medium
JP2016194924A (en) Graphical user interface (gui) shading based on context
US11978232B2 (en) Method for displaying three-dimensional augmented reality
CN108171801A (en) A kind of method, apparatus and terminal device for realizing augmented reality
CN115457179A (en) Method, apparatus, device and medium for rendering virtual object
CN111815783A (en) Virtual scene presenting method and device, electronic equipment and storage medium
CN113034697A (en) Method, device and system for augmented reality display of target equipment
CN112416218B (en) Virtual card display method and device, computer equipment and storage medium
CN111242107B (en) Method and electronic device for setting virtual object in space
CN112825198B (en) Mobile tag display method, device, terminal equipment and readable storage medium
CN113973235A (en) Interactive information display method and device and computer equipment
CN113112613B (en) Model display method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination