CN111640184A - Ancient building reproduction method, ancient building reproduction device, electronic equipment and storage medium - Google Patents
Ancient building reproduction method, ancient building reproduction device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111640184A CN111640184A CN202010504363.4A CN202010504363A CN111640184A CN 111640184 A CN111640184 A CN 111640184A CN 202010504363 A CN202010504363 A CN 202010504363A CN 111640184 A CN111640184 A CN 111640184A
- Authority
- CN
- China
- Prior art keywords
- scene object
- ancient building
- target
- scene
- virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/10—Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Geometry (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present disclosure provides an ancient building reproduction method, an ancient building reproduction device, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring a real scene image; identifying a target scene object in the real scene image based on the real scene image; based on the identified target scene object, determining a virtual ancient building matched with the target scene object and a fusion position of the virtual ancient building in the AR scene; the virtual ancient building is obtained by performing three-dimensional reconstruction according to a historical restored scene picture corresponding to the target scene object; and generating AR display data for fusing the virtual ancient building into the real scene corresponding to the scene object based on the fusion position, and displaying the AR display data through the AR equipment.
Description
Technical Field
The disclosure relates to the technical field of enhanced display, in particular to an ancient building reproduction method and device, electronic equipment and a storage medium.
Background
Generally, due to historical transition, the stored representative ancient buildings are numerous, and many of the stored ancient buildings are damaged to different degrees due to various reasons such as environment, human factors and the like, so that a user can only observe the damaged ancient buildings at a fixed place.
Therefore, the manner of visiting the ancient building at a fixed place leads to higher visiting cost for users and can not visit the original appearance of the ancient building.
Disclosure of Invention
In view of the above, the present disclosure provides at least an ancient building reproduction method, an ancient building reproduction apparatus, an electronic device, and a storage medium.
In a first aspect, the present disclosure provides a method of reconstructing an ancient building, comprising:
acquiring a real scene image;
identifying a target scene object in the real scene image based on the real scene image;
determining a virtual ancient building matched with the target scene object and a fusion position of the virtual ancient building in an Augmented Reality (AR) scene based on the identified target scene object; the virtual ancient building is obtained by performing three-dimensional reconstruction according to a historical restored scene picture corresponding to the target scene object;
and generating AR display data for fusing the virtual ancient building into the real scene corresponding to the scene object based on the fusion position, and displaying the AR display data through the AR equipment.
In the method, a target scene object in a real scene image is identified through an acquired real scene image; determining a virtual ancient building matched with the target scene object and a fusion position of the virtual ancient building in the AR scene based on the identified target scene object; the virtual ancient building is obtained by three-dimensional reconstruction according to a historical restored scene picture corresponding to the target scene object; based on fusing the position, the AR that generates and fuses virtual ancient building into the real scene that the scene object corresponds demonstrates data to demonstrate AR demonstration data through AR equipment, through demonstrating virtual ancient building for the target user, make the user can carry out careful comprehensive visit to the original appearance of ancient building, and can be in same position, each virtual ancient building of user's show, the user need not to arrive different positions and visits different ancient buildings, user's visiting cost has been reduced.
In one possible embodiment, the target scene object is reconstructed in three dimensions according to the following steps:
acquiring historical recovery scene pictures of different shooting angles corresponding to the target scene object;
generating a three-dimensional scene model of the target scene object based on a plurality of historical restored scene pictures;
and element filling is carried out on the three-dimensional scene model, and the virtual ancient building after restoration is obtained.
In one possible embodiment, the determining a virtual ancient building matched with the target scene object based on the identified target scene object includes:
comparing the target scene object with scene object materials in a pre-stored historical scene object library, and determining target historical scene object materials corresponding to the target scene object;
and determining a virtual ancient building matched with the target scene object based on the mapping relation between the stored target historical scene object materials and the virtual ancient building.
In one possible implementation, comparing the target scene object with scene object materials in a pre-stored historical scene object library, and determining target historical scene object materials corresponding to the target scene object includes:
calculating the matching degree between the target scene object and each scene object material in a pre-stored historical scene object library;
and determining target historical scene object materials corresponding to the target scene object based on the matching degree.
In a possible implementation manner, determining target historical scene object materials corresponding to the target scene object based on the matching degree includes:
determining the scene object material with the highest matching degree as a target historical scene object material corresponding to the target scene object; alternatively, the first and second electrodes may be,
and determining the scene object materials with the matching degree larger than the set matching degree threshold value as target historical scene object materials corresponding to the target scene object.
In one possible embodiment, when the target historical scene object material is multiple, determining a virtual ancient building matched with the target scene object based on a mapping relationship between the stored target historical scene object material and the virtual ancient building includes:
determining at least one candidate virtual ancient building corresponding to the target scene object based on a mapping relation between the stored target historical scene object materials and the virtual ancient buildings;
for each candidate virtual ancient building, determining the similarity of the candidate virtual ancient building based on the matching degree and the number of target historical scene object materials associated with the candidate virtual ancient building;
and determining the candidate virtual ancient building with the maximum similarity as the virtual ancient building matched with the target scene object.
In a possible embodiment, identifying a target scene object in the real scene image based on the real scene image includes:
identifying a target scene object in the real scene image based on the real scene image, and extracting a plurality of kinds of characteristic information corresponding to the target scene object;
based on the identified target scene object, determining a virtual ancient building matched with the target scene object, comprising:
and determining the virtual ancient buildings matched with the target scene object based on the extracted multiple kinds of characteristic information corresponding to the target scene object and the pre-stored characteristic information corresponding to each virtual ancient building.
In one possible embodiment, the fusion position of the virtual ancient building in the AR scene is determined by the following steps:
and determining the fusion position of the virtual ancient building in the AR scene based on the position of the target scene object in the real scene image.
The following descriptions of the effects of the apparatus, the electronic device, and the like refer to the description of the above method, and are not repeated here.
In a second aspect, the present disclosure provides an antique-building reproduction apparatus comprising:
the acquisition module is used for acquiring a real scene image;
an identification module for identifying a target scene object in the real scene image based on the real scene image;
the determining module is used for determining a virtual ancient building matched with the target scene object and a fusion position of the virtual ancient building in the AR scene based on the identified target scene object; the virtual ancient building is obtained by performing three-dimensional reconstruction according to a historical restored scene picture corresponding to the target scene object;
and the first generation module is used for generating AR display data of the virtual ancient building fused into the real scene corresponding to the scene object based on the fusion position, and displaying the AR display data through the AR equipment.
In a possible implementation, the determining module is configured to reconstruct the target scene object in three dimensions according to the following steps:
acquiring historical recovery scene pictures of different shooting angles corresponding to the target scene object;
generating a three-dimensional scene model of the target scene object based on a plurality of historical restored scene pictures;
and element filling is carried out on the three-dimensional scene model, and the virtual ancient building after restoration is obtained.
In a possible embodiment, the determining module, when determining the virtual ancient building matching the target scene object based on the identified target scene object, is configured to:
comparing the target scene object with scene object materials in a pre-stored historical scene object library, and determining target historical scene object materials corresponding to the target scene object;
and determining a virtual ancient building matched with the target scene object based on the mapping relation between the stored target historical scene object materials and the virtual ancient building.
In a possible implementation manner, when comparing the target scene object with scene object materials in a pre-stored historical scene object library and determining a target historical scene object material corresponding to the target scene object, the determining module is configured to:
calculating the matching degree between the target scene object and each scene object material in a pre-stored historical scene object library;
and determining target historical scene object materials corresponding to the target scene object based on the matching degree.
In a possible implementation manner, when determining, based on the matching degree, a target historical scene object material corresponding to the target scene object, the determining module is configured to:
determining the scene object material with the highest matching degree as a target historical scene object material corresponding to the target scene object; alternatively, the first and second electrodes may be,
and determining the scene object materials with the matching degree larger than the set matching degree threshold value as target historical scene object materials corresponding to the target scene object.
In a possible implementation manner, when the target historical scene object material is multiple, the determining module, when determining a virtual ancient building matching the target scene object based on the mapping relationship between the stored target historical scene object material and the virtual ancient building, is configured to:
determining at least one candidate virtual ancient building corresponding to the target scene object based on a mapping relation between the stored target historical scene object materials and the virtual ancient buildings;
for each candidate virtual ancient building, determining the similarity of the candidate virtual ancient building based on the matching degree and the number of target historical scene object materials associated with the candidate virtual ancient building;
and determining the candidate virtual ancient building with the maximum similarity as the virtual ancient building matched with the target scene object.
In a possible embodiment, the identifying module, when identifying the target scene object in the real scene image based on the real scene image, is configured to:
identifying a target scene object in the real scene image based on the real scene image, and extracting a plurality of kinds of characteristic information corresponding to the target scene object;
the determination module, when determining the virtual ancient building matched with the target scene object based on the identified target scene object, is configured to:
and determining the virtual ancient buildings matched with the target scene object based on the extracted multiple kinds of characteristic information corresponding to the target scene object and the pre-stored characteristic information corresponding to each virtual ancient building.
In one possible embodiment, the determining module is configured to determine the fusion position of the virtual ancient building in the AR scene by:
and determining the fusion position of the virtual ancient building in the AR scene based on the position of the scene object in the real scene image.
In a third aspect, the present disclosure provides an electronic device comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the historic building reproduction method according to the first aspect or any of the embodiments.
In a fourth aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the historic building reproduction method according to the first aspect or any of the embodiments.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 is a schematic flow chart illustrating a method for reconstructing an ancient building according to an embodiment of the present disclosure;
fig. 2 is a schematic interface diagram of an AR device in an ancient building reproduction method provided by an embodiment of the present disclosure;
fig. 3 is a schematic diagram illustrating an architecture of an ancient architecture reproducing apparatus provided by an embodiment of the present disclosure;
fig. 4 shows a schematic structural diagram of an electronic device 400 provided in an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
In order to facilitate the user to visit the original appearance of the ancient building, so that the user can finely and comprehensively observe the original appearance of the ancient building, the embodiment of the disclosure provides an ancient building reappearing method.
For the purpose of facilitating an understanding of the disclosed embodiments, a detailed description of an ancient architecture reproduction method disclosed in the disclosed embodiments will be provided first.
The execution main body of the ancient building reproduction method provided by the embodiment of the disclosure can be a server, and the server can be a cloud server or a local server.
Referring to fig. 1, a schematic flow chart of an ancient building reproduction method provided by an embodiment of the present disclosure is shown, the method including S101-S104, wherein:
s101, acquiring a real scene image;
s102, identifying a target scene object in the real scene image based on the real scene image;
s103, determining a virtual ancient building matched with the target scene object and a fusion position of the virtual ancient building in the AR scene based on the identified target scene object; the virtual ancient building is obtained by three-dimensional reconstruction according to a historical restored scene picture corresponding to the target scene object;
and S104, generating AR display data of the reality scene corresponding to the virtual ancient building fused into the scene object based on the fusion position, and displaying the AR display data through AR equipment.
In the method, a target scene object in a real scene image is identified through an acquired real scene image; determining a virtual ancient building matched with the target scene object and a fusion position of the virtual ancient building in the AR scene based on the identified target scene object; the virtual ancient building is obtained by three-dimensional reconstruction according to a historical restored scene picture corresponding to the target scene object; based on fusing the position, the AR that generates and fuses virtual ancient building into the real scene that the scene object corresponds demonstrates data to demonstrate AR demonstration data through AR equipment, through demonstrating virtual ancient building for the target user, make the user can carry out careful comprehensive visit to the original appearance of ancient building, and can be in same position, each virtual ancient building of user's show, the user need not to arrive different positions and visits different ancient buildings, user's visiting cost has been reduced.
For S101:
here, the real scene image may be any image in the real scene acquired by an imaging device provided in the AR apparatus, and the image may be an RGB image.
In specific implementation, the AR device may acquire the real-scene image in real time, and identify each frame of the real-scene image. Alternatively, the real scene image may be acquired in response to a scene acquisition request triggered by the user. For example, a user may click a scene capture button, that is, trigger a scene capture request, and after the user triggers the scene capture request, obtain a real scene image in response to the scene capture request triggered by the user.
For S102 and S103:
here, the target scene object in the real scene image may be identified from the acquired real scene image. And matching the identified target scene object with the corresponding virtual ancient building from a plurality of prestored virtual ancient buildings, and determining the fusion position of the virtual ancient building in the AR scene.
Before the real scene image is obtained, the target scene object can be subjected to three-dimensional reconstruction according to the following steps to generate a corresponding virtual ancient building:
firstly, acquiring historical recovery scene pictures of different shooting angles corresponding to the target scene object.
And secondly, generating a three-dimensional scene model of the target scene object based on a plurality of historical restored scene pictures.
And thirdly, element filling is carried out on the three-dimensional scene model, and the virtual ancient building after restoration is obtained.
Here, the history restored scene picture may be an image corresponding to a target scene object, and the image corresponding to the target scene object may be an image obtained by photographing; the image obtained by manually drawing can also be based on the description information of the target scene object; and also for images captured from video frames, etc.
After a plurality of historical restored scene pictures of different shooting angles are obtained, a three-dimensional scene model of the target scene object can be generated based on the plurality of historical restored scene pictures, and element filling is carried out on the three-dimensional scene model to obtain a restored virtual ancient building. For example, a three-dimensional scene model of the target scene object may be generated based on a plurality of historical restored scene pictures and a three-dimensional modeling algorithm, and then after element filling, the restored virtual ancient building may be obtained.
In an optional embodiment, determining a virtual ancient building matched with the target scene object based on the identified target scene object comprises:
comparing the target scene object with scene object materials in a pre-stored historical scene object library, and determining target historical scene object materials corresponding to the target scene object.
And secondly, determining a virtual ancient building matched with the target scene object based on the mapping relation between the stored target historical scene object materials and the virtual ancient building.
Here, a plurality of target historical scene object materials may be associated in advance for each virtual ancient building. For example, the target historical scene object materials of the virtual ancient building a may be a plurality of images obtained when the virtual ancient building a is shot at different shooting angles, and/or may also be images corresponding to the virtual ancient building a in different times, and/or may also be local images (the local images may be images representing characteristics of the virtual ancient building a) at different positions of the virtual ancient building a, and the like.
In an optional embodiment, the determining, based on the identified target scene object, a virtual ancient building matching the target scene object includes:
comparing the target scene object with scene object materials in a pre-stored historical scene object library, and determining target historical scene object materials corresponding to the target scene object.
And secondly, determining a virtual ancient building matched with the target scene object based on the mapping relation between the stored target historical scene object materials and the virtual ancient building.
In an optional implementation manner, comparing the target scene object with scene object materials in a pre-stored historical scene object library, and determining target historical scene object materials corresponding to the target scene object includes: calculating the matching degree between the target scene object and each scene object material in a pre-stored historical scene object library; and determining target historical scene object materials corresponding to the target scene object based on the matching degree.
In an optional implementation manner, determining, based on the matching degree, target historical scene object materials corresponding to the target scene object includes:
and in the first mode, the scene object material with the highest matching degree is determined as the target historical scene object material corresponding to the target scene object.
And secondly, determining the scene object materials with the matching degree larger than the set matching degree threshold value as target historical scene object materials corresponding to the target scene object.
In an optional embodiment, when the target historical scene object material is multiple, determining a virtual ancient building matching the target scene object based on a mapping relationship between the stored target historical scene object material and the virtual ancient building includes:
step one, determining at least one candidate virtual ancient building corresponding to a target scene object based on a mapping relation between stored target historical scene object materials and the virtual ancient buildings.
And secondly, determining the similarity of each candidate virtual ancient building based on the matching degree and the number of target historical scene object materials associated with the candidate virtual ancient building.
And step three, determining the candidate virtual ancient building with the maximum similarity as the virtual ancient building matched with the target scene object.
In specific implementation, the target scene object may be compared with each scene object material to determine a target historical scene object material of the target scene object. For example, the matching degree between the target scene object and each scene object material may be calculated, and the scene object material with the highest matching degree may be determined as the target historical scene object material corresponding to the target scene object. And determining the virtual ancient building corresponding to the target historical scene object material as the virtual ancient building matched with the target scene object.
Or, a matching degree threshold value can be set, and at least one scene object material with the matching degree greater than the matching degree threshold value is determined as a target historical scene object material; and then determining at least one candidate virtual ancient building corresponding to the target historical scene object material, determining the similarity of the candidate virtual ancient buildings based on the scene object material corresponding to each candidate virtual ancient building in the target historical scene object material, obtaining the similarity of each candidate virtual ancient building, and determining the candidate virtual ancient building with the highest similarity as the virtual ancient building matched with the target scene object.
For example, the target historical scene object materials include a scene object material a with a matching degree of 75%, a scene object material B with a matching degree of 70%, a scene object material C with a matching degree of 80%, a scene object material D with a matching degree of 82%, a scene object material E with a matching degree of 78%, and a scene object material F with a matching degree of 78%; if the scene object material A, the scene object material B and the scene object material C are associated with the virtual ancient building A, the scene object material D and the scene object material E are associated with the virtual ancient building B, and the scene object material F is associated with the virtual ancient building C, the candidate virtual ancient buildings comprise the virtual ancient building A, the virtual ancient building B and the virtual ancient building C.
Continuing with the above example, a method for determining similarity of candidate virtual ancient buildings may comprise: determining the similarity of the candidate virtual ancient buildings based on the number of each candidate virtual ancient building and the number of scene object materials included in target historical scene object materials, namely the similarity of a virtual ancient building A is as follows: 3/6 ═ 50%; the similarity of the virtual ancient building B is as follows: 2/6 ═ 33%; the similarity of the virtual ancient building C is as follows: 1/6-17%. The virtual ancient building a can be determined as a virtual ancient building matching the target scene object.
Secondly, the matching degree of each candidate virtual ancient building can be averaged, and the similarity of the candidate virtual ancient building is determined, namely the similarity of the virtual ancient building A is as follows: (75% + 70% + 80%)/3 ═ 75%; the similarity of the virtual ancient building B is as follows: (78% + 82%)/2 ═ 80%; the similarity of the virtual ancient building C is as follows: 78 percent. The virtual ancient building B can be determined as a virtual ancient building matching the target scene object. The process of determining similarity includes other methods, which are only exemplary.
During the concrete implementation, the user can be when visiting entity ancient building, can look over the virtual ancient building that matches with this entity ancient building, for example, this virtual ancient building's structure can be the same with entity ancient building's structure, this virtual ancient building can be for the virtual ancient building including the original appearance of entity ancient building promptly, and then the user can be through comparing virtual ancient building and entity ancient building, there is certain understanding to the damage degree of entity ancient building, also can visit the historical original appearance of this entity ancient building, the bandwagon effect of ancient building has been improved. Or, the user can also look over the virtual ancient building relevant with this entity ancient building when visiting entity ancient building, for example, if entity ancient building is the pagoda, the user can look over other structure type's virtual ancient building (virtual pagoda), knows the difference between entity ancient building and the virtual ancient building, improves ancient building's bandwagon effect.
In an alternative embodiment, identifying a scene object corresponding to a real scene image based on the real scene image may include: based on the real scene image, a scene object corresponding to the real scene image is identified, and various feature information corresponding to the scene object is extracted. And determining a virtual ancient building matched with the scene object based on the identified scene object, wherein the determining comprises the following steps: and determining the virtual ancient buildings matched with the scene objects based on the extracted multiple kinds of characteristic information corresponding to the scene objects and the pre-stored characteristic information corresponding to the virtual ancient buildings respectively.
Here, various feature information corresponding to the scene object may be extracted from the real scene image, for example, various feature information of the scene object may be extracted through the feature extraction network, feature information of each virtual ancient building may be extracted through the feature extraction network, and the feature information of each virtual ancient building is stored in association with the virtual ancient building. Furthermore, the virtual ancient buildings matched with the scene objects can be determined based on the extracted multiple kinds of characteristic information corresponding to the scene objects and the prestored characteristic information corresponding to the virtual ancient buildings respectively.
For example, the matching degree between the plurality of kinds of feature information corresponding to the scene object and the feature information corresponding to each virtual ancient building may be calculated, and based on the matching degree between the scene object and each virtual object, the virtual ancient building with the highest matching degree may be selected and determined as the virtual ancient building matched with the scene object. Or, inputting a plurality of kinds of feature information corresponding to the scene object and feature information of each virtual ancient building into the neural network, and determining the virtual ancient building matched with the scene object. And selecting various characteristics corresponding to the scene object according to the actual situation. For example, the plurality of features corresponding to the scene object may be volume features of a building, age features of a building, usage features of a building, and the like. For example, if the scene object is a Yuanming Garden site, the virtual ancient building determined for the scene object is the reconstructed Yuanming garden site.
In an alternative embodiment, the fusion position of the virtual ancient building in the AR scene is determined by the following steps: and determining the fusion position of the virtual ancient building in the AR scene based on the position of the scene object in the real scene image.
In specific implementation, the position of the scene object in the real scene image may be determined as the fusion position of the virtual ancient building in the AR scene. Or the fusion position of the virtual ancient building in the AR scene can be determined based on the position of the scene object in the real scene image and preset position deviation information. Alternatively, a region where no physical object exists in the real scene may be determined, and the center position of the region may be determined as the fusion position of the virtual ancient building in the AR scene.
For S104:
here, based on the fusion position, AR presentation data of a display scene corresponding to the virtual ancient building fused into the scene object is generated, and the AR presentation data may be sent to the AR device, and the AR presentation data is presented by the AR device.
In the specific implementation, when visiting the ancient building, the user can trigger the ancient building exhibition operation, respond the ancient building exhibition operation that the user triggered, show at least one virtual ancient building of storage, for example, can respond the ancient building exhibition operation that the user triggered, show the image of every virtual ancient building in at least one virtual ancient building on AR equipment in proper order, so that the user triggers the preset exhibition operation to specific virtual ancient building, namely, the user can trigger the preset exhibition operation to this virtual ancient building (specific virtual ancient building) when looking over the virtual ancient building that wants to show, respond the preset exhibition operation to specific virtual ancient building that the user triggered, acquire the reality scene image. And after the scene object corresponding to the real scene image is identified, determining the characteristic virtual ancient building as the virtual ancient building corresponding to the scene object.
Referring to fig. 2, which shows an interface schematic diagram of an AR device in an ancient building reproduction method, fig. 2 includes a target scene object 21 and a virtual ancient building 22.
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same concept, an embodiment of the present disclosure further provides an ancient building reproduction apparatus, and as shown in fig. 3, an architecture schematic diagram of the ancient building reproduction apparatus provided for the embodiment of the present disclosure includes an obtaining module 301, an identifying module 302, a determining module 303, and a first generating module 304, specifically:
an obtaining module 301, configured to obtain a real scene image;
an identifying module 302 configured to identify a target scene object in the real scene image based on the real scene image;
a determining module 303, configured to determine, based on the identified target scene object, a virtual ancient building matched with the target scene object and a fusion position of the virtual ancient building in the AR scene; the virtual ancient building is obtained by performing three-dimensional reconstruction according to a historical restored scene picture corresponding to the target scene object;
a first generating module 304, configured to generate, based on the fusion position, AR display data for fusing the virtual ancient building into the real scene corresponding to the scene object, and display the AR display data through the AR device.
In a possible implementation, the determining module 303 is configured to perform three-dimensional reconstruction on the target scene object according to the following steps:
acquiring historical recovery scene pictures of different shooting angles corresponding to the target scene object;
generating a three-dimensional scene model of the target scene object based on a plurality of historical restored scene pictures;
and element filling is carried out on the three-dimensional scene model, and the virtual ancient building after restoration is obtained.
In a possible implementation, the determining module 303, when determining the virtual ancient building matched with the target scene object based on the identified target scene object, is configured to:
comparing the target scene object with scene object materials in a pre-stored historical scene object library, and determining target historical scene object materials corresponding to the target scene object;
and determining a virtual ancient building matched with the target scene object based on the mapping relation between the stored target historical scene object materials and the virtual ancient building.
In a possible implementation manner, the determining module 303, when comparing the target scene object with scene object materials in a pre-stored historical scene object library to determine target historical scene object materials corresponding to the target scene object, is configured to:
calculating the matching degree between the target scene object and each scene object material in a pre-stored historical scene object library;
and determining target historical scene object materials corresponding to the target scene object based on the matching degree.
In a possible implementation manner, the determining module 303, when determining, based on the matching degree, a target historical scene object material corresponding to the target scene object, is configured to:
determining the scene object material with the highest matching degree as a target historical scene object material corresponding to the target scene object; alternatively, the first and second electrodes may be,
and determining the scene object materials with the matching degree larger than the set matching degree threshold value as target historical scene object materials corresponding to the target scene object.
In a possible implementation, when the target historical scene object material is multiple, the determining module 303, when determining a virtual ancient building matching the target scene object based on the mapping relationship between the stored target historical scene object material and the virtual ancient building, is configured to:
determining at least one candidate virtual ancient building corresponding to the target scene object based on a mapping relation between the stored target historical scene object materials and the virtual ancient buildings;
for each candidate virtual ancient building, determining the similarity of the candidate virtual ancient building based on the matching degree and the number of target historical scene object materials associated with the candidate virtual ancient building;
and determining the candidate virtual ancient building with the maximum similarity as the virtual ancient building matched with the target scene object.
In a possible implementation, the identifying module 302, when identifying the target scene object in the real scene image based on the real scene image, is configured to:
identifying a scene object corresponding to the real scene image based on the real scene image, and extracting various feature information corresponding to the scene object;
the determining module 303, when determining the virtual ancient building matched with the scene object based on the identified scene object, is configured to:
and determining the virtual ancient buildings matched with the scene object based on the extracted multiple kinds of characteristic information corresponding to the scene object and the pre-stored characteristic information corresponding to each virtual ancient building.
In a possible embodiment, the determining module 303 is configured to determine the fusion position of the virtual ancient building in the AR scene by:
and determining the fusion position of the virtual ancient building in the AR scene based on the position of the scene object in the real scene image.
In some embodiments, the functions of the apparatus provided in the embodiments of the present disclosure or the included templates may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, no further description is provided here.
Based on the same technical concept, the embodiment of the disclosure also provides an electronic device. Referring to fig. 4, a schematic structural diagram of an electronic device provided in the embodiment of the present disclosure includes a processor 401, a memory 402, and a bus 403. The memory 402 is used for storing execution instructions and includes a memory 4021 and an external memory 4022; the memory 4021 is also referred to as an internal memory, and is configured to temporarily store operation data in the processor 401 and data exchanged with the external memory 4022 such as a hard disk, the processor 401 exchanges data with the external memory 4022 through the memory 4021, and when the electronic device 400 operates, the processor 401 communicates with the memory 402 through the bus 403, so that the processor 401 executes the following instructions:
acquiring a real scene image;
identifying a target scene object in the real scene image based on the real scene image;
based on the identified target scene object, determining a virtual ancient building matched with the target scene object and a fusion position of the virtual ancient building in the AR scene; the virtual ancient building is obtained by performing three-dimensional reconstruction according to a historical restored scene picture corresponding to the target scene object;
and generating AR display data for fusing the virtual ancient building into the real scene corresponding to the scene object based on the fusion position, and displaying the AR display data through the AR equipment.
Furthermore, the disclosed embodiments also provide a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to perform the steps of the historic building reproduction method described in the above method embodiments.
The computer program product of the ancient building reproduction method provided by the embodiment of the disclosure includes a computer-readable storage medium storing a program code, where the program code includes instructions for executing the steps of the ancient building reproduction method described in the above method embodiment, and specific reference may be made to the above method embodiment, and details are not repeated here.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above are only specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present disclosure, and shall be covered by the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (11)
1. A method for reconstructing an ancient building, comprising:
acquiring a real scene image;
identifying a target scene object in the real scene image based on the real scene image;
based on the identified target scene object, determining a virtual ancient building matched with the target scene object and a fusion position of the virtual ancient building in the AR scene; the virtual ancient building is obtained by performing three-dimensional reconstruction according to a historical restored scene picture corresponding to the target scene object;
and generating AR display data for fusing the virtual ancient building into the real scene corresponding to the scene object based on the fusion position, and displaying the AR display data through the AR equipment.
2. The method of claim 1, wherein the target scene object is reconstructed in three dimensions according to the following steps:
acquiring historical recovery scene pictures of different shooting angles corresponding to the target scene object;
generating a three-dimensional scene model of the target scene object based on a plurality of historical restored scene pictures;
and element filling is carried out on the three-dimensional scene model, and the virtual ancient building after restoration is obtained.
3. The method of claim 1 or 2, wherein the determining a virtual ancient building matching the target scene object based on the identified target scene object comprises:
comparing the target scene object with scene object materials in a pre-stored historical scene object library, and determining target historical scene object materials corresponding to the target scene object;
and determining a virtual ancient building matched with the target scene object based on the mapping relation between the stored target historical scene object materials and the virtual ancient building.
4. The method of claim 3, wherein comparing the target scene object with scene object materials in a pre-stored historical scene object library to determine target historical scene object materials corresponding to the target scene object comprises:
calculating the matching degree between the target scene object and each scene object material in a pre-stored historical scene object library;
and determining target historical scene object materials corresponding to the target scene object based on the matching degree.
5. The method of claim 4, wherein determining target historical scene object material corresponding to the target scene object based on the matching degree comprises:
determining the scene object material with the highest matching degree as a target historical scene object material corresponding to the target scene object; alternatively, the first and second electrodes may be,
and determining the scene object materials with the matching degree larger than the set matching degree threshold value as target historical scene object materials corresponding to the target scene object.
6. The method of claim 5, wherein when the target historical scene object material is plural, determining a virtual ancient building matching the target scene object based on a mapping relationship between the stored target historical scene object material and the virtual ancient building comprises:
determining at least one candidate virtual ancient building corresponding to the target scene object based on a mapping relation between the stored target historical scene object materials and the virtual ancient buildings;
for each candidate virtual ancient building, determining the similarity of the candidate virtual ancient building based on the matching degree and the number of target historical scene object materials associated with the candidate virtual ancient building;
and determining the candidate virtual ancient building with the maximum similarity as the virtual ancient building matched with the target scene object.
7. The method of claim 1, wherein identifying a target scene object in the real scene image based on the real scene image comprises:
identifying a target scene object in the real scene image based on the real scene image, and extracting a plurality of kinds of characteristic information corresponding to the target scene object;
based on the identified target scene object, determining a virtual ancient building matched with the target scene object, comprising:
and determining the virtual ancient buildings matched with the target scene object based on the extracted multiple kinds of characteristic information corresponding to the target scene object and the pre-stored characteristic information corresponding to each virtual ancient building.
8. The method of claim 1, wherein the fusion location of the virtual ancient building in the AR scene is determined by:
and determining the fusion position of the virtual ancient building in the AR scene based on the position of the target scene object in the real scene image.
9. An ancient building reproduction apparatus, comprising:
the acquisition module is used for acquiring a real scene image;
an identification module for identifying a target scene object in the real scene image based on the real scene image;
the determining module is used for determining a virtual ancient building matched with the target scene object and a fusion position of the virtual ancient building in the AR scene based on the identified target scene object; the virtual ancient building is obtained by performing three-dimensional reconstruction according to a historical restored scene picture corresponding to the target scene object;
and the first generation module is used for generating AR display data of the virtual ancient building fused into the real scene corresponding to the scene object based on the fusion position, and displaying the AR display data through the AR equipment.
10. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the historic building reproduction method of any of claims 1 to 8.
11. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the historic building reproduction method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010504363.4A CN111640184A (en) | 2020-06-05 | 2020-06-05 | Ancient building reproduction method, ancient building reproduction device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010504363.4A CN111640184A (en) | 2020-06-05 | 2020-06-05 | Ancient building reproduction method, ancient building reproduction device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111640184A true CN111640184A (en) | 2020-09-08 |
Family
ID=72333336
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010504363.4A Pending CN111640184A (en) | 2020-06-05 | 2020-06-05 | Ancient building reproduction method, ancient building reproduction device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111640184A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113973178A (en) * | 2021-10-24 | 2022-01-25 | 云景文旅科技有限公司 | Interactive photographing processing method and device in travel process |
| CN114219923A (en) * | 2021-12-14 | 2022-03-22 | 北京凯泰铭科技文化发展有限公司 | AR, VR and MR fusion method and system applied to vestige reproduction |
| WO2022169413A1 (en) * | 2021-02-03 | 2022-08-11 | 脸萌有限公司 | Image processing method and apparatus, electronic device, and program product |
| CN117611773A (en) * | 2023-11-30 | 2024-02-27 | 天津大学 | Large-scale building augmented reality display method, system, equipment and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110689623A (en) * | 2019-08-20 | 2020-01-14 | 重庆特斯联智慧科技股份有限公司 | Tourist guide system and method based on augmented reality display |
| CN110716645A (en) * | 2019-10-15 | 2020-01-21 | 北京市商汤科技开发有限公司 | Augmented reality data presentation method and device, electronic equipment and storage medium |
| CN110874861A (en) * | 2019-11-22 | 2020-03-10 | 武汉中天云迪科技有限公司 | Three-dimensional digital image acquisition method and equipment |
-
2020
- 2020-06-05 CN CN202010504363.4A patent/CN111640184A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110689623A (en) * | 2019-08-20 | 2020-01-14 | 重庆特斯联智慧科技股份有限公司 | Tourist guide system and method based on augmented reality display |
| CN110716645A (en) * | 2019-10-15 | 2020-01-21 | 北京市商汤科技开发有限公司 | Augmented reality data presentation method and device, electronic equipment and storage medium |
| CN110874861A (en) * | 2019-11-22 | 2020-03-10 | 武汉中天云迪科技有限公司 | Three-dimensional digital image acquisition method and equipment |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022169413A1 (en) * | 2021-02-03 | 2022-08-11 | 脸萌有限公司 | Image processing method and apparatus, electronic device, and program product |
| CN113973178A (en) * | 2021-10-24 | 2022-01-25 | 云景文旅科技有限公司 | Interactive photographing processing method and device in travel process |
| CN114219923A (en) * | 2021-12-14 | 2022-03-22 | 北京凯泰铭科技文化发展有限公司 | AR, VR and MR fusion method and system applied to vestige reproduction |
| CN117611773A (en) * | 2023-11-30 | 2024-02-27 | 天津大学 | Large-scale building augmented reality display method, system, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111640184A (en) | Ancient building reproduction method, ancient building reproduction device, electronic equipment and storage medium | |
| JP6644833B2 (en) | System and method for rendering augmented reality content with albedo model | |
| CN110716645A (en) | Augmented reality data presentation method and device, electronic equipment and storage medium | |
| JP2002042169A (en) | Three-dimensional image providing system, its method, morphing image providing system, and its method | |
| CN109089038B (en) | Augmented reality shooting method and device, electronic equipment and storage medium | |
| CN111679742A (en) | Interaction control method and device based on AR, electronic equipment and storage medium | |
| CN111696215A (en) | Image processing method, device and equipment | |
| CN111638797A (en) | Display control method and device | |
| CN111640169A (en) | Historical event presenting method and device, electronic equipment and storage medium | |
| CN111651050A (en) | Method and device for displaying urban virtual sand table, computer equipment and storage medium | |
| CN111653175B (en) | Virtual sand table display method and device | |
| TW202309834A (en) | Model reconstruction method, electronic device and computer-readable storage medium | |
| CN111640203B (en) | Image processing method and device | |
| CN111625100A (en) | Method and device for presenting picture content, computer equipment and storage medium | |
| CN112905014A (en) | Interaction method and device in AR scene, electronic equipment and storage medium | |
| CN112882576A (en) | AR interaction method and device, electronic equipment and storage medium | |
| CN114153548A (en) | Display method and device, computer equipment and storage medium | |
| US20140198177A1 (en) | Realtime photo retouching of live video | |
| JP7398819B2 (en) | Three-dimensional reconstruction method and device | |
| CN113902520A (en) | Augmented reality image display method, device, equipment and storage medium | |
| CN110267079B (en) | Method and device for replacing human face in video to be played | |
| KR20160046399A (en) | Method and Apparatus for Generation Texture Map, and Database Generation Method | |
| CN113178017A (en) | AR data display method and device, electronic equipment and storage medium | |
| CN111640194A (en) | AR scene image display control method and device, electronic equipment and storage medium | |
| CN111640195A (en) | History scene reproduction method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |