WO2019128302A1

WO2019128302A1 - Method for implementing interactive operation, apparatus and client device

Info

Publication number: WO2019128302A1
Application number: PCT/CN2018/104437
Authority: WO
Inventors: 曾志荣
Original assignee: 优视科技有限公司
Priority date: 2017-12-26
Filing date: 2018-09-06
Publication date: 2019-07-04
Also published as: CN108144294B; CN108144294A

Abstract

Provided in the present invention are a method for constructing a virtual scene on the basis of a real map image for user interactions, and an apparatus and a client device. The method for implementing interactive operations comprises: sending geographic location information of a client to a server, and the server querying street view information associated with the geographic location information; the client or the server constructing a virtual street view map on the basis of the street view information; the client merging the virtual street view map with a scene framework so as to obtain a virtual scene; the client showing the virtual scene to a user thereof to enable the user to carry out interactive operations therein. Therefore, a virtual scene that is highly integrated with a real environment but also flexible and variable may be provided, thus further improving the usage experience of the user. (figure 3)

Description

互动操作实现方法、装置及客户端设备Interactive operation realization method, device and client device

技术领域Technical field

本发明涉及信息技术领域，特别是涉及一种在基于实景的虚拟场景下进行互动操作的方法、装置和客户端设备。The present invention relates to the field of information technology, and in particular, to a method, an apparatus, and a client device for performing an interactive operation in a virtual scene based on a real scene.

背景技术Background technique

随着“精灵宝可梦GO”这一游戏风靡全球，结合了实景的沉浸式应用开始进入大众视野。地理位置信息的结合提升了人们的实地参与感，AR技术则通过将计算机生成的虚拟物体、场景或***提示信息叠加到真实场景中，实现对现实的增强。图1A和1B示出了Pokémon GO游戏的截图示例。With the popularity of the game “Pokémon GO”, the immersive application combined with real scene began to enter the public eye. The combination of geographic location information enhances people's sense of participation in the field. AR technology enhances reality by superimposing computer-generated virtual objects, scenes or system prompt information into real scenes. Figures 1A and 1B show examples of screenshots of the Pokémon GO game.

虽然涉及了实地与实景，但如图1A和1B所示，该游戏只是分别使用了LBS和AR技术(图1A是地图模式的例子，图1B则是AR模式的例子)，并非在一个具体场景中融合相关技术以给出更为逼真的使用体验。Although the field and the real scene are involved, as shown in FIGS. 1A and 1B, the game only uses the LBS and AR technologies respectively (FIG. 1A is an example of a map mode, and FIG. 1B is an example of an AR mode), not in a specific scenario. Integrate related technologies to give a more realistic experience.

类似地，现有技术中缺乏对现实场景的深度融合，因此用户的使用沉浸度并不高。Similarly, the prior art lacks deep integration of real scenes, so the user's use immersion is not high.

发明内容Summary of the invention

为了解决上述至少一个问题，本发明提出了一种全新的互动操作方案，其通过获取现有的街景信息生成虚拟街景地图，再根据具体的应用场景叠加场景框架，从而能够在现有网络条件的基础上，以现有移动端设备可承受的计算量为用户提供灵活的，且与环境高度融合的互动场景。该方案还能够进一步地融合AR技术，以便为用户提供更为新奇、有趣的使用体验。In order to solve at least one of the above problems, the present invention proposes a new interactive operation scheme, which generates a virtual street view map by acquiring existing street view information, and then superimposes the scene frame according to a specific application scenario, thereby being able to be in existing network conditions. Based on the calculations that the existing mobile device can withstand, the user is provided with a flexible and highly integrated interactive scene. The solution is also able to further integrate AR technology to provide users with a more novel and interesting experience.

根据本发明的一个方面，提供了一种在客户端侧实现的互动操作实现方法，包括：将地理位置信息发送给服务器；接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息；将基于所述街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景；展示所述虚拟场景；以及响应于用户输入，在所述虚拟场景中进行交互操作。According to an aspect of the present invention, a method for implementing an interactive operation implemented on a client side includes: transmitting geographic location information to a server; and receiving related street view information related to street view information, wherein the street view information is the The street view information associated with the geographic location information queried by the server; the virtual street view map based on the street view information is merged with the scene frame to obtain a virtual scene; the virtual scene is displayed; and in response to user input, the virtual scene Interact in the scene.

由此，能够利用现有的实景图像信息，经过合理变换得出与现实环境深度结合的具体应用场景，从而以可实现的计算量和网络传输要求为用户提供沉浸式的应用体验。Thereby, the existing real-time image information can be utilized to obtain a specific application scenario combined with the depth of the real environment through reasonable transformation, thereby providing the user with an immersive application experience with achievable calculation amount and network transmission requirements.

虚拟街景地图与场景框架的融合可以是由装载的场景框架添加街景信息，而可以正好相反。例如，可以在向已装载的场景框架添加所述街景信息，以形成加载有场景框架的虚拟街景地图，所述虚拟场景是加载有场景框架的虚拟街景地图；或者基于所述街景数据构建所述虚拟街景地图，在所述虚拟街景地图上加载所述场景框架以得到所述虚拟场景。由此，可以基于具体应用场景，灵活选择虚拟场景的构建方式。The fusion of the virtual street view map and the scene frame may be to add street view information from the loaded scene frame, but the opposite is true. For example, the street view information may be added to a loaded scene frame to form a virtual street view map loaded with a scene frame, the virtual scene being a virtual street view map loaded with a scene frame; or constructing the based on the street view data a virtual street view map on which the scene frame is loaded to obtain the virtual scene. Therefore, the manner of constructing the virtual scene can be flexibly selected based on the specific application scenario.

可选地，相关街景信息还可以包括与所述街景信息相关联的地图信息，并且所述虚拟街景地图的构造还基于所述地图信息。通过引入地图信息，可以更方便高效且准确地实现虚拟街景地图的构建。Optionally, the related street view information may further include map information associated with the street view information, and the configuration of the virtual street view map is further based on the map information. By introducing map information, it is more convenient and efficient to implement the construction of virtual street view maps.

场景框架可以包括诸多实现。可选地，场景框架可以包括使用户能够与所显示的虚拟场景或其内目标进行交互的操作面板。可选地，可以至少部分基于场景框架确定虚拟街景地图的环境或显示风格。可选地，虚拟场景中的视线高度至少部分基于所述场景框架确定。The scene framework can include many implementations. Optionally, the scene framework may include an operation panel that enables the user to interact with the displayed virtual scene or objects within it. Optionally, the environment or display style of the virtual street view map may be determined based at least in part on the scene frame. Optionally, the line of sight height in the virtual scene is determined based at least in part on the scene frame.

可选地，虚拟街景地图可以是根据预定地图算法从接收到的街景信息图片中拼接生成的。从而相比于现有技术中单纯的建模或是地图模式，能够以小得多的代价为用户提供更为真实的使用场景。Optionally, the virtual street view map may be generated by splicing from the received street view information picture according to a predetermined map algorithm. Therefore, compared with the simple modeling or map mode in the prior art, the user can provide a more realistic use scenario at a much lower cost.

在用户与场景的互动中，交互操作的交互形式可以至少部分由场景框架决定。由此，能够为相同的虚拟街景地图提供多种多样的交互场景，从而提升本方案的灵活性和适用广泛程度。In the interaction between the user and the scene, the interactive form of the interaction may be determined at least in part by the scene framework. Thereby, a variety of interactive scenarios can be provided for the same virtual street view map, thereby improving the flexibility and application of the solution.

本发明的互动操作实现方法还可以进一步涉及AR技术。可选地可以基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的AR对象；以及在所述虚拟场景中加载所述AR对象。上述AR对象的加载可以是有条件的，例如虚拟场景中可以加载AR对象，但无需立即展示，而是可由特定操作、角度、任务完成等触发。上述的交互操作还可以包括与所述虚拟街景地图中加载并展示的AR对象进行的交互操作。由此，能够进一步丰富本方案的应用场景和适用范围，并提升交互的参与度和趣味性。AR对象的加载还可以响应于对目标对象的拍摄。进一步地，加载所述AR对象的相关数据被用于确定当前地理位置信息和/或所述虚拟场景的生成与更新。The interactive operation implementation method of the present invention may further relate to AR technology. Optionally, an AR object to be loaded may be acquired based on the geographic location information, the street view information, and/or the scene frame; and the AR object is loaded in the virtual scene. The loading of the above AR object may be conditional. For example, an AR object may be loaded in a virtual scene, but need not be displayed immediately, but may be triggered by a specific operation, angle, task completion, and the like. The interaction described above may also include an interaction with an AR object loaded and displayed in the virtual street view map. Thereby, the application scenario and the applicable scope of the solution can be further enriched, and the participation and interest of the interaction can be improved. The loading of the AR object can also be responsive to the shooting of the target object. Further, the relevant data for loading the AR object is used to determine current geographic location information and/or generation and update of the virtual scene.

客户端提交的地理位置信息可以是用户选定的地理位置信息或是用户的当前地理位置信息。根据选定地理信息构建虚拟场景，可以使得用户足不出户，尽情体验各种场景框架下建立在各类实际地点的虚拟场景，从而丰富用户的使用体验。倘若基于的是当前地理位置信息，则能够根据用户的实际所在地构建虚拟场景，进一步增添用户的临场感。The geographic location information submitted by the client may be the geographical location information selected by the user or the current geographic location information of the user. The virtual scene is constructed according to the selected geographic information, so that the user can experience the virtual scenes established in various actual locations under various scene frameworks without leaving the home, thereby enriching the user experience. If the current geographical location information is based on, the virtual scene can be constructed according to the actual location of the user, further adding the user's sense of presence.

根据具体的应用场景，可以为用户提供各种交互水平。可选地，可以以用户的第一视角展示所述虚拟场景。例如，可以使用移动端内置的陀螺仪等捕捉移动端的运动轨迹，将其实时反映在虚拟场景的显示内容上，从而提升应用的身临其境感。例如，在具体实现中，可以将例如用户手持的移动端的当前地理位置信息、速度信息和/或视角信息实时发送给所述服务器；并根据所述当前地理位置、速度信息和/或视角信息实时改变展示的虚拟街景地图的视角和内容。According to specific application scenarios, users can be provided with various levels of interaction. Optionally, the virtual scene may be presented in a first perspective of the user. For example, the moving trajectory of the mobile end can be captured by using a gyroscope built in the mobile terminal, and reflected in real time on the display content of the virtual scene, thereby improving the immersive feeling of the application. For example, in a specific implementation, current geographic location information, speed information, and/or perspective information of, for example, a mobile terminal held by a user may be sent to the server in real time; and real-time according to the current geographic location, speed information, and/or perspective information. Change the perspective and content of the virtual Street View map displayed.

本发明的互动操作方案还可以涉及多用户的联网操作。可选地，用户可以获取联网的一个或多个其他客户端的信息(其他用户信息)，并在该用户的虚拟街景地图中展示所述一个或多个其他客户端的化身。该用户可以经由其化身与其他客户端的化身进行交互。由此进一步提升具体场景的参与度，并为应用赋予社交性。The interactive operation scheme of the present invention may also involve multi-user networking operations. Alternatively, the user may obtain information (other user information) of one or more other clients connected to the network and display the avatars of the one or more other clients in the virtual street view map of the user. The user can interact with the avatars of other clients via their avatars. This further enhances the participation of specific scenarios and socializes the application.

根据本发明的另一个方面，提供了一种在服务器侧实现的互动操作实现方法，包括：获取客户端发送的地理位置信息；查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，其中，所述虚拟场景是将虚拟街景地图与场景框架相融合得到的。According to another aspect of the present invention, a method for implementing an interaction operation implemented on a server side includes: acquiring geographic location information sent by a client; querying street view information associated with the geographic location information or based on the street view The virtual street view map information generated by the information; the street view information or the virtual street view map information is sent to the client, so that the client displays a virtual scene in which the client user can perform an interaction operation, wherein the virtual scene is The virtual street view map is merged with the scene frame.

在涉及AR技术的情况下，该方法还可以包括：基于所述地理位置信息、所述街景信息、所述场景框架和/或预定展示条件发送要在所述虚拟场景中加载的AR对象，以供在所述虚拟场景中展示。In the case of involving AR technology, the method may further include: transmitting an AR object to be loaded in the virtual scene based on the geographical location information, the street view information, the scene frame, and/or a predetermined presentation condition, to For display in the virtual scene.

在涉及多人联网的情况下，该方法还可以包括：获取联网的一个或多个其他客户端的信息；在所述虚拟空间平台中同步所述一个或多个其他客户端的化身；以及将所述其他客户端的化身发送给所述客户端以在所述客户端的虚拟场景中展示。可选地，该方法还可以包括：实时接收所述客户端和/或其他客户端的化身的操作并进行同步处理；以及下发所述同步处理以实现所述客户端与其他客户端的化身之间的交互操作。In the case of multi-person networking, the method may further comprise: obtaining information of one or more other clients of the network; synchronizing the avatars of the one or more other clients in the virtual space platform; and The avatars of other clients are sent to the client for presentation in the virtual scene of the client. Optionally, the method may further include: receiving an operation of the avatar of the client and/or other client in real time and performing synchronization processing; and delivering the synchronization process to implement the avatar between the client and other clients. Interaction.

根据本发明的再一个方面，提供了一种在客户端侧实现的互动操作实现装置，包括：位置信息发送单元，用于将地理位置信息发送给服务器；街景信息接收单元，用于接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息；虚拟场景构建单元，用于将基于街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景；虚拟场景展示单元，用于展示所述虚拟场景；以及交互单元，用于响应于用户输入，在所述虚拟场景中进行交互操作。According to still another aspect of the present invention, an apparatus for implementing an interactive operation implemented on a client side includes: a location information sending unit for transmitting geographic location information to a server; and a street view information receiving unit for receiving and viewing a street view Information related to the street view information, wherein the street view information is street view information associated with the geographic location information queried by the server; the virtual scene construction unit is configured to merge the virtual street view map based on the street view information with the scene frame Obtaining a virtual scene; a virtual scene display unit, configured to display the virtual scene; and an interaction unit, configured to perform an interaction operation in the virtual scene in response to user input.

可选地，该互动操作实现装置还可以包括：目标对象获取单元，基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的目标对象；以及目标对象加载单元，用于在所述虚拟场景中加载所述目标对象。Optionally, the interaction operation implementation apparatus may further include: a target object acquisition unit, acquiring a target object to be loaded based on the geographical location information, the street view information, and/or the scene framework; and a target object loading unit, Loading the target object in the virtual scene.

可选地，所述目标对象加载单元在满足预定展示条件的情况下，在所述虚拟场景中展示加载的所述目标对象。Optionally, the target object loading unit displays the loaded target object in the virtual scene if a predetermined display condition is met.

相应的，所述互动操作实现装置还可以包括信息同步单元，用于在所述虚拟街景地图中同步展示所述一个或多个其他客户端的化身。Correspondingly, the interactive operation implementing device may further comprise an information synchronization unit, configured to synchronously display the avatars of the one or more other clients in the virtual street view map.

根据本发明的再一个方面，提供了一种在服务器侧实现的互动操作实现装置，包括：According to still another aspect of the present invention, an apparatus for implementing an interactive operation implemented on a server side is provided, including:

位置信息获取单元，用于查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；a location information obtaining unit, configured to query street view information associated with the geographic location information or virtual street view map information generated based on the street view information;

街景信息查询单元，用于查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；a street view information query unit, configured to query street view information associated with the geographic location information or virtual street view map information generated based on the street view information;

街景信息发送单元，用于将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，其中，所述虚拟场景是虚拟街景地图与场景框架融合得到的。a street view information sending unit, configured to send the street view information or the virtual street view map information to a client, so that the client displays a virtual scene in which the client user can perform an interaction operation, wherein the virtual scene is virtual The street view map is merged with the scene frame.

可选地，互动操作实现装置还包括：Optionally, the interaction operation implementation device further includes:

构建单元，用于基于街景信息构建虚拟街景地图，和/或在虚拟街景地图上加载场景框架以构建虚拟场景空间。A building unit for constructing a virtual street view map based on street view information, and/or loading a scene frame on the virtual street view map to construct a virtual scene space.

可选地，街景信息发送单元用于基于所述地理位置信息、所述街景信息、所述场景框架和/或预定展示条件发送要在所述虚拟场景中加载的AR对象，以供在所述虚拟场景中展示。Optionally, the street view information sending unit is configured to send an AR object to be loaded in the virtual scene based on the geographical location information, the street view information, the scene frame, and/or a predetermined display condition, for Displayed in a virtual scene.

同步单元，用于在所述虚拟空间平台中同步所述多个客户端的化身的操作与位置。And a synchronization unit, configured to synchronize operations and locations of the avatars of the plurality of clients in the virtual space platform.

根据本发明的一个方面，提供了一种客户端设备，包括：处理器；以及存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如上所述的在客户端侧实现的互动操作实现方法。According to an aspect of the present invention, a client device is provided, comprising: a processor; and a memory having stored thereon executable code that, when executed by the processor, causes the processor The interactive operation implementation method implemented on the client side as described above is performed.

根据本发明的一个方面，提供了一种服务器，包括：处理器；以及存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如上所述的在服务器侧实现的互动操作实现方法。According to an aspect of the present invention, a server is provided, comprising: a processor; and a memory having stored thereon executable code that, when executed by the processor, causes the processor to execute as above The method for implementing an interactive operation implemented on the server side.

根据本发明的再一个方面，提供了一种非暂时性机器可读存储介质，其上存储有可执行代码，当所述可执行代码被电子设备的处理器执行时，使所述处理器执行如上所述的互动操作实现方法。According to still another aspect of the present invention, there is provided a non-transitory machine readable storage medium having stored thereon executable code that, when executed by a processor of an electronic device, causes the processor to execute The interactive operation implementation method as described above.

由此，本发明通过基于实景地图图像构建虚拟场景以供用户交互，能够提供与现实环境高度融合但却灵活可变的虚拟场景，进一步提升用户的使用体验。Therefore, the present invention can provide a virtual scene that is highly integrated with the real environment but is flexible and variable, thereby constructing a virtual scene based on the real-life map image for user interaction, thereby further improving the user experience.

附图说明DRAWINGS

通过结合附图对本公开示例性实施方式进行更详细的描述，本公开的上述以及其它目的、特征和优势将变得更加明显，其中，在本公开示例性实施方式中，相同的参考标号通常代表相同部件。The above and other objects, features, and advantages of the present invention will become more apparent from the aspects of the embodiments of the invention. The same parts.

图1A和1B示出了“精灵宝可梦GO”游戏的截图示例。Figures 1A and 1B show examples of screenshots of the "Pokémon GO" game.

图2是用于实现本发明实施例的环境的示意图。2 is a schematic diagram of an environment for implementing an embodiment of the present invention.

图3示出了根据本发明一个实施例的互动操作方案的示意图。FIG. 3 shows a schematic diagram of an interactive operation scheme in accordance with one embodiment of the present invention.

图4A和4B示出了根据本发明一个实施例的赛车应用场景的例子。4A and 4B illustrate an example of a racing application scenario in accordance with one embodiment of the present invention.

图5示出了根据本发明一个实施例的客户端侧的互动操作实现方法的流程图。FIG. 5 is a flow chart showing a method of implementing an interactive operation on the client side according to an embodiment of the present invention.

图6示出了根据本发明一个实施例的服务器侧的互动操作实现方法的流程图。FIG. 6 is a flow chart showing a method of implementing an interactive operation on the server side according to an embodiment of the present invention.

图7示出了根据本发明一个实施例的在客户端侧实现的互动操作实现装置的示意图。FIG. 7 shows a schematic diagram of an interactive operation implementing apparatus implemented on a client side according to an embodiment of the present invention.

图8示出了根据本发明一个实施例的在服务器侧实现的互动操作实现装置的示意图。FIG. 8 shows a schematic diagram of an interactive operation implementing apparatus implemented on a server side according to an embodiment of the present invention.

图9是根据本发明一个实施例的客户端设备。Figure 9 is a client device in accordance with one embodiment of the present invention.

具体实施方式Detailed ways

下面将参照附图更详细地描述本公开的可选实施方式。虽然附图中显示了本公开的可选实施方式，然而应该理解，可以以各种形式实现本公开而不应被这里阐述的实施方式所限制。相反，提供这些实施方式是为了使本公开更加透彻和完整，并且能够将本公开的范围完整地传达给本领域的技术人员。Alternative embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the embodiments of the present invention are shown in the drawings, it is understood that the invention may be embodied in various forms and not limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

随着移动端处理能力的增强和网络质量的全面提升，现有的结合当前地理位置信息的应用由于其使用场景单一、真实性差和/或交互性弱，已经越来越无法满足人们生活、学习和娱乐的丰富需求。有鉴于此，本发明以相对较低的计算和网络要求实现了真实场景和虚拟场景的良好结合，从而为用户提供深度的沉浸式体验。With the enhancement of mobile processing capabilities and the overall improvement of network quality, existing applications that combine current geographic location information have become increasingly incapable of satisfying people's lives and learning due to their single use scenarios, poor authenticity, and/or weak interactivity. And the rich demand for entertainment. In view of this, the present invention achieves a good combination of real scenes and virtual scenes with relatively low computational and network requirements, thereby providing users with a deep immersive experience.

本发明实施例所提供的互动场景构建和参与方案可应用于如图2所示的环境中。图2是用于实现本发明实施例的环境200的示意图。在一个实施例中，环境200中的移动终端10可以经由网络40实现与服务器20的信息收发。服务器20可以通过访问数据库30来获取移动终端10所需的内容。在一个实施例中，该数据库30可以是存储有街景信息的街景信息数据库，服务器20则可以通过访问该数据库获取移动端10所请求的街景信息。移动终端之间(例如，10_1与10_2或10_N之间)可选地可以经由网络40彼此通信。网络40可以是广义上的用于信息传递的网络，可以包括一个或多个通信网络，诸如无线通信网络、因特网、私域网、局域网、城域网、广域网或是蜂窝数据网络等。在一个实施例中，网络40也可以包括卫星网络，由此将移动终端10的GPS信号传送给服务器20。应当注意，如果向图示200中添加或从图示200中去除附加模块，不会改变本发明的示例实施例的底层概念。另外，虽然为了方便说明而在图中示出了从数据库30到服务器20的双向箭头，但本领域技术人员可以理解的上，上述数据的收发也是可以通过网络40实现的。The interactive scenario construction and participation scheme provided by the embodiments of the present invention can be applied to the environment shown in FIG. 2. 2 is a schematic diagram of an environment 200 for implementing an embodiment of the present invention. In one embodiment, mobile terminal 10 in environment 200 can effect messaging with server 20 via network 40. The server 20 can acquire the content required by the mobile terminal 10 by accessing the database 30. In one embodiment, the database 30 may be a street view information database storing street view information, and the server 20 may obtain the street view information requested by the mobile terminal 10 by accessing the database. The mobile terminals (e.g., between 10_1 and 10_2 or 10_N) may optionally communicate with one another via the network 40. Network 40 may be a network for information transfer in a broad sense, and may include one or more communication networks, such as a wireless communication network, the Internet, a private area network, a local area network, a metropolitan area network, a wide area network, or a cellular data network. In one embodiment, network 40 may also include a satellite network, thereby transmitting GPS signals of mobile terminal 10 to server 20. It should be noted that if additional modules are added to or removed from the illustration 200, the underlying concepts of the example embodiments of the present invention are not altered. In addition, although a two-way arrow from the database 30 to the server 20 is shown in the figure for convenience of explanation, those skilled in the art can understand that the above-mentioned data can also be transmitted and received through the network 40.

移动终端10是可用来进行网络访问的任何合适的便携式电子设备，包括但不限于智能电话、平板电脑或是其他便携式客户端。服务器20则是能够通过网络访问的提供交互服务所需信息的任何服务器。图中虽然示出了多个移动终端10-1…N以及单个服务器20和数据库30，并且在随后的描述中会选择其中的一个或部分移动终端加以描述(例如，移动终端10-1)，但是本领域技术人员应该立即的是，上述1…N个移动终端旨在表示真实网络中存在的多个移动终端，示出的单个服务器20和数据库30旨在表示本发明的技术方案涉及服务器及数据库的操作。对特定编号的移动端以及单个服务器和数据库加以详述至少为了说明方便，而非暗示对移动端和服务器的类型或是位置等具有限制。 Mobile terminal 10 is any suitable portable electronic device that can be used for network access, including but not limited to a smart phone, tablet or other portable client. Server 20 is any server that is accessible over the network to provide the information needed for the interactive service. Although a plurality of mobile terminals 10-1...N and a single server 20 and database 30 are shown in the figure, one or a part of the mobile terminals will be described in the following description (for example, the mobile terminal 10-1), However, it should be immediately apparent to those skilled in the art that the above 1...N mobile terminals are intended to represent a plurality of mobile terminals existing in a real network, and the illustrated single server 20 and database 30 are intended to indicate that the technical solution of the present invention relates to a server and The operation of the database. The specific numbered mobile end and the individual servers and databases are detailed for at least convenience of explanation, and do not imply restrictions on the type or location of the mobile end and the server.

图3示出了根据本发明一个实施例的互动操作方案的示意图。在该互动操作方案的实现环境中，包括至少一台服务器S和至少一个客户端A。在互动方案包括多人联网交互的可选实施例中，该实现环境包括至少两个客户端A和B。在此，至少一台服务器S可以是图2所示环境中的服务器20，客户端A可以是图2所示的任一移动端10。该服务器S经由网络或其他连接与街景信息服务器相连，以便为客户端A获取其所在位置的街景信息。FIG. 3 shows a schematic diagram of an interactive operation scheme in accordance with one embodiment of the present invention. In the implementation environment of the interactive operation scheme, at least one server S and at least one client A are included. In an alternative embodiment where the interactive scheme includes multi-person networking interaction, the implementation environment includes at least two clients A and B. Here, at least one server S may be the server 20 in the environment shown in FIG. 2, and the client A may be any mobile terminal 10 shown in FIG. 2. The server S is connected to the Street View information server via a network or other connection to obtain the Street View information of the location of the client A.

首先，客户端A将某一地理位置信息发送给服务器S。First, client A sends a certain geographic location information to server S.

客户端A发送给服务器S的地理位置信息理论上可以是客户端A的用户选定的任何地理位置信息。例如，用户可以在地图应用中选定某一坐标，甚至在地图应用的街景模式下选定的特定街景的图像截图，并将相应的信息发送给服务器S。The geographical location information sent by client A to server S can theoretically be any geographic location information selected by the user of client A. For example, the user may select a certain coordinate in the map application, or even an image screenshot of the specific street view selected in the street view mode of the map application, and send the corresponding information to the server S.

在另一个实施例中，客户端A发送的可以是其当前地理位置信息。在此，客户端A可以利用LBS(基于位置的服务)来获取其所在的地理位置信息，并例如经由无线通讯网络(例如，4G网络或是WiFi)将其发送给服务器S。In another embodiment, client A may send its current geographic location information. Here, the client A can use the LBS (Location Based Service) to obtain the geographical location information of the location, and send it to the server S, for example, via a wireless communication network (for example, 4G network or WiFi).

基于不同的应用场景下对地理位置信息精度要求的不同，客户端A所需发送的地理位置信息也不同。考虑到智能手机在室外的GPS定位精度通常小于10米，因此可以设计基于此精度要求的应用场景。在高精度场景要求下，例如，在真实世界中第一视角跟随模式的应用场景，例如可以要求客户端A的用户额外配备精度在一米以下，甚至几厘米的手持GPS设备，该设备可以将极为精确的地理位置信息实时发送给客户端A。在另一个例子中，针对地理位置信息的高精度要求，可以首先要求客户端A例如站在特定位置朝向特定方向或是对准特定目标，以此作为交互动作的基准起点。随后通过客户端A内置的指南针、陀螺仪等再结合LBS服务，实时修正客户端A的位置和朝向。The geographic location information required by client A is different based on the accuracy requirements of geographic location information in different application scenarios. Considering that the GPS positioning accuracy of the smartphone outdoors is usually less than 10 meters, it is possible to design an application scenario based on this accuracy requirement. In the case of high-precision scene requirements, for example, in the real world, the application scene of the first angle of view following mode, for example, the user of client A may be required to additionally have a handheld GPS device with an accuracy of less than one meter or even several centimeters, and the device may Extremely accurate geographic location information is sent to Client A in real time. In another example, for high-precision requirements of geographic location information, client A may first be required to target a particular direction, for example, or to target a particular target, as a baseline starting point for the interaction. Then, through the built-in compass, gyroscope, etc. of the client A, the LBS service is combined with the LBS service to correct the position and orientation of the client A in real time.

随后，服务器S接收客户端A发送的地理位置信息，并例如在图2所示的街景信息数据库30中查询与该地理位置信息相关联的街景信息。Subsequently, the server S receives the geographical location information transmitted by the client A, and queries the street view information database 30 shown in FIG. 2 for the street view information associated with the geographical location information.

在此，街景信息指的是能够显示街道、室内、公共建筑或其他环境的360度的全景实景图像。“街景地图”是一种实景地图服务，例如，用户可以在谷歌地图的街景模式下，或是百度地图的全景模式下查询到的、针对实际景物且视角可变的图像。由于街景地图的原始数据通常是由在特定路面上行驶的采集车辆上安装的可旋转镜头采集的，因此街景地图提供的实景信息通常是以采集车辆安装镜头高度为视线高度、全视角且可移动查询的连续的街景信息。但应该理解的是，虽然被称为“街景信息”，但其实际指代的是视角可变的实景信息，实景可以并且通常是街景，但也可以指代真实“街景”之外的其他环境的实景，例如公共建筑或其内部的实景信息，自然景观的实景信息等。Here, street view information refers to a 360-degree panoramic real-life image capable of displaying a street, an indoor, a public building, or other environment. "Street View Map" is a real-life map service, for example, an image that can be queried by a user in a street view mode of Google Maps or a panoramic mode of a Baidu map, which is aimed at an actual scene and has a variable viewing angle. Since the raw data of the street view map is usually collected by a rotatable lens mounted on a collecting vehicle traveling on a specific road surface, the real scene information provided by the street view map is usually the height of the line of the installation lens of the vehicle, the full view angle and the movable Query continuous street view information. However, it should be understood that although it is called "Street View Information", it actually refers to real-life information with variable viewing angles. The real scene can and usually is a street view, but it can also refer to other environments other than the real "street view". Real scenes, such as public buildings or real-life information inside them, real-life information of natural landscapes, etc.

在一个实施例中，服务器S可以直接访问由现有的“街景地图”服务采集并维护的街景信息数据库，例如，直接访问百度或高德地图的街景信息数据库。在其他实施例中，服务器S也可以访问其自行采集、构建并维护的街景信息数据库。这在实现虚拟街景地图的范围有限时尤为使用。例如，可以采集北京故宫的(包括室内外建筑全景的)实景信息数据，构造故宫的街景(实景)数据库，并可由服务器S访问。In one embodiment, the server S can directly access the street view information database collected and maintained by the existing "Street View Map" service, for example, directly accessing the Street View information database of Baidu or Gaode maps. In other embodiments, the server S may also access a street view information database that it collects, builds, and maintains. This is especially useful when the range of virtual street view maps is limited. For example, real-time information data of the Forbidden City in Beijing (including panoramic views of indoor and outdoor buildings) can be collected to construct a street view (real view) database of the Forbidden City, which can be accessed by the server S.

基于上述街景信息，可以构建虚拟街景地图。在一个实施例中，虚拟街景地图在服务器端(即，服务器S上)构建，并经由网络发送给客户端A。客户端A直接获取上述虚拟街景地图，并进行显示或后续处理。在另一个实施例中，服务器S可以将街景信息发送给客户端A，并在客户端A完成对虚拟街景地图的构建。在又一个实施例中，服务器S可以完成虚拟街景地图的部分构建，并在客户端A处完成剩余部分的构建。Based on the above street view information, a virtual street view map can be constructed. In one embodiment, the virtual street view map is built on the server side (ie, on server S) and sent to client A via the network. Client A directly obtains the above virtual street view map and performs display or subsequent processing. In another embodiment, the server S can send street view information to the client A and complete the construction of the virtual street view map at the client A. In yet another embodiment, the server S can complete the partial construction of the virtual street view map and complete the construction of the remaining portion at the client A.

在再一个实施例中，虚拟街景地图(或其部分)的构建甚至可以在接收到来自客户端A的地理位置信息之前就已经完成。例如，服务器S可以预先存储某些特定地理位置的虚拟街景地图，或是将该虚拟街景地图(或其部分)预先下发至客户端A。当服务器S接收到来自该客户端A的与该特定地理位置相符的地理位置信息时，就直接向客户端A传输该虚拟街景地图或是启用已经下载在客户端A上的虚拟街景地图。这在特定地理位置是被频繁请求或是需要进行商业推广的地点，例如热门景点或是商业活动宣传目标时尤为有效。In still another embodiment, the construction of the virtual street view map (or portions thereof) may even be completed prior to receiving geographic location information from client A. For example, the server S may pre-store the virtual street view map of some specific geographic locations, or pre-deliver the virtual street view map (or a part thereof) to the client A. When the server S receives the geographical location information from the client A that matches the specific geographic location, the virtual street view map is directly transmitted to the client A or the virtual street view map that has been downloaded on the client A is enabled. This is especially true when a particular location is a location that is frequently requested or needs to be commercialized, such as a popular attraction or a business event promotion goal.

由此，由服务器S传送给客户端A的“相关街景信息”指的是与上述街景信息(即，与服务器查询到的与地理位置信息相关联的街景信息)有关的信息，其可以是前述的街景信息本身，可以是基于街景信息构建的部分或全部虚拟街景地图，也可以是启用客户端A本机上存储的虚拟街景地图的指令。Thus, the "relevant street view information" transmitted by the server S to the client A refers to information related to the above-mentioned street view information (ie, street view information associated with the geographical location information queried by the server), which may be the foregoing The street view information itself may be part or all of the virtual street view map built based on the street view information, or may be an instruction to enable the virtual street view map stored on the client A local machine.

在此，“虚拟街景地图”指的是基于实景图像构建的，具有高度真实感和还原性的地图空间。在一个实施例中，“虚拟街景地图”可选可以指代视角可变的、类似于三维实景的地图空间。但不同于需要大量计算的三维建模方法，本发明中的虚拟街景地图可以是根据预定地图算法从接受到的街景信息图片中拼接生成的。由此，能够以可接受的计算代价来提供使用的真实性。在一个实施例中，“虚拟街景地图”可以根据具体应用场景，具有与其相适应的重建水平。例如，在游戏场景中，可以仅根据实景重建道路等主体内容，而忽略其中的实体建筑。在商业推广场景中，重建例如需要包括实体建筑，甚至实体建筑内的具体结构，以便与具体的店铺进行操作。Here, “virtual street view map” refers to a map space constructed with real-life images and highly realistic and reductive. In one embodiment, the "virtual street view map" may optionally refer to a map space that is variable in perspective, similar to a three-dimensional real scene. However, unlike the three-dimensional modeling method that requires a large amount of calculation, the virtual street view map in the present invention may be generated by splicing from the received street view information picture according to a predetermined map algorithm. Thereby, the authenticity of the use can be provided at an acceptable computational cost. In one embodiment, the "virtual street view map" may have a level of reconstruction that is appropriate for the particular application scenario. For example, in a game scene, it is possible to reconstruct a main body content such as a road based on a real scene, and ignore the physical building therein. In commercial promotion scenarios, reconstructions, for example, need to include physical buildings, or even specific structures within a physical building, to operate with a particular store.

在一个实施例中，“虚拟街景地图”可以完全基于街景数据(例如，经由街景图片拼接)构建。在另一个实施例中，“虚拟街景地图”可以基于地图数据和街景数据构建。例如，地图数据可以用于标记道路和建筑甚至商户，以便于对街景数据(图片)中道路和/或建筑物的划分，从而实现针对不同应用场景的不同重建级别的虚拟街景地图。在此情况下，客户端A还可以从服务器S获取构建虚拟街景地图所需的地图数据。上述地图数据例如可以包括在相关街景数据中，或与其一并发送。虽然“虚拟街景地图”是一个虚拟地图空间，但其在客户端A显示给用户的，是类似于当前地图应用街景模式中的实景图像的“虚拟”版本。而这些随用户移动或操作而连续变化的虚拟街景图像可以构成虚拟街景地图空间。In one embodiment, the "virtual street view map" may be built entirely based on street view data (eg, via street view picture stitching). In another embodiment, a "virtual street view map" can be constructed based on map data and street view data. For example, map data can be used to mark roads and buildings or even merchants to facilitate the partitioning of roads and/or buildings in street view data (pictures) to enable virtual street view maps of different reconstruction levels for different application scenarios. In this case, the client A can also obtain the map data required to construct the virtual street view map from the server S. The map data described above may be included, for example, in the related street view data or transmitted therewith. Although the "virtual street view map" is a virtual map space, it is displayed to the user on the client A, which is similar to the "virtual" version of the live view image in the current map application street view mode. The virtual street view images that continuously change as the user moves or operates can form a virtual street view map space.

客户端A可以基于接收到的相关街景信息得到街景信息本身或是构建虚拟街景地图，基于街景信息的虚拟街景地图可以与场景框架相融合，以得到虚拟场景。在一个实施例中，可以在客户端A中预先加载场景框架，并装入从服务器获得的街景信息，由此使得加载的场景框架与基于街景信息的虚拟街景地图相融合，从而得到虚拟场景。在另一个实施例中，可以基于街景信息构建虚拟街景地图，并在该虚拟街景地图上加载场景框架，并向其用户展示加载了所述场景框架的虚拟街景地图，从而得到虚拟场景。在此，可以将加载了所述场景框架的虚拟街景地图或是加载了虚拟街景地图的场景框架称为“虚拟场景”。用户可以在上述虚拟场景中进行相应的交互操作，以实现与该具体场景或其中显示的对象的互动。Client A can obtain the street view information itself or construct a virtual street view map based on the received related street view information, and the virtual street view map based on the street view information can be merged with the scene frame to obtain the virtual scene. In one embodiment, the scene frame may be preloaded in the client A and loaded with street view information obtained from the server, thereby merging the loaded scene frame with the virtual street view map based on the street view information, thereby obtaining a virtual scene. In another embodiment, a virtual street view map may be constructed based on the street view information, and the scene frame is loaded on the virtual street view map, and the virtual street view map loaded with the scene frame is displayed to its user, thereby obtaining a virtual scene. Here, the virtual street view map loaded with the scene frame or the scene frame loaded with the virtual street view map may be referred to as a “virtual scene”. The user can perform corresponding interaction operations in the above virtual scene to implement interaction with the specific scene or the object displayed therein.

“框架(Framework)”可以指的是整个或部分***的可复用设计，表现为一组抽象构件及构件实例间交互的方法。在此，“场景框架”是与具体应用场景相关联的框架，即与具体应用场景相关联的可复用的设计。在具体实现中，可以通过weex、Html5或是直接植入的方式来实现上述“场景框架”的加载。"Framework" can refer to a reusable design of a whole or part of a system, represented as a set of abstract components and methods of interaction between component instances. Here, the "scene framework" is a framework associated with a specific application scenario, that is, a reusable design associated with a specific application scenario. In the specific implementation, the loading of the above “scene frame” can be implemented by weex, Html5 or directly implanted.

在一个实施例中，“场景框架”可以包括使用户能够与所显示的场景或其内目标进行交互的操作面板。In one embodiment, the "scene framework" may include an operator panel that enables a user to interact with the displayed scene or objects within it.

在一个实施例中，“场景框架”可以包括与具体应用场景相适应的环境或显示风格的设定。即，虚拟街景地图的显示风格或是显示环境，可以至少部分由场景框架所确定。In one embodiment, the "scene frame" may include settings for an environment or display style that are adapted to the particular application scenario. That is, the display style or display environment of the virtual street view map can be determined at least in part by the scene frame.

在一个实施例中，“场景框架”可以包括与具体应用场景相适应的视线高度的设定。即，虚拟场景中的视线高度至少部分基于所述场景框架确定。In one embodiment, the "scene frame" may include settings for line of sight height that are appropriate for the particular application scenario. That is, the line of sight height in the virtual scene is determined based at least in part on the scene frame.

另外，如图3所示，场景框架的加载可以在客户端A侧实现，但对该场景框架本身的获取时间不做限制。在一个实施例中，该场景框架可以是例如客户端A在下载应用(例如，移动端APP)时，应用内自带的场景框架，即，场景框架是预先下载并存储在客户端A侧的。在另一个实施例中，该场景框架可以是跟随如上相关街景信息一并从服务器S传送至客户端A的。在又一个实施例中，可以在应用更新或是传输相关街景信息时，一并下发对场景框架的更新。In addition, as shown in FIG. 3, the loading of the scene framework can be implemented on the client A side, but the acquisition time of the scene framework itself is not limited. In an embodiment, the scenario framework may be, for example, a scenario framework that is included in the application when the client A downloads the application (for example, the mobile APP), that is, the scenario frame is pre-downloaded and stored on the client A side. . In another embodiment, the scene frame may be followed by the related street view information and transmitted from the server S to the client A. In another embodiment, the update of the scene frame may be issued together when the application updates or transmits related street view information.

在本发明的实施例中，可以提供有多个场景框架。例如，可以提供如下图4所示的“赛车场景”框架，可以提供“城市射击场景”框架，“商务推广场景”框架，“古代RPG场景”框架等等。这些框架可以基于用户选择或是具体安装的应用而被加载在相同的虚拟街景地图上，由此丰富本发明实施例的应用范围。In an embodiment of the invention, multiple scene frames may be provided. For example, a "racing scene" framework as shown in Figure 4 below can be provided, which can provide a "city shooting scene" framework, a "business promotion scene" framework, an "ancient RPG scene" framework, and the like. These frameworks can be loaded on the same virtual street view map based on user selection or specifically installed applications, thereby enriching the scope of application of embodiments of the present invention.

在此，“虚拟场景”是指具体应用场景与实景地图结合后得到的地图空间。在一个更为常见的实施例中，“虚拟场景”中显示的道路至少与客户端A上传的地理位置信息相关联的物理场景(现实中的实际场景)中的道路的走向相一致，以便用户交互的顺利完成。但周围建筑甚至是道路形式都可以与现实不同。Here, the “virtual scene” refers to a map space obtained by combining a specific application scenario with a real-life map. In a more common embodiment, the road displayed in the "virtual scene" is at least consistent with the direction of the road in the physical scene (actual scene in reality) associated with the geographic location information uploaded by the client A, so that the user The interaction is completed smoothly. But the surrounding buildings and even the road forms can be different from reality.

图4A和4B示出了根据本发明一个实施例的赛车应用场景的例子。图4A示出了在街景采集车视角高度下的、视角和位置可变的街景视图的一张截图。图4B是基于该位置的虚拟街景地图并加载了赛车控制室及其形式模型框架的赛车应用场景的截图。由于图4涉及的是赛车应用场景，因此如图4B所示，在该“场景框架”中，包括用于赛车操作的面板和指标显示，且视线高度设定与汽车驾驶室高度相当。由于仅涉及路面操作，因此其虚拟街景地图中的重建主要涉及道路本身，忽略了周边建筑。加载的场景框架则用工业建筑加以填充。由此得到的虚拟场景使得用户能够身临其境地进行符合实际道路走向的赛车驾驶的同时，又在一定程度上远离自己身处生活区的事实，从而为用户提供奇妙的赛车体验。4A and 4B illustrate an example of a racing application scenario in accordance with one embodiment of the present invention. Figure 4A shows a screenshot of a view and variable position street view at the height of the street view car's viewing angle. 4B is a screenshot of a racing application scenario based on a virtual street view map of the location and loaded with a racing control room and its formal model framework. Since FIG. 4 relates to a racing application scenario, as shown in FIG. 4B, in the "scene frame", a panel and indicator display for racing operation is included, and the line of sight height setting is equivalent to the height of the car cab. Since it only involves road operations, the reconstruction in its virtual street view map mainly involves the road itself, ignoring the surrounding buildings. The loaded scene frame is filled with industrial buildings. The resulting virtual scene allows the user to immersively drive the car in line with the actual road direction while being somewhat away from the fact that he is in the living area, thus providing the user with a wonderful racing experience.

在本发明的一个实施例中，用户与具体场景或其内对象的交互操作的操作形式至少部分由所述场景框架决定。如图4B所示，当加载了赛车场景框架时，用户的交互操作形式是由该赛车场景决定的驾驶操作。In one embodiment of the invention, the operational form of the user's interaction with a particular scene or object within it is determined, at least in part, by the scene framework. As shown in FIG. 4B, when the racing scene frame is loaded, the user's interactive operation form is a driving operation determined by the racing scene.

由上可知，根据实景图像和场景框架构建的虚拟场景是至少部分与现实场景相符的虚拟空间。在将构建好的虚拟场景经由客户端A展现给用户之后，可以基于用户的屏幕或键盘操作，以连续变化的视角和位置来显示虚拟场景中变化的空间内容。而在虚拟场景的构建是基于用户的当前地理位置信息的情况下，可以基于客户端A实时上报的自身的地理位置信息，以及能由客户端A内置的各类传感器感知的用户的物理运动，以连续变化的视角和位置来显示虚拟场景中变化的空间内容。It can be seen from the above that the virtual scene constructed according to the real scene image and the scene frame is a virtual space at least partially conforming to the real scene. After the constructed virtual scene is presented to the user via the client A, the changed spatial content in the virtual scene can be displayed in a continuously changing perspective and position based on the user's screen or keyboard operation. In the case that the virtual scene is constructed based on the current geographic location information of the user, the geographic location information reported by the client A in real time and the physical motion of the user that can be perceived by various sensors built in the client A may be used. The spatial content of the changes in the virtual scene is displayed in a continuously changing perspective and position.

在一个实施例中，用于构建虚拟场景的实景图像可以是客户端A从服务器S持续获取的。例如，尤其在客户端A发送的位置是当前地理位置信息的情况下，客户端A可以持续更新其当前地理位置信息并与服务器S通信，以便持续从服务器S获取与其地理位置信息相对应的实景图像，这些实景图像用于虚拟场景中显示的空间内容的实时更新。这尤其适用于客户端A的用户在交互过程中物理位移较大的情况，网络传输性能优良的情况(例如，覆盖了高速WiFi)，和/或客户端A实时处理能力优秀的情况。In one embodiment, the live view image used to construct the virtual scene may be continuously acquired by client A from server S. For example, especially in the case where the location sent by the client A is the current geographical location information, the client A can continuously update its current geographical location information and communicate with the server S in order to continuously obtain the real scene corresponding to the geographical location information from the server S. Images, these real-life images are used for real-time updates of the spatial content displayed in the virtual scene. This is especially true when the user of client A has a large physical displacement during the interaction process, the network transmission performance is excellent (for example, high-speed WiFi is covered), and/or the real-time processing capability of client A is excellent.

在另一个实施例中，用于构建虚拟场景的实景图像可以是客户端A 从服务器S一次性获取的。即，在一次完整的虚拟场景交互中涉及到的实景范围可以是预先确定的，并可以在相关街景信息传输时将所涉及的实景图像一次性地传输给客户端A。在此，“一次性”指的是基于客户端A的单次获取请求而将全部所需的内容在应答中持续全部返回，例如，连续逐张的传输所需的例如100帧实景图像。在其他实施例中，实景图像还可以分批获取，例如，在每次经过物理热点(例如，WiFi点)时获取。上述获取方式可以基于具体应用环境而任意组合，以获得最优的实现方案。In another embodiment, the live view image used to construct the virtual scene may be obtained once by client A from server S. That is, the real scene range involved in a complete virtual scene interaction may be predetermined, and the real scene image involved may be transmitted to the client A at one time when the related street view information is transmitted. Here, "one-time" refers to continuously returning all of the required content in the response based on a single acquisition request of the client A, for example, 100 frames of real-time images required for continuous transmission. In other embodiments, the live image may also be obtained in batches, for example, each time a physical hotspot (eg, a WiFi point) is passed. The foregoing acquisition manners may be arbitrarily combined based on a specific application environment to obtain an optimal implementation scheme.

在一个实施例中，在交互过程中，可以根据当前地理位置、速度信息和/或视角信息实时改变展示虚拟场景的视角和内容。在虚拟场景预先在客户端A中加载的情况下，上述改变可以仅在客户端A侧实现。而在其他实施例中，客户端A可以将当前地理位置信息、速度信息和/或视角信息实时发送给服务器S，并经由服务器S返回的更新内容来时改变展示的虚拟场景的视角和内容。In one embodiment, during the interaction, the perspective and content of the presentation virtual scene may be changed in real time according to the current geographic location, speed information, and/or perspective information. In the case where the virtual scene is previously loaded in the client A, the above change can be implemented only on the client A side. In other embodiments, the client A may send the current geographical location information, the speed information, and/or the perspective information to the server S in real time, and change the perspective and content of the displayed virtual scene when the updated content returned by the server S.

在一个实施例中，虚拟场景的显示可以是用户的第一视角跟随显示。即，跟随用户的物理运动，连续改变虚假场景中的显示内容。上述显示内容与用户实际所处环境相一致，或者至少近似。这可以通过客户端A高精度定位能力，以及其内配备的诸如陀螺仪或水平仪的传感器来实时测量用户的运动轨迹来实现。在其他实施例中，还可以至少部分基于环境中的参照物来实现。In one embodiment, the display of the virtual scene may be the user's first perspective following display. That is, following the physical motion of the user, the display content in the fake scene is continuously changed. The above display content is consistent with, or at least approximate to, the actual environment of the user. This can be achieved by measuring the user's motion trajectory in real time through the client A's high-precision positioning capability and the sensors such as gyroscopes or spirits equipped in it. In other embodiments, it may also be implemented based, at least in part, on a reference in the environment.

本发明的互动操作方案可以进一步拓展为在由服务器S提供的共享虚拟场景平台下多个联网用户的互动操作方案。服务器S可以如图2的服务器20所示，连入多个客户端10。服务器S可以为同一场景框架下的用户维护一个共同的平台，当用户的化身在虚拟场景中接近的时候，可以彼此进行互动。不同的用户也可以通过例如参见同一虚拟场景中的某一相同场次的游戏来进行互动，例如，虚拟场景中途经某北京实际街道的某一次赛车游戏。服务器S可以在虚拟空间中同步一个或多个客户端(的用户)的化身信息。客户端A可以获取联网的一个或多个其他客户端的信息，并在其虚拟场景中展示所述一个或多个其他客户端的化身。进一步地，客户端A可以经由化身与其他客户端的化身进行交互，其中所述交互是基于加载了所述场景框架的虚拟街景地图进行的。The interactive operation scheme of the present invention can be further extended to an interactive operation scheme of a plurality of networked users under the shared virtual scene platform provided by the server S. The server S can be connected to a plurality of clients 10 as shown by the server 20 of FIG. The server S can maintain a common platform for users under the same scene framework, and can interact with each other when the user's avatars are close in the virtual scene. Different users can also interact by, for example, seeing a game of the same scene in the same virtual scene, for example, a certain racing game passing through a real street in Beijing in a virtual scene. The server S can synchronize the avatar information of one or more clients (users) in the virtual space. Client A may obtain information for one or more other clients of the network and present the avatars of the one or more other clients in their virtual scenes. Further, client A can interact with other clients' avatars via avatars, which are based on a virtual street view map that loads the scene frame.

在一个实施例中，虚拟场景中可以加载AR对象，以进一步提升本发明的互动方案的参与性和趣味性。可以基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的AR对象，并在虚拟场景中加载该AR对象。加载的AR对象，但无需立即展示，而是可由特定操作、角度、任务完成等触发。可以在满足预定展示条件的情况下，在所述虚拟场景中展示加载的所述AR对象。例如，特定AR对象可以在用户在虚拟场景中到达目标位置或其附近时展现；可以在用户与AR对象处于特定视角时展现，比如，视角正对某店铺后展示该店铺小精灵等；也可以在特定任务达成后展现，比如赛车成绩达标后出现虚拟奖杯，该奖杯例如可以位于某汽车展示店内，地图中可以生成去往该店内的路线指示，并且该奖杯可以作为在该汽车展示店内的优惠消费凭证。客户端A可以与虚拟场景中载入并展示的AR对象进行的交互操作。In one embodiment, AR objects can be loaded in a virtual scene to further enhance the participation and interest of the interactive scheme of the present invention. The AR object to be loaded may be acquired based on the geographical location information, the street view information, and/or the scene frame, and loaded in the virtual scene. The loaded AR object, but does not need to be displayed immediately, but can be triggered by a specific operation, angle, task completion, and the like. The loaded AR object may be displayed in the virtual scene if a predetermined presentation condition is met. For example, a specific AR object may be displayed when the user reaches the target location or in the vicinity of the virtual scene; may be displayed when the user and the AR object are at a specific perspective, for example, the viewing angle is opposite to a certain store, and the store elf is displayed; After the specific task is reached, for example, the virtual trophy appears after the achievement of the racing result. The trophy can be located, for example, in a car display store. The map can generate a route indication to the store, and the trophy can be used as a discount in the car display store. Consumption certificate. Client A can interact with AR objects loaded and displayed in the virtual scene.

在一个实施例中，尤其是在基于客户端A提交的当前地理位置信息生成虚拟场景的情况下(即，用户实际身处虚拟场景所基于的实际场景内的情况下)，用户可以通过使用客户端A的摄像功能拍摄某实际目标来实现AR对象的触发。可选地，上述拍摄和触发的数据还可以进一步用于客户端A本身的定位或是虚拟场景的生成。In one embodiment, particularly in the case of generating a virtual scene based on the current geographic location information submitted by the client A (ie, in the case where the user is actually within the actual scenario on which the virtual scene is based), the user may use the client The camera function of the end A captures an actual target to trigger the AR object. Optionally, the above captured and triggered data may further be used for the positioning of the client A itself or the generation of the virtual scene.

应该理解的是，虚拟场景中也可以加载其他目标对象，例如非AR对象，例如，虚拟RPG场景中融入场景并经由特定操作开启的对象，上述对象同样可以基于地理位置信息、街景信息和/或场景框架获取并在预定条件下触发。It should be understood that other target objects, such as non-AR objects, may also be loaded in the virtual scene, for example, objects in the virtual RPG scene that are integrated into the scene and opened via specific operations, and the objects may also be based on geographic location information, street view information, and/or The scene frame is acquired and triggered under predetermined conditions.

由此，如图3中的虚线箭头所显示的，在例如虚拟场景所需的全部街景信息并非在客户端A构建该虚拟场景是完全获取，而是分批或是实时从服务器S获取的情况下，在用户与该虚拟场景的交互过程中，还涉及客户端A与服务器S的信息交互，例如，实时上传当前地理位置信息/操作信息(非实地的情况下)并获取相应的街景信息。另外，在涉及多人联网的情况下，服务器需要同步多用户化身各自的位置、动作及其间的交互，并将其展示给相应范围内的用户。Thus, as shown by the dashed arrow in FIG. 3, all the street view information required in, for example, the virtual scene is not completely acquired at the client A, but is obtained in batch or in real time from the server S. In the interaction process between the user and the virtual scenario, the information interaction between the client A and the server S is also involved, for example, real-time uploading of current geographical location information/operation information (in the case of non-field) and obtaining corresponding street view information. In addition, in the case of multi-person networking, the server needs to synchronize the respective locations, actions, and interactions of the multi-user avatars and present them to users within the corresponding scope.

如上已经结合图3描述了本发明的互动操作方案的大致流程。如下将结合附图5-8示出客户端A和服务器S各自的互动操作实现方法和装置。The general flow of the interactive operational scheme of the present invention has been described above in connection with FIG. The interactive operation implementation method and apparatus of each of the client A and the server S will be described with reference to FIGS. 5-8 as follows.

图5示出了根据本发明一个实施例的客户端侧的互动操作实现方法的流程图。在步骤S510，将地理位置信息发送给服务器。在步骤S520，接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息。在步骤S530，将基于街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景。在步骤S540，展示所述虚拟场景。在步骤S550，响应于用户输入，在虚拟场景中进行交互操作。FIG. 5 is a flow chart showing a method of implementing an interactive operation on the client side according to an embodiment of the present invention. At step S510, the geographic location information is transmitted to the server. In step S520, related street view information related to the street view information is received, wherein the street view information is street view information associated with the geographical location information queried by the server. In step S530, the virtual street view map based on the street view information is merged with the scene frame to obtain a virtual scene. At step S540, the virtual scene is presented. In step S550, an interactive operation is performed in the virtual scene in response to the user input.

根据具体的应用场景，由服务器S传递给客户端A的相关街景信息可以是街景信息本身，可以是基于街景信息生成的虚拟街景地图，也可以是启用已经下载到客户端A上的相关街景信息或虚拟街景地图的指令。上述内容已在上文结合图3描述，在此不再赘述。According to a specific application scenario, the related street view information transmitted by the server S to the client A may be the street view information itself, may be a virtual street view map generated based on the street view information, or may be related to the related street view information that has been downloaded to the client A. Or instructions for a virtual street view map. The foregoing has been described above in connection with FIG. 3 and will not be described again herein.

图6示出了根据本发明一个实施例的服务器侧的互动操作实现方法的流程图。在步骤S610，获取客户端发送的地理位置信息。在步骤S620，查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息。在步骤S630，将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，其中，所述虚拟场景是将虚拟街景地图与场景框架融合得到的。FIG. 6 is a flow chart showing a method of implementing an interactive operation on the server side according to an embodiment of the present invention. In step S610, geographic location information sent by the client is obtained. In step S620, street view information associated with the geographic location information or virtual street view map information generated based on the street view information is queried. In step S630, the street view information or the virtual street view map information is sent to the client, so that the client displays a virtual scene in which the client user can perform an interaction operation, wherein the virtual scene is a virtual street view map. Blended with the scene frame.

图7示出了根据本发明一个实施例的在客户端侧实现的互动操作实现装置的示意图。如图7所示，互动操作实现装置700包括位置信息发送单元710、街景信息接收单元720、虚拟场景构建单元730、虚拟场景展示单元740和交互单元750。FIG. 7 shows a schematic diagram of an interactive operation implementing apparatus implemented on a client side according to an embodiment of the present invention. As shown in FIG. 7, the interactive operation implementing apparatus 700 includes a location information transmitting unit 710, a street view information receiving unit 720, a virtual scene building unit 730, a virtual scene displaying unit 740, and an interaction unit 750.

位置信息发送单元710可以用于将地理位置信息发送给服务器。街景信息接收单元720可以用于接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息。虚拟场景构建单元730可以用于将基于街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景。在一个实施例中，虚拟场景构建单元730可以包括基于街景信息构建虚拟街景地图的虚拟街景地图构建子单元。虚拟场景展示单元740可以用于展示所述虚拟场景。交互单元750则可响应于用户输入，在所述虚拟场景中进行交互操作。The location information sending unit 710 can be configured to send the geographic location information to the server. The street view information receiving unit 720 can be configured to receive related street view information related to the street view information, wherein the street view information is street view information that is queried by the server and associated with the geographic location information. The virtual scene construction unit 730 can be configured to fuse the virtual street view map based on the street view information with the scene frame to obtain a virtual scene. In one embodiment, virtual scene construction unit 730 can include a virtual street view map construction sub-unit that builds a virtual street view map based on street view information. The virtual scene display unit 740 can be used to display the virtual scene. The interaction unit 750 can then perform an interaction in the virtual scene in response to user input.

在一个实施例中，互动操作实现装置700还可以包括目标对象获取单元和目标对象加载单元。目标对象获取单元可以基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的目标对象。目标对象加载单元可以用于在所述虚拟场景中加载所述目标对象。目标对象加载单元在满足预定展示条件的情况下，在所述虚拟场景中展示加载的所述目标对象。该目标对象可以是AR对象。另外，在涉及多人联网等的情况下，互动操作实现装置700还可以包括信息同步单元，用于在所述虚拟街景地图中同步展示所述一个或多个其他客户端的化身。In an embodiment, the interaction operation implementing apparatus 700 may further include a target object acquisition unit and a target object loading unit. The target object acquisition unit may acquire a target object to be loaded based on the geographic location information, the street view information, and/or the scene frame. A target object loading unit can be used to load the target object in the virtual scene. The target object loading unit displays the loaded target object in the virtual scene if a predetermined presentation condition is satisfied. The target object can be an AR object. In addition, in the case of multi-person networking or the like, the interactive operation implementing apparatus 700 may further include an information synchronization unit for synchronously displaying the avatars of the one or more other clients in the virtual street view map.

图8示出了根据本发明一个实施例的在服务器侧实现的互动操作实现装置的示意图。如图8所示，互动操作实现装置800包括位置信息获取单元810、街景信息查询单元820和街景信息发送单元830。FIG. 8 shows a schematic diagram of an interactive operation implementing apparatus implemented on a server side according to an embodiment of the present invention. As shown in FIG. 8, the interactive operation implementing apparatus 800 includes a location information acquiring unit 810, a street view information inquiring unit 820, and a street view information transmitting unit 830.

互动操作实现装置800可以用于获取客户端发送的地理位置信息。街景信息查询单元820可以用于查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息。街景信息发送单元830可以将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，其中，所述虚拟场景是虚拟街景地图与场景框架融合得到的。The interactive operation implementation device 800 can be configured to obtain geographic location information sent by the client. The street view information query unit 820 may be configured to query street view information associated with the geographic location information or virtual street view map information generated based on the street view information. The street view information sending unit 830 may send the street view information or the virtual street view map information to the client, so that the client displays a virtual scene in which the client user can perform an interaction operation, wherein the virtual scene is a virtual street view. The map is merged with the scene frame.

在一个实施例中，虚拟场景、虚拟街景地图和/或其至少一部分可以在服务器侧构建。于是，互动操作实现装置800可以可选地包括构建单元840，用于基于街景信息构建虚拟街景地图，和/或在虚拟街景地图上加载场景框架以构建虚拟场景空间。在此情况下，街景信息发送单元830发送的可以是包含街景信息的内容，例如，已经构建好的虚拟街景地图，甚至是虚拟场景，或其一部分。In one embodiment, the virtual scene, virtual street view map, and/or at least a portion thereof may be built on the server side. Thus, the interactive operation implementation apparatus 800 can optionally include a building unit 840 for constructing a virtual street view map based on the street view information, and/or loading the scene frame on the virtual street view map to construct the virtual scene space. In this case, the street view information transmitting unit 830 may transmit content including street view information, for example, a virtual street view map that has been constructed, or even a virtual scene, or a part thereof.

在支持AR功能的情况下，街景信息发送单元830可以基于所述地理位置信息、所述街景信息、所述场景框架和/或预定展示条件发送要在所述虚拟场景中加载的AR对象，以供在所述虚拟场景中展示。In the case of supporting the AR function, the street view information transmitting unit 830 may transmit the AR object to be loaded in the virtual scene based on the geographical location information, the street view information, the scene frame, and/or a predetermined presentation condition, to For display in the virtual scene.

另外，位置信息获取单元810可以持续获取客户端的位置信息或操作信息，以便发送客户端所需的内容。在涉及多人联网的情况下，位置信息获取单元810可以获取联网多个客户端的信息。互动操作实现装置800可以包括同步单元，用于在所述虚拟空间平台中同步所述多个客户端的化身的操作与位置。街景信息发送单元830可以将这些同步信息发送给相应的客户端，例如将虚拟场景中涉及的其他客户端的化身发送给某一客户端以在其虚拟场景中展示。可选地，同步单元还可以实时接收所述客户端和/或其他客户端的化身的操作并进行同步处理，并由街景信息发送单元830相应地下发所述同步处理以实现所述客户端与其他客户端的化身之间的交互操作。In addition, the location information obtaining unit 810 can continuously acquire the location information or the operation information of the client to send the content required by the client. In the case of multi-person networking, the location information obtaining unit 810 can acquire information of a plurality of clients connected to the network. The interactive operation implementing apparatus 800 may include a synchronization unit for synchronizing operations and locations of the avatars of the plurality of clients in the virtual space platform. The street view information sending unit 830 can send the synchronization information to the corresponding client, for example, sending the avatars of other clients involved in the virtual scene to a certain client to display in the virtual scene. Optionally, the synchronization unit may also receive the operation of the avatar of the client and/or other clients in real time and perform synchronization processing, and the synchronization processing is performed by the street view information sending unit 830 correspondingly to implement the client and other Interaction between the avatars of the client.

图9是根据本发明一个实施例的客户端设备900。客户端设备900包括输入输出装置910、存储器920和处理器930。输入输出装置910可以是触摸屏，用于接收输入信息并显示输出信息。存储器920用于存储信息。处理器930连接至输入输出装置910和存储器920，并用于如上所述展示虚拟场景并与用户交互。Figure 9 is a client device 900 in accordance with one embodiment of the present invention. The client device 900 includes an input output device 910, a memory 920, and a processor 930. The input output device 910 can be a touch screen for receiving input information and displaying output information. Memory 920 is used to store information. Processor 930 is coupled to input and output device 910 and memory 920 and is used to present virtual scenes and interact with the user as described above.

本发明的一个实施例中，还提供服务器1000。服务器1000包括存储器1010和处理器1020。存储器1010用于存储可执行代码。处理器1020连接至存储器1010，当所述可执行代码被所述处理器1020执行时，使所述处理器1020执行上述在服务器侧实现的互动操作实现方法。In one embodiment of the invention, a server 1000 is also provided. The server 1000 includes a memory 1010 and a processor 1020. The memory 1010 is for storing executable code. The processor 1020 is coupled to the memory 1010, and when the executable code is executed by the processor 1020, causes the processor 1020 to perform the above-described interactive operation implementation method implemented on the server side.

应该理解，如上基于图3描述的可选实施例同样适用于图5-9所示方法中的客户端和服务器及其上实现的相应装置，在此不再赘述。It should be understood that the optional embodiment described above based on FIG. 3 is also applicable to the client and server in the method shown in FIG. 5-9 and corresponding devices implemented thereon, and details are not described herein again.

如上已经结合图3-9描述了本发明的互动操作方案的基本原理。通过算法生成实景街景，比传统的地图应用(例如，地图游戏)更为接近现实，以北京为例，可以很快的生成北京实际街道地图，用户认识度高，认知性强，接收度高。实景街景的生成可选基于实景图像的拼接实现，因此能够以可接受的数据处理和网络传输要求来实现至少部分信息高度还原实际场景的虚拟街景地图。可以根据加载的场景框架，来为相同的虚拟街景地图叠加不同的应用场景。例如，用于赛车游戏的工业建筑场景和操作面板，用于射击类游戏的丛林场景和操作面板等等，从而拓展了本发明的应用领域。另外，由于构建的虚拟场景可以与用户选定的地理位置或是当前所在的地理位置相关联，因此可以直接结合地图数据生成实景地图，亦可以根据关键坐标信息生成虚拟地图，远比线上任何地图应用内容丰富。The basic principles of the interactive operational scheme of the present invention have been described above in connection with Figures 3-9. The real-life streetscape is generated by the algorithm, which is closer to reality than the traditional map application (for example, map game). Taking Beijing as an example, the actual street map of Beijing can be quickly generated, and the user has high recognition, strong cognition and high reception. . The generation of the real scene scene can be selected based on the splicing of the real scene image, so that the virtual street view map of at least part of the information to restore the actual scene can be realized with acceptable data processing and network transmission requirements. Different application scenarios can be superimposed for the same virtual street view map according to the loaded scene frame. For example, industrial architectural scenes and operation panels for racing games, jungle scenes and operation panels for shooting games, and the like, thereby expanding the field of application of the present invention. In addition, since the constructed virtual scene can be associated with the geographical location selected by the user or the current geographical location, the real map can be directly generated by combining the map data, and the virtual map can be generated according to the key coordinate information, far more than any online line. The map app is rich in content.

[应用例1][Application Example 1]

用户可以使用客户端A上安装的应用，启动虚拟街景赛车游戏功能。该应用例如可以包括多个可下载或可加载至本地的场景框架，用户可以基于喜好自行选取。在赛车游戏的场景下，客户端A可以通过weex能力、H5能力或直接植入现有的赛车第一控制室及形式模型框架。The user can launch the virtual Street View racing game function using the application installed on Client A. The application may, for example, include a plurality of scene frames that are downloadable or loadable to the local, and the user may select them based on preferences. In the context of a racing game, client A can use the weex capability, H5 capability or directly into the existing racing first control room and formal model framework.

随后，服务器S可以读取客户端A的当前地理信息(例如，可由客户端A的LBS能力提供)，从地图服务或其数据库中读取地图数据相对应的读取街景数据。由于现有的街景数据是纯图片，因此可以在服务器S或是客户端A上利用专门制定地图算法，整合地图与街景(图片)信息，使得地图与图片信息结合，生成虚拟街景地图。在此，该虚拟街景地图可以不通过UI绘制，而是主要利用图片拼接得到。Subsequently, the server S can read the current geographic information of the client A (for example, can be provided by the client A's LBS capability), and read the street view data corresponding to the map data read from the map service or its database. Since the existing street view data is a pure picture, a map algorithm can be specially used on the server S or the client A to integrate the map and the street view (picture) information, so that the map and the picture information are combined to generate a virtual street view map. Here, the virtual street view map can be drawn not by the UI, but mainly by image stitching.

在街景赛车的应用场景下，可以利用地图数据和如图4A所示的街景数据(图片)，重建以实景道路为骨架的虚拟街景地图，并为虚拟街景地图添加与赛车场景相适应的建筑物和道路风格(如图4B所示)。为了增添实际参与感，虚拟街景地图中可以重建赛车路线经过的地标建筑，例如途经北京实际街道时原样呈现的天安门城楼和广场等。In the application scene of the Street View racing, you can use the map data and the street view data (picture) shown in Figure 4A to reconstruct the virtual street view map with the real road as the skeleton, and add the building suitable for the racing scene to the virtual street view map. And road style (as shown in Figure 4B). In order to add a sense of actual participation, the virtual street view map can reconstruct the landmark buildings where the racing route passes, such as the Tiananmen Gate Tower and Plaza, which are presented as they pass through the actual streets in Beijing.

在赛车游戏框架运行时，可以通过LBS位移及时间计算汽车的移动速度及到达地点等；也可以在载入地图之后，按照用户的界面操作进行交互(即，显示场景可以与用户当前面对的场景不同)。When the racing game framework is running, the moving speed and arrival location of the car can be calculated by LBS displacement and time; or after the map is loaded, the interaction can be performed according to the user's interface operation (ie, the display scene can be currently faced by the user). The scene is different).

可选地，可以建立用户联网体系，各个用户可以使用LBS接入，从而进入多人互动场景。例如，在同一地点附近的用户可以参加在同一虚拟场景范围下的同一场汽车追逐赛。Optionally, a user networking system can be established, and each user can access using the LBS to enter a multi-person interaction scenario. For example, users near the same location can participate in the same car chase in the same virtual scene range.

[应用例2][Application Example 2]

相比于前例中非严格的当前场景虚拟化实现(因为正参与赛车游戏的用户无法在游戏中自身真正达到游戏中赛车的移动速度)，用户可以启动城市寻宝RPG游戏来实现沉浸程度更深的游戏体验。Compared to the non-strict current scene virtualization implementation in the previous example (because the user who is participating in the racing game can't really reach the speed of the racing car in the game itself), the user can launch the city treasure hunt RPG game to achieve a deeper immersion game. Experience.

客户端A可以首先装载该RPG游戏的场景框架，例如包括小地图显示、跳跃、AR拍摄激活等菜单的操作界面，并实时向服务器S发送其当前地理位置。服务器S可以将地图数据和街景数据发送给客户端A，以便在客户端A上加载实景还原级别更高的虚拟街景地图。例如，除了还原道路之外，还可以还原建筑本身，并在与建筑关联的位置上设置通关所需完成的任务。通过客户端A的LBS能力及其内置的各类传感器，客户端A中实时显示的虚拟街景可以与用户真实眼见的实景保持高度一致。Client A can first load the scene frame of the RPG game, such as an operation interface including a small map display, a jump, an AR shooting activation, and the like, and send its current geographic location to the server S in real time. The server S can send the map data and the street view data to the client A, so as to load the virtual street view map with a higher level of the real scene restoration on the client A. For example, in addition to restoring the road, the building itself can be restored and the tasks required for customs clearance can be set at the location associated with the building. Through the client's LBS capability and its built-in sensors, the virtual street view displayed in real-time in client A can be highly consistent with the real scene that the user sees.

在游戏进行中，用户可以通过客户端A的相机功能扫描特定目标来启动AR对象的显示，用户可以与AR对象进行交互，以完成相应的游戏任务。另外，该AR对象的扫描和互动还可以与实景高度结合，例如，涉及扫描店铺或建筑的介绍或是优惠活动等等。During the game, the user can scan the specific target through the camera function of client A to start the display of the AR object, and the user can interact with the AR object to complete the corresponding game task. In addition, the scanning and interaction of the AR object can also be combined with the height of the real scene, for example, involving the introduction of a shop or building or a promotional event.

上文中已经参考附图详细描述了根据本发明的互动操作实现方法和装置。The method and apparatus for implementing an interactive operation in accordance with the present invention have been described in detail above with reference to the accompanying drawings.

此外，根据本发明的方法还可以实现为一种计算机程序或计算机程序产品，该计算机程序或计算机程序产品包括用于执行本发明的上述方法中限定的上述各步骤的计算机程序代码指令。Furthermore, the method according to the invention may also be embodied as a computer program or computer program product comprising computer program code instructions for performing the various steps defined above in the above method of the invention.

或者，本发明还可以实施为一种非暂时性机器可读存储介质(或计算机可读存储介质、或机器可读存储介质)，其上存储有可执行代码(或计算机程序、或计算机指令代码)，当所述可执行代码(或计算机程序、或计算机指令代码)被电子设备(或计算设备、服务器等)的处理器执行时，使所述处理器执行根据本发明的上述方法的各个步骤。Alternatively, the present invention may be embodied as a non-transitory machine readable storage medium (or computer readable storage medium, or machine readable storage medium) having stored thereon executable code (or computer program, or computer instruction code) When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), causing the processor to perform various steps of the above method in accordance with the present invention .

本领域技术人员还将明白的是，结合这里的公开所描述的各种示例性逻辑块、模块、电路和算法步骤可以被实现为电子硬件、计算机软件或两者的组合。The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both.

附图中的流程图和框图显示了根据本发明的多个实施例的***和方法的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分，所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标记的功能也可以以不同于附图中所标记的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的***来实现，或者可以用专用硬件与计算机指令的组合来实现。The flowchart and block diagrams in the Figures illustrate the architecture, functionality and operation of possible implementations of systems and methods in accordance with various embodiments of the present invention. In this regard, each block of the flowchart or block diagram can represent a module, a program segment, or a portion of code that includes one or more of the Executable instructions. It should also be noted that in some alternative implementations, the functions noted in the blocks may also occur in a different order than the ones in the drawings. For example, two consecutive blocks may be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented in a dedicated hardware-based system that performs the specified function or operation. Or it can be implemented by a combination of dedicated hardware and computer instructions.

以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。The embodiments of the present invention have been described above, and the foregoing description is illustrative, not limiting, and not limited to the disclosed embodiments. Numerous modifications and changes will be apparent to those skilled in the art without departing from the scope of the invention. The choice of terms used herein is intended to best explain the principles, practical applications, or improvements of the techniques in the various embodiments of the embodiments, or to enable those of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

一种互动操作实现方法，包括：An implementation of interactive operations, including:

将地理位置信息发送给服务器；Send geographic location information to the server;

接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息；Receiving related street view information related to the street view information, wherein the street view information is street view information associated with the geographical location information queried by the server;

将基于所述街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景；Integrating a virtual street view map based on the street view information with a scene frame to obtain a virtual scene;

展示所述虚拟场景；以及Presenting the virtual scene;

响应于用户输入，在所述虚拟场景中进行交互操作。Interactive operations are performed in the virtual scene in response to user input.
如权利要求1所述的方法，其中，所述将基于所述街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景包括：The method of claim 1, wherein the merging the virtual street view map based on the street view information with the scene frame to obtain the virtual scene comprises:

在向已装载的场景框架添加所述街景信息，以形成加载有场景框架的虚拟街景地图，所述虚拟场景是加载有场景框架的虚拟街景地图；或者，基于所述街景数据构建所述虚拟街景地图，在所述虚拟街景地图上加载所述场景框架以得到所述虚拟场景。Adding the street view information to the loaded scene frame to form a virtual street view map loaded with a scene frame, the virtual scene is a virtual street view map loaded with a scene frame; or constructing the virtual street view based on the street view data a map on which the scene frame is loaded to obtain the virtual scene.
如权利要求1或2所述的方法，其中，所述相关街景信息还包括与所述街景信息相关联的地图信息，并且，The method according to claim 1 or 2, wherein the related street view information further includes map information associated with the street view information, and

所述虚拟街景地图的构造还基于所述地图信息。The construction of the virtual street view map is also based on the map information.
如权利要求1-3任意一项所述的方法，其中，所述场景框架包括使用户能够与所显示的虚拟场景或其内目标进行交互的操作面板。The method of any of claims 1-3, wherein the scene framework includes an operation panel that enables a user to interact with the displayed virtual scene or objects within it.
如权利要求1-4任意一项所述的方法，其中，至少部分基于场景框架确定虚拟街景地图的环境或显示风格。A method according to any of claims 1-4, wherein the environment or display style of the virtual street view map is determined based at least in part on the scene frame.
如权利要求1-5任意一项所述的方法，其中，所述虚拟场景中的视线高度至少部分基于所述场景框架确定。The method of any of claims 1-5, wherein the line of sight height in the virtual scene is determined based at least in part on the scene frame.
如权利要求1-6任意一项所述的方法，还包括：The method of any of claims 1-6, further comprising:

基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的AR对象；以及Acquiring an AR object to be loaded based on the geographic location information, the street view information, and/or the scene frame;

在所述虚拟场景中加载所述AR对象。Loading the AR object in the virtual scene.
如权利要求1-7任意一项所述的方法，还包括：The method of any of claims 1-7, further comprising:

在满足预定展示条件的情况下，在所述虚拟场景中展示加载的所述AR对象。The loaded AR object is displayed in the virtual scene if the predetermined presentation condition is met.
如权利要求1-8任意一项所述的方法，还包括：The method of any of claims 1-8, further comprising:

响应于对目标对象的拍摄，加载AR对象。The AR object is loaded in response to the shooting of the target object.
如权利要求1-9任意一项所述的方法，其中，加载所述AR对象的相关数据被用于确定当前地理位置信息和/或所述虚拟场景的生成与更新。The method of any of claims 1-9, wherein the associated data loading the AR object is used to determine current geographic location information and/or generation and update of the virtual scene.
如权利要求1-10任意一项所述的方法，其中，A method according to any one of claims 1 to 10, wherein

客户端提交的地理位置信息可以是用户选定的地理位置信息或是用户的当前地理位置信息。The geographic location information submitted by the client may be the geographical location information selected by the user or the current geographic location information of the user.
如权利要求1-11任意一项所述的方法，其中，A method according to any of claims 1-11, wherein

以用户的第一视角展示所述虚拟场景。The virtual scene is presented in a first perspective of the user.
如权利要求1-12任意一项所述的方法，其中，A method according to any one of claims 1 to 12, wherein

获取联网的一个或多个其他客户端的信息，并在该用户的虚拟街景地图中展示所述一个或多个其他客户端的化身。Acquiring information for one or more other clients of the network and presenting the avatars of the one or more other clients in the virtual street view map of the user.
一种互动操作实现方法，包括：An implementation of interactive operations, including:

获取客户端发送的地理位置信息；Obtain the geographical location information sent by the client;

查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；Querying street view information associated with the geographic location information or virtual street view map information generated based on the street view information;

将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，Sending the street view information or the virtual street view map information to the client, so that the client displays a virtual scene in which the client user can perform the interaction operation,

其中，所述虚拟场景是将虚拟街景地图与场景框架相融合得到的。The virtual scene is obtained by fusing a virtual street view map with a scene frame.
如权利要求14所述的方法，其中，还包括：The method of claim 14 further comprising:

基于所述地理位置信息、所述街景信息、所述场景框架和/或预定展示条件发送要在所述虚拟场景中加载的AR对象，以供在所述虚拟场景中展示。An AR object to be loaded in the virtual scene is transmitted for presentation in the virtual scene based on the geographic location information, the street view information, the scene frame, and/or predetermined presentation conditions.
如权利要求14或15所述的方法，其中，还包括：The method of claim 14 or 15, further comprising:

获取联网的一个或多个其他客户端的信息；Get information about one or more other clients connected to the network;

在所述虚拟空间平台中同步所述一个或多个其他客户端的化身；Synchronizing an avatar of the one or more other clients in the virtual space platform;

以及将所述其他客户端的化身发送给所述客户端以在所述客户端的虚拟场景中展示。And sending the avatars of the other clients to the client for display in a virtual scenario of the client.
如权利要求14-16任意一项所述的方法，其中，还包括：The method of any of claims 14-16, further comprising:

实时接收所述客户端和/或其他客户端的化身的操作并进行同步处理；Receiving the operation of the avatar of the client and/or other client in real time and performing synchronization processing;

以及下发所述同步处理以实现所述客户端与其他客户端的化身之间的交互操作。And issuing the synchronization process to implement an interaction between the client and other clients' avatars.
一种互动操作实现装置，包括：An interactive operation implementing device includes:

位置信息发送单元，用于将地理位置信息发送给服务器；a location information sending unit, configured to send the geographic location information to the server;

街景信息接收单元，用于接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息；a street view information receiving unit, configured to receive related street view information related to the street view information, wherein the street view information is street view information associated with the geographic location information queried by the server;

虚拟场景构建单元，用于将基于街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景；a virtual scene building unit, configured to merge the virtual street view map based on the street view information with the scene frame to obtain a virtual scene;

虚拟场景展示单元，用于展示所述虚拟场景；以及交互单元，用于响应于用户输入，在所述虚拟场景中进行交互操作。a virtual scene display unit, configured to display the virtual scene; and an interaction unit, configured to perform an interaction operation in the virtual scene in response to user input.
如权利要求18所述的互动操作实现装置，还包括：The interactive operation implementation device of claim 18, further comprising:

目标对象获取单元，基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的目标对象；a target object acquisition unit that acquires a target object to be loaded based on the geographic location information, the street view information, and/or the scene frame;

以及目标对象加载单元，用于在所述虚拟场景中加载所述目标对象。And a target object loading unit, configured to load the target object in the virtual scene.
如权利要求18或19所述的互动操作实现装置，其中，The interactive operation implementing device according to claim 18 or 19, wherein

所述目标对象加载单元在满足预定展示条件的情况下，在所述虚拟场景中展示加载的所述目标对象。The target object loading unit displays the loaded target object in the virtual scene if a predetermined display condition is satisfied.
如权利要求18-20任意一项所述的互动操作实现装置，还包括：The apparatus for implementing an interactive operation according to any one of claims 18 to 20, further comprising:

信息同步单元，用于在所述虚拟街景地图中同步展示所述一个或多个其他客户端的化身。An information synchronization unit, configured to synchronously display an avatar of the one or more other clients in the virtual street view map.
一种互动操作实现装置，包括：An interactive operation implementing device includes:

位置信息获取单元，用于查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；a location information obtaining unit, configured to query street view information associated with the geographic location information or virtual street view map information generated based on the street view information;

街景信息查询单元，用于查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；a street view information query unit, configured to query street view information associated with the geographic location information or virtual street view map information generated based on the street view information;

街景信息发送单元，用于将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，其中，所述虚拟场景是虚拟街景地图与场景框架融合得到的。a street view information sending unit, configured to send the street view information or the virtual street view map information to a client, so that the client displays a virtual scene in which the client user can perform an interaction operation, wherein the virtual scene is virtual The street view map is merged with the scene frame.
如权利要求22所述的互动操作实现装置，其中，还包括：The interactive operation implementation device of claim 22, further comprising:

构建单元，用于基于街景信息构建虚拟街景地图，和/或在虚拟街景地图上加载场景框架以构建虚拟场景空间。A building unit for constructing a virtual street view map based on street view information, and/or loading a scene frame on the virtual street view map to construct a virtual scene space.
如权利要求22或23所述的互动操作实现装置，其中，The interactive operation implementing device according to claim 22 or 23, wherein

街景信息发送单元用于基于所述地理位置信息、所述街景信息、所述场景框架和/或预定展示条件发送要在所述虚拟场景中加载的AR对象，以供在所述虚拟场景中展示。The street view information transmitting unit is configured to send an AR object to be loaded in the virtual scene based on the geographical location information, the street view information, the scene frame, and/or a predetermined display condition, for being displayed in the virtual scene .
如权利要求22-24任意一项所述的互动操作实现装置，其中，还包括：The interactive operation implementing apparatus according to any one of claims 22-24, further comprising:

同步单元，用于在所述虚拟空间平台中同步所述多个客户端的化身的操作与位置。And a synchronization unit, configured to synchronize operations and locations of the avatars of the plurality of clients in the virtual space platform.
一种客户端设备，包括：A client device, including:

处理器；以及存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如权利要求1-13任意一项所述的互动操作实现方法。a processor; and a memory having stored thereon executable code for causing the processor to perform the interactive operation implementation method of any one of claims 1-13 when the executable code is executed by the processor .
一种服务器，包括：A server that includes:

处理器；以及存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如权利要求14-17任意一项所述的互动操作实现方法。a processor; and a memory having stored thereon executable code for causing the processor to perform an interactive operation implementation method according to any one of claims 14-17 when the executable code is executed by the processor .