CN113975792A

CN113975792A - Cloud game interaction method, device, equipment and storage medium

Info

Publication number: CN113975792A
Application number: CN202111255923.8A
Authority: CN
Inventors: 杨学强
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Beijing ByteDance Network Technology Co Ltd
Priority date: 2021-10-27
Filing date: 2021-10-27
Publication date: 2022-01-28

Abstract

The disclosure relates to a cloud game interaction method, a cloud game interaction device, a cloud game interaction equipment and a storage medium. The cloud game interaction method applied to the user side equipment of the cloud game comprises the following steps: receiving a data acquisition instruction of a target expansion function of the cloud game, which is sent by a relay server and acquired from cloud equipment running the cloud game; the target expansion function is realized based on the local service function in the user terminal equipment; calling a local service function based on the data acquisition instruction, and determining target local data; forwarding the target local data to the cloud end equipment through the relay server so that the cloud end equipment generates a game rendering result corresponding to the target expansion function based on the target local data; and receiving and displaying a game rendering result which is sent by the relay server and acquired from the cloud equipment. The target local data required by the target expansion function of the cloud game can be sent to the cloud end equipment from the user end equipment through the relay server, and therefore the function expansion convenience of the cloud game is improved.

Description

Cloud game interaction method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of communications technologies, and in particular, to a cloud game interaction method, apparatus, device, and storage medium.

Background

The cloud game is executed by moving a game instance originally running on the user terminal device to the cloud terminal device. The user terminal equipment does not need to download the game, but the cloud terminal equipment runs the game process, so that the game can be played on demand. The cloud device and the user side device are two different devices.

However, the cloud game implementation process is only suitable for implementing the original game functions originally developed in the game application program, and cannot rapidly implement more other functions.

Disclosure of Invention

To solve the technical problem or at least partially solve the technical problem, the present disclosure provides a cloud game interaction method, apparatus, device and storage medium.

In a first aspect, the present disclosure provides a cloud game interaction method, applied to a client device of a cloud game, where the method includes:

receiving a data acquisition instruction of a target expansion function of the cloud game sent by a relay server; the data acquisition instruction is sent to the relay server by cloud equipment running the cloud game, and the target expansion function is realized based on a local service function in the user end equipment;

calling a local service function corresponding to the target expansion function based on the data acquisition instruction, and determining the target local data;

forwarding the target local data to the cloud end equipment through the relay server, so that the cloud end equipment generates a game rendering result corresponding to the target extended function based on the target local data;

receiving and displaying the game rendering result sent by the relay server; and the game rendering result is sent to the relay server by the cloud equipment.

In a second aspect, the present disclosure provides another cloud game interaction method applied to a relay server, including:

receiving a data acquisition instruction of a target expansion function of a cloud game sent by cloud equipment running the cloud game, and forwarding the data acquisition instruction to user end equipment; the target expansion function is realized based on a local service function in the customer premise equipment, and the data acquisition instruction is used for instructing the customer premise equipment to call the local service function corresponding to the target expansion function to acquire target local data;

receiving the target local data sent by the user end equipment, and forwarding the target local data to the cloud end equipment;

receiving a game rendering result corresponding to the target expansion function generated by the cloud device based on the target local data, and forwarding the game rendering result to the user end device, so that the user end device displays the game rendering result.

In a third aspect, the present disclosure provides another cloud game interaction method applied to a cloud device running a cloud game, where the method includes:

sending a data acquisition instruction of a target expansion function of the cloud game to a relay server; the target expansion function is realized based on a local service function in user end equipment, and the data acquisition instruction is used for indicating the user end equipment to call the local service function corresponding to the target expansion function to acquire target local data;

receiving the target local data obtained by the relay server from the customer premise equipment;

and generating a game rendering result corresponding to the target expansion function based on the target local data, and forwarding the game rendering result to user end equipment through the relay server so that the user end equipment displays the game rendering result.

In a fourth aspect, the present disclosure provides a cloud game interaction apparatus configured at a user end device of a cloud game, the apparatus including:

the data acquisition instruction receiving module is used for receiving a data acquisition instruction of a target expansion function of the cloud game, which is sent by the relay server; the data acquisition instruction is sent to the relay server by cloud equipment running the cloud game, and the target expansion function is realized based on a local service function in the user end equipment;

the target local data determining module is used for calling a local service function corresponding to the target expansion function based on the data acquisition instruction and determining the target local data;

a target local data sending module, configured to forward the target local data to the cloud device via the relay server, so that the cloud device generates a game rendering result corresponding to the target extended function based on the target local data;

the game rendering result display module is used for receiving and displaying the game rendering result sent by the relay server; and the game rendering result is sent to the relay server by the cloud equipment.

In a fifth aspect, the present disclosure provides another cloud game interaction apparatus configured on a relay server, including:

the data acquisition instruction forwarding module is used for receiving a data acquisition instruction of a target expansion function of the cloud game, which is sent by cloud equipment running the cloud game, and forwarding the data acquisition instruction to user end equipment; the target expansion function is realized based on a local service function in the customer premise equipment, and the data acquisition instruction is used for instructing the customer premise equipment to call the local service function corresponding to the target expansion function to acquire target local data;

the target local data forwarding module is used for receiving the target local data sent by the user end equipment and forwarding the target local data to the cloud end equipment;

and the game rendering result forwarding module is used for receiving a game rendering result corresponding to the target expansion function generated by the cloud device based on the target local data, and forwarding the game rendering result to the user end device so that the user end device displays the game rendering result.

In a sixth aspect, the present disclosure provides another cloud game interaction apparatus configured to a cloud device running a cloud game, the apparatus including:

the data acquisition instruction sending module is used for sending a data acquisition instruction of a target expansion function of the cloud game to a relay server; the target expansion function is realized based on a local service function in user end equipment, and the data acquisition instruction is used for indicating the user end equipment to call the local service function corresponding to the target expansion function to acquire target local data;

a target local data receiving module, configured to receive the target local data obtained by the relay server from the user end device;

and the game rendering result sending module is used for generating a game rendering result corresponding to the target expansion function based on the target local data and sending the game rendering result to the user end equipment through the relay server so that the user end equipment displays the game rendering result.

In a seventh aspect, the present disclosure provides a cloud game interaction device, including:

a processor;

a memory for storing executable instructions;

the processor is configured to read an executable instruction from the memory, and execute the executable instruction to implement the cloud game interaction method applied to the client device of the cloud game in the first aspect, or implement the cloud game interaction method applied to the relay server in the second aspect, or implement the cloud game interaction method applied to the cloud device running the cloud game in the third aspect.

In an eighth aspect, the present disclosure provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the processor is enabled to implement the cloud game interaction method applied to the client device of the cloud game in the first aspect, or the cloud game interaction method applied to the relay server in the second aspect, or the cloud game interaction method applied to the cloud device for running the cloud game in the third aspect.

According to the cloud game interaction scheme, in the cloud game interaction process, the relay server forwards the data acquisition instruction to the client device from the cloud device, the client device calls the local service function corresponding to the target expansion function to obtain the target local data, the relay server forwards the target local data in the client device to the cloud device, the target local data in the target expansion function is generated in the cloud device by using the target local data, and the game rendering result is forwarded to the client device through the relay server to be played, so that the cloud device can realize the target expansion function realized based on the local service function in the client device, and the convenience in expansion of other functions except the game native function in the cloud game is improved.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.

Fig. 1 is a system architecture diagram of cloud game interaction provided by an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a cloud game interaction method applied to a client device of a cloud game according to an embodiment of the present disclosure;

fig. 3 is a schematic flowchart of a cloud game interaction method applied to a relay server according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of a cloud game interaction method applied to a cloud device running a cloud game according to an embodiment of the present disclosure;

fig. 5 is a schematic diagram of a cloud game interaction process of a cloud game user avatar setting according to an implementation of the present disclosure;

fig. 6 is a schematic diagram of a cloud game interaction process of cloud game real-time voice chat provided in an implementation of the present disclosure;

fig. 7 is a schematic structural diagram of a cloud game interaction apparatus configured on a client device of a cloud game according to an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of a cloud game interaction apparatus configured in a relay server according to an embodiment of the present disclosure;

fig. 9 is a schematic structural diagram of a cloud game interaction apparatus configured on a cloud device running a cloud game according to an embodiment of the present disclosure;

fig. 10 is a schematic structural diagram of a cloud game interaction device according to an embodiment of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

The current cloud game technology can transmit user interaction instructions and game running results through a network, so that a game function is well realized. However, in the game function implementation method based on game program development, when the function needs to be extended, the program code needs to be modified and a new version of the game application needs to be released, so that the function extension is time-consuming and labor-consuming.

Based on the above situation, embodiments of the present disclosure provide a cloud game interaction method, apparatus, device, and storage medium, so as to add a relay server between a cloud device and a client device to forward an interaction instruction and data corresponding to an expansion function between two ends, and the relay server is suitable for expansion of a cloud game function in which the cloud device obtains related data from the client device. For example, pictures stored in an album of the user side device are acquired from the user side device to set a user head portrait; for another example, a microphone of the user end device is used for voice recording to realize voice interaction and the like.

Fig. 1 shows a system architecture diagram of cloud game interaction provided by an embodiment of the present disclosure. As shown in fig. 1, the cloud game interaction system includes a cloud device 110, a relay server 120, and a client device 130 that are communicatively connected with each other. Both the cloud device 110 and the client device 130 need to be registered in the relay server 120, and the registration information includes information with unique identification, so that the relay server 120 establishes a corresponding relationship between the cloud device 110 and the client device 130. Functional interfaces that depend on implementing the expansion function need to be added in the cloud device 110 and the client device 130, respectively, and the function implementation program corresponding to the functional interfaces is a function implementation program of the local service function in the client device 130. For example, the functional interface corresponding to the at least one extended function is respectively encapsulated into a cloud Software Development Kit (SDK) and a client SDK according to the calling processes of the cloud device 110 and the client device 130, and the cloud SDK is accessed to the cloud device 110 and the client SDK is accessed to the client device 130.

When a certain extended function (i.e., a target extended function) is executed, the cloud device 110 calls a corresponding functional interface in the cloud SDK corresponding to the target extended function, and sends a data acquisition instruction to the relay server 120. The relay server 120 then sends the data obtaining instruction to the client device 130, and the data obtaining instruction is received and analyzed by the corresponding functional interface in the client SDK. Then, the user end device 130 calls a function implementation program of the corresponding local service function based on the parsed data obtaining instruction, and obtains data (i.e., target local data) required by the extended function through execution of the function implementation program. Finally, the client device 130 forwards the obtained target local data to the cloud device 110 through the relay server 120 through the corresponding functional interface in the client SDK. The cloud device 110 receives and consumes the target local data through the corresponding function interface to implement the target expansion function. For simplicity of description, the expressions of the cloud end SDK and the client end SDK will be omitted in the following embodiments, and are replaced by the cloud end device and the client end device.

The cloud game interaction method provided by the embodiment of the disclosure is first described with reference to fig. 2 to 6.

In the embodiment of the disclosure, the cloud game interaction method applied to the client device of the cloud game may be executed by the client device. The user end device may include, but is not limited to, mobile terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablets), wearable devices, etc. and fixed terminals such as digital TVs, desktop computers, smart home devices, etc.

Fig. 2 shows a flowchart of a cloud game interaction method applied to a client device of a cloud game according to an embodiment of the present disclosure. As shown in fig. 2, the method may include the steps of:

and S210, receiving a data acquisition instruction of the target expansion function of the cloud game sent by the relay server.

The data acquisition instruction is used for instructing the user end equipment to acquire target local data required by the target expansion function of the cloud game. That is to say, the data obtaining instruction can trigger the user end device to perform a certain operation to obtain the target local data corresponding to the target expansion function. The target local data is data stored in the user end device, and may be data on which the target expansion function is implemented or data generated after the target expansion function is implemented. Illustratively, the target local data includes at least one of picture, audio, video, file, and sensor data. Sensor data refers to data obtained using sensors in a customer premise equipment device.

The target expansion function is realized based on a local service function in the user terminal device, is different from a native function realized during the development of the cloud game application program, and is a function which needs to be expanded and realized after the development of the cloud game application program is completed. Optionally, the target extension function is a business function implemented depending on local data or hardware components in the customer premise equipment. The target extension function in the embodiment of the present disclosure may be a function that depends on local data in the user end device, for example, a function that depends on a local picture stored in the user end device to perform avatar setting; or a function implemented by relying on a hardware component in the client device, for example, a function of relying on a microphone in the client device for voice interaction.

Specifically, the client device receives a data acquisition instruction sent by the relay server, where the data acquisition instruction is received by the relay server from a cloud device running the cloud game in advance.

In some embodiments, the data acquisition instruction is automatically generated by the cloud device based on the target expansion function. For example, if the execution of the target expansion function is automatically triggered during the cloud game running process of the cloud device, the data acquisition instruction is automatically generated.

In other embodiments, the data acquisition instruction is generated by the cloud device based on the user interaction with the client device. Before S210, the client device generates a data obtaining request in response to a user' S trigger operation on the target expansion function, and sends the data obtaining request to the cloud device, so that the cloud device generates a data obtaining instruction based on the data obtaining request.

Specifically, firstly, the user end device displays an interactive operation control corresponding to the target expansion function to a user through a display screen of the user end device. For example, the client device presents the relevant buttons for setting up the avatar. The user then triggers the interoperation control. For example, the user clicks the interactive operation control, or the user controls to trigger the interactive operation control through voice, gesture, eye movement, and the like. At this time, the user end device detects the trigger operation of the user and generates a data acquisition request. For example, a request to take a picture is generated. Then, the client device sends the data acquisition request to the cloud device through a communication channel of the cloud game, namely a direct connection channel between the client device and the cloud device. The cloud device generates a data acquisition instruction based on the data acquisition request.

S220, calling a local service function corresponding to the target expansion function based on the data acquisition instruction, and determining target local data.

The local service function refers to a service function that the user end device can execute.

Specifically, the client device parses the received data acquisition instruction, and at least obtains descriptive information (such as a data type) including data to be acquired and an instruction parsing result of a local service function to be executed by the data. For example, for a target expansion function of selecting pictures in an album to set a user avatar, the instruction parsing result at least includes a picture type such as ". jpg" and a local service function of opening album selection data. And then, the user terminal equipment calls the corresponding local service function according to the instruction analysis result. Target local data is obtained through execution of the local service function.

And S230, forwarding the target local data to the cloud end equipment through the relay server, so that the cloud end equipment generates a game rendering result corresponding to the target expansion function based on the target local data.

Specifically, the client device sends the target local data to the relay server. And the relay server sends the target local data to the cloud end equipment. The cloud equipment realizes the target expansion function of the cloud game according to the target local data, renders the function realization process and the result thereof and generates a corresponding game rendering result.

And S240, receiving and displaying the game rendering result sent by the relay server.

Specifically, the cloud device sends the generated game rendering result to the relay server, and the relay server forwards the game rendering result to the user end device. Then, the user end device can receive a game rendering result obtained by the cloud game realizing the target expansion function, and the game rendering result is displayed to the user through the display screen of the user end device.

According to the cloud game interaction scheme applied to the user side equipment of the cloud game, in the cloud game interaction process, the relay server forwards the data acquisition instruction corresponding to the target expansion function of the cloud game to the user side equipment from the cloud side equipment, the user side equipment calls the local service function corresponding to the target expansion function to acquire the target local data, then the relay server forwards the target local data in the user side equipment to the cloud side equipment, the cloud side equipment achieves the target expansion function to generate the game rendering result, and then the relay server forwards the game rendering result in the cloud side equipment to the user side equipment to be displayed, so that convenience in expanding and achieving other service functions of the cloud game is improved.

In one embodiment of the present disclosure, the target extended function is an extended function that depends on data stored in the customer premise equipment. For example, the picture in the album of the user end device is selected to set the extended function of the user avatar. For another example, at least one of a picture, audio, video, or file in the storage space of the client device is selected to issue an extended function such as a status or comment.

In some embodiments, when the data fetch instruction is a data fetch instruction, S220 includes: calling a data selection component based on the data selection instruction, and displaying a data selection interface corresponding to the data selection component; and determining target local data in response to the selection operation of the data in the data selection interface.

The data selecting instruction is a data acquiring instruction used for indicating data selection from the customer premise equipment. The data selection component is a functional component in the customer premise equipment and is used for realizing interaction of selected data.

Specifically, the user end device calls the data selection component according to the instruction analysis result of the data selection instruction. The execution of the data selection component triggers the presentation of a data selection interface in the screen of the customer premises equipment that at least contains the data to be selected and the data selection control. The user may make a selection of data in the data selection interface. And the user end equipment determines the data corresponding to the selection operation of the user as the target local data. The data to be selected displayed in the data selection interface in the above process may be at least one of pictures, audio, video and files. Accordingly, the target local data may also be at least one of a picture, audio, video, and file. For example, pictures, audio and video are shown in the data selection interface, and if the user selects the pictures and video, the target local data is the pictures and video.

According to the technical scheme of the embodiment related to the data selection instruction, the data selection interface can be presented to the user in the process of determining the target local data, so that the target local data can be determined through the selection operation of the user on the data selection interface, and on the basis of achieving the target expansion function of data selection of the cloud game, the user interaction operation is added, so that the data selection process of the cloud game and the data selection process of the client side directly running on the user side equipment are consistent, and the use experience of the user on the target expansion function of the cloud game is improved.

In yet another embodiment of the present disclosure, the target extended function is an extended function implemented depending on a hardware component in the customer premises equipment.

In some embodiments, when the data acquisition instruction is a short-time recording instruction, the process of calling the local service function to determine the target local data includes: and calling a microphone assembly to record based on a recording starting instruction in the short-time recording instruction, and stopping recording based on a time length upper limit threshold value in the short-time recording instruction or a triggering operation of stopping recording by a user to obtain short-time audio serving as target local data.

The short-time recording instruction is an instruction for recording voice for a short time, and indicates to obtain a piece of voice data without real-time interaction. For example, the short-time recording instruction may be a recording instruction corresponding to an extended function of sending a voice message. Optionally, the short-time recording instruction at least includes a recording start instruction and an upper threshold of the recording duration (i.e., an immediate long upper threshold).

Specifically, the user end device calls a microphone assembly therein according to an instruction parsing result of the recording starting instruction, so as to record through the microphone assembly. At this point, the user may be presented with information that is being recorded with the microphone, such as a text prompt of "recording," or a recording interface that includes a microphone icon/recording icon. And when the recording duration reaches the upper limit threshold of the duration or when a user executes some operation triggers to stop calling the microphone assembly, stopping recording to obtain PCM audio data, namely short-time audio. The short-time audio may then be forwarded to the cloud device via the relay server.

In still other embodiments, when the data obtaining instruction is a real-time recording instruction, the step of calling the local service function to determine the target local data includes: and calling a microphone assembly to record based on a recording starting instruction in the real-time recording instruction to obtain real-time audio serving as target local data.

The real-time recording instruction is used for indicating to acquire real-time audio, and audio data acquired in real time need to be sent to the cloud equipment in real time to achieve real-time voice interaction, so that the real-time recording instruction indicates to acquire the audio in real time for a long time. For example, the real-time recording instruction may be a recording instruction corresponding to an extended function of a voice chat in the cloud game process. In one example, the real-time recording instruction at least comprises a recording starting instruction. In another example, the real-time recording command further includes a stop recording command because the real-time voice interaction also requires the user to actively determine whether to stop.

Specifically, the user end device calls the microphone assembly to start recording according to an instruction analysis result of the recording start instruction in the real-time recording instruction, so as to obtain PCM audio data, namely real-time audio. And continuously calling the microphone assembly to record before the user end equipment does not receive the recording stopping instruction. In addition, a real-time voice interaction interface can be displayed to the user in the recording process.

Accordingly, the process of sending the target local data to the cloud device includes: and sending the real-time audio to the relay server through a real-time data channel between the relay server and the cloud equipment, so that the relay server sends the real-time audio to the cloud equipment through the real-time data channel between the relay server and the cloud equipment. That is, the real-time voice interaction function requires a real-time data channel to be established among the cloud device, the relay server and the user end device, so as to transmit the obtained real-time audio in real time.

Further, after the real-time audio of the user end device is sent to the cloud end device in real time, the user end device receives a recording stopping instruction in the real-time recording instruction sent by the relay server, and stops calling the microphone assembly based on the recording stopping instruction, and ends real-time recording. That is to say, after receiving the recording stopping instruction in the real-time recording instruction, the user equipment may stop calling the microphone component through analysis to end the real-time recording process.

According to the technical scheme of the embodiment related to the audio data, the microphone assembly in the user terminal equipment can be called to perform short-time recording or real-time recording in the process of determining the target local data related to the audio, so that the voice interaction process of the cloud game and the voice interaction process of the client side directly running on the user terminal equipment are consistent on the basis of achieving the target extension function of the audio recording of the cloud game, and the use experience of a user on the target extension function of the cloud game is improved.

In still other embodiments, when the data fetch instruction is a device state fetch instruction, invoking the local service function to determine the target local data comprises: and calling the sensor component based on the equipment state acquisition instruction to obtain sensor data serving as target local data.

The device state obtaining instruction is used for indicating an instruction for obtaining the device state of the user end device. The device state may be at least one of the pose of the device, the temperature, the light and sound of the environment in which the device is located, and the like, which is obtained depending on various sensor components installed in the device. The sensor assembly herein may include at least one of an acceleration sensor, a gyro sensor, a photo sensor, a position sensor, a magnetic sensor, a camera, and a temperature sensor.

Specifically, after the user end device analyzes the device state acquisition instruction, the corresponding sensor component is called according to the instruction analysis result. For example, the device status obtaining instruction is to obtain the pose of the user end device, and then the user end device calls at least one of a position sensor (such as a global positioning system receiver), an acceleration sensor, a gyroscope sensor, a magnetic sensor, and the like. The data obtained by the sensor assembly (i.e., the sensor data) may be used as the target local data.

According to the technical scheme of the embodiment for acquiring the equipment state, the equipment state information of the user side equipment can be acquired, and basic data is provided for the cloud end equipment to expand the somatosensory interaction function of the cloud game, so that possibility is provided for enriching the cloud game function.

Fig. 3 shows a flowchart of a cloud game interaction method applied to a relay server according to an embodiment of the present disclosure. The cloud game interaction method applied to the relay server may be performed by a server having a communication function of data transceiving, for example, an IM server. The explanation of the same or corresponding terms as those in the above embodiments in the cloud game interaction method applied to the relay server is not repeated here. As shown in fig. 3, the method may include the steps of:

s310, receiving a data acquisition instruction of a target expansion function of the cloud game sent by cloud equipment running the cloud game, and forwarding the data acquisition instruction to user end equipment.

The target expansion function is realized based on the local service function in the customer premise equipment. And the data acquisition instruction is used for instructing the user end equipment to call the local service function corresponding to the target expansion function to acquire the target local data.

Specifically, the relay server maintains a correspondence relationship between unique registration identifiers (which may be referred to as a cloud registration identifier and a user-side registration identifier, respectively) when the cloud device and the user-side device are registered. The relay server can receive the cloud registration identification sent by the cloud device while receiving the data acquisition instruction sent by the cloud device. And then, according to the cloud registration identification and the corresponding relation, determining the user end equipment corresponding to the cloud end equipment, and sending a data acquisition instruction to the user end equipment.

And S320, receiving the target local data sent by the user terminal equipment, and forwarding the target local data to the cloud terminal equipment.

Specifically, the user end device calls a target local service function corresponding to the target expansion function according to the data acquisition instruction to acquire target local data, and sends the target local data to the relay server. And after receiving the target local data, the relay server sends the target local data to the cloud end equipment corresponding to the user end equipment.

S330, receiving a game rendering result corresponding to the target expansion function generated by the cloud device based on the target local data, and forwarding the game rendering result to the user end device so that the user end device displays the game rendering result.

Specifically, after receiving the target local data, the cloud device may implement the target expansion function, generate a game rendering result corresponding to the target expansion function, and send the game rendering result to the relay server. And the relay server sends the game rendering result to the corresponding user terminal equipment. The client device can display the game rendering result.

The cloud game interaction scheme applied to the relay server can achieve interaction between cloud end equipment and user end equipment of a cloud game through the relay server in the cloud game interaction process, so that a target expansion function of the cloud game can also perform user interaction operation and response as a client running in the user end equipment, and convenience in cloud game expansion and realization of other business functions is improved.

In the embodiment of the present disclosure, the cloud game interaction method applied to the cloud device may be executed by the cloud device running the cloud game. The cloud device may include, but is not limited to, a fixed terminal such as a desktop computer, a server cluster, or a super computer.

Fig. 4 shows a flowchart of a cloud game interaction method applied to a cloud device according to an embodiment of the present disclosure. The explanation of the same or corresponding terms as those in the above embodiments in the cloud game interaction method applied to the relay server is not repeated here. As shown in fig. 4, the method may include the steps of:

and S410, sending a data acquisition instruction of the target expansion function of the cloud game to a relay server.

Specifically, the cloud device generates a data acquisition instruction corresponding to the target expansion function, and sends the data acquisition instruction to the relay server. And the relay server forwards the data acquisition instruction to the customer premise equipment.

In some embodiments, the cloud device automatically triggers the target expansion function in the running process of the cloud game, and at this time, the cloud device may automatically generate a data acquisition instruction corresponding to the target expansion function.

In other embodiments, the user performs the triggering operation of the target expansion function in the user end device. The user end equipment generates a data acquisition request based on the triggering operation, and sends the data acquisition request to the cloud end equipment. After receiving a data acquisition request sent by the user end device, the cloud end device generates a data acquisition instruction based on the data acquisition request. Such an implementation can further enhance the interchangeability of cloud games, thereby further improving the user experience.

And S420, receiving the target local data obtained by the relay server from the user terminal equipment.

Specifically, the user end device analyzes the data acquisition instruction, calls a local service function corresponding to the target expansion function, acquires target local data, and sends the target local data to the relay server. And the relay server forwards the target local data to the cloud end equipment. The cloud device can receive the target local data sent by the relay server.

S430, generating a game rendering result corresponding to the target expansion function based on the target local data, and forwarding the game rendering result to the user end device through the relay server, so that the user end device displays the game rendering result.

Specifically, the cloud device may consume the target local data to implement a target expansion function of the cloud game. For example, after receiving a picture selected by a user from an album of the user end device, the cloud end device may set the picture as a user avatar of the user in a cloud game, so as to achieve a function of obtaining the picture from the user end device to set the cloud game user avatar. And then, the cloud equipment renders the implementation process and the result of the target expansion function to generate a game rendering result, and sends the game rendering result to the relay server. The relay server forwards the data to the customer premise equipment. And the user terminal equipment displays the game rendering result and presents the realization and the result of the target expansion function to the user.

According to the cloud game interaction scheme applied to the cloud terminal equipment for running the cloud game, the data acquisition instruction of the target expansion function of the cloud game can be sent to the relay server in the cloud game interaction process; receiving target local data obtained by a relay server from user equipment; and generating a game rendering result corresponding to the target expansion function based on the target local data, and forwarding the game rendering result to the user end equipment through the relay server so that the user end equipment displays the game rendering result. The target expansion function based on the target local data is realized, and the convenience of expanding and realizing other service functions in the cloud game is improved.

Based on the description of the above embodiments, in the embodiments of the present disclosure, the implementation process of the target expansion function of the cloud game is described in detail by taking the picture selection function and the voice chat function of the cloud game as examples.

In one example, the target extension function is to select a picture in a photo album of the user end device to set a user avatar of the cloud game. Fig. 5 shows a schematic diagram of a cloud game interaction process for a cloud game user avatar setting. As shown in fig. 5, the cloud game interaction process of the target extension function includes the following steps:

s501, sending, by a user client (app)532 in the user end device 530, a picture obtaining request generated by a user trigger to a cloud client (app)511 in the cloud end device 510. Here, the user client (app)532 refers to an app of a cloud game installed in the user-side device 530; the cloud client (app)511 refers to an app of a cloud game installed in the cloud terminal device 510.

S502 and the cloud client (app)511 send the picture obtaining request to the cloud SDK 512.

S503, the cloud SDK 512 calls a picture selection interface API based on the picture acquisition request, provides data call-back, and generates a picture acquisition instruction.

S504, the cloud SDK 512 sends the picture acquisition instruction to the IM server 520.

S505, the IM server 520 transfers the picture obtaining instruction to the client SDK 531 in the client device 530.

S506, the client SDK 531 parses the picture obtaining instruction, obtains an instruction parsing result, and invokes a picture selection component of the user-side device 530 according to the instruction parsing result. A user client (app)532 presents to the user the album 533 and associated controls for selecting the pictures contained therein. The user triggers the picture selection control and, after selecting a picture, the user client (app)532 determines the selected picture.

S507, the user client (app)532 sends the selected picture to the client SDK 531.

S508, the client SDK 531 compresses the picture and transmits IM message back to the IM server 520, wherein the IM message carries the compressed picture.

S509, the IM server 520 transfers the IM message to the cloud SDK 512.

S510, after receiving the transferred IM message, the cloud SDK 512 downloads and decompresses the picture from the IM server 520, so as to obtain the picture selected by the user.

S511, the cloud SDK 512 transmits the decompressed picture to the cloud client (app) 511.

S512, the cloud client (app)511 performs avatar setting using the decompressed picture.

In another example, the target extension function is real-time voice chat. Fig. 6 shows a schematic diagram of a cloud game interaction process for cloud game real-time voice chat. As shown in fig. 6, the cloud game interaction process of the target extension function includes the following steps:

s601, the user client (app)532 in the user end device 530 sends the recording start request of the voice chat generated by the user trigger to the cloud client (app)511 in the cloud end device 510.

S602, the cloud client (app)511 sends the recording start request to the cloud SDK 512.

S603 and the cloud SDK 512 call a recording start interface API of the real-time recording based on the recording start request, provide data call back and generate a recording start instruction.

S604, the cloud SDK 512 sends the recording starting instruction to the IM server 520.

S605, the IM server 520 transfers the recording start instruction to the client SDK 531 of the user end device 530.

S606, the client SDK 531 parses the recording start instruction, obtains an instruction parsing result, and calls the microphone component of the user end device 530 according to the instruction parsing result. A user client (app)532 may present the user with an interface to make real-time recordings with microphone 534 and obtain real-time PCM audio data output by microphone 534.

S607, the user client (app)532 sends the real-time PCM audio data to the client SDK 531.

S608, the client SDK 531 transmits real-time PCM audio data back to the IM server 520 through the real-time data channel.

S609, the IM server 520 transfers the real-time PCM audio data to the cloud SDK 512 through the real-time data channel.

S610, the cloud SDK 512 transmits the real-time PCM audio data to the cloud client (app) 511.

S611, the cloud client (app)511 consumes the real-time PCM audio data in real time.

S612, the user client (app)532 sends the record stop request of the voice chat generated by the user trigger to the cloud client (app) 511. The cloud client (app)511 sends a stop recording request to the cloud SDK 512.

S613, the cloud SDK 512 calls a recording stopping interface API of the real-time recording based on the recording stopping request to generate a recording stopping instruction. And, the cloud SDK 512 sends the recording stop instruction to the IM server 520.

S614, the IM server 520 transfers the recording stopping instruction to the client SDK 531.

S615, the client SDK 531 analyzes the recording stopping instruction to obtain an instruction analysis result, and stops calling the microphone component according to the instruction analysis result so as to stop recording by the microphone 534.

S616, the user client (app)532 sends the recording stop message generated after the recording is stopped to the client SDK 531.

S617, the client SDK 531 transmits the recording stop message back to the IM server 520 through the real-time data channel.

And S618, transferring the recording stop message to the cloud SDK 512 by the IM server 520 through the real-time data channel.

S619, the cloud SDK 512 transmits the recording stop message to the cloud client (app) 511.

S620, the cloud client (app)511 consumes the event corresponding to the stop recording message in real time.

Fig. 7 is a schematic structural diagram of a cloud game interaction apparatus configured on a client device of a cloud game according to an embodiment of the present disclosure. As shown in fig. 7, the cloud game interaction apparatus 700 configured on the client device of the cloud game may include:

a data acquisition instruction receiving module 710, configured to receive a data acquisition instruction of a target expansion function of the cloud game sent by the relay server; the data acquisition instruction is sent to the relay server by the cloud end equipment running the cloud game, and the target expansion function is realized based on the local service function in the user end equipment;

a target local data determining module 720, configured to invoke a local service function corresponding to the target expansion function based on the data obtaining instruction, and determine target local data;

the target local data sending module 730 is configured to forward the target local data to the cloud device via the relay server, so that the cloud device generates a game rendering result corresponding to the target extended function based on the target local data;

a game rendering result receiving module 740, configured to receive and display a game rendering result sent by the relay server; and the game rendering result is sent to the relay server by the cloud equipment.

The cloud game interaction device configured on the client device of the cloud game in the embodiment of the disclosure can realize the interaction of the cloud game, the data acquisition command is forwarded to the customer premise equipment from the cloud end equipment through the relay server, and after the customer premise equipment calls the local service function corresponding to the target expansion function to acquire the target local data, the target local data in the user terminal equipment is forwarded to the cloud end equipment through the relay server, the game rendering result with the target expansion function is generated in the cloud end equipment by utilizing the target local data, the game rendering result is forwarded to the user terminal equipment through the relay server to be played, the cloud device can realize the target expansion function realized based on the local service function in the user side device, and the expansion convenience of other functions except the game native function in the cloud game is improved.

In some embodiments, the target local data determination module 720 is specifically configured to:

when the data acquisition instruction is a data selection instruction, calling a data selection component based on the data selection instruction, and displaying a data selection interface corresponding to the data selection component;

responding to the selection operation of the data in the data selection interface, and determining target local data; the target local data is at least one of pictures, audio, video and files.

and when the data acquisition instruction is a short-time recording instruction, calling the microphone assembly to record based on the recording starting instruction in the short-time recording instruction, and stopping recording based on a time length upper limit threshold value in the short-time recording instruction or a triggering operation of stopping recording by a user to obtain short-time audio serving as target local data.

when the data acquisition instruction is a real-time recording instruction, based on a recording starting instruction in the real-time recording instruction, calling a microphone assembly to record, and obtaining real-time audio serving as target local data;

correspondingly, the target local data sending module 730 is specifically configured to:

and sending the real-time audio to the relay server through a real-time data channel between the relay server and the cloud equipment, so that the relay server sends the real-time audio to the cloud equipment through the real-time data channel between the relay server and the cloud equipment.

Further, the cloud game interaction apparatus 700 configured on the client device of the cloud game further includes a recording stopping module, configured to:

and after the real-time audio is sent to the relay server through a real-time data channel between the relay server and the real-time audio, receiving a recording stopping instruction in the real-time recording instructions sent by the relay server, stopping calling the microphone assembly based on the recording stopping instruction, and ending the real-time recording.

when the data acquisition instruction is an equipment state acquisition instruction, calling a sensor component based on the equipment state acquisition instruction to obtain sensor data serving as target local data;

wherein the sensor assembly comprises at least one of an acceleration sensor, a gyroscope sensor, a photosensor, a position sensor, a magnetic sensor, a camera, and a temperature sensor.

The cloud game interaction apparatus 700 configured on the client device of the cloud game further includes a data obtaining request sending module, configured to generate a data obtaining request in response to a user's trigger operation on a target expansion function before receiving a data obtaining instruction of the target expansion function of the cloud game sent by the relay server, and send the data obtaining request to the cloud device, so that the cloud device generates the data obtaining instruction based on the data obtaining request.

Fig. 8 is a schematic structural diagram of a cloud game interaction apparatus configured in a relay server according to an embodiment of the present disclosure. As shown in fig. 8, the cloud game interaction apparatus 800 configured in the relay server may include:

the data acquisition instruction forwarding module 810 is configured to receive a data acquisition instruction of a target expansion function of the cloud game, which is sent by a cloud device running the cloud game, and forward the data acquisition instruction to the user end device; the target expansion function is realized based on a local service function in the user terminal equipment, and the data acquisition instruction is used for indicating the user terminal equipment to call the local service function corresponding to the target expansion function to acquire target local data;

a target local data forwarding module 820, configured to receive target local data sent by a user end device, and forward the target local data to a cloud end device;

the game rendering result forwarding module 830 is configured to receive a game rendering result corresponding to the target expansion function generated by the cloud device based on the target local data, and forward the game rendering result to the user end device, so that the user end device displays the game rendering result.

The cloud game interaction device configured on the relay server in the embodiment of the disclosure can communicate the interaction between the cloud end equipment and the user end equipment of the cloud game through the relay server in the cloud game interaction process, so that the target expansion function of the cloud game can also perform user interaction operation and response as the same as the client end running in the user end equipment, and the expansion convenience of other functions in the cloud game except for the game native function is improved.

Fig. 9 is a schematic structural diagram of a cloud game interaction apparatus configured on a cloud device running a cloud game according to an embodiment of the present disclosure. As shown in fig. 9, the cloud game interaction apparatus 900 configured to operate a cloud device of a cloud game may include:

a data obtaining instruction sending module 910, configured to send a data obtaining instruction of a target expansion function of the cloud game to the relay server; the target expansion function is realized based on a local service function in the user terminal equipment, and the data acquisition instruction is used for indicating the user terminal equipment to call the local service function corresponding to the target expansion function to acquire target local data;

a target local data receiving module 920, configured to receive target local data obtained by the relay server from the user end device;

the game rendering result sending module 930 is configured to generate a game rendering result corresponding to the target expansion function based on the target local data, and forward the game rendering result to the user end device through the relay server, so that the user end device displays the game rendering result.

The cloud game interaction device configured on the cloud terminal device of the cloud game in the embodiment of the disclosure can forward the data acquisition instruction to the user terminal device through the relay server in the cloud game interaction process, and can receive the target local data through the relay server, so that the target expansion function based on the target local data is realized, and the expansion convenience of other functions in the cloud game except the game native function is improved.

Embodiments of the present disclosure also provide a cloud game interaction device, which may include a processor and a memory, where the memory may be used to store executable instructions. The processor may be configured to read the executable instructions from the memory and execute the executable instructions to implement the cloud game interaction method applied to the client device of the cloud game, or the cloud game interaction method applied to the relay server, or the cloud game interaction method applied to the cloud device running the cloud game in the foregoing embodiments.

Fig. 10 shows a schematic structural diagram of a cloud game interaction device provided by an embodiment of the present disclosure.

In some embodiments, the cloud game interaction device 1000 may be the device in which the cloud device 110 shown in fig. 1 is located. In other embodiments, the cloud game interaction device 1000 may be the relay server 120 shown in fig. 1. In still other embodiments, the cloud game interaction device 1000 may be the device where the client device 130 shown in fig. 1 is located.

Referring now specifically to fig. 10, a schematic diagram of a cloud gaming interaction device 1000 suitable for use in implementing embodiments of the present disclosure is shown. It should be noted that the cloud game interaction device 1000 shown in fig. 10 is only an example, and should not bring any limitation to the functions and the scope of the application of the embodiments of the present disclosure.

As shown in fig. 10, the cloud game interaction device 1000 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 1001 that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1002 or a program loaded from a storage means 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data necessary for the operation of the information processing apparatus 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other by a bus 1004. An input/output interface (I/O interface) 1005 is also connected to the bus 1004.

Generally, the following devices may be connected to the I/O interface 1005: input devices 1006 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 1007 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage devices 1008 including, for example, magnetic tape, hard disk, and the like; and a communication device 1009. The communication apparatus 1009 may allow the cloud game interaction device 1000 to perform wireless or wired communication with other devices to exchange data. While fig. 10 illustrates a cloud gaming interaction device 1000 with various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

The present disclosure also provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the processor is enabled to implement the cloud game interaction method applied to a user end device of a cloud game, or the cloud game interaction method applied to a relay server, or the cloud game interaction method applied to a cloud end device running the cloud game in the foregoing embodiments.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication means 1009, or installed from the storage means 1008, or installed from the ROM 1002. The computer program performs the above-described functions defined in the information display method of the embodiment of the present disclosure when executed by the processing device 1001.

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP, and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be included in the cloud game interaction device; or may exist separately without being assembled into the cloud game interaction device.

The computer readable medium carries one or more programs which, when executed by the cloud gaming interaction device, cause the cloud gaming interaction device to perform: the method comprises the steps of a cloud game interaction method applied to user side equipment of a cloud game, or the steps of the cloud game interaction method applied to a relay server, or the steps of the cloud game interaction method applied to cloud side equipment of a cloud game.

In embodiments of the present disclosure, computer program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of an element does not in some cases constitute a limitation on the element itself.

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A cloud game interaction method is applied to client equipment of a cloud game, and the method comprises the following steps:

2. The method according to claim 1, wherein when the data obtaining instruction is a data selecting instruction, the invoking a local service function corresponding to the target extended function based on the data obtaining instruction, and determining the target local data includes:

calling a data selection component based on the data selection instruction, and displaying a data selection interface corresponding to the data selection component;

responding to the selection operation of the data in the data selection interface, and determining the target local data; wherein the target local data is at least one of pictures, audio, video and files.

3. The method according to claim 1, wherein when the data obtaining instruction is a short-time recording instruction, the calling a local service function corresponding to the target extended function based on the data obtaining instruction, and determining the target local data includes:

and calling a microphone assembly to record based on a recording starting instruction in the short-time recording instruction, and stopping recording based on a time length upper limit threshold value in the short-time recording instruction or a triggering operation of stopping recording by a user to obtain short-time audio serving as the target local data.

4. The method according to claim 1, wherein when the data obtaining instruction is a real-time recording instruction, the calling a local service function corresponding to the target extended function based on the data obtaining instruction, and determining the target local data includes:

calling a microphone assembly to record based on a recording starting instruction in the real-time recording instruction to obtain real-time audio serving as the target local data;

the forwarding the target local data to the cloud device via the relay server comprises:

and sending the real-time audio to the relay server through a real-time data channel between the relay server and the cloud end equipment, so that the relay server sends the real-time audio to the cloud end equipment through the real-time data channel between the relay server and the cloud end equipment.

5. The method of claim 4, wherein after sending the real-time audio to the relay server over a real-time data channel with the relay server, the method further comprises:

and receiving a recording stopping instruction in the real-time recording instruction sent by the relay server, stopping calling the microphone assembly based on the recording stopping instruction, and ending the real-time recording.

6. The method according to claim 1, wherein when the data obtaining instruction is an apparatus status obtaining instruction, the calling a local service function corresponding to the target extended function based on the data obtaining instruction, and determining the target local data includes:

calling a sensor component based on the equipment state acquisition instruction to obtain sensor data serving as the target local data;

wherein the sensor assembly includes at least one of an acceleration sensor, a gyroscope sensor, a photosensor, a position sensor, a magnetic sensor, a camera, and a temperature sensor.

7. The method according to any one of claims 1 to 6, wherein before the receiving of the data acquisition instruction of the target extended function of the cloud game sent by the relay server, the method further comprises:

responding to the triggering operation of the user on the target expansion function, generating a data acquisition request, and sending the data acquisition request to the cloud end equipment so that the cloud end equipment generates the data acquisition instruction based on the data acquisition request.

8. A cloud game interaction method is applied to a relay server, and comprises the following steps:

9. A cloud game interaction method is applied to cloud equipment for running a cloud game, and comprises the following steps:

10. A cloud game interaction apparatus, configured to be a client device of a cloud game, the apparatus comprising:

11. A cloud game interaction apparatus configured to a relay server, the apparatus comprising:

12. A cloud game interaction apparatus configured to be installed in a cloud device running a cloud game, the apparatus comprising:

13. A cloud gaming interaction device, the device comprising:

a processor;

a memory for storing executable instructions;

the processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the cloud game interaction method applied to the client device of the cloud game of any one of claims 1 to 7, or implement the cloud game interaction method applied to the relay server of claim 8, or implement the cloud game interaction method applied to the cloud device of claim 9.

14. A computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the processor is enabled to implement the cloud game interaction method applied to the client device of the cloud game in any one of claims 1 to 7, or the cloud game interaction method applied to the relay server in claim 8, or the cloud game interaction method applied to the cloud device running the cloud game in claim 9.