CN118132292A

CN118132292A - Message processing method, device, electronic equipment and readable storage medium

Info

Publication number: CN118132292A
Application number: CN202410533084.9A
Authority: CN
Inventors: 彭飞; 武红杰
Original assignee: Longxin Zhongke Hefei Technology Co ltd
Current assignee: Longxin Zhongke Hefei Technology Co ltd
Priority date: 2024-04-28
Filing date: 2024-04-28
Publication date: 2024-06-04
Anticipated expiration: 2044-04-28

Abstract

The embodiment of the invention provides a message processing method, a message processing device, electronic equipment and a readable storage medium, wherein the method comprises the following steps: under the condition that cloud desktop data pushed by a server are received, sequentially and correspondingly distributing to-be-processed messages meeting a preset shunting type in the cloud desktop data to a to-be-rendered display queue and an audio queue to be played; decoding and synthesizing the first message to be processed in the display queue to be rendered based on the first target thread to obtain target frame data, and decoding and synthesizing the second message to be processed in the audio queue to be played based on the second target thread to obtain target audio; the first target thread and the second target thread are executed in parallel; and carrying out self-adaptive synchronous display and play on the target frame data and the target audio on a screen corresponding to the client. In this way, the first message to be processed and the second message to be processed are processed based on different threads respectively and simultaneously, so that the message processing time is saved, and the message processing efficiency is improved.

Description

Message processing method, device, electronic equipment and readable storage medium

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a message processing method, a device, an electronic apparatus, and a readable storage medium.

Background

With the development of cloud computing, desktop virtualization is widely applied by means of cloud terminals or cloud desktops. The virtual machine cloud desktop image of the data center may be transmitted to the user's client through a proprietary communication protocol (e.g., SPICE (Simple Protocol for Independent Computing Environments, independent computing environment simple protocol), and the client then exposes the cloud desktop image to the user based on desktop virtualization technology.

In the related art, when the client receives the audio data and the display data, a co-procedure is often executed on the basis of the single processor core to sequentially perform serial processing on the audio data in the audio channel and the display data in the display channel, for example, when the co-procedure processes the audio data in the audio channel on the basis of the entire processing capability of the single processor core, the display channel and other channels are blocked, and when the co-procedure processes the display data in the display channel on the basis of the entire processing capability of the single processor core, the audio channel and other channels are blocked. Therefore, as the cloud desktop data of different types are processed in series based on one protocol, the processing processes of the audio data and the display data cannot be synchronized, the video frames cannot be synchronized with the audio, so that the display and the play of the display data and the audio data are poor in fluency and the efficiency of displaying the play flow is low.

Disclosure of Invention

To overcome the problems in the related art, the present invention provides a message processing method, apparatus, electronic device, and readable storage medium.

In a first aspect, the present invention provides a message processing method, applied to a client, the method including:

Receiving cloud desktop data corresponding to a cloud desktop pushed by a server based on an independent computing environment simple protocol; the cloud desktop runs in a virtual machine of the server;

under the condition that the cloud desktop data are received, sequentially and correspondingly distributing the information to be processed meeting the preset shunting type in the cloud desktop data to a display queue to be rendered and an audio queue to be played;

Decoding and synthesizing the first message to be processed in the display queue to be rendered based on the first target thread to obtain target frame data, and decoding and processing the second message to be processed in the audio queue to be played based on the second target thread to obtain target audio; the first target thread and the second target thread are executed in parallel, and the first target thread and the second target thread run respectively based on different processor cores in the client;

And carrying out self-adaptive synchronous display and play on the target frame data and the target audio on a screen corresponding to the client so as to display a remote cloud desktop corresponding to the cloud desktop data on the client.

Optionally, the preset offload type includes that a message type of the cloud desktop data is a first type or a second type; the distributing the to-be-processed messages meeting the preset distribution type in the cloud desktop data to the to-be-rendered display queue and the to-be-played audio queue sequentially and correspondingly comprises the following steps:

Determining whether the cloud desktop data meets a preset distribution type or not based on the message header of the cloud desktop data;

determining cloud desktop data with the message type of a first type or a second type as the message to be processed;

Distributing a first message to be processed, the message type of which is the first type, to a to-be-rendered display queue, and distributing a second message to be processed, the message type of which is the second type, to a to-be-played audio queue.

Optionally, the method further comprises:

determining cloud desktop data which does not meet the preset distribution type as a first message;

For any one of the first messages, distributing the first message to a first message channel corresponding to the first message based on the message type of the first message;

And under the condition that the preset processing sequence corresponding to the first message channel is reached, analyzing the first message in the first message channel based on a third target thread to obtain data to be applied.

Optionally, the method further comprises:

aiming at any message to be processed, analyzing the sub-message type corresponding to the message to be processed based on the message content of the message to be processed;

And determining a processing function corresponding to the sub-message type based on a target mapping table as a target processing function corresponding to the message to be processed.

Optionally, the decoding and synthesizing process is performed on the first message to be processed in the display queue to be rendered to obtain target frame data, including:

Acquiring a first message to be processed from the display queue to be rendered;

Performing a first processing operation on the first message to be processed based on a display channel and a first target processing function corresponding to the first message to be processed to obtain target frame data;

the decoding process is performed on the second message to be processed in the audio queue to be played to obtain the target audio, including:

Acquiring a second message to be processed from the audio queue to be played;

And performing a second processing operation on the second message to be processed based on the audio channel and a second target processing function corresponding to the second message to be processed to obtain target audio.

Optionally, the method further comprises:

constructing a thread pool comprising at least two threads based on the number of processor cores corresponding to the client;

the first number of the at least two threads is the same as the second number of the processor cores, and the first target thread and the second target thread are any idle threads in the thread pool.

Optionally, the first message to be processed carries display update information; the adaptively synchronously displaying and playing the target frame data and the target audio on the screen corresponding to the client side includes:

based on a target time stamp corresponding to the target audio, acquiring display update information of target frame data matched with the target time stamp, and determining a display update area corresponding to the target frame data matched with the target time stamp;

And sending a display refreshing signal to a target display system, adaptively displaying the display updating area on a screen corresponding to the client, and synchronously playing the target audio.

Optionally, the method further comprises:

After all the first messages to be processed contained in the display queue to be rendered are processed based on the first target thread, releasing the first target thread back to a thread pool, and marking the first target thread as an idle thread;

And after all second to-be-processed messages contained in the to-be-played audio queue are processed based on the second target thread, releasing the second target thread back to the thread pool, and marking the second target thread as an idle thread.

In a second aspect, the present invention provides a message processing apparatus, for application to a client, the apparatus comprising:

the first receiving module is used for receiving cloud desktop data corresponding to the cloud desktop pushed by the server based on the independent computing environment simple protocol; the cloud desktop runs in a virtual machine of the server;

The first distribution module is used for sequentially and correspondingly distributing the to-be-processed messages meeting the preset splitting type in the cloud desktop data to a to-be-rendered display queue and an audio queue to be played under the condition that the cloud desktop data are received;

The first processing module is used for decoding and synthesizing the first message to be processed in the display queue to be rendered based on the first target thread to obtain target frame data, and decoding the second message to be processed in the audio queue to be played based on the second target thread to obtain target audio; the first target thread and the second target thread are executed in parallel, and the first target thread and the second target thread run respectively based on different processor cores in the client;

and the first display module is used for carrying out self-adaptive synchronous display and play on the target frame data and the target audio on a screen corresponding to the client so as to display a remote cloud desktop corresponding to the cloud desktop data on the client.

In a third aspect, the present invention provides an electronic device comprising: a processor, a memory and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the message processing method of any of the above first aspects when executing the program.

In a fourth aspect, the invention provides a readable storage medium, which when executed by a processor of an electronic device, enables the electronic device to perform the steps of a message processing method as in any of the embodiments of the first aspect described above.

In the embodiment of the invention, under the condition of receiving cloud desktop data pushed by a server, to-be-processed messages meeting a preset splitting type in the cloud desktop data are sequentially and correspondingly distributed to a to-be-rendered display queue and an to-be-played audio queue; decoding and synthesizing the first message to be processed in the display queue to be rendered based on the first target thread to obtain target frame data, and decoding and synthesizing the second message to be processed in the audio queue to be played based on the second target thread to obtain target audio; the first target thread and the second target thread are executed in parallel; and carrying out self-adaptive synchronous display and play on the target frame data and the target audio on a screen corresponding to the client. Therefore, cloud desktop data are pre-distributed based on the preset distribution type, and the screened information to be processed meeting the preset distribution type is further distributed to different processing queues (a display queue to be rendered and an audio queue to be played), so that the cloud desktop data in the different processing queues can be processed based on different threads, and the data processing efficiency is improved to a certain extent. Furthermore, decoding processing is carried out on the to-be-rendered display queue and the to-be-played audio queue respectively based on the first target thread and the second target thread in parallel to obtain target frame data and target audio, the first target thread and the second target thread operate based on different processor cores in the client respectively, and compared with processing of the display data and the audio data based on the co-Cheng Chuanhang in sequence, the processing flow of the first to-be-processed message and the second to-be-processed message is separated from the co-processing, and message processing is carried out based on the two threads in parallel, so that on the basis of fully utilizing the multi-core computing capability of the client, the message processing time is saved, and the message processing efficiency is improved. Meanwhile, as the first message to be processed and the second message to be processed are processed based on different threads respectively, the synchronization of the cloud desktop data processing of different types is improved, correspondingly, the display of the target frame data and the play synchronization of the target audio obtained based on the parallel thread synchronization processing are improved to a certain extent, and the display and play fluency and the audio-video display synchronization are improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of steps of a message processing method according to an embodiment of the present invention;

fig. 2 is a schematic diagram of a client architecture according to an embodiment of the present invention;

fig. 3 is a block diagram of a message processing apparatus according to an embodiment of the present invention;

fig. 4 is a block diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Fig. 1 is a flowchart of steps of a message processing method provided in an embodiment of the present invention, which is applied to a client, where the client may be a client supporting SPICE protocol, and a user may access an own virtual cloud desktop by using the client.

In the practical application of the SPICE cloud desktop scene, at least one client is connected through a server, each client is used for displaying a remote cloud desktop, and the remote cloud desktop is transmitted between the client and the server through SPICE protocols; running SPICE SERVER at the server and SPICE CLIENT at the client; SPICE protocol includes video, image, audio, mouse, and multiple communication channels corresponding to universal serial bus (Universal Serial Bus, USB), each of which can be set for different functional implementations, and each of which is a socket. The SPICE protocol is an open source cloud desktop transmission protocol of a Kernel-based virtual machine (Kernel-based Virtual Machine, KVM virtual machine), and is mainly applied to RedHat desktop virtualization and can also be applied to other cloud desktop applications. It provides a solution for interacting with the cloud desktop; SPICE clients can access the cloud desktop and use local devices (e.g., keyboard, USB, etc.) through SPICE protocols. The SPICE server is responsible for completing the interaction between the server virtual machine and SPICE client. After the client is started, connection is established with the server, and the channel connection process between the server and the client is initiated by the client. Specifically, the client may establish a primary channel (MAIN CHANNEL) connection with the server, and then create a plurality of different types of communication channels based on the primary channel. For example, the client may send a connection request (e.g., REDLINKMESS) to the server, which, upon receiving the connection request, sends a reply message (e.g., REDLINKREPLY) to the client. When the client receives the reply message (e.g., REDLINKREPLY), it is determined whether an error code is included, and if there is no error, the client encrypts the public key code at REDLINKREPLY and sends it to the server. The server receives the password and sends the connection result to the client. The client checks the connection results and if the connection results indicate that a connection is possible, a main channel (MAIN CHANNEL) connection may be established, thus establishing a session between the client and the server.

Types of communication channels may include, but are not limited to, a display channel (DISPLAYCHANNEL) for processing image commands, pictures, and video streams; an input channel (InputsChannel) for processing related contents such as keyboard and mouse inputs; the pointer channel (CursorChannel) is used for processing related contents such as the position, the visibility, the pointer shape and the like of the pointer device; a recording/reproducing channel (PlaybackChannel) for processing the sound from the server (related to the sound of the sink) so that it can be played by the client; a sound recording channel (RecordChannel) for processing sound captured from the client, etc.

As shown in fig. 1, the method may include:

Step 101, receiving cloud desktop data corresponding to a cloud desktop pushed by a server based on an independent computing environment simple protocol; and the cloud desktop runs in the virtual machine of the server.

Step 102, under the condition that the cloud desktop data is received, sequentially and correspondingly distributing the to-be-processed messages meeting the preset splitting type in the cloud desktop data to a to-be-rendered display queue and an to-be-played audio queue.

In the embodiment of the invention, in an application scene of a SPICE cloud desktop, a server receives and analyzes desktop data corresponding to desktop content of a virtual machine cloud desktop for providing desktop service, and the cloud desktop data is obtained after the desktop data is preprocessed. And after the connection between the server and the client is established, the cloud desktop data is sent to the client based on SPICE protocol. Under the condition that the client receives cloud desktop data pushed by the server, the cloud desktop data can be distributed to different communication channels for processing based on a message distribution module in the client. Before the message distribution module is used for distributing the message, in order to improve the playing effect of audio playing and picture displaying, a message distribution module can be configured in the client side in advance, and cloud desktop data of different types can be pre-distributed based on the message distribution module. Specifically, a preset distribution type needing to be distributed in advance can be preset, whether the message type of the cloud desktop data meets the preset distribution type or not is judged, and the cloud desktop data meeting the preset distribution type in the cloud desktop data is used as a message to be processed. Wherein the message to be processed may comprise a first message to be processed and a second message to be processed.

Further, the client may be configured with a message processing module, which may pre-add one or at least two message queues for storing messages to be processed of different message types when the client establishes a main channel initialization with the server. The message queue may include a display queue to be rendered and an audio queue to be played. The to-be-rendered display queue is used for storing to-be-processed messages with the message type being the first type, and the to-be-played audio queue is used for storing to-be-processed messages with the message type being the second type. The first type may include a display type, the cloud desktop data of the display type may include, for example, image data, video stream data, etc., the second type may include an audio type, the cloud desktop data of the audio type may include audio data, etc. And correspondingly distributing the message to be processed to a display queue to be rendered and an audio queue to be played based on the message type. For example, a first pending message of a first type may be distributed to a to-be-rendered display queue and a second pending message of a second type may be distributed to a to-be-played audio queue.

Step 103, decoding and synthesizing the first message to be processed in the display queue to be rendered based on the first target thread to obtain target frame data, and decoding the second message to be processed in the audio queue to be played based on the second target thread to obtain target audio; the first target thread and the second target thread are executed in parallel, and the first target thread and the second target thread run respectively based on different processor cores in the client.

In the embodiment of the invention, the client may be a multi-core processor architecture, and at least two threads may be preconfigured based on the number of processors of the client and the processing capability of the terminal. Wherein a processor core may correspond to a thread, that is, after a client starts a thread, the thread performs message processing based on the processor core, that is, using the computing power corresponding to the processor core. The message processing module of the client can start a first target thread, and decode and synthesize each first message to be processed in the display queue to be rendered through the display channel based on the first target thread to obtain processed target frame data. Meanwhile, the message processing module starts a second target thread, decodes a second message to be processed in the audio queue to be played through the audio channel based on the second target thread, and obtains processed target audio. The first target thread and the second target thread are different threads, and the first target thread and the second target thread execute in parallel, that is, the first message to be processed and the second message to be processed are processed based on different threads (the first target thread and the second target thread) at the same time. Illustratively, the first target thread may comprise a thread, and likewise, the second target thread may comprise a different thread than the first target thread. In one possible implementation manner, in order to further improve the processing efficiency of the display data and the audio data, the first target thread may include multiple threads, and the second target thread may also include multiple threads, which is not limited in this embodiment of the present invention.

Based on a first target thread, sequentially acquiring first messages to be processed in a display queue to be rendered through a display channel, decoding the first messages to be processed according to a processing function for processing the messages to be processed in the display channel, and synthesizing display data after the decoding processing. Specifically, the display data after the decoding process may be copied to a base map (buffer) to be displayed, and the target frame data may be obtained based on the display data after the decoding process and the base map (buffer) to be displayed and synthesized and fused.

And 104, performing self-adaptive synchronous display and play on the target frame data and the target audio on a screen corresponding to the client so as to display a remote cloud desktop corresponding to the cloud desktop data on the client.

In the embodiment of the invention, after the first message to be processed is decoded and synthesized to obtain the target frame data, and the second message to be processed is decoded to obtain the target audio, the target frame data and the target audio can be displayed and played in a self-adaptive synchronous manner on the screen corresponding to the client, so that the synchronization of the audio and the video at the client side is realized. In this way, desktop content of the virtual machine can be synchronously and remotely displayed on the client. Specifically, the image frame corresponding to the target frame data may be displayed on the screen corresponding to the client by transmitting a display refresh signal to the target display system (e.g., GTK display system). A call interface (e.g., connect interface) links the target audio to the audio playback device and populates the target audio to the corresponding left and right channels.

In summary, in the embodiment of the present invention, under the condition that cloud desktop data pushed by a server is received, to-be-processed messages meeting a preset splitting type in the cloud desktop data are sequentially and correspondingly distributed to a to-be-rendered display queue and a to-be-played audio queue; decoding and synthesizing the first message to be processed in the display queue to be rendered based on the first target thread to obtain target frame data, and decoding and synthesizing the second message to be processed in the audio queue to be played based on the second target thread to obtain target audio; the first target thread and the second target thread are executed in parallel; and carrying out self-adaptive synchronous display and play on the target frame data and the target audio on a screen corresponding to the client. Therefore, cloud desktop data are pre-distributed based on the preset distribution type, and the screened information to be processed meeting the preset distribution type is further distributed to different processing queues (a display queue to be rendered and an audio queue to be played), so that the cloud desktop data in the different processing queues can be processed based on different threads, and the data processing efficiency is improved to a certain extent. Furthermore, decoding processing is carried out on the to-be-rendered display queue and the to-be-played audio queue respectively based on the first target thread and the second target thread in parallel to obtain target frame data and target audio, the first target thread and the second target thread operate based on different processor cores in the client respectively, and compared with processing of the display data and the audio data based on the co-Cheng Chuanhang in sequence, the processing flow of the first to-be-processed message and the second to-be-processed message is separated from the co-processing, and message processing is carried out based on the two threads in parallel, so that on the basis of fully utilizing the multi-core computing capability of the client, the message processing time is saved, and the message processing efficiency is improved. Meanwhile, as the first message to be processed and the second message to be processed are processed based on different threads respectively, the synchronization of the cloud desktop data processing of different types is improved, correspondingly, the display of the target frame data and the play synchronization of the target audio obtained based on the parallel thread synchronization processing are improved to a certain extent, and the display and play fluency and the audio-video display synchronization are improved.

Meanwhile, by the message processing method provided by the embodiment of the invention, the display efficiency of the SPICE client and the synchronicity of audio and video display can be optimized and improved, and further, in the application scene of the SPICE cloud desktop, the SPICE cloud desktop remote experience is improved.

Optionally, the message type including the cloud desktop data is a first type or a second type.

Step 101 may comprise the steps of:

step 201, determining whether the cloud desktop data meets a preset distribution type or not based on the message header of the cloud desktop data.

In the embodiment of the invention, under the condition that cloud desktop data is received, the message header of the cloud desktop data is read, and whether the cloud desktop data belongs to the first type or the second type is judged based on the message header. The message header of the cloud desktop data can be used for representing the message type corresponding to the cloud desktop data, so that the message type corresponding to the cloud desktop data can be judged based on the message header of the cloud desktop data. For example, if the first type is a display type and the second type is an AUDIO type, and if the cloud desktop data is spice_audio_fmt_valid, the message type corresponding to the cloud desktop data may be determined to be the second type based on the message header, so as to satisfy the preset separation type. Assuming that the cloud desktop DATA is spice_msg_display_stream_data, the message type corresponding to the cloud desktop DATA can be judged to be the first type based on the message header, and the preset separation type is met.

Step 202, determining cloud desktop data with a message type of a first type or a second type as the message to be processed.

Step 203, distributing a first message to be processed with a first message type to a display queue to be rendered, and distributing a second message to be processed with a second message type to an audio queue to be played.

In the embodiment of the invention, the cloud desktop data with the message type of the first type or the second type is determined to be the message to be processed. Specifically, cloud desktop data with a message type of a first type may be determined as a first message to be processed, and cloud desktop data with a message type of a second type may be determined as a second message to be processed. Correspondingly, after judging the messages to be processed in the cloud desktop data, each message to be processed can be packaged as a context executed by the message processing module, the packaged first message to be processed is correspondingly distributed to a display queue to be rendered, and the packaged second message to be processed is correspondingly distributed to an audio queue to be played.

In the embodiment of the invention, the message types of the cloud desktop data are judged based on the message header of the cloud desktop data, and the data to be processed of different message types are respectively distributed to the corresponding queues, so that the data to be processed of different types can be split in advance, and the messages to be processed in each queue can be processed based on different threads.

Optionally, the embodiment of the invention can further comprise the following steps:

step 301, determining cloud desktop data which does not meet the preset distribution type as a first message.

In the embodiment of the invention, the cloud desktop data which does not meet the preset distribution type is determined as the first message, that is, the cloud desktop data which is not the first type and is not the second type in the cloud desktop data is determined as the first message. The first message is cloud desktop data of a non-to-be-processed message in the cloud desktop data received by the client.

Step 302, for any one of the first messages, distributing the first message to a first message channel corresponding to the first message based on a message type of the first message.

In the embodiment of the invention, after the message distribution module outputs the first message, the message type of the first message is determined based on the message distribution module, and because other message types except the first type and the second type in the client terminal respectively correspond to different message channels (such as an input channel corresponding to the input type, a pointer channel corresponding to the pointer type and the like), the first message channel corresponding to the message type can be determined based on the message type of the first message. The first message is distributed to a corresponding first message channel based on the message distribution module. For example, when the message type of the first message is an input type (e.g., a keyboard inputs a message), and the message channel corresponding to the input type is an input channel, the first message is distributed to the input channel by the message distribution module.

Step 303, under the condition that the preset processing sequence corresponding to the first message channel is reached, analyzing the first message in the first message channel based on a third target thread to obtain data to be applied.

In the embodiment of the invention, for any first message, the analysis processing is sequentially performed according to the preset processing sequence based on the third target thread, so as to obtain the data to be applied corresponding to the first message. The third target thread may be a thread or a coroutine. For example, messages in each message channel (not including the display channel and the audio channel) may be serially processed according to the co-range time slice based on the third target thread, that is, different message channels may correspond to different processing sequences according to the co-range time slice, the switching between the processing tasks of multiple message channels may be performed in the third target thread, and the third target thread may process messages of one message channel at the same time. In an exemplary embodiment, when a preset processing sequence corresponding to the first message channel is reached, the third target thread analyzes the first message through a processing function corresponding to the first message in the first message channel, so as to obtain data to be applied.

In the embodiment of the invention, aiming at any first message, the first message is distributed to the message channels corresponding to the first message, so that the other first messages received by the client side except the message to be processed are analyzed and processed according to the preset processing sequence based on the corresponding message channels, and the data to be applied is obtained. In this way, the cloud desktop data is processed based on three different threads (a first message to be processed is processed based on a first target thread, a second message to be processed is processed based on a second target thread, and the first message is processed based on a third target thread), so that the processor resource is saved to a certain extent while the display playing effect and the playing progress of the first message to be processed and the second message to be processed are not affected.

step 401, for any message to be processed, resolving a sub-message type corresponding to the message to be processed based on the message content of the message to be processed.

In the embodiment of the invention, after the message distribution module distributes the cloud desktop data to determine the message to be processed, the message header and the message content of any message to be processed (including the first message to be processed and the second message to be processed) are analyzed, and the message type and the sub-message type corresponding to the message to be processed are determined. Specifically, the message type corresponding to the message to be processed can be determined based on the message header of the message to be processed, and then the sub-message type corresponding to the message to be processed can be determined further based on the message content of the message to be processed. The message content of any message to be processed can characterize different processing operations to be executed on the message to be processed, and the message to be processed can be divided into different sub-message types according to different processing operation types. Wherein the message type may include a first type, which may include at least two sub-message types, and a second type, which may include at least two sub-message types. For example: when the first type is a display type, sub-message types that may be included include: an image flip type (characterizing that the message to be processed needs to be subjected to image flip processing), an image stretching type (characterizing that the message to be processed needs to be subjected to image stretching processing), a video rendering type (characterizing that the message to be processed needs to be subjected to video rendering processing), and the like; when the second type is an audio type, sub-message types that may be included include: an audio decompression type (characterizing a message to be processed needs to be subjected to audio decompression processing), an audio decoding type (characterizing a message to be processed needs to be subjected to decoding processing based on a decoder), and the like.

Step 402, determining a processing function corresponding to the sub-message type based on the target mapping table, and using the processing function as a target processing function corresponding to the message to be processed.

In the embodiment of the invention, a target mapping table recorded with the mapping relation between the sub-message type and the processing function is provided, and the target mapping table is used for retrieving the processing function corresponding to the sub-message type. Wherein the processing function may be a message processing handle (handle). Based on the sub-message type corresponding to any message to be processed, a processing function corresponding to the sub-message type is searched in the target mapping table, and the processing function is determined as a target processing function for processing the message to be processed. It can be appreciated that the mapping relationship between the sub-message types and the processing functions in the target mapping table can be automatically set according to the requirements, which is not limited in the embodiment of the present invention.

In the embodiment of the invention, the sub-message type corresponding to the message to be processed is determined, and the target processing function corresponding to the message to be processed is determined based on the target mapping table and the sub-message type, so that the target processing function corresponding to the message to be processed can be determined in advance and is convenient for the subsequent rapid processing of the target processing function based on the target processing function, and the message processing efficiency is improved.

Optionally, in step 102, "performing decoding synthesis processing on the first message to be processed in the display queue to be rendered to obtain the target frame data" may include the following steps:

Step 501, obtaining a first message to be processed from the display queue to be rendered.

In the embodiment of the invention, after the message distribution module distributes the first message to be processed to the display queue to be rendered, the message count reference is increased for the first message to be processed in the display queue to be rendered. The message count reference is used for representing the use state of the first message to be processed, when the message count reference is marked as 0, the first message to be processed is processed (or used), and a target program library (such as Glib library) corresponding to the client can automatically recycle the part of memory; when the message count reference flag is not 0, the first pending message is characterized as pending or being processed. Therefore, the use state of the message to be processed in the display queue to be rendered can be obtained through the message reference of the first message to be processed in the display queue to be rendered, the message to be processed in the display queue to be rendered is ensured not to be missed, and in the condition that the message count reference represents that the message to be processed is processed (or used), the memory resource can be recovered and released in time, and the memory resource waste is avoided. The client starts a first target thread, detects the number of messages in a display queue to be rendered, and acquires a first message to be processed from the display queue to be rendered according to the sequence of the queue under the condition that the number of the messages in the display queue to be rendered is greater than 0.

Step 502, performing a first processing operation on the first message to be processed based on the display channel and a first target processing function corresponding to the first message to be processed, so as to obtain target frame data.

In the embodiment of the invention, based on the display channel and the first target processing function in the display channel, the first processing operation is performed on the first message to be processed, so as to obtain target frame data. The first target processing function is determined based on the sub-message type corresponding to the first message to be processed and the target mapping table. For example, when the sub-message type corresponding to the first message to be processed is the image stretching type, the first target processing function corresponding to the first message to be processed may be a function (for example DISPLAY HANDLE) for performing image stretching processing on the first message to be processed, and accordingly, the first processing operation may include an image stretching processing operation, a synthesizing operation, and the like, so that the image stretching processing is performed on the first message to be processed based on the first target processing function and the display channel, and the data after the image stretching processing is synthesized, to obtain the target frame data. After the target frame data is obtained, the message reference of the corresponding first message to be processed can be released, for example, the message reference of the first message to be processed is marked as 0, and the memory can be recovered by the characterization target program library.

Accordingly, the step 102 of "decoding the second to-be-processed message in the to-be-played audio queue to obtain the target audio" may include the following steps:

Step 503, obtaining a second message to be processed from the audio queue to be played.

In the embodiment of the invention, after the message distribution module distributes the second message to be processed to the audio queue to be played, the message count reference is increased for the second message to be processed in the audio queue to be played. The message count reference is used for representing the use state of the second message to be processed, when the message count reference is marked as 0, the second message to be processed is processed (or used), and a target program library (such as Glib library) corresponding to the client can automatically recycle the part of memory; when the message count reference flag is not 0, the second pending message is characterized as pending or being processed. Therefore, the use state of the message to be processed in the audio queue to be played can be obtained through the message reference of the first message to be processed in the audio queue to be played, the message to be processed in the audio queue to be played is ensured not to be missed, and in addition, under the condition that the message count reference represents that the message to be processed is processed (or used), the memory resource can be recovered and released in time, and the memory resource waste is avoided. The client starts a second target thread, detects the number of messages in the audio queue to be played, and acquires second messages to be processed from the audio queue to be played according to the queue sequence under the condition that the number of the messages in the audio queue to be played is greater than 0.

Step 504, performing a second processing operation on the second message to be processed based on the audio channel and a second target processing function corresponding to the second message to be processed, so as to obtain target audio.

In the embodiment of the invention, based on the audio channel and the second target processing function in the audio channel, the second processing operation is performed on the second message to be processed to obtain the target audio. The second target processing function is determined based on the sub-message type corresponding to the second message to be processed and the target mapping table. For example, in the case where the sub-message type corresponding to the second message to be processed is an audio decompression type, the second target processing function corresponding to the second message to be processed may be a function (for example, audio handle) for performing audio decompression processing on the second message to be processed, and accordingly, the second processing operation may include an audio decompression processing operation. Therefore, the second message to be processed is subjected to audio decompression processing based on the second target processing function and the display channel, and target audio is obtained. After the target audio is obtained, the message reference of the corresponding second message to be processed can be released, for example, the message reference of the second message to be processed is marked as 0, and the memory can be recovered by the characterization target program library.

In the embodiment of the invention, the first message to be processed is subjected to the first processing operation through the display channel and the first target processing function, and the second message to be processed is subjected to the second processing operation through the audio channel and the second target processing function, so that different processing operations can be performed on different types of messages to be processed by using different processing functions, and the diversity of message processing is improved.

Optionally, the embodiment of the invention can comprise the following steps:

Step 601, constructing a thread pool comprising at least two threads based on the number of processor cores corresponding to the client;

In the embodiment of the invention, the message processing module can acquire the number of processor cores of the terminal where the client is located when the client and the server establish the main channel initialization. The number of processor cores may be determined based on the number of processors the client contains and the number of processor cores of each processor. And setting at least two threads according to the terminal computing capacity of the terminal where the client is located, namely the number of processor cores, and constructing a thread pool. Wherein the first number of at least two threads may be the same as the second number of processor cores, i.e. the number of threads comprised in the thread pool may be equal to the number of processor cores, one for each processor core, i.e. each processor core is adapted to run a processing thread.

When the message processing module processes cloud desktop data, a thread with any idle thread state can be selected from a thread pool in advance to serve as a first target thread, a second target thread and a third target thread.

In the embodiment of the invention, the threads in the thread pool are configured based on the number of the processors, so that the balanced allocation of the processor resources is realized while the full utilization of the computing capacity of the terminal is ensured, and the message processing efficiency is improved to a certain extent.

Optionally, the first message to be processed carries image data and display update information. The first message to be processed carries display update information for indicating a display update area which needs to be updated in the image data, and the display update information can include coordinate information of the display update area. Based on the coordinate information of the display update area, an area and a position corresponding to the display update area in a base map (buffer) to be displayed can be determined.

Step 103 may comprise the steps of:

Step 701, based on a target timestamp corresponding to the target audio, obtaining display update information of target frame data matched with the target timestamp, and determining a display update area corresponding to the target frame data matched with the target timestamp.

In the embodiment of the invention, after the target audio and the target frame data are determined, the target frame data are adaptively displayed in a screen and the target audio is played according to the timestamp corresponding to the target audio. And acquiring a target time stamp corresponding to the target audio, and determining target frame data matched with the target time stamp based on the target time stamp to serve as a frame to be displayed. And acquiring display update information of a first message to be processed corresponding to the frame to be displayed, and determining a corresponding display update area in a base map (buffer) to be displayed based on the display update information. The target frame data matched with the target timestamp may be the same target frame data as the target timestamp, that is, the timestamp of the target frame data matched with the target timestamp is the same as the target timestamp corresponding to the target audio. The target frame data matched with the target time stamp is determined, so that the synchronization of the target audio and the target frame data in a screen of the client according to the actual playing progress of the sink machine can be ensured, and the display and playing synchronism and accuracy are improved.

In one possible implementation manner, there may be a situation that the target frame data obtained by the processing of the first target thread and the target audio obtained by the processing of the second target thread are not matched at the same time, and at this time, the target frame data matched with the target timestamp needs to be selected as an actual frame to be displayed according to the target timestamp of the target audio. And playing the frame to be displayed and the target audio in the client screen in a self-adaptive mode. It can be understood that, in general, the time stamp of the target frame data processed by the first target thread and the time stamp of the target audio processed by the second target thread are matched at the same time, and at this time, the target frame data and the target audio can be directly and adaptively played in the client screen at the same time, so as to ensure the consistency of the display and playing of the audio data and the display data.

Step 702, a display refresh signal is sent to a target display system, the display update area is adaptively displayed on a screen corresponding to the client, and the target audio is synchronously played.

In the embodiment of the invention, a display refreshing signal is sent to a target display system so as to call related components of the target display system to carry out self-adaptive display refreshing on a screen corresponding to a client side in a display updating area, and meanwhile, target audio is played. Illustratively, the display channel may send a display refresh signal to the GTK display system, invoking GTK WIDGET the associated components to refresh the display update area in the corresponding screen displayed on the client. The audio channel may call alsa to play the target audio and populate the target audio to the corresponding left and right channels.

In the embodiment of the invention, the target frame data matched with the target time stamp is determined based on the time stamp corresponding to the target audio, so that the display playing consistency of the audio data and the display data can be ensured to a certain extent.

Optionally, the embodiment of the invention can comprise the following steps:

step 801, after all the first messages to be processed contained in the display queue to be rendered are processed based on the first target thread, the first target thread is released back to a thread pool, and the first target thread is marked as an idle thread.

Step 802, after processing all the second pending messages contained in the audio queue to be played based on the second target thread, releasing the second target thread back to the thread pool, and marking the second target thread as an idle thread.

In the embodiment of the invention, after all the first messages to be processed in the display queue to be rendered are processed based on the first target thread, the first target thread can be released back to the thread pool, and the thread state of the first target thread is marked as idle. Similarly, after all the second pending messages in the audio queue to be played based on the second target thread are processed, the second target thread can be released back to the thread pool, and the thread state of the second target thread is marked as idle.

It can be understood that, in the case that the message splitting module and the message distributing module split and distribute the message to be processed to the display queue to be rendered and the audio queue to be played, the message processing module may select the idle thread as the first target thread and the second target thread again to process the message to be processed.

In the embodiment of the invention, under the condition that the first target thread and the second target thread process the information to be processed, the first target thread and the second target thread are released to the thread pool in time, so that subsequent call to the threads is facilitated, and the thread utilization efficiency is improved.

For example, fig. 2 provides a schematic architecture diagram of a client according to an embodiment of the present invention, and as shown in fig. 2, the client may include a message splitting module, a message distributing module, and a message processing module. And under the condition that the client receives the cloud desktop data, the cloud desktop data is shunted based on the message shunting module, and display data and audio data are screened out. Based on the target mapping table, determining processing functions corresponding to each display data and each audio data, distributing the packaged display data to a display queue to be rendered based on the message distribution module, and distributing the packaged audio data to an audio queue to be played. Correspondingly, the message count references are added to the display data in the display queue to be rendered and the audio data in the audio queue to be played. And the message processing module selects an idle thread from the thread pool as a first target thread, detects the quantity of display data in the display queue to be rendered based on the first target thread, acquires the display data and performs decoding synthesis processing based on a processing function corresponding to the display data when the quantity is greater than 0, obtains target frame data, and releases message counting references of the display data. And meanwhile, the message processing module selects an idle thread from the thread pool as a second target thread, detects the number of the audio data in the audio queue to be played based on the second target thread, acquires the audio data and performs decoding processing based on a processing function corresponding to the audio data when the number is greater than 0, obtains target audio, and releases the message counting reference of the audio data. And carrying out self-adaptive synchronous display and play on the target frame data and the target audio on a screen corresponding to the client. And after all the first to-be-processed messages in the to-be-rendered display queue are processed by the first target thread, releasing the first target thread back to the thread pool, and similarly, after all the second to-be-processed messages in the to-be-played audio queue are processed by the second target thread, releasing the second target thread back to the thread pool.

Because the scheme for processing cloud desktop data based on the co-Cheng Chuanhang can play the audio corresponding to the video frame only after the display data is decoded, when the processing process of the display data is complicated and takes a long time, for example, rendering and displaying time is long under the condition of full screen processing of the display image, the problems that sound cannot be played in time, delay and silence occur intermittently and the like are caused, and the playing effect is poor. According to the embodiment of the invention, the first message (display data), the second message (audio data) to be processed and the first message (other data except the display data and the audio data in the cloud desktop data) are respectively played and rendered by calling different threads based on the multi-processing core, and particularly, the message processing module synchronously processes the display data and the audio data by concurrency of the first target thread and the second target thread, the processing processes of the threads are independent, the screen display effect and the sound playing effect of the SPICE client are not influenced, and the rendering efficiency is further improved. Compared with the cloud desktop data processing based on the co-processing unit Cheng Chuanhang, the display data and the audio data are separated from the co-processing unit, namely the co-processing unit does not need to process the display data and the audio data, and the co-processing efficiency is improved to a certain extent.

Fig. 3 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present invention, which is applied to a client, and as shown in fig. 3, the apparatus may specifically include:

The first receiving module 901 is configured to receive cloud desktop data corresponding to a cloud desktop pushed by a server based on an independent computing environment simple protocol; the cloud desktop runs in a virtual machine of the server;

The first distributing module 902 is configured to, when receiving the cloud desktop data, sequentially and correspondingly distribute a message to be processed in the cloud desktop data, where the message meets a preset splitting type, to a display queue to be rendered and an audio queue to be played;

The first processing module 903 is configured to perform decoding synthesis processing on the first to-be-processed message in the to-be-rendered display queue based on a first target thread to obtain target frame data, and perform decoding processing on the second to-be-processed message in the to-be-played audio queue based on a second target thread to obtain target audio; the first target thread and the second target thread are executed in parallel, and the first target thread and the second target thread run respectively based on different processor cores in the client;

And the first display module 904 is configured to perform adaptive synchronous display and play on a screen corresponding to the client for the target frame data and the target audio, so as to display a remote cloud desktop corresponding to the cloud desktop data on the client.

The embodiment of the invention provides a message processing device, which sequentially and correspondingly distributes to-be-processed messages meeting a preset split type in cloud desktop data to a to-be-rendered display queue and an audio queue to be played under the condition that the cloud desktop data pushed by a server are received; decoding and synthesizing the first message to be processed in the display queue to be rendered based on the first target thread to obtain target frame data, and decoding and synthesizing the second message to be processed in the audio queue to be played based on the second target thread to obtain target audio; the first target thread and the second target thread are executed in parallel; and carrying out self-adaptive synchronous display and play on the target frame data and the target audio on a screen corresponding to the client. Therefore, cloud desktop data are pre-distributed based on the preset distribution type, and the screened information to be processed meeting the preset distribution type is further distributed to different processing queues (a display queue to be rendered and an audio queue to be played), so that the cloud desktop data in the different processing queues can be processed based on different threads, and the data processing efficiency is improved to a certain extent. Furthermore, decoding processing is carried out on the to-be-rendered display queue and the to-be-played audio queue respectively based on the first target thread and the second target thread in parallel to obtain target frame data and target audio, the first target thread and the second target thread operate based on different processor cores in the client respectively, and compared with processing of the display data and the audio data based on the co-Cheng Chuanhang in sequence, the processing flow of the first to-be-processed message and the second to-be-processed message is separated from the co-processing, and message processing is carried out based on the two threads in parallel, so that on the basis of fully utilizing the multi-core computing capability of the client, the message processing time is saved, and the message processing efficiency is improved. Meanwhile, as the first message to be processed and the second message to be processed are processed based on different threads respectively, the synchronization of the cloud desktop data processing of different types is improved, correspondingly, the display of the target frame data and the play synchronization of the target audio obtained based on the parallel thread synchronization processing are improved to a certain extent, and the display and play fluency and the audio-video display synchronization are improved.

Optionally, the preset offload type includes that a message type of the cloud desktop data is a first type or a second type; the first distribution module 902 includes:

The first determining module is used for determining whether the cloud desktop data meets a preset distribution type or not based on the message header of the cloud desktop data;

The second determining module is used for determining cloud desktop data with the message type of the first type or the second type as the message to be processed;

The first distributing sub-module is used for distributing a first message to be processed, the message type of which is the first type, to the to-be-rendered display queue, and distributing a second message to be processed, the message type of which is the second type, to the to-be-played audio queue.

Optionally, the apparatus may specifically include:

The third determining module is used for determining cloud desktop data which does not meet the preset distribution type as a first message;

The second distributing module is used for distributing the first message to a first message channel corresponding to the first message based on the message type of the first message aiming at any first message;

the first analyzing module is used for analyzing the first message in the first message channel based on the third target thread to obtain data to be applied under the condition that the preset processing sequence corresponding to the first message channel is reached.

Optionally, the apparatus may specifically include:

The second analyzing module is used for analyzing the sub-message type corresponding to any message to be processed based on the message content of the message to be processed;

And the fourth determining module is used for determining a processing function corresponding to the sub-message type based on the target mapping table and taking the processing function as a target processing function corresponding to the message to be processed.

Optionally, the first processing module 903 includes:

the first acquisition module is used for acquiring a first message to be processed from the display queue to be rendered;

The first processing sub-module is used for performing first processing operation on the first message to be processed based on the display channel and a first target processing function corresponding to the first message to be processed to obtain target frame data;

the second acquisition module is used for acquiring a second message to be processed from the audio queue to be played;

and the second processing sub-module is used for performing second processing operation on the second message to be processed based on the audio channel and a second target processing function corresponding to the second message to be processed to obtain target audio.

Optionally, the apparatus may specifically include:

The first construction module is used for constructing a thread pool comprising at least two threads based on the number of processors corresponding to the client;

the first number of the at least two threads is the same as the second number of the processors, and the first target thread and the second target thread are any idle threads in the thread pool.

Optionally, the first message to be processed carries display update information; the first display module 904 includes:

The third acquisition module is used for acquiring display update information of target frame data matched with the target time stamp based on the target time stamp corresponding to the target audio, and determining a display update area corresponding to the target frame data matched with the target time stamp;

And the first sending module is used for sending a display refreshing signal to a target display system, carrying out self-adaptive display on the display updating area on a screen corresponding to the client, and synchronously playing the target audio.

Optionally, the apparatus may specifically include:

The first release module is used for releasing the first target thread back to a thread pool after all the first to-be-processed messages contained in the to-be-rendered display queue are processed based on the first target thread, and marking the first target thread as an idle thread;

And the second release module is used for releasing the second target thread back to the thread pool after all second to-be-processed messages contained in the to-be-played audio queue are processed based on the second target thread, and marking the second target thread as an idle thread.

The present invention also provides an electronic device, see fig. 4, comprising: a processor 1001, a memory 1002 and a computer program 10021 stored on the memory and executable on the processor, which when executed implements the message processing method of the preceding embodiments.

The present invention also provides a readable storage medium which, when executed by a processor of an electronic device, enables the electronic device to perform the message processing method of the foregoing embodiments.

For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with the teachings herein. The required structure for a construction of such a system is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It will be appreciated that the teachings of the present invention described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be construed as reflecting the intention that: i.e., the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some or all of the components in a sorting device according to the present invention may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present invention may also be implemented as an apparatus or device program for performing part or all of the methods described herein. Such a program embodying the present invention may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as names.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

It should be noted that all actions for acquiring signals, information or data in the present application are performed in compliance with the corresponding data protection legislation policy of the location and obtaining the authorization granted by the owner of the corresponding device.

The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.

Claims

1. A message processing method, applied to a client, the method comprising:

2. The method of claim 1, wherein the preset offload type includes a first type or a second type of message type of the cloud desktop data; the distributing the to-be-processed messages meeting the preset distribution type in the cloud desktop data to the to-be-rendered display queue and the to-be-played audio queue sequentially and correspondingly comprises the following steps:

3. The method according to claim 2, wherein the method further comprises:

4. The method according to claim 1, wherein the method further comprises:

5. The method of claim 4, wherein the decoding and synthesizing the first message to be processed in the display queue to be rendered to obtain the target frame data includes:

Acquiring a second message to be processed from the audio queue to be played;

6. The method according to claim 1, wherein the method further comprises:

7. The method of claim 1, wherein the first message to be processed carries display update information; the adaptively synchronously displaying and playing the target frame data and the target audio on the screen corresponding to the client side includes:

8. The method according to claim 1, wherein the method further comprises:

9. A message processing apparatus for application to a client, the apparatus comprising:

10. An electronic device, comprising:

A processor, a memory and a computer program stored on the memory and executable on the processor, the processor implementing the message processing method according to any of claims 1-8 when the program is executed.

11. A readable storage medium, characterized in that instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the message processing method of any one of claims 1-8.