CN113885759B

CN113885759B - Notification message processing method, device, system and computer readable storage medium

Info

Publication number: CN113885759B
Application number: CN202110715005.2A
Authority: CN
Inventors: 秦国昊; 赵洋
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2021-06-25
Filing date: 2021-06-25
Publication date: 2022-08-26
Anticipated expiration: 2041-06-25
Also published as: CN113885759A; CN115599566A

Abstract

The embodiment of the application discloses a notification message processing method, a device, a system and a computer readable storage medium, relating to the technical field of terminals. The method comprises the following steps: monitoring a newly added notification message in a notification bar by the first device in the process of screen projection to the second device; if the first notification message is newly added to the notification bar, acquiring the message content of the first notification message from the monitoring information of the first notification message; and sending the message content to the second equipment, and displaying a second notification message associated with the message content of the first notification message in a message aggregation page according to the message content after the second equipment receives the message content. Therefore, if the second device switches the screen projection window of the first device to the background, the user can also sense that the notification bar of the first device receives a new notification message in time through the message aggregation page displayed by the second device, and therefore the notification bar message of the screen projection device can be prevented from being omitted by the user to a certain extent in the screen projection process.

Description

Notification message processing method, device, system and computer readable storage medium

Technical Field

The present application relates to the field of terminal technologies, and in particular, to a notification message processing method, device, system, and computer-readable storage medium.

Background

With the development of terminal technology and internet technology, the multi-screen cooperation technology is widely applied. The multi-screen cooperative technology refers to cooperative work of multiple screens, and includes that screen content of one device is projected to other devices to be displayed. For example, the content of the screen of the mobile phone is projected to the computer for displaying, so that the user can watch the content displayed on the screen of the mobile phone on the computer, thereby realizing the on-screen display between the mobile phone and the computer.

Currently, in the process of projecting a screen to a second device by a first device, a display interface of the second device may include a screen projecting window, where the screen projecting window displays screen content of the first device. However, in the process that the user uses the second device, if the user switches the screen projecting window in the second device to the background display, for example, the screen projecting window is minimized or other windows are switched to the foreground, so that the screen projecting window is blocked by other windows, and the like, under these circumstances, if the notification bar of the first device receives a new notification message, because the screen projecting window is not present in the display interface of the second device currently viewed or used by the user, the user cannot timely know that the notification bar of the first device receives the notification message, which may cause the notification bar message of the first device to be missed by the user. Therefore, a method for preventing the notification bar message of the screen projection device from being missed by the user in the screen projection process is needed.

Disclosure of Invention

The application provides a notification message processing method, device, system and computer readable storage medium, which can solve the problem that notification bar messages of screen projection equipment may be missed by a user in a screen projection process in the related art.

In order to achieve the purpose, the following technical scheme is adopted in the application:

in a first aspect, a notification message processing method is provided, and is applied to a first device, and the method includes:

monitoring a newly added notification message in the notification bar in the process of screen projection to the second device; if the first notification message is newly added to the notification bar, acquiring the message content of the first notification message from the monitoring information of the first notification message; and sending the message content of the first notification message to the second device, so that the second device displays a second notification message in the message aggregation page according to the message content after receiving the message content, wherein the second notification message is associated with the message content of the first notification message.

Therefore, in the process that the first device casts the screen to the second device, if the second device switches the screen casting window of the first device to the background, the user can also timely sense that the notification bar of the first device receives new notification messages through the message aggregation page displayed by the second device, and therefore the notification bar messages of the screen casting device during the screen casting process can be prevented from being omitted by the user to a certain extent.

As an example, the projection screen according to the embodiment of the present application may be a mirror projection screen. The mirror image screen projection refers to projecting screen content displayed by the first device onto the second device for displaying, so that the display screen of the first device and the screen projection window on the second device display the same content.

When the second device switches the screen-projecting window to the background, the user cannot timely perceive a new notification message received by the notification bar of the first device through the second device, and thus the user may miss the notification bar message of the first device. In the embodiment of the application, in order to avoid omission of the notification bar message of the first device in the process of screen projection from the first device to the second device, the first device may monitor a new notification message in the notification bar of the first device in the process of screen projection from the second device.

The first notification message is a notification message newly added to the notification bar, and the first notification message may be a notification message pushed to the notification bar by the first application, that is, the first application is a push application notified by the first message.

The monitoring information of the first notification message refers to information obtained by monitoring the first notification message newly added in the notification bar. The listening information of the first notification message may include message content, message subject and application identification of the first application of the first notification message, and may further include information such as ID and launch time of the first notification message.

In addition, if the first notification message is newly added to the notification bar, the application identifier of the first application may also be obtained from the monitoring information of the first notification message, and the application identifier of the first application is sent to the second device. The first application is an application for pushing the first message notification to the notification bar. The application identifier of the first application is used for uniquely identifying the first application, and may include one or more of an application name, an application package name, and an application icon of the first application.

As an example, if a first notification message is added to the notification bar, a message structure of the first notification message may be obtained from the monitored information of the first notification message, where the message structure includes at least message content, and may further include a message topic, an application identifier of the first application, and the like. Wherein the message subject may be a keyword of the message content of the first notification message or an identification of the contact sending the first notification message. The identifier of the contact may be a name or an ID of the contact, which is not limited in this embodiment of the application.

As an example, the first device includes a first screen projection module and a message monitoring service, and the message monitoring service may monitor a notification message newly added in the notification bar during a screen projection process from the first screen projection module to the second device.

As one example, the first device may further include a listening module, a first message conversion module, and a first transmission module. And if the message monitoring service monitors that the notification bar is added with the first notification message, calling back the monitoring module according to the monitoring information of the first notification message so as to send the monitoring information of the first notification message to the monitoring module. The monitoring module acquires monitoring information of the first notification message after being called back by the message monitoring service, and sends the monitoring information of the first notification message to the first message conversion module. The first message conversion module acquires the message content of the first notification message from the monitoring information of the first notification message and sends the message content of the first notification message to the first transmission module. The first transmission module transmits the message content to the second device.

As one example, the operation of obtaining the message structure of the first notification message may include: and acquiring the monitoring information of the first notification message, analyzing the monitoring information of the first notification message according to the analysis mode of the first data structure, and extracting the message structure of the first notification message from the analysis result.

It should be noted that the listening information of the first notification message is generally information of the first data structure. Therefore, after the monitoring information of the first notification message is acquired, the monitoring information of the first notification message needs to be analyzed according to the analysis mode of the first data structure, so as to obtain an analysis result, where the analysis result is the specific information content in the monitoring information of the first notification message. After the parsing result is obtained, the message structure of the required first notification message may be extracted from the parsing result. For example, the message content, the message subject, and the application identification of the first application of the first notification message may be extracted from the parsing result.

As an example, the interception information of the first notification message may be obtained by a message interception service. For example, the monitoring information of the first notification message is the monitoring information generated by the message monitoring service when the message monitoring service monitors that the first notification message is added to the notification bar.

In addition, after the message structure of the first notification message is acquired, in order to facilitate transmission of the message structure of the first notification message, the message structure of the first notification message may be encapsulated first. For example, the message structure of the first notification message may be first encapsulated into first encapsulation information in a first data format, and then the first encapsulation information may be converted into second encapsulation information in a second data format.

As one example, the first data format and the second data format may be a JavaScript object notation (JSON) format and a Byte (Byte) format, respectively. That is, the message structure of the first notification message may be encapsulated into the first encapsulation information in the JSON format, and then the first encapsulation information in the JSON format is converted into the second encapsulation information in the Byte format. It should be understood that the first data format and the second data format may also be other data formats, which is not limited in this embodiment of the application.

As an example, the first device may establish a communication connection with the second device before transmitting the message content to the second device; the message content is then transmitted to the second device over the communication connection.

Wherein the communication connection may be a socket connection or the like. In addition, the communication connection may be a long connection or a short connection, which is not limited in the embodiments of the present application.

The short connection is a communication connection established when data needs to be sent, and if the data needing to be sent does not exist within a preset time length after the data is sent, the connection is automatically interrupted. If the short connection is established, the connection is established once when data needs to be sent, and the connection is interrupted after the task is finished, so that the power consumption of the equipment can be reduced.

A long connection is a connection that can maintain the connection characteristics for a long time after a communication connection is established, i.e. without active interruption after the connection is established. If a long connection is established, the connection can be used for transmitting information all the time after the connection is established once, and the operation of frequently establishing the connection can be reduced.

As one example, in response to a screen-in operation, the first device establishes a long connection with the second device as the communication connection. Or, in response to acquiring the message content of the first notification message, the first device establishes a short connection with the second device as the communication connection.

In addition, after the message content is sent to the second device, the first device may further receive an application identifier of the first application sent by the second device, and start the first application according to the application identifier of the first application. The first application is an application for pushing a first notification message to the notification bar, and the application identifier of the first application is sent to the first device by the second device when the triggering operation of the displayed second notification message is detected.

That is, for the second notification message displayed on the message aggregation page of the second device, the user may perform a trigger operation on the second notification message to pull the screen projection window switched to the background, and send the application identifier of the first application to the first device to trigger the first device to start the first application.

After the first device starts the first application, the display screen of the first device can display the application interface of the first application. Accordingly, the screen projection window of the second device will also display the same content as the display screen of the first device, i.e. the second device also displays the application interface of the first application in the screen projection window. Therefore, the user can conveniently reply to the first notification message through the screen projection window of the second device, and the method for quickly replying the notification bar message of the screen projection device through the screen projection device is provided.

In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the method according to any one of the first aspect is implemented.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method according to any one of the above first aspects.

In a fifth aspect, embodiments of the present application provide a chip system, where the chip system includes a processor, and the processor is coupled with a memory, and executes a computer program stored in the memory to implement the method according to any one of the above first aspects. The chip system can be a single chip or a chip module consisting of a plurality of chips.

In a sixth aspect, embodiments of the present application provide a computer program product, which, when run on an electronic device, causes the electronic device to perform the method of any one of the above first aspects.

It is to be understood that, for the beneficial effects of the second aspect to the sixth aspect, reference may be made to the relevant description in the first aspect, and details are not described herein again.

In a seventh aspect, a notification message processing method is provided, and is applied to a second device, where the method includes:

receiving message content of a first notification message sent by first equipment in the process of screen projection by the first equipment, wherein the first notification message is a newly added notification message in a notification bar of the first equipment; and displaying a second notification message in the message aggregation page according to the message content, wherein the second notification message is associated with the message content of the first notification message.

In addition, after the second notification message is displayed in the message aggregation page according to the message content, the second device may further acquire a first trigger operation on the second notification message; responding to the first trigger operation, if the screen projecting window of the first equipment is switched to the background, switching the screen projecting window switched to the background to the foreground, and identifying the application of the first application to the first equipment.

The first application is an application for pushing a first notification message to the notification bar, the application identifier of the first application is used for starting the first application according to the application identifier of the first application after the first device receives the application identifier of the first application, and the screen projection window is used for displaying screen projection content of the first device.

As an example, the second device may further receive information such as an application identifier of the first application sent by the first device. For example, the second device may receive a message structure of a first notification message sent by the first device, where the message structure of the first notification message includes a message content, a message subject, an application identifier of the first application, and the like of the first notification message.

As an example, the second device may first receive second encapsulation information in a second data format sent by the first device, where the second encapsulation information includes message content of the first message. Then converting the second packaging information into first packaging information in a first data format; and decapsulating the first encapsulation information, and extracting the message content of the first notification message from the decapsulation result.

That is, if the message content of the first notification message received by the second device is sent in the form of the second encapsulation information, after receiving the second encapsulation information, the first device may convert the second encapsulation information into the first encapsulation information in the first data format, decapsulate the first encapsulation information, and extract the message content of the first notification message from the decapsulation result. For example, if the second encapsulation information is in the Byte format, the second encapsulation information in the Byte format may be converted into the first encapsulation information in the JSON format, and then the first encapsulation information in the JSON format may be decapsulated to obtain the message content of the first notification message.

As an example, after the second device receives the message content of the first notification message, the message content may be added to the message management list. The message management list is used for storing message contents of newly added notification messages in a notification bar of the first device, and the message aggregation page is used for displaying the notification messages corresponding to the message contents stored in the message management list. And then, refreshing the message aggregation page according to the message management list so as to display a second notification message on the refreshed message aggregation page.

Wherein the message management list is used for storing the message data of the second data structure. For example, the message management list may be a mapping (map) structure for storing message data of a key-value pair mapping structure. The mapping (map) may be hash mapping (hash map) or linked hash mapping (linked hash map), and the type of the mapping (map) in the embodiment of the present application is not limited.

As an example, taking the message data as the message data of the key-value pair mapping structure and the message management list as the map as an example, the message content may be encapsulated into the message data of the key-value pair mapping structure, and then the message data may be added to the message management list.

In addition, after receiving the message content of the first notification message, the second device may automatically refresh the message aggregation page according to the message content of the first notification message, and display the refreshed message aggregation page. Or first obtaining a second trigger operation for opening the message aggregation page, responding to the second trigger operation, refreshing the message aggregation page according to the message content of the first notification message, and displaying the refreshed message aggregation page.

As an example, after receiving the message content of the first notification message sent by the first device, the second device may also perform a new message alert. The new message reminder is used for indicating that a new notification message is added to the message aggregation page, so that a notification bar of the first equipment side of the user is reminded of receiving the new notification message, and the effect of timely notifying the user is achieved.

The form of the new message reminder may include various forms. For example, the form of the new message reminder may include displaying a new message identifier on an application icon of a specific application, or highlighting the application icon of the specific application in a task bar, or displaying a notification pop-up window, or the like, or may also include other reminding forms such as a sound reminder, and the form of the new message reminder is not limited in the embodiment of the present application.

As one example, the second device includes a second screen projection module, a second transmission module, a message management module, and a display module. The second device can receive the screen projection of the first device through the second screen projection module. In the process that the second screen projection module receives the screen projection of the first device, the second transmission module can receive the message content of the first notification message sent by the first device. The message management module may obtain the message content of the first notification message received by the second transmission module, and add the message content to the message management list. The display module may refresh the message aggregation page according to the message management list, and display the refreshed message aggregation page, where the refreshed message aggregation page includes the second notification message.

In addition, the second device may further include a second message conversion module. The second transmission module may receive second encapsulation information in a second data format sent by the first device, and send the second encapsulation information to the second message conversion module. Wherein the second encapsulation information includes message content of the first message. The second message conversion module may convert the second encapsulation information into first encapsulation information in the first data format, decapsulate the first encapsulation information, and extract the message content of the first notification message from the decapsulation result. Then, the message content of the first notification message is sent to the message management module.

In an eighth aspect, embodiments of the present application provide an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the method according to any one of the above seventh aspects is implemented.

In a ninth aspect, embodiments of the present application provide a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the method according to any one of the above seventh aspects.

In a tenth aspect, embodiments of the present application provide a chip system, where the chip system includes a processor, the processor is coupled with a memory, and the processor executes a computer program stored in the memory to implement the method according to any one of the above seventh aspects. The chip system can be a single chip or a chip module consisting of a plurality of chips.

In an eleventh aspect, embodiments of the present application provide a computer program product, which, when run on an electronic device, causes the electronic device to perform the method of any one of the above seventh aspects.

It is understood that the beneficial effects of the above eighth to eleventh aspects can be referred to the related description of the above seventh aspect, and are not repeated herein.

In a twelfth aspect, a message processing system is provided, which includes a first device and a second device, and the first device performs screen projection to the second device.

The first device is used for monitoring a newly added notification message in the notification bar in the process of screen projection to the second device; if the first notification message is newly added to the notification bar, acquiring the message content of the first notification message from the monitoring information of the first notification message; transmitting the message content to the second device;

the second equipment is used for receiving the message content sent by the first equipment in the screen projection process of the first equipment; and displaying a second notification message in the message aggregation page according to the message content, wherein the second notification message is associated with the message content of the first notification message.

The first device is further configured to obtain an application identifier of the first application from the monitoring information of the first notification message, and send the application identifier of the first application to the second device, where the first application is an application that pushes the first notification message to the notification bar;

and the second device is also used for receiving the application identification of the first application sent by the first device.

The second equipment is also used for acquiring a first trigger operation of the second notification message; responding to a first trigger operation, if a screen projecting window of first equipment is switched to a background, switching the screen projecting window switched to the background to a foreground, and sending an application identifier of a first application to the first equipment, wherein the screen projecting window is used for displaying screen projecting content of the first equipment;

the first device is further used for receiving the application identifier and starting the first application according to the application identifier.

And the second device is also used for carrying out new message reminding, and the new message reminding is used for indicating that a new notification message is added to the message aggregation page.

Drawings

FIG. 1 is a schematic diagram of a multi-collaboration system provided by an embodiment of the application;

FIG. 2 is a schematic diagram illustrating a screen projected from a mobile phone to a computer according to an embodiment of the present disclosure;

FIG. 3 is a diagram of a drop-down notification bar according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of another embodiment of the present application showing a mobile phone projecting a screen to a computer;

FIG. 5 is a schematic diagram of a screen projected from a mobile phone to a computer according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a screen projected from a mobile phone to a computer according to an embodiment of the present application;

fig. 7 is a block diagram of a software architecture of a first device according to an embodiment of the present disclosure;

fig. 8 is a block diagram of a software architecture of a second device according to an embodiment of the present disclosure;

FIG. 9 is a diagram of a software architecture of a notification message processing system according to an embodiment of the present application;

fig. 10 is a schematic diagram of establishing a long connection according to an embodiment of the present application;

fig. 11 is a schematic diagram of establishing a short connection according to an embodiment of the present application;

fig. 12 is a schematic flowchart of adding message data to a map according to an embodiment of the present application;

fig. 13 is a flowchart of a message processing method according to an embodiment of the present application;

FIG. 14 is a diagram illustrating a computer performing a new message alert according to an embodiment of the present application;

FIG. 15 is a logic diagram of a notification message processing process according to an embodiment of the present application;

fig. 16 is a hardware structure diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In addition, in the description of the embodiments of the present application, "a plurality" means two or more than two.

The following is an exemplary description of relevant matters that may be involved in embodiments of the present application.

(1) A multi-screen cooperative system. Referring to fig. 1, fig. 1 is a schematic diagram of a multi-coordination system according to an embodiment of the present disclosure. As shown in fig. 1, the multi-screen cooperative system includes a first device 10 and a second device 20, and the first device 10 and the second device 20 are connected through a wired network or a wireless network.

Wherein, the first device 10 can perform screen projection to the second device 20, and the second device 20 can display the screen projection content of the first device 10. That is, the first device 10 is a screen projection device, and the second device 20 is a screen projected device.

For example, the first device 10 may first establish a communication connection with the second device 20 and then project a screen to the second device 20 through the communication connection. The communication connection may be a wireless fidelity (WIFI) connection or a bluetooth connection, and of course, other communication connections may also be used, which is not limited in this embodiment of the present application.

The first device 10 or the second device 20 may be a mobile phone, a tablet computer, a wearable device, an in-vehicle device, an Augmented Reality (AR)/Virtual Reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a Personal Digital Assistant (PDA), or other electronic devices with a multi-screen cooperative function. In addition, the types of the first device 10 and the second device 20 may be the same or different, and the embodiments of the present application do not limit the types of the first device 10 and the second device 20.

As an example, during the process of the first device 10 projecting the screen to the second device 20, the second device 20 may display a projection window in the display interface, the projection window being used for displaying the projection content of the first device 10. For example, the screen projection window is used to display the screen content of the first device 10, that is, the display content in the screen projection window is the same as the content displayed on the screen of the first device 10.

Referring to fig. 2, taking the first device 10 as a mobile phone and the second device 20 as a computer as an example, in a process of projecting a screen to the computer by the mobile phone, the mobile phone displays the user interface 21, the computer displays the user interface 22, the user interface 22 includes a screen projection window 221, and a content displayed in the screen projection window 221 is the same as a display content of the user interface 21. Therefore, the user can watch the content displayed by the mobile phone through the screen projection window of the computer.

As an example, the first device 10 is installed with a multi-screen collaborative application, the second device 20 is also installed with the multi-screen collaborative application, the first device 10 may perform screen projection on the second device 20 through the installed multi-screen collaborative application, and the second device 20 may perform display on screen projection content of the first device 10 through the installed multi-screen collaborative application.

The multi-screen cooperative system can be applied to various scenes. For example, in a multi-person conference, one member may project the screen content of his mobile phone onto a projector to share the material on his mobile phone to other members for viewing. Or, in a home scene, the user can project the screen content of the mobile phone to the smart television for watching in the living room. The application scene of the multi-screen cooperative system is not limited in the embodiment of the application.

It should be noted that the screen projection generally includes a push screen projection and a mirror screen projection. The multi-screen cooperative system is mainly applied to a mirror image screen projection scene, and can be applied to a push screen projection scene.

The screen pushing and projecting means that a multimedia resource of one device is projected to other devices to be played, wherein the multimedia resource comprises audio and video resources. For example, a video on a mobile phone is put on a smart television for playing, so that the use of the mobile phone by a user is not influenced in the process, and the user can still operate the mobile phone. The mirror image screen projection means that the screen content of one device is projected to other devices for display, that is, the whole screen of one device is displayed on other devices in a mirror image mode. For example, the screen content of the mobile phone is put on the projector for displaying, and in the process, what content is displayed by the mobile phone and what content is displayed by the projector are both equivalent to displaying on the same screen.

In addition, during the process of the first device 10 projecting the screen to the second device 20, the second device 20 may also perform reverse control on the first device 10 according to the projection window in the second device 20.

(2) And a notification bar.

At present, in order to fully utilize the screen space of an intelligent terminal, a multi-level interface structure is mostly adopted for a display interface, wherein the most typical application is a notification bar, and the notification bar is also called a status bar and the like. The notification bar is used for displaying a shortcut menu and notification messages. The notification message refers to a notification type message, and may include a system notification message, a notification message pushed by a third-party application, and the like. For example, the message may include a short message sent by a contact pushed by a short message application, a communication message sent by a contact pushed by an instant messaging application, or some news messages pushed by a news application.

As one example, the notification message in the notification bar may be displayed via a notification card. For example, the notification bar includes a shortcut menu and at least one notification card for displaying a corresponding notification message. In addition, one notification card may display one or more notification messages of one application, i.e., notification messages of the same application may be displayed on one notification card. Further, the notification message may also be displayed on the screen in the form of a notification window.

Normally, the notification bar is in a hidden state, and when a user needs to view the notification bar, the intelligent terminal can be triggered to display the notification bar through a pull-down operation and the like. The pull-down operation refers to an operation of touching and sliding down from the top of the screen. It should be understood that, with the development of terminal technology, other ways may also be used to trigger the electronic device to display the notification bar, which is not limited in this application.

Referring to fig. 3, fig. 3 is a schematic diagram of a pull-down notification bar according to an embodiment of the present disclosure. As shown in fig. 3, when the mobile phone displays the user interface 301, if the user wants to view the notification bar, the user may perform a pull-down operation on the user interface 301 by touching from the top of the screen and sliding downward. The user interface 301 includes Huaqi video, sports health, weather, browser and settings applications, etc. In response to the user's pull-down operation, the cellular phone switches to the user interface 302, and the user interface 302 includes a notification bar including the shortcut menu 31 and the notification card 32. The notification card 32 displays a communication message pushed by the instant messaging application.

In conjunction with the multi-screen collaborative system, the first device 10 in the multi-screen collaborative system may be an intelligent terminal with a notification bar function. In the process of the first device 10 zooming into the second device 20, if the notification bar of the first device 10 receives a new notification message, the first device 10 may display a notification popup on the display interface, where the notification popup is used to display a notification message newly received by the notification bar. The notification popup window may be in the form of a notification card, and the like, which is not limited in the embodiment of the present application. In addition, the first device 10 may also alert the user that a new notification message is received by the notification bar by vibrating or ringing.

When the notification bar of the first device 10 receives a new notification message and displays a notification popup in the display interface, if the second device 20 displays a screen-projecting window, the screen-projecting window will also display the same content as the display screen of the first device 10, that is, the screen-projecting window of the second device 20 also displays the same notification popup. In this way, the user can timely obtain a new notification message received by the notification bar of the first device 10 through the screen-projecting window in the second device 10.

Referring to fig. 4, fig. 4 is a schematic view illustrating another mobile phone projecting a screen to a computer according to an embodiment of the present application. As shown in fig. 4, in the process of the mobile phone projecting a screen to the computer, if the notification bar of the mobile phone receives a communication message sent by a contact pushed by the instant messaging application, the mobile phone may display a user interface 401. The user interface 401 includes a notification popup 41 at the top, where the notification popup 41 includes an application icon and an application name of the instant messaging application, a contact name of the messaging message: "qinxx", and message content of a messaging message pushed by an instant messaging application: "conference from the conference room! ". Of course, the notification pop-up window 41 may also include other contents, which is not limited in this embodiment. At the same time, the computer displays a user interface 402. User interface 402 includes a drop window 42, and drop window 42 displays the same content as user interface 401, i.e., the same notification popup is also included within drop window 42. Therefore, the user can timely know the notification message newly received by the notification bar of the mobile phone through the notification popup window in the screen-projecting window 42 of the computer, and then timely check the notification message.

However, in the multi-screen coordination system shown in fig. 1, in the process that the first device 10 projects a screen to the second device 20, if the second device 20 switches the screen projecting window to the background according to the operation of the user, for example, the screen projecting window is minimized or other windows are switched to the foreground, so that the screen projecting window is blocked by other windows, in this case, if the notification bar of the first device 20 receives a new notification message, since the user cannot see the screen projecting window on the second device 20, when the user views or uses the second device 20, the user cannot timely sense that the notification bar of the first device 10 receives the new notification message through the second device 20, and further, the notification bar message of the first device 10 may be missed.

Referring to fig. 5, fig. 5 is a schematic view illustrating a screen projected from a mobile phone to a computer according to an embodiment of the present application. In the process that the mobile phone projects the screen to the computer, a user can watch the video by using the computer, and in order to avoid the shielding of the screen projecting window on the video, the user can minimize the screen projecting window. In this case, if the notification bar of the mobile phone receives a communication message sent by a contact pushed by the instant messaging application, the mobile phone displays the user interface 501, the user interface 501 includes a notification popup 51 at the top, and the notification popup 51 includes a communication message pushed by the instant messaging application. Meanwhile, the computer displays a user interface 502, wherein the user interface 502 comprises a video playing interface and a bottom task bar, and the task bar comprises a minimized icon 52 of a screen-projected window.

As can be seen from fig. 5, in the screen projection process, if the screen projection window in the screen projection device is switched to the background, the screen projection window will not be displayed in the display interface of the screen projection device, so that in the process of viewing or using the screen projection device by the user, if the notification bar of the screen projection device receives a new notification message, the user will not be able to know the notification bar message of the screen projection device in time through the screen projection device, which may cause the notification bar message of the screen projection device to be omitted by the user.

In the embodiment of the application, a message processing method is provided to avoid the problem that a notification bar message of a screen projection device may be omitted by a user when the screen projection window is switched to a background by the screen projection device. Based on the message processing method, in combination with the multi-screen collaboration system shown in fig. 1, in the process that the first device 10 is projecting the screen to the second device 20, if the notification bar of the first device 10 receives the first notification message, the first device 10 may trigger the second device 20 to display a second notification message associated with the message content of the first notification message in the message aggregation page. In this way, even if the second device 20 switches the screen-projecting window to the background, the user can timely know the notification bar message of the first device 10 based on the second message notification displayed in the message aggregation page of the specified application, thereby avoiding the notification bar message of the first device 10 from being missed by the user.

For example, the first device 10 may trigger the second device 20 to display the second notification message in the message aggregation page of the specified application. The designated application may be a system application or a third-party application, the third-party application may be a multi-screen cooperative application or an equipment manager application, and the like, and may also be other types of applications, and the application type of the designated application is not limited in the embodiment of the present application.

Referring to fig. 6, fig. 6 is a schematic diagram illustrating a screen shot by a mobile phone to a computer according to another embodiment of the present disclosure. In the process of projecting the screen to the computer by the mobile phone, the user can switch the screen projecting window in the computer to the background. In this case, if the notification bar of the mobile phone receives a communication message sent by a contact pushed by the instant messaging application, the mobile phone displays the user interface 601. The user interface 601 includes a notification popup 61 at the top, the notification popup 61 including a pushed messaging message of an instant messaging application. Meanwhile, the computer may display a message aggregation page 602 of the multi-screen collaborative application, where the message aggregation page 602 includes the notification popup 62, and the notification popup 62 also displays a communication message pushed by the instant messaging application and received by the notification bar of the mobile phone.

As shown in fig. 6, the message aggregation page 602 of the multi-screen collaborative application is a latest message page of the multi-screen collaborative application. It should be understood that the message aggregation page 602 may also be another page of the multi-screen collaborative application, which is not limited in this embodiment.

In addition, in order to facilitate the user to reply the message, after the second device 20 displays the second notification message on the message aggregation page of the designated application, if the trigger operation on the second notification message is detected, the second device 20 may further switch the screen projection window switched to the background to the foreground, and send the application identifier of the first application to the first device 10, so as to instruct the first device 10 to start the first application according to the application identifier of the first application, and display the application interface of the first application on the display interface.

The first application is an application that pushes a first notification message to the notification bar of the first device 10. The application identifier of the first application may be an application name, an Identity Document (ID), an application package name, and the like of the first application, which is defined in this embodiment of the present application.

In this way, the second device 20 can display the screen projection window on the foreground again, and the screen projection window displays the same content as the screen of the first device 10, that is, the screen projection window also displays the application interface of the first application. In this way, it is convenient for the user to reply to the first notification message in the screen-projection window of the second device 20, so as to provide a method for quickly replying to the notification bar message of the screen-projection device through the screen-projected device.

Referring to fig. 6, in the process of the mobile phone projecting the screen to the computer, if the notification bar of the mobile phone receives the communication message pushed by the instant messaging application, the computer may display a message aggregation page 602 of the multi-screen collaborative application, where the message aggregation page 602 includes a notification popup 62, and the notification popup 62 also displays the communication message pushed by the instant messaging application and received by the notification bar of the mobile phone. In this case, if the user wants to reply to the communication message through the computer, the user may click on the notification popup 62. In response to the click operation of the user, the computer can send the application identifier of the instant messaging application to the mobile phone. After receiving the application identifier, the mobile phone may start the instant messaging application according to the application identifier of the instant messaging application, and display an application interface 603 of the instant messaging application. In addition, in response to the click operation of the user, the computer may also switch the screen projection window switched to the background to the foreground again, so that the screen projection window 63 switched to the foreground again is displayed on the user interface 602. An application interface of the instant messaging application of the mobile phone is also displayed in the screen-projecting window 63, and the user can quickly reply to the communication message in the screen-projecting window 63.

Next, a software system architecture of the first device 10 and the second device 20 in the multi-screen collaborative system shown in fig. 1 will be described.

The software architecture of the first device 10 will be described first.

The software system of the first device 10 may employ a hierarchical architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes an Android system with a hierarchical architecture as an example, and exemplarily illustrates a software structure of the first device 10. Referring to fig. 7, fig. 7 is a block diagram of a software architecture of a first device 10 according to an embodiment of the present disclosure.

As shown in fig. 7, the layered architecture divides the software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. The layered architecture divides the Android system into four layers, namely an application layer, an application framework (framework) layer, an Android runtime (Android runtime) and system library, and a kernel layer from top to bottom. The application layer is also called an application layer, and the application framework layer is also called an application framework layer.

The application layer may include a series of application packages. As shown in fig. 7, the application package may include a first multi-screen collaborative application, a notification bar, an instant messaging application, a short message application, a camera application, a gallery application, a WLAN application, a bluetooth application, a music application, a video application, and the like.

The first multi-screen cooperative application may include a screen projection module, a monitoring module, a message conversion module, a transmission module, and the like. The screen projection module is used for projecting the screen content of the first device 10. The monitoring module is configured to monitor the notification message received by the notification bar of the first device 10. The message conversion module is used for converting the information monitored by the monitoring module, for example, analyzing and packaging the monitored information. The transmission module is used for transmitting the information converted by the message conversion module.

The application framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions.

As shown in fig. 7, the application framework layer may include a message listening service (notification list service), a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

The message listening service is used for listening to the change of the notification message in the notification bar, such as listening to a new notification message in the notification bar, or listening to a deletion of the notification message in the notification bar. In addition, if the message monitoring service monitors that the notification bar is added with a new notification message, the message monitoring service can also call back a monitoring module included in the first multi-screen cooperative application in the application layer to send the monitoring information to the monitoring module.

The window manager is used for managing window programs. The window manager can obtain the size of the display screen, and judge whether a notification bar, a locked screen, a captured screen and the like exist.

The content provider is used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

The view system includes visual controls such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide the communication functions of the first device 10. Such as management of call status (including connection, hangup, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and the like.

The notification manager enables applications to display notification information in the notification bar, can be used to convey notification-type messages, can disappear automatically after a short dwell, and does not require user interaction. Such as a notification manager used to inform download completion, message alerts, etc. The notification manager may also be a notification that appears in the form of a chart or scroll bar text at the top of the system in a notification bar, such as a notification of a background running application, or a notification that appears on the screen in a dialog window. For example, a text message is prompted in the notification bar, a prompt tone is given, the electronic device vibrates, an indicator light flashes, etc.

The Android Runtime comprises a core library and a virtual machine. The Android runtime is responsible for scheduling and managing an Android system.

The core library comprises two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the application framework layer run in a virtual machine. The virtual machine executes java files of the application layer and the application framework layer as binary files. The virtual machine is used for performing the functions of object life cycle management, stack management, thread management, safety and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. For example: surface managers (surface managers), Media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), and the like.

The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

The media library supports a variety of commonly used audio, video format playback and recording, and still image files, among others. The media library may support a variety of audio-video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, and the like.

The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

The software architecture of the second device 20 is described below.

The software system of the second device 20 may employ a hierarchical architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. Next, taking an Android system with a hierarchical architecture as an example, a software structure of the second device 20 is exemplarily described. Referring to fig. 8, fig. 8 is a block diagram of a software architecture of a second device 20 according to an embodiment of the present disclosure.

As shown in fig. 8, the layered architecture divides the software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. The layered architecture divides the Android system into four layers, namely an application layer, an application framework (framework) layer, an Android runtime (Android runtime) and system library, and a kernel layer from top to bottom.

The application layer may include a series of application packages. As shown in fig. 8, the application package may include applications such as a second multi-screen collaborative application, a notification bar, instant messaging, short messages, a camera, a gallery, WLAN, bluetooth, music, video, and the like.

The second multi-screen collaborative application may include a screen projection module, a transmission module, a message conversion module, a message management module, a display module, and the like. The screen projection module is used for receiving screen projection of other equipment. The transmission module is used for receiving the relevant information of the notification bar message sent by other devices in the process of screen projection to the second device 20. The message conversion module is used for converting the information received by the transmission module, for example, decapsulating the related information of the received notification bar message. The message management module is used for managing the related information of the notification bar message. The display module is used for displaying the corresponding notification message according to the related information of the notification bar message.

As shown in FIG. 8, the application framework layers may include a window manager, content provider, view system, phone manager, resource manager, notification manager, and the like.

The window manager is used for managing window programs. The window manager can obtain the size of the display screen, judge whether a notification bar exists, lock the screen, intercept the screen and the like.

The content provider is used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and answered, browsing history and bookmarks, phone books, etc.

The phone manager is used to provide the communication functions of the second device 20. Such as management of call status (including on, off, etc.).

The notification manager enables applications to display notification information in the notification bar, can be used to convey notification-type messages, can disappear automatically after a short dwell, and does not require user interaction. Such as a notification manager used to inform download completion, message alerts, etc. The notification manager may also be a notification that appears in the form of a chart or scroll bar text at the top of the system in a notification bar, such as a notification of a background running application, or a notification that appears on the screen in a dialog window. For example, a text message is prompted in the notification bar, a prompt tone is given, the electronic device vibrates, an indicator light flashes, and the like.

The application layer and the application framework layer run in a virtual machine. And executing java files of the application program layer and the application program framework layer into a binary file by the virtual machine. The virtual machine is used for performing the functions of object life cycle management, stack management, thread management, safety and exception management, garbage collection and the like.

The 2D graphics engine is a drawing engine for 2D drawing.

Next, taking the example that the mobile phone projects a screen to the computer, and both the mobile phone and the computer adopt an Android system with a layered architecture, a notification message processing process provided by the embodiment of the present application is described in detail.

Referring to fig. 9, fig. 9 is a software architecture diagram of a notification message processing system according to an embodiment of the present application, as shown in fig. 9, the notification message processing system includes a mobile phone 91 and a computer 92. The software system of the mobile phone 91 includes an application layer and an application framework layer. The software architecture of the computer 92 also includes an application layer and an application framework layer.

The application layer of the mobile phone 91 includes applications such as a notification bar and a first multi-screen cooperative application. The first multi-screen collaborative application includes a first screen projecting module 911, a monitoring module 912, a first message conversion module 913, and a first transmission module 914. The application framework layer of the handset 91 includes a message listening service 915.

The first screen projecting module 911 is configured to implement a multi-screen coordination function. In the embodiment of the present application, a listening module 912, a first message conversion module 913, and a first transmission module 914 are additionally added in the multi-screen collaborative application, so that the processing of the notification message in the notification bar is realized through these modules.

The application layer of the computer 92 includes a second multi-screen cooperative application, and the second multi-screen cooperative application includes a second screen projection module 921, a display module 922, a message management module 923, a second message conversion module 924, and a second transmission module 925.

The second screen projecting module 921 is configured to implement a multi-screen coordination function. In the embodiment of the present application, a display module 922, a message management module 923, a second message conversion module 924, and a second transmission module 925 are additionally added in the second multi-screen collaborative application, so that the notification message in the notification bar on the side of the mobile phone 91 is reminded through these modules.

It should be understood that, in the embodiment of the present application, the monitoring module 912, the first message conversion module 913, and the first transmission module 914 on the side of the mobile phone 91 are only described as being integrated in a multi-screen cooperative application of the mobile phone 91, but in other embodiments, these modules may also be integrated in other applications of the mobile phone 91, for example, integrated in a system application or a newly developed application besides an existing application, which is not limited in the embodiment of the present application.

Similarly, the embodiment of the present application is only described in an example of a multi-screen cooperative application in which the display module 922, the message management module 923, the second message conversion module 924, and the second transmission module 925 on the computer 92 side are integrated on the computer 92 side, and in other embodiments, these modules may also be integrated in other applications on the computer 92 side, for example, in a system application, a computer administrator application, or a newly developed application besides an existing application, which is not limited in this embodiment of the present application.

Next, referring to the software architecture diagram of the notification message processing system shown in fig. 9, a detailed description will be given of a processing flow of the notification message, taking as an example that the notification bar of the mobile phone 91 receives the first notification message from the first application in the process of the mobile phone 91 projecting the screen to the computer 92. As shown in fig. 9, the message processing flow includes the following steps:

step 1, the mobile phone 91 starts the first screen projecting module 911 in response to the screen projecting operation, the first screen projecting module 911 establishes a screen projecting connection with a second screen projecting module 922 in the computer 92 after starting, and sends a screen projecting image to the second screen projecting module 922 through the screen projecting connection.

The screen-projecting operation is an operation for triggering the mobile phone 91 to project a screen to the computer 92. The screen projection operation can be click operation, press operation, voice operation or gesture operation, and the like, and the multi-screen cooperative triggering operation is not limited in the embodiment of the application.

For example, when a user wants to screen the screen content of the mobile phone 91 to the computer 92 for displaying, the user may open the first multi-screen cooperative application of the mobile phone, and in response to an operation of the user, the mobile phone 91 starts the first multi-screen cooperative application, and scans the screen-projectable device through the first multi-screen cooperative application. The user can select the computer 92 from the scanned screen-projectable devices, and in response to the selection operation of the user, the mobile phone 91 can start the first screen-projectable module 911 in the first multi-screen collaborative application.

If the screen projecting operation of the user is detected, the mobile phone 91 may trigger the first screen projecting module 911 to start. After the first screen projecting module 911 is started, the first screen projecting module 911 can establish screen projecting connection with the second screen projecting module 922, send a screen projecting image to the second screen projecting module 922 through the screen projecting connection, and the computer 92 displays the received screen projecting image, so that screen projecting between the mobile phone 91 and the computer 92 is realized.

As one example, the first screen projecting module 911 includes an encoding unit and a first transmission unit. The encoding unit is configured to acquire an image displayed on a screen of the mobile phone 91, encode the acquired image, and obtain a screen projection image. The first transmission unit is used for establishing screen projection connection with a second transmission unit in the second screen projection module 921, and sending the screen projection image to the second transmission unit through the screen projection connection.

The second screen projection module 921 includes a second transmission unit and a decoding unit. The second transmission unit is used for receiving the screen projection image sent by the first transmission unit. The decoding unit is configured to decode the screen-projected image received by the second transmission unit, and trigger the computer 92 to display the decoded screen-projected image in the screen-projected window.

As an example, the second screen projecting module 921 may further include an inverse control unit. The counter control unit is used for detecting a trigger event of a user to the screen projection window, and converting the trigger event to the screen projection window into a trigger event to the display screen of the mobile phone 91. The second transmission unit is further configured to send the converted trigger event to the first transmission unit, and after receiving the trigger event, the first transmission unit may trigger the mobile phone 91 to respond to the trigger event, so as to implement reverse control of the mobile phone 91 by the computer 92.

Step 2, the handset 91 starts the monitoring module 912, and the monitoring module 912 binds the message monitoring service 915 of the application framework layer after starting.

After the monitoring module 912 is started, it can bind with the message monitoring service 915 of the application framework layer, so that the monitoring module 912 inherits the message monitoring service 915, and can obtain the monitoring information of the notification bar from the message monitoring service 915.

The message monitoring service 915 is used for monitoring the change of the notification message in the notification bar. For example, the newly added notification message in the notification bar is monitored to obtain the monitored information of the newly added notification message. The monitoring information may include message content of the newly added notification message and application identifiers such as an application identifier of the push application, which may include one or more of an application name, an application package name, and an application icon.

It should be understood that the listening information may also include other content, such as a message subject of the notification message. The message subject may be key information in the message content or an identification of the contact sending the first notification message, etc. The identifier of the contact may be a name or an ID of the contact, which is not limited in this embodiment of the application.

In addition, the message monitoring service 915 may also monitor the notification message with the notification bar removed to obtain the monitoring information of the removed notification message. The interception information includes the message content of the removed notification message and the application identification of the push application, and may further include the message subject of the removed notification message, and the like.

As one example, message listening service 915 may be a service provided by the system for listening for notification messages in a notification bar. For example, the message monitoring service 915 may be a notifiationlistenerservice class provided by the system.

After the monitoring module 912 binds the message monitoring service 915, the message monitoring service 915 may call back the monitoring module 912 when it detects that a notification message is newly added to the notification bar, and may send the monitoring information of the newly added notification message to the monitoring module 912 when the call back occurs. Therefore, through the binding message monitoring service 915, the monitoring module 912 can obtain the monitoring information of the notification message newly added in the notification bar.

As an example, the listening module 912 includes a binding interface through which the message listening service 915 of the application framework layer can be bound. The binding interface may be a start service method, etc.

It should be noted that the monitoring module 912 may be started after the first screen projecting module 911 is started, for example, after the first screen projecting module 911 is started, the first screen projecting module 911 triggers the monitoring module 912 to start; alternatively, the monitoring module 912 may also be started together with the first screen projecting module 911, for example, when the screen projecting operation is monitored, the monitoring module 912 and the first screen projecting module 911 may be triggered to start at the same time. The embodiment of the present application does not limit the starting sequence of the first screen projecting module 911 and the monitoring module 912.

In step 3, if the message monitoring service 915 monitors that the first notification message is added to the notification bar, the message monitoring service calls back the monitoring module 912 to send the monitoring information of the first notification message to the monitoring module 912.

The monitoring information of the first notification message may include the message content of the first notification message, the application package name, the application icon of the first application, the contact name of the first notification message, and the like. The first application pushes an application of the first notification message to the notification bar, such as an instant messaging application, a social application, or a short message application.

It should be understood that the monitoring information of the first notification message may also include other contents, and the specific content of the monitoring information of the first notification message is not limited in this embodiment of the application.

It should be noted that the monitored information of the first notification message monitored by the message monitoring service 915 is typically information of the first data structure, for example, the monitored information of the first notification message is a StatusBarNotification object of the first notification message. The StatusB arNotification object may include an ID, message content, and launch time of the first notification message, and an application package name, an application icon, and the like of the push application.

The callback parameter when the message monitoring service 915 callbacks the monitoring module 912 may include monitoring information of the first notification message, so that the monitoring information of the first notification message can be sent to the monitoring module 912 through the callback monitoring module 912, so that the monitoring module 912 obtains the monitoring information of the first notification message.

As an example, the monitoring module 912 includes a new message interface and a removed message interface, where the new message interface is used to obtain monitoring information of a notification message added to the notification bar, and the removed message interface is used to obtain monitoring information of a notification message removed from the notification bar. For example, the new message interface is an onnotifiationpost method, and the removed message interface is an onnotifiationpost method.

If the message monitoring service 915 monitors that the notification bar has a new first notification message, the message monitoring service 915 may call back the new message interface of the monitoring module 912, so that the monitoring module 912 can obtain the monitoring information of the first notification message through the new message interface.

In addition, if the message monitoring service 915 detects that the notification bar removes the third notification message, the message monitoring service 915 may call back the removal message interface of the monitoring module 912, so that the monitoring module 913 interface obtains the monitoring information of the third notification message through the removal message interface. For example, if the user clicks the third notification message in the notification bar, or performs a deletion operation on the third notification message in the notification bar, the message monitoring service 915 may monitor that the notification bar removes the third notification message.

Step 4, the monitoring module 912 sends the monitoring information of the first notification message to the first message conversion module 913, and the first message conversion module 913 parses the monitoring information of the first notification message, extracts the message structure of the first notification message from the parsing result, and encapsulates the message structure of the first notification message to obtain the encapsulation information.

The interception information of the first notification message is typically interception information of a first data structure. The message conversion module 913 may analyze the monitored information of the first notification message according to the analysis manner of the first data structure, and the obtained analysis result is the specific information content in the monitored information of the first data structure. Then, a message structure of the required first notification message may be extracted from the parsing result.

In addition, in order to facilitate transmission of the message structure of the first notification message, the message structure of the first notification message may be further encapsulated into encapsulation information that facilitates transmission. For example, the message structure of the first notification message is encapsulated into information in a JSON (JavaScript Object Notation) format for facilitating transmission, so as to obtain the first encapsulated information. And then converting the first encapsulation information into the information in the Byte format to obtain second encapsulation information.

It should be understood that, when the message structure of the first notification message is encapsulated, the first notification message may also be encapsulated into information in other data formats that are convenient for transmission, and the encapsulation form of the message structure of the first notification message is not limited in this embodiment of the application.

As an example, referring to fig. 9, the first message conversion module 913 includes a parsing unit and an encapsulation unit. The analyzing unit is used for analyzing the monitoring information of the first notification message, extracting the message structure of the first notification message from the analysis result, and sending the message structure of the first notification message to the packaging unit. The encapsulation unit is used for encapsulating the message structure of the first notification message to obtain encapsulation information.

In step 5, the first message conversion module 913 sends the encapsulation information to the first transmission module 914.

For example, the first message conversion module 913 may send the second encapsulation information in the Byte format to the first transmission module 914.

It should be understood that the embodiment of the present application is only described by taking as an example that the first message conversion module 913 extracts the message structure of the first notification message from the parsing result, and sends the message structure of the first notification message to the first transmission module 914, and in other embodiments, other forms of information may also be extracted from the parsing result and sent to the first transmission module 914, for example, directly extracting the message content of the first notification message, etc.

The first transmission module 914 establishes a communication connection with a second transmission module 925 in the computer 92, step 6.

Wherein the communication connection is an information transmission channel established between the cellular phone 91 and the computer 92 for transmitting information related to the notification message in the notification bar on the side of the cellular phone 91. The communication connection may be a socket connection, and certainly, may also be a communication connection in other forms, which is not limited in this embodiment of the present application.

Taking the communication connection as a socket connection as an example, when the socket connection is established, the first transmission module 914 may serve as a server side, the second transmission module 925 serves as a client side, and the client side initiates a connection request to the server side to establish the socket connection between the client side and the server side.

Referring to fig. 9, the first transmission module 914 may include a server-side connection interface, a server-side end connection interface, a server-side transmission interface, and a server-side reception interface. The second transmission module 925 includes a client connection interface, a client termination connection interface, a client reception interface, and a client transmission interface. The first transmission module 914 may establish a socket connection with the client connection interface in the second transmission module 925 through the server connection interface. In addition, when disconnection is required, the first transmission module 914 may also disconnect the established socket connection with the client end connection in the second transmission module 925 through the server end connection interface.

For example, the server-side connection interface, the server-side end connection interface, the server-side transmission interface, and the server-side reception interface in the first transmission module 914 may be a socketServerConnect (), a socketserverdisconnectconnect (), a socketserversend (), and a socketServerRecv (), respectively. The client connection interface, the client termination connection interface, the client reception interface, and the client transmission interface in the second transmission module 925 may be a socketClientConnect (), a socketClientrD isConnect (), a socketClientSend (), and a socketClientRecv (), respectively.

It should be noted that, in the embodiment of the present application, the first transmission module 914 triggers to establish the communication connection with the second transmission module 925 when there is information to be transmitted, and in other embodiments, the first transmission module may also establish the communication connection with the second transmission module 925 at other times, for example, when responding to a screen-casting operation, establish the communication connection with the second transmission module 925. For example, a communication connection with the second transmission module 925 is established when the first screen projecting module 911 is started or after the first screen projecting module 911 is started.

As an example, in the embodiment of the present application, the first transmission module 914 may establish a communication connection with the second transmission module 925 in the following two ways:

the first implementation mode comprises the following steps: the first transfer module 914 establishes a short connection with the second transfer module 925 as the communication connection when there is information to be transferred. The short connection is a communication connection established when data needs to be sent, and the connection is automatically disconnected if no data needs to be sent within a specific time period after the data is sent.

In the case of establishing a short connection, the connection is established each time there is data to be sent and the task is disconnected upon completion. Thus, the time for maintaining the connection can be reduced, and the connection consumption can be further reduced.

Referring to fig. 10, fig. 10 is a schematic diagram illustrating establishing a long connection according to an embodiment of the present application. As shown in fig. 10, after a Transmission Control Protocol (TCP)/user data packet protocol (UDP) connection is established between a server-side connection interface of a first transmission module and a client-side connection interface of a second transmission module, a server-side transmission interface of the first transmission module may transmit data to a client-side reception interface of the second transmission module, and then receive a data Acknowledgement Character (ACK) returned by the second transmission module, where the data ACK is used to indicate that a receiving end receives the data. And if the data transmission is not needed within the specific time length of receiving the data ACK, disconnecting the connection. And then, if data needs to be sent, connection needs to be reestablished, the data is sent through the established connection, and if the data needing to be sent does not exist within a specific time length after the data ACK is received, the connection is disconnected again.

The second implementation mode comprises the following steps: the first transmission module 914 establishes a long connection with the second transmission module 925 as the communication connection in response to the screen-in operation. A long connection is a connection that can maintain the connection characteristics for a long time after a communication connection is established, i.e. without active interruption after the connection is established.

As an example, the first transmission module 914 may be initialized at the start-up of the first screen projecting module 911 or after the first screen projecting module 911, during which a long connection is established with the second transmission module 925.

In the case of a long connection being established, the connection can be used to transmit data all the time after a connection is established, without the need to interrupt the connection after the data has been transmitted. In this way, the operation of frequently establishing a connection can be reduced.

Referring to fig. 11, fig. 11 is a schematic diagram illustrating establishing a short connection according to an embodiment of the present application. As shown in fig. 10, after a Transmission Control Protocol (TCP)/User Datagram Protocol (UDP) connection is established between the server-side connection interface of the first transmission module and the client-side connection interface of the second transmission module, the server-side transmission interface of the first transmission module may transmit data to the client-side reception interface of the second transmission module, receive data ACK returned by the second transmission module, and maintain the connection characteristics. That is, if data transmission is not required within a specific time period of receiving the data ACK, disconnection is not required. Thereafter, if there is data to be sent, the data may be sent again through the established connection.

In step 7, the first transmission module 914 sends the encapsulated information to the second transmission module 925 via the communication connection.

Referring to fig. 8, taking the communication connection as a socket connection as an example, the first transmission module 914 may send the encapsulation information to the client receiving interface in the second transmission module 925 through the server sending interface, so that the second transmission module 925 can obtain the encapsulation information sent by the first transmission module 914 through the client receiving interface.

For example, the first transmission module 914 may send the second encapsulation information in the Byte format to the second pass-through module 925 through the communication connection.

Step 8, after receiving the encapsulation information, the second transmission module 925 sends the encapsulation information to the second message conversion module 924, and the second message conversion module 924 decapsulates the encapsulation information to obtain a message structure of the first notification message, and encapsulates the message structure of the first notification message into message data of the second data structure.

The second data structure refers to a data structure that can be managed by the message management module 923. For example, the second data structure may be a key-value mapping structure. Wherein, key is the application identifier in the message structure, and value is the message structure.

Since the message management module 923 is configured to manage the information of the second data structure, in order to facilitate management of the message management module 923, after decapsulating the encapsulated information to obtain the message structure of the first notification message, the message structure of the first notification message may be encapsulated into message data of the second data structure, and then the message data of the second data structure may be sent to the message management module 923.

As an example, assuming that the data format of the encapsulation information is the Byte format, the encapsulation information in the Byte format may be converted into the encapsulation information in the JSON format, and then the encapsulation information in the JSON format may be decapsulated to obtain the message structure of the first notification message. In addition, if the second data structure is a key-value mapping structure, after the encapsulation information is decapsulated to obtain the message structure of the first notification message, the application identifier in the message structure of the first notification message may be extracted as a key, the message structure of the first notification message may be taken as a value, and the key-value mapping structure formed by the key and the value may be determined as the message data of the second data structure.

As an example, referring to fig. 9, the second message conversion module 924 includes a decapsulating unit and an encapsulating unit, where the decapsulating unit is configured to decapsulate the received encapsulated information to obtain a message structure of the first notification message. The encapsulating unit is configured to encapsulate a message structure of the first notification message into message data of a second data structure, so that the message management module 923 manages the message data.

In step 9, the second message conversion module 924 sends the message data of the second data structure to the message management module 923, and the message management module 923 adds the message data to the message management list.

The message management module 923 is configured to manage the message data of the second data structure to provide the message data to the display module 922 for display. The management includes addition, deletion, combination and the like of message data.

The message management module 923 maintains a message management list to which the message management module 923 may add in order message data of the second data structure sent by the second message converting module 924.

Wherein the message management list is used for storing the message data of the second data structure. For example, the message management list may be a map (map) structure for storing message data of a key-value pair map structure. The mapping (map) may be hash mapping (hash map) or linked hash mapping (linked hash map), and the type of the mapping (map) in the embodiment of the present application is not limited.

Taking the message data with the key-value pair mapping structure as the message data and the message management list as the map as examples, after receiving the message data with the key-value pair mapping structure sent by the second message converting module 924, the message managing module 923 may add the message data to the map with the application identifier in the message data as the key.

Referring to fig. 12, fig. 12 is a schematic flowchart illustrating a process of adding message data according to an embodiment of the present application. As shown in fig. 12, in the process of adding the message data to the map, the second message management module 923 may further obtain a key of the message data as the target application identifier, and then detect whether there is message data whose key is the target application identifier in the message management list. And if the message data of which the key is the target application identifier exists in the message management list, updating the message data of which the key is the target application identifier according to the message data. And if the message data of which the key is the target application identifier does not exist in the message management list, adding the message data of which the key is the target application identifier into the map.

As an example, the message management module 923 includes an add message (add notification) interface for adding message data to the message management list and a delete message (delete notification) interface for deleting message data from the message management list. After receiving the message data sent by the second message conversion module 924, the second message conversion module 924 may add the message data to the message management list by adding a message interface.

In step 10, the message management module 923 triggers the display module 922 to refresh the message aggregation page, so as to display the second notification message indicated by the message data in the message aggregation page.

The second notification message is associated with the message content of the first notification message, and may include the message content of the first notification message or other information obtained by processing the message content of the first notification message. In addition, a message subject of the first notification message and one or more of an application name, an application package name, and an application icon of the first application may also be included.

The second notification message may be obtained by aggregating data content in the message data. For example, the message content and the message subject in the message data, and the application name, the application package name, and the application icon of the first application are aggregated into a notification message as a second notification message according to a preset rule.

The message aggregation page corresponds to the message management list, and the message aggregation page is used for displaying notification messages corresponding to the message data stored in the message management list. After the message data of the first notification message is newly added to the message management list, the message management module 923 may trigger the display module 922 to refresh the message aggregation page, so that the display module 922 displays the second notification message indicated by the message data of the first notification message in the refreshed message aggregation page.

As one example, the display module 922 includes a display unit, and the display module 922 may refresh the message aggregation page through the display unit to display the second notification message in the refreshed message aggregation page.

In step 11, the display module 922 triggers a new message alert after refreshing the message aggregation page.

The new message reminder is used for reminding the message aggregation page that a new notification message is added, and further reminding the user that a notification bar on the mobile phone side has the new notification message, so that the effect of timely notifying the user is achieved.

The form of the new message alert may include a variety of forms. For example, the form of the new message reminder may include displaying a new message identifier on an application icon of the second multi-screen collaborative application, or highlighting and displaying an application icon of the multi-screen application in a task bar, or displaying a notification popup window, or the like, or may also include other reminding forms such as a voice reminder, and the form of the new message reminder is not limited in the embodiment of the present application.

The new message identifier is used for indicating that a new notification message is added to the message aggregation page of the second multi-screen collaborative application, and may be a highlighted dot or the number of new notification messages. The form of the highlighted icon may include flashing display, highlighting display, and the like, which is not limited in the embodiment of the present application. The notification popup may include a second notification message or other alerting information, such as a notification popup may be displayed at the top of the display screen that includes the second notification message.

As an example, the display module 922 includes a reminding unit, and the display module 922 can remind a new message through the reminding unit.

In step 12, if the display module 922 detects the trigger operation on the second notification message, the screen projection window is switched to the foreground, and the trigger message management module 923 deletes the message data of the first notification message from the message management list.

The display module 922 may further detect a trigger operation of the user on the second notification message after the second notification message is displayed in the message aggregation page. The trigger action on the second notification message is indicative of having viewed the second notification message. The trigger operation may be a click operation, a press operation, a voice operation, a gesture operation, or the like, which is not limited in this embodiment of the present application.

Since the message management list maintained by the message management module 923 corresponds to the message aggregation page displayed by the display module 922, after the user triggers the second notification message in the message aggregation page, the message management module 923 indicates that the second notification message is viewed, and in this case, the message management module 923 may delete the message data of the first notification message from the message management list.

In addition, after the message management module 923 can delete the message data of the first notification message from the message management list, the message management module 923 may further trigger the display module 922 to refresh the message aggregation page to delete the second notification message from the message aggregation page.

Wherein, to throw the screen window and switch to the foreground and show can include: and restoring the minimized screen projection window, or displaying the screen projection window which is blocked by other windows on the upper layer of other windows.

As an example, referring to fig. 7, the message management module 923 includes a delete message interface by which message data of the first notification message may be deleted from the message management list.

In step 13, the message management module 923 sends the application package name of the first application to the second transmission module 925, where the first application is a push application of the first notification message.

As an example, the key value of the message data of the first notification message is an application package name of the first application. The message management module 923 may obtain in advance a key value of the message data of the first notification message to be deleted before deleting the message data of the first notification message from the message management list, obtain an application package name of the first application, and then send the application package name of the first application to the second transmission module 925.

In step 14, the second transmission module 925 sends the application package name of the first application to the first transmission module 914 of the handset 91 through the communication connection.

Taking the communication connection as a socket connection as an example, the second transmission module 925 may send the application packet name of the first application to a server-side receiving interface in the first transmission module 914 through the client-side sending interface, so that the first transmission module 914 may obtain the application packet name of the first application sent by the second transmission module 925 through the server-side receiving interface.

In step 15, the first transmission module 914 receives the application package name of the first application, and triggers the mobile phone 91 to start the first application according to the application package name of the first application.

After the mobile phone 91 starts the first application, the screen of the mobile phone 91 may display an application interface of the first application. Accordingly, the screen-casting window of the computer 92 will also display the same content as the screen of the cell phone 91, i.e. the computer 92 also displays the application interface of the first application in the screen-casting window. Therefore, the first notification message can be conveniently replied in the screen projection window by the user, and the technical scheme for rapidly replying the notification bar message of the screen projection equipment through the screen projection equipment is provided.

The application interface of the first application may be a main interface of the first application, or may also be a message interface including the first notification message or a message reply interface of the first notification message, and the like, which is not limited in this embodiment of the application.

It should be understood that, in this embodiment, the application package name of the first application is sent by the second transmission module 925 to the first transmission module 914, for example, to instruct the cell phone 91 to start the first application, while in other embodiments, the second transmission module 925 sends other application identifiers of the first application to the first transmission module 914, for example, sends the name or ID of the first application, and this is not limited in this embodiment of the present application.

Next, taking an example that a notification bar of a mobile phone receives a new notification message in a process of a screen being projected to a computer by the mobile phone, a detailed description is given of a message processing method provided in an embodiment of the present application with reference to the drawings.

Referring to fig. 13, fig. 13 is a flowchart of a message processing method according to an embodiment of the present application, and as shown in fig. 13, the method includes the following steps:

step 1301: and the mobile phone projects a screen to the computer.

The user can perform screen projection operation on the mobile phone to project the mobile phone to the computer. The mobile phone can detect the screen projection operation of the user and respond to the screen projection operation to project the screen to the computer.

The screen projection operation may be a click operation, a press operation, a voice operation, a gesture operation, or the like, which is not limited in the embodiment of the present application. For example, the user may perform a screen-casting operation through a multi-screen collaborative application installed on a mobile phone.

As an example, when a user wants to screen a screen content of a mobile phone onto a computer for display, the user may first open a multi-screen collaborative application of the mobile phone. And responding to the opening operation of the user, starting the multi-screen cooperative application by the mobile phone, and scanning the screen-projectable equipment through the multi-screen cooperative application. And the user selects a computer which wants to screen from the scanned screen-projection equipment, and the mobile phone can project the screen to the computer in response to the selection operation of the user.

As an example, the projection screen according to the embodiment of the present application may be a mirror projection screen. The mirror image screen projection means that screen contents displayed by the mobile phone are projected to the computer for displaying, so that the screen projection windows of the mobile phone and the computer display the same contents.

As an example, referring to fig. 9, the mobile phone may start the first screen projecting module 911 in response to a screen projecting operation of the user. After being started, the first screen projecting module 911 establishes screen projecting connection with a second screen projecting module 922 in the computer 92, and sends a screen projecting image to the second screen projecting module 922 through the screen projecting connection. The computer 92 receives the projected image through the second projection module 922, and displays the projected image. For a specific implementation process, reference may be made to the related description in step 1 in the embodiment of fig. 9, and details of the embodiment of the present application are not described herein again.

Step 1302: and the computer displays a screen projection window in the display interface, and the screen projection window is used for displaying the screen projection content of the mobile phone.

After the computer accepts the screen projection of the mobile phone, a screen projection window can be created on a display interface of the computer, and the screen projection window is used for displaying screen projection contents of the mobile phone, such as displaying contents displayed by a screen of the mobile phone.

Step 1303: and the computer responds to the switching operation of switching the screen projection window to the background and switches the screen projection window to the background.

When a user wants to switch the screen projection window to the background in the process of watching or using the computer, the user can execute corresponding switching operation. And responding to the switching operation of the user, the computer can switch the screen projection window to the background, so that the display interface of the computer does not display the screen projection window any more.

The switching operation may include an operation of minimizing a screen projection window, an operation of switching another window to a foreground, and the like. Accordingly, switching the screen projection window to the background may include: and minimizing the projection window for display, or switching other windows to the foreground, and the like.

Step 1304: and monitoring a newly added notification message in the notification bar by the mobile phone in the screen projection process of the computer.

Under the condition that the screen projection window is switched to the background by the computer, a user cannot sense new notification messages received by the notification bar of the mobile phone in time through the computer, and further the user may miss the notification bar messages of the mobile phone. In the embodiment of the application, in order to avoid missing the notification bar message of the mobile phone in the process of screen-casting the mobile phone to the computer, the mobile phone can monitor the newly added notification message in the notification bar in the process of screen-casting the mobile phone to the computer.

As an example, referring to fig. 9, an application framework layer of the mobile phone includes a message monitoring service 915, and the mobile phone can monitor a notification message newly added in the notification bar through the message monitoring service 915. For a specific implementation process, reference may be made to the related description in step 2, and details of the embodiment of the present application are not described herein.

If the mobile phone monitors that the notification bar is added with the first notification message, the message content of the first notification message can be acquired from the monitoring information of the first notification message, so that the message content of the first notification message is sent to the computer device. Of course, other contents, such as a message subject of the first notification message and an application identifier of the push application, may also be obtained from the monitoring information of the first notification message. The embodiment of the present application will be described with an example of obtaining a message structure of a first notification message from monitoring information of the first notification message, where the message structure includes a message content, a message topic, and an application identifier of a push application.

Step 1305: if the mobile phone monitors that the notification bar is added with the first notification message, the message structure of the first notification message is obtained from the monitoring information of the first notification message, and the message structure comprises message content, message subject and application identification of the first application.

The application identifier of the first application is used for uniquely identifying the first application, and may include one or more of an application name, an application package name, and an application icon of the first application. The message topic may be a keyword of the message content of the first notification message or an identification of the contact sending the first notification message. The identifier of the contact may be a name or an ID of the contact, which is not limited in this embodiment of the application.

It should be understood that the embodiment of the present application is only described by taking as an example that the message structure of the first notification message includes message content, message subject and application identifier of the first application, and in other embodiments, the message structure of the first notification message includes at least message content. That is, the first notification message may not include the message subject or the application identifier of the first application, or may include other related information of the first notification message, except for the message content, which is not limited in this embodiment of the present application.

The monitoring information of the first notification message refers to information obtained by monitoring the first notification message newly added in the notification bar. The listening information of the first notification message may include the message content, the message subject and the application identification of the first application of the first notification message, and may further include information such as an ID and an origination time of the first notification message.

It should be noted that the listening information of the first notification message is typically information of the first data structure. Therefore, after the monitoring information of the first notification message is acquired, the monitoring information of the first notification message needs to be analyzed according to the analysis mode of the first data structure, so as to obtain an analysis result, where the analysis result is the specific information content in the monitoring information of the first notification message. After the parsing result is obtained, the message structure of the required first notification message may be extracted from the parsing result. For example, the message content, the message subject, and the application identification of the first application of the first notification message may be extracted from the parsing result.

As an example, referring to fig. 9, the application framework layer of the handset includes a message listening service 915 and the application layer of the handset includes a listening module 913. The mobile phone may start the monitoring module 913 when initiating a screen projection with the computer, and the monitoring module 913 binds the message monitoring service 915 after starting. The message monitoring service 915 monitors the newly added notification message in the notification bar, and if the first notification message is monitored in the notification bar, the monitoring module 913 may be called back to send the monitoring information of the first notification message to the monitoring module 913. The specific process may refer to the related descriptions in step 2 to step 3 in the embodiment of fig. 9, and the embodiment of the present application is not described herein again.

In addition, after the message structure of the first notification message is acquired, in order to facilitate transmission of the message structure of the first notification message, the message structure of the first notification message may be encapsulated first. For example, the message structure of the first notification message may be encapsulated into the first encapsulation information in the first data format, and then the first encapsulation information may be converted into the second encapsulation information in the second data format.

As one example, the first data format and the second data format may be a JavaScript object notation (JSON) format and a Byte (Byte) format, respectively. That is, the message structure of the first notification message may be encapsulated into the first encapsulation information in the JSON format, and then the first encapsulation information in the JSON format is converted into the second encapsulation information in the Byte format.

It should be understood that the first data format and the second data format may also be other data formats, which is not limited in this embodiment of the application.

Referring to fig. 9, the application layer of the mobile phone includes a first message conversion module 913, and the message structure of the first notification message may be encapsulated by the first message conversion module 913. The specific process may refer to the related description in step 4 in the embodiment of fig. 9, and the embodiment of the present application is not described herein again.

Step 1306: the mobile phone establishes communication connection with the computer.

After the message structure of the first notification message is acquired, the message structure of the first notification message may be used as data to be sent. In the embodiment of the application, the mobile phone can establish communication connection with the computer when data needs to be sent, so that the data is sent to the computer through the established communication connection.

Wherein the communication connection may be a socket connection. In addition, the communication connection may be a long connection or a short connection, which is not limited in the embodiments of the present application.

It should be noted that, in the embodiment of the present application, the communication connection with the computer is only established when there is data to be sent, but of course, the communication connection may also be established at other times, for example, after the screen projection operation is acquired, the communication connection is established in response to the screen projection operation. The embodiment of the present application does not limit the timing of establishing the communication connection.

As an example, referring to fig. 9, the application layer of the mobile phone includes a first transmission module 914, the application layer of the computer includes a second transmission module 925, and the first transmission module 914 can establish a communication connection with the second transmission module 925. For a specific implementation process, reference may be made to the related description of step 6, and details of the embodiment of the present application are not described herein again.

Step 1309: the handset sends a message structure of a first notification message to the computer through the communication connection.

As an example, if the message structure of the first notification message is encapsulated before the message structure of the first notification message is sent, in which case the encapsulated message structure may be sent to the computer over the communication connection. For example, the second encapsulated message is sent to the computer over the communication connection.

As an example, referring to fig. 9, the application layer of the handset includes a first transmission module 914, the application layer of the computer includes a second transmission module 925, and the first transmission module 914 may send the message structure of the first notification message to the second transmission module 925. For a specific implementation process, reference may be made to the related description of step 9, and details of the embodiment of the present application are not described herein again.

After receiving the message content of the first notification message sent by the mobile phone, the computer device may display a second notification message associated with the message content of the first notification message in the message aggregation page according to the message content.

The message aggregation page can be used for displaying notification messages associated with the notification messages received by the notification bar of the mobile phone. For example, the message aggregation page may be a message aggregation page of a specific application, and the specific application may be a system application or a third-party application. The embodiment of the present application will be described with the message aggregation page as a message aggregation page of a multi-screen collaborative application as an example.

Step 1308: the computer receives a message structure of the first notification message through the communication connection, refreshes a message aggregation page of the multi-screen cooperative application according to the message structure of the first notification message, wherein the refreshed message aggregation page comprises a second notification message, and the second notification message is associated with the message content of the first notification message.

Wherein the second notification message is a notification message associated with the message content of the first notification message, and may indicate the first notification message. For example, the second notification message may include the message content of the first notification message or other information obtained by processing the message content of the first notification message, such as key information obtained by extracting a keyword from the message content of the first notification message. The second notification message may further include a message subject of the first notification message, an application identification of the first application, and the like.

For example, the second notification message may be a notification message obtained by aggregating a message structure of the first notification message, that is, the computer may aggregate the message structure into the notification message to obtain the second notification message, and then display the second notification message in a message aggregation page of the specified application.

It should be understood that the first notification message and the second notification message may be displayed in the same form, or may have a certain difference, and this is not limited in this embodiment of the application.

According to the embodiment of the application, the newly received notification message in the notification bar at the mobile phone side can be displayed in the message aggregation page, so that a user can timely acquire the notification bar message at the mobile phone side through the message aggregation page of the multi-screen collaborative application, and the omission of the notification bar message at the mobile phone side is avoided.

It should be noted that the multi-screen collaborative application may also expand the function of displaying the notification message newly received in the notification bar on the basis of the existing page, and use the page after the function expansion as the message aggregation page. Or, the multi-screen collaborative application may add a page dedicated to displaying a newly received notification message in the notification bar on the mobile phone side, and use the added page as a message aggregation page. That is, the message aggregation page may be an existing page of the multi-screen collaborative application, or may also be a page newly extended by the multi-screen collaborative application, which is not limited in this embodiment of the present application.

For example, the multi-screen collaborative application is provided with a recent message page for displaying a recent state of the multi-screen collaborative application. In the embodiment of the application, the page can be functionally expanded, so that the page can also display a newly received notification message in a notification bar on the mobile phone side.

In this embodiment of the present application, the second notification message may be displayed in the message aggregation page according to the message structure of the first notification message, and an action of displaying the second notification message in the message aggregation page according to the message structure of the first notification message may be triggered by a user or may be automatically triggered by a computer, which is not limited in this embodiment of the present application.

In one possible implementation, after receiving the message structure of the first notification message, the computer may display, in response to receiving the message structure of the first notification message, the second notification message in a message aggregation page of the multi-screen collaborative application directly according to the message structure of the first message. That is, the operation of displaying the second notification message in the message aggregation page may be automatically triggered after receiving the message structure of the first notification message without user triggering.

For example, after receiving the message structure of the first notification message, the computer may trigger the multi-screen collaborative application to display a message aggregation page, where the message aggregation page includes the second notification message.

In another possible implementation manner, after receiving the message structure of the first notification message, when the triggering operation on the message aggregation page is detected, the computer may display the second notification message in the message aggregation page according to the message structure of the first message. That is, the operation of displaying the second notification message in the message aggregation page may be triggered by the user.

For example, after receiving the message structure of the first notification message, the computer may obtain an opening operation of the user on the message aggregation page, and in response to the opening operation, display the message aggregation page, where the message aggregation page includes the second notification message.

In addition, the computer may add the message structure of the first notification message to the message management list after receiving the message structure of the first notification message. The message management list is used for storing a message structure of the newly added notification message in the notification bar of the mobile phone. The message aggregation page corresponds to the message management list and is used for displaying the notification messages corresponding to the message structures stored in the message management list. After the message structure of the first notification message is added to the message management list, the message aggregation page may be triggered to refresh, so that the second notification message corresponding to the message structure of the first notification message is displayed in the refreshed message aggregation page.

As an example, referring to fig. 9, a multi-screen collaborative application of a computer includes a message management module 923 and a display module 922, the message management module 923 maintains a message management list, and a message structure of a first notification message may be added to the message management list. After adding the message structure of the first notification message to the message management list, the message management module 923 may trigger the display module 922 to refresh the message aggregation page, so as to display a second notification message corresponding to the message structure of the first notification message in the refreshed message aggregation page. For a specific implementation process, reference may be made to the related descriptions of step 9 and step 13, which is not described herein again in this embodiment of the present application.

In addition, if the message structure of the first notification message received by the computer is sent in the form of the second encapsulation information, after receiving the second encapsulation information, the computer may convert the second encapsulation information into the first encapsulation information in the first data format, and then decapsulate the first encapsulation information, thereby obtaining the message structure of the first notification message. For example, if the second encapsulation information is in the Byte format, the second encapsulation information in the Byte format may be converted into the first encapsulation information in the JSON format, and then the first encapsulation information in the JSON format may be decapsulated to obtain the message structure of the first notification message.

As an example, referring to fig. 9, the multi-screen collaborative application of the computer includes a second message conversion module 924, and the computer may convert the second package information through the second message conversion module 924 to obtain a message structure of the first notification message. The specific implementation process may refer to the related description of step 8, which is not limited in this embodiment of the present application.

Step 1309: and the computer carries out new message reminding, and the new message reminding is used for reminding the message aggregation page that a new notification message is added.

The new message prompt is used for prompting the user that a new notification message is added to the message aggregation page of the specified application, and further prompting the user that a notification bar on the mobile phone side receives the new notification message, so that the effect of timely notifying the user is achieved.

After sensing the new message reminding of the computer, the user can open the message aggregation page of the specified application in time to view the notification messages in the message aggregation page. In response to the user's open operation, the computer device may display a message aggregation page for the specified application, the message aggregation page including the second notification message.

The form of the new message alert may include a variety of forms. For example, the form of the new message reminder may include displaying a new message identifier on an application icon of the multi-screen collaborative application, or highlighting the application icon of the multi-screen application in a task bar, or displaying a notification popup window, or the like, or may also include other reminding forms such as a voice reminder, and the form of the new message reminder is not limited in the embodiment of the present application.

The new message identifier is used for indicating that a new notification message is added to the message aggregation page of the multi-screen collaborative application, and may be a highlighted dot or the number of new notification messages. The form of the highlighted icon may include flashing display, highlighting display, and the like, which is not limited in the embodiment of the present application. The notification popup may include a second notification message or other alerting information, such as a notification popup may be displayed at the top of the display screen that includes the second notification message.

Referring to fig. 14, fig. 14 is a schematic diagram illustrating a computer performing a new message alert according to an embodiment of the present disclosure. As shown in fig. 14, if the notification bar of the mobile phone receives a communication message pushed by the instant messaging application, the mobile phone displays a user interface 1401, the user interface 1401 includes a notification popup 141 at the top, and the notification popup 141 includes a communication message pushed by the instant messaging application. Meanwhile, the computer may display a user interface 1402, where the user interface 1402 also includes a video display interface and a taskbar, and the taskbar includes an application icon 142 of the multi-screen cooperative application and a window icon of a screen-casting window switched to the background. A new message identifier, which is the number of new notification messages added to the message aggregation page of "1", is displayed on the application icon 142 of the multi-screen collaborative application.

Step 1310: and the computer responds to the triggering operation of the second notification message, switches the screen projection window switched to the background to the foreground, and sends the application package name of the first application to the mobile phone through the communication connection.

In the embodiment of the application, for the second notification message displayed on the message aggregation page of the multi-screen cooperative application, a user can perform a trigger operation on the second notification message to pull up the screen-casting window switched to the background, and send the application identifier of the first application to the mobile phone to trigger the mobile phone to start the first application.

The triggering operation may be a click operation, a press operation, a voice operation, a gesture operation, or the like, which is not limited in this embodiment of the application. For example, the trigger operation may be a click operation on the second notification message.

The application package name of the first application is used for indicating the mobile phone to start the first application according to the application package name of the first application. It should be understood that, in this embodiment of the present application, the name of the application package of the first application is sent to the computer to illustrate by taking the example of instructing the mobile phone to start the first application, and in other embodiments, other application identifiers of the first application may also be sent to the mobile phone, for example, the name or the ID of the first application is sent, which is not limited in this embodiment of the present application.

Wherein, switching the screen projection window switched to the background to the foreground comprises: if the screen projection window is minimized, the minimized screen projection window can be restored; if the screen projecting window is shielded by other windows, the screen projecting window can be displayed above the other windows.

As one example, in response to the triggering operation, the computer may also delete the message structure of the first notification message from the message management list.

For example, referring to fig. 9, the computer includes a display module 922, a message management module 923 and a second transmission module 925, and if the display module 922 detects a trigger operation on the second notification message, the message management module 923 may be triggered to delete message data of the first notification message from the message management list, the display module 922 is triggered to switch the screen-on window to the foreground, and the second transmission module 925 is triggered to send an application packet name of the first application to the first transmission module 914. The specific implementation process may refer to the related descriptions in step 12 to step 14, which is not limited in this application.

Step 1311: the mobile phone receives the application identifier through communication connection, and starts the first application according to the application identifier.

After the mobile phone starts the first application, the display screen of the mobile phone can display the application interface of the first application. Correspondingly, the screen projection window of the computer will also display the same content as the screen of the mobile phone, i.e. the computer also displays the application interface of the first application in the screen projection window. Therefore, the user can conveniently reply the first notification message in the screen projection window, and the method for quickly replying the notification bar message of the screen projection equipment through the screen projection equipment is provided.

The application interface of the first application displayed on the display screen of the mobile phone may be a main interface of the first application, or may also be a message interface including the first notification message, or a message reply interface of the first notification message, which is not limited in this embodiment of the application.

In addition, after the computer device displays the application interface of the first application in the screen projection window, the computer device can also obtain a trigger event of the user to the screen projection window, and in response to the trigger event to the screen projection window, the computer device can also send the trigger event to the mobile phone so that the mobile phone can respond to the trigger event, thereby realizing reverse control of the mobile phone through the computer device and facilitating the user to reply the notification message received by the mobile phone side through the screen projection window of the computer.

The trigger event of the screen projection window can be a trigger event of an application interface of a first application in the screen projection window. For example, the trigger event may be an event that opens a message reply interface of the first notification message, or an event that replies to the first notification message, or the like.

Referring to fig. 15, fig. 15 is a logic diagram illustrating a notification message processing procedure according to an embodiment of the present application. As shown in fig. 15, the handset can start the message listening service in the background during the process of the handset being dropped on the computer. If the notification bar of the mobile phone receives the notification message pushed by the wechat application, the mobile phone can obtain the notification message newly added to the notification bar through the message monitoring service, and then analyze the notification message to obtain the application package name, the application icon, and the message content and the message theme of the notification message of the pushed application, namely the wechat application. And then, packaging the analyzed information into a JSON object, and converting the JSON object into packaging information in a Byte format. Then, a socket connection with the computer is established, and the encapsulation information is sent to the computer through the socket connection.

After receiving the encapsulation information, the computer converts the encapsulation information in the Byte format into a JSON object, and then decapsulates the JSON object to obtain a message structure of the notification message. And then, aggregating the message structure of the notification message into the notification message, and displaying the aggregated notification message and performing new message reminding on a message aggregation page of the multi-screen cooperative application. And then, if the computer equipment detects the click operation on the notification message displayed on the message aggregation page, switching the screen projection window switched to the background to the foreground, and sending the application package name of the WeChat application to the mobile phone. After receiving the application package name, the mobile phone can start the WeChat application according to the package name, so that screen throwing windows of the mobile phone and the computer both display application interfaces of the WeChat application, and a user can reply to a notification message pushed by the WeChat application conveniently according to the screen throwing window of the computer.

In the embodiment of the application, in the process of screen projection from the mobile phone to the computer, the mobile phone can monitor the newly added notification message in the notification bar. If the mobile phone monitors that the first notification message is added to the notification bar, the message structure of the first notification message can be obtained, and the message structure includes the message content of the first notification message. Then, the message structure of the first notification message is sent to the computer, and a second notification message is displayed in the message aggregation page of the specified application according to the message structure by the computer, wherein the second notification message comprises the message content of the first notification message. Therefore, under the condition that the screen projection window is switched to the background by the computer, the user can timely sense that the notification bar of the mobile phone receives new notification messages through the message aggregation page displayed by the computer, and therefore the notification bar messages of the screen projection equipment are prevented from being omitted by the user to a certain extent in the screen projection process.

Next, a hardware configuration of the electronic device according to the present application will be described.

Fig. 16 is a hardware structure diagram of an electronic device 100 according to an embodiment of the present application, where the electronic device 100 may be the first device 10 or the second device 20 shown in fig. 1. The electronic device may be, but is not limited to, a mobile phone, a tablet computer, a smart large screen (also referred to as a smart television), or a wearable device.

As shown in fig. 16, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, a Subscriber Identity Module (SIM) card interface 195, and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It is to be understood that the illustrated structure of the embodiment of the present invention does not specifically limit the electronic device 100. In other embodiments of the present application, electronic device 100 may include more or fewer components than shown, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units, such as: the processor 110 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.

The controller may be, among other things, a neural center and a command center of the electronic device 100. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 110, thereby increasing the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (I2C) interface, an integrated circuit built-in audio (I2S) interface, a Pulse Code Modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a general-purpose input/output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and/or a Universal Serial Bus (USB) interface, etc.

The I2C interface is a bi-directional synchronous serial bus that includes a serial data line (SDA) and a Serial Clock Line (SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces, respectively. For example: the processor 110 may be coupled to the touch sensor 180K via an I2C interface, such that the processor 110 and the touch sensor 180K communicate via an I2C bus interface to implement the touch functionality of the electronic device 100.

The I2S interface may be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 via an I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 may communicate audio signals to the wireless communication module 160 via the I2S interface, enabling answering of calls via a bluetooth headset.

The PCM interface may also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, audio module 170 and wireless communication module 160 may be coupled by a PCM bus interface. In some embodiments, the audio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to implement a function of answering a call through a bluetooth headset. Both the I2S interface and the PCM interface may be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communications. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example: the processor 110 communicates with a bluetooth module in the wireless communication module 160 through a UART interface to implement a bluetooth function. In some embodiments, the audio module 170 may transmit the audio signal to the wireless communication module 160 through a UART interface, so as to realize the function of playing music through a bluetooth headset.

MIPI interfaces may be used to connect processor 110 with peripheral devices such as display screen 194, camera 193, and the like. The MIPI interface includes a Camera Serial Interface (CSI), a Display Serial Interface (DSI), and the like. In some embodiments, processor 110 and camera 193 communicate through a CSI interface to implement the capture functionality of electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal and may also be configured as a data signal. In some embodiments, a GPIO interface may be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface may also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

The USB interface 130 is an interface conforming to the USB standard specification, and may be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transmit data between the electronic device 100 and a peripheral device. And the earphone can also be used for connecting an earphone and playing audio through the earphone. The interface may also be used to connect other terminal devices, such as AR devices and the like.

It should be understood that the connection relationship between the modules according to the embodiment of the present invention is only illustrative and is not limited to the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners or a combination of multiple interface connection manners in the above embodiments.

The charging management module 140 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the electronic device 100. The charging management module 140 may also supply power to the terminal device through the power management module 141 while charging the battery 142.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140 and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 may also be used to monitor parameters such as battery capacity, battery cycle count, battery state of health (leakage, impedance), etc. In some other embodiments, the power management module 141 may also be disposed in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be disposed in the same device.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution including 2G/3G/4G/5G wireless communication applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module 150 may receive the electromagnetic wave from the antenna 1, filter, amplify, etc. the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modem processor, and convert the signal into electromagnetic wave through the antenna 1 to radiate the electromagnetic wave. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then passes the demodulated low frequency baseband signal to a baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional modules, independent of the processor 110.

The wireless communication module 160 may provide a solution for wireless communication applied to the electronic device 100, including Wireless Local Area Networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), bluetooth (bluetooth, BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), and the like. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification on the signal, and convert the signal into electromagnetic waves through the antenna 2 to radiate the electromagnetic waves.

In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 and antenna 2 is coupled to wireless communication module 160 so that electronic device 100 can communicate with networks and other devices through wireless communication techniques. The wireless communication technology may include global system for mobile communications (GSM), General Packet Radio Service (GPRS), code division multiple access (code division multiple access, CDMA), Wideband Code Division Multiple Access (WCDMA), time-division code division multiple access (time-division code division multiple access, TD-SCDMA), Long Term Evolution (LTE), LTE, BT, GNSS, WLAN, NFC, FM, and/or IR technologies, etc. The GNSS may include a Global Positioning System (GPS), a global navigation satellite system (GLONASS), a beidou satellite navigation system (BDS), a quasi-zenith satellite system (QZSS), and/or a Satellite Based Augmentation System (SBAS).

The electronic device 100 implements display functions via the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 194 is used to display images, video, and the like. The display screen 194 includes a display panel. The display panel may adopt a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), a flexible light-emitting diode (FLED), a miniature, a Micro-oeld, a quantum dot light-emitting diode (QLED), and the like. In some embodiments, the electronic device 100 may include 1 or N display screens 194, with N being a positive integer greater than 1.

The electronic device 100 may implement a shooting function through the ISP, the camera 193, the video codec, the GPU, the display 194, the application processor, and the like.

The ISP is used to process the data fed back by the camera 193. For example, when a photo is taken, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converting into an image visible to naked eyes. The ISP can also carry out algorithm optimization on the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in camera 193.

The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In some embodiments, the electronic device 100 may include 1 or N cameras 193, N being a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process digital image signals and other digital signals. For example, when the electronic device 100 selects a frequency bin, the digital signal processor is used to perform fourier transform or the like on the frequency bin energy.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 may play or record video in a variety of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

The NPU is a neural-network (NN) computing processor that processes input information quickly by using a biological neural network structure, for example, by using a transfer mode between neurons of a human brain, and can also learn by itself continuously. Applications such as intelligent recognition of the electronic device 100 can be implemented by the NPU, for example: image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to extend the memory capability of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music, video, etc. are saved in an external memory card.

The internal memory 121 may be used to store computer-executable program code, which includes instructions. The processor 110 executes various functional applications of the electronic device 100 and data processing by executing instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, and the like) required by at least one function, and the like. The storage data area may store data (such as audio data, phone book, etc.) created during use of the electronic device 100, and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), and the like.

The electronic device 100 may implement audio functions via the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. Such as music playing, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also called a "horn", is used to convert the audio electrical signal into a sound signal. The electronic apparatus 100 can listen to music through the speaker 170A or listen to a hands-free call.

The receiver 170B, also called "earpiece", is used to convert the electrical audio signal into an acoustic signal. When the electronic apparatus 100 receives a call or voice information, it can receive voice by placing the receiver 170B close to the ear of the person.

The microphone 170C, also referred to as a "microphone," is used to convert sound signals into electrical signals. When making a call or transmitting voice information, the user can input a voice signal to the microphone 170C by speaking the user's mouth near the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C to achieve a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may further include three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and perform directional recording.

The headphone interface 170D is used to connect a wired headphone. The headset interface 170D may be the USB interface 130, or may be an Open Mobile Terminal Platform (OMTP) standard interface of 3.5mm, or a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

The pressure sensor 180A is used for sensing a pressure signal, and converting the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. The pressure sensor 180A can be of a wide variety, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor may be a sensor comprising at least two parallel plates having an electrically conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the strength of the pressure from the change in capacitance. When a touch operation is applied to the display screen 194, the electronic apparatus 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic apparatus 100 may also calculate the touched position from the detection signal of the pressure sensor 180A. In some embodiments, the touch operations that are applied to the same touch position but different touch operation intensities may correspond to different operation instructions. For example: and when the touch operation with the touch operation intensity smaller than the first pressure threshold value acts on the short message application icon, executing an instruction for viewing the short message. And when the touch operation with the touch operation intensity larger than or equal to the first pressure threshold value acts on the short message application icon, executing an instruction of newly building the short message.

The gyro sensor 180B may be used to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of electronic device 100 about three axes (i.e., the x, y, and z axes) may be determined by gyroscope sensor 180B. The gyro sensor 180B may be used for photographing anti-shake. For example, when the shutter is pressed, the gyro sensor 180B detects a shake angle of the electronic device 100, calculates a distance to be compensated for by the lens module according to the shake angle, and allows the lens to counteract the shake of the electronic device 100 through a reverse movement, thereby achieving anti-shake. The gyroscope sensor 180B may also be used for navigation, somatosensory gaming scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, electronic device 100 calculates altitude from barometric pressure values measured by barometric pressure sensor 180C to assist in positioning and navigation.

The magnetic sensor 180D includes a hall sensor. The electronic device 100 may detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 may detect the opening and closing of the flip according to the magnetic sensor 180D. And then according to the opening and closing state of the leather sheath or the opening and closing state of the flip cover, the automatic unlocking of the flip cover is set.

The acceleration sensor 180E may detect the magnitude of acceleration of the electronic device 100 in various directions (typically three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. The method can also be used for recognizing the posture of the electronic equipment 100, and is applied to horizontal and vertical screen switching, pedometers and other applications.

A distance sensor 180F for measuring a distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, taking a picture of a scene, the electronic device 100 may utilize the distance sensor 180F to range to achieve fast focus.

The proximity light sensor 180G may include, for example, a Light Emitting Diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light to the outside through the light emitting diode. The electronic device 100 detects infrared reflected light from nearby objects using a photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 may determine that there are no objects near the electronic device 100. The electronic device 100 can utilize the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear for talking, so as to automatically turn off the screen to achieve the purpose of saving power. The proximity light sensor 180G may also be used in a holster mode, a pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense the ambient light level. Electronic device 100 may adaptively adjust the brightness of display screen 194 based on the perceived ambient light level. The ambient light sensor 180L may also be used to automatically adjust the white balance when taking a picture. The ambient light sensor 180L may also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

The fingerprint sensor 180H is used to collect a fingerprint. The electronic device 100 may utilize the collected fingerprint characteristics to unlock a fingerprint, access an application lock, photograph a fingerprint, answer an incoming call with a fingerprint, and so on.

The temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 implements a temperature processing strategy using the temperature detected by temperature sensor 180J. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 performs a reduction in performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, the electronic device 100 heats the battery 142 when the temperature is below another threshold to avoid the low temperature causing the electronic device 100 to shut down abnormally. In other embodiments, when the temperature is lower than a further threshold, the electronic device 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

The touch sensor 180K is also referred to as a "touch panel". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is used to detect a touch operation applied thereto or nearby. The touch sensor can communicate the detected touch operation to the application processor to determine the touch event type. Visual output associated with the touch operation may be provided through the display screen 194. In other embodiments, the touch sensor 180K may be disposed on a surface of the electronic device 100, different from the position of the display screen 194.

The bone conduction sensor 180M can acquire a vibration signal. In some embodiments, the bone conduction sensor 180M may acquire a vibration signal of the human vocal part vibrating the bone mass. The bone conduction sensor 180M may also contact the human pulse to receive the blood pressure pulsation signal. In some embodiments, the bone conduction sensor 180M may also be disposed in a headset, integrated into a bone conduction headset. The audio module 170 may analyze a voice signal based on the vibration signal of the bone mass vibrated by the sound part acquired by the bone conduction sensor 180M, so as to implement a voice function. The application processor can analyze heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

The keys 190 include a power-on key, a volume key, and the like. The keys 190 may be mechanical keys. Or may be touch keys. The electronic apparatus 100 may receive a key input, and generate a key signal input related to user setting and function control of the electronic apparatus 100.

The motor 191 may generate a vibration cue. The motor 191 may be used for incoming call vibration cues, as well as for touch vibration feedback. For example, touch operations applied to different applications (e.g., photographing, audio playing, etc.) may correspond to different vibration feedback effects. The motor 191 may also respond to different vibration feedback effects for touch operations applied to different areas of the display screen 194. Different application scenes (such as time reminding, receiving information, alarm clock, game and the like) can also correspond to different vibration feedback effects. The touch vibration feedback effect may also support customization.

Indicator 192 may be an indicator light that may be used to indicate a state of charge, a change in charge, or a message, missed call, notification, etc.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be brought into and out of contact with the electronic apparatus 100 by being inserted into the SIM card interface 195 or being pulled out of the SIM card interface 195. The electronic device 100 may support 1 or N SIM card interfaces, N being a positive integer greater than 1. The SIM card interface 195 may support a Nano SIM card, a Micro SIM card, a SIM card, etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 may also be compatible with different types of SIM cards. The SIM card interface 195 may also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as communication and data communication. In some embodiments, the electronic device 100 employs esims, namely: an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: the computer program code can be carried to any entity or apparatus, such as the first device or the second device, the recording medium, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, and software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.

Finally, it should be noted that: the above description is only an embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A notification message processing method applied to a first device, the method comprising:

monitoring a newly added notification message in the notification bar in the process of screen projection to the second device;

if the notification bar is monitored to newly add a first notification message, acquiring the message content of the first notification message from the monitoring information of the first notification message;

and sending the message content to the second device, wherein the message content is used for displaying a second notification message in a message aggregation page of a specified application according to the message content under the condition that a screen projection window of the first device is switched to a background after the second device receives the message content, and the second notification message is associated with the message content of the first notification message.

2. The method of claim 1, wherein after sending the message content to the second device, further comprising:

receiving an application identifier of a first application sent by the second device, wherein the first application is an application for pushing the first notification message to the notification bar, and the application identifier of the first application is sent by the second device to the first device after detecting a trigger operation on the displayed second notification message;

and starting the first application according to the application identifier of the first application.

3. The method of claim 2, further comprising:

acquiring an application identifier of the first application from the monitoring information of the first notification message;

and sending the application identification of the first application to the second device.

4. The method according to any of claims 1-3, wherein the monitoring information of the first notification message is information of a first data structure;

the obtaining the message content of the first notification message from the monitoring information of the first notification message includes:

analyzing the monitoring information of the first notification message according to the analysis mode of the first data structure;

and extracting the message content of the first notification message from the analysis result.

5. The method of any of claims 1-4, wherein prior to sending the message content to the second device, further comprising:

packaging the message content into first packaging information in a first data format;

converting the first encapsulation information into second encapsulation information in a second data format;

the sending the message content to the second device includes:

and sending the second packaging information to the second equipment.

6. The method of claim 5, wherein the first data format is a JavaScript object notation (JSON) data format and the second data format is a byte data format.

7. The method of any of claims 1-6, wherein prior to sending the message content to the second device, further comprising:

establishing a communication connection with the second device;

the sending the message content to the second device includes:

and sending the message content to the second device through the communication connection.

8. The method of claim 7, wherein the establishing a communication connection with the second device comprises:

establishing a long connection with the second device as the communication connection in response to a screen-casting operation;

alternatively, the first and second electrodes may be,

and establishing a short connection with the second equipment as the communication connection in response to the acquired message content of the first notification message.

9. The method of any of claims 1-8, wherein the first device comprises a first screen projection module and a message listening service;

in the process of projecting the screen to the second device, monitoring a newly added notification message in the notification bar, including:

and in the process that the first screen projection module projects the screen to the second equipment, the message monitoring service monitors the newly added notification message in the notification bar.

10. The method of claim 9, wherein the first device further comprises a listening module, a first message conversion module, and a first transmission module;

if the notification bar is monitored to newly add a first notification message, acquiring the message content of the first notification message from the monitoring information of the first notification message, including:

if the message monitoring service monitors that a first notification message is newly added to the notification bar, the message monitoring service calls back the monitoring module according to the monitoring information of the first notification message so as to send the monitoring information of the first notification message to the monitoring module;

the monitoring module acquires monitoring information of the first notification message after being called back by the message monitoring service, and sends the monitoring information of the first notification message to the first message conversion module;

the first message conversion module acquires the message content of the first notification message from the monitoring information of the first notification message, and sends the message content of the first notification message to the first transmission module;

the sending the message content to the second device includes:

and the first transmission module transmits the message content to the second equipment.

11. A notification message processing method applied to a second device, the method comprising:

receiving message content of a first notification message sent by first equipment in the process of being thrown by the first equipment, wherein the first notification message is a newly added notification message in a notification bar of the first equipment;

and under the condition that the screen projection window of the first device is switched to the background, displaying a second notification message in a message aggregation page of a specified application according to the message content, wherein the second notification message is associated with the message content of the first notification message.

12. The method of claim 11, wherein after displaying the second notification message in the message aggregation page of the specific application according to the message content, further comprising:

acquiring a first trigger operation on the second notification message;

responding to the first trigger operation, switching the screen projection window switched to the background to the foreground, and sending an application identifier of a first application to the first equipment;

the first application is an application for pushing the first notification message to the notification bar, the application identifier of the first application is used for starting the first application according to the application identifier of the first application after the first device receives the application identifier of the first application, and the screen projection window is used for displaying screen projection content of the first device.

13. The method of claim 12, further comprising:

and receiving the application identification of the first application sent by the first equipment in the process of being projected by the first equipment.

14. The method according to any of claims 11-13, wherein said receiving the message content of the first notification message sent by the first device comprises:

receiving second encapsulation information in a second data format sent by the first device, wherein the second encapsulation information comprises the message content of the first notification message;

converting the second packaging information into first packaging information in a first data format;

and decapsulating the first encapsulation information, and extracting the message content of the first notification message from the decapsulation result.

15. The method according to any one of claims 11-14, wherein said displaying the second notification message in a message aggregation page according to the message content comprises:

adding the message content to a message management list, wherein the message management list is used for storing message content of newly added notification messages in a notification bar of the first device, and the message aggregation page is used for displaying the notification messages corresponding to the message content stored in the message management list;

refreshing the message aggregation page according to the message management list;

displaying a refreshed message aggregation page, wherein the refreshed message aggregation page comprises the second notification message.

16. The method of claim 15, wherein adding the message content to a message management list comprises:

encapsulating the message content into message data of a key-value pair mapping structure;

adding the message data to the message management list.

17. The method of claim 15, wherein displaying the refreshed message aggregation page comprises:

acquiring a second trigger operation, wherein the second trigger operation is used for opening the message aggregation page;

and responding to the second trigger operation, and displaying the refreshed message aggregation page.

18. The method according to any of claims 11-17, wherein after receiving the message content of the first notification message sent by the first device, further comprising:

and carrying out new message reminding, wherein the new message reminding is used for indicating that a new notification message is added to the message aggregation page.

19. The method of any of claims 11-18, wherein the second device comprises a second screen projection module, a second transmission module, a message management module, and a display module;

the receiving of the message content of the first notification message sent by the first device in the process of being projected by the first device includes:

in the process that the second screen projection module receives the screen projection of the first device, the second transmission module receives the message content of the first notification message sent by the first device;

the displaying the second notification message in the message aggregation page according to the message content includes:

the message management module acquires the message content of the first notification message received by the second transmission module, and adds the message content to a message management list, wherein the message management list is used for storing the message content of the notification message newly added in a notification bar of the first device, and the message aggregation page is used for displaying the notification message corresponding to the message content stored in the message management list;

and the display module refreshes the message aggregation page according to the message management list and displays the refreshed message aggregation page, wherein the refreshed message aggregation page comprises the second notification message.

20. The method of claim 19, wherein the second device further comprises a second message conversion module;

the second transmission module receives the message content of the first notification message sent by the first device, and the message content includes:

the second transmission module receives second encapsulation information of a second data format sent by the first device, wherein the second encapsulation information comprises the message content of the first notification message;

after the second transmission module receives the message content of the first notification message sent by the first device, the method further includes:

the second transmission module sends the second encapsulation information to the second message conversion module;

the second message conversion module converts the second packaging information into first packaging information in a first data format; and decapsulating the first encapsulated information, extracting the message content of the first notification message from the decapsulation result, and sending the message content of the first notification message to the message management module.

21. An electronic device, comprising a memory and a processor;

the memory is used for storing a program supporting the electronic device to execute the method of any one of claims 1-10 and storing data involved in implementing the method of any one of claims 1-10; the processor is configured to execute programs stored in the memory.

22. An electronic device, comprising a memory and a processor;

the memory is used for storing a program supporting the electronic device to execute the method of any one of claims 11-20 and storing data involved in implementing the method of any one of claims 11-20; the processor is configured to execute programs stored in the memory.

23. A computer-readable storage medium having instructions stored therein, which when executed on a computer, cause the computer to perform the method of any one of claims 1-20.

24. A computer-readable storage medium having instructions stored therein, which when executed on a computer, cause the computer to perform the method of any one of claims 11-20.

25. A message processing system is characterized in that the system comprises a first device and a second device, wherein the first device casts a screen to the second device;

the first device is used for monitoring a newly added notification message in the notification bar in the process of screen projection to the second device; if the notification bar is monitored to newly add a first notification message, acquiring the message content of the first notification message from the monitoring information of the first notification message; sending the message content to the second device;

the second device is used for receiving the message content sent by the first device in the process of being projected by the first device; and under the condition that the screen projection window of the first device is switched to the background, displaying a second notification message in a message aggregation page of a specified application according to the message content, wherein the second notification message is associated with the message content of the first notification message.

26. The system of claim 25,

the first device is further configured to obtain an application identifier of a first application from monitoring information of the first notification message, and send the application identifier of the first application to the second device, where the first application is an application that pushes the first notification message to the notification bar;

the second device is further configured to receive an application identifier of the first application sent by the first device.

27. The system of claim 25 or 26,

the second device is further configured to acquire a first trigger operation on the second notification message; responding to the first trigger operation, if the screen projecting window of the first device is switched to the background, switching the screen projecting window switched to the background to the foreground, and sending an application identifier of a first application to the first device, wherein the screen projecting window is used for displaying screen projecting contents of the first device, and the first application is an application for pushing the first notification message to the notification bar;

the first device is further configured to receive the application identifier, and start the first application according to the application identifier.

28. The system of any one of claims 25-27,

the second device is further configured to perform a new message alert, where the new message alert is used to indicate that a new notification message is added to the message aggregation page.