CN115309299B - Desktop card display method, device, terminal, storage medium and program product - Google Patents

Desktop card display method, device, terminal, storage medium and program product Download PDF

Info

Publication number
CN115309299B
CN115309299B CN202211117854.9A CN202211117854A CN115309299B CN 115309299 B CN115309299 B CN 115309299B CN 202211117854 A CN202211117854 A CN 202211117854A CN 115309299 B CN115309299 B CN 115309299B
Authority
CN
China
Prior art keywords
card
desktop
control
cards
content
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211117854.9A
Other languages
Chinese (zh)
Other versions
CN115309299A (en
Inventor
陈科鑫
张晓帆
董伟鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN202211117854.9A priority Critical patent/CN115309299B/en
Publication of CN115309299A publication Critical patent/CN115309299A/en
Application granted granted Critical
Publication of CN115309299B publication Critical patent/CN115309299B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The embodiment of the application discloses a display method, a display device, a display terminal, a storage medium and a display program product for desktop cards, and belongs to the field of man-machine interaction. The method comprises the following steps: acquiring a desktop card, wherein the desktop card is a card assembly for information display on a desktop; based on the card content of the desktop card, carrying out card clustering on the desktop card to obtain a card clustering result, wherein the card clustering result comprises a plurality of card clusters; integrating the desktop cards in the same card cluster to obtain a desktop integration component, wherein the desktop integration component is an independent desktop card or a desktop card group formed by a plurality of desktop cards; and displaying the desktop integration component. By adopting the scheme provided by the embodiment of the application, the terminal desktop cards are clustered and integrated, so that the desktop card display is more ordered, and application information pushed by the desktop card is clearer.

Description

Desktop card display method, device, terminal, storage medium and program product
Technical Field
The embodiment of the application relates to the technical field of man-machine interaction, in particular to a display method, a device, a terminal, a storage medium and a program product of a desktop card.
Background
The technology of the intelligent terminal is mature, so that the information recommended by the intelligent terminal for the user is richer and more various, and the desktop card is a classical information presenting mode for the user in the intelligent terminal.
In general, the desktop card is arranged on a negative screen, information of an application is displayed to a user, the user can change the display position and the display state of the desktop card through sliding or long-press operation, and after the desktop card is triggered, the user can jump to an application interface to provide corresponding services for the user.
However, as application information tends to be diversified, the number and variety of desktop cards are gradually increased, the display interface is also more complex, the display of the desktop cards is poor in regularity, and the problem of difficulty in selection is caused because the user is not benefited to intuitively acquire the application information.
Disclosure of Invention
The embodiment of the application provides a display method, a device, a terminal, a storage medium and a program product of a desktop card. The technical scheme is as follows:
in one aspect, the present application provides a method for displaying a desktop card, where the method includes:
acquiring a desktop card, wherein the desktop card is a card assembly for information display on a desktop;
based on the card content of the desktop card, carrying out card clustering on the desktop card to obtain a card clustering result, wherein the card clustering result comprises a plurality of card clusters;
Integrating the desktop cards in the same card cluster to obtain a desktop integration component, wherein the desktop integration component is an independent desktop card or a desktop card group formed by a plurality of desktop cards;
and displaying the desktop integration component.
In another aspect, the present application provides a device for displaying a desktop card, the device comprising:
the desktop card is a card-type component for information display on the desktop;
the clustering module is used for carrying out card clustering on the desktop cards based on the card content of the desktop cards to obtain card clustering results, wherein the card clustering results comprise a plurality of card clusters;
the desktop card integration module is used for integrating the desktop cards in the same card cluster to obtain a desktop integration component, wherein the desktop integration component is an independent desktop card or a desktop card group formed by a plurality of desktop cards;
and the display module is used for displaying the desktop integration assembly.
In another aspect, an embodiment of the present application provides a terminal, where the terminal includes a processor and a memory; the memory stores at least one instruction for execution by the processor to implement a method of displaying a desktop card as described in the above aspects.
In another aspect, embodiments of the present application provide a computer readable storage medium having at least one program code stored therein, the program code loaded and executed by a processor to implement a method for displaying a desktop card as described in the above aspect.
In another aspect, embodiments of the present application provide a computer program product comprising computer instructions stored in a computer-readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the method of displaying a desktop card provided in various alternative implementations of the above aspects.
In this embodiment, after the terminal obtains the desktop card, cluster and integrate the desktop card according to the card content of desktop card, at last show the desktop integration subassembly after the integration, the terminal can be according to the content of desktop card, the self-adaptation integrates the desktop card for the organization form of desktop card is not limited to fixed, or the irregular display form, more is favorable to clearly showing application information to the user.
Drawings
FIG. 1 is a flowchart of a method for displaying desktop cards according to an exemplary embodiment of the present application;
FIG. 2 illustrates a flowchart of a desktop card display method provided in another exemplary embodiment of the present application;
FIG. 3 illustrates a schematic diagram of desktop card integration provided in an exemplary embodiment of the present application;
FIG. 4 illustrates a schematic diagram of desktop card integration provided in accordance with another exemplary embodiment of the present application;
FIG. 5 illustrates a flowchart of a process for obtaining a desktop card provided in another exemplary embodiment of the present application;
FIG. 6 is a schematic diagram of a process for acquiring card content according to an exemplary embodiment of the present application;
FIG. 7 is a flowchart illustrating a process for generating a card feature vector according to another exemplary embodiment of the present application;
FIG. 8 illustrates a schematic diagram of determining a first key control provided by an exemplary embodiment of the present application;
FIG. 9 illustrates a schematic diagram of determining a second key control provided by an exemplary embodiment of the present application;
fig. 10 is a schematic structural diagram illustrating a terminal acquiring an association degree of an application interface control according to an exemplary embodiment of the present application;
FIG. 11 is a flowchart illustrating a method for displaying a desktop card according to another exemplary embodiment of the present application;
FIG. 12 is a schematic diagram illustrating display area partitioning provided in an exemplary embodiment of the present application;
FIG. 13 is a diagram of a desktop card update layout according to an exemplary embodiment of the present application;
FIG. 14 illustrates a desktop card update layout diagram provided in accordance with another exemplary embodiment of the present application;
FIG. 15 is a flowchart illustrating a method for pushing a desktop card according to an exemplary embodiment of the present disclosure;
FIG. 16 illustrates a schematic diagram of pushing of a desktop card provided in an exemplary embodiment of the present application;
FIG. 17 is a block diagram showing a display device of a desktop card according to one embodiment of the present application;
fig. 18 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
It should be noted that all application information collected in the present application is collected under the consent and authorization of the user, and the collection, use and processing of relevant application data need to comply with relevant laws and regulations and standards of relevant countries and regions.
Fig. 1 shows a flowchart of a method for displaying a desktop card according to an exemplary embodiment of the present application, where the method includes:
step 101, obtaining a desktop card, wherein the desktop card is a card assembly for information display on a desktop.
The desktop card is a card component for information display to the user, is usually arranged on a negative screen and is an interface control. After the desktop card is triggered, the application interface corresponding to the card content can be jumped to.
The desktop card is generated based on the application interface, and is generated by acquiring information of the application interface, processing and then performing art rendering. And the other mode is to fill the content based on the card template, and after the terminal obtains the application interface information, the terminal fills the corresponding content into the proper template according to the set template, so as to generate the desktop card.
Step 102, based on the card content of the desktop card, carrying out card clustering on the desktop card to obtain a card clustering result, wherein the card clustering result comprises a plurality of card clusters.
The card content of the desktop card comprises explicit content and implicit content, wherein the explicit content refers to content displayed by the desktop card and can be seen by a user, for example, information displayed by the desktop card, an application corresponding to the desktop card and the like; the implicit content refers to content which is stored in an internal cache of the terminal and is invisible to the user, for example, a jump link of an application interface corresponding to the desktop card, important data in the process of generating the desktop card, and the like.
After obtaining desktop cards, the terminal clusters the desktop cards according to the card content of the desktop cards, wherein clustering refers to dividing the desktop cards into different types of clusters according to a certain specific standard (such as distance), so that the similarity of the desktop cards in the same card type cluster is as large as possible, and meanwhile, the difference of the desktop cards not in one type of cluster is as large as possible, namely, the desktop cards of the same type are aggregated together as much as possible after clustering, and the desktop cards of different types are separated as much as possible.
And 103, integrating the desktop cards in the same card cluster to obtain a desktop integration component, wherein the desktop integration component is an independent desktop card or a desktop card group formed by a plurality of desktop cards.
Because the variety and the quantity of the application information are very rich, and the interface size of the terminal for displaying the desktop card is limited, the desktop card needs to be integrated.
The card clustering result can be used as a basis for integrating desktop cards, and the desktop cards in the same card type cluster in the card clustering result are integrated to obtain the desktop integration component.
Integration of desktop cards essentially integrates similar desktop cards into a new card or new deck based on the relevance of the card content of the desktop cards. Optionally, the desktop integration component may integrate the card content by using the desktop cards in the same card cluster to generate a new desktop card; the desktop cards with irregular arrangement can be integrated into a new desktop card group which is used as a desktop integration component, so that relevant application information can be conveniently displayed for a user.
Step 104, displaying the desktop integration component.
And displaying the desktop integrated assembly obtained after integrating the desktop cards on a desktop or a negative screen to display application information for a user.
To sum up, in the embodiment of the present application, after the terminal obtains the desktop card, the terminal clusters and integrates the desktop card according to the card content of the desktop card, and finally displays the integrated desktop integration component, and the terminal can adaptively integrate the desktop card according to the content of the desktop card, so that the organization of the desktop card is not limited to a fixed or irregular display form, which is more beneficial to clearly displaying application information to the user.
Fig. 2 shows a flowchart of a desktop card display method according to another exemplary embodiment of the present application, where the method includes:
step 201, obtaining a desktop card.
The implementation of this step may refer to step 101, and this embodiment is not described herein.
Step 202, obtaining a card feature vector of a desktop card, wherein the card feature vector is generated based on a card content vector corresponding to card content of at least one content dimension, and the content dimension comprises text and a control.
When the terminal clusters the desktop cards, the terminal needs to cluster according to the similarity of the desktop cards, and the similarity of the desktop cards needs to be measured through the numerical value of the similarity of the desktop cards. The similarity value of the desktop card needs to be calculated according to the feature vector of the desktop card.
The mode of the terminal obtaining the card feature vector of the desktop card is different according to the mode of generating the desktop card.
In one possible implementation, the desktop card is generated based on the application interface, and the card feature vector of the desktop card is generated in the process of generating the desktop card.
In another possible implementation, the desktop card is obtained by filling content based on a card template, and the card feature vector of the desktop card is not generated when the desktop card is generated. Therefore, the card content line vector processing of at least one content dimension in the desktop card is needed to obtain the card content vector. And obtaining a card characteristic vector for generating the desktop card based on the card content vector.
Wherein the card content dimension comprises two dimensions of text and control. And the terminal acquires the text content of the desktop card as card content under the condition that the desktop card only contains text dimension, and takes the text content vector obtained after the text content is encoded as card feature vector. Under the condition that the desktop card comprises a text dimension and a control dimension, text content of the text dimension and control content of the control dimension are obtained to serve as card content, the card content of the two dimensions is encoded in different encoding modes to obtain a card content vector, and then the text content vector and the control content vector are weighted and averaged to obtain a card feature vector. The Text content refers to the Text attribute of the desktop card, namely the Text attribute. The control content may contain image information of the control and structured information of the control.
In one possible implementation, after the terminal obtains the text content of the desktop card, a word segmentation algorithm is used to segment the text content into a plurality of words independent of each other, and the words are segmented text. And performing sliding pooling processing on word vectors corresponding to each word segmentation text to obtain text vectors corresponding to the text content. On one hand, the terminal can obtain the structured information of the desktop card by obtaining the control tree of the desktop card display interface, and on the other hand, the terminal can obtain the control icon of the desktop card control content in a screenshot mode after confirming the position of the desktop card in the display interface. After the control content is obtained, the control content code of the desktop card is obtained through image processing, image coding and structuring coding. And then carrying out weighted average on the control content codes and the text content codes to obtain the card feature vector of the desktop card.
And 203, clustering the desktop cards based on the vector distance between the card feature vectors to obtain a card clustering result.
After the terminal acquires the feature vectors of the desktop cards, the desktop cards need to be clustered according to the similarity between the desktop cards, and the similarity of the desktop cards is obtained by calculating the feature vectors of the cards. The similarity between two desktop cards can be determined by Sim (C 1 ，C 2 )＝cosine(F(C 1 )，F(C 2 ) And (3) calculating, wherein C represents a desktop card, and F (C) represents a card feature vector corresponding to the desktop card.
After the terminal obtains the similarity between the desktop cards, the desktop cards need to be clustered according to a similarity threshold. Desktop cards with similarity higher than the similarity threshold value can be clustered to obtain a card class cluster, and desktop cards with similarity lower than the similarity threshold value cannot be clustered into a card class cluster. The similarity threshold is set, the value of the similarity threshold can be adjusted according to the state of the display interface of the terminal, and when the complexity of the display interface of the terminal is high, the similarity threshold is correspondingly lowered, so that more desktop cards can be clustered together, and the display interface control is saved. When the complexity of the display interface of the terminal is low, the similarity threshold value is correspondingly increased by the terminal, so that desktop cards with higher content correlation degree of the cards are clustered together, the display regularity of the desktop cards is stronger, and various information can be provided more clearly.
Step 204, integrating the plain text desktop cards based on the texts contained in the plain text desktop cards in the same card class cluster to obtain an integrated desktop card, wherein the integrated desktop card is an independent desktop card, and the plain text desktop card does not contain controls.
After the terminal clusters the desktop cards, the desktop cards which only contain text content, namely plain text desktop cards, may be in one card class cluster. Under the condition, the terminal acquires texts contained in all desktop cards in the card clusters, extracts similar texts in text contents of all desktop cards in the card clusters from the texts, extracts key words from the similar texts, and serves as key texts of the card clusters and also serves as key texts of the integrated desktop cards.
Before generating the desktop integration cards, the terminal needs to refine and classify the desktop cards in each class cluster, so that the similarity of the desktop cards contained in each classified result after refinement and classification is higher, and then corresponding type names are marked for different classified results for displaying in the desktop integration cards.
As shown in fig. 3, after the terminal clusters the desktop cards, integrating the multiple plain text desktop cards 301 of the notification class obtained by the clustering to obtain a second desktop card 302, where the "message alert" is a key text, "message notification", "news notification" and "health notification" are obtained by finely classifying the desktop cards in the card class cluster, and the terminal marks different types of names, such as "message notification", "news notification", "health notification" and the like, for the classification result of the finely classified desktop cards and displays the names in the desktop integration card.
Step 205, integrating the controllable desktop cards in the same card cluster into a desktop card group, wherein the controllable desktop cards comprise texts and controls.
When the terminal clusters the desktop cards, the card clusters may contain the desktop cards with the controls, and if the method is adopted with the pure text desktop cards, the desktop cards with the controls originally cannot realize the function of controlling the application through the desktop cards directly after integration. Therefore, for the desktop card with the control, the terminal only needs to adjust the original desktop card to a proper size and then integrate the desktop card into a new desktop card group. The desktop deck is a desktop integration component that is essentially a box-type control whose key text is extracted from the text of desktop cards in the deck.
As shown in fig. 4, the controllable desktop cards of the same card cluster are integrated, a first desktop card 401 with a key text of "map navigation" and a second desktop card 402 with a key text of "travel scene" are integrated with a third desktop card 403 with a key text of "travel assistant", so as to obtain a desktop card group 404 with a key text of "convenience travel".
Step 206, displaying the desktop integration component.
And displaying the integrated desktop card or desktop card group after integration in a display interface.
In the embodiment of the application, the terminal clusters the desktop cards based on the similarity of the card contents of the desktop cards, so that the desktop card display is more organized. And the desktop integration components for integrating different desktop cards in the card clustering result into different forms reduce the complexity of the desktop cards, and the integration modes of the desktop cards are more flexible and various, so that different kinds of application information can be displayed more clearly.
Since the types of the application information are different, the display content of the desktop card corresponding to the application information is also different. And a part of desktop cards only need to display text information to the user, and the user can jump to the corresponding application interface by triggering the corresponding desktop cards. The other part of desktop cards needs to be provided with corresponding control controls for directly controlling the application, for example, the desktop card of the player with play and pause buttons, and a user can directly control the application through the control controls in the desktop card or jump to the corresponding application interface by triggering the desktop card. Correspondingly, the generation method is different for different desktop card types.
The manner in which desktop cards are generated will be described below with respect to one exemplary embodiment.
Fig. 5 shows a flowchart of a process for obtaining a desktop card according to another exemplary embodiment of the present application, where the process includes:
step 501, extracting key interface content of at least one content dimension from interface content of an application interface.
And the terminal acquires at least one content dimension of the text dimension and the control dimension from the application interface as key interface content. In general, text content exists in an application interface, and the text content is more specific than other forms of content, so the terminal first obtains text dimension content as key interface content.
Furthermore, when the terminal comprises the control, the dimension content of the control can also be used as key interface content to be displayed in the interface card.
In one possible implementation manner, after the terminal obtains the control tree of the application interface, the text attribute of the control is extracted from the control information corresponding to the control by traversing each node in the control tree, wherein the text attribute comprises at least one of an explicit text attribute and an implicit text attribute. The explicit Text attribute refers to a Text attribute of a control, which is visible to a user, and Text displayed on an application interface in a Text form. The implicit text attribute, i.e., the invisible text attribute, refers to the content description text attribute of the control. The ContentDescription text attribute is an attribute written by the developer inside the control, the text describing the control functionality is invisible in the application interface, and the developer makes the interface beautiful and hides part of the text into the ContentDescription text attribute.
The terminal acquires Text content by traversing each node of the control tree, so that Text attributes and content description Text attributes of each control of the application interface can be acquired, and the Text attributes and the content description Text attributes of some controls are null due to different development habits of developers.
In another possible implementation manner, the terminal extracts control coordinates from control information corresponding to the application interface control in the control tree, then intercepts a control image of the application interface control from the application interface based on the control coordinates, and simultaneously, the terminal extracts structured information from the control information of the interface control of the control tree. And the structured information and the control image are used as control dimension content together.
And step 502, performing text summarization processing on the text to obtain a key text.
For text dimension content in interface content, a corresponding algorithm is needed to simplify the information. For display in a desktop card.
Alternatively, the terminal extracts key text of the text dimension content from the text dimension content using MMR (maximum boundary correlation, maximal Marginal Relevance). Text summarization is the computation of a piece of text that is most representative of its subject for a given piece of text sequence. Common algorithms are divided into generative and decimated, where decimated is the use of a portion of text in the original text to represent the subject matter of the entire piece of text. Because the terminal application information is updated faster, the MMR extraction algorithm with higher operation efficiency is adopted for text abstract processing. Setting the dimension content of the original text as C, d i For a subset of phrases in text-dimensional content, λsim (C, d i ) Representing the similarity of a subset of phrases to the original text dimension content C,representing the similarity of a subset of phrases to the processed abstract text,the index of k sentences returned by the search is represented, wherein the two expressions are connected by a minus sign, so that the extracted text abstract can express the meaning of the content of the whole text dimension and has diversity. Where λ is a parameter for controlling the diversity of the summary. The MMR algorithm formula for text summarization processing is as follows: />And the terminal takes the result of the text abstract processing as key text.
In step 503, in the case that the application interface includes a control, a key control is extracted from the control.
For the control dimension content in the application interface, the terminal needs to extract the key control in the application interface and the control associated with the key control as the key control.
In one possible implementation, the terminal may perform the acquisition of the key control through a key control prediction algorithm.
For example, a play control of a video player page may be used as a key control, and a pause control is a control associated with the play control.
And 504, taking the extracted key interface content as card content, and generating a desktop card.
Wherein the key interface content includes at least one of key text and key controls. After the terminal obtains the key Interface content, the key Interface content is used as card content, and is rendered through a corresponding art UI (User Interface), so that a desktop card corresponding to the application Interface can be obtained. The present embodiment does not describe the art UI rendering in detail.
Step 505, generating a card feature vector of the desktop card based on the key interface content.
The key interface content is the card content of the desktop card, and the embodiment of this step may refer to the above step 101, and this embodiment is not described herein.
As shown in fig. 6, in one possible implementation, the message notification class information, the consultation summary class information, the item reminding class information and the drawing recommendation class information may correspond to plain text desktop cards, so that when the desktop cards are generated, plain text information is extracted from the application interface, and then the key text is acquired through the text information summary algorithm for display in the corresponding desktop cards. The reminding item information, the service pushing information and the service control information may correspond to the controllable desktop card, so that when the desktop card is generated, application information with a control relation is extracted from an application interface, and then calculation is performed through a key associated control prediction algorithm to obtain a key text and a key control, and the key text and the key control are processed and then displayed on the desktop card.
It should be noted that, the type of the application information may be the same as the type of the application information divided in fig. 6, namely: a message notification class, a consulting abstract class, a reminder class, a service push class and a service control class. Alternatively, more application information types may be partitioned according to different dimensions in iterative updating of application information. The method for dividing the application information types and the divided application information types are not limited.
According to the embodiment of the application interface, the key text and the key control of the application interface are respectively obtained by adopting different processing modes according to the text dimension and the control dimension of the application interface and are used as key contents of the application interface, and the desktop card is generated according to the key contents in a mode of rendering by the art UI, so that the generation of the desktop card is not limited to the filling of the fixed mould, and the problem of single form of the desktop card is solved.
For the control dimension content in the application interface, the terminal needs to determine the first control from the control tree and then determine the second control associated with the first control, and the first control and the second control are used as key controls together. The process of determining key controls in the case where the application interface contains control dimensions will be described below with one exemplary embodiment.
Fig. 7 is a flowchart of a process for generating a card feature vector according to another exemplary embodiment of the present application, where the process includes:
step 701, obtaining a control tree of an application interface, wherein the control tree is used for representing the topological structure of each control in the application interface.
The terminal can directly acquire a control tree corresponding to the application interface through an Accessibility (barrier-free service) interface defined by the system.
The control tree is composed of interface controls of the application interface. The control tree is a tree-shaped structure and comprises all controls in the application interface, each control has corresponding control information, and each control corresponds to one node in the control tree. When the terminal acquires the control tree, the terminal can obtain the control coordinates of the controls corresponding to each node in the control tree, and each control has one control coordinate corresponding to the control coordinate one by one.
After the terminal acquires the control tree, a first key control and a second key control related to the first key control need to be determined from the control tree, and the first key control and the second key control are extracted for display in the desktop card. The specific implementation steps are as follows.
Step 702, control information of each control is obtained from a control tree, the control information comprises at least one of text information, image information and structural information, and the structural information is used for representing a hierarchical structure of the control in the control tree.
The Text information in the control information refers to Text words in the application interface, and the Text attribute of the control comprises a visible Text attribute (i.e. Text attribute) and a non-visible Text attribute (i.e. content description attribute).
The image information refers to images in the application interface, including icons, plain images and text images displayed in the form of images.
The structured information is used for representing the hierarchical structure of the controls in the application interface, each control is a container, and when a developer develops the application interface, the controls are set according to a certain hierarchical relationship. The terminal can determine other controls which have certain association with the control, such as child controls or father controls of the control and the like by acquiring structural modal information of a certain control.
And step 703, coding the control information to obtain a control original coding vector of the control.
After the terminal acquires the control information of different types, the terminal needs to encode the control information of different types respectively by different encoding modes because the characteristics of the information of different types are different.
The text information is encoded by a text encoder; the image information is encoded by an image encoder; and the structured information is encoded by adopting a structured encoder to obtain the corresponding modal codes of the different modal information.
In one possible implementation, binary coding is adopted for the structured information, and the position codes of the control in the first order traversal sequence of the control tree and the position codes of the control in the middle order traversal sequence of the control tree are spliced to obtain a structured code vector; for the text information, firstly, word segmentation algorithm is carried out on the text information of the control, then word vectors of each word are queried by taking the word as a unit, and weighted average is carried out on the word vectors, so that text coding vectors of the control can be obtained; depth features of the image are extracted for the image information terminal using a pre-trained CNN (Convolutional Neural Network, neural network model) and then converted into one-dimensional vectors. I.e. the image coding vector.
After the terminal obtains the coding vectors corresponding to the control information of each type, different coding vectors are fused to obtain the original coding vector of the control. Optionally, the terminal fuses the coding vectors corresponding to the different kinds of control information through the feature fusion network to obtain the original coding vector of the control.
And step 704, performing self-attention coding on the original coding vector of the control to obtain the self-attention coding vector of the control.
And after the terminal acquires the original coding vector of the control, extracting the self-attention coding vector of the control by using a self-attention mechanism of the transducer model.
Step 705, determining a first key control based on the control self-attention encoding vector of the corresponding control of the key text and the control self-attention encoding vector of other controls.
After the terminal determines the key text, the relevance between the two controls can be calculated according to the self-attention coding vector of the control corresponding to the key text and the self-attention coding vector of other controls in the application interface. And determining the control with the degree of association higher than the set threshold value corresponding to the key text as a first key control.
As shown in fig. 8, the association degree of the two controls connected by the solid line is higher than the set threshold. The text control 802 corresponding to the key text in the application interface 801 has a relation of association degree among the video control 803, the first icon control 804, the second icon control 805 and the third icon control 806 through calculation, wherein the relation degree of the video control 803 and the text control 802 corresponding to the key text is higher than a set threshold value, so that the video control 803 is determined to be the first key control.
In one possible embodiment, the terminal may calculate the association between two controls using an MLP (Multi-Layer Perceptron), where the association between two controls is calculated based on the self-attention encoding vectors of the two controls. And inputting the self-attention coding vectors of the two controls into the MLP model to obtain the association degree of the two self-attention coding vectors, namely the association degree of the two controls. And after the association degree between the application interface controls is obtained, sorting the association degree from high to low, and determining the control with the association degree higher than the set threshold value corresponding to the key text as the first key control.
Step 706, determining a second key control associated with the first key control based on the control self-attention encoding vector of the first key control and the control self-attention encoding vectors of the other controls.
After the terminal determines the first key, based on the self-attention coding vector of the first key control, calculating by adopting the MLP model and the self-attention coding vectors of other controls to obtain the association degree between the first key control and the other controls, and determining the control with the association degree higher than the set threshold value with the first key control as the second key control.
When the same application interface has a plurality of key texts, each key text corresponding control has a control with association degree, and the control with association degree higher than the threshold value in the control with association degree of each key text corresponding control can be respectively determined as a first key control, and at the moment, one application interface can generate a plurality of first key controls. And similarly, determining the control with the association degree higher than the set threshold value from the controls with the association degree of each first key control as a second key control, and displaying the second key control, the key text and the first key control in the desktop card.
Each group of associated controls and the corresponding key text can be displayed as card content of one controllable desktop card. As shown in fig. 9, in the application interface 901, three controls corresponding to "personalized customization", "display and brightness", "notification and status bar" of the key text are respectively a first control 902, a second control 903, a third control 904, three first key controls 905 respectively corresponding to the three controls and three second key controls 906 respectively associated with the first key controls, where the degree of association of the current interface is calculated.
Fig. 10 shows a schematic structural diagram of a terminal for obtaining the relevance of an application interface control, where the terminal encodes text information, image information and structured information in the application interface 1001 through a structure encoder 1002, a text encoder 1003 and an image encoder 1004 respectively, so as to obtain three corresponding encoding vectors. And (3) splicing and fusing the three coding vectors to obtain the original coding vector of the control. And then performing self-attention coding on the original coding vector of the control by using a transducer model 1005 to obtain the self-attention coding vector of the control. And finally, performing relevance calculation on the control self-attention coding vector and the self-attention coding vectors of other controls through the MLP model 1006 to obtain the relevance among the controls.
And 707, vectorizing the key text to obtain a text feature vector.
The text vectorization process represents the key text as a real number vector which can be identified by a computer, and the key text can be represented as text feature vectors of the levels of words, sentences and the like according to different sizes of fine granularity.
Alternatively, text vectorization may be implemented by a statistical-based method, or by a neural network-based method.
Step 708, generating a card feature vector of the desktop card based on the text feature vector and the control self-attention encoding vector of the key control.
And carrying out weighted average on the control self-attention coding vector and the text feature vector of the key text to obtain the card feature vector of the desktop card.
According to the method and the device for generating the desktop card, after the terminal obtains the application interface information, the self-attention coding vector of the control is generated through the transducer model, the first key control and the second key control are determined through the MLP model to calculate the association degree, and then the card feature vector of the desktop card can be generated, the controllable desktop control can be generated according to the association relation of the control in the application interface, the integrity of the display information of the desktop control is benefited, meanwhile, a user can control a plurality of functions of the application through one desktop control, and the practicability is higher.
In the above embodiment, after the terminal clusters the desktop cards, the desktop cards are directly integrated based on the card clustering result.
In another possible implementation manner, after obtaining the desktop card clustering result, the terminal needs to determine a relative loading factor based on the current card integration proportion and the display duty ratio, and integrate the desktop cards in the same card cluster to obtain the desktop card integration component under the condition that the relative loading factor is greater than a threshold value. The card display duty ratio is used for representing the current integration degree of the desktop card, and the card display duty ratio is used for representing the screen duty ratio of the desktop card displayed at present.
n cur For indicating the number of desktop cards (independent desktop cards or desktop card sets) independently displayed in a display page, n total For representing the total number of desktop cards (including desktop cards in the desktop deck) presented on the display interface, S cur Screen area occupied for a currently displayed desktop cardS is the total resolution area of the screen, then the relative loading factor Where λ is an adjustable parameter for controlling the propensity to integrate into a stand-alone desktop card or group of desktop cards.
The relative loading factor may be used to characterize the saturation level of the display interface, with greater relative loading factors indicating more crowded and more complex desktop cards of the display interface. Under the condition that the relative loading factor is larger than the set threshold, the terminal integrates the desktop cards, and when the relative loading factor is too large, the terminal can automatically optimize the direction in which the relative loading factor can be greatly reduced by adopting the method of the embodiment of the application, for example, a plurality of desktop cards are integrated into a new independent desktop card.
For desktop display cards, the desktop display cards should be arranged according to the browsing sequence of the user, and according to modern reading habits, the desktop cards can be ordered from top to bottom and from left to right according to the frequency of using the desktop cards by the user in general. However, arranging the desktop cards according to the frequency of use of the desktop cards may result in that the display position of the desktop cards, which is low-frequency but important, cannot be easily noticed by the user, and may also cause a problem that the display position of the desktop cards is not suitable. For example, the terminal acquires that a user can carry out an important meeting on a tomorrow, but the number of times that the terminal receives the information is small, and the desktop card corresponding to the reminding information of the important meeting is very low in use frequency, so that the desktop card is arranged according to the use frequency of the desktop card, which results in that the low-frequency high-priority important meeting reminding card can be arranged behind the high-frequency low-priority news card, and the effect of effective information recommendation cannot be achieved.
Fig. 11 shows a flowchart of a method for displaying a desktop card according to another exemplary embodiment of the present application, where the method includes:
step 1101, obtaining a desktop card.
The implementation of this step may refer to step 101, and this embodiment is not described herein.
Step 1102, based on the card content of the desktop card, performing card clustering on the desktop card to obtain a card clustering result, wherein the card clustering result comprises a plurality of card clusters.
The implementation of this step may refer to step 102, and this embodiment is not described herein.
In step 1103, desktop cards in the same card cluster are integrated to obtain a desktop integration component, where the desktop integration component is an independent desktop card or a desktop card group formed by multiple desktop cards.
The implementation of this step may refer to step 103, and this embodiment is not described herein.
Step 1104, determining a target display area to which the desktop integration component belongs based on the card content of the desktop card in the desktop integration component, wherein the different display areas correspond to different display priorities, and the display priorities are related to the card content.
The terminal adopts a data barrel dividing mode to divide the display area of the desktop card into a plurality of display areas with display priorities from top to bottom, wherein the display priority of the display area at the top is higher. As shown in fig. 12, the display interface 1201 is divided into different priorities, a first priority area 1202, a second priority area 1203, a third priority area 1204, and the like from top to bottom, respectively.
The virtual division of the display area division potential in the figure at the system layer is not visible in the display interface.
The display areas with different display priorities correspond to different desktop integration components, the terminal divides desktop cards belonging to the same priority into display areas with the same display priority based on the priorities of the card contents of the desktop cards in the desktop integration components, the display priorities of the display areas correspond to the priorities of the card contents in the desktop integration components, for example, a target display area to which the desktop card component with the highest card content priority belongs is a first priority area.
Step 1105, determining a target display position of the desktop integration component in the target display area based on the historical use frequency of the desktop card in the desktop integration component, wherein the different display positions correspond to different display priorities, and the display priorities and the historical use frequency are in positive correlation.
And in the display area corresponding to each display priority, the terminal ranks the desktop integration components again according to the use frequency of the user, and determines the target display position of the desktop integration components in the target display area.
The priority of the display position is in positive correlation with the use frequency of the user, and the higher the use frequency of the user is, the higher the display priority corresponding to the desktop card is, and the more striking display position is in the display interface.
The display area range corresponding to each display priority is not fixed, expansion and contraction adjustment can be performed according to actual use conditions, and display areas with different display priorities can be overlapped, so that the maximum use efficiency of a display interface is ensured. However, there is a set upper limit on the display area range of different display priorities, and when the desktop card in the display area corresponding to a certain display priority is full and cannot be integrated further, the desktop card with a relatively low frequency is stored in the system buffer.
At step 1106, the desktop integration component is displayed at a target display location of the target display area.
After the terminal determines the target display position of the desktop card, displaying a corresponding desktop control at the position, wherein the display position is changed according to the updating of the display interface by the terminal.
In the embodiment of the application, the terminal firstly divides the display priority of the display interface based on the card content priority of the desktop card, then arranges the display positions in the corresponding area of each display priority based on the use frequency of the user, so that the problem that the low-frequency high-priority card is divided at a position behind the divided position is avoided, the desktop card can be adaptively adjusted in a targeted manner, meanwhile, the recommended information of the desktop card is clearer, and the highest utilization rate of the display interface is ensured.
In addition, the terminal can adaptively adjust the display layout and the display content of the desktop card based on the screen display state, wherein the screen display state comprises a screen display size and/or a screen available display area.
In one possible implementation, when the display interface is occupied by other controls, such as a video playing window, or the display interface is occupied by a split-screen application, the available display area of the desktop card is limited, and at this time, the terminal performs adaptive adjustment on the display layout of the desktop card again according to the available display area of the current screen. For example, in fig. 13, the original layout manner of the terminal is shown in a first display interface 1301, after a video playing window 1303 is added to the display interface, the terminal performs adaptive adjustment of the display layout again on the desktop cards in the application suggestions, and the adjusted desktop card group 1304 with the key text being "application suggestions" is obtained, and the second display interface 1302 is obtained after adjustment.
In another possible implementation, when the terminal has a folding screen, the size of the display screen may be changed according to the operation of the user, and at this time, adaptive layout of the desktop card is also required.
In the desktop card generation process, the card content of the desktop card is stored in a cache region of the system. When the desktop cards are laid out, the desktop cards to be displayed are selected according to the actual screen size, and the display form and the display position of the desktop cards are planned.
When the screen display area is adjusted, the desktop cards on the original display interface and the desktop cards in the cache area are adjusted in a self-adaptive mode, and integration of the desktop cards is performed again. When the display area of the desktop card is changed from large to small, the terminal further integrates the desktop card, and stores the desktop card which is not displayed in the display area into the buffer area; when the display area of the desktop card is changed from small to large, the terminal executes the integrated reverse process or redisplays the desktop card in the buffer area. If the display area cannot be filled, the terminal can execute the process of generating the desktop cards again, modify the association threshold value and fill more display contents for each desktop card.
As shown in fig. 14, the display interface 1401 is a desktop card display area under a small screen, the second display interface 1402 is a display area of a desktop card after the folding screen is opened, when the desktop card display area is changed from small to large, the terminal releases and displays the desktop cards corresponding to the keywords stored in the buffer area, such as "hot application", "hot good", "daily poetry", and the like, and simultaneously, the terminal re-executes the desktop card generation flow, regenerates the desktop card, adds music information into the card content, and fills the display content of the music playing card. Accordingly, when the size of the display screen becomes smaller, the second display interface 1402 is converted into the display interface 1401, and the desktop card whose display is canceled is stored in the buffer.
Fig. 15 shows a pushing manner of a desktop card according to an exemplary embodiment of the present application, where the manner includes:
step 1501, selecting a target desktop card from desktop cards.
On the one hand, the terminal screens out target desktop cards from the desktop cards based on the control relation between the current equipment and the cross-terminal equipment. Under the condition that the cross-terminal equipment is controlled by the current equipment, the target desktop card does not contain the desktop card for controlling the current equipment.
Under the condition that the desktop card is pushed to the controlled device by the main control device, the terminal screens out the uncontrollable desktop card from the desktop card as a target desktop card due to the lower level of the controlled device.
For example, the current equipment is a smart phone, the cross-terminal equipment is a smart air conditioner, the equipment level of the smart phone is higher than that of the smart air conditioner, and the smart air conditioner is a controlled equipment, so that when the terminal pushes the desktop card from the smart phone of the current equipment to the smart air conditioner, the desktop card in the state of the controllable main control equipment can be filtered, and the uncontrollable desktop card is used as a target desktop card.
Under the condition that the desktop card is sent to the main control equipment by the controlled equipment, the controlled equipment can send all desktop cards to the main control equipment because the main control equipment is higher in grade, and the main control equipment can control the controlled equipment through the controllable desktop card.
For example, the smart refrigerator pushes desktop cards to the smart phone, and because the smart phone equipment level is higher, the smart refrigerator takes all desktop cards as target desktop cards, and a user can control the smart refrigerator by operating the mobile phone.
When one device pushes desktop cards to the other device under the condition that the device grades of the two devices are the same, all desktop cards are used as target desktop cards, and a user can control the other device by controlling any device. For example, in fig. 16, the display interface 1601 is a display interface of a smart phone, the second display interface 1602 is a display interface of a smart watch, and the two devices have the same level, so that a music playing card 1603 capable of controlling the smart phone can be displayed on the smart watch, and meanwhile, the smart phone integrates the sports card 1604 and the weather card 1605 to obtain a desktop card group 1607, which is used as a target desktop card and displayed in the second display interface 1602 of the smart watch.
On the other hand, the terminal determines the type of the target card supported to be displayed by the cross-terminal device, and screens out the target desktop card from the desktop cards based on the type of the target card.
Because the processing performance of different devices is different, the card type supported and displayed by the cross-terminal device is different from that of the current device, before the card pushing, the terminal needs to screen the desktop card of the current device according to the target card type supported by the cross-terminal device and then push the desktop card to the cross-terminal device.
For example, the current device is a smart phone, the cross-terminal device is a smart refrigerator, the smart refrigerator does not support a music playing function, and correspondingly, the music playing desktop card is not supported, so that before the desktop card is pushed, the terminal filters out the music playing desktop card of the smart phone, and then takes other desktop cards as target desktop cards.
In step 1502, the target desktop card is pushed to the cross-end device, so that the cross-end device displays the target desktop card, the cross-end device supports displaying card content in the target desktop card, and the cross-end device has control authority of the target desktop card.
After the terminal determines the target desktop card, pushing the target desktop card to the cross-terminal device for display, wherein the cross-terminal device can support card content in the target desktop card, such as a music playing function, a video playing function, a message notifying function and the like.
After the target desktop card is displayed by the cross-end device, the cross-end device can control the target desktop card, for example, a user can manually remove, stretch, shift and the like the target desktop card in the cross-end device display page.
In the embodiment of the application, after the desktop card can be screened according to the actual condition of the cross-terminal device, the desktop card is pushed to the cross-terminal device for display, so that the cross-device pushing of the desktop card is realized, the application range is wider, the control of the cross-terminal device to the current device can be realized, and the practicability is strong.
After the desktop card is generated, the card content is dynamically updated along with the time and the transformation of the application information. Correspondingly, the terminal can also adjust the integration mode and the layout mode of the card according to the use condition of the display interface. The dynamic update of the card content may change according to the change of the application information, for example, the desktop card content of the music list may be dynamically updated along with the update of the music list in the music playing application. The desktop card content of the home surveillance video can also be changed according to the actual shot video. In addition, the display state of the desktop card can automatically re-render the desktop card according to operations such as dragging, zooming and the like of the card by a user.
The following are device embodiments of the present application, which may be used to perform method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.
Fig. 17 is a block diagram of a display device of a desktop card according to an embodiment of the present application. The apparatus may include:
the acquiring module 1701 is configured to acquire a desktop card, where the desktop card is a carded component for displaying information on a desktop;
The clustering module 1702 is configured to perform card clustering on the desktop card based on the card content of the desktop card to obtain a card clustering result, where the card clustering result includes a plurality of card clusters;
the integration module 1703 is configured to integrate the desktop cards in the same card cluster to obtain a desktop integration component, where the desktop integration component is an independent desktop card or a desktop card group formed by multiple desktop cards;
and the display module 1704 is used for displaying the desktop integration component.
Optionally, the clustering module 1702 includes:
the system comprises a vector acquisition unit, a control unit and a control unit, wherein the vector acquisition unit is used for acquiring a card feature vector of the desktop card, the card feature vector is generated based on a card content vector corresponding to the card content of at least one content dimension, and the content dimension comprises a text and a control;
and the card clustering unit is used for clustering the desktop cards based on the vector distance among the card feature vectors to obtain the card clustering result.
Optionally, the vector obtaining unit is configured to:
under the condition that the desktop card is generated based on an application interface, acquiring the card feature vector of the desktop card in the card generation process;
Under the condition that the desktop card is filled with content based on a card template, vectorizing the card content of at least one content dimension in the desktop card to obtain a card content vector; the card feature vector is generated based on the card content vector.
Optionally, in a case that the desktop card is generated based on the application interface, the apparatus further includes:
the content extraction module is used for extracting key interface content of at least one content dimension from the interface content of the application interface;
the card generation module is used for taking the extracted key interface content as the card content and generating the desktop card;
and the vector generation module is used for generating the card feature vector of the desktop card based on the key interface content.
Optionally, the content extraction module is configured to:
performing text summarization processing on the text to obtain a key text;
and extracting key controls from the controls under the condition that the controls are contained in the application interface.
Optionally, the content extraction module is configured to:
acquiring a control tree of the application interface, wherein the control tree is used for representing the topological structure of each control in the application interface;
Determining a first key control and a second key control associated with the first key control from the control tree;
and extracting the first key control and the second key control.
Optionally, the content extraction module is configured to:
acquiring control information of each control from the control tree, wherein the control information comprises at least one of text information, image information and structural information, and the structural information is used for representing the hierarchical structure of the control in the control;
coding the control information to obtain a control original coding vector of the control;
performing self-attention coding on the original coding vector of the control to obtain a self-attention coding vector of the control;
determining the first key control based on the control self-attention coding vector of the corresponding control of the key text and the control self-attention coding vectors of other controls;
the second key control associated with the first key control is determined based on the control self-attention encoding vector of the first key control and the control self-attention encoding vectors of other controls.
Optionally, the vector generation module is configured to:
Vectorizing the key text to obtain a text feature vector;
and generating the card feature vector of the desktop card based on the text feature vector and the control self-attention coding vector of the key control.
Optionally, the integration module 1703 is configured to:
integrating the plain text desktop cards based on texts contained in the plain text desktop cards in the same card type cluster to obtain integrated desktop cards, wherein the integrated desktop cards are independent desktop cards, and the plain text desktop cards do not contain controls;
and integrating the controllable desktop cards in the same card cluster into the desktop card group, wherein the controllable desktop cards comprise texts and controls.
Optionally, the apparatus further includes:
the relative loading factor determining module is used for determining a relative loading factor based on a current card integration proportion and a card display duty ratio, wherein the current card integration proportion is used for representing the current integration degree of the desktop card, and the card display duty ratio is used for representing the screen duty ratio of the desktop card currently displayed;
the integration module 1703 is configured to:
and under the condition that the relative loading factor is larger than a threshold value, integrating the desktop cards in the same card cluster to obtain the desktop integration component.
Optionally, the display module 1704 is configured to:
determining a target display area to which the desktop integration component belongs based on card content of the desktop card in the desktop integration component, wherein different display areas correspond to different display priorities, and the display priorities are related to the card content;
determining a target display position of the desktop integration component in the target display area based on the historical use frequency of the desktop card in the desktop integration component, wherein different display positions correspond to different display priorities, and the display priorities and the historical use frequency are in positive correlation;
displaying the desktop integration component at the target display location of the target display area.
Optionally, the apparatus further includes:
and the self-adapting module is used for carrying out self-adapting adjustment on the display layout and the display content of the desktop card based on the screen display state, wherein the screen display state comprises a screen display size and/or a screen available display area.
Optionally, the apparatus further includes:
the card screening module is used for screening target desktop cards from the desktop cards;
the card pushing module is used for pushing the target desktop card to the cross-terminal equipment so that the cross-terminal equipment can display the target desktop card, the cross-terminal equipment supports to display card content in the target desktop card, and the cross-terminal equipment has control authority of the target desktop card.
Optionally, the card screening module is configured to:
screening the target desktop card from the desktop cards based on the control relation between the current equipment and the cross-terminal equipment, wherein the target desktop card does not contain the desktop card for controlling the current equipment under the condition that the cross-terminal equipment is controlled by the current equipment;
determining the type of the target card supported to be displayed by the cross-terminal equipment; and screening the target desktop card from the desktop cards based on the target card type.
Fig. 18 is a block diagram illustrating a structure of a terminal according to an exemplary embodiment of the present application. The terminal 1800 may be implemented as a terminal in the various embodiments described above. The terminal 1800 may include one or more of the following components: a processor 1810 and a memory 1820.
The processor 1810 may include one or more processing cores. The processor 1810 uses various interfaces and circuits to connect various components within the overall terminal 1800, performing various functions and processing data for the terminal 1800 by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 1820, and invoking data stored in the memory 1820. Alternatively, the processor 1810 may be implemented in hardware in at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 1810 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), a Neural network processor (Neural-network Processing Unit, NPU), a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the touch display screen; the NPU is used to implement artificial intelligence (Artificial Intelligence, AI) functionality; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 1810 and may be implemented on a single chip.
The Memory 1820 may include random access Memory (Random Access Memory, RAM) or Read-Only Memory (ROM). Optionally, the memory 1820 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 1820 may be used to store instructions, programs, code sets, or instruction sets. The memory 1820 may include a stored program area that may store instructions for implementing an operating system, instructions for at least one function (e.g., a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described below, etc.; the storage data area may store data (e.g., audio data, phonebook) created according to the use of the terminal 1800, etc.
In addition, those skilled in the art will appreciate that the structure of the terminal 1800 illustrated in the above-described figures is not limiting and that the terminal may include more or fewer components than illustrated, or may combine certain components, or a different arrangement of components. For example, the terminal 1800 may further include a display screen, a camera module, a microphone, a speaker, a radio frequency circuit, an input unit, a sensor (such as an acceleration sensor, an angular velocity sensor, a light sensor, etc.), an audio circuit, a WiFi module, a power supply, a bluetooth module, etc., which are not described herein.
Embodiments of the present application also provide a computer readable storage medium storing at least one program code that is loaded and executed by a processor to implement the desktop card display method according to the above embodiments.
Embodiments of the present application provide a computer program product comprising computer instructions stored in a computer-readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the method of displaying a desktop card provided in various alternative implementations of the above aspects.
It should be understood that references herein to "a plurality" are to two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. In addition, the step numbers described herein are merely exemplary of one possible execution sequence among steps, and in some other embodiments, the steps may be executed out of the order of numbers, such as two differently numbered steps being executed simultaneously, or two differently numbered steps being executed in an order opposite to that shown, which is not limited by the embodiments of the present application.
The foregoing description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, since it is intended that all modifications, equivalents, improvements, etc. that fall within the spirit and scope of the invention.

Claims (24)

1. A method for displaying a desktop card, the method comprising:
under the condition of generating a desktop card based on an application interface, performing text summarization processing on a text of the application interface to obtain a key text;
under the condition that the application interface comprises controls, a control tree of the application interface is obtained, wherein the control tree is used for representing the topological structure of each control in the application interface;
determining a first key control and a second key control associated with the first key control from the control tree, wherein the key text and the key control belong to key interface content;
extracting the first key control and the second key control;
taking the extracted key interface content as card content, and generating the desktop card, wherein the desktop card is a card assembly for information display on a desktop;
acquiring the desktop card;
Based on the card content of the desktop card, carrying out card clustering on the desktop card to obtain a card clustering result, wherein the card clustering result comprises a plurality of card clusters;
integrating the desktop cards in the same card cluster to obtain a desktop integration component, wherein the desktop integration component is an independent desktop card or a desktop card group formed by a plurality of desktop cards;
and displaying the desktop integration component.
2. The method of claim 1, wherein the clustering the desktop cards based on the card content of the desktop cards comprises:
acquiring a card feature vector of the desktop card, wherein the card feature vector is generated based on a card content vector corresponding to the card content of at least one content dimension, and the content dimension comprises a text and a control;
and clustering the desktop cards based on the vector distance between the card feature vectors to obtain the card clustering result.
3. The method of claim 2, wherein the obtaining the card feature vector of the desktop card comprises:
under the condition that the desktop card is generated based on the application interface, acquiring the card feature vector of the desktop card in the card generation process;
Under the condition that the desktop card is filled with content based on a card template, vectorizing the card content of at least one content dimension in the desktop card to obtain a card content vector; the card feature vector is generated based on the card content vector.
4. The method of claim 1, wherein the determining a first key control from the control tree and a second key control associated with the first key control comprises:
acquiring control information of each control from the control tree, wherein the control information comprises at least one of text information, image information and structural information, and the structural information is used for representing the hierarchical structure of the control in the control;
coding the control information to obtain a control original coding vector of the control;
performing self-attention coding on the original coding vector of the control to obtain a self-attention coding vector of the control;
determining the first key control based on the control self-attention coding vector of the corresponding control of the key text and the control self-attention coding vectors of other controls;
The second key control associated with the first key control is determined based on the control self-attention encoding vector of the first key control and the control self-attention encoding vectors of other controls.
5. The method of claim 2, wherein the obtaining the card feature vector of the desktop card comprises:
vectorizing the key text to obtain a text feature vector;
and generating the card feature vector of the desktop card based on the text feature vector and the control self-attention coding vector of the key control.
6. The method of claim 1, wherein the integrating the desktop cards in the same card cluster results in a desktop integration assembly comprising:
integrating the plain text desktop cards based on texts contained in the plain text desktop cards in the same card type cluster to obtain integrated desktop cards, wherein the integrated desktop cards are independent desktop cards, and the plain text desktop cards do not contain controls;
and integrating the controllable desktop cards in the same card cluster into the desktop card group, wherein the controllable desktop cards comprise texts and controls.
7. The method according to claim 1, wherein the method further comprises:
determining a relative loading factor based on a current card integration ratio used for representing the current integration degree of the desktop card and a card display duty ratio used for representing the screen duty ratio of the desktop card currently displayed;
the step of integrating the desktop cards in the same card cluster to obtain a desktop integration assembly comprises the following steps:
and under the condition that the relative loading factor is larger than a threshold value, integrating the desktop cards in the same card cluster to obtain the desktop integration component.
8. The method of claim 1, wherein the displaying the desktop integration component comprises:
determining a target display area to which the desktop integration component belongs based on card content of the desktop card in the desktop integration component, wherein different display areas correspond to different display priorities, and the display priorities are related to the card content;
determining a target display position of the desktop integration component in the target display area based on the historical use frequency of the desktop card in the desktop integration component, wherein different display positions correspond to different display priorities, and the display priorities and the historical use frequency are in positive correlation;
Displaying the desktop integration component at the target display location of the target display area.
9. The method according to claim 1, wherein the method further comprises:
and carrying out self-adaptive adjustment on the display layout and the display content of the desktop card based on a screen display state, wherein the screen display state comprises a screen display size and/or a screen available display area.
10. The method according to claim 1, wherein the method further comprises:
screening target desktop cards from the desktop cards;
pushing the target desktop card to a cross-terminal device so that the cross-terminal device can display the target desktop card, the cross-terminal device supports displaying card content in the target desktop card, and the cross-terminal device has control authority of the target desktop card.
11. The method of claim 10, wherein the screening the target desktop card from the desktop cards comprises:
screening the target desktop card from the desktop cards based on the control relation between the current equipment and the cross-terminal equipment, wherein the target desktop card does not contain the desktop card for controlling the current equipment under the condition that the cross-terminal equipment is controlled by the current equipment;
Determining the type of the target card supported to be displayed by the cross-terminal equipment; and screening the target desktop card from the desktop cards based on the target card type.
12. A display device for a desktop card, the device comprising:
the content extraction module is used for carrying out text summarization processing on the text of the application interface under the condition that a desktop card is generated based on the application interface, so as to obtain a key text;
the content extraction module is further configured to obtain a control tree of the application interface when the application interface includes a control, where the control tree is used to characterize a topology structure of each control in the application interface;
the content extraction module is further configured to determine a first key control and a second key control associated with the first key control in the control tree, where the key text and the key control belong to key interface content;
the content extraction module is further configured to extract the first key control and the second key control;
the card generation module is used for taking the extracted key interface content as card content and generating the desktop card, wherein the desktop card is a card assembly used for information display on a desktop;
The acquisition module is used for acquiring the desktop card;
the clustering module is used for carrying out card clustering on the desktop cards based on the card content of the desktop cards to obtain card clustering results, wherein the card clustering results comprise a plurality of card clusters;
the desktop card integration module is used for integrating the desktop cards in the same card cluster to obtain a desktop integration component, wherein the desktop integration component is an independent desktop card or a desktop card group formed by a plurality of desktop cards;
and the display module is used for displaying the desktop integration assembly.
13. The apparatus of claim 12, wherein the clustering module comprises:
the system comprises a vector acquisition unit, a control unit and a control unit, wherein the vector acquisition unit is used for acquiring a card feature vector of the desktop card, the card feature vector is generated based on a card content vector corresponding to the card content of at least one content dimension, and the content dimension comprises a text and a control;
and the card clustering unit is used for clustering the desktop cards based on the vector distance among the card feature vectors to obtain the card clustering result.
14. The apparatus of claim 13, wherein the vector acquisition unit is configured to:
Under the condition that the desktop card is generated based on the application interface, acquiring the card feature vector of the desktop card in the card generation process;
under the condition that the desktop card is filled with content based on a card template, vectorizing the card content of at least one content dimension in the desktop card to obtain a card content vector; the card feature vector is generated based on the card content vector.
15. The apparatus of claim 12, wherein the content extraction module is configured to:
acquiring control information of each control from the control tree, wherein the control information comprises at least one of text information, image information and structural information, and the structural information is used for representing the hierarchical structure of the control in the control;
coding the control information to obtain a control original coding vector of the control;
performing self-attention coding on the original coding vector of the control to obtain a self-attention coding vector of the control;
determining the first key control based on the control self-attention coding vector of the corresponding control of the key text and the control self-attention coding vectors of other controls;
The second key control associated with the first key control is determined based on the control self-attention encoding vector of the first key control and the control self-attention encoding vectors of other controls.
16. The apparatus of claim 13, wherein the vector acquisition unit is configured to:
vectorizing the key text to obtain a text feature vector;
and generating the card feature vector of the desktop card based on the text feature vector and the control self-attention coding vector of the key control.
17. The apparatus of claim 12, wherein the integration module is configured to:
integrating the plain text desktop cards based on texts contained in the plain text desktop cards in the same card type cluster to obtain integrated desktop cards, wherein the integrated desktop cards are independent desktop cards, and the plain text desktop cards do not contain controls;
and integrating the controllable desktop cards in the same card cluster into the desktop card group, wherein the controllable desktop cards comprise texts and controls.
18. The apparatus of claim 12, wherein the apparatus further comprises:
The relative loading factor determining module is used for determining a relative loading factor based on a current card integration proportion and a card display duty ratio, wherein the current card integration proportion is used for representing the current integration degree of the desktop card, and the card display duty ratio is used for representing the screen duty ratio of the desktop card currently displayed;
the integration module is used for:
and under the condition that the relative loading factor is larger than a threshold value, integrating the desktop cards in the same card cluster to obtain the desktop integration component.
19. The apparatus of claim 12, wherein the display module is configured to:
determining a target display area to which the desktop integration component belongs based on card content of the desktop card in the desktop integration component, wherein different display areas correspond to different display priorities, and the display priorities are related to the card content;
determining a target display position of the desktop integration component in the target display area based on the historical use frequency of the desktop card in the desktop integration component, wherein different display positions correspond to different display priorities, and the display priorities and the historical use frequency are in positive correlation;
Displaying the desktop integration component at the target display location of the target display area.
20. The apparatus of claim 12, wherein the apparatus further comprises:
and the self-adapting module is used for carrying out self-adapting adjustment on the display layout and the display content of the desktop card based on the screen display state, wherein the screen display state comprises a screen display size and/or a screen available display area.
21. The apparatus of claim 12, wherein the apparatus further comprises:
the card screening module is used for screening target desktop cards from the desktop cards;
the card pushing module is used for pushing the target desktop card to the cross-terminal equipment so that the cross-terminal equipment can display the target desktop card, the cross-terminal equipment supports to display card content in the target desktop card, and the cross-terminal equipment has control authority of the target desktop card.
22. The apparatus of claim 21, wherein the card screening module is configured to:
screening the target desktop card from the desktop cards based on the control relation between the current equipment and the cross-terminal equipment, wherein the target desktop card does not contain the desktop card for controlling the current equipment under the condition that the cross-terminal equipment is controlled by the current equipment;
Determining the type of the target card supported to be displayed by the cross-terminal equipment; and screening the target desktop card from the desktop cards based on the target card type.
23. A terminal, the terminal comprising a processor and a memory; the memory stores at least one instruction for execution by the processor to implement the method of displaying a desktop card as claimed in any one of claims 1 to 11.
24. A computer readable storage medium having stored therein at least one program code loaded and executed by a processor to implement a method of displaying a desktop card as claimed in any one of claims 1 to 11.
CN202211117854.9A 2022-09-14 2022-09-14 Desktop card display method, device, terminal, storage medium and program product Active CN115309299B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211117854.9A CN115309299B (en) 2022-09-14 2022-09-14 Desktop card display method, device, terminal, storage medium and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211117854.9A CN115309299B (en) 2022-09-14 2022-09-14 Desktop card display method, device, terminal, storage medium and program product

Publications (2)

Publication Number Publication Date
CN115309299A CN115309299A (en) 2022-11-08
CN115309299B true CN115309299B (en) 2024-02-23

Family

ID=83866806

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211117854.9A Active CN115309299B (en) 2022-09-14 2022-09-14 Desktop card display method, device, terminal, storage medium and program product

Country Status (1)

Country Link
CN (1) CN115309299B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116420989B (en) * 2023-06-09 2023-08-22 广州美术学院 Intelligent office workstation

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107733988A (en) * 2017-08-18 2018-02-23 上海掌门科技有限公司 A kind of method and apparatus for being used to provide desktop
CN108803973A (en) * 2018-06-01 2018-11-13 西安领讯卓越信息技术有限公司 A kind of video information component groups display methods for mobile terminal
CN108804491A (en) * 2018-03-27 2018-11-13 优视科技新加坡有限公司 item recommendation method, device, computing device and storage medium
CN111176766A (en) * 2019-12-16 2020-05-19 青岛海信移动通信技术股份有限公司 Communication terminal and component display method
CN111752647A (en) * 2019-03-27 2020-10-09 西安欧思奇软件有限公司 Card information display method and device, computer equipment and storage medium
CN112068744A (en) * 2020-08-26 2020-12-11 深圳传音控股股份有限公司 Interaction method, mobile terminal and storage medium
CN113434235A (en) * 2021-06-30 2021-09-24 大众问问(北京)信息科技有限公司 Vehicle-mounted desktop display method and device, computer equipment and storage medium
CN113778574A (en) * 2020-06-08 2021-12-10 荣耀终端有限公司 Card sharing method, electronic equipment and communication system
CN114510170A (en) * 2021-03-04 2022-05-17 聚好看科技股份有限公司 Component display method and display device
CN114816610A (en) * 2021-01-29 2022-07-29 华为技术有限公司 Page classification method, page classification device and terminal equipment
CN114842270A (en) * 2022-05-31 2022-08-02 中冶赛迪技术研究中心有限公司 Target image classification method and device, electronic equipment and medium
CN115033331A (en) * 2022-06-28 2022-09-09 Oppo广东移动通信有限公司 Card display method and related product

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8578330B2 (en) * 2007-06-11 2013-11-05 Sap Ag Enhanced widget composition platform
US9824159B2 (en) * 2011-03-17 2017-11-21 Red Hat, Inc. Assigning labels to desktop items

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107733988A (en) * 2017-08-18 2018-02-23 上海掌门科技有限公司 A kind of method and apparatus for being used to provide desktop
CN108804491A (en) * 2018-03-27 2018-11-13 优视科技新加坡有限公司 item recommendation method, device, computing device and storage medium
CN108803973A (en) * 2018-06-01 2018-11-13 西安领讯卓越信息技术有限公司 A kind of video information component groups display methods for mobile terminal
CN111752647A (en) * 2019-03-27 2020-10-09 西安欧思奇软件有限公司 Card information display method and device, computer equipment and storage medium
CN111176766A (en) * 2019-12-16 2020-05-19 青岛海信移动通信技术股份有限公司 Communication terminal and component display method
CN113778574A (en) * 2020-06-08 2021-12-10 荣耀终端有限公司 Card sharing method, electronic equipment and communication system
CN112068744A (en) * 2020-08-26 2020-12-11 深圳传音控股股份有限公司 Interaction method, mobile terminal and storage medium
CN114816610A (en) * 2021-01-29 2022-07-29 华为技术有限公司 Page classification method, page classification device and terminal equipment
CN114510170A (en) * 2021-03-04 2022-05-17 聚好看科技股份有限公司 Component display method and display device
CN113434235A (en) * 2021-06-30 2021-09-24 大众问问(北京)信息科技有限公司 Vehicle-mounted desktop display method and device, computer equipment and storage medium
CN114842270A (en) * 2022-05-31 2022-08-02 中冶赛迪技术研究中心有限公司 Target image classification method and device, electronic equipment and medium
CN115033331A (en) * 2022-06-28 2022-09-09 Oppo广东移动通信有限公司 Card display method and related product

Also Published As

Publication number Publication date
CN115309299A (en) 2022-11-08

Similar Documents

Publication Publication Date Title
WO2020221013A1 (en) Image processing method and apparaus, and electronic device and storage medium
US8869211B2 (en) Zoomable content recommendation system
KR20180055708A (en) Device and method for image processing
WO2012169135A1 (en) Information processing device, information processing method and computer program product
CN108595573B (en) Page display method and device, storage medium and electronic equipment
CN115309299B (en) Desktop card display method, device, terminal, storage medium and program product
KR20190088406A (en) Video playback device and controlling method thereof
CN111814475A (en) User portrait construction method and device, storage medium and electronic equipment
CN102982054A (en) Information processor, information processing method and program
KR101646926B1 (en) Method and system of deep concept hioerarchy for reconstruction in multi-modality data
CN114548276A (en) Method and device for clustering data, electronic equipment and storage medium
CN116227629B (en) Information analysis method, model training method, device and electronic equipment
CN115935049A (en) Recommendation processing method and device based on artificial intelligence and electronic equipment
CN113313066A (en) Image recognition method, image recognition device, storage medium and terminal
CN116957678A (en) Data processing method and related device
US20220326846A1 (en) Electronic device and method to provide sticker based on content input
CN115562556A (en) Interface display method, device, terminal and storage medium
CN111797874A (en) Behavior prediction method, behavior prediction device, storage medium and electronic equipment
CN115454554A (en) Text description generation method, text description generation device, terminal and storage medium
CN111915701A (en) Button image generation method and device based on artificial intelligence
CN110490084A (en) Detection method, device, the network equipment and the storage medium of target object
CN117236347B (en) Interactive text translation method, interactive text display method and related device
CN110222590B (en) Image difference judgment method and device and electronic equipment
CN112908319B (en) Method and equipment for processing information interaction
US20240086051A1 (en) Editing User Interfaces using Free Text

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant