CN113283921A - Business data processing method and device and cloud server - Google Patents

Business data processing method and device and cloud server Download PDF

Info

Publication number
CN113283921A
CN113283921A CN202010103095.5A CN202010103095A CN113283921A CN 113283921 A CN113283921 A CN 113283921A CN 202010103095 A CN202010103095 A CN 202010103095A CN 113283921 A CN113283921 A CN 113283921A
Authority
CN
China
Prior art keywords
data
vertex
individuals
individual
matrix
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010103095.5A
Other languages
Chinese (zh)
Inventor
刘陈魏
万艺
沈驰雄
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202010103095.5A priority Critical patent/CN113283921A/en
Publication of CN113283921A publication Critical patent/CN113283921A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201Market modelling; Market analysis; Collecting market data
    • G06Q30/0203Market surveys; Market polls
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/23Clustering techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/048Activation functions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Finance (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Strategic Management (AREA)
  • Molecular Biology (AREA)
  • Development Economics (AREA)
  • Accounting & Taxation (AREA)
  • Health & Medical Sciences (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Game Theory and Decision Science (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

In the method, after business data with multilayer incidence relation is obtained, the business data can be converted into input data with a first structure, the input data with the first structure can be operated by using a preset neural network, and then the converted data is processed by using the preset neural network, so that a result of processing the business data is obtained, the effect of processing the business data with multiple structures through artificial intelligence is achieved, and the processing efficiency of the business data can be improved.

Description

Business data processing method and device and cloud server
Technical Field
The application relates to the technical field of computers, in particular to a business data processing method and device and a cloud server.
Background
Artificial Intelligence (AI) is a theory, method, technique and application that utilizes a digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results, it attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making. Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision and reasoning, human-computer interaction, recommendation and search, AI basic theory, and the like.
As an example, the artificial intelligence may divide the acquired multiple service data (for example, image data, log data, and the like) into different groups or different subsets through a clustering technique, and then perform analysis processing to obtain data characteristics of the service data in the same group or the same subset, so as to perform decision making or related recommendation, and the like.
However, the clustering technology can only process service data with a single-layer structure, and in an actual use scenario, more service data with multiple structures are processed. For example, in the user consumption behavior data, each user may purchase multiple types of goods, and each good has certain attributes, such as price, purchase quantity, production place, etc., so that, in the consumption behavior data of different users, there may be association among multiple aspects of the goods type, price, purchase quantity, production place, i.e. multiple structures of business data.
Therefore, how to process the business data with the multiple structures by artificial intelligence is a technical problem to be solved urgently at present.
Disclosure of Invention
The application provides a method and a device for processing service data and a cloud server, which are used for processing the service data with multiple structures.
In a first aspect, a method for processing service data is provided, where the service data is first obtained, where the service data may be understood as service data to be processed, and the service data includes a plurality of individual data respectively corresponding to a plurality of individuals when performing a service, where an individual may also be understood as an individual performing the service, for example, in a retail research scenario, each user purchasing a commodity may be regarded as an individual, or may be understood as an execution object corresponding to the service, and for example, in a base station configuration scenario, each base station that needs to be configured may be regarded as an individual. Each individual data includes at least one sub-individual, which can be understood as a sub-object necessary for the individual to complete the business, for example, in a retail investigation scene, each user must purchase a commodity to complete a consumption behavior, and the commodity is the sub-individual; for another example, in a base station configuration scenario, each base station must be configured to use a different outsourcing service, and the outsourcing service is a sub-individual. Moreover, a plurality of individual data included in the business data have a multilayer incidence relation.
Then, the service data is converted into input data having a first structure, thereby obtaining input data corresponding to the service data, the first structure including vertices having feature information, each vertex corresponding to one of the plurality of individuals of the service data, the feature information of one vertex including information determined from vertex sub-individuals, the vertex sub-individuals being understood as sub-individuals included in the individual data of the individual corresponding to the vertex, and for example, may include information determined from attributes of the vertex sub-individuals, and one connecting edge including information determined from a multi-layer association relationship between the plurality of individual data.
After the input data with the first structure is obtained, a preset neural network is used for calculating the input data, and the class of each individual in a plurality of individuals included in the business data is obtained.
In the technical scheme, after the service data (which can be understood as data of a multi-structure) with a multi-layer incidence relation is obtained, the service data is converted into data which can be calculated by using a preset neural network, and then the converted data is processed by using the preset neural network, so that a result of processing the service data is obtained, the effect of processing the service data of the multi-structure through artificial intelligence is realized, and the processing efficiency of the service data can be improved.
In a possible design, a plurality of vertices of the input data may be obtained according to a plurality of individuals included in the service data, each vertex includes feature information of the vertex, at least one connecting edge between the plurality of vertices is determined according to at least one layer of incidence relation in the multi-layer incidence relation of the service data, and finally, the input data corresponding to the service data is obtained according to the plurality of vertices and the at least one connecting edge.
In the above technical solution, each vertex in the input data is not an isolated point, and the vertex also retains the feature information of each individual data, so that the input data can better retain the features of each individual, and because the input data further includes a connecting edge determined according to at least one layer association relationship, the multi-layer association relationship of the business data can be stored in the input data as much as possible, and the input data can better retain the relationship between the individuals in the business data.
In one possible design, the manner of determining at least one connected edge between the plurality of vertices according to at least one layer of incidence relation in the multi-layer incidence relation may include, but is not limited to, the following:
first, according to at least one layer of the multi-layer association relationship, specifically, according to one layer of the multi-layer association relationship or the multi-layer association relationship, shared information between every two vertices is determined, and then, according to the shared information between every two vertices, at least one connecting edge between the multiple vertices is obtained.
In the technical scheme, the continuous edges are determined through the shared information between the vertexes, and the association relation between the individuals can be accurately acquired. For example, in a retail investigation scenario, where one layer of association may be a commodity purchased by consumption behavior data of each user, shared information between two vertices corresponding to two users may be determined according to whether the two users purchase the same commodity, and then a connecting edge may be determined. For another example, in a base station configuration scenario, where one layer of association relationship may be to configure an outsourcing service used by each base station, shared information between two vertices corresponding to two base stations may be determined according to whether the two base stations need to use the same outsourcing service, and then a connection edge may be determined.
In a possible design, the input data further includes a type of each of at least one of the connected edges, the type of each of the connected edges is determined according to the content of the shared information between two vertices connected by the connected edge, and if the content of the shared information corresponding to two connected edges is the same, the types of the two connected edges are the same.
In the above technical solution, the input data can completely retain the characteristics of the multiple structures of the service data by carrying the type of each connecting edge in the input data.
In a possible design, the content of the shared information includes the same sub-individuals included in the individual data of the two individuals respectively corresponding to the two vertices connected by one connecting edge. For example, in a retail investigation scene, if a user a, a user B, and a user C all purchase a commodity X, a connecting edge is included between the user a and the user B, a connecting edge is included between the user B and the user C, the user a and the user C also include a connecting edge, and since shared information corresponding to each connecting edge is that the commodity X is purchased, the three connecting edges are of the same type.
In the above technical solution, whether a connecting edge exists between two individuals may be determined by whether the individual data corresponding to the two individuals includes the same sub-individual, or may be determined in other manners, which is only an example.
In one possible design, the total number of occurrences of the same sub-individual in the plurality of individual data is less than or equal to a preset threshold. The preset threshold may be preset, for example, to 2; the total number of individuals included in the service data and/or the total number of types of shared information between individuals may also be determined, for example, the preset threshold is smaller than the total number of individuals, and if the total number of individuals is 3, the preset threshold may be 1 or 2, and the like. In the above example, if the retail investigation scene includes only 3 users, i.e., user a to user C, and all of the 3 users purchase the product X, assuming that the preset threshold is 2, since the total number of times that the product X appears in the individual data is 3, which is greater than the preset threshold, when determining the connecting edge between the 3 vertices, the product X needs to be excluded, that is, if the user a and the user B purchase only the product X, in this case, there is no shared information between two vertices corresponding to the user a and the user B, respectively.
In the above technical solution, in order to avoid that the plurality of vertices are connected pairwise and the plurality of vertices are not unique, the sub-individuals used for determining whether there is a connection between two individuals may be that the total number of occurrences in the plurality of individual data is less than or equal to a threshold, where the threshold is not limited, that is, the sub-individuals shared by the plurality of vertices may be eliminated, so as to ensure the uniqueness between the vertices.
In one possible design, after the input data is obtained, an input matrix and a continuous matrix corresponding to the input data are obtained, where elements in a column vector or a row vector of the input matrix are used to indicate feature information corresponding to a vertex in the input data and an initial type of the vertex, that is, the feature information and the initial type of each vertex are used as a row vector or a column vector of the input matrix, the initial type is determined according to a preset judgment condition, which may be preset or determined according to business data, for example, in a retail investigation scenario, the preset condition may be a magnitude relationship between a total price of a commodity purchased by a user and a preset price, and if the total price is greater than the preset price, the first category is a first category, otherwise, for example, in a base station configuration scenario, the preset condition may be that a size relationship between the number of outsourcing services used by each base station and a preset number is configured, and if the number of used outsourcing services is greater than the preset number, the type is the first type, otherwise, the type is the second type. It should be noted that the type is only used to distinguish whether each data is the same type, and is not used to distinguish which type. Elements in a row vector or a column vector of the edge matrix are used to indicate a type of an edge included in one vertex of the input data, and as an example, there may be a correspondence between the row vector or the column vector of the input matrix and the row vector or the column vector of the edge matrix. For example, if the feature information and the initial type of the first vertex are used as the first row vector of the input matrix, the first row vector of the edge-connected matrix is used to indicate the type of the edge-connected included in the vertex; and if the feature information and the initial type of the first vertex are taken as the first column vector of the input matrix, indicating the type of the connected edge included by the vertex by using the first column vector of the connected edge matrix. As another example, there may be no correspondence between the row vector or the column vector of the input matrix and the row vector or the column vector of the edge matrix. For example, the first row vector of the input matrix is used to indicate the feature information and the initial type of the first vertex, and the third column vector of the edge matrix is used to indicate the type of the edge included in the first vertex, which does not limit the relationship between the input matrix and the edge matrix in the present application. Then, according to the edge matrix, taking the class to which the plurality of individuals belong as a loss function of the preset neural network, and using the preset neural network to calculate the input matrix to obtain the class to which each individual in the plurality of individuals included in the service data belongs.
In the technical scheme, after the input data is divided into the input matrix and the continuous edge matrix, the preset neural network is trained, so that the class to which each individual belongs is obtained, and the processing mode is simple. Of course, the input data may be trained in other ways, and is not limited herein.
In a possible design, a preset neural network may be used to train input data, and a class to which each individual belongs may be obtained directly according to an output result of the preset neural network, or may be processed in the following manner:
firstly, according to the continuous edge matrix, taking the class to which the plurality of individuals belong as a loss function of the preset neural network, using the preset neural network to calculate the input matrix, and then obtaining an output matrix of the last hidden layer of the preset neural network, wherein a row vector in the output matrix is a state vector corresponding to each individual. And finally, clustering the output matrix to obtain the classes of the plurality of individuals.
In the technical scheme, the neural network and the clustering process are combined to process the input data, so that the accuracy of the processing result can be improved.
In a possible design, considering that a density-based clustering method has a certain tolerance to noise and abnormal points, in order to improve the accuracy of the processing result, the output matrix may be clustered using the density-based clustering method.
In one possible design, the output matrix may be subjected to dimensionality reduction, and then the output matrix may be clustered in a density-based clustering manner. By the dimension reduction processing, the amount of calculation can be reduced, and the processing efficiency can be improved.
In a second aspect, an apparatus for processing traffic data is provided, the apparatus including means for performing the method of the first aspect. As an example, the apparatus includes a first obtaining module, a second obtaining module, and a processing module. Wherein:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring service data, the service data comprises a plurality of individual data which respectively correspond to a plurality of individuals when the service is executed, each individual data comprises at least one sub-individual, and a multilayer incidence relation exists among the plurality of individual data;
a second obtaining module, configured to obtain input data corresponding to the service data, where the input data has a first structure, where the first structure includes vertices having feature information and multiple types of connected edges, each vertex corresponds to one of the multiple individuals, the feature information of one vertex includes information determined according to a vertex sub-individual, the vertex sub-individual is a sub-individual included in individual data of the individual corresponding to the vertex, and the connected edges include information determined according to the multilayer association relationship between the multiple individual data;
and the processing module is used for calculating the input data by using a preset neural network to acquire the classes to which the individuals belong.
In a third aspect, an apparatus for processing service data is provided, the apparatus including: a memory for storing a program; a processor for executing the memory-stored program, the processor being adapted to perform the method of the first aspect when the memory-stored program is executed.
In a fourth aspect, a cloud server is provided, which includes: data collection station, memory, preprocessor and arithmetic circuit, wherein:
the data acquisition unit is used for acquiring service data and storing the service data in a memory, the service data comprises a plurality of individual data respectively corresponding to a plurality of individuals when the service is executed, each individual data comprises at least one sub-individual, and a multilayer incidence relation exists among the individual data;
the preprocessor is used for acquiring the business data from the memory and acquiring input data corresponding to the business data, wherein the input data has a first structure, the first structure comprises vertexes with characteristic information and multiple types of connecting edges, each vertex corresponds to one of the multiple individuals, the characteristic information of one vertex comprises information determined according to vertex sub-individuals, the vertex sub-individuals are sub-individuals included in the individual data of the individual corresponding to the vertex, and the connecting edges comprise information determined according to the multilayer incidence relation among the multiple individual data;
the operation circuit is used for operating the input data by using a preset neural network to obtain the class to which the plurality of individuals belong.
In a fifth aspect, there is provided a computer readable medium storing program code for execution by a device, the program code comprising instructions for performing the method of the first aspect.
In a sixth aspect, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the method of the first aspect described above.
In a seventh aspect, a chip is provided, where the chip includes a processor and a data interface, and the processor reads instructions stored in a memory through the data interface to perform the method in the first aspect.
In one possible design, the chip may further include a memory having instructions stored therein, and the processor is configured to execute the instructions stored on the memory, and when the instructions are executed, the processor is configured to perform the method of the first aspect.
In an eighth aspect, an electronic device is provided, which includes the service data processing apparatus in any one of the second aspect to the fourth aspect.
Drawings
FIG. 1 is a schematic diagram of one example of data of a graph structure;
fig. 2 is a schematic structural diagram of an example of the neural network GNN;
fig. 3 is a schematic diagram of an example of a chip hardware structure for performing the service data processing method according to an embodiment of the present application;
fig. 4 is a flowchart of an example of a method for processing service data according to an embodiment of the present application;
fig. 5 is a flowchart of an example of a manner of obtaining data of a graph structure corresponding to the service data according to an embodiment of the present application;
FIG. 6 is a diagram illustrating an example of graph structured data provided by an embodiment of the present application;
FIG. 7 is a diagram illustrating another example of graph structured data provided by an embodiment of the present application;
FIG. 8 is a diagram illustrating another example of graph structured data provided by an embodiment of the present application;
fig. 9 is a schematic diagram of an example of a service data processing apparatus provided in an embodiment of the present application;
fig. 10 is a schematic diagram of another example of a service data processing apparatus according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.
The method for processing the service data can be applied to the scenes of pattern recognition, image processing, retail investigation, text classification and classification of data with multiple structures. Specifically, the service data processing method in the embodiment of the application can be used in a retail investigation scene and a base station configuration scene. In the following, a brief description is given of a retail investigation scenario and a base station configuration scenario, respectively.
Retail investigation scenario:
the retail investigation scene mainly analyzes consumption behavior data generated in the process of executing consumption business by a user. During the process of executing the consuming business, each user may purchase a plurality of commodities, for example, user 1 purchases 1 commodity a, 2 commodities B and 3 commodities C, user 2 purchases 1 commodity a and 1 commodity D, and user 3 purchases 2 commodities C and E. Each item has multiple attributes, which may include price, place of production, etc., and each item may have different attributes. The type and quantity of the goods purchased by each user and the attribute of each goods are recorded, and the consumption behavior data shown in table 1 can be obtained. Table 1 illustrates an example in which the attribute of a product is the unit price of the product.
TABLE 1
Figure BDA0002387520380000061
In a retail investigation scenario, after the consumption behavior data of the user is obtained, the consumption behavior data is usually processed and analyzed to obtain the consumption behavior characteristics of each user, where the consumption behavior characteristics may be the consumption capability of the user, or what type of goods the user prefers to purchase, and the like. Then, according to the consumption behavior characteristics of each user, different users can be recommended with commodities matched with the consumption behavior characteristics or different sales strategies can be made.
A base station configuration scene:
configuring a base station often requires purchasing multiple outsourcing services. Different configurations of base stations, different types of outsourcing services are required and the number of outsourcing services of each type is different, each having a different price etc. For example, base station 1 needs to purchase Q11 outsourcing services a1 and Q12 outsourcing services a2, base station 2 needs to purchase Q22 outsourcing services a2 and Q24 outsourcing services a4, and base station 3 needs to purchase Q33 outsourcing services A3 and Q34 outsourcing services a4, and the outsourcing services required for configuring other base stations are similar to the configuration process of the 3 base stations, which is not described herein. The unit prices of outsourcing service A1 to outsourcing service AM are P1 to PM in sequence. Recording the type and quantity of outsourced services to be purchased for configuring each base station and the unit price of each outsourced service, base station configuration cost data as shown in table 2 can be obtained.
TABLE 2
Figure BDA0002387520380000062
Under the base station configuration scene, the cost data of different base stations can be analyzed and processed to obtain cost baseline models for configuring different base stations.
The two application scenarios are only examples, and the service data processing method provided by the present application can also be applied to other service scenarios with data having multiple structures. The other business scenario may be a store analysis scenario, in which the structures of the stores and their surrounding population (including age, gender, and mobile phone brand) are obtained to provide intelligent site selection basis and store operation strategy for each store. The other business scenario may also be a chip analysis scenario, in which each chip and its test item set near a compliance boundary may be obtained, so that quality, performance, grade, and flow optimization of each chip may be better understood. Or, the method can also be applied to a database scene, and the method is integrated into a function with an analysis function, so that a user can analyze the data in the database system. Alternatively, the method may be applied to a cloud platform, and the like, and as long as a scene of data with multiple structures is involved, the method provided by the present application may be used, and an application scene is not limited herein.
For the sake of understanding, the related terms related to the embodiments of the present application will be described below.
(1) Data of multiple structure
The data of the multiple structure can be understood as including a multi-layer association relationship between business data corresponding to different objects (users or individuals). For example, in the consumption behavior data shown in table 1, there may be an overlap between the items purchased by different users, for example, user 1 and user 2 both purchase item a, and then user 1 and user 2 have a relationship in the types of the purchased items. Further, if the unit prices of the product C purchased by the user 1 and the product D purchased by the user 2 are the same, the user 1 and the user 2 have a relationship in the unit prices of the purchased products. As can be seen, in the consumption behavior data shown in table 1, the service data corresponding to different users include a multi-layer association relationship, that is, the consumption behavior data is data with a multi-structure.
(2) Clustering
Clustering is a method of dividing a plurality of objects into a plurality of disjoint subsets according to a preset specific standard by using feature engineering, and the purpose of the method is to make the data similarity in the same subset as large as possible, and the difference of data objects not in the same subset as large as possible, and each subset is usually called a "cluster".
Clustering can be classified into several types:
1. a hierarchical based clustering method. For example, the data of each object is used as a cluster, and then the clusters are combined according to a certain standard in each step of clustering, and the process is repeated continuously until the preset number of clustered clusters is reached.
2. Clustering methods based on partitioning. For example, k objects are randomly selected as initial points, and then based on some criteria (the euclidean distance within a cluster is minimum and the euclidean distance between clusters is maximum), the data for each object is divided into clusters formed with k initial points, resulting in k clusters.
3. A density-based clustering method. For example, the connectability between the respective objects is examined according to the density between the data of the respective objects, and the cluster is continuously expanded based on connectable samples to obtain a final clustering result.
4. A grid-based clustering method. For example, the data space of each object is divided into grid cells, the data of each object is mapped into the grid cells, and the density of each grid cell is calculated. And judging whether each grid cell is a high-density cell or not according to a preset threshold value, wherein the adjacent dense cell groups form a class.
5. A model-based clustering method. For example, some methods cluster data of each object based on mathematical models, and common mathematical models include probability generation models, decision trees, neural networks, and the like.
(3) Data of graph structure
The graph structure is a data structure that models a plurality of objects and their associations between the plurality of objects. Referring to fig. 1, which is a schematic diagram of an example of a graph structure, fig. 1 includes 3 vertices, and a connecting edge is included between any two vertices, where each vertex corresponds to an object, and the connecting edge between any two vertices represents an association relationship between the two objects, and the association relationships between the two vertices may be the same or different. Graph structures may be used to represent data in a number of domains including social networking, science, knowledge graphs, and the like.
(4) Neural network
The neural network may be composed of neural units, which may be referred to as xsAnd an arithmetic unit with intercept 1 as input, the output of which may be:
Figure BDA0002387520380000081
wherein s is 1, 2, … … n, n is a natural number greater than 1, and W issIs xsWeight of (2)And b is the bias of the neural unit. f is an activation function (activation functions) of the neural unit for introducing a nonlinear characteristic into the neural network to convert an input signal in the neural unit into an output signal. The output signal of the activation function may be used as an input to the next layer of neural network. As an example, the activation function may be a sigmoid function, a hyperbolic tangent function (Tanh () function), a modified linear function (ReLU), or the like, and the activation function is not limited herein. A neural network is a network formed by a number of the above-mentioned single neural units joined together, i.e. the output of one neural unit may be the input of another neural unit. The input of each neural unit can be connected with the local receiving domain of the previous layer to extract the characteristics of the local receiving domain, and the local receiving domain can be a region composed of a plurality of neural units.
(5) Deep neural network
Deep Neural Networks (DNNs), also known as multi-layer Neural networks, can be understood as Neural networks having many hidden layers, where "many" has no particular metric. From the division of DNNs by the location of different layers, neural networks inside DNNs can be divided into three categories: input layer, hidden layer, output layer. Generally, the first layer is an input layer, the last layer is an output layer, and the middle layers are hidden layers. The layers are all connected, that is, any neuron of the ith layer is necessarily connected with any neuron of the (i + 1) th layer. Although DNN appears complex, it is not really complex in terms of the work of each layer, simply the following linear relational expression:
Figure BDA0002387520380000082
wherein the content of the first and second substances,
Figure BDA0002387520380000083
is the input vector of the input vector,
Figure BDA0002387520380000084
is the output vector of the output vector,
Figure BDA0002387520380000085
is an offset vector, W is a weight matrix (also called coefficient), and α () is an activation function. Each layer is only for the input vector
Figure BDA0002387520380000086
Obtaining the output vector through such simple operation
Figure BDA0002387520380000087
Due to the large number of DNN layers, the coefficient W and the offset vector
Figure BDA0002387520380000088
The number of the same is large. The definition of these parameters in DNN is as follows: taking coefficient W as an example: assume that in a three-layer DNN, the linear coefficients of the 4 th neuron of the second layer to the 2 nd neuron of the third layer are defined as
Figure BDA0002387520380000089
The superscript 3 represents the number of layers in which the coefficient W is located, while the subscripts correspond to the third layer index 2 of the output and the second layer index 4 of the input. The summary is that: the coefficients of the kth neuron of the L-1 th layer to the jth neuron of the L-1 th layer are defined as
Figure BDA00023875203800000810
Note that the input layer is without the W parameter. In deep neural networks, more hidden layers make the network more able to depict complex situations in the real world. Theoretically, the more parameters the higher the model complexity, the larger the "capacity", which means that it can accomplish more complex learning tasks. The final goal of the process of training the deep neural network, i.e., learning the weight matrix, is to obtain the weight matrix (the weight matrix formed by the vectors W of many layers) of all the layers of the deep neural network that is trained.
(6) Convolutional neural network
A Convolutional Neural Network (CNN) is a deep neural Network with a Convolutional structure. The convolutional neural network includes a feature extractor consisting of convolutional layers and sub-sampling layers. The feature extractor may be viewed as a filter and the convolution process may be viewed as convolving an input image or convolved feature plane (feature map) with a trainable filter. The convolutional layer is a neuron layer for performing convolutional processing on an input signal in a convolutional neural network. In convolutional layers of convolutional neural networks, one neuron may be connected to only a portion of the neighbor neurons. In a convolutional layer, there are usually several characteristic planes, and each characteristic plane may be composed of several neural units arranged in a rectangular shape. The neural units of the same feature plane share weights, where the shared weights are convolution kernels. Sharing weights may be understood as the way image information is extracted is location independent. The underlying principle is: the statistics of a certain part of the image are the same as the other parts. Meaning that image information learned in one part can also be used in another part. The same learned image information can be used for all positions on the image. In the same convolution layer, a plurality of convolution kernels can be used to extract different image information, and generally, the greater the number of convolution kernels, the more abundant the image information reflected by the convolution operation.
The convolution kernel can be initialized in the form of a matrix of random size, and can be learned to obtain reasonable weights in the training process of the convolutional neural network. In addition, sharing weights brings the direct benefit of reducing connections between layers of the convolutional neural network, while reducing the risk of overfitting.
(7) Loss function
In the process of training the deep neural network, because the output of the deep neural network is expected to be as close to the value really expected to be predicted as possible, the weight vector of each layer of the neural network can be updated according to the difference between the predicted value of the current network and the really expected target value (of course, an initialization process is usually carried out before the first updating, namely parameters are preset for each layer in the deep neural network), for example, if the predicted value of the network is high, the weight vector is adjusted to be slightly lower, and the adjustment is carried out continuously until the deep neural network can predict the really expected target value or the value which is very close to the really expected target value. Therefore, it is necessary to define in advance "how to compare the difference between the predicted value and the target value", which are loss functions (loss functions) or objective functions (objective functions), which are important equations for measuring the difference between the predicted value and the target value. Taking the loss function as an example, if the higher the output value (loss) of the loss function indicates the larger the difference, the training of the deep neural network becomes the process of reducing the loss as much as possible.
(8) Graph Neural Network (GNN) is a deep Neural Network that can be used to process data of Graph structures, with the goal of learning the representation/state embedding (state embedding) of each vertex in the data of the Graph structure. In particular, each vertex v is denoted hv∈Rm,RmA data space formed for all data in the data of the graph structure. The output of each vertex v is denoted Ov,hvAnd OvThe following expression is satisfied:
hv=f(xv,xco[v],hne[v],xne[v]) (2)
Ov=g(hv,xv) (3)
wherein f () and g () are the local migration function and the local data function of GNN, x, respectivelyv,xco[v],hne[v],xne[v]Is the feature vector of the vertex, the feature vector of the edge connected to the vertex, the representation vector of the neighbor vertex of the vertex, and the feature vector of the neighbor vertex. GNN updates the representation h of each vertex by multiple iterationsvThereby minimizing the penalty function for all supervised vertices, which satisfies the expression:
Figure BDA0002387520380000091
wherein p is the number of all supervised vertices, tiIs the true state of the ith vertex, OiIs the predicted state of the ith vertex.
GNNs can be divided into several types, the main difference of which is the information collection per vertex and the updating way of the representation per vertex.
1. Convolution-based GNN
The main idea is to apply a convolution kernel on the graph-structured data, which may include, for example, a spectrum (spectral) -convolution-based GNN that extracts information and updates the state of each vertex by means of eigenvalues and eigenvectors of the laplacian matrix of the graph-structured data.
2. GNN based on door mechanism (gate)
GNNs based on the portal mechanism use a Gated Repeat Unit (GRU) and a portal control method in a long-short-term memory (LSTM) to transfer vertex information in graph-structured data.
3. Attention (attention) -based GNN
Attention-based GNNs use a self-attention (self-attention) strategy to update the representation of each vertex according to each vertex's attention on its neighbor vertices.
4. GNN based on residual connection (skip connection)
Because the GNN is a deep neural network, noise introduced by the GNN in the information transfer process increases with the number of vertices which increase exponentially, and residual connection can effectively improve the noise introduced by the GNN, so that the residual connection is introduced by the GNN.
Compared with the traditional deep neural network, the GNN can keep the data of the graph structure not deformed after being translated, and the GNN also takes the edges in the data of the graph structure into account, and the state update of each vertex is obtained by weighted summation of the states of the adjacent vertices, which is consistent with the association relationship between each edge representing each two vertices in the graph structure.
The application of GNNs to vertices may include vertex classification or vertex attribute prediction, among others. For example, a part of vertices (10% or less vertices) in the data of the graph structure is labeled, and the attributes of all the vertices and the adjacency matrix are input to the GNN, so that the classification of each vertex can be obtained. The main application scenarios may include picture classification, text classification, or anomaly detection, etc. For another example, after the attributes of all vertices and the adjacency matrix are input to the GNN, the unknown attribute of each vertex can be obtained. The main application scenarios may include predicting traffic flow, predicting weather, or learning molecular characteristics, etc.
The application of the GNN to the edge mainly includes predicting hidden association relations between vertices by using known association relations between vertices in the graph-structured data. For example, to complement a knowledge graph or to perform new drug testing, etc.
In addition, in the embodiments of the present application, "a plurality" means two or more, and in view of this, the embodiments of the present application may also mean "at least two". "at least one" is to be understood as meaning one or more, for example one, two or more. For example, including at least one means including one, two, or more, and does not limit which ones are included, for example, including at least one of A, B and C, then including may be A, B, C, A and B, A and C, B and C, or a and B and C. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/", unless otherwise specified, generally indicates that the preceding and following related objects are in an "or" relationship. Unless stated to the contrary, the embodiments of the present application refer to the ordinal numbers "first", "second", etc., for distinguishing between a plurality of objects, and do not limit the sequence, timing, priority, or importance of the plurality of objects.
With diversification and complication of service types, service data with a multi-structure is more and more common in practical use scenes. Because the incidence relation between the data is complex, the traditional clustering technology can not process the service data with multiple structures. Specifically, the conventional clustering technique extracts partial features corresponding to the objects according to a preset specific standard by a feature engineering method, and then performs clustering processing on the plurality of objects according to the partial features.
Taking the business data in the retail research scenario shown in table 1 as an example, a conventional clustering technique will be described. For example, the preset specific standard is the total price of the commodities purchased by each user, and since the conventional clustering technique only focuses on the total price of the commodities purchased by each user, i.e. does not focus on the association relationship among the commodities purchased by each user, the business data shown in table 1 can be converted into that shown in table 3.
TABLE 3
User number Total number of purchased goods Total price of goods General categories of purchased goods
User
1 6 1600 3
User 2 3 300 2
User 3 2 850 1
In table 3, the total price of 6 items purchased by user 1 is 1600, the total price of 3 items purchased by user 2 is 300, and the total price of 2 items purchased by user 3 is 850. Then, according to the total prices of the commodities purchased by different users, clustering is carried out on all the users, and commodities close to the total prices of the commodities purchased by the users of the same class are recommended.
As can be seen from comparing table 1 and table 3, if the conventional clustering technique is forcibly adopted to analyze and process the data with multiple structures, the data with multiple structures is substantially converted into the data with single-layer structures, and after the data with multiple structures is converted into the data with single-layer structures, part of the incidence relation in the data with multiple structures is lost.
In view of this, the present application provides a method for processing service data, which is used for processing service data with multiple structures.
The business data processing method can be applied to electronic equipment. The electronic device may be a terminal, such as a mobile phone terminal, a tablet computer, a notebook computer, an Augmented Reality (AR) device, a Virtual Reality (VR) device, a vehicle-mounted terminal, and the like; the electronic device may also be a server, for example, a server capable of providing a service required by a user, where the service may include services corresponding to various application scenarios applicable to the foregoing application, for example, a server providing a consumption service for the user in the foregoing retail research scenario, such as a server of some e-commerce platforms, or the server may be a server providing outsourcing service purchase service in a base station configuration scenario, or may be a server of a cloud service platform, or the server may also be embedded in an existing database system, such as a server of a certain data center, or the server may also be an independent server, and in this embodiment of the application, a specific form of the electronic device is not limited.
The electronic equipment can acquire the service data of multiple structures corresponding to the user when the user executes the service. For example, if the electronic device is a server providing a consumption service, after the user executes the consumption service using the mobile phone terminal, the server providing the consumption service generates consumption behavior data corresponding to the user. Or, the electronic device may perform data interaction with an external device to obtain service data corresponding to the user when executing the service. For example, the electronic device is an independent server, which is labeled as server a, where the server a is configured to process preset service data, where the preset service data may be consumption behavior data of a user, and then, when the server providing the consumption service generates the consumption behavior data of the user, the consumption behavior data of the user is sent to the server a. When the electronic device obtains the service data of the multiple structure corresponding to the user when executing the service, the electronic device first pre-processes the service data, and converts the data structure of the service data from the multiple structure to a first structure, where the first structure is a non-single-layer structure and a non-linear data structure, and the first structure may be a graph structure, for example. After the structure conversion of the service data is completed, the electronic device processes the service data with the first structure according to a preset target model/rule to obtain the type of each user or the hidden association relationship among the users.
It should be noted that the preset target model/rule may be obtained by inputting data having the first structure into a neural network capable of processing data having a non-linear structure and training the neural network. The data with the first structure may be stored in a training library of the electronic device in advance, or the data with the first structure may be received from other devices, which is not limited herein.
It should be noted that there may be one or more preset target models/rules in the electronic device, and if there is only one preset target model/rule, the same target model/rule is used for processing the service data of different services; if there are multiple preset target models/rules, each preset target model/rule is used for processing one kind of business data, the multiple preset target models/rules may be generated based on different training data.
In the embodiment of the present application, the preset target model/rule may be a graph neural network GNN. Please refer to fig. 2, which is a schematic structural diagram illustrating an example of the neural network GNN. As shown in fig. 2, the graphical neural network GNN may include N processing layers, N being an integer greater than or equal to 3. The first layer of the graph neural network GNN is an input layer 201, which is responsible for receiving data having the first structure, and the last layer of the graph neural network GNN is an output layer 203, which is used for outputting processing results of the graph neural network GNN, such as outputting a class to which each vertex in the data having the first structure belongs or outputting an association relationship between each vertex in the data having the first structure. The hidden layer 202 may include one or more layers except the first and last layers, and in fig. 2, the hidden layer 202 includes a plurality of layers (as shown by the dashed boxes in fig. 2). Each of the hidden layers 202 can receive input data and output a calculation result, and the hidden layers 202 are responsible for processing input data having the first structure. Each hidden layer 202 represents a logical level of data processing, and data may be processed through multiple levels of logic through multiple hidden layers. Each hidden layer 202 corresponds to a weight matrix for data processing, and a weight value of each element in the weight matrix may be predefined or trained to extract a feature vector of each vertex in the data from the input data. The output layer 203 is configured to determine a type to which each vertex belongs or an association relationship between the vertices according to the feature vector of each vertex extracted by the hidden layer 202. In general, the output layer 203 has a loss function for calculating the prediction error, and once the forward propagation of the entire graph neural network GNN (the propagation from the input layer 201 to the output layer 203 is shown as forward propagation in fig. 2) is completed, the backward propagation (the propagation from the output layer 203 to the input layer 201 is shown as backward propagation in fig. 2) starts to update the weight values of the layers in the hidden layer 202 as described above, so as to reduce the loss of the graph neural network GNN and reduce the error between the output result of the output layer 203 and the ideal result.
It should be noted that the graph neural network GNN shown in fig. 2 is only an example of an object model for processing data of a nonlinear structure, and in a specific application, the preset object model/rule may also exist in the form of other network models.
A hardware structure of a chip provided in an embodiment of the present application is described below. The chip may be disposed in the electronic device, so as to complete the analysis and processing of the data with multiple structures by the electronic device, that is, the processing of the business data with multiple structures by the electronic device may be implemented in the chip.
Please refer to fig. 3, which is a diagram illustrating an example of the hardware structure of the chip. As shown in fig. 3, the chip includes a neural network processor 30, the neural network processor 30 may be mounted as a coprocessor on a Central Processing Unit (CPU), and the CPU distributes tasks of data processing to the neural network processor 30. The neural network processor 30 includes a preprocessor 301, a controller 302, an arithmetic circuit 303, and a memory 304. The preprocessor 301 is configured to receive the service data with the multi-structure, convert the service data with the multi-structure into data with a first structure, and store the data with the first structure in the memory 304. The controller 302 is configured to control the operation circuit 303 to fetch the data with the first structure from the memory 304 and perform an operation.
In some implementations, the arithmetic circuitry 303 includes a plurality of Processing Elements (PEs) internally, which may include, for example, a two-dimensional systolic array or electronic circuitry capable of performing mathematical operations such as multiplication and addition or may be a general-purpose matrix processor.
In some implementations, the memory 304 may include a weight memory 3041, an input memory 3042, and a unified memory 3043. The weight memory 3041 is used to store weight data corresponding to each layer in the graph neural network GNN shown in fig. 2, the input memory 3042 is used to store an output result of the preprocessor 301 after processing the service data with multiple structures, that is, the service data with the first structure, and the unified memory 3043 is used to store the service data with multiple structures and an operation result of the operation circuit 303 on the service data with the first structure. The weight memory 3041, the input memory 3042, and the unified memory 3043 may be on-chip (on-chip) memories.
Taking the operation process of the first hidden layer implemented by the operation circuit 303 as an example, the operation circuit 303 first obtains the weight matrix corresponding to the first hidden layer from the weight memory 3041, and buffers the weight matrix on each PE in the operation circuit 303. Then, the arithmetic circuit 303 acquires the input service data having the first structure from the input memory 3042, performs an arithmetic operation with the acquired weight matrix, obtains an arithmetic result, and stores the arithmetic result in the unified memory 3043. That is, the operation of each layer in the neural network GNN shown in fig. 2 can be performed by the operation circuit 303.
Of course, the chip may also include other hardware modules, for example, a bus interface unit, which may be used for interacting with the CPU, or a memory for storing instructions used by the preprocessor 301 and the controller 302, which are not listed here.
Next, a method for processing service data provided in the embodiment of the present application is described with reference to the drawings. Referring to fig. 4, a flowchart of an example of the method is shown. The method may be specifically executed by the aforementioned electronic device, for example, by a CPU of the electronic device, or, if the aforementioned electronic device includes a chip as shown in fig. 3, the method may also be executed by the neural network processor 30 of the chip, and the application is not limited thereto.
And S41, acquiring the service data.
In the embodiment of the present application, the service data is a plurality of individual data generated by a plurality of individuals when executing corresponding services, and the plurality of individual data corresponds to the plurality of individuals one to one, that is, each individual generates one (or one group of) individual data when executing the service. Each individual data comprises at least one sub-individual, each sub-individual comprises at least one attribute, multiple layers of association relations exist among the multiple individual data, a first layer of association relation in the multiple layers of association relations is used for indicating whether two individual data comprise the same sub-individual, and a second layer of association relation in the multiple layers of association relations is used for indicating whether the attributes of different sub-individuals in the two individual data are the same. It is understood that the service data is a service data of a multiple structure.
As an example, in a retail research scenario, the service refers to a consumption service, and each user performing the consumption service is an individual, for example, if the obtained service data is consumption behavior data shown in table 1, the individuals performing the consumption service include user 1, user 2, and user 3. Each sub-entity can be understood as an object of execution of the service. For example, in a consuming business, an object, i.e., an article purchased by a user, and an individual piece of each article, a type, i.e., an attribute included in each sub-individual, are executed. The method comprises the steps that a user 1 and a user 2 both purchase a commodity A, so that a first-layer incidence relation exists between individual data corresponding to the user 1 and individual data corresponding to the user 2; although the commodity C purchased by the user 1 and the commodity D purchased by the user 2 are different commodities, the unit prices of the two commodities are the same, and thus there is a second-layer association relationship between the individual data corresponding to the user 1 and the individual data corresponding to the user 2, and therefore the consumption behavior data is business data having a multiple structure.
The service data in the embodiment of the present application, that is, the service data corresponding to the scenario involving the data with multiple structures, is similar to the consumption behavior data in each scenario, and the service data in each scenario is not described herein one by one.
The service data may be sent to the electronic device by a server providing each service, in which case the electronic device may be a separate server; alternatively, the service data may be acquired by the electronic device according to the operation of the user, in which case, the electronic device may be a server for providing each service. Here, the manner in which the electronic device acquires the service data is not limited.
And S42, acquiring input data corresponding to the service data.
In the embodiment of the present application, the input data is data having a first structure, and the first structure is a non-single-layer and non-linear data structure, and the first structure includes vertices having feature information and multiple types of edges. As an example, the first structure may be a graph structure. The input data may be determined according to a multi-layer association relationship between the plurality of individual data, that is, data of a graph structure corresponding to the business data may be acquired according to the multi-layer association relationship between the plurality of individual data. Of course, the first structure may also be other data structures capable of maintaining the association relationship between a plurality of individual data as much as possible, and for convenience of description, the first structure is taken as a diagram structure as an example hereinafter.
Referring to fig. 5, the manner of acquiring the data of the graph structure corresponding to the service data is as follows:
s51, using the plurality of individuals as the vertexes of the graph structure data.
For example, the consumption behavior data shown in table 1 includes 3 individuals, namely, user 1, user 2, and user 3, and the number of each vertex may be the number of the user, so that 3 vertices (vertex 1, vertex 2, and vertex 3, respectively) are obtained as shown in fig. 6.
For example, the base station arrangement cost data shown in table 2 includes 5 individuals, i.e., base stations 1 to 5, and 5 vertices (vertices 1 'to 5', respectively) as shown in fig. 7 are obtained.
It should be noted that, each vertex included in the data of the graph structure not only has a number, but also each vertex carries feature information, the feature information includes information determined according to a vertex sub-individual, and the vertex sub-individual can be understood as a sub-individual included in the individual data of the individual corresponding to the vertex, and as an example, the feature information may be obtained according to attribute information of the vertex sub-individual. For example, in the base station configuration cost data shown in table 2, the characteristic information carried by each vertex may be the number of outsourced services purchased by the corresponding individual, the number of purchased respective outsourced services, the unit price of the outsourced services, and the total cost. Alternatively, the calculation may be performed based on the data shown in table 2, and may include, for example, the sum of the number of all outsourced services purchased, the total cost, and the average price of the outsourced services. Of course, the feature information may also include other contents, and is not limited herein.
As an example, the characteristic information carried by each vertex includes the sum of the number of all outsourced services purchased, the total cost, and the average price of the outsourced services. The feature information for the vertex corresponding to the base station 1, i.e., the vertex 1', is [ Q11+ Q12, Q11 × P1+ Q12 × P2, Q11 × P1+ Q12 × P2/Q11+ Q12 ].
And S52, determining shared information between every two individual data according to the multilayer incidence relation.
In the embodiment of the present application, the shared information is determined according to at least one layer of the multi-layer association relationship. It is to be understood that the shared information may be determined according to one of the relationships in the multi-layer relationship, or the shared information may be determined according to two of the relationships in the multi-layer relationship, or the shared information may be determined according to more of the relationships in the multi-layer relationship. Thus, the manner in which the shared information between each two individual data is determined is associated with the association used.
A first example will be described with reference to consumption behavior data shown in table 1 as an example of a method of determining shared information.
As can be seen from table 1, the consumption behavior data at least includes two layers of relationships, the first layer of relationships is that different users may purchase the same product. The second layer association includes the same unit price for different commodities. It should be noted that the second-layer association relationship may also use other attributes of the sub-individuals, for example, the types of the commodities are used, and the second-layer association relationship may be the same for different types of commodities. Three ways of determining shared information include:
1. and determining shared information according to the first-layer incidence relation.
It is understood that it is determined whether two individual data include the same sub-individual, and if so, shared information exists between the two individuals, and the two individual data include the same sub-individual, that is, shared information of the two individuals.
For example, if the individual data corresponding to the user 1 and the user 2 both include the commodity a, the shared information between the user 1 and the user 2 is the shared commodity a. If the individual data corresponding to the user 1 and the user 3 respectively includes the commodity C, the shared information between the user 1 and the user 3 is the shared commodity C. Since the individual data corresponding to each of the users 2 and 3 does not include the same product, there is no shared information between the users 2 and 3.
2. And determining the shared information according to the incidence relation of the second layer.
It is understood that it is determined whether unit prices of different sub-individuals included in the data of the two individuals are the same, if the unit prices are the same, shared information exists between the two individuals, and the shared information of the two individuals is different sub-individuals including the same unit price.
For example, in the individual data corresponding to each of the user 1 and the user 2, the unit prices of the item C and the item D are both 200, and the shared information between the user 1 and the user 2 is the item C and the item D having the shared unit price of 200. In the individual data corresponding to the user 1 and the user 3, the single items of the commodity B and the commodity E are both 450, and the shared information between the user 1 and the user 3 is the commodity B and the commodity E with the shared unit price of 450. The individual data respectively corresponding to the user 2 and the user 3 includes the commodity C and the commodity D each having a single commodity of 200, and therefore, the shared information between the user 2 and the user 3 is the commodity C and the commodity D each having a shared unit price of 200.
3. And determining shared information according to the first layer incidence relation and the second layer incidence relation.
It is to be understood that, whether the same sub-individuals are included in the two individual data or not, and whether the unit prices of different sub-individuals included in the two individual data are the same or not, if the same sub-individuals are included in the two individual data and the unit prices of different sub-individuals are the same, shared information exists between the two individuals, and the shared information of the two individuals is different sub-individuals including the same sub-individuals and the same unit price.
For example, the individual data corresponding to the user 1 and the user 2 respectively includes the item a, and the unit prices of the item C and the item D are both 200, so that the shared information between the user 1 and the user 2 is the shared item a and the item C and the item D have the shared unit price of 200. The individual data respectively corresponding to the user 1 and the user 3 includes a commodity C, and the singlets of the commodity B and the commodity E are both 450, so that the shared information between the user 1 and the user 3 is the shared commodity C and the commodities B and E with the shared unit price of 450. The individual data corresponding to the user 2 and the user 3 includes the same product C and the same product D, but does not include the same product, and therefore, there is no shared information between the user 2 and the user 3.
A second example will be described with reference to the base station configuration cost data shown in table 2 as an example, to determine the shared information.
The manner of determining the shared information between the respective base stations based on the base station configuration cost data shown in table 2 is similar to the example shown in table 1, and the manner of determining the shared information between the respective base stations will be described below by taking one of the manners as an example. In the following, the same outsourced service is purchased for example in the case of determining the correlation of the shared information.
As can be seen from table 2, when the outsourced service a2 is included in the base station configuration cost data corresponding to the base station 1 and the base station 2, the shared information between the base station 1 and the base station 2 is the shared outsourced service a 2. The outsourcing service a4 is included in the base station configuration cost data corresponding to each of the base station 2 and the base station 3, and therefore the shared information between the base station 2 and the base station 3 is the shared outsourcing service a 4. The base station configuration cost data corresponding to the base station 1 and the base station 4 respectively comprises outsourcing service a1 and outsourcing service a2, and the shared information between the base station 1 and the base station 4 is shared outsourcing service a1 and outsourcing service a 2. By analogy, this is not to be taken as an enumeration.
And S53, connecting all vertexes corresponding to the individuals with the shared information to form a connecting edge of the graph structure data, so as to obtain the graph structure data.
After the shared information between the two individual data is determined, the vertex corresponding to each individual is connected according to the shared information.
For example, in the consumption behavior data shown in table 1, based on whether the same child is included in the two individual data, it is determined that the shared information between the user 1 and the user 2 is the shared commodity a, the shared information between the user 1 and the user 3 is the shared commodity C, and no shared information exists between the user 2 and the user 3, so that the vertex 1 and the vertex 2 are connected, the vertex 1 and the vertex 3 are connected, and the connected edges of the graph structure are formed, and the data of the graph structure shown in fig. 6 is obtained.
For another example, in the base station configuration cost data shown in table 2, it is determined whether the configuration base station has purchased the same outsource service, and the shared information between the base station 1 and the base station 2 is shared outsource service a2, the shared information between the base station 2 and the base station 3 is shared outsource service a4, the shared information between the base station 1 and the base station 4 is shared outsource service a1 and outsource service a2, the shared information between the base station 3 and the base station 5 is shared outsource service A3, and the shared information between the base station 4 and the base station 5 is shared outsource service A3, so that the vertex 1 'and the vertex 2' are connected, the vertex 2 'and the vertex 3' are connected, the vertex 1 'and the vertex 4' are connected, the vertex 3 'and the vertex 5' are connected, and the connection of the graph structure is formed, and the data of the graph structure shown in fig. 7 is obtained.
In the embodiment of the present application, the graph structure data obtained according to the shared information between the individuals further includes the type of each connected edge. The type of each side is determined according to the content of the shared information between individuals. If the contents of the shared information of the two groups of individuals are the same, the types of the connecting edges respectively corresponding to the two groups of individuals are the same; if the contents of the shared information of the two groups of individuals are different, it is indicated that the types of the connecting edges respectively corresponding to the two groups of individuals are different. Wherein each group of individuals includes two individuals for whom shared information exists.
For example, if the type of the connection between the user 1 and the user 2 is labeled as type 1, since the shared information between the user 1 and the user 3 is the shared commodity C, which is different from the shared information between the user 1 and the user 2, the type of the connection between the user 1 and the user 3 may be labeled as type 2, as shown in fig. 6.
For another example, if the type of the connection edge between the base station 1 and the base station 2 is denoted as type a, the type of the connection edge between the base station 2 and the base station 3 may be denoted as type B because the shared information between the base station 1 and the base station 2 is different from the shared information between the base station 2 and the base station 2. Since the shared information between the base station 1 and the base station 4 is different from the shared information between the base station 1 and the base station 2, and the shared information between the base station 2 and the base station 3, respectively, the type of the connection edge between the base station 1 and the base station 4 can be labeled as type C. By analogy, it is determined to label the type of the connecting edge between base station 3 and base station 5 as type D and the type of the connecting edge between base station 4 and base station 5 as type D, as shown in fig. 7.
It should be noted that the type of each continuous edge in the data of the graph structure may be only used to indicate whether the types of any two continuous edges are the same or different, for example, when the corresponding identifiers of two continuous edges are the same, it is determined that the types of the two continuous edges are the same, otherwise, the types of the two continuous edges are considered to be different. Alternatively, in order to describe the data of the graph structure more accurately, the mapping relationship between the connection type and the shared information corresponding to the data of each graph structure may be separately stored. For example, for the data of the graph structure shown in fig. 7, the mapping relationship between the connection edge type and the shared information as shown in table 4 may be stored.
TABLE 4
Type of continuous edge Sharing information
A Shared outsourcing service A2
B Shared outsourcing service A4
C Shared outsourcing services A1 and A2
D Shared outsourcing service A3
In addition, the mapping relationship between the edge linking type and the shared information shown in table 4 may be associated with the service type, that is, the service data of the same service type share the same mapping relationship, and the service data of different service types correspond to different mapping relationships. For example, if the electronic device can only process the consumption behavior data as shown in table 1, only one mapping relationship needs to be stored in the electronic device; if the electronic device is capable of processing the consumption behavior data as shown in table 1 and the base station configuration cost data as shown in table 2, a mapping relationship corresponding to the consumption behavior data and a mapping relationship corresponding to the base station configuration cost data need to be stored in the electronic device.
In other embodiments, when the traffic data includes a large amount of individual data, there may be more shared information between vertices. For example, in the base station configuration cost data shown in table 2, the outsourced service a2 is shared among the base station configuration cost data corresponding to each of the base station 1, the base station 2 and the base station 4, and if the base station configuration cost data shown in table 2 includes 6 or more base stations, the outsourced service a2 may be shared among more base stations. In order to avoid the uniqueness loss between the vertices in the graph structure due to the connection of the vertices, the children shared by the vertices can be eliminated. The culled sub-individuals may be sub-individuals sharing the sub-individuals with a number reaching a threshold value, that is, the total number of times the sub-individuals appear in the plurality of individual data is less than or equal to a threshold value, which may be preset, or may be determined according to the total number of individuals included in the business data and/or the total number of types of shared information among the individuals, which is not limited herein.
Taking the base station configuration cost data shown in table 2 as an example, assuming that the threshold is set to 3, since there are 3 base stations sharing the outsource service a2 among the 5 base stations, that is, the base station 1, the base station 2, and the base station 4, that is, the number of individuals sharing the outsource service a2 reaches the threshold, the outsource service a2 can be removed when determining the shared information between the base stations. It is thus determined that there is no shared information between the base station 1 and the base station 2, the shared information between the base station 2 and the base station 3 is the shared outsourcing service a4, the shared information between the base station 1 and the base station 4 is the shared outsourcing service a1, the shared information between the base station 3 and the base station 5 is the shared outsourcing service A3, and the shared information between the base station 4 and the base station 5 is the shared outsourcing service A3, thereby obtaining data of the graph structure shown in fig. 8. Unlike the data of the graph structure shown in fig. 7, in fig. 8, vertex 1 'and vertex 2' are not connected, the connection type of vertex 1 'and vertex 4' becomes the connection type of shared outsource service a1, and the data of the graph structure shown in fig. 8 includes only 3 connection types, which are the connection type of shared outsource service a4, the connection type of shared outsource service a1, and the connection type of shared outsource service A3, respectively.
In the above technical solution, the shared information between the individuals in the data with the multiple structures is utilized to convert the multiple data into the data with the first structure, such as the data with the graph structure, so that the loss of the associated information between the individuals caused when the data with the multiple structures is converted into the data with the single-layer structure when the clustering processing is performed on the multiple objects by using the characteristic engineering method can be avoided, and the associated information between the individuals can be kept through the different types of connection edges between the individuals.
And S43, calculating the input data by using a preset neural network to obtain the classes to which the individuals belong.
And after the data with the multiple structures are converted into the data with the first structure, inputting the data with the first structure into a preset neural network for training. As an example, the preset neural network may be a preset graph neural network, and the preset neural network is taken as the graph neural network as an example for description hereinafter.
In the embodiment of the present application, the obtaining of the type to which the plurality of individuals belong through training by using the preset neural network may include, but is not limited to, the following two ways.
First, these two modes will be briefly described.
In the first mode, the preset graph neural network is trained by directly using data with a first structure, and the type of each individual is determined according to the output result of the graph neural network. That is, the classification of the data having the first structure is achieved directly using a preset graph neural network. In this case, the preset neural network of the graph obtains the classifications to which the plurality of individuals belong in the data with the first structure, and the classifications to which the plurality of individuals belong are associated according to the label carried by each individual. For example, when the plurality of individuals input the preset graph neural network, the plurality of individuals include 3 kinds of labels, which are respectively a middle age, a young age and a child, and after the classification is performed by the graph neural network, the individuals with the same kind of labels are classified into one classification.
In a second mode, the data with the first structure is used for training a preset graph neural network to obtain a plurality of state vectors, and then the plurality of state vectors are clustered to obtain classes to which the plurality of individuals belong. That is, state vectors of the respective individuals are extracted using a preset graph neural network, and then the plurality of individuals are clustered using clustering processing. In this case, the result of clustering the plurality of individuals is not associated with the original type of each individual. For example, when the data having the first structure is input to the preset neural network for processing, the plurality of individuals may be classified into two types, for example, male and female, according to a preset rule, but after the clustering process, there may be 3 or more types of the plurality of individuals, and the clustering result is learned according to a clustering algorithm.
For ease of understanding, the two modes are described in detail below.
The first mode is as follows:
as an example, the predetermined graph neural network may be a relational-graph convolutional neural network (R-GCN). The schematic structure of the R-GCN is similar to the structure shown in FIG. 2, and includes an input layer, a hidden layer and an output layer, which are not described herein again. In R-GCN, the update formula of each individual in one hidden layer is as follows:
Figure BDA0002387520380000181
wherein the content of the first and second substances,
Figure BDA0002387520380000182
is the state matrix of the ith vertex at the l-th level, and σ () is the activation function, which may be max (0,), for example, and is not limited herein.
Figure BDA0002387520380000183
And j is one vertex in the adjacent vertex set. c. Ci,rThe weight values for the normalization are, for example,
Figure BDA0002387520380000184
is the initial parameter (or initial weight matrix) of the l-th layer,
Figure BDA0002387520380000185
the l-th layer needs to learn parameters, i.e. parameters that can be updated by back-propagation through the loss function.
As an example, the R-GCN may include 2 hidden layers, the first hidden layer may include 16 neurons, and the second hidden layer may include 48 neurons, although in practical use, the number of hidden layers in the R-GCN and the number of neurons included in each hidden layer may be set according to a use situation, and are not limited herein.
In the embodiment of the application, the first structure is used as a graph structure, the graph structure data is used for training a preset graph neural network, an input matrix and a continuous edge matrix corresponding to the graph structure data are firstly acquired, and then according to the input matrix and the continuous edge matrix, classes to which a plurality of individuals belong are used as targets for training the R-GCN, so that an output result is obtained.
In this way, elements in a column vector or a row vector of the input matrix are used to indicate feature information corresponding to a vertex in the data having the first structure and a label of the vertex. Wherein the label information of each vertex is determined according to the information included in each vertex. For example, taking the consumption behavior data as an example, the consumption behavior data may include a description of each user, for example, whether the gender of the user is male or female, or whether the age group of the user belongs to the middle-aged or the elderly of the young, so that each vertex can be labeled according to the information. Specifically, what kind of information is adopted as the label can be set according to the actual use requirement, and is not limited herein. The column vector or row vector in the edge matrix is used to indicate the type of edge included in each vertex.
Next, the types of the connected edges included in each vertex and the feature information of the vertex will be described by taking the data of the graph structure shown in fig. 7 as an example.
For the vertex corresponding to the base station 1, i.e. vertex 1 ', vertex 1 ' includes two connected edges, as shown in table 4, the types of the two connected edges related to vertex 1 ' are connected edge type a and connected edge type C, respectively, and may be represented by character 1 or character 3. In the data of the graph structure shown in fig. 7, 5 vertices are included, and taking a row vector of the edge-connected matrix as an example to indicate the type of the edge-connected included in a vertex, a row vector includes 5 elements, a first element is used to indicate the type of the edge-connected included in the vertex and vertex 1 ', a second element is used to indicate the type of the edge-connected included in the vertex and vertex 2', and so on, which are not necessarily described herein. Since there is a connecting edge between vertex 1 ' and vertex 2 ', and the type is 1, and there is a connecting edge between vertex 1 ' and vertex 4 ', and the type is 3, a row vector corresponding to vertex 1 ' in the connecting edge matrix is [0, 1, 0, 3, 0 ].
For the input matrix, as an example, the feature information of each vertex may be represented by three parameters, that is, "total amount of purchased goods by user," total price of purchased goods, "and" average price of purchased goods "corresponding to the vertex, but other parameters may be selected, and the selection is not limited herein. The label of each vertex may be whether the base station belongs to a large base station or a small base station, although other parameters may be selected, and are not limited herein. Taking the above example as an example, the vertex 1 'is a small base station, denoted by-1, so that the feature information of the vertex 1' can be denoted as [ -1, Q11+ Q12, Q11 × P1+ Q12 × P2, Q11 × P1+ Q12 × P2/Q11+ Q12 ].
It should be noted that, in this case, there is a correspondence between the row vector and the column vector between the input matrix and the edge matrix, that is, the column vector or the row vector located at the same position of the input matrix and the edge matrix is used to indicate information of the same vertex. For example, if the first column vector of the edge-connected matrix is used to indicate the edge-connected type included in the first vertex, the first column vector of the input matrix is used to indicate the feature information of the first vertex and the initial type of the vertex.
And sequentially acquiring a row vector or a column vector corresponding to each vertex in the continuous edge matrix and the input matrix according to the mode to obtain the continuous edge matrix and the input matrix. And after the input matrix and the edge matrix are obtained, inputting the input matrix into the R-GCN for training, and summing various edges according to row vectors or column vectors corresponding to all vertexes in the edge matrix, thereby obtaining a training result of the R-GCN. As an example, the training result may be: the vertex 1 'and the vertex 2' are in the same classification, and the classification can be determined as a small base station type according to the label of each vertex; vertices 3 'through 5' are the same class and may be identified as a large base station type based on the label.
In the technical scheme, when the graph neural network is trained, not only the labels of the vertexes and the information of the connecting edges included in the graph structure data are input, but also the characteristic information of the vertexes is input, so that the beneficial information can be fully utilized, and the accuracy of data analysis is improved.
The second mode is as follows:
in the embodiment of the application, a first structure is used as a graph structure, a preset graph neural network is trained by using graph structure data, an input matrix and an edge connecting matrix corresponding to the graph structure data are firstly acquired, then, according to the input matrix and the edge connecting matrix, classes to which a plurality of individuals belong are used as targets, R-GCN is trained to acquire an output matrix of a last hidden layer of the preset graph neural network, a row vector in the output matrix is a state vector corresponding to each individual, and finally, clustering processing is performed on the output matrix to acquire classes to which the plurality of individuals belong.
In this way, the process of training the preset neural network using the data having the first structure differs from the first way in two points. The first difference is that the input matrix of the acquired data having the first structure is different. The second point is different in that, in the second mode, the output result of the graph neural network to the input matrix is not required to be obtained, but the output matrix of the last hidden layer of the graph neural network is obtained, and the row vector in the output matrix of the last hidden layer is the state vector corresponding to each individual, and then the state vector is subjected to clustering processing to obtain the clustering result. These two different points will be described below.
The first difference is: and inputting the matrix.
In this manner, the input matrix is used to indicate the feature information corresponding to each vertex and the initial type of the vertex. The initial type is determined according to a preset determination condition, for example, for the consumption behavior data, the preset determination condition may be whether the total price of the goods purchased by the user exceeds a threshold, if so, the initial type of the vertex is determined to be a first type and may be marked as 1, otherwise, the initial type of the vertex is determined to be a second type and may be marked as-1. The threshold may be preset, or may be an average value of the total consumption of all users, or may be other values, which is not limited herein. The feature information and the edge matrix corresponding to each vertex are similar to those in the first method, and are not described herein again. It should be noted that the initial type may be determined according to the information included in each individual, in this case, similar to the tags in the first manner, but the initial type may also be customized, so the initial type is more flexible than the tags in the first manner.
In this manner, the input matrix may include, but is not limited to, the following two forms:
in the first form, the row vector or the column vector of the input matrix is used to indicate the feature information corresponding to a vertex and the initial type of the vertex. The row vector or the column vector in the input matrix may also be referred to as a feature vector corresponding to one vertex. As an example, with one of the vertices, e.g. vertex 1' as shown in fig. 7, the feature vector can be expressed as follows:
[-1,Q11+Q12,Q11*P1+Q12*P2,Q11*P1+Q12*P2/Q11+Q12]
wherein, the first element in the feature vector is used to indicate the initial type of the vertex, and a plurality of subsequent elements are used to indicate the feature information of the vertex. In the above example, -1 indicates that the initial type of the vertex 1 'is the second type, [ Q11+ Q12, Q11 × P1+ Q12 × P2, Q11 × P1+ Q12 × P2/Q11+ Q12] is the feature information in the vertex 1'.
And sequentially acquiring the characteristic vector of each vertex according to the mode, and then sequentially arranging according to the size of the vertex number to obtain an input matrix corresponding to the data of the graph structure. For example, the input matrix is obtained by using the eigenvector corresponding to vertex 1 'as the first row vector of the input matrix, the eigenvector corresponding to vertex 2' as the second row vector of the input matrix, and so on.
In the second form, the input matrix may also be used to indicate the number of the vertex, for example, the number of the vertex may be added to the feature vector corresponding to each vertex. It should be noted that, when a vertex number is added to a special vector, the number may be preprocessed, for example, there are 5 vertices in total, a vector corresponding to a first vertex after being preprocessed may be [1, 0, 0, 0, 0], a vector corresponding to a second vertex after being preprocessed may be [0, 1, 0, 0, 0], and so on. Of course, other pre-processing may be performed on the numbers, which is not illustrated here. Hereinafter, the preprocessing of the above example is performed on the numbers as an example. Taking the graph structure described in fig. 7 as an example, the feature vector of the vertex 1' can be expressed as follows:
[-1,1,0,0,0,0,Q11+Q12,Q11*P1+Q12*P2,Q11*P1+Q12*P2/Q11+Q12]
wherein the first element is used to indicate the initial type of the vertex, the next 5 elements are used to indicate the number of the vertex, and the remaining elements are used to indicate the feature information of the vertex. In this case, the feature vectors of the plurality of vertices may be arbitrarily combined, for example, the feature vector corresponding to vertex 1 'may be the second row vector of the input matrix, and the feature vector corresponding to vertex 2' may be the first row vector of the input matrix.
It should be noted that, in the embodiments of the present application, a specific form of the input matrix is not limited.
And after the input matrix is obtained, inputting the input matrix into the graph neural network, and training by taking the classes to which the plurality of individuals belong as targets according to the edge connection matrix to obtain a plurality of state vectors. As shown in table 5, an example of an output matrix of the last hidden layer of the graph neural network.
TABLE 5
Figure BDA0002387520380000201
Figure BDA0002387520380000211
In Table 5, the last hidden layer includes 48 neurons, which are labeled v 1-v 48.
After the output matrix is obtained, the output matrix can be processed by using a traditional clustering method to obtain a clustering result, namely the class of each individual. As an example, a clustering method belonging to density may be employed. For example, considering that density-based spatial clustering with noise (DBSCAN) has a certain tolerance to noise and outliers and can solve the clustering problem of irregular shapes or non-convex types, the output matrix may be clustered using DBSCAN.
In order to reduce the complexity of the clustering process, before the output matrix is clustered by using a clustering method, the output matrix can be subjected to dimensionality reduction, and then the output matrix subjected to dimensionality reduction is clustered. As an example, the output matrix may be dimension-reduced using a t-distributed stored neighboring embedding (TSNE). Of course, this is not limited in the embodiments of the present application.
In the above technical solution, the output matrix of the last hidden layer in the graph neural network can be understood as the state vector of each individual is obtained through the graph neural network, and feature engineering is not required to extract feature vectors, so that the implementation method is simple and has universality.
In the embodiments provided in the present application, the methods provided in the embodiments of the present application are introduced from the perspective of the electronic device, respectively. In order to implement the functions in the method provided by the embodiments of the present application, the electronic device may include a hardware structure and/or a software module, and the functions are implemented in the form of a hardware structure, a software module, or a hardware structure and a software module. Whether any of the above-described functions is implemented as a hardware structure, a software module, or a hardware structure plus a software module depends on the particular application and implementation constraints of the technical solution.
Fig. 9 shows a schematic structural diagram of a device 900 for processing service data. The apparatus 900 may be an electronic device or an apparatus built in the electronic device, and may implement the function of the electronic device in the method provided by the embodiment of the present application. The apparatus 900 may be a hardware structure, a software module, or a hardware structure plus a software module. The apparatus 900 may be implemented by a system-on-a-chip. In the embodiment of the present application, the chip system may be composed of a chip, and may also include a chip and other discrete devices.
The apparatus 900 may include a first obtaining module 901, a second obtaining module 902, and a processing module 903.
A first obtaining module 901, configured to obtain service data, where the service data includes a plurality of individual data respectively corresponding to a plurality of individuals when performing a service, each individual data includes at least one sub-individual, and multiple layers of association relationships exist among the plurality of individual data;
a second obtaining module 902, configured to obtain input data corresponding to the service data, where the input data has a first structure, where the first structure includes vertices having feature information and multiple types of connected edges, each vertex corresponds to one of the multiple individuals, the feature information of one vertex includes information determined according to attributes of children individuals included in the individuals corresponding to the vertex, and the connected edges include information determined according to the multilayer association relationship between the multiple types of individual data;
a processing module 903, configured to train a preset neural network using the input data, and obtain classes to which the multiple individuals belong.
The first obtaining module 901 may be used to perform step S41 in the embodiment shown in fig. 4, and/or other processes for supporting the techniques described herein.
The first obtaining module 901 may be used for the apparatus 900 to communicate with other modules, and may be a circuit, a device, an interface, a bus, a software module, a transceiver, or any other apparatus that can implement communication, for example, may obtain service data from other apparatuses.
The second obtaining module 902 may be used to perform step S42 in the embodiment shown in fig. 4, or to perform steps S51-S53 in the embodiment shown in fig. 5, and/or other processes for supporting the techniques described herein.
The processing module 903 may perform step S43 in the embodiment shown in fig. 4, and/or other processes for supporting the techniques described herein.
All relevant contents of each step related to the above method embodiment may be referred to the functional description of the corresponding functional module, and are not described herein again.
The division of the modules in the embodiments of the present application is schematic, and only one logical function division is provided, and in actual implementation, there may be another division manner, and in addition, each functional module in each embodiment of the present application may be integrated in one processor, may also exist alone physically, or may also be integrated in one module by two or more modules. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
Fig. 10 is a schematic hardware structure diagram of a service data processing apparatus according to an embodiment of the present application. The apparatus 1000 for processing service data shown in fig. 10 (the apparatus 1000 may be specifically a computer device or an electronic device) includes a memory 1001, a processor 1002, a communication interface 1003, and a bus 1004. The memory 1001, the processor 1002, and the communication interface 1003 are communicatively connected to each other via a bus 1004.
The Memory 1001 may be a Read Only Memory (ROM), a static Memory device, a dynamic Memory device, or a Random Access Memory (RAM). The memory 1001 may store a program, and when the program stored in the memory 1001 is executed by the processor 1002, the processor 1002 and the communication interface 1003 are used to execute the steps of the method for processing service data according to the embodiment of the present application.
The processor 1002 may be a general-purpose Central Processing Unit (CPU), a microprocessor, an Application Specific Integrated Circuit (ASIC), a Graphics Processing Unit (GPU), or one or more Integrated circuits, and is configured to execute related programs to implement functions required to be executed by units in the training apparatus of the XX network in the embodiment of the present Application, or to execute the method for training the XX network in the embodiment of the present Application.
The processor 1002 may also be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the XX network training method of the present application may be implemented by integrated logic circuits of hardware or instructions in the form of software in the processor 1002. The processor 1002 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 1001, and the processor 1002 reads information in the memory 1001, and completes, in combination with hardware thereof, functions required to be executed by a unit included in the service data processing apparatus according to the embodiment of the present application, or executes a service data processing method according to the embodiment of the method of the present application.
The communication interface 1003 enables communication between the apparatus 1000 and other devices or communication networks using transceiver means such as, but not limited to, a transceiver. For example, the service data (such as consumption behavior data and base station configuration cost data in the embodiment of the present application) may be acquired through the communication interface 1003.
Bus 1004 may include a pathway to transfer information between various components of device 1000 (e.g., memory 1001, processor 1002, communication interface 1003).
It should be understood that the first obtaining module 901 in the processing apparatus 900 of the service data may correspond to the communication interface 1003 in the processing apparatus 1000 of the service data, and the second obtaining module 902 and the processing module 903 may correspond to the processor 1002.
It should be noted that although the apparatus 1000 shown in fig. 10 shows only memories, processors, and communication interfaces, in a specific implementation, those skilled in the art will appreciate that the apparatus 1000 also includes other components necessary for normal operation. Also, those skilled in the art will appreciate that the apparatus 1000 may also include hardware components to implement other additional functions, according to particular needs. Furthermore, those skilled in the art will appreciate that apparatus 1000 may also include only those components necessary to implement embodiments of the present application, and need not include all of the components shown in FIG. 10.
The embodiment of the present application further provides a cloud server, which may include a data collector, a memory, a preprocessor, and an arithmetic circuit, where the data collector may be equivalent to the communication interface 1003 in the processing apparatus 1000 for business data shown in fig. 10, the memory may be equivalent to the memory 1001 in the processing apparatus 1000 for business data shown in fig. 10, the preprocessor and the arithmetic circuit may be equivalent to the processor 1002 in the processing apparatus 1000 for business data shown in fig. 10, and the data collector, the memory, the preprocessor, and the arithmetic circuit may be connected through the bus 1004 in the apparatus shown in fig. 10.
The embodiment of the present application further provides a computer readable medium, which stores a program code for device execution, where the program code includes a processing method for executing the service data in the foregoing embodiment.
The embodiments of the present application also provide a computer program product containing instructions, which when run on a computer, causes the computer to execute the method for processing business data in the foregoing embodiments.
The embodiment of the present application further provides a chip, where the chip includes a processor and a data interface, and the processor reads an instruction stored in a memory through the data interface to execute the method for processing the service data in the foregoing embodiment.
In a possible design, the chip may further include a memory, the memory stores instructions, and the processor is configured to execute the instructions stored in the memory, and when the instructions are executed, the processor is configured to execute the processing method of the business data in the foregoing embodiment.
The embodiment of the present application further provides an electronic device, where the electronic device includes a service data processing apparatus that executes the service data processing method in the foregoing embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (23)

1. A method for processing service data is characterized by comprising the following steps:
acquiring service data, wherein the service data comprises a plurality of individual data respectively corresponding to a plurality of individuals when the service is executed, each individual data comprises at least one sub-individual, and a multilayer incidence relation exists among the plurality of individual data;
acquiring input data corresponding to the business data, wherein the input data have a first structure, the first structure comprises vertexes with characteristic information and multiple types of connecting edges, each vertex corresponds to one of the multiple individuals, the characteristic information of one vertex comprises information determined according to vertex sub-individuals, the vertex sub-individuals are sub-individuals included in the individual data of the individual corresponding to the vertex, and the connecting edges comprise information determined according to the multilayer incidence relation among the multiple individual data;
and calculating the input data by using a preset neural network to obtain the class to which the plurality of individuals belong.
2. The method of claim 1, wherein obtaining input data corresponding to the service data comprises:
obtaining a plurality of vertices of the input data according to the plurality of individuals, each vertex including feature information of the vertex;
determining at least one connecting edge among the plurality of vertexes according to at least one layer of incidence relation in the multilayer incidence relation;
and acquiring the input data according to the plurality of vertexes and the at least one connecting edge.
3. The method of claim 2, wherein determining at least one continuous edge between the plurality of vertices according to at least one of the plurality of levels of associations comprises:
determining shared information between every two vertexes according to at least one layer of incidence relation in the multilayer incidence relation;
and acquiring the at least one connecting edge according to the shared information between every two vertexes.
4. The method according to claim 3, wherein the input data further includes a type of each of the at least one continuous edge, the type of each continuous edge is determined according to the content of the shared information between two vertices connected by the continuous edge, and if the content of the shared information corresponding to two continuous edges is the same, the types of the two continuous edges are the same.
5. The method according to claim 3 or 4, wherein the content of the shared information includes the same sub-individuals included in the individual data of two individuals, and the two individuals correspond to two vertexes connected by one connecting edge respectively.
6. The method of claim 5, wherein the total number of occurrences of the same sub-individual in the plurality of individual data is less than or equal to a preset threshold.
7. The method according to any one of claims 1-6, wherein the operation on the input data pairs using a preset neural network to obtain the class to which the plurality of individuals belong comprises:
acquiring an input matrix and a continuous matrix corresponding to the input data, wherein elements in a column vector or a row vector of the input matrix are used for indicating feature information corresponding to a vertex in the input data and an initial type of the vertex, and the initial type is determined according to a preset judgment condition; elements in a row vector or a column vector of the edge-connected matrix are used for indicating the type of an edge connected included by one vertex in the input data;
and according to the continuous edge matrix, taking the class to which the plurality of individuals belong as a loss function of the preset neural network, and using the preset neural network to calculate the input matrix to obtain the class to which the plurality of individuals belong.
8. The method according to claim 7, wherein the obtaining the class to which the plurality of individuals belong by using the preset neural network to operate the input matrix according to the continuous edge matrix by taking the class to which the plurality of individuals belong as a loss function of the preset neural network comprises:
according to the continuous edge matrix, taking the class to which the plurality of individuals belong as a loss function of the preset neural network, and using the preset neural network to calculate the input matrix to obtain an output matrix of the last hidden layer of the preset neural network, wherein a row vector in the output matrix is a state vector corresponding to each individual;
and clustering the output matrix to obtain the classes to which the individuals belong.
9. The method of claim 8, wherein clustering the output matrix comprises:
and clustering the output matrix by using a density clustering-based mode.
10. The method of claim 9, wherein prior to clustering the output matrix using density-based clustering, the method further comprises:
and performing dimension reduction processing on the output matrix.
11. A device for processing service data, comprising:
the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring service data, the service data comprises a plurality of individual data which respectively correspond to a plurality of individuals when the service is executed, each individual data comprises at least one sub-individual, and a multilayer incidence relation exists among the plurality of individual data;
a second obtaining module, configured to obtain input data corresponding to the service data, where the input data has a first structure, where the first structure includes vertices having feature information and multiple types of connected edges, each vertex corresponds to one of the multiple individuals, the feature information of one vertex includes information determined according to a vertex sub-individual, the vertex sub-individual is a sub-individual included in individual data of the individual corresponding to the vertex, and the connected edges include information determined according to the multilayer association relationship between the multiple individual data;
and the processing module is used for calculating the input data by using a preset neural network to acquire the classes to which the individuals belong.
12. The apparatus of claim 11, wherein the second obtaining module is specifically configured to:
obtaining a plurality of vertices of the input data according to the plurality of individuals, each vertex including feature information of the vertex;
determining at least one connecting edge among the plurality of vertexes according to at least one layer of incidence relation in the multilayer incidence relation;
and acquiring the input data according to the plurality of vertexes and the at least one connecting edge.
13. The apparatus of claim 12, wherein the second obtaining module is specifically configured to:
determining shared information between every two vertexes according to at least one layer of incidence relation in the multilayer incidence relation;
and acquiring the at least one connecting edge according to the shared information between every two vertexes.
14. The apparatus according to claim 13, wherein the input data further includes a type of each of the at least one continuous edge, the type of each continuous edge is determined according to content of shared information between two vertices connected by the continuous edge, and if the content of the shared information corresponding to two continuous edges is the same, the types of the two continuous edges are the same.
15. The apparatus according to claim 13 or 14, wherein the content of the shared information includes the same sub-individuals included in the individual data of two individuals, and the two individuals correspond to two vertexes connected by one connecting edge respectively.
16. The apparatus of claim 15, wherein the total number of occurrences of the same sub-individual in the plurality of individual data is less than or equal to a preset threshold.
17. The apparatus according to any one of claims 11-16, wherein the processing module is specifically configured to:
acquiring an input matrix and a continuous matrix corresponding to the input data, wherein elements in a column vector or a row vector of the input matrix are used for indicating feature information corresponding to a vertex in the input data and an initial type of the vertex, and the initial type is determined according to a preset judgment condition; elements in a row vector or a column vector of the edge-connected matrix are used for indicating the type of an edge connected included by one vertex in the input data;
and according to the continuous edge matrix, taking the class to which the plurality of individuals belong as a loss function of the preset neural network, and using the preset neural network to calculate the input matrix to obtain the class to which the plurality of individuals belong.
18. The apparatus of claim 17, wherein the processing module is specifically configured to:
according to the continuous edge matrix, taking the class to which the plurality of individuals belong as a loss function of the preset neural network, and using the preset neural network to calculate the input matrix to obtain an output matrix of the last hidden layer of the preset neural network, wherein a row vector in the output matrix is a state vector corresponding to each individual;
and clustering the output matrix to obtain the classes to which the individuals belong.
19. The apparatus of claim 18, wherein the processing module is specifically configured to:
and clustering the output matrix by using a density clustering-based mode.
20. The apparatus of claim 19, wherein the processing module is further configured to:
and performing dimension reduction processing on the output matrix.
21. The cloud server is characterized by comprising a data collector, a memory, a preprocessor and an arithmetic circuit, wherein:
the data acquisition unit is used for acquiring service data and storing the service data in a memory, the service data comprises a plurality of individual data respectively corresponding to a plurality of individuals when the service is executed, each individual data comprises at least one sub-individual, and a multilayer incidence relation exists among the individual data;
the preprocessor is used for acquiring the business data from the memory and acquiring input data corresponding to the business data, wherein the input data has a first structure, the first structure comprises vertexes with characteristic information and multiple types of connecting edges, each vertex corresponds to one of the multiple individuals, the characteristic information of one vertex comprises information determined according to vertex sub-individuals, the vertex sub-individuals are sub-individuals included in the individual data of the individual corresponding to the vertex, and the connecting edges comprise information determined according to the multilayer incidence relation among the multiple individual data;
the operation circuit is used for operating the input data by using a preset neural network to obtain the class to which the plurality of individuals belong.
22. An apparatus for processing service data, the apparatus comprising:
a memory for storing a software program;
a processor for reading the software program in the memory and performing the method of any one of claims 1-10.
23. A computer storage medium, characterized in that the storage medium has stored therein a software program which, when read and executed by one or more processors, implements the method of any one of claims 1-10.
CN202010103095.5A 2020-02-19 2020-02-19 Business data processing method and device and cloud server Pending CN113283921A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010103095.5A CN113283921A (en) 2020-02-19 2020-02-19 Business data processing method and device and cloud server

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010103095.5A CN113283921A (en) 2020-02-19 2020-02-19 Business data processing method and device and cloud server

Publications (1)

Publication Number Publication Date
CN113283921A true CN113283921A (en) 2021-08-20

Family

ID=77274890

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010103095.5A Pending CN113283921A (en) 2020-02-19 2020-02-19 Business data processing method and device and cloud server

Country Status (1)

Country Link
CN (1) CN113283921A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110009093A (en) * 2018-12-07 2019-07-12 阿里巴巴集团控股有限公司 For analyzing the nerve network system and method for relational network figure
CN110321484A (en) * 2019-06-18 2019-10-11 中国平安财产保险股份有限公司 A kind of Products Show method and device
CN110543586A (en) * 2019-09-04 2019-12-06 北京百度网讯科技有限公司 Multi-user identity fusion method, device, equipment and storage medium
CN110826706A (en) * 2018-08-10 2020-02-21 北京百度网讯科技有限公司 Data processing method and device for neural network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110826706A (en) * 2018-08-10 2020-02-21 北京百度网讯科技有限公司 Data processing method and device for neural network
CN110009093A (en) * 2018-12-07 2019-07-12 阿里巴巴集团控股有限公司 For analyzing the nerve network system and method for relational network figure
CN110321484A (en) * 2019-06-18 2019-10-11 中国平安财产保险股份有限公司 A kind of Products Show method and device
CN110543586A (en) * 2019-09-04 2019-12-06 北京百度网讯科技有限公司 Multi-user identity fusion method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
Ghaderizadeh et al. Hyperspectral image classification using a hybrid 3D-2D convolutional neural networks
Sim et al. Is deep learning for image recognition applicable to stock market prediction?
Xie et al. Point clouds learning with attention-based graph convolution networks
Qi et al. Convolutional neural network based detection and judgement of environmental obstacle in vehicle operation
Wang et al. Nonparametric density estimation for high‐dimensional data—Algorithms and applications
KR102005628B1 (en) Method and system for pre-processing machine learning data
Yeganejou et al. Interpretable deep convolutional fuzzy classifier
CN109934285B (en) Deep learning-based image classification neural network compression model system
CN109063719B (en) Image classification method combining structure similarity and class information
JP5591178B2 (en) Method for classifying objects in test images
Nawaz et al. AI-based object detection latest trends in remote sensing, multimedia and agriculture applications
Hameed et al. Class distribution-aware adaptive margins and cluster embedding for classification of fruit and vegetables at supermarket self-checkouts
Ge et al. Adaptive hash attention and lower triangular network for hyperspectral image classification
Sevastyanov et al. On methods for improving the accuracy of multi-class classification on imbalanced data.
Huang et al. View-based weight network for 3D object recognition
Muneer et al. Genetic algorithm based intelligent system for estate value estimation
CN110110628B (en) Method and equipment for detecting degradation of frequency synthesizer
CN115393631A (en) Hyperspectral image classification method based on Bayesian layer graph convolution neural network
Park et al. Unified convolutional neural network for direct facial keypoints detection
Zhang et al. Complex image recognition algorithm based on immune random forest model
Ouadiay et al. Simultaneous object detection and localization using convolutional neural networks
Nemade et al. Co-occurrence patterns based fruit quality detection for hierarchical fruit image annotation
CN111738812B (en) Information pushing method and system based on user group micro-segmentation
CN113283921A (en) Business data processing method and device and cloud server
Shi et al. Fuzzy support tensor product adaptive image classification for the internet of things

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination