WO2017193685A1 - 社交网络中数据的处理方法和装置 - Google Patents
社交网络中数据的处理方法和装置 Download PDFInfo
- Publication number
- WO2017193685A1 WO2017193685A1 PCT/CN2017/075931 CN2017075931W WO2017193685A1 WO 2017193685 A1 WO2017193685 A1 WO 2017193685A1 CN 2017075931 W CN2017075931 W CN 2017075931W WO 2017193685 A1 WO2017193685 A1 WO 2017193685A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- knowledge
- topic
- network node
- updated
- concept
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 99
- 238000012545 processing Methods 0.000 title claims abstract description 32
- 239000013598 vector Substances 0.000 claims description 238
- 238000005070 sampling Methods 0.000 claims description 67
- 238000009826 distribution Methods 0.000 claims description 54
- 238000012935 Averaging Methods 0.000 claims description 16
- 238000005295 random walk Methods 0.000 claims description 16
- 230000035508 accumulation Effects 0.000 claims description 8
- 238000009825 accumulation Methods 0.000 claims description 8
- 230000000717 retained effect Effects 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000003993 interaction Effects 0.000 abstract description 6
- 230000015654 memory Effects 0.000 description 33
- 230000006870 function Effects 0.000 description 17
- 238000004422 calculation algorithm Methods 0.000 description 13
- 238000002474 experimental method Methods 0.000 description 11
- 238000003672 processing method Methods 0.000 description 11
- 238000011160 research Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 4
- 230000001360 synchronised effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000001186 cumulative effect Effects 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 1
- 244000141359 Malus pumila Species 0.000 description 1
- 240000008790 Musa x paradisiaca Species 0.000 description 1
- 241000220324 Pyrus Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 235000021016 apples Nutrition 0.000 description 1
- 235000021015 bananas Nutrition 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 235000021017 pears Nutrition 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000699 topical effect Effects 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/01—Social networking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/027—Frames
Definitions
- the present application relates to the field of computers, and more particularly to a method and apparatus for data processing in a social network in the field of computers.
- a social network can be an online social platform with a large user base, such as Facebook, Twitter, Sina Weibo, WeChat, etc., or a social network that is not based on an online platform, such as an academic social network. And offline network of relationships.
- An important research topic of social network data mining is how to understand the behavior of massive users and explore the interests and concerns of massive users. For example, in an online social network, we want to mine the interests and concerns of each user to more accurately advertise and personalize user recommendations. As far as academic social networks are concerned, each researcher is a node in this academic social network. The academic partnership between them constitutes a huge academic social network. We hope to explore the research of each researcher. Interest, so that researchers can more effectively help researchers find a research direction, conduct academic cooperation recommendations, and enhance knowledge-based academic search capabilities.
- Embodiments of the present application provide a method and apparatus for data processing in a social network, capable of establishing an effective relationship between a social network and a knowledge map.
- an embodiment of the present application provides a data processing method in a social network, where the method includes:
- the D i-th information of the weak weak supervision supervision information includes a network node D i social network associated with the network node and D i M i th knowledge of concepts, wherein the The network node d i is represented as an embedded vector, and the M i knowledge concepts are respectively represented as embedded vectors, D and M i are positive integers greater than 1, respectively, and i is a positive integer ranging from 1 to D;
- the heterogeneous Bayesian model including T topics and D documents, and the jth topic among the T topics includes a parameter ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k , the i-th document in the D documents includes the network node d i and the M i knowledge concepts, and the i-th document further includes M i th of said concept knowledge of the correspondence M i th first topic of hidden variables, the network node and the second topic d i hidden variables and parameters ⁇ one correspondence, the network node corresponding to a second topic d i
- the hidden variable can be one.
- ⁇ j r represents the mean of the Gaussian distribution of the network node d i corresponding to the jth topic
- ⁇ j r represents the accuracy of the Gaussian distribution of the network node d i corresponding to the jth topic
- ⁇ jm k represents the mean of the Gaussian distribution of the mth knowledge concept in the M i knowledge concepts corresponding to the jth topic
- ⁇ jm k represents the M i knowledge corresponding to the jth topic accuracy of m Gaussian distribution concept in the concept of knowledge
- ⁇ d i indicates the topic of the network node number distribution
- T is a positive integer greater than 1
- j is a positive integer ranging from 1 to T
- m is a positive integer ranging from 1 to M i ;
- a method for processing data in a social network by determining a series of weak supervisory information indicating interaction information between a network node and a knowledge concept, and generating a heterogeneous Bayesian model according to the weak supervisory information,
- the heterogeneous Bayesian model represents a connection model between a social network and a knowledge map, and then based on the heterogeneous Bayesian model, an effective relationship between a given network node and a given knowledge concept can be determined.
- the probability that the first network node and the first knowledge concept become neighbors can be calculated by:
- the probability that a given network node a and a given knowledge concept 1 become neighbors is the conditional probability of the network node a and the knowledge concept 1.
- the value of the conditional probability is relatively large, it indicates that the knowledge concept 1 is the core knowledge concept of the network node a.
- the knowledge concept is expressed as the edge knowledge concept of the network node a. Therefore, the embodiment of the present application can conveniently distinguish the core knowledge concept and the edge knowledge concept related to each network node in the social network.
- the generating the heterogeneous Bayesian model according to the D weak supervision information including:
- the initial values of the M i first topic hidden variables and the initial values of the second topic hidden variables are randomly determined, where the number of second topic hidden variables is the same as the number of network nodes.
- conditional probability of the network node topic d i represents the second topic d i hidden variable node in a given network, among other topics than the second topic hidden variables d i hidden variables of the network node, the the second topic hidden variables, in the case where the conditional probability for each concept knowledge embedded network node vector and the vector d i is embedded.
- the document -d represents a document other than the document d
- y -d represents a second topic hidden variable of the document -d
- the function G'( ⁇ ) is defined as:
- the initial value of M i the first topic of hidden variables, conditional probability of the i-th weak supervision information M i and a knowledge of the topics in the concept of the subject M i th first hidden variable t a b -sampling update, wherein a conditional probability of a topic of each of the conditional probabilities of the topics of the M i knowledge concepts indicates that the first topic hidden variable of each of the knowledge concepts is given in addition to each Other topic hidden variables other than the first topic hidden variable of the knowledge concept, the M i first topic hidden variables, the embedded vector of each of the knowledge concepts, and the embedded vector of the network node d i Conditional probability in the case.
- the updated M i first topic hidden variables and the updated The second topic hidden variable and the parameters ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k are sample updated to generate the heterogeneous Bayesian model.
- the updated M i after performing the t b sampling update on the M i first topic hidden variables and the second topic hidden variable respectively, respectively, the updated M i
- the first topic hidden variable, the updated second topic hidden variable, and the parameters ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k are updated, including:
- conditional probability of the i-th topic weak supervision information and the network node D i the topic of the second hidden variables for the updated update sampling times t p .
- conditional probability of the topic of the network node d i and the conditional probability of the topic of the M i knowledge concepts are respectively the above formula (1) and formula (2).
- the topic distribution parameter ⁇ d is updated as follows:
- updating the ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , and ⁇ jm k respectively includes: respectively, the parameters ⁇ , ⁇ j r , ⁇ j r , j jm k , ⁇ jm k performs N sampling updates, wherein each update of the N updates performs t p sampling update on the updated M i first topic hidden variables,
- the updated M i first topic hidden variables are updated by t p sampling and updated by ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k respectively to obtain N times updated ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k ;
- the ⁇ jm k after the N updates is cumulatively averaged, and the average value obtained by accumulating ⁇ jm k is taken as the ⁇ jm k parameter of the updated heterogeneous Bayesian model.
- the updated M i first topic hidden variables, the updated second topic hidden variables, and the parameters ⁇ , ⁇ j r , ⁇ j respectively r, ⁇ jm k, ⁇ jm k after the update further comprising:
- the embedded vector d i and the network node M i th vector of the embedded knowledge of concepts, in a case of determining the initial given the hidden variables M i th first and second latent variables The number is like.
- the log likelihood of the embedded vector without the updated network node d i and the embedded vector without the updated M i knowledge concept is referred to as the initial log likelihood.
- said M is determined after the i-th embedded knowledge vector updated concept.
- the determining, according to the initial log likelihood, the embedded vector that is updated by the network node d i includes:
- the gradient of the embedded representation of the network node is:
- the determining, according to the initial log likelihood, respectively, the embedded vector after the updated M i knowledge concepts including:
- the gradient of the embedded representation of the knowledge concept is:
- the gradient M, M i th concept learning knowledge and concepts of the knowledge of the embedded vector M i represents a concept of knowledge according to the concept of embedded knowledge of the knowledge of M i th concept, determining said updated after each An embedded vector of i knowledge concepts, wherein the knowledge concept learning rate of each of the M i knowledge concepts represents a learning rate of a gradient of the embedded vector of each knowledge concept.
- the method further includes:
- an update log likelihood the number of embedded vector after update after update embedded network node vector d i and M i of the concept of a likelihood of knowledge.
- the network node embedded vector d i is replaced with the updated embedded vector d i of the network node, the M i th vector of embedded knowledge embedded vector concept replacing the updated knowledge M i th concept;
- update the network node learning rate to a product of the network node learning rate and a first attenuation index
- the M i knowledge concepts The knowledge concept learning rate is respectively updated to the product of the knowledge concept learning rate of the M i knowledge concepts and the second attenuation index, and the embedded vector of the network node d i is kept unchanged, and the M i knowledge concepts are retained.
- the embedded vector is invariant, wherein the first attenuation index represents a degree of attenuation of the network node learning rate, and the second attenuation index represents each knowledge concept learning rate of the M i knowledge concept learning rates The degree of attenuation.
- first attenuation index and the second attenuation index may be the same or different, which is not limited in the present application.
- the determining D weak supervision information includes:
- Determining D initial weak supervision information wherein the i-th initial weak supervision information in the D initial weak supervision information includes the network node d i and all knowledge concepts related to the network node di;
- the unsupervised generation rate r is a preset ratio of the number of determined knowledge concepts M i to the number of all knowledge concepts, and r ranges from 0 to 1.
- the concept of knowledge The score can be expressed as:
- n is The number of occurrences in all knowledge concepts D d .
- the method before the determining D weak supervision information, the method further includes:
- the converted corpus is input to the first skipgram model, and an embedded vector of each knowledge concept in the knowledge concept set is obtained.
- the method before the determining D weak supervision information, the method further includes:
- the weighted probability of the network node can be expressed as:
- the first skipgram model and the second skipgram model may be the same or different, and the present application does not limit this.
- the embodiment of the present application provides a device for processing data in a social network, which is used to perform the method in any of the foregoing first aspect or any possible implementation manner of the first aspect.
- the device includes A module of the method of any of the preceding aspects or any of the possible implementations of the first aspect.
- an embodiment of the present application provides an apparatus for data processing in a social network, where the apparatus includes: a memory, a processor, a transceiver, and a bus system. Wherein the memory and the processor are coupled by the bus system for storing instructions for executing instructions stored by the memory, and when the processor executes the instructions stored by the memory, the performing causes the processing The method of the first aspect or any possible implementation of the first aspect is performed.
- the embodiment of the present application provides a computer readable medium for storing a computer program, the computer program comprising instructions for executing the method in the first aspect or any possible implementation manner of the first aspect.
- FIG. 1 is an application scenario diagram of a method for data processing in a social network according to an embodiment of the present application.
- FIG. 2 is a schematic flowchart of a method for data processing in a social network according to an embodiment of the present application.
- FIG. 3 is a schematic block diagram of a heterogeneous Bayesian model generated in a method of data processing in a social network according to an embodiment of the present application.
- FIG. 4 is a schematic block diagram of an apparatus for data processing in a social network according to an embodiment of the present application.
- FIG. 5 is a schematic block diagram of an apparatus for data processing in a social network according to another embodiment of the present application.
- FIG. 1 is a schematic block diagram showing an application scenario of a method for data processing in a social network according to an embodiment of the present application. It can be understood that a large-scale social network may have many network nodes, and a large-scale corpus may have many knowledge concepts or terms. FIG. 1 only schematically shows some network nodes and corpora in the social network. Part of the knowledge concept or entry.
- the social network in FIG. 1 may include a plurality of network nodes, which may be, for example, a, b, c, d, or e in the figure.
- the network nodes a, b, c, d, and e can be different experts, academic institutions, or research teams, respectively.
- network nodes a, b, c, d, and e may be different bloggers, respectively.
- the understanding of social networks requires portraying the network structure of social networks.
- the method of distributed vector embedding can be applied to the social network according to Deepwalk, and an embedded vector is learned for each social network network node, that is, the network structure of the social network can be characterized.
- the embedded vector may be referred to as an embedded vector representation.
- the corpus in Figure 1 can be all the knowledge concepts in Wikipedia or all the entries in Baidu Encyclopedia.
- 1, 2, 3, 4, 5, 6, 7, or 8 in the corpus are Wikipedia.
- Different knowledge concepts, or 1, 2, 3, 4, 5, 6, 7, or 8 in the corpus are different entries in Baidu Encyclopedia, respectively, and these knowledge concepts (or terms) can be, for example, big data, Smart home, machine learning, Tsinghua University, optical communication, Long Term Evolution (LTE), etc.
- LTE Long Term Evolution
- the topic model can be, for example, a probabilistic Latent Semantic Indexing (pLSI) model and a Latent Dirichlet Allocation (LDA) model.
- pLSI probabilistic Latent Semantic Indexing
- LDA Latent Dirichlet Allocation
- LDA is a Bayesian generation model that ultimately generates each word in each document by generating a topic distribution for each document and generating a word distribution for each topic.
- the advantage of the topic model is that it can characterize the semantics of different words through topic hidden variables and constrain them with the distribution of documents and words.
- the topic model can only output discrete words and cannot output continuous semantic representations.
- the words corresponding to the topic may be basketball, Yao Ming, the World Cup, the Olympics, etc.; when the topic is fruit, the words corresponding to the topic may be apples, pears, bananas, and the like.
- a topic can also include multiple words.
- the topic model can generate multiple topic distributions for each document, generate multiple word distributions for each topic, and finally generate each document in each document. Words.
- the embedded model is a learning method of distributed vector embedded representation, such as the Word2Vec learning method.
- the Word2Vec learning method learns the continuous vector embedded representation of each word by optimizing the likelihood of occurrence of a word in a given context.
- the position of each word in the feature space characterizes the semantics of the word.
- Distributed vector embedded learning The advantage is that each word can be represented on a continuous feature space, and the optimization target can characterize the second-order co-occurrence relationship. However, it is difficult for the embedded model to understand the implicit topic of each word.
- the effective connection may be a conditional probability of the network node and the knowledge concept (or term).
- the conditional probability between a network node in a large-scale social network and a knowledge concept (or term) in a corpus may also be referred to as a link probability between the network node and the knowledge concept (or term), or become the The probability that a network node becomes a neighbor with this knowledge concept.
- a network node can have an effective connection with multiple knowledge concepts.
- network node b has an effective association with knowledge concepts (or terms) 2, knowledge concepts (or terms) 4, and knowledge concepts (or terms) 5, respectively.
- a knowledge concept (or term) can also have an effective connection with multiple network nodes at the same time.
- the knowledge concept (or term) 6 has an effective relationship with the network node e and the network node d, respectively. Therefore, there can be complex links between large-scale social networks and large-scale knowledge maps.
- the structural information of the social network and the textual semantic information in the corpus are two kinds of heterogeneous information.
- only one type of information can be processed, and there is no processing method for associating the two kinds of information. Therefore, there is a lack of universal connection between large-scale social networks and large-scale knowledge maps in the prior art, and it is not convenient to use knowledge maps through social networks, that is, existing technologies cannot be between social networks and knowledge maps. Establish the above effective links.
- a data processing method in a social network provided by an embodiment of the present application can establish an effective relationship between a social network and a knowledge map, thereby effectively distinguishing core knowledge concepts and edges related to each network node in the social network. Knowledge concept.
- the embodiment of the present application can learn the embedded vector representation of each knowledge concept unsupervised in the corpus, and the embedded vector representation can describe the text semantics of the knowledge concept.
- the method for determining the embedded vector representation of each knowledge concept in the embodiment of the present application can be applied to the academic social network system Aminer, and the corpus C is all knowledge concepts in Wikipedia.
- the method for determining the embedded vector representation of each knowledge concept in the embodiment of the present application can also be applied to other social networks, such as Sina Weibo, in which all the entries in Baidu Encyclopedia can be used as the corpus C.
- the corpus may be converted according to the knowledge concept set, the length of the corpus, and the maximum N-gram length in the corpus, the knowledge concept set including the Describe all knowledge concepts in D weak supervisory information.
- V k based on the set required knowledge corpus conversion concept.
- N represents the length of the word
- gram represents the segment obtained by dividing the original word by the length N.
- N takes the minimum rule. If the length of the original word is greater than 5, the minimum value of N is 3. If the length of the original word is equal to 5, the minimum value is 2, otherwise the minimum value is 1.
- N is then decremented by one to continue all N-gram in the corpus of knowledge and concepts set of matching V k, if the N-gram in the set of knowledge exists in the concept of V k, N-gram in the corpus will replace a word .
- the enumeration of the N-gram starts from the largest N, and the value of N is 2. Therefore, if one knowledge concept is a string of another knowledge concept, only the longest string is considered.
- the converted corpus is input to the skipgram model, and an embedded vector representation of each knowledge concept in the knowledge concept set can be obtained.
- the objective function of the skipgram model is to calculate the conditional probability that the distance between the current word and the current word is less than the threshold given the currently examined word.
- the embodiment of the present application adopts a historical softmax to speed up the training, that is, according to the word frequency generation, a Huffman greedy algorithm is used to generate a Huffman tree, in Huff. After the Man tree is generated, the conditional probability is expressed as:
- w is the word to be predicted
- w I as the context word
- L (w) represents the Huffman tree from the root node to the path length of the word w
- n (w, j) represents the Huffman The root node to the jth node on the path of the word w.
- Ch(w) represents a child of the w node in the Huffman tree. Since the Huffman tree is a binary tree, each node has 0 or 2 children, and ch(w) is a non-leaf node. Any one specified in advance. Is an indicator function that returns 1 if the variable is true, otherwise -1.
- ⁇ ( ⁇ ) is a softmax function.
- the historical softmax is to organize all the nodes into a binary tree. There is a fork in each non-leaf node. The probability is divided into two halves. The probability of reaching a certain node is equal to the product of all the assigned probabilities of the path. It is easy to see that according to this property, it is no longer necessary to normalize the probability because the probability of all leaf nodes adds up to 1.
- the embodiment of the present application can obtain the semantics of the knowledge concept through a large amount of unlabeled data, thereby helping to distinguish different topics of the knowledge concept, and distinguishing the main knowledge concept and the secondary knowledge concept corresponding to the social network user.
- the time complexity of the method for determining the embedded vector representation of the knowledge concept is reduced from the original linear level to the logarithmic level, and can be applied to large-scale corpora.
- an embedded vector representation of each network node may be learned from an unsupervised manner in a large-scale social network, and the embedded vector representation can describe the structural information of the social network.
- the method for determining the embedded vector representation of each network node in the embodiment of the present application can be applied to the academic social network system Aminer, and the network node is each expert scholar.
- the method for determining the embedded vector representation of each network node in the embodiment of the present application can also be applied to other social networks such as Sina Weibo, where the network node is each blogger.
- an unsupervised learning of an embedded vector representation of each network node from a large-scale social network may first determine a weighted probability of each network node in the social network, wherein each of the network nodes The weighted probability indicates the probability that each of the network nodes accesses the next network node, and all network nodes in the social network include all of the D weak supervisory information.
- the DeepWalk algorithm can be used to learn the embedding of each node in the social network.
- the DeepWalk algorithm generates a certain number of random walks from each node, and then trains the sequence of nodes obtained by random walks as a corpus to train the Skipgram model.
- the DeepWalk algorithm is proposed for an unweighted undirected graph, but in an embodiment of the invention, the edges on the social network should be entitled to be undirected. Therefore, the DeepWalk algorithm cannot be directly applied.
- a weighted probability is used to select the next visited node.
- the first item on the right side of the equal sign indicates that the node u must be a neighbor of v, and n u, v represents the integer weight on the u and v sides.
- determining, according to a weighted probability of each network node, at least one random walk sequence of each network node, each of the network nodes At least one random walk sequence is input to the skipgram model, and an embedded vector representation of each of the network nodes can be obtained.
- the embodiment of the present application performs sampling according to the above probability, and may generate a random walk sequence with a weighted undirected graph.
- a random walk sequence with a weighted undirected graph.
- an embedded representation of each node on the social network can be learned.
- the embedded representation of unsupervised learning network nodes in social networks can effectively solve the sparseness problem of data and make full use of the network structure to help generate social knowledge maps.
- the skipgram model for determining the embedded vector representation of the knowledge concept may be the same as the skipgram model for determining the embedded vector representation of the network node, or may be different, which is not limited by the embodiment of the present application.
- FIG. 2 is a schematic flowchart of a method 100 for data processing in a social network according to an embodiment of the present application.
- the method 100 includes:
- the i-th weak D Supervising weak supervision information includes a network node D i social network associated with the network node and D i M i th knowledge of concepts, wherein the network node is represented as D i
- the embedded vector representation, the M i knowledge concepts are respectively represented as embedded vector representations, D and M i are positive integers greater than 1, respectively, and i is a positive integer ranging from 1 to D.
- the input of the heterogeneous Bayesian model to be generated is a series of weak supervisory information, such as the above D weak supervisory information.
- the D network nodes in the D weak supervisory information are part of the network nodes in the experimentally available large-scale social network. Because the weak supervisory information includes a series of network nodes and all knowledge concepts that interact with each network node, the weak supervisory information can represent the interaction information between the network nodes and the text.
- embodiments of the present application are able to perform the establishment of a heterogeneous Bayesian model based on given weak supervisory information.
- the method for determining weak supervisory information in the embodiment of the present application can be applied to the academic social network system Aminer, in which case the weak supervisory information is the knowledge concept contained in all the papers of a researcher, that is, the title of each paper published and Summary.
- the method for determining weak supervision information in the embodiment of the present application can also be applied to other social networks such as Sina Weibo, where the weak supervision information is all the microblogs published by each blogger.
- the algorithm proposes an unsupervised generating method for weak supervisory information, that is, the embodiment of the present application can filter out all weak supervisory information. Part of the data, using the filtered data to train a better performing model.
- the following is a method for filtering data according to an embodiment of the present application.
- the embodiment of the present application needs to determine D initial weak supervisory information, where the i-th initial weak supervisory information in the D initial weak supervisory information includes the network node d i and all knowledge related to the network node di concept.
- the M i knowledge concepts are a score greater than each of the knowledge concepts except the M i knowledge concepts, the score of each knowledge concept indicating that each knowledge concept is at the center of all the knowledge concepts
- the degree of location, the unsupervised generation rate r is a preset ratio of the number of determined knowledge concepts M i to the number of all knowledge concepts, and r ranges from 0 to 1.
- a weighted sum of cosine similarities of a certain knowledge concept and all other knowledge concepts is calculated according to the following formula.
- n is The number of occurrences in all knowledge concepts D d .
- the weighted sum is used as the knowledge concept.
- Knowledge concept The higher the score, the more central the knowledge concept is in the knowledge concept in D d .
- Knowledge concept in the above calculation process The number of occurrences in D d as weights, making The more you appear, the more you contribute to its score.
- ⁇ r knowledge concept with the largest score is screened as D' d .
- the heterogeneous Bayesian model includes T topics and D documents, and the jth topic among the T topics includes parameters ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k , the D the i-th document comprising the document and the network node D i M i-th concept knowledge, the i-th document further comprising the M and M i the i-th concept of knowledge correspondence of first hidden topic a variable, a second topic hidden variable corresponding to the network node d i and a parameter ⁇ .
- the superscript r represents a researcher (researcher), that is, one of a plurality of network node types
- the superscript k represents a keyword, that is, one of a plurality of knowledge concept types.
- ⁇ j r represents the mean value of the Gaussian distribution of the network node d i corresponding to the jth topic
- ⁇ j r represents the accuracy of the Gaussian distribution of the network node d i corresponding to the jth topic
- ⁇ jm k represents the mean of the Gaussian distribution of the mth knowledge concept in the M i knowledge concepts corresponding to the jth topic
- ⁇ jm k represents the M i corresponding to the jth topic knowledge of m concept accuracy Gaussian distribution concept knowledge
- ⁇ d i representing the subject of a number of distributed network nodes
- T is a positive integer greater than 1
- j is a positive integer ranging from 1 to T
- m is Is a positive integer ranging from 1 to M i .
- the heterogeneous Bayesian model in the embodiments of the present application is capable of establishing a connection between heterogeneous embedded representations through LDA-like topic hidden variables.
- a weak supervisory information item may be referred to as a document. Therefore, in the heterogeneous Bayesian model, each document may include a social network node and interact with the social network node. All knowledge concepts.
- the method of data processing in the social network of the embodiment of the present application has universality. Different knowledge maps can be generated depending on the social network and corpus entered.
- input is a scholar in academic social networks, and the corresponding corpus is some knowledge related to scholar' research interests.
- interactive information is a paper published by scholars, then you can generate a knowledge map of scholar' research interests.
- the input is a blogger in the entertainment industry of Weibo social network
- the corpus is the knowledge concept related to film and television entertainment
- the interactive information is the post published by the blogger, then the generated knowledge map is the most concerned about the entertainment of the film and television. hot spot.
- the first topic hidden variable is referred to as a topic hidden variable z
- the second topic hidden variable is referred to as y.
- ⁇ is the hyperparameter of the Dirichlet distribution
- f r is the embedded representation of the social network node
- f k is the embedded representation of the knowledge concept
- ⁇ k is the normal Gamma a priori hyperparameter of the knowledge concept embedded representation
- ⁇ r is The social network node embeds the representation of the normal Gamma a priori hyperparameter.
- ⁇ k and ⁇ k are generated from the distribution NG( ⁇ k ) for the network node
- Each dimension of the embedded representation generates ⁇ r and ⁇ r from the distribution NG( ⁇ r ).
- the dimension represented by the embedded vector is approximately 200.
- the subscripts of ⁇ k , ⁇ k , ⁇ r , and ⁇ r are omitted for the sake of simplicity.
- a topic polynomial distribution ⁇ d is generated from the Dir ( ⁇ ) distribution.
- the topic z of the knowledge concept can be generated from the multinomial distribution ⁇ d , for each dimension of each knowledge concept w, generated from a Gaussian distribution N( ⁇ k z , ⁇ k z )
- the embedded vector represents f k .
- the topic y of the network node may also be generated from the uniform of the topic z of all knowledge concepts in the document d, and the Gaussian distribution N ( ⁇ r y , ⁇ for each dimension of the embedded vector representation of the network node r y ) generates an embedded vector representation f k .
- the distribution of ⁇ obeys the Dirichlet distribution, and the hyperparameter is ⁇ .
- the probability can be expressed as:
- the parameters ⁇ and ⁇ of the Gaussian distribution are generated by the normal Gamma distribution, and the hyperparameters are ⁇ r and ⁇ k . Therefore, the probability of a Gaussian distribution parameter can be written as follows:
- the subscript t represents the topic
- e represents a certain dimension of the embedded vector representation
- the hyperparameter ⁇ of the normal Gamma is expanded into four scalar parameters ⁇ 0 , ⁇ 0 , ⁇ 0 , ⁇ 0 ⁇ .
- the probability of generating a topic from a multinomial distribution is as follows:
- the subscript d represents a document
- the subscript m represents a concept of knowledge
- the probability of generating a topic of a network node from the topic of knowledge concept is a uniform distribution. Since a knowledge concept topic may appear multiple times, the probability that the topic is generated is proportional to the number of occurrences:
- the subscript d represents a document
- the subscript m represents a concept of knowledge
- the topic of the network node must appear in the topic of the knowledge concept, because the probability of the topic that has not appeared is zero.
- the Laplacian smoothing technique is adopted, so the probability of generating a network node topic can be rewritten as follows:
- ⁇ k and ⁇ k are values in the corresponding dimension of the corresponding topic, for the sake of brevity, the subscript is omitted here.
- ⁇ r and ⁇ r are values in the corresponding dimension of the corresponding topic, and for the sake of brevity, the subscript is omitted here.
- n d is a vector of length T
- each element in the vector Represents the number of knowledge concepts in document d that are assigned to topic t.
- ⁇ is a hyperparameter of the Dirichlet function and is also a vector of length T.
- E r and E k respectively represent conceptual knowledge and dimensions of the embedded network node representation.
- the Gibbs sampling is used to perform probabilistic reasoning on the model.
- the conditional probability of each variable in the case of other variables is derived.
- the topic distribution parameter ⁇ d is updated as follows:
- the prior art does not update the embedded representation in the Gaussian LDA model in which the embedded representation model and the topic model are combined.
- it is proposed to update the embedded vector representation of the network node and the knowledge concept, so that the weak supervision learning can be better utilized to correct the deficiency of the embedded representation obtained by the unsupervised learning.
- the objective function is defined as the log likelihood of the embedded representation given the hidden variables z and y, as follows:
- the embodiment of the present application directly obtains the embedded representation of the closed-form from the above objective function.
- the embodiment of the present application proposes to update the embedded representation by using a gradient descent method.
- the gradient of the embedded representation of the network node is:
- the topic hidden variable and the model parameter of the model may be learned according to the D weak monitoring information.
- the learning process of the model is divided into three phases: initialization, burn-in, and sampling.
- the initial values of the M i first topic hidden variables and the initial values of the second topic hidden variables may be randomly determined.
- the number of second topic hidden variables is the same as the number of network nodes, that is, the second topic hidden variable has a one-to-one correspondence with the network nodes.
- the topic hidden variables y and z can be initialized, and each topic hidden variable is randomly assigned to a topic by uniform.
- the topic hidden variable z also has M i and corresponds one-to-one with the above-mentioned M i knowledge concepts
- the topic hidden variable y has one, that is, the network node d i Corresponding topic hidden variable.
- the subscript of the topic dependent variable z is omitted.
- the method of collapsed Gibbs sampling is used for reasoning.
- the values of other variables are fixed, the conditional probability distribution of the currently investigated variable given the other variables is calculated, and then the samples are sampled from the distribution.
- the number of iterations of burn-in is set to t b , that is, the iteration of t b times in the burn-in phase.
- M i according to the initial values of the first topic of hidden variables, conditional probability of the i-th weak supervision information M i and a knowledge of the topics in the concept of the M i th
- the first topic hidden variable is t b sample update.
- conditional probability of the topic of each knowledge concept in the conditional probability of the topic of the M i knowledge concept indicates that the first topic hidden variable of each knowledge concept is given in addition to each of the knowledge concepts A conditional probability in the case of a topic hidden variable other than the topic hidden variable, the second topic hidden variable, the embedded vector of each of the knowledge concepts, and the embedded vector of the network node d i .
- Conditional probability of the network node topic d i represents the second topic d i hidden variable node in a given network, among other topics than the second topic hidden variables d i hidden variables of the network node, the Conditional probability in the case of M i first topic hidden variables, embedded vectors of each of the knowledge concepts, and embedded vectors of the network node d i .
- the topic hidden variable z of each knowledge concept in the model is sampled and generated according to the above formula (2), and the topic of each network node in the model is generated.
- the hidden variable is sampled and generated according to the above formula (1).
- the values of the other variables in the formulas (1) and (2) are determined based on the above D weak supervisory information.
- the topic hidden variables y and z can be sampled and updated.
- the model parameter or the embedded vector representation is not updated.
- the initial value of the topic hidden variable y is the value of the topic hidden variable y after the first iteration update
- the initial value of the topic hidden variable z is the value of the topic hidden variable z after the first iteration update. It should be noted that in the t b iterations of the burn-in phase.
- the initial value of the topic hidden variable y for each iteration is the value of the topic hidden variable y after the last iteration update
- the initial value of the topic hidden variable z is the value of the topic hidden variable z after the last iteration update.
- the updated M i first topic hidden variables and the updated second topic convergence respectively, the updated M i first topic hidden variables, the updated second topic hidden variables and the respectively
- the parameters ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , and ⁇ jm k are sampled and updated.
- the topic hidden variables y and z of the model basically enter a stable state, that is, after the topic hidden variables y and z of the model basically converge, the learning phase of the model enters the sampling phase.
- the topic hidden variables y and z and the model parameters can be sampled and updated separately.
- the maximum number of iterations of the sampling phase is set to t m , that is, the sampling phase performs at most t m iterations. It is also possible to set the model parameter update period to t p .
- the sample update variables y and z and the model parameters are respectively updated, which may be based on the updated M i first topic hidden variables, the ith weak monitoring information, and the said M i th conditional probability topical knowledge of concepts, topics of M i th first hidden variables for the updated sampling times t p updated; according to the second subject the updated hidden variables, the i-th topics weak supervision conditional probability information d i and the network node, the second subject of the updated hidden variables for updating sampling times t p; Out of the updated M i th hidden variables and a first topic After the updated second topic hidden variable is updated by t p sampling, ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k are updated respectively.
- time t p may be updated sample of the updated M i th first latent variables subject, the subject of the second hidden variables for the updated after updating sampling times t p, respectively, ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , and ⁇ jm k are updated once.
- updating the ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , and ⁇ jm k respectively includes: respectively, the parameters ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k performs N sampling updates, wherein each update in the N updates performs t p sampling update on the updated M i first topic hidden variables, for the update
- the following M i first hidden variables are t p -sampling updates and ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k are updated respectively to obtain ⁇ , ⁇ j r after N updates. , ⁇ j r , ⁇ jm k , ⁇ jm k .
- the ⁇ is cumulatively averaged after N times of updating, and the average value obtained by ⁇ is added as the ⁇ parameter of the updated heterogeneous Bayesian model;
- ⁇ ⁇ j r is cumulatively averaged for N times, and the average value obtained by ⁇ j r is added as the ⁇ j r parameter of the updated heterogeneous Bayesian model;
- the average value of ⁇ jm k after N updates is averaged, and the average value obtained by accumulating ⁇ jm k is used as the ⁇ jm k parameter of the updated heterogeneous Bayesian model;
- the average value of the ⁇ jm k after the N update is cumulatively averaged, and the average value obtained by accumulating ⁇ jm k is taken as the ⁇ jm k parameter of the updated heterogeneous Bayesian model.
- model parameters obtained by calculating the plurality of periods t p may be accumulated, and the average value of all the read model parameters in the sampling stage is taken as the final model parameter.
- the number of iterations of the topic hidden variable in the sampling phase can be set to t l . Therefore, after t p -sampling update of the topic hidden variable and updating of the model parameters ⁇ , ⁇ k , ⁇ r , ⁇ k and ⁇ r , the topic hidden variable and the model parameters ⁇ , ⁇ can be continued as described above. k , ⁇ r , ⁇ k and ⁇ r are updated. It will be appreciated, when a total of a subject hidden variables updated sampling times t l, the model parameter ⁇ , ⁇ k, ⁇ r, ⁇ k and ⁇ r conducted (t l / t p) update.
- the embedding vector network nodes and updating the concept of knowledge representation may comprise: the network node according embedded vector representation and embedded d i M i th vector of the conceptual representation of knowledge, it is determined given the M An initial log likelihood in the case of i first hidden variables and the second hidden variable; determining an embedded vector representation of the updated network node d i according to the initial log likelihood; An initial log likelihood is used to determine an embedded vector representation of the updated M i knowledge concepts, respectively.
- the embedded vector representation of each network node and knowledge concept can be updated.
- the number of iterations of the embedded vector representation can be set to t e times, and the initial learning rate of each network node is lr r , and the initial learning rate of each knowledge concept is lr k .
- the initial log likelihood may be L above.
- An embedded vector representation of i determines an updated embedded vector representation of the network node d i , wherein the network node learning rate represents a learning rate of a gradient of the embedded vector of the network node d i .
- the embedding vector for each network node represents f r
- the gradient g 1 of each network node can be calculated according to formula (8).
- the updated embedded vector representation of the network node can be determined according to f r +lr r ⁇ g 1 .
- the embedding vector for each knowledge concept represents f k
- the gradient g 2 of each knowledge concept can be calculated according to formula (9).
- the updated embedded vector representation of the knowledge concept can be determined from f k +lr k ⁇ g 2 .
- the update log likelihood of the first hidden variable and the second hidden variable.
- the update log likelihood is greater than or equal to the initial log-likelihood, will be embedded in the network node vector d i is represented as replacing the updated embedded vector d i represents a network node, the embedded vector M i represents a concept of knowledge representation is replaced after updating the embedded vector M i th concept of knowledge.
- update the network node learning rate to a product of the network node learning rate and a first attenuation index
- the M i knowledge concepts The knowledge concept learning rate is respectively updated to the product of the knowledge concept learning rate of the M i knowledge concepts and the second attenuation index
- the embedded vector representation of the network node d i is retained, and the M i knowledge concepts are retained.
- the embedded vector representation is unchanged, wherein the first attenuation index represents a degree of attenuation of the network node learning rate, and the second attenuation index represents each knowledge concept learning rate of the M i knowledge concept learning rates The degree of attenuation.
- first attenuation index and the second attenuation index may be the same or different, which is not limited in the present application.
- the embedded vector representation of the network node d i is retained unchanged, i.e., the embedded vector representation of the network node d i is not replaced with the updated embedded vector representation of the network node d i .
- Retaining the embedded vector M i represents a concept of knowledge unchanged, i.e. does not embed the vector M represents the i-th concept knowledge embedded vector is replaced after the number M i represents updated knowledge concept.
- the embodiment of the present application uses a technique of dynamically adjusting the gradient descent learning rate. Before each gradient descent, the log likelihood of the current model is calculated, then the gradient descent is attempted, and the log likelihood of the model after iteration is again calculated.
- the embodiment of the present application adopts an embedded representation after the gradient is dropped. If the log likelihood decreases, the current learning rate is too large. At this time, the learning rate of each network node and indication concept is multiplied by an attenuation index, and the update of the embedded representation by the current iteration is discarded.
- the iterative process of the sampling phase performs t m times to obtain the topic hidden variables z and y of the model, and the model parameters ⁇ , ⁇ k , ⁇ r , ⁇ k , ⁇ r , and an embedded vector representation of each network node and knowledge concept after the update.
- S130 Determine, according to the heterogeneous Bayesian model, an effective association between the first network node and the first knowledge concept.
- the first network node is one of all the network nodes in the D weak supervisory information, that is, the first network node belongs to the D weak supervisory information
- the first knowledge concept is One of all the knowledge concepts in the D weak supervisory information, that is, the first knowledge concept belongs to the D weak supervisory information.
- the first network node may be a given network node d
- the first knowledge concept may be a given knowledge concept w
- the first network node and the first knowledge concept are both obtained in the above heterogeneous
- the probability of becoming a neighbor in the Bayesian model ie, the social knowledge map generated in the embodiment of the present application
- the probability that d and w become neighbors in the social knowledge map is defined as y d is generated by the topic of z w , and then the probability of the two embedded vector representations is separately generated.
- the summation of z w can be used to obtain the probability that the two match in the social knowledge map.
- the probability that a given network node a and a given knowledge concept 1 become neighbors is the conditional probability of the network node a and the knowledge concept 1.
- the value of the conditional probability is relatively large, it indicates that the knowledge concept 1 is the core knowledge concept of the network node a.
- the knowledge concept is expressed as the edge knowledge concept of the network node a. Therefore, the embodiment of the present application can conveniently distinguish the core knowledge concept and the edge knowledge concept related to each network node in the social network.
- a method for processing data in a social network by determining a series of weak supervisory information indicating interaction information between a network node and a knowledge concept, and generating a heterogeneous Bayesian model according to the weak supervisory information,
- the heterogeneous Bayesian model represents a connection model between a social network and a knowledge map, and then based on the heterogeneous Bayesian model, an effective relationship between a given network node and a given knowledge concept can be determined.
- the heterogeneous Bayesian model generated by the embodiment of the present application has accuracy and universality.
- the method of data processing in the social network of the embodiment of the present application is compared with other baseline algorithms for multiple comparison experiments, and all of them are effectively verified.
- the following is a brief introduction to the baseline algorithm in the experiment.
- a.AM-base is a research interest extraction method before the Aminer system.
- the method adopts the Microsoft Structured Query Language (SQL) server (Server) keyword extraction application based on natural language processing techniques such as word stemming, filtering vocabulary, POS tagging, and word boundary detection.
- SQL Structured Query Language
- API Application Programming Interface
- the CountKG method directly matches the N-gram in the weak supervisory information with the vocabulary in the knowledge map, extracts all the successfully matched N-grams as keywords, and sorts according to the word frequency.
- GenVector is a method of data processing proposed by the embodiment of the present application.
- GenVector includes heterogeneous Bayesian models, learning of embedded representations of network nodes and knowledge concepts, and unsupervised generation methods of weakly supervised information.
- GenVector-E is an algorithm for embedding the portion representing the update based on the data processing method of the embodiment of the present application.
- e.GenVector-M is based on the data processing method of the embodiment of the present application, and the heterogeneous Bayesian model is removed, and is randomly generated directly from the weak supervisory information.
- f.GenVector-R is based on the data processing method of the embodiment of the present application, and the heterogeneous Bayesian model is removed, directly according to the following formula:
- NTN Neutral tensor network
- the first experiment was a matching experiment on a personal homepage.
- the experiment uses the researcher's personal homepage to determine whether a knowledge concept is the researcher's main research interest, that is, using Aminer's homepage extraction algorithm to capture a part of the researcher's homepage from the Internet, and then through the knowledge concept collection with Wikipedia. The matching is obtained, and the researcher's research interest is obtained.
- the result of the experiment is a data set containing 1874 researchers, and these knowledge concepts are used as correct answers to compare the accuracy of the data processing methods in different social networks.
- the accuracy of the GenVector is improved by 5.58% with respect to the AM-base, which indicates that the data processing method in the social network of the embodiment of the present application can update the original algorithm instead of the AMiner.
- GenVector's accuracy is improved relative to GenVector-E, indicating that updates to the embedded representation can yield better results.
- the accuracy of GenVector is 18.88% higher than that of GenVector-M.
- the accuracy of GenVector is 6.98% higher than that of GenVector-R, indicating that the heterogeneous Bayesian embedding model plays an important role in the algorithm.
- the accuracy rate of GenVector is 43.05% higher than that of CountKB, which means that the simple comparison of statistics through the knowledge base does not work, and further understanding of semantics is needed.
- the accuracy rate of GenVector is 4.70% higher than that of Author-topic, indicating that the introduction of embedded representation can enhance the expressive ability of the model.
- GenVector's accuracy rate is 18.28% higher than NTN, indicating that using only the embedded representation as a classifier input does not yield good results.
- the second experiment was a matching experiment on the LinkedIn personal page.
- the experiment determines whether a knowledge concept is the researcher's by the researcher's personal page on LinkedIn.
- the main research interest is to use the heterogeneous network matching algorithm COSNET to connect the Aminer social network with the LinkedIn network, select the node with the highest connection probability, and then map the researcher on the AMiner to the LinkedIn account.
- the author's personal page is then crawled from LinkedIn and a column is drawn from it as a standard answer to research interests. Set the probability threshold of COSNET higher to ensure the reliability of the heterogeneous network connection, and then select all the researchers who are not empty. After the above steps, a data set containing 222 researchers was obtained.
- GenVector is improved compared to GenVector-E, GenVector-M, and GenVector-R, and it defeats the methods of modeling from different angles such as AM-base, Author-topic, and NTN.
- the embodiment of the present application Compared with the method of data processing in the previous social network, the embodiment of the present application has the characteristics of high speed and high accuracy.
- the embodiment of the present application optimizes the data processing method in the social network, so that the calculation speed is increased by 60 times, and the single iteration time is accelerated from the original 3 hours to 3 minutes.
- a method for processing data in a social network by determining a series of weak supervisory information indicating interaction information between a network node and a knowledge concept, and generating a heterogeneous Bayesian model according to the weak supervisory information,
- the heterogeneous Bayesian model represents a connection model between a social network and a knowledge map, and then based on the heterogeneous Bayesian model, an effective relationship between a given network node and a given knowledge concept can be determined.
- FIG. 4 is a schematic block diagram of an apparatus 300 for data processing in a social network according to an embodiment of the present application.
- the device 300 includes:
- the determining unit 310 is configured to determine D weak supervisory information, where the i-th weak supervisory information in the D weak supervisory information includes the network node d i in the social network and the M i related to the network node d i Knowledge concept, wherein the network node d i is represented as an embedded vector representation, the M i knowledge concepts are respectively represented as embedded vector representations, D and M i are respectively positive integers greater than 1, and i is a value range of 1 A positive integer to D.
- the generating unit 320 is configured to generate a heterogeneous Bayesian model according to the D weak monitoring information, where the heterogeneous Bayesian model includes T topics and D documents, and the jth of the T topics
- the topic includes parameters ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k
- the i-th document in the D documents includes the network node d i and the M i knowledge concepts
- the ith documents further comprising a number M i M i-one correspondence concept knowledge of first topic hidden variable d i with the network node and the network node and the second one to one topic d i hidden variables a A corresponding parameter ⁇ .
- ⁇ j r represents the mean of the Gaussian distribution of the network node d i corresponding to the jth topic
- ⁇ j r represents the accuracy of the Gaussian distribution of the network node d i corresponding to the jth topic
- ⁇ jm k represents the mean of the Gaussian distribution of the mth knowledge concept in the M i knowledge concepts corresponding to the jth topic
- ⁇ jm k represents the M i knowledge corresponding to the jth topic accuracy of m Gaussian distribution concept in the concept of knowledge
- ⁇ d i indicates the topic of the network node number distribution
- T is a positive integer greater than 1
- j is a positive integer ranging from 1 to T
- m is A positive integer ranging from 1 to M i .
- the determining unit 310 is further configured to determine, according to the heterogeneous Bayesian model, an effective association between the first network node and the first knowledge concept.
- the first network node is one of all the network nodes in the D weak supervisory information, that is, the first network node belongs to the D weak supervisory information
- the first knowledge concept is One of all the knowledge concepts in the D weak supervisory information, that is, the first knowledge concept belongs to the D weak supervisory information.
- a method for processing data in a social network by determining a series of weak supervisory information indicating interaction information between a network node and a knowledge concept, and generating a heterogeneous Bayesian model according to the weak supervisory information,
- the heterogeneous Bayesian model represents a connection model between a social network and a knowledge map, and then based on the heterogeneous Bayesian model, an effective relationship between a given network node and a given knowledge concept can be determined.
- the probability that the first network node and the first knowledge concept become neighbors can be calculated by:
- the probability that a given network node a and a given knowledge concept 1 become neighbors is the conditional probability of the network node a and the knowledge concept 1.
- the value of the conditional probability is relatively large, it indicates that the knowledge concept 1 is the core knowledge concept of the network node a.
- the knowledge concept is expressed as the edge knowledge concept of the network node a. Therefore, the embodiment of the present application can conveniently distinguish the core knowledge concept and the edge knowledge concept related to each network node in the social network.
- the generating unit 320 is specifically configured to:
- An initial value of the M i first topic hidden variables and an initial value of the second topic hidden variable are randomly determined.
- conditional probability of the network node topic d i represents the second topic d i hidden variable node in a given network, among other topics than the second topic hidden variables d i hidden variables of the network node, the the second topic hidden variables, in the case where the conditional probability for each concept knowledge embedded network node vector and the vector d i is embedded.
- the document -d represents a document other than the document d
- y -d represents a second topic hidden variable of the document -d
- the function G'( ⁇ ) is defined as:
- the initial value of M i the first topic of hidden variables, conditional probability of the i-th weak supervision information M i and a knowledge of the topics in the concept of the subject M i th first hidden variable t a b -sampling update, wherein a conditional probability of a topic of each of the conditional probabilities of the topics of the M i knowledge concepts indicates that the first topic hidden variable of each of the knowledge concepts is given in addition to each Other topic hidden variables other than the first topic hidden variable of the knowledge concept, the M i first topic hidden variables, the embedded vector of each of the knowledge concepts, and the embedded vector of the network node d i Conditional probability in the case.
- the updated M i first topic hidden variables and the updated The second topic hidden variable and the parameters ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k are sample updated to generate the heterogeneous Bayesian model.
- the generating unit 320 performs t b sampling updates on the M i first topic hidden variables and the second topic hidden variables respectively, respectively Updating the updated M i first topic hidden variables, the updated second topic hidden variables, and the parameters ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k include:
- the topic hidden variable is t p -sampling update.
- conditional probability of the i-th topic weak supervision information and the network node D i the topic of the second hidden variables for the updated update sampling times t p .
- conditional probability of the topic of the network node d i and the conditional probability of the topic of the M i knowledge concepts are respectively the above formula (1) and formula (2).
- the topic distribution parameter ⁇ d is updated as follows:
- the generating unit 320 is specifically configured to: perform N times sampling update on the parameters ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k , respectively, where said N times are updated in each update performed on the updated subject M i th first hidden variables updated sampling times t p, of the updated hidden topic M i th first variable time t p Sampling update and updating ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k respectively, and obtaining N times updated ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k , respectively, and obtaining N times updated ⁇ , ⁇ j r , ⁇ j r , ⁇ jm k , ⁇ jm k , ⁇ jm k ,
- the ⁇ jm k after the N updates is cumulatively averaged, and the average value obtained by accumulating ⁇ jm k is taken as the ⁇ jm k parameter of the updated heterogeneous Bayesian model.
- the generating unit 320 is further configured to:
- the log likelihood of the embedded vector representation of the network node d i that is not updated and the embedded vector representation of the M i knowledge concepts that are not updated is referred to as the initial log likelihood.
- the generating unit 320 determines, according to the initial log likelihood, the updated embedded vector representation of the network node d i , including:
- the gradient of the embedded representation of the network node is:
- the network node learning rate represents a learning rate of a gradient of the embedded vector of the network node d i .
- the generating unit respectively determines, according to the initial log likelihood, an updated embedded vector representation of the M i knowledge concepts, including:
- the concept of the gradient embedded knowledge M i-th concept of knowledge representation, knowledge of M i-th concept learning concepts and knowledge of the embedded vector M i-th concept of knowledge representation, respectively, of the determining the updated M An embedded vector representation of i knowledge concepts, wherein the knowledge concept learning rate of each of the M i knowledge concepts represents a learning rate of a gradient of the embedded vector of each knowledge concept.
- the generating unit 320 is further configured to:
- the embedding vector updated the network node, and d i represents the updated embedded vector M i th concept of knowledge representation, determining a given number M i of the first and second latent variables in hidden variables
- the update of the case is log likelihood.
- the log likelihood of the embedded vector representation after the update of the network node d i and the update of the M i knowledge concepts is referred to as the update log likelihood.
- the update log likelihood is equal to or greater than the initial log-likelihood, will be embedded in the network node vector d i is represented as replacing the updated embedded vector d i represents a network node, the embedded vector M i represents a concept of knowledge representation is replaced after updating the embedded vector M i th concept of knowledge;
- update the network node learning rate to a product of the network node learning rate and a first attenuation index
- the M i knowledge concepts The knowledge concept learning rate is respectively updated to the product of the knowledge concept learning rate of the M i knowledge concepts and the second attenuation index, and the embedded vector representation of the network node d i is retained unchanged, and the M i knowledge concepts are retained.
- the embedded vector representation is unchanged, wherein the first attenuation index represents a degree of attenuation of the network node learning rate, and the second attenuation index represents each knowledge concept learning rate of the M i knowledge concept learning rates The degree of attenuation.
- first attenuation index and the second attenuation index may be the same or different, which is not limited in the present application.
- the determining unit 310 is specifically configured to:
- Determining D initial weak supervision information wherein the i-th initial weak supervision information in the D initial weak supervision information includes the network node d i and all knowledge concepts related to the network node di;
- the unsupervised generation rate r is a preset ratio of the number of determined knowledge concepts M i to the number of all knowledge concepts, and r ranges from 0 to 1.
- the concept of knowledge The score can be expressed as:
- n is The number of occurrences in all knowledge concepts D d .
- the device further includes:
- a conversion unit configured to convert the corpus according to a knowledge concept set, a length of a corpus, and a maximum N-gram length of the corpus, the knowledge concept set including all knowledge concepts in the D weak supervision information ;
- a first acquiring unit configured to input the converted corpus to the first skipgram model, and obtain an embedded vector representation of each knowledge concept in the knowledge concept set.
- the determining unit 310 is further configured to:
- the weighted probability of the network node can be expressed as:
- the apparatus further includes a second obtaining unit, configured to input at least one random walk sequence of each of the network nodes to a second skipgram model to obtain an embedded vector representation of each of the network nodes.
- the first skipgram model and the second skipgram model may be the same or different, and the application does not limit this.
- apparatus 400 can include a processor 410, a memory 420, and a bus system 430.
- the memory 420 can be used to store code and the like executed by the processor 410.
- bus system 430 which in addition to the data bus includes a power bus, a control bus, and a status signal bus.
- bus system 430 in addition to the data bus includes a power bus, a control bus, and a status signal bus.
- bus system 430 various buses are labeled as bus system 430 in the figure.
- each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 410 or an instruction in a form of software.
- the steps of the method disclosed in the embodiments of the present application may be directly implemented as a hardware processor, or may be performed by a combination of hardware and software modules in the processor.
- the software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like.
- the storage medium is located in the memory 420, and the processor 410 reads the information in the memory 420 and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.
- the processor may be an integrated circuit chip with signal processing capabilities.
- each step of the foregoing method embodiment may be completed by an integrated logic circuit of hardware in a processor or an instruction in a form of software.
- the processor may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a Field Programmable Gate Array (FPGA), or the like. Programming logic devices, discrete gates or transistor logic devices, discrete hardware components.
- the methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed.
- the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
- the steps of the method disclosed in the embodiments of the present application may be directly implemented by the hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor.
- the software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like.
- the storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the above method.
- the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory.
- the non-volatile memory may be a read-only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (Erasable PROM, EPROM), or an electric Erase programmable read only memory (EEPROM) or flash memory.
- the volatile memory can be a Random Access Memory (RAM) that acts as an external cache.
- RAM Random Access Memory
- many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (Synchronous DRAM).
- SDRAM Double Data Rate SDRAM
- DDR SDRAM Double Data Rate SDRAM
- ESDRAM Enhanced Synchronous Dynamic Random Access Memory
- SLDRAM Synchronous Connection Dynamic Random Access Memory
- DR RAM direct memory bus random access memory
- the apparatus 300 shown in FIG. 4 or the apparatus 400 shown in FIG. 5 can implement the respective processes corresponding to the foregoing method embodiments shown in FIG. 2 and FIG. 3. Specifically, the apparatus 300 or the apparatus 400 can refer to FIG. 2 and FIG. The description in 3, in order to avoid repetition, will not be repeated here.
- the size of the sequence numbers of the foregoing processes does not mean the order of execution sequence, and the order of execution of each process should be determined by its function and internal logic, and should not be applied to the embodiment of the present application.
- the implementation process constitutes any limitation.
- the disclosed systems, devices, and methods may be implemented in other manners.
- the device embodiments described above are merely illustrative.
- the division of the unit is only a logical function division.
- there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
- the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
- the functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the present application, which is essential to the prior art or part of the technical solution, may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present application.
- the foregoing storage medium includes: a U disk, a mobile hard disk, a read only memory ROM, a random access memory RAM, a magnetic disk, or an optical disk, and the like, which can store program codes.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Business, Economics & Management (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
一种社交网络中数据处理的方法和装置,该方法包括:确定D个弱监督信息(S110),所述D个弱监督信息中的第i个弱监督信息包括社交网络中的网络节点d i和与所述网络节点d i相关的M i个知识概念,所述弱监督信息表示网络节点和知识概念的交互信息;根据所述D个弱监督信息,生成异构贝叶斯模型(S120),所述异构贝叶斯模型表示社交网络和知识图谱之间的连接模型;根据所述异构贝叶斯模型,确定第一网络节点和第一知识概念的有效联系(S130)。可以方便地区分与社交网络中每个网络节点相关的核心知识概念和边缘知识概念。
Description
本申请要求于2016年05月11日提交中国专利局、申请号为201610311864.4、发明名称为“社交网络中数据的处理方法和装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请本申请涉及计算机领域,更具体的,涉及计算机领域中社交网络中数据处理的方法和装置。
社交网络可以是拥有巨大用户群体的在线社交平台,比如脸书(Facebook),推特(Twitter),新浪微博和微信等,也可以是不以在线平台为依托的社交网络,比如学术社交网络和线下人际关系网络。
社交网络数据挖掘的一个重要研究课题就是如何理解海量用户的行为,挖掘海量用户的兴趣和关注点。比如,在在线社交网络中,我们希望挖掘每个用户的兴趣和关注点,从而更精准地投放广告和进行个性化用户推荐。就学术社交网络而言,每一位研究者都是这个学术社交网络中的一个节点,他们之间存在的学术合作关系组成了一张庞大的学术社交网络,我们希望挖掘每个研究者的研究兴趣,从而能够更有效地帮助科研人员找到某一个研究方向的其他研究人员,进行学术合作推荐,增强以知识为基础的学术搜索功能。
此外,群体智能的兴起诞生了一些大规模知识图谱,这些知识图谱上有丰富的语料,大规模的知识概念,以及知识概念之间的关系。对于大规模知识图谱的应用将有助于人们对海量的互联网数据进行语义理解。
但是目前大规模社交网络和大规模知识图谱之间普遍缺乏联系,不能方便地通过社交网络使用知识图谱。
发明内容
本申请实施例提供了社交网络中数据处理的方法和装置,能够在社交网络和知识图谱之间建立有效的联系。
第一方面,本申请实施例提供了一种社交网络中数据处理的方法,该方法包括:
确定D个弱监督信息,所述D个弱监督信息中的第i个弱监督信息包括社交网络中的网络节点di和与所述网络节点di相关的Mi个知识概念,其中,所述网络节点di表示为嵌入式向量,所述Mi个知识概念分别表示为嵌入式向量,D和Mi分别为大于1的正整数,i为取值范围为1至D的正整数;
根据所述D个弱监督信息,生成异构贝叶斯模型,所述异构贝叶斯模型包括T个话题和D个文档,所述T个话题中的第j个话题包括参数μj
r、λj
r、μjm
k、λjm
k,所述D个文档中的第i个文档包括所述网络节点di和所述Mi个知识概念,所述第i个文档还包括与所述Mi个知识概念一一对应的Mi个第一话题隐变量、与所述网络节点di一一对应的第
二话题隐变量和参数θ,所述网络节点di对应的第二话题隐变量可以为一个。
其中,μj
r表示所述第j个话题对应的所述网络节点di的高斯分布的均值,λj
r表示所述第j个话题对应的所述网络节点di的高斯分布的精度,μjm
k表示所述第j个话题对应的所述Mi个知识概念中的第m个知识概念的高斯分布的均值,λjm
k表示所述第j个话题对应的所述Mi个知识概念中的m个知识概念的高斯分布的精度,θ表示所述网络节点di的话题多项分布,T为大于1的正整数,j为取值范围为1至T的正整数,m为取值范围为1至Mi的正整数;
根据所述异构贝叶斯模型,确定第一网络节点与第一知识概念连接之间的有效联系,其中,所述第一网络节点为所述D个弱监督信息中的所有网络节点之中的一个,所述第一知识概念为所述D个弱监督信息中的所有知识概念之中的一个。
本申请实施例的社交网络中数据处理的方法,通过确定一系列弱监督信息,该弱监督信息表示网络节点和知识概念的交互信息,并根据该弱监督信息,生成异构贝叶斯模型,该异构贝叶斯模型表示社交网络和知识图谱之间的连接模型,然后根据该异构贝叶斯模型,能够确定给定的网络节点和给定的知识概念之间的有效联系。
可选的,本申请实施例中,第一网络节点与第一知识概念成为邻居的概率可以通过下式计算:
本申请实施例中,给定的网络节点a和给定的知识概念1的成为邻居的概率即为该网络节点a和知识概念1的条件概率。当该条件概率的值比较大时,表示该知识概念1为该网络节点a的核心知识概念。反之,当该条件概率的值比较小时,表示该知识概念为该网络节点a的边缘知识概念。因此,本申请实施例可以方便地区分与社交网络中每个网络节点相关的核心知识概念和边缘知识概念。
可选的,本申请实施例中,所述根据所述D个弱监督信息,生成所述异构贝叶斯模型,包括:
随机确定所述Mi个第一话题隐变量的初始值和所述第二话题隐变量的初始值,这里,第二话题隐变量的数量与网络节点的数量相同。
根据所述第二话题隐变量的初始值、所述第i个弱监督信息和所述网络节点di的话题的条件概率,对所述第二话题隐变量进行tb次抽样更新,其中,所述网络节点di的话题的条件概率表示所述网络节点di的第二话题隐变量在给定除所述网络节点di的第二话题隐变量之外的其他话题隐变量、所述第二话题隐变量、所述每个知识概念的嵌入式向量和所述网络节点di的嵌入式向量的情况下的条件概率。
对于文档d,在给定文档-d的第二话题隐变量y-d和第一话题隐变量z、网络节点的嵌入式表示和知识概念的嵌入式表示的情况下,网络节点的话题yd=t的条件概率可以写成:
其中,n是所有y=t对应的f的数目。假设x是所有y=t的对应的嵌入式向量f的第e维组成的向量。n'=n-nd,nd是文档中y=t对应的f的数目。
根据所述Mi个第一话题隐变量的初始值、所述第i个弱监督信息和所述Mi个知识概念的话题的条件概率,对所述Mi个第一话题隐变量进行tb次抽样更新,其中,所述Mi个知识概念的话题的条件概率中的每个知识概念的话题的条件概率表示所述每个知识概念的第一话题隐变量在给定除所述每个知识概念的第一话题隐变量之外的其他话题隐变量、所述Mi个第一话题隐变量、所述每个知识概念的嵌入式向量和所述网络节点di的嵌入式向量的情况下的条件概率。
本申请实施例中,在给定话题隐变量z-dm、y、网络节点的嵌入式表示和知识概念的嵌入式表示的情况下,文档d中的知识概念m的话题zdm=t的条件概率可以写成:
其中,z-dm表示除了文档d中的知识概念m之外的第一话题隐变量。
在对所述Mi个第一话题隐变量和所述第二话题隐变量分别进行tb次抽样更新之后,分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行抽样更新,以生成所述异构贝叶斯模型。
可选的,本申请实施例中,所述在对所述Mi个第一话题隐变量和所述第二话题隐变量分别进行tb次抽样更新之后,分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行更新,包括:
根据所述更新后的Mi个第一话题隐变量、所述第i个弱监督信息和所述所述Mi个知识概念的话题的条件概率,对所述更新后的Mi个第一话题隐变量进行tp次抽样更新;
根据所述更新后的第二话题隐变量、所述第i个弱监督信息和所述网络节点di的话题的条件概率,对所述更新后的第二话题隐变量进行tp次抽样更新。
这里,所述网络节点di的话题的条件概率和所述Mi个知识概念的话题的条件概率分别为上述公式(1)和公式(2)。
在对所述更新后的Mi个第一话题隐变量和所述更新后的第二话题隐变量分别进行tp次抽样更新之后,分别对θ、μj
r、λj
r、μjm
k、λjm
k进行更新。
本申请实施例中,对于文档d,话题分布参数θd更新如下:
可选的,本申请实施例中,所述分别对θ、μj
r、λj
r、μjm
k、λjm
k进行更新包括:分别对所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行N次抽样更新,其中,所述N次更新中的每次更新均执行对所述更新后的Mi个第一话题隐变量进行tp次抽样更新,对所述更新后的Mi个第一话题隐变量进行tp次抽样更新和分别对θ、μj
r、λj
r、μjm
k、λjm
k进行更新,得到N次更新后的θ、μj
r、λj
r、μjm
k、λjm
k;
对所述N次更新后的θ累加求平均值,将θ累加求得的平均值作为更新后的所述异构贝叶斯模型的θ参数;
对所述N次更新后的μj
r累加求平均值,将μj
r累加求得的平均值作为更新后的所述异构贝叶斯模型的μj
r参数;
对所述N次更新后的λj
r累加求平均值,将λj
r累加求得的平均值作为更新后的所述异构贝叶斯模型的λj
r参数;
对所述N次更新后的μjm
k累加求平均值,将μjm
k累加求得的平均值作为更新后的所述异构贝叶斯模型的μjm
k参数;
对所述N次更新后的λjm
k累加求平均值,将λjm
k累加求得的平均值作为更新后的所述异构贝叶斯模型的λjm
k参数。
可选的,本申请实施例中,所述分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行更新之后,还包括:
根据所述网络节点di的嵌入式向量和所述Mi个知识概念的嵌入式向量,确定在给定所述Mi个第一隐变量和所述第二隐变量的情况下的初始对数似然。
给定第一话题隐变量和第二话题隐变量情况下的嵌入表示的对数似然的计算公式,如下:
这里,将没有更新的网络节点di的嵌入式向量和没有更新的Mi个知识概念的嵌入式向量的对数似然称为初始对数似然。
根据所述初始对数似然,确定所述网络节点di更新后的嵌入式向量;
根据所述初始对数似然,分别确定所述Mi个知识概念更新后的嵌入式向量。
可选的,本申请实施例中,所述根据所述初始对数似然,确定所述网络节点di更新后的嵌入式向量,包括:
根据所述初始对数似然确定所述网络节点di的嵌入式向量的梯度;
具体的,网络节点的嵌入表示的梯度为:
根据所述网络节点di的嵌入式向量的梯度、网络节点学习率和所述网络节点di的嵌入式向量,确定所述网络节点di更新后的嵌入式向量,其中,所述网络节点学习率表示所述所述网络节点di的嵌入式向量的梯度下降的学习率。
可选的,本申请实施例中,所述根据所述初始对数似然,分别确定所述Mi个知识概念更新后的嵌入式向量,包括:
根据所述初始对数似然确定所述Mi个知识概念的嵌入式向量的梯度;
具体的,知识概念的嵌入表示的梯度为:
根据所述Mi个知识概念的知识概念的嵌入表示的梯度、Mi个知识概念的知识概念学习率和所述Mi个知识概念的嵌入式向量,分别确定所述更新后的所述Mi个知识概念的嵌入式向量,其中,所述Mi个知识概念中的每个知识概念的知识概念学习率表示所述每个知识概念的嵌入式向量的梯度下降的学习率。
可选的,本申请实施例中,所述方法还包括:
根据所述网络节点di的更新后的嵌入式向量和所述Mi个知识概念的更新后的嵌入式向量,确定在给定所述Mi个第一隐变量和所述第二隐变量的情况下的的更新对数似然。
这里,将网络节点di的更新之后的嵌入式向量和Mi个知识概念的更新之后的嵌入式向量的对数似然称为更新对数似然。
若所述更新对数似然大于或等于所述初始对数似然,则将所述网络节点di的嵌入式向量替换为所述网络节点di的更新后的嵌入式向量,将所述Mi个知识概念的嵌入式向量替换为所述Mi个知识概念的更新后的嵌入式向量;
若所述更新对数似然小于所述初始对数似然,则将所述网络节点学习率更新为所述网络节点学习率与第一衰减指数的乘积,将所述Mi个知识概念的知识概念学习率分别更新为所述Mi个知识概念的知识概念学习率与第二衰减指数的乘积,并且保留所述网络节点di的嵌入式向量不变,保留所述Mi个知识概念的嵌入式向量不变,其中,所述第一衰减指数表示所述网络节点学习率的衰减程度,所述第二衰减指数表示所述Mi个知识概念学习率中的每个知识概念学习率的衰减程度。
这里,第一衰减指数与第二衰减指数可以相同,也可以不同,本申请对此不作限定。
可选的,本申请实施例中,所述确定D个弱监督信息,包括:
确定D个初始弱监督信息,所述D个初始弱监督信息中的第i个初始弱监督息包括所述网络节点di和与所述网络节点di相关的所有知识概念;
根据所述所有知识概念中每个知识概念的分值和无监督生成率r,确定所述Mi个知识概念,其中,所述Mi个知识概念中的每一个知识概念的分值大于所述所有知识概念中
除所述Mi个知识概念之外的每一个知识概念的分值,所述每个知识概念的分值表示所述每个知识概念处于所述所有知识概念的中心位置的程度,所述无监督生成率r为确定的知识概念的数量Mi与所述所有知识概念的数量的预设比值,r的取值范围为0至1。
可选的,本申请实施例中,在所述确定D个弱监督信息之前,所述方法还包括:
根据知识概念集合、语料的长度和所述语料的最大的N-gram长度,对所述语料进行转换,所述知识概念集合包括所述D个弱监督信息中的所有知识概念;
将转换后的所述语料输入至第一skipgram模型,获取所述知识概念集合中每个知识概念的嵌入式向量。
可选的,本申请实施例中,在所述确定D个弱监督信息之前,所述方法还包括:
确定所述社交网络中的每个网络节点的加权概率,其中,所述每个网络节点的加权概率表示所述每个网络节点访问下一个网络节点的概率,所述社交网络中所有网络节点包括所述D个弱监督信息中的所有网络节点;
具体的,网络节点的加权概率可以表示为:
根据所述每个网络节点的加权概率,确定所述每个网络节点的至少一个随机游走序列;
将所述每个网络节点的至少一个随机游走序列输入至第二skipgram模型,获取所述每个网络节点的嵌入式向量。
本申请实施例中,该第一skipgram模型和该第二skipgram模型可以相同,也可以不同,本申请本申请对此不作限定。
第二方面,本申请实施例提供了一种社交网络中数据处理的装置,用于执行上述第一方面或第一方面的任意可能的实现方式中的方法,具体的,该装置包括用于执行上述第一方面或第一方面任意可能的实现方式中的方法的模块。
第三方面,本申请实施例提供了一种社交网络中数据处理的装置,该装置包括:存储器、处理器、收发器和总线***。其中,该存储器和该处理器通过该总线***相连,该存储器用于存储指令,该处理器用于执行该存储器存储的指令,并且当该处理器执行该存储器存储的指令时,该执行使得该处理器执行第一方面或第一方面的任意可能的实现方式中的方法。
第四方面,本申请实施例提供了一种计算机可读介质,用于存储计算机程序,该计算机程序包括用于执行第一方面或第一方面的任意可能的实现方式中的方法的指令。
图1是本申请本申请实施例的社交网络中数据处理的方法的应用场景图。
图2是本申请本申请一个实施例的社交网络中数据处理的方法的示意性流程图。
图3是本申请本申请一个实施例的社交网络中数据处理的方法中生成的异构贝叶斯模型的示意性框图。
图4是本申请本申请一个实施例的社交网络中数据处理的装置的示意性框图。
图5是本申请本申请另一个实施例的社交网络中数据处理的装置的示意性框图。
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述。
图1示出了本申请实施例的社交网络中数据处理的方法的应用场景的示意性框图。可以理解,大规模的社交网络中可以具有很多个网络节点,大规模的语料中可以具有很多个知识概念或词条,图1仅仅示意性的示出了社交网络中的部分网络节点和语料中的部分知识概念或词条。
具体的,图1中的社交网络可以包括多个网络节点,该网络节点例如可以为图中的a、b、c、d或e。例如,在学术社会网络***Aminer中,网络节点a、b、c、d和e可以分别为不同的专家学者、学术机构或者科研团队。在其他社交网络如新浪微博中,网络节点a、b、c、d和e可以分别为不同的博主。
对社交网络的理解,需要对社交网络的网络结构进行刻画。例如,可以根据Deepwalk将分布式向量嵌入的方法应用到社交网络上,为每一个社交网络的网络节点学习得到一个嵌入式向量,即可以完成对社交网络的网络结构进行刻画。本申请实施例中,可以称嵌入式向量为嵌入向量表示。
图1中语料可以为***(Wikipedia)中的所有知识概念或者百度百科内的所有词条,例如,语料中的1、2、3、4、5、6、7或8分别为***中的不同的知识概念,或者语料中的1、2、3、4、5、6、7或8分别为百度百科内的不同的词条,这些知识概念(或词条)例如可以为大数据、智能家居、机器学习、清华大学、光通信、长期演进(Long Term Evolution,LTE)等。
目前主要通过话题模型或嵌入模型对语料中的知识概念(或词条)的文本语义进行建模和分析。
话题模型例如可以为概率潜在语义分析模型(probabilistic Latent Semantic Indexing,pLSI)和潜在狄利克雷分布模型(Latent Dirichlet Allocation,LDA)。
LDA是一种贝叶斯生成模型,通过对每一个文档生成一个话题分布和对每一个话题生成一个词分布来最终生成每一个文档里的每一个词。话题模型的优点是能够通过话题隐变量对不同词的语义进行刻画,并用文档和词的分布进行约束。但是,话题模型只能输出离散的词,不能输出连续的语义表示。
例如,当话题为体育时,该话题对应的词可以为篮球、姚明、世界杯和***等等;当话题为水果时,该话题对应的词可以为苹果、梨和香蕉等等。因为一个文档中可以包括多个话题,一个话题还可以包括多个词,话题模型可以对每个文档生成多个话题分布,对每个话题生成多个词分布,最终生成每个文档中的每个词。
嵌入模型是一种分布式向量嵌入表示的学习方法,例如Word2Vec学习方法。Word2Vec学习方法通过优化给定语境中一个词出现的似然来学习每一个词的连续向量嵌入表示。每一个词在特征空间中的位置表征了这个词的语义。分布式向量嵌入学习的
优点是能够对每一个词得到一个在连续特征空间上的表示,并且优化目标可以刻画二阶共现关系。但是,嵌入模型难以理解每个词对应的隐含话题。
图1中社交网络中的网络节点与语料中的知识概念(或词条)连接的直线表示该网络节点与该知识概念(或词条)的有效联系,px表示该网络节点与该知识概念(或词条)的条件概率。也就是说,本申请实施例中,该有效联系可以为该网络节点与该知识概念(或词条)的条件概率。大规模的社交网络中的网络节点与语料中的知识概念(或词条)之间的条件概率也可以称为该网络节点与该知识概念(或词条)之间的链接概率,或者成为该网络节点与该知识概念成为邻居的概率。
如图1所示,一个网络节点可以与多个知识概念之间具有有效联系。例如,网络节点b分别与知识概念(或词条)2、知识概念(或词条)4和知识概念(或词条)5具有有效联系。同时,一个知识概念(或词条)也可以同时和多个网络节点之间具有有效联系。例如,知识概念(或词条)6分别与网络节点e和网络节点d具有有效联系。因此,大规模的社交网络和大规模的知识图谱之间可以具有复杂的联系。
但是,社交网络的结构信息和语料中的文本语义信息为两种异构的信息,现有技术中只能够针对其中的一种信息做处理,并没有将两种信息关联起来的处理方法。因此,现有技术中的大规模的社交网络和大规模的知识图谱之间缺乏普遍的联系,并不能方便的通过社交网络使用知识图谱,即现有技术不能够在社交网络和知识图谱之间建立上述有效的联系。
本申请实施例提供的一种社交网络中数据处理的方法,能够在社交网络和知识图谱之间建立有效的联系,进而可以有效地区分与社交网络中每个网络节点相关的核心知识概念和边缘知识概念。
可选的,本申请实施例可以在语料中无监督地学习每个知识概念的嵌入向量表示,嵌入向量表示能够对知识概念的文本语义进行刻画。
本申请实施例的确定每个知识概念的嵌入向量表示的方法可以应用于学术社会网络***AMiner中,这时语料C即为Wikipedia中的所有知识概念。此外,本申请实施例的确定每个知识概念的嵌入向量表示的方法也可以应用于其他社交网络如在新浪微博中,这时可将百度百科内的所有词条作为语料C。
这里,在语料中无监督地学习每个知识概念的嵌入向量表示时,可以根据知识概念集合、语料的长度和语料中的最大N-gram长度,对语料进行转换,所述知识概念集合包括所述D个弱监督信息中的所有知识概念。
具体的,对于给定语料C和知识概念集合Vk,需要根据该知识概念集合Vk对语料进行转换。在对语料进行转换时,需要确定语料的长度L和最大的N-gram长度。
首先需要确定N-gram的长度。具体的,N表示词的长度,gram表示按长度N切分原词得到的词段。N取最小值规则,如果原词的长度大于5,则N最小值为3,如果原词的长度等于5,则最小值为2,否则最小值为1。
然后按照对N从最大取值到2,分别对语料进行线性扫描,即当N取最大值时,将语料中的所有N-gram和知识概念集合Vk进行匹配,如果该N-gram在知识概念集合Vk中存在,则将语料中的N-gram替换为一个单词。然后N减小1,继续将语料中的所有N-gram和知识概念集合Vk进行匹配,如果该N-gram在知识概念集合Vk中存在,则将语料中的N-gram替换为一个单词。本申请实施例中,N-gram的枚举从最大的N开始,
直至N的取值为2。因此,如果一个知识概念是另一个知识概念的字串,则只考虑长度最长的字符串。
本申请实施例中,在对语料进行转换之后,将转换之后的语料输入至skipgram模型,可以获取该知识概念集合中的每个知识概念的嵌入向量表示。
具体的,skipgram模型的目标函数是给定当前考察的单词,计算所有与当前词在上下文中距离小于阈值的条件概率。
在计算条件概率的归一化项时,本申请实施例采用了用hierarchical softmax来加速训练的速度,即根据词频生成采用哈夫曼(Huffmann)贪心算法生成一棵哈夫曼树,在哈夫曼树生成之后,条件概率表示为:
其中,w是要预测的词,wI是作为上下文的词,L(w)表示从哈夫曼树的根节点到w这个词的路径长度,n(w,j)表示从哈夫曼树的根节点到w这个词的路径上的第j个节点。ch(w)表示哈夫曼树中w节点的一个孩子,由于哈夫曼树是一棵二叉树,所以每个节点有0或2个孩子,ch(w)为非叶子节点的两个孩子中事先任意指定的某一个。是一个指标函数,如果变量为真那么返回1,否则返回-1。σ(·)是softmax函数。
当n(w,j+1)=ch(n(w,j))时,指标函数的值为1,上述条件概率可以表示为:
而当n(w,j+1)≠ch(n(w,j))时,指标函数的值为-1,上述条件概率可以表示为:
根据softmax函数的性质,上述两个概率加起来为1。所以hierarchical softmax即是将所有节点组织成一棵二叉树,在每一个非叶子节点存在一个分叉,该概率分成两半,最后到某一个节点的概率就等于路径所有分配的概率的乘积。容易看出,根据这个性质,无需再对概率进行归一化,因为所有叶子节点的概率加起来等于1。
这样,本申请实施例可以通过大量无标注数据得到知识概念的语义,从而可以帮助区分知识概念的不同话题,区分社交网络用户对应的主要知识概念和次要知识概念。
本申请实施例中,上述确定知识概念的嵌入向量表示的方法的时间复杂度从原来的线性级别降低为对数级别,可以适用于大规模语料。可选的,本申请实施例中,可以从大规模社交网络中无监督的学习每个网络节点的嵌入向量表示,该嵌入向量表示能够对社交网络的结构信息进行刻画。
本申请实施例中确定每个网络节点的嵌入向量表示的方法可以应用于学术社会网络***AMiner中,这时网络节点即为每一位专家学者。本申请实施例中确定每个网络节点的嵌入向量表示的方法还可以应用于在其他社交网络如新浪微博中,这时网络节点是每一个博主。
本申请实施例中,从大规模社交网络中无监督的学习每个网络节点的嵌入向量表示,首先可以确定所述社交网络中的每个网络节点的加权概率,其中,所述每个网络节点的加权概率表示所述每个网络节点访问下一个网络节点的概率,所述社交网络中所有网络节点包括所述D个弱监督信息中的所有网络节点。
具体的,本申请实施例中,可以采用DeepWalk算法学习社交网络中每个节点的嵌
入向量表示。DeepWalk算法是从每一个节点开始产生一定数量的随机游走,然后将随机游走得到的节点序列作为语料训练Skipgram模型。
DeepWalk算法是针对无权无向图提出来的,但是在发明实施例中,社交网络上的边应该是有权无向的。因此,DeepWalk算法并不能直接应用。
本申请实施例中在每次进行随机游走时,都使用一个加权概率来选择下一个访问的节点。形式化地,假定当前节点是v,那么我们在选择随机游走的下一个节点u时,采用如下的概率进行抽样:
其中,等号右边第一项表示该节点u必须是v的邻居,nu,v表示u和v边上的整数权值。
在确定所述社交网络中的每个网络节点的加权概率之后,根据所述每个网络节点的加权概率,确定所述每个网络节点的至少一个随机游走序列,将所述每个网络节点的至少一个随机游走序列输入至skipgram模型,能够获取所述每个网络节点的嵌入向量表示。
具体的,本申请实施例根据上述的概率进行抽样,可以生成带权无向图的随机游走序列。将该带权无向图的随机游走序列输入到skipgram模型,可以学习得到社交网络上每个节点的嵌入表示。在社交网络中无监督学习网络节点的嵌入表示能够有效地解决数据的稀疏性问题,并充分利用网络结构来帮助生成社交知识图谱。通过将每一个网络节点映射到一个连续的特征空间,我们可以从语义上有效的刻画每一个节点。
本申请实施例中,确定知识概念的嵌入向量表示的skipgram模型与确定网络节点的嵌入向量表示的skipgram模型可以相同,也可以不同,本申请实施例对此不作限定。
图2示出了本申请实施例的一种社交网络中数据处理的方法100的示意性流程图。该方法100包括:
S110,确定D个弱监督信息。
所述D个弱监督信息中的第i个弱监督信息包括社交网络中的网络节点di和与所述网络节点di相关的Mi个知识概念,其中,所述网络节点di表示为嵌入向量表示,所述Mi个知识概念分别表示为嵌入向量表示,D和Mi分别为大于1的正整数,i为取值范围为1至D的正整数。
本申请实施例中,需要生成的异构贝叶斯模型的输入是一系列弱监督信息,例如上述D个弱监督信息。该D个弱监督信息中的D个网络节点为实验可获得的大规模社交网络中的部分网络节点。因为弱监督信息包括一系列的网络节点和与每个网络节点交互过的所有知识概念,因此弱监督信息能够表示网络节点与文本的交互信息。这样,本申请实施例能够根据给定的弱监督信息进行异构贝叶斯模型的建立。
本申请实施例的确定弱监督信息的方法可以应用于学术社会网络***AMiner中,这时弱监督信息即为一个研究者的所有论文中包含的知识概念,即其发表过每篇论文的题目和摘要。本申请实施例的确定弱监督信息的方法还可以应用在其他社交网络如新浪微博中,此时弱监督信息即为每个博主发表过的所有微博。
可选的,本申请实施例中,为了避免由于使用全部弱监督信息而引入噪声,本算法提出了对弱监督信息的无监督生成方法,即本申请实施例能够过滤掉全部弱监督信息中
的部分数据,用过滤之后的数据来训练得到表现更好的模型。下面为本申请实施例的过滤数据的方法。
首先,本申请实施例需要确定D个初始弱监督信息,所述D个初始弱监督信息中的第i个初始弱监督息包括所述网络节点di和与所述网络节点di相关的所有知识概念。
然后,根据所述所有知识概念中每个知识概念的分值和无监督生成率r,确定所述Mi个知识概念,其中,所述Mi个知识概念中的每一个知识概念的分值大于所述所有知识概念中除所述Mi个知识概念之外的每一个知识概念的分值,所述每个知识概念的分值表示所述每个知识概念处于所述所有知识概念的中心位置的程度,所述无监督生成率r为确定的知识概念的数量Mi与所述所有知识概念的数量的预设比值,r的取值范围为0至1。
具体的,对于一个网络节点d的所有知识概念Dd,根据下面公式计算某一知识概念同其他所有知识概念的余弦相似度的加权和。
本申请实施例中,将上述加权和作sw作为该知识概念的分值。知识概念分值越高,说明该知识概念处于Dd中知识概念中的越靠中心的位置。上述计算过程中,把知识概念出现在Dd中的次数作为权重,使得出现次数越多,对其分值的贡献越大。在计算完分值之后,筛选分值最大的|Dd|×r个知识概念作为D′d。
S120,根据所述D个弱监督信息,生成异构贝叶斯模型。
所述异构贝叶斯模型包括T个话题和D个文档,所述T个话题中的第j个话题包括参数μj
r、λj
r、μjm
k、λjm
k,所述D个文档中的第i个文档包括所述网络节点di和所述Mi个知识概念,所述第i个文档还包括与所述Mi个知识概念一一对应的Mi个第一话题隐变量、与所述网络节点di对应的第二话题隐变量和参数θ。这里,上标r表示研究者(researcher),即多种网络节点类型中的其中一种,上标k表示关键字(keyword),即多种知识概念类型中的其中一种。
其中,上述μj
r表示所述第j个话题对应的所述网络节点di的高斯分布的均值,λj
r表示所述第j个话题对应的所述网络节点di的高斯分布的精度,μjm
k表示所述第j个话题对应的所述Mi个知识概念中的第m个知识概念的高斯分布的均值,λjm
k表示所述第j个话题对应的所述Mi个知识概念中的m个知识概念的高斯分布的精度,θ表示所述网络节点di的话题多项分布,T为大于1的正整数,j为取值范围为1至T的正整数,m为取值范围为1至Mi的正整数。
本申请实施例中的该异构贝叶斯模型能够在异构的嵌入表示之间通过类似LDA的话题隐变量建立联系。按照话题模型的传统,本申请实施例中,可以将一个弱监督信息条目称为一个文档,因此该异构贝叶斯模型中,每个文档可以包含社交网络节点以及与该社交网络节点交互过的所有知识概念。
可理解,本申请实施例的社交网络中数据处理的方法具有普适性。根据输入的社交网络及语料库不同,可以生成不同的知识图谱。
比如输入是学术社交网络中的学者,对应语料库是学者研究兴趣相关的一些知识概
念,交互信息是学者发表的论文,那么就可以生成学者的研究兴趣的知识图谱。如果输入是微博社交网络中影视娱乐方面的博主,语料库是影视娱乐相关的知识概念,交互信息是博主发表的帖子,那么生成的知识图谱就是微博博主最关注的影视娱乐相关的热点。
下文中,将第一话题隐变量称为话题隐变量z,将第二话题隐变量称为y。
图2是本申请实施例的异构贝叶斯模型的图的表示。这里,α是狄利克雷分布的超参数,fr是社交网络节点的嵌入表示,fk是知识概念的嵌入表示,τk是知识概念嵌入表示的normal Gamma先验的超参数,τr是社交网络节点嵌入表示的normal Gamma先验的超参数。
根据上述的异构贝叶斯模型,对于T个话题中的每一个话题t,对于知识概念的嵌入向量表示的每一个维度,从分布NG(τk)生成μk和λk,对于网络节点嵌入表示的每一维度,从分布NG(τr)生成μr和λr。本申请实施例中,嵌入向量表示的维度大概为200。这里,为了简便,省略了μk、λk、μr和λr的下标。
本申请实施例中,对于D个文档中的每个文档d,从Dir(α)分布生成话题多项分布θd。对于文档d中的每一个知识概念w,可以从多项分布θd生成知识概念的话题z,对于每一个知识概念w的每一个维度,从高斯分布N(μk
z,λk
z)生成嵌入向量表示fk。本申请实施例中,还可以从文档d中的所有知识概念的话题z中uniform生成网络节点的话题y,对于网络节点的嵌入向量表示的每一个维度,从高斯分布N(μr
y,λr
y)生成嵌入向量表示fk。
本申请实施例中的异构贝叶斯模型的联合概率分布可以写成如下形式:
p(θ,μr,λr,μk,λk,z,y,fr,fk;α,τr,τk)=p(θ;θ)p(μr,λr;τr)p(μk,λk;τk)
p(z|θ)p(fk|y,μk,λk)p(fr|y,μr,λr)p(y|z)
下面对上述等式右边的每一项进行单独展开。
θ的分布服从狄利克雷分布,超参数是α,其概率可以表示为:
其中,下面d表示文档,下标t表示话题。
高斯分布的参数μ和λ由normal Gamma分布生成,超参数是τr和τk。因此高斯分布参数的概率可以写成如下形式:
其中,下标t表示话题,e表示嵌入向量表示的某一个维度,normal Gamma的超参数τ被展开成四个标量参数{μ0,λ0,α0,β0}。注意,作为normal Gamma分布超参数的μ0,λ0,α0与模型中的高斯参数μk,λk,μr,λr以及狄利克雷超参数α含义不同。
从多项分布生成话题的概率如下:
其中,下标d表示文档,下标m表示知识概念。
从知识概念的话题生成网络节点的话题的概率是一个uniform分布,由于一个知识概念话题可能出现多次,所以该话题被生成的概率正比于出现的次数:
其中,下标d表示文档,下标m表示知识概念。
本申请实施例中,如果采用上述式子作为生成网络节点话题的概率,网络节点的话题必须在知识概念的话题中出现过,因为没有出现过的话题的概率是零。为了使得概率分布更加平滑,采用了拉普拉斯平滑的技巧,所以可以将生成网络节点话题的概率改写为下式:
其中拉普拉斯平滑系数l∈(0,1)。
其中,λk和μk均是在对应话题对应维度下的取值,为了简洁,这里省略了其下标。
其中,λr和μr均是在对应话题对应维度下的取值,为了简洁,这里省略了其下标。
这里,可以借鉴吉布斯采样的思想,对模型参数μr,λr,μk,λk,θ进行积分,得到变量关于模型超参数的联合概率分布。
对参数θ进行积分,
对参数μk,λk进行积分,
其中,函数G(·)定义为:
其中,n是所有y=t对应的f的数目。假设x是所有y=t的对应的嵌入向量表示f的第e维组成的向量,则:
αn=α0+n/2
κn=κ0+n
相似地,可以对参数μr,λr进行积分,
综上,通过对参数μr,λr,μk,λk,θ进行积分,可以得到对所有模型参数积分之后所有变量关于模型超参数的联合概率分布,如下:
其中,Er和Ek分别表示网络节点和知识概念嵌入表示的维度。
本申请实施例中,采用吉布斯采样对模型进行概率推理,为了进行吉布斯采样,需要推导每个变量在给定其他变量情况下的条件概率。
对于文档d,在给定文档-d的话题隐变量y-d、z、知识概念的嵌入向量表示和网络节点的嵌入向量表示的情况下,网络节点的话题yd=t的条件概率可以写成:
其中,n是所有y=t对应的f的数目。假设x是所有y=t的对应的嵌入向量表示f的第e维组成的向量。n'=n-nd,nd是文档中y=t对应的f的数目。
本申请实施例中,在给定话题隐变量z-dm、y、知识概念的嵌入向量表示和网络节点的嵌入向量表示的情况下,知识概念的话题zdm=t的条件概率可以写成:
其中,z-dm表示除了文档d中的知识概念m之外的话题隐变量。
本申请实施例中,对于文档d,话题分布参数θd更新如下:
现有技术在嵌入表示模型和话题模型结合的高斯LDA模型中,并没有对嵌入表示进行更新。在本申请实施例中,提出对网络节点和知识概念的嵌入向量表示也进行更新,这样可以更好地利用弱监督学习纠正非监督学习得到的嵌入表示的不足。
将目标函数定义为给定隐变量z和y的情况下的嵌入表示的对数似然,如下:
为了最大化嵌入向量表示的对数似然,本申请实施例直接从上述目标函数得到closed-form的嵌入表示。但是由于每次抽样的话题是具有随机性的,这样得到的嵌入表示容易受到话题的随机性影响,变动太大。因此本申请实施例提出采用梯度下降的方法对嵌入表示进行更新。
知识概念的嵌入表示的梯度为:
网络节点的嵌入表示的梯度为:
在本申请实施例的社交网络中数据处理的方法,可以根据上述D个弱监督信息,学习模型的话题隐变量和模型参数。模型的学习过程分为3个阶段:初始化、burn-in和抽样阶段。
在初始化阶段,可以随机确定所述Mi个第一话题隐变量的初始值和所述第二话题隐变量的初始值。这里,第二话题隐变量的数量与网络节点的数量相同,即该第二话题隐变量与网络节点一一对应。
具体的,可以对话题隐变量y和z进行初始化,每个话题隐变量被uniform随机分配到一个话题。例如,可以初始化zdm=t和yd=t。应注意,对于包含网络节点di的文档,其话题隐变量z也具有Mi个,并且与上述Mi个知识概念一一对应,其话题隐变量y具有一个,即为该网络节点di对应的话题隐变量。这里,为了简洁,省略了话题因变量z的下标。
在burn-in阶段,采用collapsed吉布斯采样的方法进行推理。在每一轮迭代中,都固定其他变量的值,计算当前考察的变量在给定其他变量的情况下的条件概率分布,然后从分布中进行抽样。
在模型的学习过程中,设置burn-in的迭代次数为tb,即burn-in阶段进行tb次的迭代。
本申请实施例中,可以根据所述Mi个第一话题隐变量的初始值、所述第i个弱监督信息和所述Mi个知识概念的话题的条件概率,对所述Mi个第一话题隐变量进行tb次抽样更新。根据所述第二话题隐变量的初始值、所述第i个弱监督信息和所述网络节点di的话题的条件概率,对所述第二话题隐变量进行tb次抽样更新。
其中,所述Mi个知识概念的话题的条件概率中的每个知识概念的话题的条件概率表示所述每个知识概念的第一话题隐变量在给定除所述每个知识概念的第一话题隐变量之外的其他话题隐变量、所述第二话题隐变量、所述每个知识概念的嵌入式向量和所述网络节点di的嵌入式向量的情况下的条件概率。所述网络节点di的话题的条件概率表示所述网络节点di的第二话题隐变量在给定除所述网络节点di的第二话题隐变量之外的其他话题隐变量、所述Mi个第一话题隐变量、所述每个知识概念的嵌入式向量和所述网络节点di的嵌入式向量的情况下的条件概率。
具体的,在第一次迭代时,在初始化后,对模型中的每个知识概念的话题隐变量z,根据上述公式(2)对z进行抽样生成,对模型中的每一个网络节点的话题隐变量,根据上述公式(1)对y进行抽样生成。这里,公式(1)和(2)中的其他变量的值是根据上述D个弱监督信息确定的。
这样,在第一次迭代后,能够对话题隐变量y和z进行抽样更新。这时,为了消除话题隐变量初始值对模型的影响,不对模型参数或嵌入向量表示进行更新。
在第二次进行迭代时,话题隐变量y的初始值为第一次迭代更新之后话题隐变量y的值,话题隐变量z初始值为第一次迭代更新之后话题隐变量z的值。应注意,在burn-in阶段的tb次迭代中。每一次迭代的话题隐变量y的初始值均为上一次迭代更新之后话题隐变量y的值,话题隐变量z初始值均为上一次迭代更新之后话题隐变量z的值。
在更新后的Mi个第一话题隐变量和更新后的第二话题收敛之后,分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行抽样更新。
具体的,经过tb次迭代,模型的话题隐变量y和z基本进入稳定状态之后,也就是在模型的话题隐变量y和z基本收敛之后,模型的学习阶段进入了抽样阶段。在抽样阶段,可以分别对话题隐变量y和z、模型参数进行抽样更新。
在模型的学习过程中,设置抽样阶段的最大迭代次数为tm,即抽样阶段最多进行tm次迭代。还可以设置模型参数更新周期为tp。
这样,在抽样阶段,分别对话题隐变量y和z、模型参数进行抽样更新具体可以为根据所述更新后的Mi个第一话题隐变量、所述第i个弱监督信息和所述所述Mi个知识概念的话题的条件概率,对所述更新后的Mi个第一话题隐变量进行tp次抽样更新;根据所述更新后的第二话题隐变量、所述第i个弱监督信息和所述网络节点di的话题的条件概率,对所述更新后的第二话题隐变量进行tp次抽样更新;在对所述更新后的Mi个第一话题隐变量和所述更新后的第二话题隐变量分别进行tp次抽样更新之后,分别对θ、μj
r、λj
r、μjm
k、
λjm
k进行更新。
这里,可以在对所述更新后的Mi个第一话题隐变量进行tp次抽样更新、对所述更新后的第二话题隐变量进行tp次抽样更新之后,分别对θ、μj
r、λj
r、μjm
k、λjm
k进行一次更新。
具体的,在抽样阶段的第一次迭代中,在对每个网络节点的话题隐变量和知识概念的话题隐变量进行tp次抽样更新之后,根据上述公式(3)、(5)、(7)、(4)和(6)获取更新的模型参数θ、λk、λr、μk和μr。这里,话题隐变量y和z的更新方法与上述burn-in阶段相同。
可选的,本申请实施例中,所述分别对θ、μj
r、λj
r、μjm
k、λjm
k进行更新包括:分别对所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行N次抽样更新,其中,N次更新中的每次更新均执行对所述更新后的Mi个第一话题隐变量进行tp次抽样更新,对所述更新后的Mi个第一话题隐变量进行tp次抽样更新和分别对θ、μj
r、λj
r、μjm
k、λjm
k进行更新,得到N次更新后的θ、μj
r、λj
r、μjm
k、λjm
k。
对N次更新后的θ累加求平均值,将θ累加求得的平均值作为更新后的所述异构贝叶斯模型的θ参数;
对N次更新后的μj
r累加求平均值,将μj
r累加求得的平均值作为更新后的所述异构贝叶斯模型的μj
r参数;
对N次更新后的λj
r累加求平均值,将λj
r累加求得的平均值作为更新后的所述异构贝叶斯模型的λj
r参数;
对N次更新后的μjm
k累加求平均值,将μjm
k累加求得的平均值作为更新后的所述异构贝叶斯模型的μjm
k参数;
对N次更新后的λjm
k累加求平均值,将λjm
k累加求得的平均值作为更新后的所述异构贝叶斯模型的λjm
k参数。
具体的,可以对多个周期tp中计算获得的模型参数进行累加,将抽样阶段所有读取的模型参数的平均值作为最终的模型参数。
这时,可以设置抽样阶段的话题隐变量的迭代次数为tl。因此,在对话题隐变量进行tp次抽样更新、对模型参数θ、λk、λr、μk和μr进行一次更新之后,可以继续按照上述方式对话题隐变量和模型参数θ、λk、λr、μk和μr进行更新。可以理解,这时总共对话题隐变量进行了tl次抽样更新,对模型参数θ、λk、λr、μk和μr进行了(tl/tp)次更新。将(tl/tp)次更新之后的(tl/tp)个θ、λk、λr、μk和μr分别进行累加并求取平均值,分别将获得的θ、λk、λr、μk和μr的平均值作为θ、λk、λr、μk和μr。
可选的,本申请实施例中,分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行抽样更新之后,还可以对网络节点和知识概念的嵌入向量表示进行更新。
具体的,对网络节点和知识概念的嵌入向量表示进行更新,可以包括:根据所述网络节点di的嵌入向量表示和所述Mi个知识概念的嵌入向量表示,确定在给定所述Mi个第一隐变量和所述第二隐变量的情况下的初始对数似然;根据所述初始对数似然,确定更新后的所述网络节点di的嵌入向量表示;根据所述初始对数似然,分别确定更新后的所述Mi个知识概念的嵌入向量表示。
具体的,在对话题隐变量的迭代次数为tl之后,可以对每个网络节点和知识概念的
嵌入向量表示进行更新。这时,可以设置嵌入向量表示的迭代次数为te次,每个网络节点学习初始学习率为lrr,每个知识概念初始学习率为lrk。
在嵌入向量表示的第一次迭代中,首先需要确定当前的模型的初始对数似然。具体的,该初始对数似然可以为上述L。
然后,根据所述初始对数似然确定所述网络节点di的嵌入向量表示的梯度;根据所述网络节点di的嵌入向量表示的梯度、所述网络节点学习率和所述网络节点di的嵌入向量表示,确定所述网络节点di的更新后的嵌入向量表示,其中,所述网络节点学习率表示所述所述网络节点di的嵌入式向量的梯度下降的学习率。
这里,还可以根据所述初始对数似然确定所述Mi个知识概念的嵌入向量表示的梯度;根据所述Mi个知识概念的知识概念的嵌入表示的梯度、所述Mi个知识概念的知识概念学习率和所述Mi个知识概念的嵌入向量表示,分别确定所述Mi个知识概念的更新后的嵌入向量表示,其中,所述Mi个知识概念中的每个知识概念的知识概念学习率表示所述每个知识概念的嵌入式向量的梯度下降的学习率。
具体的,对每一个网络节点的嵌入向量表示fr,可以根据公式(8)计算每个网络节点的梯度g1。这时,可以根据fr+lrr×g1确定网络节点的更新后的嵌入向量表示。
对每一个知识概念的嵌入向量表示fk,可以根据公式(9)计算每个知识概念的梯度g2。这时,可以根据fk+lrk×g2确定知识概念的更新后的嵌入向量表示。
可选的,本申请实施例中,还可以根据所述网络节点di的更新后的嵌入向量表示和所述Mi个知识概念的更新后的嵌入向量表示,确定在给定所述Mi个第一隐变量和所述第二隐变量的情况下的的更新对数似然。
如果所述更新对数似然大于或等于所述初始对数似然,则将所述网络节点di的嵌入向量表示替换为所述网络节点di的更新后的嵌入向量表示,将所述Mi个知识概念的嵌入向量表示替换为所述Mi个知识概念的更新后的嵌入向量表示。
如果所述更新对数似然小于所述初始对数似然,则将所述网络节点学习率更新为所述网络节点学习率与第一衰减指数的乘积,将所述Mi个知识概念的知识概念学习率分别更新为所述Mi个知识概念的知识概念学习率与第二衰减指数的乘积,保留所述网络节点di的嵌入向量表示不变,并且保留所述Mi个知识概念的嵌入向量表示不变,其中,所述第一衰减指数表示所述网络节点学习率的衰减程度,所述第二衰减指数表示所述Mi个知识概念学习率中的每个知识概念学习率的衰减程度。
这里,第一衰减指数与第二衰减指数可以相同,也可以不同,本申请对此不作限定。
这里,保留所述网络节点di的嵌入向量表示不变,即不将所述网络节点di的嵌入向量表示替换为所述网络节点di的更新后的嵌入向量表示。保留所述Mi个知识概念的嵌入向量表示不变,即不将所述Mi个知识概念的嵌入向量表示替换为所述Mi个知识概念的更新后的嵌入向量表示。
具体的,本申请实施例中,由于梯度下降的学习率lrr和lrk比较难设置,本申请实施例使用了一个动态调整梯度下降学习率的技巧。每次进行梯度下降之前,先计算当前模型的对数似然,然后尝试进行梯度下降,再次计算迭代之后的模型的对数似然。
可以理解,如果对数似然上升,说明当前的学习率合适,本申请实施例采用梯度下降之后的嵌入表示。如果对数似然下降,说明当前的学习率过大,这时,将每个网络节点和指示概念的学习率乘以一个衰减指数,并放弃当前迭代对嵌入表示的更新。
这样,完成了抽样阶段的第一次迭代。因为本申请实施例设置的最大迭代次数为tm,因此抽样阶段的迭代过程进行tm次将获得模型的话题隐变量z和y,模型参数θ、λk、λr、μk、μr,和更新之后的每个网络节点和知识概念的嵌入向量表示。
S130,根据所述异构贝叶斯模型,确定第一网络节点与第一知识概念的有效联系。
这里,所述第一网络节点为所述D个弱监督信息中的所有网络节点之中的一个,即所述第一网络节点属于所述D个弱监督信息,所述第一知识概念为所述D个弱监督信息中的所有知识概念之中的一个,即所述第一知识概念属于所述D个弱监督信息。
具体的,该第一网络节点可以为给定的一个网络节点d,该第一知识概念可以为给定的一个知识概念w,第一网络节点和第一知识概念两者在上述得到的异构贝叶斯模型(即本申请实施例中生成的社交知识图谱)中成为邻居的概率为下式:
上式中,省略了其他给定的模型变量。这里,将d和w在社交知识图谱中成为邻居的概率定义为yd通过zw的话题生成后,再分别生成得到两者嵌入向量表示的概率。这里,对zw进行枚举求和,就能得到两者在社交知识图谱中匹配的概率。
本申请实施例中,给定的网络节点a和给定的知识概念1的成为邻居的概率即为该网络节点a和知识概念1的条件概率。当该条件概率的值比较大时,表示该知识概念1为该网络节点a的核心知识概念。反之,当该条件概率的值比较小时,表示该知识概念为该网络节点a的边缘知识概念。因此,本申请实施例可以方便地区分与社交网络中每个网络节点相关的核心知识概念和边缘知识概念。
本申请实施例的社交网络中数据处理的方法,通过确定一系列弱监督信息,该弱监督信息表示网络节点和知识概念的交互信息,并根据该弱监督信息,生成异构贝叶斯模型,该异构贝叶斯模型表示社交网络和知识图谱之间的连接模型,然后根据该异构贝叶斯模型,能够确定给定的网络节点和给定的知识概念之间的有效联系。
本申请实施例生成的异构贝叶斯模型具有准确性和普适性。将本申请实施例的社交网络中数据处理的方法与其他基线算法进行多次对比实验,均得到了有效地验证。下面是对试验中基线算法的简单介绍。
a.AM-base是AMiner***之前的研究兴趣抽取方法。该方法采用基于括词干化,过滤词表,词性标注(POS tagging),词语边界检测等自然语言处理技术的微软结构化查询语言(Structured Query Language,SQL)服务器(Server)关键词抽取应用程序编程接口(Application Programming Interface,API)。通过该API,可以根据上下文依赖关系从文本中抽取关键词之后,该方法依据关键词的频度对关键词进行排序。
b.CountKG方法直接把弱监督信息中的N-gram与知识图谱中的词表进行匹配,抽出所有匹配成功的N-gram作为关键词,按照词频进行排序。
c.GenVector是本申请实施例提出来的数据处理的方法。GenVector包括异构贝叶斯模型,网络节点和知识概念嵌入表示的学习,以及弱监督信息的无监督生成方法。
d.GenVector-E是在本申请实施例的数据处理的方法的基础上去掉了嵌入表示更新的部分的算法。
e.GenVector-M是在本申请实施例的数据处理的方法的基础上去掉异构贝叶斯模型,直接从弱监督信息中随机生成。
f.GenVector-R是在本申请实施例的数据处理的方法的基础上去掉异构贝叶斯模型,直接根据下式:
对给出的分数对知识概念进行排序。
g.作者-话题模型(Author-Topic model)。
h.张量神经网络(Neural tensor network,NTN),该算法使用学习得到的网络节点和知识概念的嵌入表示作为NTN的输入,使用弱监督信息作为训练数据。
下面两个实验描述及其结果是在所进行的几个实验中选取的最具代表性的实验。
第一个实验为个人主页的匹配实验。
该实验通过研究者的个人主页来确定某一个知识概念是否是该研究者的主要研究兴趣,即使用AMiner的主页抽取算法从互联网抓取一部分研究者的主页,接着通过与***的知识概念集合进行匹配,得到研究者的研究兴趣,例如该实验的结果为一个包含1874个研究者的数据集,并用这些知识概念作为正确答案来比较不同的社交网络中数据处理的方法的准确率。
对于每一种社交网络中数据处理的方法,选取每一个研究者排名前五的知识概念,然后计算这五个知识概念的准确率。实验结果如下表1所示:
表1
方法 | 准确率 | 方法 | 准确率 |
GenVector | 77.9402% | AM-base | 73.8189% |
GenVector-E | 77.8548% | CountKB | 54.4823% |
GenVector-M | 65.5608% | Author-topic | 74.4397% |
GenVector-R | 72.8495% | NTN | 65.8911% |
从表1中可以看出,本申请实施例的社交网络中的数据处理的方法的准确率显著地高于基线方法。
具体的,GenVector的准确率相对于AM-base提升了5.58%,说明本申请实施例的社交网络中的数据处理的方法可以较好地更新取代AMiner原有的算法。GenVector的准确率相对于GenVector-E取得提升,说明对于嵌入表示的更新可以得到更好的结果。GenVector的准确率相对于GenVector-M提升了18.88%,GenVector的准确率相对于GenVector-R提升了6.98%,说明异构贝叶斯嵌入模型在算法中发挥了重要作用。GenVector的准确率相对于CountKB提升了43.05%,说明简单的通过知识库匹配统计不能工作,还需要进一步对语义进行理解。GenVector的准确率相对于Author-topic提升了4.70%,说明引入嵌入表示可以增强模型的表达能力。GenVector的准确率相对于NTN提升了18.28%,说明仅仅使用嵌入表示作为分类器的输入不能得到很好的结果。
第二个实验为LinkedIn个人页面的匹配实验。
该实验通过研究者在LinkedIn上的个人页面来确定一个知识概念是否是该研究者的
主要研究兴趣,即使用异构网络匹配算法COSNET将AMiner社交网络和LinkedIn网络进行连接,选取连接概率最大的节点,然后将AMiner上的研究者对应到LinkedIn的账号上。接着从LinkedIn上抓取该研究者的个人页面,并从中抽取一栏,作为研究兴趣的标准答案。将COSNET的概率阈值设置得较高,以保证异构网络连接的可靠性,然后选取所有一栏不为空的研究者。经过上述步骤可得到一个包含222个研究者的数据集。
对于每一种社交网络中数据处理的方法,选取每一个研究者排名前五的知识概念,以LinkedIn个人主页上的一栏为正确答案,计算这五个知识概念的准确率。实验结果如下表2所示:
表2
方法 | 准确率 | 方法 | 准确率 |
GenVector | 26.8468% | AM-base | 24.5195% |
GenVector-E | 26.5765% | CountKB | 25.4954% |
GenVector-M | 24.6695% | Author-topic | 26.4864% |
GenVector-R | 26.3063% | NTN | 24.3243% |
从表2中可以看出,本申请实施例的社交网络中的数据处理的方法的准确率显著地高于基线方法。
具体的,GenVector的准确率相对于GenVector-E,GenVector-M,GenVector-R都有提升,且击败了AM-base,Author-topic,NTN等从不同角度进行建模的方法。
相较于之前的社交网络中数据处理的方法,本申请实施例具有速度快,准确率高的特点。本申请实施例对社交网络中数据处理的方法进行了优化,使计算的速度提升了60倍,单次迭代时间从原来的3小时加快到3分钟。
本申请实施例的社交网络中数据处理的方法,通过确定一系列弱监督信息,该弱监督信息表示网络节点和知识概念的交互信息,并根据该弱监督信息,生成异构贝叶斯模型,该异构贝叶斯模型表示社交网络和知识图谱之间的连接模型,然后根据该异构贝叶斯模型,能够确定给定的网络节点和给定的知识概念之间的有效联系。
图4为本申请实施例的社交网络中数据处理的装置300的示意性框图。该装置300包括:
确定单元310,用于确定D个弱监督信息,所述D个弱监督信息中的第i个弱监督信息包括社交网络中的网络节点di和与所述网络节点di相关的Mi个知识概念,其中,所述网络节点di表示为嵌入向量表示,所述Mi个知识概念分别表示为嵌入向量表示,D和Mi分别为大于1的正整数,i为取值范围为1至D的正整数。
生成单元320,用于根据所述D个弱监督信息,生成异构贝叶斯模型,所述异构贝叶斯模型包括T个话题和D个文档,所述T个话题中的第j个话题包括参数μj
r、λj
r、μjm
k、λjm
k,所述D个文档中的第i个文档包括所述网络节点di和所述Mi个知识概念,所述第i个文档还包括与所述Mi个知识概念一一对应的Mi个第一话题隐变量、与所述网络节点di一一对应的第二话题隐变量和与所述网络节点di一一对应的参数θ。
其中,μj
r表示所述第j个话题对应的所述网络节点di的高斯分布的均值,λj
r表示所
述第j个话题对应的所述网络节点di的高斯分布的精度,μjm
k表示所述第j个话题对应的所述Mi个知识概念中的第m个知识概念的高斯分布的均值,λjm
k表示所述第j个话题对应的所述Mi个知识概念中的m个知识概念的高斯分布的精度,θ表示所述网络节点di的话题多项分布,T为大于1的正整数,j为取值范围为1至T的正整数,m为取值范围为1至Mi的正整数。
所述确定单元310还用于根据所述异构贝叶斯模型,确定第一网络节点与第一知识概念的有效联系。
这里,所述第一网络节点为所述D个弱监督信息中的所有网络节点之中的一个,即所述第一网络节点属于所述D个弱监督信息,所述第一知识概念为所述D个弱监督信息中的所有知识概念之中的一个,即所述第一知识概念属于所述D个弱监督信息。
本申请实施例的社交网络中数据处理的方法,通过确定一系列弱监督信息,该弱监督信息表示网络节点和知识概念的交互信息,并根据该弱监督信息,生成异构贝叶斯模型,该异构贝叶斯模型表示社交网络和知识图谱之间的连接模型,然后根据该异构贝叶斯模型,能够确定给定的网络节点和给定的知识概念之间的有效联系。
可选的,本申请实施例中,第一网络节点与第一知识概念成为邻居的概率可以通过下式计算:
本申请实施例中,给定的网络节点a和给定的知识概念1的成为邻居的概率即为该网络节点a和知识概念1的条件概率。当该条件概率的值比较大时,表示该知识概念1为该网络节点a的核心知识概念。反之,当该条件概率的值比较小时,表示该知识概念为该网络节点a的边缘知识概念。因此,本申请实施例可以方便地区分与社交网络中每个网络节点相关的核心知识概念和边缘知识概念。
可选的,本申请实施例中,生成单元320具体用于:
随机确定所述Mi个第一话题隐变量的初始值和所述第二话题隐变量的初始值。
根据所述第二话题隐变量的初始值、所述第i个弱监督信息和所述网络节点di的话题的条件概率,对所述第二话题隐变量进行tb次抽样更新,其中,所述网络节点di的话题的条件概率表示所述网络节点di的第二话题隐变量在给定除所述网络节点di的第二话题隐变量之外的其他话题隐变量、所述第二话题隐变量、所述每个知识概念的嵌入式向量和所述网络节点di的嵌入式向量的情况下的条件概率。
对于文档d,在给定文档-d的第二话题隐变量y-d、第一话题隐变量z、网络节点的嵌入式表示和知识概念的嵌入式表示的情况下,网络节点的话题yd=t的条件概率可以写成:
其中,n是所有y=t对应的f的数目。假设x是所有y=t的对应的嵌入向量表示f的第e维组成的向量。n'=n-nd,nd是文档中y=t对应的f的数目。
根据所述Mi个第一话题隐变量的初始值、所述第i个弱监督信息和所述Mi个知识概念的话题的条件概率,对所述Mi个第一话题隐变量进行tb次抽样更新,其中,所述Mi个知识概念的话题的条件概率中的每个知识概念的话题的条件概率表示所述每个知识概念的第一话题隐变量在给定除所述每个知识概念的第一话题隐变量之外的其他话题隐变量、所述Mi个第一话题隐变量、所述每个知识概念的嵌入式向量和所述网络节点di的嵌入式向量的情况下的条件概率。
本申请实施例中,在给定话题隐变量z-dm、y、网络节点的嵌入式表示和知识概念的嵌入式表示的情况下,文档d中的知识概念m的话题zdm=t的条件概率可以写成:
其中,z-dm表示除了文档d中的知识概念m之外的第一话题隐变量。
在对所述Mi个第一话题隐变量和所述第二话题隐变量分别进行tb次抽样更新之后,分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行抽样更新,以生成所述异构贝叶斯模型。
可选的,本申请实施例中,所述在所述生成单元320在对所述Mi个第一话题隐变量和所述第二话题隐变量分别进行tb次抽样更新之后,分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行更新,包括:
根据所述更新后的Mi个第一话题隐变量、所述第i个弱监督信息和所述所述Mi个知识概念的话题的条件概率,对所述更新后的Mi个第一话题隐变量进行tp次抽样更新。
根据所述更新后的第二话题隐变量、所述第i个弱监督信息和所述网络节点di的话题的条件概率,对所述更新后的第二话题隐变量进行tp次抽样更新。
这里,所述网络节点di的话题的条件概率和所述Mi个知识概念的话题的条件概率分别为上述公式(1)和公式(2)。
在对所述更新后的Mi个第一话题隐变量和所述更新后的第二话题隐变量分别进行tp次抽样更新之后,分别对θ、μj
r、λj
r、μjm
k、λjm
k进行更新。
本申请实施例中,对于文档d,话题分布参数θd更新如下:
可选的,本申请实施例中,所述生成单元320具体用于:分别对所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行N次抽样更新,其中,所述N次更新中的每次更新均执行对所述更新后的Mi个第一话题隐变量进行tp次抽样更新,对所述更新后的Mi个第一话题隐变量进行tp次抽样更新和分别对θ、μj
r、λj
r、μjm
k、λjm
k进行更新,得到N次更新后的θ、μj
r、λj
r、μjm
k、λjm
k,
对所述N次更新后的θ累加求平均值,将θ累加求得的平均值作为更新后的所述异构贝叶斯模型的θ参数;
对所述N次更新后的μj
r累加求平均值,将μj
r累加求得的平均值作为更新后的所述异构贝叶斯模型的μj
r参数;
对所述N次更新后的λj
r累加求平均值,将λj
r累加求得的平均值作为更新后的所述异构贝叶斯模型的λj
r参数;
对所述N次更新后的μjm
k累加求平均值,将μjm
k累加求得的平均值作为更新后的所述异构贝叶斯模型的μjm
k参数;
对所述N次更新后的λjm
k累加求平均值,将λjm
k累加求得的平均值作为更新后的所述异构贝叶斯模型的λjm
k参数。
可选的,本申请实施例中,在分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行抽样更新分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj
r、λj
r、μjm
k、λjm
k进行抽样更新之后,所述生成单元320还用于:
根据所述网络节点di的嵌入向量表示和所述Mi个知识概念的嵌入向量表示,确定在给定所述Mi个第一隐变量和所述第二隐变量的情况下的初始对数似然。
给定第一话题隐变量和第二话题隐变量情况下的嵌入表示的对数似然的计算公式,如下:
这里,将没有更新的网络节点di的嵌入向量表示和没有更新的Mi个知识概念的嵌入向量表示的对数似然称为初始对数似然。
根据所述初始对数似然,确定所述网络节点di的更新后的嵌入向量表示;
根据所述初始对数似然,分别确定所述Mi个知识概念的更新后的嵌入向量表示。
可选的,本申请实施例中,所述生成单元320根据所述初始对数似然,确定所述网络节点di的更新后的嵌入向量表示,包括:
根据所述初始对数似然确定所述网络节点di的嵌入向量表示的梯度;
具体的,网络节点的嵌入表示的梯度为:
根据所述网络节点di的嵌入向量表示的梯度、网络节点学习率和所述网络节点di的嵌入向量表示,确定所述更新后的所述网络节点di的嵌入向量表示,其中,所述网络节点学习率表示所述所述网络节点di的嵌入式向量的梯度下降的学习率。
可选的,本申请实施例中,所述生成单元根据所述初始对数似然,分别确定所述Mi个知识概念的更新后的嵌入向量表示,包括:
根据所述初始对数似然确定所述Mi个知识概念的嵌入向量表示的梯度;
知识概念的嵌入表示的梯度为:
根据所述Mi个知识概念的知识概念的嵌入表示的梯度、Mi个知识概念的知识概念学习率和所述Mi个知识概念的嵌入向量表示,分别确定所述更新后的所述Mi个知识概念的嵌入向量表示,其中,所述Mi个知识概念中的每个知识概念的知识概念学习率表示所述每个知识概念的嵌入式向量的梯度下降的学习率。
可选的,本申请实施例中,所述生成单元320具体还用于:
根据所述网络节点di的更新后的嵌入向量表示和所述Mi个知识概念的更新后的嵌入向量表示,确定在给定所述Mi个第一隐变量和所述第二隐变量的情况下的的更新对数似然。
这里,将网络节点di的更新之后的嵌入向量表示和Mi个知识概念的更新之后嵌入向量表示的对数似然称为更新对数似然。
若所述更新对数似然大于或等于所述初始对数似然,则将所述网络节点di的嵌入向量表示替换为所述网络节点di的更新后的嵌入向量表示,将所述Mi个知识概念的嵌入向量表示替换为所述Mi个知识概念的更新后的嵌入向量表示;
若所述更新对数似然小于所述初始对数似然,则将所述网络节点学习率更新为所述网络节点学习率与第一衰减指数的乘积,将所述Mi个知识概念的知识概念学习率分别更新为所述Mi个知识概念的知识概念学习率与第二衰减指数的乘积,并且保留所述网络节点di的嵌入向量表示不变,保留所述Mi个知识概念的嵌入向量表示不变,其中,所述第一衰减指数表示所述网络节点学习率的衰减程度,所述第二衰减指数表示所述Mi个知识概念学习率中的每个知识概念学习率的衰减程度。
这里,第一衰减指数与第二衰减指数可以相同,也可以不同,本申请对此不作限定。
可选的,本申请实施例中,所述确定单元310具体用于:
确定D个初始弱监督信息,所述D个初始弱监督信息中的第i个初始弱监督息包括所述网络节点di和与所述网络节点di相关的所有知识概念;
根据所述所有知识概念中每个知识概念的分值和无监督生成率r,确定所述Mi个知
识概念,其中,所述Mi个知识概念中的每一个知识概念的分值大于所述所有知识概念中除所述Mi个知识概念之外的每一个知识概念的分值,所述每个知识概念的分值表示所述每个知识概念处于所述所有知识概念的中心位置的程度,所述无监督生成率r为确定的知识概念的数量Mi与所述所有知识概念的数量的预设比值,r的取值范围为0至1。
可选的,本申请实施例中,所述装置还包括:
转换单元,用于根据知识概念集合、语料的长度和所述语料的最大的N-gram长度,对所述语料进行转换,所述知识概念集合包括所述D个弱监督信息中的所有知识概念;
第一获取单元,用于将转换后的所述语料输入至第一skipgram模型,获取所述知识概念集合中每个知识概念的嵌入向量表示。
可选的,本申请实施例中,所述确定单元310还用于:
确定所述社交网络中的每个网络节点的加权概率,其中,所述每个网络节点的加权概率表示所述每个网络节点访问下一个网络节点的概率,所述社交网络中所有网络节点包括所述D个弱监督信息中的所有网络节点;
根据所述每个网络节点的加权概率,确定所述每个网络节点的至少一个随机游走序列;
具体的,网络节点的加权概率可以表示为:
所述装置还包括第二获取单元,用于将所述每个网络节点的至少一个随机游走序列输入至第二skipgram模型,获取所述每个网络节点的嵌入向量表示。
本申请实施例中,该第一skipgram模型和该第二skipgram模型可以相同,也可以不同,本申请对此不作限定。
应注意,本申请实施例中,确定单元310和生成单元320可以由处理器实现。如图5所示,装置400可以包括处理器410、存储器420和总线***430。其中,存储器420可以用于存储处理器410执行的代码等。
装置400中的各个组件通过总线***430耦合在一起,其中总线***430除包括数据总线之外,还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见,在图中将各种总线都标为总线***430。
在实现过程中,上述方法的各步骤可以通过处理器410中的硬件的集成逻辑电路或者软件形式的指令完成。结合本申请实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成,或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器,闪存、只读存储器,可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器420,处理器410读取存储器420中的信息,结合其硬件完成上述方法的步骤。为避免重复,这里不再详细描述。
应注意,本申请上述方法实施例可以应用于处理器中,或者由处理器实现。处理器可能是一种集成电路芯片,具有信号的处理能力。在实现过程中,上述方法实施例的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器可以是通用处理器、数字信号处理器(Digital Signal Processor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现成可编程门阵列(Field Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成,或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器,闪存、只读存储器,可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器,处理器读取存储器中的信息,结合其硬件完成上述方法的步骤。
可以理解,本申请实施例中的存储器可以是易失性存储器或非易失性存储器,或可包括易失性和非易失性存储器两者。其中,非易失性存储器可以是只读存储器(Read-Only Memory,ROM)、可编程只读存储器(Programmable ROM,PROM)、可擦除可编程只读存储器(Erasable PROM,EPROM)、电可擦除可编程只读存储器(Electrically EPROM,EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory,RAM),其用作外部高速缓存。通过示例性但不是限制性说明,许多形式的RAM可用,例如静态随机存取存储器(Static RAM,SRAM)、动态随机存取存储器(Dynamic RAM,DRAM)、同步动态随机存取存储器(Synchronous DRAM,SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM,DDR SDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM,ESDRAM)、同步连接动态随机存取存储器(Synchlink DRAM,SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM,DR RAM)。应注意,本文描述的***和方法的存储器旨在包括但不限于这些和任意其它适合类型的存储器。
图4所示的装置300或图5所示的装置400能够实现前述图2和图3所示的方法实施例对应的各个过程,具体的,该装置300或装置400可以参见上述图2和图3中的描述,为避免重复,这里不再赘述。
应理解,本文中术语“和/或”,仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。另外,本文中字符“/”,一般表示前后关联对象是一种“或”的关系。
应理解,在本申请的各种实施例中,上述各过程的序号的大小并不意味着执行顺序的先后,各过程的执行顺序应以其功能和内在逻辑确定,而不应对本申请实施例的实施过程构成任何限定。
本领域普通技术人员可以意识到,结合本文中所公开的实施例中描述的各方法步骤和单元,能够以电子硬件、计算机软件或者二者的结合来实现。为了清楚地说明硬件和软件的可互换性,在上述说明中已经按照功能一般性地描述了各实施例的步骤及组成。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。本领域普通技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
在本申请所提供的几个实施例中,应该理解到,所揭露的***、装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个***,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。
另外,在本申请本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本申请本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器ROM、随机存取存储器RAM、磁碟或者光盘等各种可以存储程序代码的介质。
尽管通过参考附图并结合优选实施例的方式对本申请本申请进行了详细描述,但本申请本申请并不限于此。在不脱离本申请本申请的精神和实质的前提下,本领域普通技术人员可以对本申请本申请的实施例进行各种等效的修改或替换,而这些修改或替换都应在本申请本申请的涵盖范围内。
Claims (22)
- 一种社交网络中数据处理的方法,其特征在于,包括:确定D个弱监督信息,所述D个弱监督信息中的第i个弱监督信息包括社交网络中的网络节点di和与所述网络节点di相关的Mi个知识概念,其中,所述网络节点di表示为嵌入式向量,所述Mi个知识概念分别表示为嵌入式向量,D和Mi分别为大于1的正整数,i为取值范围为1至D的正整数;根据所述D个弱监督信息,生成异构贝叶斯模型,所述异构贝叶斯模型包括T个话题和D个文档,所述T个话题中的第j个话题包括参数μj、λj、μjm、λjm,所述D个文档中的第i个文档包括所述网络节点di和所述Mi个知识概念,所述第i个文档还包括与所述Mi个知识概念一一对应的Mi个第一话题隐变量、与所述网络节点di一一对应的第二话题隐变量和与所述网络节点di一一对应的参数θ,其中,μj表示所述第j个话题对应的所述网络节点di的高斯分布的均值,λj表示所述第j个话题对应的所述网络节点di的高斯分布的精度,μjm表示所述第j个话题对应的所述Mi个知识概念中的第m个知识概念的高斯分布的均值,λjm表示所述第j个话题对应的所述Mi个知识概念中的m个知识概念的高斯分布的精度,θ表示所述网络节点di的话题多项分布,T为大于1的正整数,j为取值范围为1至T的正整数,m为取值范围为1至Mi的正整数;根据所述异构贝叶斯模型,确定第一网络节点和第一知识概念的有效联系,其中,所述第一网络节点为所述D个弱监督信息中的所有网络节点之一,所述第一知识概念为所述D个弱监督信息中的所有知识概念之一。
- 根据权利要求1所述的方法,其特征在于,所述根据所述D个弱监督信息,生成异构贝叶斯模型,包括:随机确定所述Mi个第一话题隐变量的初始值和所述第二话题隐变量的初始值;根据所述Mi个第一话题隐变量的初始值、所述第i个弱监督信息和所述Mi个知识概念的话题的条件概率,对所述Mi个第一话题隐变量进行tb次抽样更新,其中,所述Mi个知识概念的话题的条件概率中的每个知识概念的话题的条件概率表示所述每个知识概念的第一话题隐变量在给定除所述每个知识概念的第一话题隐变量之外的其他话题隐变量、所述第二话题隐变量、所述每个知识概念的嵌入式向量和所述网络节点di的嵌入式向量的情况下的条件概率;根据所述第二话题隐变量的初始值、所述第i个弱监督信息和所述网络节点di的话题的条件概率,对所述第二话题隐变量进行tb次抽样更新,其中,所述网络节点di的话题的条件概率表示所述网络节点di的第二话题隐变量在给定除所述网络节点di的第二话题隐变量之外的其他话题隐变量、所述Mi个第一话题隐变量、所述每个知识概念的嵌入式向量和所述网络节点di的嵌入式向量的情况下的条件概率;在对所述Mi个第一话题隐变量和所述第二话题隐变量分别进行tb次抽样更新之后,分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj、λj、μjm、λjm进行更新,以生成所述异构贝叶斯模型。
- 根据权利要求2所述的方法,其特征在于,所述在对所述Mi个第一话题隐变量和所述第二话题隐变量分别进行tb次抽样更新之后,分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj、λj、μjm、λjm进行更新,包括:根据所述更新后的Mi个第一话题隐变量、所述第i个弱监督信息和所述所述Mi个知 识概念的话题的条件概率,对所述更新后的Mi个第一话题隐变量进行tp次抽样更新;根据所述更新后的第二话题隐变量、所述第i个弱监督信息和所述网络节点di的话题的条件概率,对所述更新后的第二话题隐变量进行tp次抽样更新;在对所述更新后的Mi个第一话题隐变量和所述更新后的第二话题隐变量分别进行tp次抽样更新之后,分别对θ、μj、λj、μjm、λjm进行更新。
- 根据权利要求3所述的方法,其特征在于,所述分别对θ、μj、λj、μjm、λjm进行更新包括:分别对所述参数θ、μj、λj、μjm、λjm进行N次更新,其中,所述N次更新中的每次更新均执行对所述更新后的Mi个第一话题隐变量进行tp次抽样更新,对所述更新后的Mi个第一话题隐变量进行tp次抽样更新和分别对θ、μj、λj、μjm、λjm进行更新,得到N次更新后的θ、μj、λj、μjm、λjm,对所述N次更新后的θ累加求平均值,将θ累加求得的平均值作为更新后的所述异构贝叶斯模型的θ参数;对所述N次更新后的μj累加求平均值,将μj累加求得的平均值作为更新后的所述异构贝叶斯模型的μj参数;对所述N次更新后的λj累加求平均值,将λj累加求得的平均值作为更新后的所述异构贝叶斯模型的λj参数;对所述N次更新后的μjm累加求平均值,将μjm累加求得的平均值作为更新后的所述异构贝叶斯模型的μjm参数;对所述N次更新后的λjm累加求平均值,将λjm累加求得的平均值作为更新后的所述异构贝叶斯模型的λjm参数。
- 根据权利要求2-4中任一项所述的方法,其特征在于,所述分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj、λj、μjm、λjm进行更新之后,还包括:根据所述网络节点di的嵌入式向量和所述Mi个知识概念的嵌入式向量,确定在给定所述Mi个第一隐变量和所述第二隐变量的情况下的初始对数似然;根据所述初始对数似然,确定所述网络节点di的更新后的嵌入式向量;根据所述初始对数似然,分别确定所述Mi个知识概念的更新后的嵌入式向量。
- 根据权利要求5所述的方法,其特征在于,所述根据所述初始对数似然,确定所述网络节点di的更新后的嵌入式向量,包括:根据所述初始对数似然确定所述网络节点di的嵌入式向量的梯度;根据所述网络节点di的嵌入式向量的梯度、网络节点学习率和所述网络节点di的嵌入式向量,确定所述所述网络节点di的更新后的嵌入式向量,其中,所述网络节点学习率表示所述所述网络节点di的嵌入式向量的梯度下降的学习率。
- 根据权利要求5所述的方法,其特征在于,所述根据所述初始对数似然,分别确定所述Mi个知识概念的更新后的嵌入式向量,包括:根据所述初始对数似然分别确定所述Mi个知识概念的嵌入式向量的梯度;根据所述Mi个知识概念的知识概念的嵌入表示的梯度、所述Mi个知识概念的知识概念学习率和所述Mi个知识概念的嵌入式向量,分别确定所述所述Mi个知识概念的更新后的嵌入式向量,其中,所述Mi个知识概念中的每个知识概念的知识概念学习率表示所述每个知识概念的嵌入式向量的梯度下降的学习率。
- 根据权利要求5至7中任一项所述的方法,其特征在于,所述方法还包括:根据所述网络节点di的更新后的嵌入式向量和所述Mi个知识概念的更新后的嵌入式向量,确定在给定所述Mi个第一隐变量和所述第二隐变量的情况下的的更新对数似然;若所述更新对数似然大于或等于所述初始对数似然,则将所述网络节点di的嵌入式向量替换为所述网络节点di的更新后的嵌入式向量,将所述Mi个知识概念的嵌入式向量替换为所述Mi个知识概念的更新后的嵌入式向量;若所述更新对数似然小于所述初始对数似然,则将所述网络节点学习率更新为所述网络节点学习率与第一衰减指数的乘积,将所述Mi个知识概念的知识概念学习率分别更新为所述Mi个知识概念的知识概念学习率与第二衰减指数的乘积,并且保留所述网络节点di的嵌入式向量不变,保留所述Mi个知识概念的嵌入式向量不变,其中,所述第一衰减指数表示所述网络节点学习率的衰减程度,所述第二衰减指数表示所述Mi个知识概念学习率中的每个知识概念学习率的衰减程度。
- 根据权利要求1-8中任一项所述的方法,其特征在于,所述确定D个弱监督信息,包括:确定D个初始弱监督信息,所述D个初始弱监督信息中的第i个初始弱监督信息包括所述网络节点di和与所述网络节点di相关的所有知识概念;根据所述所有知识概念中每个知识概念的分值和无监督生成率r,确定所述Mi个知识概念,其中,所述Mi个知识概念中的每一个知识概念的分值大于所述所有知识概念中除所述Mi个知识概念之外的每一个知识概念的分值,所述每个知识概念的分值表示所述每个知识概念处于所述所有知识概念的中心位置的程度,所述无监督生成率r为确定的知识概念的数量Mi与所述所有知识概念的数量的预设比值,r的取值范围为0至1。
- 根据权利要求1-9中任一项所述的方法,其特征在于,在所述确定D个弱监督信息之前,所述方法还包括:根据知识概念集合、语料的长度和所述语料的最大N-gram长度,对所述语料进行转换,所述知识概念集合包括所述D个弱监督信息中的所有知识概念;将转换后的所述语料输入至skipgram模型,获取所述知识概念集合中每个知识概念的嵌入式向量。
- 根据权利要求1-9中任一项所述的方法,其特征在于,在所述确定D个弱监督信息之前,所述方法还包括:确定所述社交网络中的每个网络节点的加权概率,其中,所述每个网络节点的加权概率表示所述每个网络节点访问下一个网络节点的概率,所述社交网络中所有网络节点包括所述D个弱监督信息中的所有网络节点;根据所述每个网络节点的加权概率,确定所述每个网络节点的至少一个随机游走序列;将所述每个网络节点的至少一个随机游走序列输入至skipgram模型,获取所述每个网络节点的嵌入式向量。
- 一种社交网络中数据处理的装置,其特征在于,包括:确定单元,用于确定D个弱监督信息,所述D个弱监督信息中的第i个弱监督信息包括社交网络中的网络节点di和与所述网络节点di相关的Mi个知识概念,其中,所述网络节点di表示为嵌入式向量,所述Mi个知识概念分别表示为嵌入式向量,D和Mi分别 为大于1的正整数,i为取值范围为1至D的正整数;生成单元,用于根据所述D个弱监督信息,生成异构贝叶斯模型,所述异构贝叶斯模型包括T个话题和D个文档,所述T个话题中的第j个话题包括参数μj、λj、μjm、λjm,所述D个文档中的第i个文档包括所述网络节点di和所述Mi个知识概念,所述第i个文档还包括与所述Mi个知识概念一一对应的Mi个第一话题隐变量、与所述网络节点di一一对应的第二话题隐变量和与所述网络节点di一一对应的参数θ,其中,μj表示所述第j个话题对应的所述网络节点di的高斯分布的均值,λj表示所述第j个话题对应的所述网络节点di的高斯分布的精度,μjm表示所述第j个话题对应的所述Mi个知识概念中的第m个知识概念的高斯分布的均值,λjm表示所述第j个话题对应的所述Mi个知识概念中的m个知识概念的高斯分布的精度,θ表示所述网络节点di的话题多项分布,T为大于1的正整数,j为取值范围为1至T的正整数,m为取值范围为1至Mi的正整数;所述确定单元还用于根据所述异构贝叶斯模型,确定第一网络节点与第一知识概念之间的有效联系,其中,所述第一网络节点为所述D个弱监督信息中的所有网络节点之一,所述第一知识概念为所述D个弱监督信息中的所有知识概念之一。
- 根据权利要求12所述的装置,其特征在于,生成单元具体用于:随机确定所述Mi个第一话题隐变量的初始值和所述第二话题隐变量的初始值;根据所述Mi个第一话题隐变量的初始值、所述第i个弱监督信息和所述Mi个知识概念的话题的条件概率,对所述Mi个第一话题隐变量进行tb次抽样更新,其中,所述Mi个知识概念的话题的条件概率中的每个知识概念的话题的条件概率表示所述每个知识概念的第一话题隐变量在给定除所述每个知识概念的第一话题隐变量之外的其他话题隐变量、所述第二话题隐变量、所述每个知识概念的嵌入式向量和所述网络节点di的嵌入式向量的情况下的条件概率;根据所述第二话题隐变量的初始值、所述第i个弱监督信息和所述网络节点di的话题的条件概率,对所述第二话题隐变量进行tb次抽样更新,其中,所述网络节点di的话题的条件概率表示所述网络节点di的第二话题隐变量在给定除所述网络节点di的第二话题隐变量之外的其他话题隐变量、所述Mi个第一话题隐变量、所述每个知识概念的嵌入式向量和所述网络节点di的嵌入式向量的情况下的条件概率;在对所述Mi个第一话题隐变量和所述第二话题隐变量分别进行tb次抽样更新之后,分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj、λj、μjm、λjm进行更新,以生成所述异构贝叶斯模型。
- 根据权利要求13所述的装置,其特征在于,所述在所述生成单元对所述Mi个第一话题隐变量和所述第二话题隐变量分别进行tb次抽样更新之后,分别对所述更新后的Mi个第一话题隐变量、所述更新后的第二话题隐变量和所述参数θ、μj、λj、μjm、λjm进行更新,包括:根据所述更新后的Mi个第一话题隐变量、所述第i个弱监督信息和所述所述Mi个知识概念的话题的条件概率,对所述更新后的Mi个第一话题隐变量进行tp次抽样更新;根据所述更新后的第二话题隐变量、所述第i个弱监督信息和所述网络节点di的话题的条件概率,对所述更新后的第二话题隐变量进行tp次抽样更新;在对所述更新后的Mi个第一话题隐变量和所述更新后的第二话题隐变量分别进行tp次抽样更新之后,分别对θ、μj、λj、μjm、λjm进行更新。
- 根据权利要求13所述的装置,其特征在于,所述生成单元具体用于:分别对所述参数θ、μj、λj、μjm、λjm进行N次抽样更新,其中,所述N次更新中的每次更新均执行对所述更新后的Mi个第一话题隐变量进行tp次抽样更新,对所述更新后的Mi个第一话题隐变量进行tp次抽样更新和分别对θ、μj、λj、μjm、λjm进行更新,得到N次更新后的θ、μj、λj、μjm、λjm,对所述N次更新后的θ累加求平均值,将θ累加求得的平均值作为更新后的所述异构贝叶斯模型的θ参数;对所述N次更新后的μj累加求平均值,将μj累加求得的平均值作为更新后的所述异构贝叶斯模型的μj r参数;对所述N次更新后的λj累加求平均值,将λj累加求得的平均值作为更新后的所述异构贝叶斯模型的λj参数;对所述N次更新后的μjm累加求平均值,将μjm累加求得的平均值作为更新后的所述异构贝叶斯模型的μjm参数;对所述N次更新后的λjm累加求平均值,将λjm累加求得的平均值作为更新后的所述异构贝叶斯模型的λjm参数。
- 根据权利要求13-15中任一项所述的装置,其特征在于,所述生成单元还用于:根据所述网络节点di的嵌入式向量和所述Mi个知识概念的嵌入式向量,确定在给定所述Mi个第一隐变量和所述第二隐变量的情况下的初始对数似然;根据所述初始对数似然,确定所述网络节点di的更新后的嵌入式向量;根据所述初始对数似然,分别确定所述Mi个知识概念的更新后的嵌入式向量。
- 根据权利要求16所述的装置,其特征在于,所述生成单元根据所述初始对数似然,确定所述网络节点di的更新后的嵌入式向量,包括:根据所述初始对数似然确定所述网络节点di的嵌入式向量的梯度;根据所述网络节点di的嵌入式向量的梯度、所述网络节点学习率和所述网络节点di的嵌入式向量,确定所述所述网络节点di的更新后的嵌入式向量,其中,所述网络节点学习率表示所述所述网络节点di的嵌入式向量的梯度下降的学习率。。
- 根据权利要求16所述的装置,其特征在于,所述生成单元根据所述初始对数似然,分别确定所述Mi个知识概念的更新后的嵌入式向量,包括:根据所述初始对数似然分别确定所述Mi个知识概念的嵌入式向量的梯度;根据所述Mi个知识概念的知识概念的嵌入表示的梯度、所述Mi个知识概念的知识概念学习率和所述Mi个知识概念的嵌入式向量,分别确定所述所述Mi个知识概念的更新后的嵌入式向量,其中,所述Mi个知识概念中的每个知识概念的知识概念学习率表示所述每个知识概念的嵌入式向量的梯度下降的学习率。
- 根据权利要求16至18中任一项所述的装置,其特征在于,所述生成单元还用于:根据所述所述网络节点di的更新后的嵌入式向量和所述Mi个知识概念的更新后的嵌入式向量,确定在给定所述Mi个第一隐变量和所述第二隐变量的情况下的的更新对数似然;若所述更新对数似然大于或等于所述初始对数似然,则将所述网络节点di的嵌入式向量替换为所述网络节点di的更新后的嵌入式向量,将所述Mi个知识概念的嵌入式向量 替换为所述Mi个知识概念的更新后的嵌入式向量;若所述更新对数似然小于所述初始对数似然,则将所述网络节点学习率更新为所述网络节点学习率与第一衰减指数的乘积,将所述Mi个知识概念的知识概念学习率分别更新为所述Mi个知识概念的知识概念学习率与第二衰减指数的乘积,并且保留所述网络节点di的嵌入式向量不变,保留所述Mi个知识概念的嵌入式向量不变,其中,所述第一衰减指数表示所述网络节点学习率的衰减程度,所述第二衰减指数表示所述Mi个知识概念学习率中的每个知识概念学习率的衰减程度。
- 根据权利要求12-19中任一项所述的装置,其特征在于,所述确定单元具体用于:确定D个初始弱监督信息,所述D个初始弱监督信息中的第i个初始弱监督信息包括所述网络节点di和与所述网络节点di相关的所有知识概念;根据所述所有知识概念中每个知识概念的分值和无监督生成率r,确定所述Mi个知识概念,其中,所述Mi个知识概念中的每一个知识概念的分值大于所述所有知识概念中除所述Mi个知识概念之外的每一个知识概念的分值,所述每个知识概念的分值表示所述每个知识概念处于所述所有知识概念的中心位置的程度,所述无监督生成率r为确定的知识概念的数量Mi与所述所有知识概念的数量的预设比值,r的取值范围为0至1。
- 根据权利要求12-20任一项所述的装置,其特征在于,所述装置还包括:转换单元,用于根据知识概念集合、语料的长度和所述语料的最大N-gram长度,对所述语料进行转换,所述知识概念集合包括所述D个弱监督信息中的所有知识概念;第一获取单元,用于将转换后的所述语料输入至skipgram模型,获取所述知识概念集合中每个知识概念的嵌入式向量。
- 根据权利要求12-20任一项所述的装置,其特征在于,所述确定单元还用于:确定所述社交网络中的每个网络节点的加权概率,其中,所述每个网络节点的加权概率表示所述每个网络节点访问下一个网络节点的概率,所述社交网络中所有网络节点包括所述D个弱监督信息中的所有网络节点;根据所述每个网络节点的加权概率,确定所述每个网络节点的至少一个随机游走序列;所述装置还包括第二获取单元,用于将所述每个网络节点的至少一个随机游走序列输入至skipgram模型,获取所述每个网络节点的嵌入式向量。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610311864.4 | 2016-05-11 | ||
CN201610311864.4A CN107369098B (zh) | 2016-05-11 | 2016-05-11 | 社交网络中数据的处理方法和装置 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017193685A1 true WO2017193685A1 (zh) | 2017-11-16 |
Family
ID=60266019
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2017/075931 WO2017193685A1 (zh) | 2016-05-11 | 2017-03-08 | 社交网络中数据的处理方法和装置 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN107369098B (zh) |
WO (1) | WO2017193685A1 (zh) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110727794A (zh) * | 2018-06-28 | 2020-01-24 | 上海传漾广告有限公司 | 一种网络语义收集分析及内容概括分析***及方法 |
WO2020056984A1 (zh) * | 2018-09-19 | 2020-03-26 | 平安科技(深圳)有限公司 | 最短路径查询方法、***、计算机设备和存储介质 |
CN111611530A (zh) * | 2020-05-20 | 2020-09-01 | 杭州中奥科技有限公司 | 案件与人员关系分析方法、装置及电子设备 |
CN111640033A (zh) * | 2020-04-11 | 2020-09-08 | 中国人民解放军战略支援部队信息工程大学 | 一种网络水军的检测方法及装置 |
CN112580676A (zh) * | 2019-09-29 | 2021-03-30 | 北京京东振世信息技术有限公司 | 聚类方法、聚类装置、计算机可读介质及电子设备 |
CN113255918A (zh) * | 2021-04-13 | 2021-08-13 | 国家计算机网络与信息安全管理中心 | 强化聚合知识指导的生成常识推理方法 |
CN113792123A (zh) * | 2021-11-17 | 2021-12-14 | 广州极天信息技术股份有限公司 | 一种基于数据驱动的领域知识图谱构建方法及*** |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109523373B (zh) * | 2018-11-13 | 2022-07-15 | 深圳前海微众银行股份有限公司 | 远程核身方法、设备及计算机可读存储介质 |
CN109508390B (zh) * | 2018-12-28 | 2021-12-14 | 北京金山安全软件有限公司 | 基于知识图谱的输入预测方法、装置和电子设备 |
CN114255056A (zh) * | 2020-09-19 | 2022-03-29 | 华为技术有限公司 | 广告显示方法及电子设备 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7433853B2 (en) * | 2004-07-12 | 2008-10-07 | Cardiac Pacemakers, Inc. | Expert system for patient medical information analysis |
CN103207860A (zh) * | 2012-01-11 | 2013-07-17 | 北大方正集团有限公司 | 舆情事件的实体关系抽取方法和装置 |
CN103268607A (zh) * | 2013-05-15 | 2013-08-28 | 电子科技大学 | 一种弱监督条件下的共同对象检测方法 |
CN104463249A (zh) * | 2014-12-09 | 2015-03-25 | 西北工业大学 | 一种基于弱监督学习框架的遥感图像机场检测方法 |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2010045463A2 (en) * | 2008-10-15 | 2010-04-22 | Government Of The United States As Represented By The Secretary Of The Army | Clinical decision model |
CN103581270B (zh) * | 2012-08-08 | 2015-12-16 | 腾讯科技(深圳)有限公司 | 用户推荐方法和*** |
CN103226580B (zh) * | 2013-04-02 | 2016-03-30 | 西安交通大学 | 一种面向交互文本的话题识别方法 |
CN105550190B (zh) * | 2015-06-26 | 2019-03-29 | 许昌学院 | 面向知识图谱的跨媒体检索*** |
-
2016
- 2016-05-11 CN CN201610311864.4A patent/CN107369098B/zh active Active
-
2017
- 2017-03-08 WO PCT/CN2017/075931 patent/WO2017193685A1/zh active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7433853B2 (en) * | 2004-07-12 | 2008-10-07 | Cardiac Pacemakers, Inc. | Expert system for patient medical information analysis |
CN103207860A (zh) * | 2012-01-11 | 2013-07-17 | 北大方正集团有限公司 | 舆情事件的实体关系抽取方法和装置 |
CN103268607A (zh) * | 2013-05-15 | 2013-08-28 | 电子科技大学 | 一种弱监督条件下的共同对象检测方法 |
CN104463249A (zh) * | 2014-12-09 | 2015-03-25 | 西北工业大学 | 一种基于弱监督学习框架的遥感图像机场检测方法 |
Non-Patent Citations (1)
Title |
---|
YANG, ZHILIN: "da4gui1mo2she4jiao1zhi1shi2tu2pu2dezi4dong4sheng2cheng2yan2jiu1", TSINGHUA UNIVERSITY-DATABASE OF OUTSTANDING WORKS OF STUDENTS, 18 March 2016 (2016-03-18), pages 16 - 30 * |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110727794A (zh) * | 2018-06-28 | 2020-01-24 | 上海传漾广告有限公司 | 一种网络语义收集分析及内容概括分析***及方法 |
WO2020056984A1 (zh) * | 2018-09-19 | 2020-03-26 | 平安科技(深圳)有限公司 | 最短路径查询方法、***、计算机设备和存储介质 |
CN112580676A (zh) * | 2019-09-29 | 2021-03-30 | 北京京东振世信息技术有限公司 | 聚类方法、聚类装置、计算机可读介质及电子设备 |
CN111640033A (zh) * | 2020-04-11 | 2020-09-08 | 中国人民解放军战略支援部队信息工程大学 | 一种网络水军的检测方法及装置 |
CN111611530A (zh) * | 2020-05-20 | 2020-09-01 | 杭州中奥科技有限公司 | 案件与人员关系分析方法、装置及电子设备 |
CN111611530B (zh) * | 2020-05-20 | 2023-11-17 | 杭州中奥科技有限公司 | 案件与人员关系分析方法、装置及电子设备 |
CN113255918A (zh) * | 2021-04-13 | 2021-08-13 | 国家计算机网络与信息安全管理中心 | 强化聚合知识指导的生成常识推理方法 |
CN113792123A (zh) * | 2021-11-17 | 2021-12-14 | 广州极天信息技术股份有限公司 | 一种基于数据驱动的领域知识图谱构建方法及*** |
Also Published As
Publication number | Publication date |
---|---|
CN107369098A (zh) | 2017-11-21 |
CN107369098B (zh) | 2021-10-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2017193685A1 (zh) | 社交网络中数据的处理方法和装置 | |
Dhingra et al. | Embedding text in hyperbolic spaces | |
CN107133213B (zh) | 一种基于算法的文本摘要自动提取方法与*** | |
US11301637B2 (en) | Methods, devices, and systems for constructing intelligent knowledge base | |
Sun et al. | Sentiment analysis for Chinese microblog based on deep neural networks with convolutional extension features | |
CN104834747B (zh) | 基于卷积神经网络的短文本分类方法 | |
JP7316721B2 (ja) | 対象領域およびクライアント固有のアプリケーション・プログラム・インタフェース推奨の促進 | |
WO2021051518A1 (zh) | 基于神经网络模型的文本数据分类方法、装置及存储介质 | |
CN110969020A (zh) | 基于cnn和注意力机制的中文命名实体识别方法、***及介质 | |
CN111104518A (zh) | 用于从用户生成的内容建立演化本体的***和方法 | |
CN113392209B (zh) | 一种基于人工智能的文本聚类方法、相关设备及存储介质 | |
JP7303195B2 (ja) | 対象領域およびクライアント固有のアプリケーション・プログラム・インタフェース推奨の促進 | |
US11227183B1 (en) | Section segmentation based information retrieval with entity expansion | |
CN113761868B (zh) | 文本处理方法、装置、电子设备及可读存储介质 | |
Jin et al. | Multi-label sentiment analysis base on BERT with modified TF-IDF | |
Fang et al. | MANNER: A variational memory-augmented model for cross domain few-shot named entity recognition | |
CN109977194B (zh) | 基于无监督学习的文本相似度计算方法、***、设备及介质 | |
Bhuvaneswari et al. | A deep learning approach for the depression detection of social media data with hybrid feature selection and attention mechanism | |
CN113111136B (zh) | 一种基于ucl知识空间的实体消歧方法及装置 | |
Dong et al. | DC‐BiGRU‐CNN Algorithm for Irony Recognition in Chinese Social Comments | |
He et al. | Distant supervised relation extraction via long short term memory networks with sentence embedding | |
Batista | Large-scale semantic relationship extraction for information discovery | |
Deepthi et al. | Query-Based Retrieval Using Universal Sentence Encoder. | |
Wang et al. | Academic Activities Transaction Extraction Based on Deep Belief Network | |
CN113076754A (zh) | 一种基于知识集成的虚假评论检测方法及*** |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17795303 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 17795303 Country of ref document: EP Kind code of ref document: A1 |