CN116308854A - Information cascading popularity prediction method and system based on probability diffusion - Google Patents

Abstract

The invention belongs to the technical field of information propagation, and discloses an information cascade popularity prediction method and system based on probability diffusion, which are used for modeling time irregularity of cascade events and uncertainty of information propagation based on a neural ordinary differential equation and a diffusion probability model and carrying out cascade popularity prediction. Firstly, constructing information cascade data into a cascade social graph, a cascade graph and a cascade sequence, wherein the cascade graph and the cascade sequence are used for feature learning of downstream structure and sequence models; then, a gating mechanism based on a normal differential equation and time perception obtains cascading implicit characteristics; on the basis, from the perspective of space-time hidden variables, combining a conditional diffusion probability model and an implicit ordinary differential equation to obtain an implicit characteristic of uncertain cascade; finally, popularity prediction is performed by using the cascade implicit features and the cascade uncertain implicit features. The method can be used for continuous time state modeling and propagation uncertainty modeling of information cascading, and can be used for predicting popularity of the information cascading better.

Description

Information cascading popularity prediction method and system based on probability diffusion

Technical Field

The invention belongs to the technical field of information propagation, relates to information cascade popularity prediction, in particular to information diffusion (Information Diffusion) and popularity prediction (Popularity Prediction) in Deep Learning (Deep Learning), and relates to a method based on a neural ordinary differential equation (Neural Ordinary Differential Equations, NODEs) and a diffusion probability model (Diffusion Probabilistic Model, DPM).

Background

With the widespread use of social media, such as Twitter, newwave microblogs, etc., it has become one of the main sources of information for users to generate and disseminate information and guide users' daily decisions. The forwarding or sharing behavior of users facilitates rapid propagation of information on a social platform, facilitating generation of information cascades, such as: the initial information release (e.g., news, blogs) of the user and the sharing and forwarding of other users form an information cascade. The information cascade popularity prediction is to predict the scale of a forwarding user after a certain cascade (text pushing, microblog and the like) passes a specific period of time by observing the propagation evolution process (such as forwarding user and time) of the early stage of information. The accurate prediction of the popularity of the information cascade is beneficial to the rapid and effective propagation of the information, and can bring great economic and social effects, which also gets great attention in academia and industry. In the propagation of information, there are generally two important phenomena: the temporal irregularities of cascading events (e.g., users may forward or share news, tweets, etc. at any time) and the inherent uncertainty of information dissemination. The existing accurate prediction method of the information cascade popularity is mainly divided into three types: (1) probability generation model: modeling an information propagation process by using a time point process, such as a poisson process, a hopus process and the like, by using a strength function of a newly-appearing cascade event; (2) Based on the model of the characteristics, the method predicts the cascade popularity by exploring and designing the characteristics such as cascade structure, time, user attribute, information content and the like; (3) The deep learning model models cascading events mainly using simple event sequence models, such as a recurrent neural network (Recurrent Neural Network, RNN), long Short-Term Memory (LSTM), and gated loop units (Gated Recurrent Unit, GRU). However, the existing method ignores the time irregularity of cascading events and the uncertainty of information propagation in the actual information diffusion process, so that accurate prediction of the popularity of information cascading is difficult to realize, and the prediction effect is not ideal.

Disclosure of Invention

The invention aims at solving the technical problems in the prior art, and designs a novel information cascade popularity prediction method and system based on a neural ordinary differential equation and a diffusion probability model, which only use the structural information of a cascade social graph and a cascade graph and the diffusion information of the cascade, simulate the time irregularity of a cascade event and the uncertainty of information propagation in the information propagation process, and improve the accuracy of cascade popularity prediction.

The method is characterized in that a framework based on a neural ordinary differential equation and a diffusion probability model is constructed to model time irregularity of cascade events and uncertainty of information propagation, and cascade popularity prediction is carried out. Firstly, constructing collected cascade data into a cascade social graph, a cascade graph and a cascade sequence, wherein the cascade graph, the cascade graph and the cascade sequence are used for feature learning of a downstream structure and sequence model; a novel time-aware neural frequent differentiation module (Temporal Ordinary Differential Equations, T-ODE) is then designed to model the temporal irregularities of cascading events by generalizing the discrete feature states in the RNN to continuous-time dynamics defined by the ODE. Meanwhile, the input information and time interval information between cascade events are considered through a gating mechanism to update the implicit state of cascade, so that the learned cascade characteristics can better accord with the real information propagation process. On the basis, from the perspective of space-time hidden variables, a cascade uncertainty modeling module (DPM-ODE) combining a conditional diffusion probability model and an implicit ordinary differential equation (Latent Ordinary Differential Equations) is designed, and the module simultaneously considers the uncertainty of cascade evolution (time dependence of a cascade diagram) and the uncertainty of user space association (space structure of the cascade diagram). In this module, the conditional diffusion probability model approximates the posterior distribution score of the user structural features on the condition of continuous time cascading features, thereby reconstructing the cascading graph structural features to simulate the uncertainty of the user spatial correlation. The conditional probability diffusion model is trained through explicit cascading structure generation, and useful correlations between structural features can be observed. On the basis, the implicit ordinary differential equation defines a generating process with time according to the deterministic evolution of the initial cascade state, and simultaneously acquires cascade characteristics of propagation uncertainty. Through the design of two modules, the continuous time dynamic and propagation uncertainty representation of the cascade is combined and input into a fully connected layer for popularity prediction. The method realizes a more novel popularity modeling mode and improves the accuracy of cascade popularity prediction.

Based on the inventive thought, the invention provides an information cascade popularity prediction method based on probability diffusion, which comprises the following steps:

s1, constructing cascade data into a social graph, a cascade graph and a cascade sequence; then obtaining global structural features according to the structural attributes of the cascade global social graph; obtaining node similarity characteristics according to the cascade diagram; then, splicing the global structural features and the node similarity features to obtain a structural embedded representation of the user;

s2, acquiring the implicit state of the current moment according to the implicit state of the previous moment and the embedded representation of the user structure; then obtaining an ODE implicit state by using a first ODE solver according to the implicit state at the previous moment and the implicit state at the current moment; then, the implicit state and the ODE implicit state at the current moment are cascaded to obtain a cascaded implicit state; then obtaining an updated cascade implicit state according to the implicit state at the current moment and the cascade implicit state and outputting cascade implicit characteristics through full connection based on a time-aware gating mechanism;

s3, generating a target cascade structure embedded representation of a user based on the diffusion probability model by taking the cascade implicit characteristic obtained in the step S2 as a condition, and resampling; then, evolution is carried out on the resampling result in a probability space by using a second ODE solver to obtain an implicit characteristic with uncertain cascade;

S4, predicting the cascade popularity according to the cascade implicit characteristic obtained in the step S2 and the cascade uncertain implicit characteristic splicing result.

In step S1, the cascade data is constructed as a social graph

Cascading diagram->

And a cascading sequence. The social graph->

Mainly consisting of all subscribers to the cascading sequence. The cascade diagram->

Mainly consists of any one of the subscribers of the cascade sequence.

The cascaded global social graph is then learned using sparse matrix decomposition (sparsematrixfactor)

Obtaining global structural feature E _g (see C.Donnat, M.Zitnik, D.Hallac, and J.Leskovic, "Learning structural node embeddings via diffusion wavelets," in SIGKDD,2018, pp.1320-132); modeling a cascade Graph using Graph Wavelets model>

Obtaining node similarity features E _c (see J.Zhang, Y.Dong, Y.Wang, J.Tang, and M.Ding, "ProNE: fast and scalable network representation learning," in IJCAI, macao, china, aug.10-16,2019, pp.4278-4284); finally to E _g And E is _c And performing splicing operation to obtain the structural representation E of the user.

Because of the randomness of human response behavior, the forwarding of information may occur at any time, resulting in an irregular time sequence in the information cascade. The invention promotes the cascade state in the RNN to continuous time dynamic defined by ODE through step S2, and simultaneously considers input information and time interval information between cascade events through a gating mechanism.

Firstly, acquiring the implicit state of the current moment by utilizing an LSTM unit according to the implicit state of the previous moment and the embedded representation of the user structure so as to avoid the gradient disappearance or explosion phenomenon caused by using an ODE solver.

Given at time t _i-1 Is a concatenation of implicit state pairs (c) _i-1 ,h _i-1 ) And user u's structure embedding

First they are input into LSTM cell to generate new implicit stateState (c) _i ,h′ _i ):

Wherein θ _l Representing model parameters that can be learned in LSTM cells.

Then we will h _i-1 Inputting the data to a first ODE solver based on Euler algorithm to obtain an ODE implicit state z _i ：

z _i ＝ODESolver(f _ω ,h _i-1 ,h′ _i ,(t _i-1 ,t _i ))

Through the above operation, using a first ODE solver based on the Euler algorithm to evaluate the cascade hidden states between successive observations, an irregular time interval t is constructed _i-1 And t _i A cascade of continuous times in between.

To construct the actual time t _i We will implicit state h 'at the current time' _i And ODE implicit State z _i As input, learn the true characteristics of the cascade; the GRU unit is used to update the implicit state of cascade, i.e. the implicit state of current time and the implicit state of ODE are cascade to obtain the implicit state h _i . The above process is expressed as:

h″ _i ＝GRUCell(θ _g ,h′ _i ,z _i )

wherein,,

is the ODE from time t _i-1 By time t _i Is a solution of h _i Is the cascade implicit state after GRU unit update, θ _g Representing model parameters that are learnable in the GRU unit;

finally, a time-aware gating mechanism (T-Gate) is designed to integrate the implicit representations of the first two steps to generate a continuous-time cascade of states.

The neural ordinary differential equation regards the parameter update in the neural network as a process of solving the ordinary differential equation, and from the point of view of the numerical method, the discrete layer of the neural network can be regarded as the Euler discretization of the differential equation:

wherein h (t) =h _t . The neural network is composed of f _ω (. Cndot.) parameterization, building a continuous dynamics of cascading implicit states. The parameter update process of NODE can be regarded as solving ODE numerically.

Given an implicit state h' _i And h' _i Updating concatenated implicit state h using time-aware gating mechanisms _i ：

h _i ＝v _i ⊙h″ _i +(1-v _i )⊙h′ _i ,

Wherein,,

representing time gating; Δt (delta t) _i Representing t _i -t _i-1 . Time gating->

The help model determines the continuous time state that needs to be delivered.

Finally, we calculate the output state o through a full link layer ₁ …o _n }. Wherein,,

and n represents the number of users evolving in the early stage. Typically, we use the last output state o _n Representing a cascade implicit feature Z.

In step S3, the information propagation uncertainty exists not only in the evolution process of the cascade (time dependence of the cascade graph) but also in the spatial correlation between users (spatial structure of the cascade graph). The invention integrates a diffusion probability model and an implicit neural ordinary differential equation to model information diffusion uncertainty from the perspective of space-time hidden variables. The invention reconstructs cascading information cascading structural features by approximating posterior distribution scores of user structural features on the basis of a designed conditional diffusion probability model by taking continuous time cascading features as conditions. The conditional probability diffusion model is used for structure generation and useful correlations between structural features can be observed. On the basis, the implicit ordinary differential equation defines a generating process with time according to the deterministic evolution of the initial cascade state, and simultaneously acquires cascade characteristics of propagation uncertainty. Based on the above analysis, step S3 of the present invention comprises the following sub-steps:

s31 given information cascade diagram

And social graph->

And structural feature E, obtaining an initial data distribution of q (E ₀ )，E ₀ ＝E；

S32 based on the diffusion probability model, the forward process gradually adds noise to the data to reduce the prior q (E ₀ ) Converting into Gaussian distribution which is easy to process; obtaining model distribution through the inverse process of the diffusion probability model; obtaining user structure embedded E by model distributed sampling ^ta ；

The uncertainty of the structure embedding is captured in a probabilistic manner using a diffusion probability model. The conditional diffusion probability model uses the cascade implicit characteristic Z learned in step S2 ^co Structure embedding for generating target concatenation using =z as condition

Probability generation of the diffusion probability model is aimed at using model distribution +.>

To estimate the true conditional data distribution +.>

In this step, based onThe diffusion probability model, forward process gradually adds a priori q (E ₀ ) Converting into Gaussian distribution which is easy to process; the forward process of each time step R e {0,1,2, …, R } is defined as a gaussian change:

in the formula, = _r Controlling a process of adding gaussian noise to data for a fixed constant;

the gaussian distribution is represented by the formula,

mean, beta _r I represents variance; inverse process simulation conditional distribution using conditional diffusion probability model

Defining a conditional denoising function E _θ :(E ^ta |Z ^co )→E ^ta It will Z ^co As input:

in the method, in the process of the invention,

mean value of gaussian distribution; />

Representing the variance of the gaussian distribution;

we set alpha _r ：＝1-β _r And

distribution q (E) _r |E ₀ ) Expressed as:

the goal of (2) is to eliminate gaussian noise added during diffusion. By minimizing the variational negative log-likelihood, the parameter θ can be fitted to the data distribution q (E ₀ )：

The above formula can be simplified and effectively trained by random gradient descent:

Wherein the noise

∈ _θ Is the output of a noise prediction model integrating a residual neural network, a convolution neural network and an attention mechanism, and the input of the noise prediction model is represented by a user structure +.>

The step of time r and the step of S2 are obtained to obtain a cascade implicit characteristic Z; wherein (1)>

The sampling process may be defined as:

wherein,,

and->

By sampling from r to 0 we obtain a sample E ^ta For modeling the evolution uncertainty described below.

S33 embedding E according to the user structure ^ta Acquiring an initial cascade state

Then reconstructing an initial cascade state based on the variation self-encoder;

embedding E according to user structure ^ta According to the step S2 given above, the initial cascade state is obtained by inverse solution

The extrapolation property of the ODE is then utilized to model the cascading evolution dynamics with uncertainty based on the variational self-encoder (VariationalAutoencoders, VAEs) framework. Assume that

Is a true posterior distribution, we use model distribution based on the VAE framework +.>

To approximate the true posterior distribution->

Wherein phi is a parameter of the neural network, by using a linear transformation from +.>

Derived mean->

Sum of variances->

Combining with the re-parameterization technique, re-sampling to obtain a reconstructed initial cascade shapeStatus- >

Wherein ζ is sampled from the normal ethernet distribution.

S34, evolution is carried out on the reconstructed initial cascade state in a probability space by using a second ODE solver to obtain an implicit characteristic Z with uncertain cascade ^′ _T ：

Wherein f _ξ (. Cndot.) is the ODE function of calculating the derivative, t _n Indicating the last observation instant of the concatenated sequence. In this way we have generated a continuous evolutionary trajectory in which each point represents an implicit variable p (Z 'following a posterior distribution' _t |Z′ ₀ ,…,Z′ _t-1 ). Finally, we optimize model parameters by maximizing the lower bound of Evidence (ELBO):

in the method, in the process of the invention,

expressed as +.>

Is the desire under conditions; />

Represents the KL divergence Kullback-Leibler divergence.

In step S4, the cascade popularity is obtained by using the multi-layer perceptron according to the cascade implicit characteristic and the cascade uncertain implicit characteristic splicing result obtained in step S2.

First, we combine the cascade implicit feature Z from step S2 with the cascade uncertain implicit feature Z from step S3 ^′ _T Performing splicing operations, and then inputting them into Multi-layer perceptron (Multi-Layer Perceptrons, MLPs) for cascade popularityAnd (3) degree prediction:

wherein Concat (-) represents the splice operation. In the training process, we target the Mean Square Log Error (MSLE), combining MSLE and ELBO to train the model. The final loss function is defined as:

Where N is the total number of cascades, P _k (t _p ) Representing the number of users of the kth real forwarding cascade,

representing popularity of the kth prediction.

The invention further provides an information cascading popularity prediction system based on probability diffusion, which comprises the following steps:

the structure embedding representation acquisition module is used for constructing cascade data into a social graph, a cascade graph and a cascade sequence; then obtaining global structural features according to the structural attributes of the cascade global social graph; obtaining node similarity characteristics according to the cascade diagram; then, splicing the global structural features and the node similarity features to obtain a structural embedded representation of the user;

time-aware neural ordinary differential module T-ODE: the implicit state acquisition module is used for acquiring the implicit state of the current moment according to the implicit state of the previous moment and the embedded representation of the user structure; then obtaining an ODE implicit state by using a first ODE solver according to the implicit state at the previous moment and the implicit state at the current moment; then, the implicit state and the ODE implicit state at the current moment are cascaded to obtain a cascaded implicit state; then obtaining an updated cascade implicit state according to the implicit state at the current moment and the cascade implicit state and outputting cascade implicit characteristics through full connection based on a time-aware gating mechanism;

Cascaded uncertainty modeling module DPM-ODE: generating a target cascade structure embedded representation of a user based on a diffusion probability model by taking cascade implicit characteristics obtained by a time-aware neural ordinary differential module T-ODE as conditions, and resampling; then, evolution is carried out on the resampling result in a probability space by using a second ODE solver to obtain an implicit characteristic with uncertain cascade;

and the cascade popularity prediction model is used for predicting cascade popularity according to the cascade implicit characteristics and the cascade uncertain implicit characteristic splicing result.

The neural ordinary differential module T-ODE of the time perception comprises:

the LSTM unit is used for acquiring the implicit state of the current moment according to the implicit state of the previous moment and the embedded representation of the user structure;

the first ODE solver obtains an ODE implicit state according to the implicit state at the previous moment and the implicit state at the current moment;

the GRU unit is used for cascading the implicit state at the current moment and the implicit state of the ODE to obtain a cascading implicit state;

the updating unit is used for updating the cascade implicit state based on a time-aware gating mechanism;

and the full connection layer is used for carrying out full connection processing on the splicing result of the global structural features and the node similarity features to obtain the structural embedded representation of the user.

The cascade uncertainty modeling module DPM-ODE comprises:

an initial data distribution acquisition unit for giving an information cascade graph

And social graph->

And structural feature E, obtaining an initial data distribution of q (E ₀ )，E ₀ ＝E；

Uncertainty-based user structure embedding generation unit, forward process gradually adds a priori q (E ₀ ) Converting into Gaussian distribution which is easy to process; obtaining model distribution through the inverse process of the diffusion probability model; then pass throughSampling the model distribution to obtain the embedded E of the user structure ^ta ；

Variable self-encoder for obtaining user structure embedded E according to neural ordinary differential module T-ODE perceived by time ^ta Is of the initial cascade state of (1)

Reconstructing an initial cascade state;

a second ODE solver for evolving the reconstructed initial cascade state in the probability space to obtain a cascade uncertainty implicit feature Z ^′ _T ：

Wherein f _ξ (. Cndot.) is the ODE function that calculates the derivative.

The information cascade popularity prediction system based on probability diffusion uses a multi-layer perceptron as a cascade popularity prediction model.

Compared with the prior art, the invention has the following beneficial effects:

1. the method is based on a neural ordinary differential equation and a diffusion probability model, can be used for continuous time state modeling and propagation uncertainty modeling of information cascade, and can be used for predicting popularity of the information cascade better.

2. The method is extremely important for understanding the information evolution process in the social network and explaining the cascade epidemic reasons; for example, the method and the device for predicting the forwarding quantity of a certain microblog in a future period can be used for downstream tasks such as marketing design, rumor prediction and the like.

3. The invention provides a neural ordinary differential equation module (T-ODE) of time perception, which considers the influence of time factors in the information propagation process and is used for modeling the time irregularity of cascade events and capturing continuous time dynamics of time perception.

4. The invention provides a cascade uncertain modeling module (DPM-ODE) combining a conditional diffusion probability model and an implicit neural ordinary differential equation; the conditional diffusion probability model reconstructs the information cascade structure characteristics of cascade by taking the continuous time cascade characteristics as the conditions, and on the basis, an implicit ordinary differential equation defines the generation process along with time according to the deterministic evolution of the initial cascade state, and simultaneously acquires the cascade characteristics of propagation uncertainty.

Drawings

FIG. 1 is a schematic diagram of an information cascade depicting an information cascade diffusion process and popularity prediction tasks.

Fig. 2 is a schematic diagram of the information cascade your popularity prediction flow based on probability diffusion in the present invention.

Fig. 3 is a schematic diagram of a noise prediction model.

Interpretation of the terms

Information concatenation (Information Cascade): fig. 1 illustrates this process in one example: after a root node issues an information content, the information content browsed by the attention of the root node is shared or forwarded. The information content propagates through the social network via the forwarding behavior of the user and facilitates the generation of the information cascade. The information cascade popularity prediction task is a classical task, predicts the scale of a cascade (text push, microblog, etc.), and is potentially affected by the user after a period of observation. The theoretical basis can be referred to in the literature [ J.Cheng, L.Adamic, P.A.Dow, J.M.Kleinberg, and J.Leskovic.can cascades be predicted In Proc.of WWW,2014 ]

Neural ordinary differential equation (Neural Ordinary Differential Equations, NODEs): the ordinary differential equation parameterizes the derivatives of hidden states by using neural networks, rather than discrete sequences of hidden layers as used in traditional models (e.g., resNet and RNN), and has a balance between numerical accuracy and computation while saving significant memory costs. The theoretical basis can be referred to in the literature [ R.T.Chen, Y.Rubanova, J.Bettencourt, and D.Duvenaud. Neal ordinary differential equivalents. In NeurIPS,2018, pp.6572-6583 ]

Diffusion probability model (Diffusion Probabilistic Model, DPM): the diffusion probability model can flexibly model complex data through a Markov chain, and gradually converts the data distribution into a distribution which is easy to process by adding noise through a forward diffusion process. A back diffusion process is then defined to generate data in a generative manner. The theoretical basis can be referred to in the literature [ J.Sohl-Dickstein, E.Weiss, N.Maheswaranathan, and S.Ganguli.deep unsupervised learning using nonnequibrimthermodammals. In ICML,2015 ]

The log function used in the present invention is based on 10.

Detailed Description

The invention will be further described with reference to the accompanying drawings.

Example 1

As shown in fig. 2, the information cascade popularity prediction method based on probability diffusion provided in this embodiment includes the following steps:

s1, constructing cascade data into a social graph, a cascade graph and a cascade sequence; then obtaining global structural features according to the structural attributes of the cascade global social graph; obtaining node similarity characteristics according to the cascade diagram; then, splicing the global structural features and the node similarity features to obtain a structural embedded representation of the user;

S2, acquiring the implicit state of the current moment according to the implicit state of the previous moment and the embedded representation of the user structure; then obtaining an ODE implicit state by using a first ODE solver according to the implicit state at the previous moment and the implicit state at the current moment; then, the implicit state and the ODE implicit state at the current moment are cascaded to obtain a cascaded implicit state; then obtaining an updated cascade implicit state according to the implicit state at the current moment and the cascade implicit state and outputting cascade implicit characteristics through full connection based on a time-aware gating mechanism;

s3, generating a target cascade structure embedded representation of a user based on the diffusion probability model by taking the cascade implicit characteristic obtained in the step S2 as a condition, and resampling; then, evolution is carried out on the resampling result in a probability space by using a second ODE solver to obtain an implicit characteristic with uncertain cascade;

s4, predicting the cascade popularity according to the cascade implicit characteristic obtained in the step S2 and the cascade uncertain implicit characteristic splicing result.

The above steps S1 to S4 are explained in detail below.

Step S1 is that cascade data are built into a social graph

Cascading diagram->

And a cascading sequence. The social graph->

Mainly consisting of all subscribers to the cascading sequence. The cascade diagram- >

Mainly consists of any one of the subscribers of the cascade sequence. Then learn the cascaded global social graph using sparse matrix decomposition (sparsematrixfactor)>

Obtaining global structural feature E _g (see C.Donnat, M.Zitnik, D.Hallac, and J.Leskovic, "Learning structural node embeddings via diffusion wavelets," in SIGKDD,2018, pp.1320-132); modeling a cascade Graph using Graph Wavelets model>

Obtaining node similarity features E _c (see J.Zhang, Y.Dong, Y.Wang, J.Tang, and M.Ding, "ProNE: fast and scalable network representation learning," in IJCAI, macao, china, aug.10-16,2019, pp.4278-4284); finally to E _g And E is _c And performing splicing operation to obtain the structural representation E of the user.

Step S2 is to acquire the implicit state of the current time by using the LSTM unit according to the implicit state of the previous time and the embedded representation of the user structure. Given at time t _i-1 Is a concatenation of implicit state pairs (c) _i-1 ,h _i-1 ) And user u's structure embedding

First they are input into LSTM unit to generate newImplicit state (c) _i ,h′ _i ):

Wherein θ _l Representing model parameters that can be learned in LSTM cells.

Then we will h _i-1 Inputting the data to a first ODE solver based on Euler algorithm to obtain an ODE implicit state z _i ：

z _i ＝ODESolver(f _ω ,h _i-1 ,h′ _i ,(t _i-1 ,t _i ))

Updating the cascade implicit state by using GRU unit, i.e. cascade the implicit state at the current time and ODE implicit state to obtain cascade implicit state h _i . The above process is expressed as:

h″ _i ＝GRUCell(θ _g ,h′ _i ,z _i )

wherein,,

is the ODE from time t _i-1 By time t _i Is a solution of h _i Is the cascade implicit state after GRU unit update, θ _g Representing model parameters that can be learned in the GRU units.

Given an implicit state h' _i And h' _i Updating concatenated implicit state h using time-aware gating mechanisms _i ：

h _i ＝ν _i ⊙h″ _i +(1-ν _i )⊙h′ _i ,

Wherein,,

representing time gating.

Finally, we calculate the output state o through a full link layer ₁ …o _n }. Wherein,,

and n represents the early stage of the modelingNumber of users. Typically, we use the last output state o _n Representing a cascade implicit feature Z.

The step S3 comprises the following sub-steps:

s31 given information cascade diagram

And social graph->

And structural feature E, obtaining an initial data distribution of q (E ₀ )，E ₀ ＝E；

S32 based on the diffusion probability model, the forward process gradually adds noise to the data to reduce the prior q (E ₀ ) Converting into Gaussian distribution which is easy to process; obtaining model distribution through the inverse process of the diffusion probability model; obtaining user structure embedded E by model distributed sampling ^ta ；

Based on the diffusion probability model, the forward process gradually adds a priori q (E ₀ ) Converting into Gaussian distribution which is easy to process; the forward process of each time step R e {0,1,2, …, R } is defined as a gaussian change:

wherein beta is _r Controlling a process of adding gaussian noise to data for a fixed constant;

the gaussian distribution is represented by the formula,

mean, beta _r I represents the variance.

Inverse process simulation conditional distribution using conditional diffusion probability model

Defining a conditional denoising function E _θ :(E ^ta |Z ^co )→E ^ta It will Z ^co As input:

in the method, in the process of the invention,

mean value of gaussian distribution; />

Representing the variance of the gaussian distribution.

We set alpha _r ：＝1-β _r And

distribution q (E) _r |E ₀ ) Expressed as:

the goal of (2) is to eliminate gaussian noise added during diffusion. By minimizing the variational negative log-likelihood, the parameter θ can be fitted to the data distribution q (E ₀ )：

The above formula can be simplified and effectively trained by random gradient descent:

wherein the noise

∈ _θ Is the predicted output of the noise prediction model. The noise prediction model is shown in fig. 3 and includes an input layer, an attention layer, and an output layer. The input layer consists of a first convolution layer, a time step encoder, a first multi-layer perceptron (MLP) and a second convolution layer in parallel; the first convolution layer and the second convolution layer are both 1×1 convolution layers; the input of the first convolution is the user structural representation +. >

I.e. < ->

The time-step encoder input is a time-step r, which is used to learn the parameter matrix W from the time-step r _r The corresponding row of the middle index is taken as output; the input of the first multi-layer perceptron is the cascade implicit characteristic Z obtained in the step S2; the outputs of the first convolution layer, the time step encoder and the first multi-layer perceptron are spliced in dimension to obtain a result which is used as the input of the second convolution layer; the attention layer is output and +_in a second convolution layer using a conventional structure (see Vaswani, ashish, et al, "Attention is all you need." InNIPS, 2017)>

The sum is used as the input of the attention layer, and the output of the attention layer is used as the input of the output layer; taking the second multi-layer perceptron as an output layer, wherein the output E of the second multi-layer perceptron _θ 。

The sampling process may be defined as:

wherein,,

and->

By sampling from r to 0 we obtain a sample E ^ta For modeling the evolution uncertainty described below.

S33 embedding E according to the user structure ^ta Acquiring an initial cascade state

The initial concatenation state is then reconstructed from the encoder based on the variance.

Embedding E according to user structure ^ta According to the step S2 given above, the initial cascade state is obtained by inverse solution

The extrapolation property of the ODE is then utilized to model the cascading evolution dynamics with uncertainty based on the variational self-encoder (VariationalAutoencoders, VAEs) framework. Assume that

Is a true posterior distribution, we use model distribution based on the VAE framework +.>

To approximate the true posterior distribution->

Wherein phi is a parameter of the neural network, by using a linear transformation from +.>

Derived mean->

Sum of variances->

Resampling, combined with a repartitioning technique, results in a reconstructed initial cascade state +.>

Wherein ζ is extracted from the distribution of the n-TaiAnd (5) sampling.

S34 an initial cascade state Z 'reconstructed in probability space using a second ODE solver' ₀ Evolution is carried out to obtain implicit characteristic Z 'with uncertain cascade' _T ：

Wherein f _ξ (. Cndot.) is the ODE function of calculating the derivative, t _n Indicating the last observation instant of the concatenated sequence. In this way we have generated a continuous evolutionary trajectory in which each point represents an implicit variable p (Z 'following a posterior distribution' _t |Z′ ₀ ,…,Z′ _t-1 ). Finally, we optimize model parameters by maximizing the lower bound of Evidence (ELBO):

in the method, in the process of the invention,

expressed as +.>

Is the desire under conditions; />

Represents the KL divergence Kullback-Leibler divergence.

Step S4, namely, the cascade implicit characteristic Z obtained in step S2 and the cascade uncertain implicit characteristic Z obtained in step S3 are combined ^′ _T Performing a stitching operation, and then inputting them into a Multi-layer perceptron (Multi-Layer Perceptrons, MLPs) for cascade popularity prediction:

wherein Concat (-) represents the splice operation.

In the training process, we target the Mean Square Log Error (MSLE), combining MSLE and ELBO to train the model. The final loss function is defined as:

where N is the total number of cascades, P _k (t _p ) Representing the number of users of the kth real forwarding cascade,

representing popularity of the kth prediction.

Example 2

The embodiment provides an information cascade popularity prediction system based on probability diffusion, which comprises the following components:

the structure embedding representation acquisition module is used for constructing cascade data into a social graph, a cascade graph and a cascade sequence; then obtaining global structural features according to the structural attributes of the cascade global social graph; obtaining node similarity characteristics according to the cascade diagram; then, splicing the global structural features and the node similarity features to obtain a structural embedded representation of the user;

time-aware neural ordinary differential module T-ODE: the implicit state acquisition module is used for acquiring the implicit state of the current moment according to the implicit state of the previous moment and the embedded representation of the user structure; then obtaining an ODE implicit state by using a first ODE solver according to the implicit state at the previous moment and the implicit state at the current moment; then, the implicit state and the ODE implicit state at the current moment are cascaded to obtain a cascaded implicit state; then obtaining an updated cascade implicit state according to the implicit state at the current moment and the cascade implicit state and outputting cascade implicit characteristics through full connection based on a time-aware gating mechanism;

Cascaded uncertainty modeling module DPM-ODE: generating a target cascade structure embedded representation of a user based on a diffusion probability model by taking cascade implicit characteristics obtained by a time-aware neural ordinary differential module T-ODE as conditions, and resampling; then, evolution is carried out on the resampling result in a probability space by using a second ODE solver to obtain an implicit characteristic with uncertain cascade;

and the cascade popularity prediction model is used for predicting cascade popularity according to the cascade implicit characteristics and the cascade uncertain implicit characteristic splicing result.

The neural ordinary differential module T-ODE of the time perception comprises:

the LSTM unit is used for acquiring the implicit state of the current moment according to the implicit state of the previous moment and the embedded representation of the user structure;

the first ODE solver obtains an ODE implicit state according to the implicit state at the previous moment and the implicit state at the current moment;

the GRU unit is used for cascading the implicit state at the current moment and the implicit state of the ODE to obtain a cascading implicit state;

the updating unit is used for updating the cascade implicit state based on a time-aware gating mechanism;

and the full connection layer is used for carrying out full connection processing on the splicing result of the global structural features and the node similarity features to obtain the structural embedded representation of the user.

The cascade uncertainty modeling module DPM-ODE comprises:

an initial data distribution acquisition unit for giving an information cascade graph

And social graph->

And structural feature E, obtaining an initial data distribution of q (E ₀ )，E ₀ ＝E；

Uncertainty-based user structure embedding generation unit, forward process gradually adds a priori q (E ₀ ) Converting into Gaussian distribution which is easy to process; obtaining model distribution through the inverse process of the diffusion probability model; obtaining the embedded E of the user structure by distributed sampling of the model ^ta ；

Variable self-encoder for obtaining user structure embedded E according to neural ordinary differential module T-ODE perceived by time ^ta Is of the initial cascade state of (1)

Reconstructing an initial cascade state;

a second ODE solver for evolving the reconstructed initial cascade state in the probability space to obtain a cascade uncertainty implicit feature Z ^′ _T ：

Wherein f _ξ (. Cndot.) is the ODE function that calculates the derivative.

The cascade popularity prediction model is a multi-layer perceptron.

The training process of the information cascade popularity prediction system based on probability diffusion is as follows: training the information cascade popularity prediction system based on probability diffusion according to the steps S1-S4 by using training data and according to the loss function

Acquiring a loss value, and optimizing system network parameters through random gradient descent (SGD); repeating the above process until the loss value becomes stable.

After the training of the information cascade popularity prediction system based on probability diffusion is completed, inputting the known information cascade into the system, and obtaining the prediction result of the information cascade popularity according to the steps S1-S4 given above.

Application example

The predictive effect of the probability diffusion based information cascade popularity prediction system (CasDO) provided by the examples on three different real datasets (Twitter, weibo, and APS), a first dataset source reference [ L.Weng, F.Menczer, and y. -y.ahn.visual prediction and communitystructure in social networks.scientific Reports, vol.3, no.1, pp.1-6,2013 ], a second dataset source reference [ Q.Cao, H.Shen, K.Cen, W.Ouyang, and x.cheng.deep hawkes: bridging the gap between prediction and understanding of information cascades.in CIKM,2017 ], a third dataset source reference [ https:// journ aps.org/dataseconds ]. The data ratio of the training set to the sample in the test set is 7:1.5:1.5.

Meanwhile, the information cascade popularity prediction system (CasDO) based on probability diffusion provided by the invention is compared with 5 different baseline models (Feature-Deep, deepHawkes, casCN, latentODE, casFlow), MSLE is used as an evaluation index (the smaller the value is, the better the prediction effect is), and the prediction result is shown in a table 1.

Table 1: effect of popularity predictions on application data sets

The rest of the methods in the table are described as follows:

Feature-Deep: and extracting structural features and time features from the information cascade data, and inputting the structural features and the time features into a two-layer multi-layer perceptron for prediction. [ X.Xu, F.Zhou, K.Zhang, S.Liu, and G.Trajcevski.CasFlow: exploring hierarchical structures and propagation uncertainty for cascade prediction.InTKDE, pp.1-14,2021 ]

Deepfhawkes: the method integrates the deep neural network into the popularity prediction of the point-in process, and considers three main aspects of the Hox process, namely the influence of users, the self-excitation mechanism and the time attenuation. [ Q.Cao, H.Shen, K.Cen, W.Ouyang, and X.Cheng. DeepHawkes: bridging the gap between prediction and understanding of information cascades. In CIKM,2017 ]

CasCN: the method combines a recurrent neural network and a graph convolution network, and utilizes time and structure information to carry out cascading prediction. It captures the evolution process by sampling the sub-cascade diagram and using LSTM. [ X.Chen, F.Zhou, K.Zhang, G.Trajcevski, T.Zhong, and F.zhang. Information diffusion prediction via recurrent cascades con-version. InICDE, 2019 ]

LatentODE: it extends the discrete RNN to continuous time concealment dynamics defined by the ODE. It treats the potential representation as a time series variable in the RNN, being able to handle any time interval between observations. [ Y.Rubanova, R.T.Chen, and D.Duvenaud.Latent odes for irregularly sampled time series.In NeuroIPS, 2019 ]

CasFlow: it learns the local and global structures in the information cascade and uses the variation self-encoder and regularized streams to enhance the learned cascade representation. [ X.Xu, F.Zhou, K.Zhang, S.Liu, and G.Trajcevski.CasFlow: exploring hierarchical structures and propagation uncertainty for cascade prediction.InTKDE, pp.1-14,2021 ]

As can be seen from the experimental results in Table 1, the probability diffusion-based information cascade popularity prediction system (CasDO) provided by the invention can greatly improve the accuracy of popularity prediction compared with other baseline models.

Therefore, the information cascading popularity prediction system based on probability diffusion can model the time irregularity of cascading events and the uncertainty of information propagation in the information propagation process, and can also improve the accuracy of popularity prediction. Experiments on three real data sets demonstrate the superior performance of the present invention over the most advanced baseline model. The improvement of the performance of the proposal shows that the advantages of the neural ordinary differential equation are combined with the diffusion probability model, so that the information cascade diffusion process can be effectively simulated, and the information cascade popularity can be predicted more accurately.

In summary, the invention relates to a neural ordinary differential equation and a diffusion probability model, designs time irregularity of a time-aware neural ordinary differential equation modeling cascade event, and better simulates an information propagation process in the real world by promoting discrete cascade states of an information cascade to continuous time dynamics. The diffusion probability model and the implicit ordinary differential equation are then fused to model the uncertainty associated with the information cascade. The diffusion probability model learns uncertainty of user space association by reconstructing the structural features of the cascaded graph, and on the basis, an implicit ordinary differential equation is utilized to define a generating process along with time according to initial cascade state deterministic evolution, so that cascade features of propagation uncertainty are obtained. Finally, the aim of predicting the popularity of the information cascade is achieved by utilizing the continuous time characteristic and the propagation uncertainty characteristic of the cascade.

Those of ordinary skill in the art will recognize that the embodiments described herein are for the purpose of aiding the reader in understanding the principles of the present invention and should be understood that the scope of the invention is not limited to such specific statements and embodiments. Those of ordinary skill in the art can make various other specific modifications and combinations from the teachings of the present disclosure without departing from the spirit thereof, and such modifications and combinations remain within the scope of the present disclosure.

Claims (10)

1. The information cascade popularity prediction method based on probability diffusion is characterized by comprising the following steps of:

s1, constructing cascade data into a social graph, a cascade graph and a cascade sequence; then obtaining global structural features according to the structural attributes of the cascade global social graph; obtaining node similarity characteristics according to the cascade diagram; then, splicing the global structural features and the node similarity features to obtain a structural embedded representation of the user;

s2, acquiring the implicit state of the current moment according to the implicit state of the previous moment and the embedded representation of the user structure; then obtaining an ODE implicit state by using a first ODE solver according to the implicit state at the previous moment and the implicit state at the current moment; then, the implicit state and the ODE implicit state at the current moment are cascaded to obtain a cascaded implicit state; then obtaining an updated cascade implicit state according to the implicit state at the current moment and the cascade implicit state and outputting cascade implicit characteristics through full connection based on a time-aware gating mechanism;

S3, generating a target cascade structure embedded representation of a user based on the diffusion probability model by taking the cascade implicit characteristic obtained in the step S2 as a condition, and resampling; then, evolution is carried out on the resampling result in a probability space by using a second ODE solver to obtain an implicit characteristic with uncertain cascade;

s4, predicting the cascade popularity according to the cascade implicit characteristic obtained in the step S2 and the cascade uncertain implicit characteristic splicing result.

2. The method for predicting popularity of information cascade based on probability diffusion as claimed in claim 1, wherein in step S1, cascade data is constructed as social graph

Cascading diagram->

A cascading sequence; then learn the cascade global social graph +.>

Obtaining global structural feature E _g The method comprises the steps of carrying out a first treatment on the surface of the Modeling a cascade graph using a graph wavelet model +.>

Obtaining node similarity features E _c The method comprises the steps of carrying out a first treatment on the surface of the Finally to E _g And E is _c And performing splicing operation to obtain the structural representation E of the user.

3. The method for predicting the popularity of the information cascade based on the probability diffusion according to claim 1, wherein in the step S2, the implicit state of the current time is obtained by using the LSTM unit according to the implicit state of the previous time and the embedded representation of the user structure.

4. The method for predicting popularity of information cascade based on probability diffusion according to claim 1, wherein in step S2, the implicit state of the current time and the implicit state of ODE are cascaded by using a GRU unit to obtain the implicit state of cascade,

h″ _i ＝GRUCell(θ _g ,h′ _i ,z _i ),

wherein,,

is the ODE from time t _i-1 By time t _i Is a solution of h _i Is the cascade implicit state after GRU unit update, θ _g Representing model parameters that are learnable in the GRU unit;

given an implicit state h' _i And h' _i Updating concatenated implicit state h using time-aware gating mechanisms _i ：

h _i ＝ν _i ⊙h″ _i +(1-ν _i )⊙h′ _i ，

Wherein,,

representing time gating.

5. The method for predicting popularity of a cascade of information based on probability diffusion of claim 1, wherein step S3 comprises the sub-steps of:

s31 given information cascade diagram

And social graph->

And structural feature E, obtaining an initial data distribution of q (E ₀ )，E ₀ ＝E；

S32 based on the diffusion probability model, the forward process gradually adds noise to the data to reduce the prior q (E ₀ ) Converting into Gaussian distribution which is easy to process; obtaining model distribution through the inverse process of the diffusion probability model; obtaining user structure embedded E by model distributed sampling ^ta ；

S33 embedding E according to the user structure ^ta Acquiring an initial cascade state

Then reconstructing an initial cascade state based on the variation self-encoder;

s34 makingEvolution of the reconstructed initial cascade state in the probability space by using a second ODE solver to obtain an implicit characteristic Z 'with uncertain cascade' _T ：

Wherein f _ξ (. Cndot.) is the ODE function that calculates the derivative.

6. The information cascade popularity prediction method based on probability diffusion according to claim 1, wherein in step S4, cascade popularity is obtained by using a multi-layer perceptron according to the concatenation result of the cascade implicit feature and the cascade uncertain implicit feature obtained in step S2.

7. An information cascade popularity prediction system based on probability diffusion, comprising:

the structure embedding representation acquisition module is used for constructing cascade data into a social graph, a cascade graph and a cascade sequence; then obtaining global structural features according to the structural attributes of the cascade global social graph; obtaining node similarity characteristics according to the cascade diagram; then, splicing the global structural features and the node similarity features to obtain a structural embedded representation of the user;

time-aware neural ordinary differential module T-ODE: the implicit state acquisition module is used for acquiring the implicit state of the current moment according to the implicit state of the previous moment and the embedded representation of the user structure; then obtaining an ODE implicit state by using a first ODE solver according to the implicit state at the previous moment and the implicit state at the current moment; then, the implicit state and the ODE implicit state at the current moment are cascaded to obtain a cascaded implicit state; then obtaining an updated cascade implicit state according to the implicit state at the current moment and the cascade implicit state and outputting cascade implicit characteristics through full connection based on a time-aware gating mechanism;

Cascaded uncertainty modeling module DPM-ODE: generating a target cascade structure embedded representation of a user based on a diffusion probability model by taking cascade implicit characteristics obtained by a time-aware neural ordinary differential module T-ODE as conditions, and resampling; then, evolution is carried out on the resampling result in a probability space by using a second ODE solver to obtain an implicit characteristic with uncertain cascade;

and the cascade popularity prediction model is used for predicting cascade popularity according to the cascade implicit characteristics and the cascade uncertain implicit characteristic splicing result.

8. The information cascade popularity prediction system based on probability diffusion of claim 7, wherein the time-aware neural ordinary differential module T-ODE comprises:

the LSTM unit is used for acquiring the implicit state of the current moment according to the implicit state of the previous moment and the embedded representation of the user structure;

the first ODE solver obtains an ODE implicit state according to the implicit state at the previous moment and the implicit state at the current moment;

the GRU unit is used for cascading the implicit state at the current moment and the implicit state of the ODE to obtain a cascading implicit state;

the updating unit is used for updating the cascade implicit state based on a time-aware gating mechanism;

And the full connection layer is used for carrying out full connection processing on the splicing result of the global structural features and the node similarity features to obtain the structural embedded representation of the user.

9. The information cascade popularity prediction system of claim 7, wherein the cascade uncertainty modeling module DPM-ODE comprises:

an initial data distribution acquisition unit for giving an information cascade graph

And social graph->

And structural feature E, obtaining an initial data distribution of q (E ₀ )，E ₀ ＝E；

Uncertainty-based user structure embedding generation unit, forward process gradually adds a priori q (E ₀ ) Converting into Gaussian distribution which is easy to process; obtaining model distribution through the inverse process of the diffusion probability model; obtaining the embedded E of the user structure by distributed sampling of the model ^ta ；

Variable self-encoder for obtaining user structure embedded E according to neural ordinary differential module T-ODE perceived by time ^ta Is of the initial cascade state of (1)

Reconstructing an initial cascade state;

a second ODE solver for evolving the reconstructed initial cascade state in the probability space to obtain a cascade uncertainty implicit feature Z' _T ：

Wherein f _ξ (. Cndot.) is the ODE function that calculates the derivative.

10. The information cascade popularity prediction system based on probability diffusion according to claim 7, wherein a multi-layer perceptron is used as a cascade popularity prediction model.

Images