CN115099886B - Long-short interest sequence recommendation method, device and storage medium - Google Patents
Long-short interest sequence recommendation method, device and storage medium Download PDFInfo
- Publication number
- CN115099886B CN115099886B CN202210575237.7A CN202210575237A CN115099886B CN 115099886 B CN115099886 B CN 115099886B CN 202210575237 A CN202210575237 A CN 202210575237A CN 115099886 B CN115099886 B CN 115099886B
- Authority
- CN
- China
- Prior art keywords
- user
- commodity
- embedded
- vector
- interest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000003860 storage Methods 0.000 title claims abstract description 13
- 239000013598 vector Substances 0.000 claims abstract description 318
- 230000003993 interaction Effects 0.000 claims abstract description 62
- 230000006399 behavior Effects 0.000 claims abstract description 52
- 239000002775 capsule Substances 0.000 claims abstract description 24
- 230000007246 mechanism Effects 0.000 claims abstract description 19
- 230000007774 longterm Effects 0.000 claims abstract description 16
- 239000011159 matrix material Substances 0.000 claims description 63
- 230000006870 function Effects 0.000 claims description 38
- 238000012549 training Methods 0.000 claims description 30
- 230000004913 activation Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 17
- 238000013528 artificial neural network Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 230000009471 action Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 238000012163 sequencing technique Methods 0.000 claims description 4
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 4
- 238000005070 sampling Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000012512 characterization method Methods 0.000 description 5
- 230000002776 aggregation Effects 0.000 description 4
- 238000004220 aggregation Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000001914 filtration Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000004931 aggregating effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 235000009854 Cucurbita moschata Nutrition 0.000 description 1
- 240000001980 Cucurbita pepo Species 0.000 description 1
- 235000009852 Cucurbita pepo Nutrition 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 239000012633 leachable Substances 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 235000020354 squash Nutrition 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0631—Item recommendations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
- G06Q30/0202—Market predictions or forecasting for commercial activities
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Economics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Game Theory and Decision Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method, a device and a storage medium for recommending long and short interest sequences, wherein the method comprises the following steps: constructing a graph according to the interaction data of the user and the commodity; drawing and accumulating to extract multi-order information of the commodity and the user, constructing a user-commodity interaction sequence after obtaining initial embedded vectors of the commodity and the user, inputting the sequence into a transducer module, and learning the long-term interesting embedded vectors of the user; acquiring short-time user behavior embedded vectors, inputting the short-time user behavior embedded vectors into a capsule network, and acquiring k short-time interest embedded vectors of a user; the method comprises the steps of obtaining weights of a single commodity embedded vector and each user interest embedded vector through an attention mechanism module by k short-time interest embedded vectors, long-time interest embedded vectors and commodity embedded vectors, and then obtaining a final embedded vector of a user through weighting; and calculating the click possibility of interaction between the user embedded vector and the commodity embedded vector, and realizing commodity recommendation. The method and the device can effectively improve the recommendation effect and can be widely applied to the field of sequence recommendation.
Description
Technical Field
The invention relates to the technical field of commodity time sequence recommendation, in particular to a method, a device and a storage medium for recommending long and short interest sequences.
Background
With the progress and development of network technology, mobile phones, computers and other devices are gradually popularized, network shopping becomes a part of daily life of residents, network sales becomes an important component of business income ratio, various large electronic commerce platforms are constantly generating a large amount of user and commodity interaction information data, the data express a large amount of important information in macroscopic and microscopic levels, electronic commerce enterprises can fully utilize the data information research to predict interest and hobbies of customers, better recommend commodities and services for the customers, and better sell goods for the merchants, and fully meet the demands of the merchants and the users; therefore, the research on an effective and lightweight recommendation algorithm by using the interaction data of the user and the commodity is of great significance. When a user purchases a commodity, the relationship between the commodity purchased before and the commodity to be purchased is generally considered, so that the interaction sequence of the user and the commodity contains important information for the purchase of the next commodity; recommending the next commodity they really need to users from among a huge number of commodities becomes extremely important; the time cost of searching the commodity by the user is greatly reduced, and meanwhile, the problems of stock and online new commodity and the like are solved for the merchant.
On the basis of a prediction algorithm of a next commodity based on a sequence of user interaction with the commodity, the existing algorithm generally faces three problems; the first problem is that the traditional collaborative filtering algorithm faces the problem of data sparsity, so that the recommendation effect is poor; the second problem is that the traditional neural network algorithm can express the expression embedded vector of the user by a fixed vector, which brings the problem that hot goods are often recommended when the user is recommending, and the recall capability of the goods in the less public domain is insufficient; a third problem is that more recommended networks do not introduce time information, and in reality the user's token vector will change over time.
Disclosure of Invention
In order to solve at least one of the technical problems existing in the prior art to a certain extent, the invention aims to provide a method, a device and a storage medium for recommending long and short interest sequences.
The technical scheme adopted by the invention is as follows:
a recommendation method of long and short interest sequences comprises the following steps:
Acquiring data comprising an interaction sequence of a user and a commodity;
Constructing a graph of the user and the commodity according to the obtained data, and inputting the constructed graph into a graph neural network to obtain an initial embedding vector of the user and the commodity;
Learning to obtain a user short-time behavior embedded vector according to the initial embedded vector of the commodity;
according to the short-time behavior embedded vectors of the user, learning to obtain K short-time user interest embedded vectors;
Adding the user and commodity interaction sequence and the position embedding vector, and inputting the added sequence and the position embedding vector into a transducer module to obtain a long-term user interest embedding vector;
merging the long-time user interest embedded vector and the initial embedded vector of the user into the K short-time user interest embedded vectors to obtain K+1 user interest embedded vectors;
learning the weight of each interest embedded vector through the attention mechanism between the interest embedded vector of the user and the commodity embedded vector, and constructing a final embedded vector of the user;
and obtaining a commodity prediction result according to the inner product of the commodity embedding vector and the final embedding vector of the user.
Further, the step of constructing a graph of the user and the commodity according to the obtained data, inputting the constructed graph into a graph neural network, and obtaining an initial embedded vector of high-order semantics of the user and the commodity, including:
constructing a graph of the user and the commodity according to the obtained data, and gathering information between the nodes into a central node through multi-layer graph convolution to express an initial embedding vector of the user and the commodity;
wherein, the expression of the initial embedded vector obtained through graph convolution is as follows:
Wherein E (L) represents that after the graph convolution layer I, the obtained commodity and the user are integrated by the embedded vector; w 1 (l) and W 2 (l) are learnable parameter matrices; each node is represented to integrate own information, add an identity matrix I, multiply the identity matrix I with an initialization embedded matrix, and aggregate the neighbor information of the user/commodity; /(I) Indicating that the correlation between the user and the commodity is integrated.
Further, the learning to obtain the user short-time behavior embedded vector according to the initial embedded vector of the commodity comprises the following steps:
The expression of the interaction sequence of the user and the commodity is as follows: representing an interaction sequence of a user and commodities, and sequencing according to interaction time; wherein/> An embedded vector with the article ID of 1, and m is the article ID;
The short-time behavior embedded vector of the user is obtained by means-pooling processing the commodity embedded vector of the interaction sequence of the user and the commodity N is a user ID; m is the commodity ID.
Further, the learning to obtain K short-time user interest embedded vectors according to the short-time user behavior embedded vectors includes:
and inputting the short-time behavior embedded vectors of the users into a capsule network, and approaching a high-level capsule from a low-level capsule through a process of iterating a plurality of dynamic routes to finally obtain K short-time interest embedded vectors of the users, wherein K is a preset super parameter.
Further, the expression of the position embedding vector is as follows:
In the method, in the process of the invention, Information is embedded in the position of the index value even for representing the commodity of the user and the interaction, and the index value is/is evenTimestamp information representing interactions of the item with the user,/>Information is embedded in the positions of the commodity with the index value being odd, l represents the index value of the commodity, and d is the dimension of commodity embedding.
Further, the expression of the K+1 user interest embedding vectors is as follows:
In the method, in the process of the invention, Representing the user k+1 interest embedding matrices, n is the user ID, i is the ID of the specific interest embedding vector,/>Is an interest embedded vector;
the expression of the user's final embedded vector is as follows:
Wherein σ is a softmax nonlinear activation function; i j is a commodity embedded vector, and V i is a user embedded vector; u i is the user-embedded vector that is ultimately obtained by weighted summing the short-time and long-time user-interest embedded vectors, w ij is the weight value of each user-interest embedded vector. Wherein formula (1) is abbreviated, and formula (2) is a detailed calculation process of u i.
Further, the obtaining a commodity predicting result according to the inner product of the commodity embedding vector and the final embedding vector of the user comprises the following steps:
For user u i and positive example merchandise Negative sample commodity/>The output predicted value needs to be subjected to supervision training, and the loss of training is defined as follows:
o＝{(u,i,j)|(u,i)∈R+,(u,j)∈R-}
wherein R + is the observed sample and R - is the unobserved sample; sigma is a nonlinear activation function; theta represents A learnable parameter; /(I)Representing a user interaction skin predictive preference score with the commodity.
Further, the long and short interest sequence recommendation method further comprises the step of constructing an end-to-end model:
constructing a long-term user interest embedded vector by using user and commodity interaction data through a pre-trained graphic neural network part, a user behavior layer part, a multi-interest capsule network part and transformers, and constructing an attention mechanism layer between the multi-interest and commodity embedded vector and a click rate prediction part to form an end-to-end long-short interest sequence recommendation algorithm;
Model parameter learning is performed on the training data set by using a random gradient descent method until the model converges.
The invention adopts another technical scheme that:
A long and short interest sequence recommendation device, comprising:
At least one processor;
At least one memory for storing at least one program;
The at least one program, when executed by the at least one processor, causes the at least one processor to implement the method described above.
The invention adopts another technical scheme that:
A computer readable storage medium, in which a processor executable program is stored, which when executed by a processor is adapted to carry out the method as described above.
The beneficial effects of the invention are as follows: according to the method and the device, a plurality of short-time and long-time interest embedded vectors are generated for the user, so that different interest demands of the user can be better supported, and the recommendation effect is further improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description is made with reference to the accompanying drawings of the embodiments of the present invention or the related technical solutions in the prior art, and it should be understood that the drawings in the following description are only for convenience and clarity of describing some embodiments in the technical solutions of the present invention, and other drawings may be obtained according to these drawings without the need of inventive labor for those skilled in the art.
FIG. 1 is a flowchart illustrating steps of a method for recommending long and short interest sequences according to the present embodiment;
FIG. 2 is a schematic diagram of the pre-training phase of the model in this embodiment;
fig. 3 is a structural diagram of a main model block of the model in the present embodiment.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention. The step numbers in the following embodiments are set for convenience of illustration only, and the order between the steps is not limited in any way, and the execution order of the steps in the embodiments may be adaptively adjusted according to the understanding of those skilled in the art.
In the description of the present invention, it should be understood that references to orientation descriptions such as upper, lower, front, rear, left, right, etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of description of the present invention and to simplify the description, and do not indicate or imply that the apparatus or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be construed as limiting the present invention.
In the description of the present invention, a number means one or more, a number means two or more, and greater than, less than, exceeding, etc. are understood to not include the present number, and above, below, within, etc. are understood to include the present number. The description of the first and second is for the purpose of distinguishing between technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present invention, unless explicitly defined otherwise, terms such as arrangement, installation, connection, etc. should be construed broadly and the specific meaning of the terms in the present invention can be reasonably determined by a person skilled in the art in combination with the specific contents of the technical scheme.
As shown in fig. 1, the embodiment provides a method for recommending long and short interest sequences, which includes firstly preprocessing user and commodity interaction data through a pre-training stage, sampling and constructing an adjacency matrix of the user and the data, extracting commodity and user multi-order information through multiple graph convolution, constructing a user and commodity interaction sequence after initial embedded vectors of the commodity and the user are obtained, inputting the sequence into a transducer module, and learning the user long-term interest embedded vectors. Acquiring a short-time user behavior embedded vector through a sequence extraction layer; inputting the time interest embedded vectors into a capsule network to obtain k short-time interest embedded vectors of a user; the method comprises the steps of obtaining weights of a single commodity embedded vector and each user interest embedded vector through an attention mechanism module by k short-time interest embedded vectors, long-time interest embedded vectors and commodity embedded vectors, and then obtaining a final embedded vector of a user through weighting; calculating the click possibility of interaction between the user embedded vector and the commodity embedded vector, and recommending top k commodity; finally, model training is supervised by an objective function BPRLoss, and network parameters are learned by gradient back propagation until convergence. The method comprises the following steps:
s1, acquiring data comprising an interaction sequence of a user and a commodity.
The user and commodity interaction data comprise user and commodity interaction and user and commodity interaction sequence data; sampling and preprocessing data, filtering invalid data, constructing an adjacency matrix of a user and a commodity by using interactive data, and constructing and obtaining the adjacency matrix of the user and the commodity after standardized processing:
Wherein the method comprises the steps of The adjacent matrix is subjected to standardized processing, n is the number of users, and m is the number of commodities;
Where D is the degree matrix and N u and N i are the set of neighbors for the user and the commodity.
S2, constructing a graph of the user and the commodity according to the obtained data, and inputting the constructed graph into a graph neural network to obtain an initial embedded vector of the user and the commodity.
Referring to fig. 2, in the pre-training stage, a user and a commodity graph are input into a graph neural network, and through multi-layer graph convolution, information between nodes can be better combined to be converged into a central node, and characteristics of the nodes not only comprise own characteristics, but also are directly connected with neighbor characteristics, and are fused with high-order line drawing characteristics, so that initial embedded vectors of the user and the commodity are better expressed.
The expression of the initial embedding vector obtained through the graph convolution is as follows:
wherein, After the layer l is convolved in the representation diagram, the obtained commodity and the user are integrated by the embedded vector; and/> Is a parameter matrix which can be learned; /(I)The method is equivalent to that each node blends own information into the node, adds the identity matrix I, multiplies the identity matrix I by the initialized embedded matrix, and aggregates the neighbor information of the user/commodity; /(I)The correlation between the user and the commodity is fused; and finally, adding the two, and updating the embedded vectors of the user and the commodity.
User embedded vector obtained after convolution of each layer of graph: Commodity embedding vector: /(I)
Splice each layer together:
Commodity (3): i= { I 1,i2,…im, }; wherein i n is the commodity embedded vector at the nth position, n is more than or equal to 1 and less than or equal to m, and m is the total number of the articles;
The user: u= { U 1,u2,…ul, }; wherein u n is the user embedded vector at the nth position, t is more than or equal to 1 and less than or equal to L, and L is the total number of users;
Pretraining stage, for user u i and positive example commodity Negative sample commodity/>The output predicted value needs to be subjected to supervision training, and the loss of training can be defined as follows:
here, o represents: o= { (u, i, j) | (u, i) ∈r +,(u,j)∈R- } where R + is the observed sample and R - is the non-observed sample; sigma is a nonlinear activation function; theta represents Parameters can be learned, and the overfitting problem of the small model is regularized using L 2.
S3, learning to obtain a short-time behavior embedded vector of the user according to the initial embedded vector of the commodity.
And (3) the user and quotient interaction sequence is learned by means of means-pooling or max-pooling and the like to obtain the short-time behavior embedding vector of the user.
Sequence of user interactions with merchandise: representing an interaction sequence of a user and commodities, and sequencing according to interaction time; wherein/> The object ID is an embedded vector of 1, and m is the object ID. The method comprises the steps of making means-pooling on an article embedding vector of a user and commodity interaction sequence to finally obtain a user short-time behavior embedding vector/>N is a user ID; m is the commodity ID.
S4, learning to obtain K short-time user interest embedded vectors according to the short-time behavior embedded vectors of the user.
Learning to obtain a short-time behavior representation embedded vector of a user, and inputting the short-time behavior representation embedded vector into a capsule network as follows: the number of the capsules needed is dynamically determined by the interaction times of the users.
The formula: k 'u＝max(1,min(K,log2(|Iu |)) wherein K' u is the dynamic user interest number, the dynamic user interest number is obtained by comparing 1 with the maximum value of min (K, log 2(|Iu |); wherein I u is the number of user interactions, K is a super parameter set by human.
Learning to obtain a short-time behavior representation embedded vector of a user, and inputting the short-time behavior representation embedded vector into a capsule network as follows:
the learning method of the capsule network is that the short-time interest embedded vector of the user is finally obtained by the process of approaching a high-level capsule from a low-level capsule through the process of iterating a plurality of dynamic routes, and the formula is as follows:
The formula: Wherein b ij is random initialization, b ij～N(0,σ2), and obtaining the weight value of each short-time behavior vector through softamx (b ij);
further, the method comprises the steps of, The embedded vector is the short-time user behavior, and j is the number of user interests;
by mapping the embedded vectors of short-term user behavior to a bilinear map matrix Obtaining a short-time interest embedded vector of the user through weighted summation; wherein/>Embedding a vector/>, for a user's short-term interests after passing through a nonlinear activation function
Further, obtainThen, it is input into a nonlinear activation function: wherein/> Embedding a vector/>, for a user's short-term interests after passing through a nonlinear activation functionNext, embedding vectors by dot multiplying the interest embedded vectors by short-term user behavior; the formula: The degree of correlation of the short-time user behavior embedded vector and the interest embedded vector can be regarded as; finally, adding the initial value b ij to update b ij;
The formula:
and finally obtaining K short-time user interest embedded vectors through iterating the steps for a plurality of times.
S5, adding the user and commodity interaction sequence and the position embedding vector, and inputting the added sequence and the position embedding vector into a transducer module to obtain the long-term user interest embedding vector.
The expression of the position embedding vector is as follows:
Where d is the dimension in which the article is embedded, Where i is the user ID, j is the merchandise ID, and k is the dimension of the location embedding vector.
By utilizing the periodicity of the trigonometric function, semantic information at a specific moment can be captured, and the periodic relationship and relativity between time points can be depicted to generate embedded characterization. The method can introduce different cycle sizes, thereby extracting different periodicity relations in different dimensions. Meanwhile, unlike the position information, the time axis can be extended infinitely, and the representation mode based on the trigonometric function mapping can be adapted to time stamps with any size, so that untrained time in the test process can be processed.
S6, integrating the long-time user interest embedded vector and the initial embedded vector of the user into the K short-time user interest embedded vectors to obtain K+1 user interest embedded vectors.
The long-time user interest embedded vector and the user initial embedded vector are fused into K short-time interest embedded vectors, and K+1 user interest embedded vectors are finally obtained. The expression of the K+1 user interest embedding vectors is as follows:
In the method, in the process of the invention, Representing the user k+1 interest embedding matrices, n is the user ID, i is the ID of the specific interest embedding vector,/>Is an interest embedded vector.
S7, learning the weight of each interest embedded vector through an attention mechanism between the interest embedded vector of the user and the commodity embedded vector, and constructing a final embedded vector of the user.
The expression of the user's final embedded vector is as follows:
Wherein σ is a softmax nonlinear activation function; i j is a commodity embedded vector, and V i is a user embedded vector; u i is the user-embedded vector that is ultimately obtained by weighted summing the short-term and long-term user-interest embedded vectors.
And S8, obtaining a commodity prediction result according to the inner product of the commodity embedding vector and the final embedding vector of the user.
Pretraining stage, for user u i and positive example commodityNegative sample commodity/>The output predicted value needs to be subjected to supervision training, and the loss of training can be defined as follows:
o＝{(u,i,j)|(u,i)∈R+,(u,j)∈R-}
wherein R + is the observed sample and R - is the unobserved sample; sigma is a nonlinear activation function; theta represents Parameters can be learned, and the overfitting problem of the small model is regularized using L 2.
S9, constructing an end-to-end model, and performing parameter learning and updating by using training data.
The method comprises the steps of constructing a long-term user interest embedded vector by user interaction data and commodity interaction data through a pre-trained graphic neural network part, a user behavior layer part, a multi-interest capsule network part and transformers, constructing an attention mechanism layer between the multi-interest and commodity embedded vector and a click rate prediction part, forming an end-to-end long and short interest sequence recommendation algorithm based on an attention mechanism, and performing model parameter learning on a training data set by using a random gradient descent method until a model converges.
The above method is explained in detail below in connection with the following consistent and specific examples.
The embodiment provides a long and short interest sequence recommending method based on an attention mechanism, which is used for constructing a diagram of a user and a commodity and entering a pre-training module by sampling and filtering invalid data; inputting the constructed user and commodity graph into a graph neural network, and obtaining initial embedded vectors of the user and commodity nodes fused with other nodes after multi-layer convolution; splicing the obtained commodity initial embedded vector with a leachable position vector, inputting the commodity initial embedded vector into a user behavior layer, learning to obtain a user short-time behavior embedded vector by means of means-pooling or max-pooling and the like, inputting the commodity initial embedded vector into a capsule network, and learning K short-time user interest embedded vectors through the iterative multiple dynamic routing process; the method comprises the steps of obtaining a global user interest embedded vector by inputting a user and commodity interaction sequence into a transducer module; splicing the global user interest embedded vector and the K short-time interest embedded vectors with the initial user embedded vector to obtain K+1 user interest embedded vectors; the method comprises the steps of constructing a final embedded vector of a user through the weight of each interest embedded vector of the attention mechanism between the interest embedded vector of the user and the commodity embedded representation vector; obtaining a commodity prediction result by utilizing the inner product of the commodity and the user embedded vector; and constructing an end-to-end model, and performing parameter learning and updating by using training data. The method specifically comprises the following steps:
and step 1, data preparation.
The data to be prepared mainly comprises user interaction with the commodity and time information. Downloading taobao, tianmao or movie-1m data sets on the public data set, wherein each sample comprises commodities interacted by a user and the user, and particularly interaction time information; traversing all users, removing the user interaction number with the commodity is less than 30, and filtering invalid data.
And 2, the pre-training model learns initial embedded vectors of the user and the commodity.
Constructing a graph of commodities and users to obtain a key dictionary with adjacent matrix dimensions (6040+3706 ) based on sparse matrixes, and then converting the key dictionary into a li sparse matrix; constructing a sparse matrix-based key dictionary dimension (6040, 3706) according to the number of users and commodities, and inserting the matrix into a value of an adjacent matrix according to an index (6040, 6040:); inserting a transpose of the matrix into a value according to index (6040: 6040) in the adjacency matrix; then, the adjacency matrix is converted into a key dictionary based on a sparse matrix, and the adjacency matrix construction of the user and the commodity is completed.
R epsilon R n×m, an adjacent matrix, n is the number of users and m is the number of commodities;
Constructing an adjacency matrix of the commodity and the user through R; and then standardized to obtain/>
Wherein D is a degree matrix, and N u and N i are sets of adjacent points of the user and the commodity; the degree matrix is dot-product with the adjacency matrix because the nodes with large degree have larger values in their characterization and the nodes with small degree have smaller values, which may lead to gradient extinction or gradient explosion. Finally, normalization of the adjacency matrix is realized.
Further, normalization processing is required for the adjacency matrix; firstly, creating a unit matrix with dimensions (9746 ) and adding the unit matrix with an adjacent matrix, wherein the aggregation characterization of the nodes does not contain own characteristics, the characterization is the characteristic aggregation of adjacent nodes and does not contain own node characteristics, so that the closed loop is added by the unit matrix fusion, which acts as adding the attribute of each node. The specific expression is as follows:
further, inputting the user, the commodity and the constructed adjacency matrix into a pre-training model; first initializing initial embedding vectors of a user (6040, 64) and a commodity (3706, 64), and linear variation matrix dimensions (64, 64) and bias matrices (64, 1) in a graph neural network; the adjacent matrix based on the csr sparse matrix is changed to be sparse tensor, and the dimension is 9746, 9746.
In each training process of the model, firstly, sampling a data sample; there are a variety of ways of sampling; randomly extracting 4060 users from the training set 6040; traversing each user, and selecting a positive sample and a negative sample from the training set; and finally, obtaining three lists of the commodity ids of the positive sample and the negative sample corresponding to the user id after sampling.
Inputting the three list-formed adjacency matrices into a graph neural network model; the adjacency matrix passes through a dropout layer (0.1), and part of nodes are disabled, so that the effect is more beneficial to generalization of a test set, and the influence of overfitting is reduced; the user and commodity initial embedding vectors are concatenated in columns to form an overall initial embedding matrix tensor, dimensions (9746, 64) which are included in a list.
Entering a first layer graph rolling network, carrying out matrix multiplication on adjacent matrixes (9748, 9746) and integrally initialized embedded vector matrixes (9746, 64) to obtain an edge representation embedded matrix, carrying out information aggregation on all neighbors of users and commodities from user to user and commodity to commodity, carrying out feature extraction on the information through a linear change layer, and obtaining a summary neighbor information representation embedded matrix (9746, 64).
Further, embedding neighbor aggregated information into a self node, performing dot multiplication on the integrally initialized embedded vector matrix and the representing embedded matrix for summarizing the neighbor information, so that the neighbor aggregated information is aggregated on a central node, and the transmission and aggregation of commodity to users and commodity information from users are realized, and the dimension (9746, 64); it is activated through the linear change layer and the nonlinearity.
Adding the two parts of user-to-user transmission and commodity-to-user information transmission, and then realizing that a layer of graph roll-up network is put into a list after dropout and regularization treatment, wherein the obtained matrix is still a dimension (9746, 64); after the multiple-time graph convolution, the user and the commodity can extract higher-order information, and the information representing the embedded vector is enriched;
the specific calculation formula can be written as:
Wherein the method comprises the steps of After the layer l is convolved in the representation diagram, the obtained commodity and the user are integrated by the embedded vector; and/> Is a parameter matrix which can be learned; /(I)The method is equivalent to that each node blends own information into the node, adds the identity matrix I, multiplies the identity matrix I by the initialized embedded matrix, and aggregates the neighbor information of the user/commodity; /(I)The correlation between the user and the commodity is fused; and finally, adding the two, and updating the embedded vectors of the user and the commodity.
Finally, splicing the user obtained by the multi-layer graph convolution with the commodity representation embedded matrix to obtain a final integrated final embedded matrix after passing through the multi-layer graph neural network, wherein the dimension is (9746, 256) and the assumption is subjected to 3-layer graph convolution;
User embedded vector obtained after convolution of each layer of graph:
Commodity embedding vector:
Splice each layer together:
Commodity (3): i= { I 1,i2,…im, }
Wherein i n is the commodity embedded vector at the nth position, n is more than or equal to 1 and less than or equal to m, and m is the total number of the articles;
the user: u= { U 1,u2,…ul, }
Wherein u n is the user embedded vector at the nth position, t is more than or equal to 1 and less than or equal to L, and L is the total number of users.
And obtaining an embedding matrix corresponding to the commodity by splitting the integral finalized embedding matrix, extracting the sampled user id and commodity id list, and extracting the embedding vector corresponding to the positive and negative sample commodity of the user.
And respectively carrying out inner product on the embedded vector of the user and the positive and negative sample representation embedded vector, wherein the loss function is as follows:
here, o represents: o= { (u, i, j) | (u, i) ∈r +,(u,j)∈R- } where R + is the observed sample and R - is the non-observed sample; sigma is a nonlinear activation function; theta represents Parameters can be learned, and the overfitting problem of the small model is regularized using L 2.
Finally, supervising the model training through BPRLoss loss functions, and updating parameters through gradient feedback. After the training of the pre-training module is completed, the learned embedded vectors of the user and the commodity representation are input into the main model.
And 3, inputting the initial embedded vectors of the commodities to a user behavior layer, and learning the short-time user behavior embedded vectors.
Randomly extracting 1000 users in a training set to obtain a user id sequence (1000), a user and commodity purchasing sequence (1000, 50), a user and positive sample sequence (1000, 1), and a user and negative sample sequence (1000, 10); the above data are all expanded in one dimension.
Commodity (3): i= { I 1,i2,…im, }
Wherein i n is the commodity embedded vector at the nth position, n is more than or equal to 1 and less than or equal to m, and m is the total number of the articles;
the user: u= { U 1,u2,…ul, }
Wherein u n is the user embedded vector at the nth position, t is more than or equal to 1 and less than or equal to L, and L is the total number of users.
Further, inputting a user and commodity interaction sequence and a user purchasing commodity interaction quantity sequence into a user behavior layer; the sequence of each user is used as a sequence through 50 sliding windows, n sequences of interaction between the user and different commodities are obtained, and commodity representation embedded vectors of the user are aggregated into n short-time user behavior representation embedded vectors in a mean-pooling mode; the action layer is used for averaging adjacent 50 commodity embedded vectors of a user to obtain short-time action embedded vectors of the user, and n short-time action embedded vectors of the user are obtained through the action of a sliding window.
Sequence of user interactions with merchandise: representing an interaction sequence of a user and commodities, and sequencing according to interaction time; wherein/> The object ID is an embedded vector of 1, and m is the object ID. The method comprises the steps of making means-pooling on an article embedding vector of a user and commodity interaction sequence to finally obtain a user short-time behavior embedding vector/>N is a user ID; m is the commodity ID.
And 4, inputting the short-time user behavior embedded vector into a capsule network, and learning the short-time user interest embedded vector.
After n short-time user behavior embedding vectors of a user are obtained, the n short-time user behavior embedding vectors are input into a capsule network; the capsule network has the function of finally aggregating n input vectors into one vector for output, and has the function of clustering.
Specifically: in the model, the capsule network is used for embedding a plurality of short-time user behaviors into vectors through the process of iterating a plurality of dynamic routes, and k short-time user interest embedded vectors are extracted.
The formula: k 'u＝max(1,min(K,log2(|Iu |)) where K' u is the number of dynamic user interests, obtained by taking the maximum of 1 and min (K, log 2(|Iu |)); wherein I u is the number of user interactions, K is a super parameter set by human.
The learning method of the capsule network is that the short-time interest embedded vector of the user is finally obtained by the process of approaching the low-level capsule to the high-level capsule through the process of iterating multiple dynamic routes, and the formula is as follows:
The formula: Where b ij is random initialization, b ij～N(0,σ2), and obtaining the weight value of each short-time behavior vector through softamx (b ij).
Further, the method comprises the steps of,The embedded vector is the short-time user behavior, and j is the number of user interests; /(I)
By mapping the embedded vectors of short-term user behavior to a bilinear map matrixObtaining a short-time interest embedded vector of the user through weighted summation; wherein/>Embedding a vector/>, for a user's short-term interests after passing through a nonlinear activation function
Further, obtainThen, it is input into a nonlinear activation function: /(I)Wherein/>Embedding a vector/>, for a user's short-term interests after passing through a nonlinear activation functionNext, embedding vectors by dot multiplying the interest embedded vectors by short-term user behavior; the formula: /(I)The degree of correlation of the short-time user behavior embedded vector and the interest embedded vector can be regarded as; finally, adding the initial value b ij to update b ij;
The formula:
and finally obtaining K short-time user interest representation embedded vectors through iterating the steps for a plurality of times.
Dynamic routing process: firstly initializing b ij, obtaining the weight of each short-time user behavior embedded vector through a softmax (b ij) sequence, obtaining the vector after aggregating a plurality of behavior embedded vectors through weighted summation, entering a squash nonlinear activation function, finally obtaining a new embedded vector, and finally adding the new embedded vector with the previous short-time user behavior embedded vector to obtain a new bij value; through iteration for many times, n user behavior embedded vectors can be finally aggregated into k short-time user interest embedded vectors with user behavior characteristics extracted.
And 5, the sequence of the interaction between the user and the commodity is used for learning the long-term user interest embedded vector through a transformation module.
Specifically, a sequence of user interaction with commodities is extracted through a transformation module to obtain a long-term interest embedded vector of the user, wherein the long-term interest embedded vector is used as the global interest of the user; the main steps include 2 steps.
Referring to fig. 3, firstly, a position embedding layer is reflected, and the layer is used for giving position information to commodity embedding vectors of interactions between a user and commodities, so that time information can be better reserved in a transducer module, and long-term information and short-term information can be reserved when commodity features are extracted from the transducer module; giving each commodity embedded vector time information; the specific method comprises the following steps:
The formula:
Where d is the dimension in which the article is embedded, Where i is the user ID, j is the merchandise ID, and k is the dimension of the location embedding vector;
By utilizing the periodicity of the trigonometric function, semantic information at a specific moment can be captured, and the periodic relationship and relativity between time points can be depicted to generate embedded characterization. The scheme can introduce different period sizes, thereby extracting different periodicity relations in different dimensions. Meanwhile, unlike the position information, the time axis can be extended infinitely, and the representation mode based on the trigonometric function mapping can be adapted to time stamps with any size, so that untrained time in the test process can be processed.
Next d is the dimension of the word embedding,The range of the representation is greater than [ 1,1 ]; manipulating position codes using different functions in different dimensions, the representation space of high latitudes is more meaningful, so each dimension of different position codes gives a different/>This allows for each dimension to contain certain location information, with the coding being different from location to location. /(I)
And then, the representation embedded vector of the commodity after being integrated with the time information is input into a multi-head attention mechanism module, and the module can learn the relationship between commodities well. The embedded vector of one commodity is obtained by weighting and summing other commodities through the form of the attention mechanism of query, key and value.
And 6, combining the long-term and short-term user interest embedded vectors by using an attention mechanism.
Wherein/>An interest embedding vector representing all users, n is the user ID, i is the ID of the specific interest embedding vector,/>Is an interest embedded vector.
An attention mechanism layer; the k short-time user interest embedded vectors and a long-time user interest embedded vector are weighted and summed together through an attention mechanism to obtain a final user embedded vector;
Wherein the method comprises the steps of Respectively three globally shared trainable parameter matrices, Wherein d model is the dimension of the commodity embedding vector and k is the dimension of the parameter matrix;
After three vectors of a query vector, a key vector and a value of each commodity embedded vector are obtained, the query vector is used for matching each key vector, the query vector and the key vector are subjected to dot multiplication, the correlation weight of the query vector and the key vector is calculated, the query vector and the value vector are subjected to dot multiplication, the correlation weight is obtained, and a Softmax activation function is used for obtaining a corresponding weight value; obtaining each commodity embedded vector by weighting, summing and updating; finally, each head is connected with the W o parameter matrix in a butt joint mode through a plurality of heads,
For k+1 user interest embedded vectors, our final goal is to synthesize the last user embedded vector; therefore, the k+1 user interest embedded vector passes through a layer Relu nonlinear activation function, so that the values are all larger than 0, gradient propagation disappearance is prevented, and gradient transfer is facilitated.
Inputting k+1 user interest embedded vectors into an attention mechanism layer, and taking the k+1 user interest embedded vectors as keys; then obtaining the embedded vector of the commodity according to the id value of the positive sample individual item sampled randomly; the negative sample also obtains the commodity embedding vector of the negative sample.
The intrinsic mechanism is that firstly, a commodity embedded vector is used as a query, and is subjected to inner product with each interest embedded vector, then a probability value is obtained through Softmax, and then the embedded vector of a user is obtained through probability value weighted summation.
Finally, the last output is used as a long-time interest embedded vector of the user, and the long-time interest embedded vector is added into a short-time user interest representation embedded vector; and the initial embedded vectors of the users in the pre-training stage are added together, so that the embedded vectors of the short-time user interests are enriched.
Normalizing the attention value by using a Softmax function to obtain the attention probability distribution of each interest embedded vector:
The formula:
The formula: />
wherein, For the attention of each region, w ij is the vector of the last attention of each region, and its value is equal toEqual in size, w ij has a sum of 1 for each element;
Wherein σ is a softmax nonlinear activation function; i j is a commodity embedded vector, and V i is a user embedded vector; u i is the user-embedded vector that is ultimately obtained by weighted summing the short-term and long-term user-interest embedded vectors.
And 7, predicting the click rate by a fraction predicting layer.
Obtaining a predicted value by utilizing a mode that the commodity represents an embedded vector and the user represents the embedded vector to do dot product;
The formula:
The formula: BPRLoss:
Here, o represents: loss is a Loss value, o= { (u, i, j) | (u, i) ∈r +,(u,j)∈R- } where R + is the observed sample and R - is the non-observed sample; sigmoid is a nonlinear activation function; theta represents Parameters can be learned, and the overfitting problem of the small model is regularized using L 2.
And 8, training a model.
Constructing an end-to-end model, and performing parameter learning and updating by using training data, wherein the method specifically comprises the following steps:
Firstly, preprocessing user and commodity interaction data through a pre-training stage, sampling to construct an adjacent matrix of the user and the data, convolving and extracting commodity and user multi-order information by using a plurality of graphs, constructing a user and commodity interaction sequence after initial embedded vectors of the commodity and the user are obtained, inputting the sequence into a transducer module, and expressing the embedded vectors by long-term global interests of the user at a learning position; the second step is to obtain a short-time user behavior embedded vector through a sequence extraction layer; inputting the interest expression embedded vectors into a capsule network to obtain k interest expression embedded vectors of a user; the method comprises the steps of obtaining weights of a single commodity embedded vector and each user interest embedded vector through an attention mechanism module by using k short-time interest embedded vectors, a global interest embedded vector and commodity embedded vectors, and then obtaining a final embedded vector of a user through weighting; calculating the interaction probability of the user embedded vector and the commodity embedded vector, and recommending top k commodity; finally, model training is supervised by an objective function BPRLoss, and network parameters are learned by gradient back propagation until convergence.
The embodiment also provides a device for recommending long and short interest sequences, which comprises:
At least one processor;
At least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement the method illustrated in fig. 1.
The long and short interest sequence recommending device of the embodiment can execute the long and short interest sequence recommending method provided by the embodiment of the method, can execute the implementation steps of any combination of the embodiment of the method, and has the corresponding functions and beneficial effects of the method.
Embodiments of the present application also disclose a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions may be read from a computer-readable storage medium by a processor of a computer device, and executed by the processor, to cause the computer device to perform the method shown in fig. 1.
The embodiment also provides a storage medium which stores instructions or programs capable of executing the long and short interest sequence recommending method provided by the embodiment of the method, and when the instructions or programs are run, any combination of the embodiments of the executable method can implement steps, so that the method has corresponding functions and beneficial effects.
In some alternative embodiments, the functions/acts noted in the block diagrams may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Furthermore, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed, and in which sub-operations described as part of a larger operation are performed independently.
Furthermore, while the invention is described in the context of functional modules, it should be appreciated that, unless otherwise indicated, one or more of the described functions and/or features may be integrated in a single physical device and/or software module or one or more functions and/or features may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary to an understanding of the present invention. Rather, the actual implementation of the various functional modules in the apparatus disclosed herein will be apparent to those skilled in the art from consideration of their attributes, functions and internal relationships. Accordingly, one of ordinary skill in the art can implement the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are merely illustrative and are not intended to be limiting upon the scope of the invention, which is to be defined in the appended claims and their full scope of equivalents.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a read-only memory (ROM), a random access memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the foregoing description of the present specification, reference has been made to the terms "one embodiment/example", "another embodiment/example", "certain embodiments/examples", and the like, means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
While the preferred embodiment of the present application has been described in detail, the present application is not limited to the above embodiments, and various equivalent modifications and substitutions can be made by those skilled in the art without departing from the spirit of the present application, and these equivalent modifications and substitutions are intended to be included in the scope of the present application as defined in the appended claims.
Claims (5)
1. The long and short interest sequence recommending method is characterized by comprising the following steps of:
Acquiring data comprising an interaction sequence of a user and a commodity;
Constructing a graph of the user and the commodity according to the obtained data, and inputting the constructed graph into a graph neural network to obtain an initial embedding vector of the user and the commodity;
Learning to obtain a user short-time behavior embedded vector according to the initial embedded vector of the commodity;
according to the short-time behavior embedded vectors of the user, learning to obtain K short-time user interest embedded vectors;
Adding the user and commodity interaction sequence and the position embedding vector, and inputting the added sequence and the position embedding vector into a transducer module to obtain a long-term user interest embedding vector;
merging the long-time user interest embedded vector and the initial embedded vector of the user into the K short-time user interest embedded vectors to obtain K+1 user interest embedded vectors;
learning the weight of each interest embedded vector through the attention mechanism between the interest embedded vector of the user and the commodity embedded vector, and constructing a final embedded vector of the user;
Obtaining a commodity prediction result according to the inner product of the commodity embedding vector and the final embedding vector of the user;
constructing a graph of the user and the commodity according to the obtained data, inputting the constructed graph into a graph neural network, and obtaining an initial embedding vector of high-order semantics of the user and the commodity, wherein the method comprises the following steps:
constructing a graph of the user and the commodity according to the obtained data, and gathering information between the nodes into a central node through multi-layer graph convolution to express an initial embedding vector of the user and the commodity;
wherein, the expression of the initial embedded vector obtained through graph convolution is as follows:
Wherein E (l) represents that after the graph convolution layer I, the obtained commodity and the user are integrated by the embedded vector; w 1 (l) and W 2 (l) are learnable parameter matrices; Representing that each node blends own information into, adds an identity matrix I, multiplies the identity matrix I by an initialization embedded matrix, and aggregates information of neighbors of users or commodities, wherein/> Is an adjacency matrix, I is an identity matrix; Indicating that the correlation between the user and the commodity is integrated, and the ". As shown in the drawing, the". Sur represents multiplication by element;
the learning to obtain the user short-time behavior embedded vector according to the initial embedded vector of the commodity comprises the following steps:
The expression of the interaction sequence of the user and the commodity is as follows: representing an interaction sequence of a user and commodities, and sequencing according to interaction time; wherein/> An embedded vector with an object ID of m;
The short-time behavior embedded vector of the user is obtained by means-pooling processing the commodity embedded vector of the interaction sequence of the user and the commodity N is a user ID;
the expression of the position embedding vector is as follows:
In the method, in the process of the invention, Information is embedded in the position of the index value even for representing the commodity of the user and the interaction, and the index value is/is evenTimestamp information representing interactions of the item with the user,/>Information is embedded in the positions of the index values of the commodity representing the interaction between the user and the commodity, l 1 represents the index value of the commodity, and d is the dimension of commodity embedding;
the expression of the K+1 user interest embedding vectors is as follows:
Where V i n denotes the user k+1 interest embedding matrices, n is the user ID, i is the ID of the interest embedding vector, Is an interest embedding vector, i e (1, k+1);
the expression of the user's final embedded vector is as follows:
yi＝attention(Ij,Vi n,Vi n)＝Viσ(ViIj)
wherein σ is a softmax nonlinear activation function; i j is a commodity embedded vector, and V i is a user embedded vector; u i is the user-embedded vector that is ultimately obtained by weighted summing the short-time and long-time user-interest embedded vectors, w ij is the weight value of each user-interest embedded vector.
2. The method for recommending long and short interest sequences according to claim 1, wherein learning to obtain K short-term user interest embedded vectors according to the short-term user action embedded vectors comprises:
And inputting the short-time behavior embedded vectors of the users into a capsule network, and obtaining K short-time interest embedded vectors of the users through the process of iterative repeated dynamic routing, wherein K is a preset super parameter.
3. The method for recommending long and short interest sequences according to claim 1, further comprising the step of constructing an end-to-end model:
constructing a long-term user interest embedded vector by using user and commodity interaction data through a pre-trained graphic neural network part, a user behavior layer part, a multi-interest capsule network part and transformers, and constructing an attention mechanism layer between the multi-interest and commodity embedded vector and a click rate prediction part to form an end-to-end long-short interest sequence recommendation algorithm;
Model parameter learning is performed on the training data set by using a random gradient descent method until the model converges.
4. A long and short interest sequence recommendation device, characterized by comprising:
At least one processor;
At least one memory for storing at least one program;
When executed by the at least one processor, causes the at least one processor to implement the method of any of claims 1-3.
5. A computer readable storage medium, in which a processor executable program is stored, characterized in that the processor executable program is for performing the method according to any of claims 1-3 when being executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210575237.7A CN115099886B (en) | 2022-05-25 | 2022-05-25 | Long-short interest sequence recommendation method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210575237.7A CN115099886B (en) | 2022-05-25 | 2022-05-25 | Long-short interest sequence recommendation method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115099886A CN115099886A (en) | 2022-09-23 |
| CN115099886B true CN115099886B (en) | 2024-04-19 |
Family
ID=83288591
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210575237.7A Active CN115099886B (en) | 2022-05-25 | 2022-05-25 | Long-short interest sequence recommendation method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115099886B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115374369B (en) * | 2022-10-20 | 2023-04-07 | 暨南大学 | News diversity recommendation method and device based on graph neural network |
| CN116562992B (en) * | 2023-07-11 | 2023-09-29 | 数据空间研究院 | Method, device and medium for recommending items for modeling uncertainty of new interests of user |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111932336A (en) * | 2020-07-17 | 2020-11-13 | 重庆邮电大学 | Commodity list recommendation method based on long-term and short-term interest preference |
| CN112990972A (en) * | 2021-03-19 | 2021-06-18 | 华南理工大学 | Recommendation method based on heterogeneous graph neural network |
| CN114519145A (en) * | 2022-02-22 | 2022-05-20 | 哈尔滨工程大学 | Sequence recommendation method for mining long-term and short-term interests of users based on graph neural network |
| CN114528490A (en) * | 2022-02-18 | 2022-05-24 | 哈尔滨工程大学 | Self-supervision sequence recommendation method based on long-term and short-term interests of user |
-
2022
- 2022-05-25 CN CN202210575237.7A patent/CN115099886B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111932336A (en) * | 2020-07-17 | 2020-11-13 | 重庆邮电大学 | Commodity list recommendation method based on long-term and short-term interest preference |
| CN112990972A (en) * | 2021-03-19 | 2021-06-18 | 华南理工大学 | Recommendation method based on heterogeneous graph neural network |
| CN114528490A (en) * | 2022-02-18 | 2022-05-24 | 哈尔滨工程大学 | Self-supervision sequence recommendation method based on long-term and short-term interests of user |
| CN114519145A (en) * | 2022-02-22 | 2022-05-20 | 哈尔滨工程大学 | Sequence recommendation method for mining long-term and short-term interests of users based on graph neural network |
Non-Patent Citations (1)
| Title |
|---|
| 基于多维社交关系嵌入的深层图神经网络推荐方法;何昊晨;***;;计算机应用;20201010;第40卷(第10期);第2796-2800页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115099886A (en) | 2022-09-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11526680B2 (en) | Pre-trained projection networks for transferable natural language representations | |
| Pan et al. | Study on convolutional neural network and its application in data mining and sales forecasting for E-commerce | |
| Yang et al. | Click-through rate prediction in online advertising: A literature review | |
| Liu et al. | A novel deep hybrid recommender system based on auto-encoder with neural collaborative filtering | |
| CN115082147B (en) | Sequence recommendation method and device based on hypergraph neural network | |
| Liu et al. | Detection of spam reviews through a hierarchical attention architecture with N-gram CNN and Bi-LSTM | |
| CN115099886B (en) | Long-short interest sequence recommendation method, device and storage medium | |
| Osoba et al. | Fuzzy cognitive maps of public support for insurgency and terrorism | |
| Li et al. | Deep probabilistic matrix factorization framework for online collaborative filtering | |
| CN110781407A (en) | User label generation method and device and computer readable storage medium | |
| Sheng et al. | Enhanced graph neural network for session-based recommendation | |
| CN116150480A (en) | User personalized demand prediction method integrating multi-mode comment information | |
| Du et al. | based hierarchical attention cooperative neural networks for recommendation | |
| Tan et al. | Recommendation Based on Users’ Long‐Term and Short‐Term Interests with Attention | |
| Garcia-Vega et al. | Learning from data streams using kernel least-mean-square with multiple kernel-sizes and adaptive step-size | |
| Bouzidi et al. | Deep learning-based automated learning environment using smart data to improve corporate marketing, business strategies, fraud detection in financial services, and financial time series forecasting | |
| Wei et al. | DLGNN: A Double-layer Graph Neural Network Model Incorporating Shopping Sequence Information for Commodity Recommendation. | |
| Weng et al. | GAIN: Graph attention & interaction network for inductive semi-supervised learning over large-scale graphs | |
| Dongbo et al. | Intelligent chatbot interaction system capable for sentimental analysis using hybrid machine learning algorithms | |
| Zou et al. | Deep field relation neural network for click-through rate prediction | |
| Li | Cross‐Border E‐Commerce Intelligent Information Recommendation System Based on Deep Learning | |
| Fang | Making recommendations using transfer learning | |
| Liu | Deep learning in marketing: a review and research agenda | |
| Li et al. | From edge data to recommendation: A double attention-based deformable convolutional network | |
| Liu et al. | Hands-On Deep Learning Architectures with Python: Create deep neural networks to solve computational problems using TensorFlow and Keras |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |