CN117196763A - Commodity sequence recommending method based on time sequence perception self-attention and contrast learning - Google Patents
Commodity sequence recommending method based on time sequence perception self-attention and contrast learning Download PDFInfo
- Publication number
- CN117196763A CN117196763A CN202311025453.5A CN202311025453A CN117196763A CN 117196763 A CN117196763 A CN 117196763A CN 202311025453 A CN202311025453 A CN 202311025453A CN 117196763 A CN117196763 A CN 117196763A
- Authority
- CN
- China
- Prior art keywords
- sequence
- attention
- commodity
- user
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000008447 perception Effects 0.000 title claims abstract description 29
- 238000012549 training Methods 0.000 claims abstract description 36
- 230000000694 effects Effects 0.000 claims abstract description 14
- 238000004364 calculation method Methods 0.000 claims abstract description 11
- 238000007781 pre-processing Methods 0.000 claims abstract description 8
- 239000011159 matrix material Substances 0.000 claims description 52
- 230000006870 function Effects 0.000 claims description 36
- 239000013598 vector Substances 0.000 claims description 35
- 230000003993 interaction Effects 0.000 claims description 16
- 238000013528 artificial neural network Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 15
- 238000012360 testing method Methods 0.000 claims description 13
- 238000012795 verification Methods 0.000 claims description 13
- 230000004913 activation Effects 0.000 claims description 6
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 230000006872 improvement Effects 0.000 claims description 3
- 230000001965 increasing effect Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 2
- 238000012216 screening Methods 0.000 claims description 2
- 230000006399 behavior Effects 0.000 abstract description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 108091026890 Coding region Proteins 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a commodity sequence recommending method based on time sequence perception self-attention and contrast learning, which comprises the following steps: 1) Data preprocessing, namely organizing user history data into a user behavior sequence; 2) Constructing a sequence recommendation model based on time sequence perception self-attention and contrast learning, wherein the sequence recommendation model comprises a sequence recommender of an improved transducer and a contrast learning module, and training the model; 3) And inputting the historical behavior sequence data of the user into the trained model, and recommending the commodity purchased next by the user according to the prediction score generated by the model. According to the invention, real time interval information is introduced into a self-attention mechanism, and the capability of capturing complex sequence dependence of a model is enhanced through the calculation of decoupled time sequence sensing multi-head self-attention; meanwhile, contrast learning is introduced, under the problems of data sparseness and noise deviation, the user preference relation can be completely modeled, and the commodity recommendation effect is improved.
Description
Technical Field
The invention relates to the technical field of recommendation systems, in particular to a commodity sequence recommendation method based on time sequence perception self-attention and contrast learning.
Background
The recommendation system makes recommendation to the user according to the user history information, helps the user to quickly find interesting commodities, and brings great benefit to merchants. Unlike traditional recommendation techniques, sequential recommendation can dynamically simulate user-commodity interactions and capture complex dependencies, better modeling user interests. Recently, as the transducer shows strong performance in the coding sequence, and is also introduced into sequence recommendation, the transducer-based sequence recommendation model utilizes a self-attention network SAN to learn the importance among commodities, better models the sequence dependence relationship, and shows strong performance superior to other sequence models.
Although the sequential recommendation method using SAN has achieved excellent effects in commodity recommendation tasks, the currently existing methods mostly have the following problems:
1. most of the current methods only simulate the time sequence of user-object interaction, and do not utilize real time information, so that the modeled sequence relationship is incomplete;
2. most of the current methods directly superimpose commodity information and position information, and calculate attention score through SAN, which not only limits the expressive force of an attention matrix, but also introduces noise, so that the modeled sequence relationship is inaccurate;
3. current methods mostly rely on recommendation tasks to optimize a large number of parameters, which suffer from data sparsity and noise problems, which make them difficult to learn high quality user characterization, resulting in sub-optimal commodity recommendation results.
The problems existing above cause that the existing sequence recommendation model using SAN cannot model sequence dependence well, resulting in poor commodity recommendation results.
Disclosure of Invention
The invention aims to overcome the defects and shortcomings of the prior art, and provides a commodity sequence recommending method based on time sequence sensing self-attention and contrast learning, which calculates an attention matrix by using decoupled time sequence sensing multi-head self-attention through introducing a real time interval, so that a model can model more complex sequence dependence; meanwhile, a contrast learning module is introduced, a plurality of pairs of positive and negative samples are respectively constructed by utilizing random data level enhancement and model level enhancement, a sequence recommending task is combined with the contrast learning task, the trouble of data sparseness and noise is solved, the learning of user preference is enhanced, and the recommending effect is improved.
In order to achieve the above purpose, the technical scheme provided by the invention is as follows: the method provides a sequence recommendation model based on time sequence perception self-attention and contrast learning, the model is composed of two parts, the first part is a sequence recommender based on an improved transducer, the sequence recommender comprises an embedding module, a self-attention module and a prediction module, after data is encoded by the embedding module, the self-attention module captures information of an embedded vector, and finally the prediction module gives commodity preference scores, wherein the improvement comprises the following steps: (1) constructing embedded vectors of different information in an embedded module, converting a time interval into an embedded matrix by using an improved Hamming window function, introducing time information and capturing the same time interval dependence; (2) modifying the attention in the self-attention module into decoupled time sequence perception multi-head self-attention, respectively calculating different attention matrixes, and enhancing the modeling capacity of the model; the second part is a contrast learning module which uses random data level enhancement and model level enhancement to construct a plurality of pairs of positive and negative samples, performs contrast learning tasks, assists training and improves recommendation effects;
the specific implementation of the commodity sequence recommending method comprises the following steps:
1) Collecting historical data in a user real scene, including a user ID, a commodity ID and an interaction time stamp, and preprocessing the historical data: classifying with user ID, organizing user u' S data into a user sequence S of length n u Finally, dividing all user sequences into a training set, a verification set and a test set;
2) Sending the training set into a sequence recommendation model for training, performing iterative optimization on a loss function of the model by using an Adam optimizer until the model converges, screening optimal model parameters by using a verification set, and finally evaluating a model effect with the optimal parameters by using a test set to obtain a sequence recommendation model with optimal performance based on time sequence perception self-attention and contrast learning;
3) Acquiring historical data of a user u needing commodity recommendation, and organizing the historical data into a user sequence S through preprocessing in the step 1) u Will S u And (3) inputting the commodity preference score into the sequence recommendation model with the optimal performance obtained in the step (2), and finally obtaining the preference score of the user for the commodity by a prediction module of the model, and recommending the commodity with the greatest interest to the user according to the preference score.
Further, in step 1), historical data of the user in a real transaction scene is obtained and preprocessed, and the method comprises the following steps:
1.1 Acquiring user history data including user ID, commodity ID and interaction time stamp, classifying the user ID, and organizing the interaction data of user u into a user sequence S with length n u ＝(v u ，t u ，p u ) The method comprises the steps of carrying out a first treatment on the surface of the U e U, U is the user set, v u Is a time ordered sequence of product IDs interacting with user u, t u Is the time stamp sequence of the interaction of the user u with the commodity, p u The relative position sequence of the interaction between the user u and the commodity is n in length;
1.2 A) converting the time stamp sequence t u Processed into a time interval sequence T u ：
In the method, in the process of the invention,is t u The timestamp of the corresponding position in Min (·) is a minimum function, r max Is the maximum time interval, defined as follows:
where T is the set of time intervals for all sequences,is a scaling factor, max (·) is a maximum function; obtaining a time interval sequence T u After that, the user sequence becomes S u ＝(v u ，T u ，p u )；
1.3 Dividing the training set, the verification set and the test set, wherein the specific operation is as follows: for user sequence S u Taking the commodity ID, the time interval and the position corresponding to the last position in the sequence as a test set, taking the penultimate position in the sequence as a verification set and taking the rest as a training set; the above operation is repeated for all user sequences.
Further, in step 2), the sequence recommender based on the improved transducer and the contrast learning module perform joint training of the recommendation task and the contrast learning task, the training set divided in step 1) is sent into the sequence recommendation model for training, and the Adam optimizer is utilized to perform loss function on the modelOptimizing until the model converges:
in the method, in the process of the invention,is recommended task loss, < >>The loss of the learning task is compared, and lambda is the super parameter of the control proportion;
after each training round, the optimal parameters of the model are reserved according to the result of the prediction module on the verification set, the training is stopped after the model is converged, the model effect with the optimal parameters is evaluated by using the test set, and a sequence recommendation model with optimal performance and based on time sequence perception self-attention and comparison learning is obtained.
Further, the sequence recommender based on the improved transducer performs the learning of recommendation tasks, and the specific operations are as follows: the preprocessed data is sent to a sequence encoder of a sequence recommender for coding learning, wherein the sequence encoder consists of an embedding module and a self-attention module; and predicting commodities purchased by a user according to a prediction module by using the sequence vector output by the sequence encoder, wherein the specific structure is as follows:
1. and (3) an embedding module: the commodity ID, the position and the time interval sequence are respectively encoded into embedded vectors through the embedding module and are continuously updated in model training, and the specific process is as follows:
a. commodity embedding/location embedding: first, initializing a commodity embedded vector matrix For vector representation, |V| is the number of commodity sets, and d is the embedding dimension; for any commodity item, its initial vector is derived from X, expressed asThus, the user sequence S u Commodity ID sequence v in (a) u Coded as commodity embedding matrix E I ：
Wherein n is the sequence length, x 1 、x 2 、x n Commodity vectors at positions corresponding to the sequences; similarly, a position embedding matrix can be obtained
b. Time interval embedding: first, the time interval sequence T u Normalized to the value of [0, num), num being a set number; initializing an embedding matrixe 1 、e 2 、e nums An embedded vector which is a corresponding value; in order to capture the weight relation between time intervals, a modified hamming window function is used +.>Calculating a weight matrix Values in the matrix are:
wherein T is i u Is T u Time interval of position i, range of iThe circumference is 1 to n, j is 0 to nums-1 integer value, pi is the circumference rate, alpha is the window size parameter, x represents the input of the function; calculating a final time interval embedding matrix according to the embedding matrix M and the weight matrix W
2. Improved self-attention module: after information of three embedded matrixes is captured by utilizing decoupled time sequence sensing multi-head self-attention, inputting the information into a feedforward layer neural network, and increasing nonlinearity of a model; to mitigate gradient cancellation and overfitting, an LD (·) operation is performed after the output of each sub-layer, including residual connection, layer normalization, and Dropout operations; stacking the L layers of self-attention modules to obtain the final L-th layer output H L+1 ：
H l+1 ＝LD(PFFN(LD(DTMH(H l ，E T ，E P ))))；l∈[1，L]
H 1 ＝E I
Wherein H is l+1 Is the output of layer l+1, H l Is the output of layer I, L being in the range 1 to L, H 1 Is the output of layer 1, DTMH (·) is decoupled time-series aware multihop self-attention, PFFN (·) is feed-forward layer neural network, and the process for computation of DTMH (·) and PFFN (·) is as follows:
a. decoupled time-series aware multi-headed self-attention: using decoupled time sequence perception multi-head self-attention capturing embedded information to enhance modeling capability of a model, wherein a calculation formula is as follows:
A l+1 ＝DTMH(H l ，E T ，E P )＝[D_head 1 ；D_head 2 ；…；D_head h ]W O
wherein A is l+1 Is the multi-head self-attention output of layer l+1, H l Is the entire self-attention module output of layer l, D_head k Is the kth attention score, D_att k Is the kth head attention matrix sum, kth attention matrix for goods, time interval and location, +.> Is a projection matrix corresponding to each attention head, k is the serial number of the head, h is the attention head number, W O For the parameter matrix, softmax (·) is the normalized exponential function, ++>Representing a transpose of the matrix; respectively calculating and fusing three attention matrixes of each head, calculating the attention score of a single head, and finally splicing the attention scores of the h heads and sending the attention scores into a feedforward layer neural network;
b. feedforward layer neural network: to impart more nonlinear capabilities to the model, a feed-forward layer neural network is applied, including two linear transforms PFFN, FFN and a gel activation, where the gel is used to replace the original gel:
FFN(x)＝GELU(xW (1) +b (1) )W (2) +b (2)
wherein A is l Is the multi-headed self-attention output of the first layer,is A l Vector representation of corresponding position in W (1) 、W (2) 、b (1) 、b (2) Is a shared learnable parameter;
3. and a prediction module: after stacking the L layers of self-attention blocks, a combined representation H of the commodity, location and time interval is obtained L+1 The method comprises the steps of carrying out a first treatment on the surface of the Taking H L+1 The last vector h u As a representation of the user sequence, then the next commodity prediction task is constructed as a classification task for the entire commodity set, the given sequence representing h u And a commodity embedded vector matrix X, predicting the score of the commodityThe calculation is as follows:
recommending the user according to the size of the predicted commodity score;
4. recommended loss: adopting a log likelihood loss function as a recommended task loss so as to optimize model parameters until the model converges; wherein for each user u and for each location t, a penalty is recommendedIs calculated as:
in the method, in the process of the invention,representing the predicted next commodity representation, σ is a nonlinear activation function, ++>Is commodity v t+1 Is embedded in the representation of->Is commodity v j Is embedded in the representation, v j Is a sampled negative commodity; summing the losses of all users to obtain the final recommended loss +.>
Further, the contrast learning module performs contrast learning tasks, builds the contrast learning tasks by utilizing random data level enhancement and model level enhancement, and assists in training of recommended tasks; the random data level enhancement adopts three element operations of clipping, shielding and disordered, and each time, one operation is randomly selected to construct different views of the same sequence in a set proportion; the contrast learning module also uses a model-level enhancement to construct different views, namely, views are constructed through two different Dropout masks in the sequence encoder, so that semantic information of the original sequence is reserved; thus, given a batch number B sequence, the sequence is obtained through two model-level enhancement and two data-level enhancement, respectively, and finally through a sequence encoder For the output enhanced by the model stage, +.>Is output after being enhanced by the random data level; will->And->And->Paired calculation, the same position is regarded as positive pair of samples, the rest as negative pair of samples, and the contrast loss is calculated>The following are provided:
in the method, in the process of the invention,representing the corresponding positive sample, +.>Representing the corresponding negative samples, B is the sequence number value of 1 to B, m is the value of 1 to 2B, sim (& gt) is the calculated similarity function, & lt/L>Is an indication function; summing up to obtain the final contrast loss->
Further, in step 3), first, history data of the user u who needs to make a commodity recommendation is collected, and the history data is used in step 1)Organizing historical data into a user sequence S u Will S u Inputting the sequence recommendation model with optimal performance and based on time sequence perception self-attention and contrast learning obtained in the step 2), encoding by a sequence encoder, finally obtaining the preference scores of the user on all commodities by a prediction module, sorting according to the size of the preference scores, recommending the user u based on TOPN from large to small, namely recommending a set of N commodities with the largest preference score for the commodity which is most likely to be purchased next by the user u.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. the invention uses an embedding module to respectively construct commodity, position and time interval embedding vectors, and for time interval characteristics, uses an improved Hamming window function to introduce time information into a model and captures the relationship of the same time interval, so that the sequence relationship of model modeling is complete.
2. According to the invention, the original attention in the self-attention module is transformed into decoupled time sequence perception multi-head self-attention, the attention scores of three embedded matrixes are respectively calculated, the modeling capacity of the model on data is enhanced, and the modeling sequence relation is more accurate.
3. According to the invention, a comparison learning module is used, a plurality of pairs of positive and negative samples are respectively constructed by utilizing random data level enhancement and model level enhancement, comparison learning tasks are carried out, learning of a recommendation task is assisted, and the recommendation effect can be improved under the influence of data sparseness and noise.
4. The method solves the problems of the conventional SAN-based sequence recommendation model, improves the commodity recommendation effect, improves the user experience and brings benefits to merchants.
Drawings
Fig. 1 is a flowchart of a commodity sequence recommending method based on time sequence sensing self-attention and contrast learning in an embodiment of the present invention.
Fig. 2 is a schematic diagram of a commodity sequence recommendation method based on time-series sensing self-attention and contrast learning in an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of an improved self-attention module of a commodity sequence recommendation method based on time sequence sensing self-attention and contrast learning according to an embodiment of the present invention.
Detailed Description
The invention will be further illustrated with reference to specific examples.
As shown in fig. 1, the present embodiment provides a commodity sequence recommending method based on time sequence perception self-attention and contrast learning, the method provides a sequence recommending model based on time sequence perception self-attention and contrast learning, the model is composed of two parts, the first part is a sequence recommender based on improved transducer, the sequence recommender comprises an embedding module, a self-attention module and a predicting module, after data is encoded by the embedding module, information of an embedded vector is captured by the self-attention module, and finally commodity preference scores are given by the predicting module, wherein the improvement comprises: (1) constructing embedded vectors of different information in an embedded module, converting a time interval into an embedded matrix by using an improved Hamming window function, introducing time information and capturing the same time interval dependence; (2) modifying the attention in the self-attention module into decoupled time sequence perception multi-head self-attention, respectively calculating different attention matrixes, and enhancing the modeling capacity of the model; the second part is a contrast learning module which uses random data level enhancement and model level enhancement to construct a plurality of pairs of positive and negative samples, performs contrast learning tasks, assists training and improves recommendation effects; which comprises the following steps:
s1) acquiring historical behavior data of a user in a real scene, and preprocessing, wherein the specific process is as follows:
s11) acquiring user history data comprising user ID, commodity ID and interaction time stamp, classifying the user ID, and organizing the interaction data of the user u into a user sequence S with the length of n u ＝(v u ，t u ，p u ) The method comprises the steps of carrying out a first treatment on the surface of the U e U, U is the user set; v u Is a time ordered sequence of product IDs interacting with user u, t u Is the time stamp sequence of the interaction of the user u with the commodity, when v u And t u If the length of (2) is greater than n, then n bits are continuously intercepted in each sequence, and if the length is less than n, then the sequence is in orderFilling the right end of the column until the length is n; p is p u Is the relative position sequence of the interaction between the user u and the commodity;
s12) the time stamp sequence t is processed u Processed into a time interval sequence T u ：
Wherein the method comprises the steps ofIs t u The timestamp of the corresponding position in Min (·) is a minimum function, r max Is the maximum time interval, defined as follows:
t is the set of time intervals for all sequences,is a scaling factor, max (·) is a maximum function; obtaining a time interval sequence T u After that, the user sequence becomes S u ＝(v u ，T u ，p u )；
S13) dividing a training set, a verification set and a test set, wherein the specific operation is as follows: for user sequence S u Taking the commodity ID, the time interval and the position corresponding to the last position in the sequence as a test set, taking the penultimate position in the sequence as a verification set and taking the rest as a training set; the above operation is repeated for all user sequences.
S2) constructing a sequence recommendation model based on time sequence perception self-attention and contrast learning, wherein a sequence recommender based on an improved transducer and a contrast learning module are respectively constructed to perform joint training of recommendation tasks and contrast learning tasks, the training set divided in the step 1) is sent into the sequence recommendation model to train, and the Adam optimizer is utilized to train a loss function of the modelOptimizing until the model converges:
is recommended task loss, < >>The loss of the learning task is compared, and lambda is the super parameter of the control proportion;
after each training round, the optimal parameters of the model are reserved according to the result of the prediction module on the verification set, the training is stopped after the model is converged, the model effect with the optimal parameters is evaluated by using the test set, and a sequence recommendation model with optimal performance and based on time sequence perception self-attention and comparison learning is obtained.
The sequence recommendation model based on time sequence perception self-attention and contrast learning is constructed, as shown in fig. 2, and the sequence recommendation device and the contrast learning module based on improved transducer are required to be respectively constructed.
The sequence recommender based on the improved transducer is constructed to learn recommendation tasks, and the specific structure is as follows:
1. and (3) an embedding module: the commodity ID, the position and the time interval sequence are respectively encoded into embedded vectors through the embedding module and are continuously updated in model training, and the specific process is as follows:
a. commodity embedding/location embedding: first, initializing a commodity embedded vector matrix For vector representation, |V| is the number of commodity sets, and d is the embedding dimension; for any commodity item, its initial vector canObtained from X, expressed as +.>Thus, the user sequence S can be u Commodity ID sequence v in (a) u Coded as commodity embedding matrix E I ：
Where n is the sequence length, x 1 、x 2 、x n Embedding commodities in positions corresponding to the sequences; similarly, a position embedding matrix can be obtained
b. Time interval embedding: first, the time interval sequence T u Normalized to the value of [0, num), num being a set number; initializing an embedding matrixe 1 、e 2 、e nums An embedded vector which is a corresponding value; in order to capture the weight relation between time intervals, a modified hamming window function is used +.>Calculating a weight matrix Values in the matrix are:
T i u is T u The time interval of the position i, i ranges from 1 to n, j is an integer value from 0 to nums-1, pi is a circumference rate, alpha is a window size parameter, and x represents the input of a function; calculating a final time interval embedding matrix according to the embedding matrix M and the weight matrix W
2. Improved self-attention module: as shown in fig. 3, after three pieces of information of the embedded matrix are captured by using decoupled time sequence perception multi-head self-attention, the information is input into a feedforward layer neural network, and the nonlinearity of the model is increased; to mitigate gradient cancellation and overfitting, an LD (·) operation is performed after the output of each sub-layer, including residual connection, layer normalization, and Dropout operations; stacking the L layers of self-attention modules to obtain the final L-th layer output H L+1 ：
H l+1 ＝LD(PFFN(LD(DTMH(H l ，E T ，E P ))))；l∈[1，L]
H 1 ＝E I
Wherein H is l+1 Is the output of layer l+1, H l Is the output of layer I, L being in the range 1 to L, H 1 Is the output of layer 1, DTMH (·) is decoupled time-series aware multihop self-attention, PFFN (·) is feed-forward layer neural network, and the process for computation of DTMH (·) and PFFN (·) is as follows:
a. decoupled time-series aware multi-headed self-attention: using decoupled time sequence perception multi-head self-attention capturing embedded information to enhance modeling capability of a model, wherein a calculation formula is as follows:
A l+1 ＝DTMH(H l ，E T ，E P )＝[D_head 1 ；D_head 2 ；…；D_head h ]W O
wherein A is l+1 Is the multi-head self-attention output of layer l+1, H l Is the entire self-attention module output of layer l, D_head k Is the kth attention score, D_att k Is the kth head attention matrix sum, kth attention matrix for goods, time interval and location, +.> Is a projection matrix corresponding to each attention head, k is the serial number of the head, h is the attention head number, W O For the parameter matrix, softmax (·) is the normalized exponential function, ++>Representing a transpose of the matrix; respectively calculating and fusing three attention matrixes of each head, calculating the attention score of a single head, and finally splicing the attention scores of the h heads and sending the attention scores into a feedforward layer neural network;
b. feedforward layer neural network: to impart more nonlinear capabilities to the model, a feed-forward layer neural network is applied, which includes two linear transformations and a gel activation, where the gel is used to replace the original gel:
FFN(x)＝GELU(xW (1) +b (1) )W (2) +b (2)
wherein A is l Is the multi-headed self-attention output of the first layer,is A l Vector representation of corresponding position in W (1) 、W (2) 、b (1) 、b (2) Is a shared learnable parameter;
3. and a prediction module: after stacking the L layers of self-attention blocks, a combined representation H of the commodity, location and time interval is obtained L+1 The method comprises the steps of carrying out a first treatment on the surface of the Taking H L+1 The last vector h u As a representation of the user sequence, then the next commodity prediction task is constructed as a classification task for the entire commodity set, the given sequence representing h u And a commodity embedded vector matrix X, predicting the score of the commodityThe calculation is as follows:
recommending the user according to the size of the predicted commodity score;
4. recommended loss: adopting a log likelihood loss function as a recommended task loss so as to optimize model parameters until the model converges; wherein for each user u and for each location t, a penalty is recommendedCan be calculated as:
wherein,representing the predicted next commodity representation, σ is a nonlinear activation function, ++>Is commodity v t+1 Is embedded in the representation of->Is commodity v j Is embedded in the representation, v j Is a sampled negative commodity; summing the losses of all users to obtain the final recommended loss:
a contrast learning module is constructed to carry out contrast learning tasks, the random data level enhancement and the model level enhancement are utilized to construct contrast learning tasks, and training of recommended tasks is assisted; the random data level enhancement adopts three element operations of clipping, shielding and disordered, and each operation is randomly selected to construct different views of the same sequence in a certain proportion; the contrast learning module also uses a model-level enhancement to construct different views, namely, views through two different Dropout masks in the sequence encoder, which greatly preserves the semantic information of the original sequence; thus, given a batch number B sequence, the sequence is obtained through two model-level enhancement and two data-level enhancement, respectively, and finally through a sequence encoder For the output enhanced by the model stage, +.>D is the embedding dimension for the output after the random data level enhancement; will->And->And->Paired calculation, the same position is regarded as positive pair of samples, the rest as negative pair of samples, the contrast loss can be calculated>The following are provided:
wherein,representing the corresponding positive sample, +.>Representing the corresponding negative samples, B is the sequence number value of 1 to B, m is the value of 1 to 2B, sim (& gt) is the calculated similarity function, & lt/L>Is an indication function; summing results in a final contrast loss:
s3) firstly, collecting historical data of a user u needing commodity recommendation, and organizing the historical data into a user sequence S by utilizing the preprocessing operation in the step 1) u Will S u Inputting the sequence recommendation model with optimal performance and based on time sequence perception self-attention and contrast learning obtained in the step S2), encoding by a sequence encoder, finally obtaining the preference scores of the user on all commodities by a prediction module, sorting according to the size of the preference scores, recommending the user u based on TOPN from large to small, namely recommending a set of N commodities with the largest preference score for the next most likely purchased commodity of the user u.
In summary, after the method is adopted, the invention applies a new method for commodity sequence recommendation, by introducing a real time interval into the SAN and providing a multi-head self-attention calculation attention matrix by using decoupled time sequence perception, the problems of incomplete modeling and inaccuracy in SAN-based sequence recommendation are solved, meanwhile, a contrast learning module is introduced, the problems of data sparseness and noise in a recommendation system are solved, the recommendation effect is improved, a proper commodity is recommended for a user according to a user history behavior sequence, the user experience is improved, and meanwhile, the invention creates value for merchants, has practical popularization value and is worth popularizing.
The above embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention, so variations in shape and principles of the present invention should be covered.
Claims (6)
1. The commodity sequence recommending method based on time sequence perception self-attention and contrast learning is characterized in that the method provides a sequence recommending model based on time sequence perception self-attention and contrast learning, the model is composed of two parts, the first part is a sequence recommender based on an improved transducer, the sequence recommender comprises an embedding module, a self-attention module and a prediction module, after data are encoded by the embedding module, information of an embedded vector is captured by the self-attention module, and finally commodity preference scores are given by the prediction module, wherein the improvement comprises the following steps: (1) constructing embedded vectors of different information in an embedded module, converting a time interval into an embedded matrix by using an improved Hamming window function, introducing time information and capturing the same time interval dependence; (2) modifying the attention in the self-attention module into decoupled time sequence perception multi-head self-attention, respectively calculating different attention matrixes, and enhancing the modeling capacity of the model; the second part is a contrast learning module which uses random data level enhancement and model level enhancement to construct a plurality of pairs of positive and negative samples, performs contrast learning tasks, assists training and improves recommendation effects;
the specific implementation of the commodity sequence recommending method comprises the following steps:
1) Collecting historical data in a user real scene, including a user ID, a commodity ID and an interaction time stamp, and preprocessing the historical data: classifying with user ID, organizing user u' S data into a user sequence S of length n u Finally, dividing all user sequences into a training set, a verification set and a test set;
2) Sending the training set into a sequence recommendation model for training, performing iterative optimization on a loss function of the model by using an Adam optimizer until the model converges, screening optimal model parameters by using a verification set, and finally evaluating a model effect with the optimal parameters by using a test set to obtain a sequence recommendation model with optimal performance based on time sequence perception self-attention and contrast learning;
3) Acquiring historical data of a user u needing commodity recommendation, and organizing the historical data into a user sequence S through preprocessing in the step 1) u Will S u And (3) inputting the commodity preference score into the sequence recommendation model with the optimal performance obtained in the step (2), and finally obtaining the preference score of the user for the commodity by a prediction module of the model, and recommending the commodity with the greatest interest to the user according to the preference score.
2. The method for recommending commodity sequences based on time-series aware self-attention and contrast learning according to claim 1, wherein in step 1), historical data of a user in a real transaction scene is acquired and preprocessed, comprising the steps of:
1.1 Acquiring user calendar)History data including user ID, commodity ID and interaction time stamp, classifying the interaction data of user u into a user sequence S of length n by user ID u ＝(v u ,t u ,p u ) The method comprises the steps of carrying out a first treatment on the surface of the U e U, U is the user set, v u Is a time ordered sequence of product IDs interacting with user u, t u Is the time stamp sequence of the interaction of the user u with the commodity, p u The relative position sequence of the interaction between the user u and the commodity is n in length;
1.2 A) converting the time stamp sequence t u Processed into a time interval sequence T u ：
In the method, in the process of the invention,is t u The timestamp of the corresponding position in Min (·) is a minimum function, r max Is the maximum time interval, defined as follows:
where T is the set of time intervals for all sequences,is a scaling factor, max (·) is a maximum function; obtaining a time interval sequence T u After that, the user sequence becomes S u ＝(v u ,T u ,p u )；
1.3 Dividing the training set, the verification set and the test set, wherein the specific operation is as follows: for user sequence S u Taking the commodity ID, the time interval and the position corresponding to the last position in the sequence as a test set, taking the penultimate position in the sequence as a verification set and taking the rest as a training set; the above operation is repeated for all user sequences.
3. The method for recommending commodity sequences based on time-series aware self-attention and contrast learning according to claim 2, wherein in step 2), based on the combined training of recommendation tasks and contrast learning tasks by the sequence recommender and contrast learning module of the improved fransformer, the training set divided in step 1) is sent into a sequence recommendation model for training, and the loss function of the model is used by Adam optimizerOptimizing until the model converges:
in the method, in the process of the invention,is recommended task loss, < >>The loss of the learning task is compared, and lambda is the super parameter of the control proportion;
after each training round, the optimal parameters of the model are reserved according to the result of the prediction module on the verification set, the training is stopped after the model is converged, the model effect with the optimal parameters is evaluated by using the test set, and a sequence recommendation model with optimal performance and based on time sequence perception self-attention and comparison learning is obtained.
4. The method for recommending commodity sequences based on time-series aware self-attention and contrast learning according to claim 3, wherein the sequence recommender based on improved transducer performs the learning of recommendation tasks, and the specific operations are as follows: the preprocessed data is sent to a sequence encoder of a sequence recommender for coding learning, wherein the sequence encoder consists of an embedding module and a self-attention module; and predicting commodities purchased by a user according to a prediction module by using the sequence vector output by the sequence encoder, wherein the specific structure is as follows:
1. and (3) an embedding module: the commodity ID, the position and the time interval sequence are respectively encoded into embedded vectors through the embedding module and are continuously updated in model training, and the specific process is as follows:
a. commodity embedding/location embedding: first, initializing a commodity embedded vector matrixFor vector representation, |V| is the number of commodity sets and d is the embedding dimension; for any commodity item, its initial vector is derived from X, denoted +.>Thus, the user sequence S u Commodity ID sequence v in (a) u Coded as commodity embedding matrix E I ：
Wherein n is the sequence length, x 1 、x 2 、x n Commodity vectors at positions corresponding to the sequences; similarly, a position embedding matrix can be obtained
b. Time interval embedding: first, the time interval sequence T u Normalized to the value of [0, num), num being a set number; initializing an embedding matrixe 1 、e 2 、e nums An embedded vector which is a corresponding value; in order to capture the weight relation between time intervals, a modified hamming window function is used +.>Calculating a weight matrix Values in the matrix are:
wherein T is i u Is T u The time interval of the position i, i ranges from 1 to n, j is an integer value from 0 to nums-1, pi is a circumference rate, alpha is a window size parameter, and x represents the input of a function; calculating a final time interval embedding matrix according to the embedding matrix M and the weight matrix W
2. Improved self-attention module: after information of three embedded matrixes is captured by utilizing decoupled time sequence sensing multi-head self-attention, inputting the information into a feedforward layer neural network, and increasing nonlinearity of a model; to mitigate gradient cancellation and overfitting, an LD (·) operation is performed after the output of each sub-layer, including residual connection, layer normalization, and Dropout operations; stacking the L layers of self-attention modules to obtain the final L-th layer output H L+1 ：
H l+1 ＝LD(PFFN(LD(DTMH(H l ,E T ,E P ))))；l∈[1,L]
H 1 ＝E I
Wherein H is l+1 Is the output of layer l+1, H l Is the output of layer I, L being in the range 1 to L, H 1 Is the output of layer 1, DTMH (·) is decoupled time-series aware multihop self-attention, PFFN (·) is feed-forward layer neural network, and the process for computation of DTMH (·) and PFFN (·) is as follows:
a. decoupled time-series aware multi-headed self-attention: using decoupled time sequence perception multi-head self-attention capturing embedded information to enhance modeling capability of a model, wherein a calculation formula is as follows:
A l+1 ＝DTMH(H l ,E T ,E P )＝[D_head 1 ；D_head 2 ；...；D_head h ]W O
wherein A is l+1 Is the multi-head self-attention output of layer l+1, H l Is the entire self-attention module output of layer l, D_head k Is the kth attention score, D_att k Is the kth head attention matrix sum, kth attention matrix for goods, time interval and location, +.> Is a projection matrix corresponding to each attention head, k is the serial number of the head, h is the attention head number, W O For the parameter matrix, softmax (·) is the normalized exponential function, ++>Representing a transpose of the matrix; respectively calculating and fusing three attention matrixes of each head, calculating the attention score of a single head, and finally splicing the attention scores of the h heads and sending the attention scores into a feedforward layer neural network;
b. feedforward layer neural network: to impart more nonlinear capabilities to the model, a feed-forward layer neural network is applied, including two linear transforms PFFN, FFN and a gel activation, where the gel is used to replace the original gel:
FFN(x)＝GELU(xW (1) +b (1) )W (2) +b (2)
wherein A is l Is the multi-headed self-attention output of the first layer,is A l Vector representation of corresponding position in W (1) 、W (2) 、b (1) 、b (2) Is a shared learnable parameter;
3. and a prediction module: after stacking the L layers of self-attention blocks, a combined representation H of the commodity, location and time interval is obtained L+1 The method comprises the steps of carrying out a first treatment on the surface of the Taking H L+1 Last vector of (3)h u As a representation of the user sequence, then the next commodity prediction task is constructed as a classification task for the entire commodity set, the given sequence representing h u And a commodity embedded vector matrix X, predicting the score of the commodityThe calculation is as follows:
recommending the user according to the size of the predicted commodity score;
4. recommended loss: adopting a log likelihood loss function as a recommended task loss so as to optimize model parameters until the model converges; wherein for each user u and for each location t, a penalty is recommendedIs calculated as:
in the method, in the process of the invention,representing the predicted next commodity representation, σ is a nonlinear activation function, ++>Is commodity v t+1 Is embedded in the representation of->Is commodity v j Is embedded in the representation of v j Is a sampled negative commodity; summing the losses of all users to obtain the final recommended loss +.>
5. The commodity sequence recommending method based on time sequence perception self-attention and contrast learning according to claim 4, wherein the contrast learning module performs contrast learning tasks, builds the contrast learning tasks by utilizing random data level enhancement and model level enhancement, and assists in training of the recommended tasks; the random data level enhancement adopts three element operations of clipping, shielding and disordered, and each time, one operation is randomly selected to construct different views of the same sequence in a set proportion; the contrast learning module also uses a model-level enhancement to construct different views, namely, views are constructed through two different Dropout masks in the sequence encoder, so that semantic information of the original sequence is reserved; thus, given a batch number B sequence, the sequence is obtained through two model-level enhancement and two data-level enhancement, respectively, and finally through a sequence encoder For the output enhanced by the model stage, +.>Is output after being enhanced by the random data level; will->And->And->Paired calculation, the same position is regarded as positive pair of samples, the rest as negative pair of samples, and the contrast loss is calculated>The following are provided:
in the method, in the process of the invention,representing the corresponding positive sample, +.>Representing the corresponding negative samples, B is the sequence number value of 1 to B, m is the value of 1 to 2B, sim (& gt) is the calculated similarity function, & lt/L>Is an indication function; summing up to obtain the final contrast loss->
6. The method for recommending commodity sequences based on time-series aware self-attention and contrast learning according to claim 5, wherein the method comprises the following steps: in step 3), first, the history data of the user u who needs to make commodity recommendation is collected, and the history data is organized into a user sequence S by the preprocessing operation in step 1) u Will S u The time sequence sensing self-attention sum based on the optimal performance obtained in the step 2) is inputComparing the learned sequence recommendation model, coding by a sequence coder, finally obtaining the preference scores of the user on all commodities by a prediction module, sorting according to the size of the preference scores, recommending the user u based on TOPN from large to small, namely recommending a set of N commodities with the largest preference scores for the next most likely purchased commodity of the user u.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311025453.5A CN117196763A (en) | 2023-08-15 | 2023-08-15 | Commodity sequence recommending method based on time sequence perception self-attention and contrast learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311025453.5A CN117196763A (en) | 2023-08-15 | 2023-08-15 | Commodity sequence recommending method based on time sequence perception self-attention and contrast learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117196763A true CN117196763A (en) | 2023-12-08 |
Family
ID=88995188
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311025453.5A Pending CN117196763A (en) | 2023-08-15 | 2023-08-15 | Commodity sequence recommending method based on time sequence perception self-attention and contrast learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117196763A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117474637A (en) * | 2023-12-28 | 2024-01-30 | 中国海洋大学 | Personalized commodity recommendation method and system based on time sequence diagram convolution network |
| CN117520864A (en) * | 2024-01-08 | 2024-02-06 | 四川易利数字城市科技有限公司 | Multi-feature fusion intelligent matching method for data elements |
| CN118035568A (en) * | 2024-04-12 | 2024-05-14 | 暨南大学 | Intelligent recommendation method for education problems with knowledge mastering degree perception |
-
2023
- 2023-08-15 CN CN202311025453.5A patent/CN117196763A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117474637A (en) * | 2023-12-28 | 2024-01-30 | 中国海洋大学 | Personalized commodity recommendation method and system based on time sequence diagram convolution network |
| CN117474637B (en) * | 2023-12-28 | 2024-04-16 | 中国海洋大学 | Personalized commodity recommendation method and system based on time sequence diagram convolution network |
| CN117520864A (en) * | 2024-01-08 | 2024-02-06 | 四川易利数字城市科技有限公司 | Multi-feature fusion intelligent matching method for data elements |
| CN117520864B (en) * | 2024-01-08 | 2024-03-19 | 四川易利数字城市科技有限公司 | Multi-feature fusion intelligent matching method for data elements |
| CN118035568A (en) * | 2024-04-12 | 2024-05-14 | 暨南大学 | Intelligent recommendation method for education problems with knowledge mastering degree perception |
| CN118035568B (en) * | 2024-04-12 | 2024-07-05 | 暨南大学 | Intelligent recommendation method for education problems with knowledge mastering degree perception |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117196763A (en) | Commodity sequence recommending method based on time sequence perception self-attention and contrast learning | |
| CN111274398A (en) | Method and system for analyzing comment emotion of aspect-level user product | |
| CN108647226B (en) | Hybrid recommendation method based on variational automatic encoder | |
| CN112819604A (en) | Personal credit evaluation method and system based on fusion neural network feature mining | |
| CN112328900A (en) | Deep learning recommendation method integrating scoring matrix and comment text | |
| CN112258262B (en) | Session recommendation method based on convolution self-attention network | |
| CN113627266B (en) | Video pedestrian re-recognition method based on transform space-time modeling | |
| CN114493755B (en) | Self-attention sequence recommendation method fusing time sequence information | |
| CN113628059A (en) | Associated user identification method and device based on multilayer graph attention network | |
| CN110781401A (en) | Top-n project recommendation method based on collaborative autoregressive flow | |
| CN113139850A (en) | Commodity recommendation model for relieving data sparsity and commodity cold start | |
| CN114757432A (en) | Future execution activity and time prediction method and system based on flow log and multi-task learning | |
| CN114239935A (en) | Prediction method for non-uniform track sequence | |
| CN115393968A (en) | Audio-visual event positioning method fusing self-supervision multi-mode features | |
| CN114004220A (en) | Text emotion reason identification method based on CPC-ANN | |
| CN115471771A (en) | Video time sequence action positioning method based on semantic level time sequence correlation modeling | |
| CN114117029B (en) | Solution recommendation method and system based on multi-level information enhancement | |
| CN116976505A (en) | Click rate prediction method of decoupling attention network based on information sharing | |
| CN114662652A (en) | Expert recommendation method based on multi-mode information learning | |
| US20220207321A1 (en) | Convolution-Augmented Transformer Models | |
| CN115952360B (en) | Domain self-adaptive cross-domain recommendation method and system based on user and article commonality modeling | |
| CN112818256A (en) | Recommendation method based on neural collaborative filtering | |
| CN112052795A (en) | Video behavior identification method based on multi-scale space-time feature aggregation | |
| CN115374283A (en) | Double-graph attention network-based aspect category emotion classification method | |
| CN116257786A (en) | Asynchronous time sequence classification method based on multi-element time sequence diagram structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |