CN111881359B - Ordering method, ordering system, ordering equipment and ordering storage medium in internet information retrieval - Google Patents
Ordering method, ordering system, ordering equipment and ordering storage medium in internet information retrieval Download PDFInfo
- Publication number
- CN111881359B CN111881359B CN202010773050.9A CN202010773050A CN111881359B CN 111881359 B CN111881359 B CN 111881359B CN 202010773050 A CN202010773050 A CN 202010773050A CN 111881359 B CN111881359 B CN 111881359B
- Authority
- CN
- China
- Prior art keywords
- model
- ranking
- evaluation index
- ndcg
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000011156 evaluation Methods 0.000 claims abstract description 74
- 238000012163 sequencing technique Methods 0.000 claims abstract description 66
- 238000012549 training Methods 0.000 claims abstract description 60
- 239000011159 matrix material Substances 0.000 claims description 72
- 239000013598 vector Substances 0.000 claims description 45
- 238000013139 quantization Methods 0.000 claims description 9
- 238000005070 sampling Methods 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 3
- 230000003542 behavioural effect Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 14
- 238000005457 optimization Methods 0.000 abstract description 10
- 238000006243 chemical reaction Methods 0.000 abstract description 4
- 238000012545 processing Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 230000006399 behavior Effects 0.000 description 4
- 230000001186 cumulative effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000013441 quality evaluation Methods 0.000 description 2
- 235000019640 taste Nutrition 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000013210 evaluation model Methods 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009182 swimming Effects 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9538—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0631—Item recommendations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/12—Hotels or restaurants
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Tourism & Hospitality (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Finance (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Primary Health Care (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a sorting method, a system, equipment and a storage medium in internet information retrieval, wherein the method comprises the following steps: obtaining a training sample; constructing a sequencing model; performing iterative training on the sequencing model based on the training sample, generating an evaluation index on the sequencing index, and optimizing the model according to the evaluation index; and inputting search results obtained by the user and queried in real time in the model to obtain the scores and the sequences of the candidate objects. According to the invention, iteration of linear model parameters is realized by using training samples, the model is evaluated in the iteration, and training and optimization of the model are completed through evaluation indexes. The sorting method has high interpretability while improving the self-adaptive learning capability and the sorting effect of sorting, and can clearly reflect the logic basis of the recommended sorting. The user can easily understand the recommended principle, so that stronger recognition feeling is generated, and experience feeling and conversion rate are improved.
Description
Technical Field
The present invention relates to the field of internet retrieval technologies, and in particular, to a method, a system, an apparatus, and a storage medium for ordering in internet information retrieval.
Background
The internet retrieval technology is applied to a plurality of fields including an e-commerce recommendation system, a search engine, a question-answering system and the like. The ranking method plays a very important role in the search technology. Ranking methods have received considerable attention from researchers in recent years because the effects produced by these ranking methods directly affect the user experience of these applications. There are various indexes for evaluating the effect of the sorting method, such as NDCG (Normalized DCG), MRR (Mean Reciprocal Rank, average reciprocal sorting), ERR (Expected Reciprocal Rank, desired reciprocal sorting), and the like. Because of these non-convex, non-leadership issues, students have proposed a variety of methods to optimize the rank learning model, including: (1) Optimizing the rank-learned model with some agent evaluation index (loss function); (2) And optimizing the sorting algorithm in index calculation to be led.
The method for optimizing the model by using the proxy loss function converts the sorting problem into a general classified and regressed machine learning problem, and can optimize the model by using different machine learning models and skills. Representative methods are as follows: listNet, rankNet, SVMRank, DLCM, QILCM, etc. However, these transformed proxy loss functions only form the lower bound of the ranking index, and the optimization process of the model is equivalent to maximizing the lower bound of the ranking index. There is a gap between such conversions and the true evaluation index of the ranking.
Another approach is to optimize the ranking algorithm that calculates the ranking indicators. A representative method is SoftRank. This method converts the deterministic scoring of the candidate into a normal distribution with a mean score, which is equivalent to introducing noise. This approach introduces a certain uncertainty in its optimization.
In addition, the order learning algorithm model with better effect is designed based on a tree model and a neural network model at present. Since the feature dimensions used to construct these models are hidden, the disadvantage is that they are poorly interpretable and do not clearly reflect the basis of ranking and recommendation. In some of the above-mentioned fields of internet retrieval, the interpretability plays a very important role and is an important index for evaluating the ranking method.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide a sequencing method, a sequencing system, sequencing equipment and a storage medium in Internet information retrieval, which have high interpretability while improving sequencing self-adaptive learning capability and sequencing effect.
The embodiment of the invention provides a sorting method in internet information retrieval, which comprises the following steps:
obtaining a training sample, wherein the training sample comprises a feature matrix corresponding to each search result, each feature in the feature matrix is defined as a feature dimension, and the feature dimension comprises the features of a user, the features of a candidate object and the associated features of the user and the candidate object;
constructing a ranking model based on the linear model, wherein the ranking model is configured to rank each candidate object after scoring according to the input feature matrix and the weight of each feature dimension, and output the ranking result of each candidate object;
performing iterative training on the sequencing model based on the training sample, calculating an evaluation index of the sequencing model based on a predicted sequencing result and a reference sequencing result output by the sequencing model, and optimizing the weight of each feature dimension according to the evaluation index;
and acquiring a feature matrix corresponding to the search result queried by the user in real time, and inputting the feature matrix into the sequencing model to obtain the score and sequencing of each candidate object.
Optionally, the embodiment of the present invention performs iterative training on the ranking model based on the training samples in S100, including the following steps:
inputting a feature matrix of a training sample into the sequencing model to obtain a predicted sequencing result of each candidate object output by the sequencing model;
obtaining a first evaluation index by adopting a preset evaluation index quantization algorithm according to the prediction sequencing result;
obtaining a reference sequencing result of each candidate object according to the correlation degree between a preset user and the candidate object;
obtaining a second evaluation index by adopting a preset evaluation index quantization algorithm according to the reference sorting result;
calculating the ratio of the first evaluation index to the second evaluation index as the evaluation index of the ranking model;
and optimizing the weight of the sequencing model according to the evaluation index.
Optionally, the evaluation index quantization algorithm evaluates the sorting result by adopting an ndcg@top K evaluation index, where K is the length of a candidate object list recommended preferentially to the user;
and carrying out quantitative scoring on the prediction sequencing result and the reference sequencing result by adopting the NDCG@Top K evaluation index to respectively obtain the ratio NDCG value of the DCG and the IDCG. The larger the NDCG value, the higher the ranking evaluation index.
Specifically, according to the evaluation index, iterating and optimizing the weight of the sorting model, wherein each iterative optimization comprises the following steps:
calculating the average value mu of a matrix w of weight vectors under each dimension j, and calculating a corresponding covariance matrix C through the weight vector matrix w and the average value mu, wherein C is a real matrix, and C is E R d×d D is the number of feature dimensions, and m times of sampling are carried out by using a multi-element Gaussian distribution N (mu, C) with the mean value of mu and the covariance matrix of C to obtain new m weight vectors wi, wi E R d And (3) obtaining score vectors of all candidate objects under m single weight vectors and corresponding m NDCG values through a sequencing model, wherein each weight vector wi corresponds to one NDCG value, the matrix w of the weight vector corresponds to m NDCG values in total, the optimal N_best NDCG values are obtained, and the N_best weight vectors corresponding to the NDCG values are used for calculating the mean and covariance.
Optionally, the sampling frequency m is a positive integer in the range of [4+log (d), d//2], so as to achieve a better model convergence effect.
Optionally, optimizing the weight of the ranking model according to the evaluation index, and the iterative process includes the following steps:
when the first iteration times are not greater than a preset threshold t, recording the maximum NDCG value as NDCG_best in each iteration, selecting corresponding N_best weight vectors according to the maximum N_best NDCG values, taking m/2 or other reasonable positive integers by N_best, calculating the mean value and covariance C of the weight vectors, and sampling the multi-element Gaussian distribution N (mu, C) with the mean value of mu and covariance matrix of C for m times to generate m weight vectors;
when the iteration times are greater than a preset threshold t, the method comprises the steps of ordering, entering a second iteration, monitoring NDCG_best, updating his_NDCG=NDCG_best until a preset condition is achieved, and outputting the weight vector matrix w_best corresponding to his_NDCG when the iteration is stopped, wherein offline training is completed.
Optionally, the process of implementing the preset condition includes:
and judging whether NDCG_best generated by each iteration is larger than his_NDCG, if so, updating his_NDCG=NDCG_best, if so, increasing the Patient value, and if the Patient value is not updated continuously for Pmax times, taking w_best corresponding to the his_NDCG value at the moment as the weight of the sorting model.
Optionally, when the ranking model is constructed based on the linear model, initial weights of all feature dimensions in the ranking model are obtained by multi-element gaussian random sampling, and m weight vectors { w are generated 1 ，w 2 ，...，w n In each weight w i Has the dimension d, w i The superscript i in (a) denotes the i-th vector.
Optionally, the user features include basic features of the user and query features of the user using the search engine; the basic features of the user include the identity features of the user and the behavioral features of the user.
The embodiment of the invention also provides a sequencing system in internet information retrieval, which comprises the following steps:
the sample acquisition module is used for acquiring a training sample, wherein the training sample comprises a feature matrix corresponding to each search result, each feature in the feature matrix is defined as a feature dimension, and the features in the feature matrix comprise the features of a user, the features of a candidate object and the associated features of the user and the candidate object;
the model construction module is used for constructing a sorting model based on the linear model, wherein the sorting model is configured to score each candidate object according to the input feature matrix and the weight of each feature dimension and then sort the candidate objects, and output the sorting result of each candidate object;
the model training module is used for carrying out iterative training on the sequencing model based on the training sample, calculating an evaluation index of the sequencing model based on a prediction sequencing result and a reference sequencing result output by the sequencing model, and optimizing the weight of each feature dimension according to the evaluation index;
and the result ordering module is used for acquiring a feature matrix corresponding to the search result queried by the user in real time, inputting the feature matrix into the ordering model, and obtaining the score and the ordering of each candidate object.
The embodiment of the invention also provides ordering equipment in internet information retrieval, which comprises the following steps:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the ranking method in internet information retrieval via execution of the executable instructions.
The embodiment of the invention also provides a computer readable storage medium for storing a program, which when executed, implements the steps of the ranking method in internet information retrieval.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
The sorting method, the system, the equipment and the storage medium in the internet information retrieval have the following beneficial effects:
the invention is based on a simple linear model, realizes the self iteration of the linear model parameters by using a training sample, evaluates the model in the iteration process, completes the training and optimization of the model by the obtained evaluation index, and realizes the optimization of the model and the evaluation of the model by the sequencing method in parallel.
The sequencing method disclosed by the invention has high interpretability while improving the self-adaptive learning capability and the sequencing effect. The feature dimension used for constructing the model is explicit, overcomes the defect of poor interpretability, and can clearly reflect the recommended ordering and the logic basis between features. The user can easily understand the recommended principle, so that stronger recognition feeling is generated, and experience feeling and conversion rate are improved.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings.
FIG. 1 is a flow chart of a method of ranking in Internet information retrieval according to an embodiment of the present invention;
FIG. 2 is a logical schematic diagram of a ranking method in Internet information retrieval according to an embodiment of the present invention;
FIG. 3 is a schematic diagram showing the ordering method in Internet information retrieval according to an embodiment of the present invention;
FIG. 4 is a flow diagram of a ranking system in Internet information retrieval according to an embodiment of the invention;
FIG. 5 is a schematic diagram of a ranking apparatus in Internet information retrieval according to an embodiment of the present invention;
fig. 6 is a schematic structural view of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.
As shown in fig. 1, an embodiment of the present invention provides a sorting method in internet information retrieval, including the following steps:
s100: obtaining a training sample, wherein the training sample comprises a feature matrix corresponding to each search result, each feature in the feature matrix is defined as a feature dimension, and the feature dimension comprises the features of a user, the features of a candidate object and the associated features of the user and the candidate object;
s200: constructing a ranking model based on the linear model, wherein the ranking model is configured to rank each candidate object after scoring according to the input feature matrix and the weight of each feature dimension, and output the ranking result of each candidate object;
s300: performing iterative training on the sequencing model based on the training sample, calculating an evaluation index of the sequencing model based on a predicted sequencing result and a reference sequencing result output by the sequencing model, and optimizing the weight of each feature dimension according to the evaluation index;
s400: and acquiring a feature matrix corresponding to the search result queried by the user in real time, and inputting the feature matrix into the sequencing model to obtain the score and sequencing of each candidate object.
In the method of this embodiment, the serial number of each step is merely used to distinguish the steps, and is not limited to the specific execution sequence of the steps, and the execution sequence between the steps may be adjusted and changed as required.
An embodiment of the ranking method in internet information retrieval is described below with reference to fig. 2. The logic diagram shown in fig. 2 is only an example of ranking by hotel, and should not be construed as limiting the function and scope of use of the embodiment of the present invention, and the candidates for ranking may be any candidates for ranking, including but not limited to documents, web pages, commodities, shops, etc.
As shown in FIG. 1, the ranking method of the present invention can be roughly divided into an offline training section and an online ranking section. The off-line training part can be divided into S100 for obtaining hotel sequencing training samples, S200 for constructing a sequencing model, and S200 for carrying out iterative training based on the training samples to generate parameters of the model. The on-line sorting part S400 mainly copies the model after off-line training to an on-line model, acquires a feature matrix corresponding to the candidate object result queried by the user in real time, inputs the feature matrix into the sorting model, and obtains the score and the sorting of each candidate object. The method specifically comprises the following steps:
s100, as shown in FIG. 3, obtaining a training sample comprising feature matrixes corresponding to candidate hotels, wherein each feature in the feature matrixes is defined as a feature dimension, and each feature is defined as a feature dimensionThe symptom dimension is a ranking factor that affects ranking. The feature dimension is explicit and comprises the features of the user, the features of the candidate hotels and the associated features of the user and the candidate hotels; compared with recessive models, the models trained by linear feature dimensions have higher interpretability, so that a user can understand the reasons behind the recommendation more easily, and stronger recognition is generated; the characteristic matrix is a real matrix, and X epsilon R is used n×d It is indicated that each element in the matrix is a real number. n is the number of candidate hotels obtained by user inquiry, d is the number of hotel feature dimensions in the training sample, and x is used for the number of candidate hotels i Representing the ith candidate hotel
The characteristics of the user comprise basic characteristics of the user and query characteristics of the user;
the user basic characteristics comprise one or more of identity characteristics such as gender, age, user star level, registration duration and the like of the user, and one or more of behavior characteristics such as clicking, browsing, ordering, paying, canceling and the like of the user. The inquiry characteristics of the user comprise preference setting and filtering options of the user for inquiring the hotel, such as one or more of floors, rooms, gyms, swimming pools, chinese and western restaurants and the like;
the association features of the user and the candidate hotels comprise one or more of features of the hotels collected by the user, features of the hotels ordered by the user, features of the hotels clicked by the user, behavior features of other users similar to the interests and tastes of the user, features of other hotels similar to the candidate hotels of interest to the user, and the like;
the candidate hotel features include one or more of candidate hotel category, region, distance to traffic hub, historical user score, price location in the same class of candidate hotels, etc.
For example, x 1 For one candidate of all n candidates, which has d feature dimensions, the feature matrix thus constructed is expressed as:
from equation oneAny candidate x i Expressed as all feature dimensions:
s200, as shown in FIG. 3, constructing a ranking model based on a linear model, wherein the ranking model is configured to score each candidate object according to an input feature matrix and the weight of each feature dimension, and output the ranking result of each candidate object according to the score;
assuming that the ordering model is f and the parameter is w epsilon R d R represents a real matrix, where each element is a real number. The score vector of a model for a candidate i is expressed as
s i ＝f(x i )＝x i W (formula IV)
Where f represents a scoring model, represents dot product, and represents the score of all hotels in combination as a matrix:
s=f (X) =x·w (formula five)
Wherein S is E R nxm Score matrix for all candidates, w ε R dxm The weight matrix consists of m weight vectors w. And S is obtained, and hotels can be ranked according to the score of each candidate object.
The above represents a linear scoring model where the only unknown parameter is w. The self-evaluation and training process of the model is that of solving and optimizing w.
S300, performing iterative training on the sorting model in S200 based on the training sample in S100, solving and optimizing w, as shown in FIG. 3, and comprising the following steps:
s310, inputting a feature matrix X of a training sample into the ranking model f to obtain a predicted ranking result of each candidate object output by the ranking model, wherein the predicted ranking result comprises the following steps:
obtaining m weight vectors { w 1 ，w 2 ，...，w m Any one of w i (i is more than 1 and less than or equal to m) satisfies w i ∈R d By the formula four of S200,calculating a score vector s of the candidate object under each weight vector i ∈R n 1 < i.ltoreq.m, representing the scores of the n candidate objects under the weight vector i, respectively, and s i In which n elements are ordered to obtain a predicted ordering of candidate objectsSimilarly, m prediction orders can be obtained using a w weight matrix.
S320, obtaining a reference ordering result of each candidate object according to the correlation degree between a preset user and the candidate object, wherein the reference ordering result comprises the following steps:
obtaining Y epsilon R n Representing the ordering of the relevance of the candidate objects, where y= { Y 1 ，y 2 ，...，y n }. Any y i Representing the relevance of the i candidate object to the user. The relevance of the same user and different candidate objects is independent. The relevance of the same candidate object to different users is also independent.
Using S310, the reference ranking { r } of the similarly available candidates 1 ，r 2 ，…r n }，1＜r≤n；
The value of the correlation is preset by a number of influencing factors, as and only as a reference or contrast to the ranking. Different Y have different effects on the ranking evaluation in the ranking method. The influence factor on decision Y may be different from the choice of feature dimensions for decision X, and the role in the ordering method is also different and should not be confused. Y may be obtained by a variety of methods, such as expert evaluation or deep learning models, etc. It should be noted that the ordering of the same candidate set, generated by comparison with different Y's, may be different, which may not be interpretable. In the invention, the constant Y is used as a standard for evaluating the ranking method, in other words, when Y is changed, the ranking method of the invention should be retrained and optimized. But the Y value remains unchanged during the training, optimization process.
S330, selecting an evaluation index of the sorting model, scoring the predicted sorting generated by the sorting model in S310 to obtain a first evaluation index, and obtaining a second evaluation index by adopting a preset evaluation index quantization algorithm according to the reference sorting result, wherein the specific steps comprise:
obtaining a breakage accumulated gain according to the prediction ranking result by adopting a preset evaluation index quantization algorithm, wherein the breakage accumulated gain is expressed as DCG (Discounted Cumulative Gain, an ranking evaluation index), obtaining IDCG (Ideal Discounted Cumulative Gain, an ideal measurement search engine evaluation index) according to the reference ranking result by adopting a preset evaluation index quantization algorithm, and the evaluation index is expressed as IDCG, and calculating the ratio of the DCG to the IDCG as the evaluation index NDCG (Normalized Discounted Cumulative Gain) of the ranking model specifically comprises:
and selecting NDCG@K as an evaluation index of the quantitative ordering effect. NDCG represents normalized loss cumulative gain, with each score being taken as a gain. For items without user feedback, its gain is typically set to 0.K is the length of the candidate list that is preferentially recommended to the user. And accumulating the gains of K candidate objects in the list, and dividing the gains by a break value to obtain a DCG value. Taking the IDCG obtained by the preset correlation degree Y as a reference, and taking the NDCG as the ratio of DCG to IDCG. The range of NDCG values is (0, 1), the larger the value, the higher the ranking evaluation index:
as shown in formula six, DCG can be obtained by predicting the ranking result. Where K may be a reasonable positive integer, taking k=10 as an example, and the quantized evaluation index is dcg@10. The sum sigma represents the sum after the correlation and ranking calculation corresponding to the top 10 candidate objects are taken, rel j The relevance of the candidate in the j position in the rank. For the use of w i Calculated score vector s i ，rel j Is s i The j candidate object of the list is scored, and the score is ordered according to the rule from big score to small score, so as to obtain the ordering position of each candidate objectAnd a relevance score corresponding to the location.
Ranking according to the relevance Y thereof to obtain the ranking { r } of each candidate object 1 ，r 2 ，...r j ...r n And j is more than 1 and less than or equal to n. The ranking and the relevance are used for calculating IDCG@10 of the candidate object, and the calculation formula is as follows:
for example, if the query results are candidate objects D1, D2, D3, D4, D5, D6, and the relevance scores are {3,2,3,0,1,2}, respectively, DCG 6 8.10:
CG 6 ＝3+2+3+0+1+2＝11
if the reference correlation degree of the candidate object in the query result is {3,3,2,2,1,0}, calculating IDCG 6 8.69.
And calculating the ratio of the DCG to the IDCG as an evaluation index NDCG to obtain DCG/IDCG=0.932.
As previously described, a w i Generating an NDCG. Similarly, m NDCGs can be obtained using the weight matrix w, where the optimal NDCG is denoted as ndcg_best.
It is to be understood that when other evaluation methods are employed, the first quality evaluation index and the second quality evaluation index are not limited to DCG and IDCG, and are different depending on the evaluation algorithm employed.
S340, optimizing the weight of the sorting model according to iteration of the evaluation index, comprising the following steps:
as shown in fig. 3, the threshold value of the first iteration is preset to be t, where t may be 50, 100 or other reasonable positive integers, so that the weight vector and the corresponding ordering effect after t iterations are approximately close to the target range. The magnitude of m determines the magnitude of the computation and affects the convergence effect of the model, and in one embodiment, the value of m is [4+log (d), d//2], and d//2 represents an integer. The optimal ndcg_best in the last iteration is denoted his_ndcg.
When the iteration number is smaller than a preset threshold t, recording NDCG_best in the iteration. Obtaining N_best weight vectors w corresponding to the largest N_best NDCG values i N_best takes m/2 or other reasonable positive integers, calculates the mean value and covariance C of the N_best weight vector, samples the multi-element Gaussian distribution N (mu, C) with the mean value of mu and the covariance matrix of C for m times, and generates m new weight vectors; the m weight vectors are input as weight vectors in S310 for the next iteration.
When the iteration times are larger than a preset threshold t, the ordering method enters a second iteration, and whether the NDCG value at the moment meets the termination condition or not is judged, wherein the threshold of the preset second iteration is Pmax. And monitoring the NDCG_best and updating his_NDCG=NDCG_best until a preset condition is realized, and outputting a weight vector matrix w_best corresponding to the his_NDCG when the iteration is stopped. The process for realizing the preset condition comprises the following steps:
and judging whether NDCG_best generated by each iteration is larger than his_NDCG, if so, updating his_NDCG=NDCG_best, and if so, increasing the Patience value. If the continuous Pmax times Patience value is not updated, taking the w_best corresponding to the his_NDCG value at the moment as the weight of the sorting model.
Furthermore, in an embodiment of the present invention, the maximum value Pmax and the maximum value t of the Patience can be adjusted during the implementation of the method, so as to reduce the operation amount, increase the effective calculation, and improve the convergence efficiency.
In another embodiment of the present invention, the first iteration and the second iteration are combined into a third iteration, and the third iteration implements the functions of all the first iteration and the second iteration, that is, both calculates a new NDCG value in each iteration and determines whether a termination condition is satisfied, where the termination condition uses the update frequency of the NDCG value or the difference between the new and old values during each update as a scale.
In other embodiments, S330 may select other evaluation indexes such as MAP as a method for quantifying the ranking effect.
S400, acquiring a feature matrix corresponding to a search result queried by a user in real time, and inputting the feature matrix into the ranking model to obtain the score and ranking of each candidate object.
In one embodiment of the invention, the initial weight of each feature dimension in the ranking model is obtained by multi-element Gaussian random sampling, and m weight vectors { w 1 ，w 2 ，...，w m In each weight w i Has the dimension d, w i The superscript i in (a) denotes the i-th vector. The size of m determines the magnitude of the calculation amount and affects the convergence effect of the model.
The mean value of each dimension j of the weight matrix w, and its corresponding covariance matrix:
where j represents any feature dimension and i represents the i-th weight vector. The covariance matrix is generated as C E R d×d The element is sigma. Randomly sampling a multi-element Gaussian distribution N (mu, C) with a mean value mu and a covariance matrix C for m times and generating new m weight vectors { w 1 ，w 2 ，...，w m }。
In another embodiment of the present invention, the associated features of the user and the candidate object include one or more of the features of the object collected by the user, the features of the object ordered by the user, the features of the object clicked by the user, the behavior features of other users similar to the interests and tastes of the user, the features of other objects similar to the candidate object of interest to the user, and so on;
the basic characteristics of the user comprise one or more of identity characteristics such as gender, age, user star level, registration duration and the like of the user and one or more of behavior characteristics such as clicking, browsing, ordering, paying, canceling and the like of the user, and the inquiry characteristics of the user comprise one or more of preference setting and filtering options of the user using a search engine;
the candidate object features comprise one or more of candidate object category, region, distance from traffic hub, historical user score, price and similar candidate object price.
The invention provides a sort system in internet information retrieval, as shown in fig. 4, the sort system comprises:
the sample acquisition module M100 is used for acquiring a training sample, wherein the training sample comprises a feature matrix corresponding to each search result, each feature in the feature matrix is defined as a feature dimension, and the features in the feature matrix comprise the features of a user, the features of a candidate object and the associated features of the user and the candidate object;
the model construction module M200 is used for constructing a ranking model based on a linear model, wherein the ranking model is configured to score each candidate object according to the input feature matrix and the weight of each feature dimension and rank the candidate objects, and output the ranking result of each candidate object;
the model training module M300 is used for carrying out iterative training on the sequencing model based on the training sample, calculating an evaluation index of the sequencing model based on a predicted sequencing result and a reference sequencing result output by the sequencing model, and optimizing the weight of each characteristic dimension according to the evaluation index;
the result ordering module M400 is used for acquiring a feature matrix corresponding to the search result queried by the user in real time, inputting the feature matrix into the ordering model, and obtaining the score and the ordering of each candidate object.
In the sorting system in internet information retrieval of the present invention, the functions of each module may be implemented by adopting the specific implementation manner of the sorting method in internet information retrieval as described above, which is not described herein. For example, the sample acquiring module M100 may implement acquiring a training sample by using the embodiment of step S100, the model building module M200 may build a ranking model by using the embodiment of step S200, the model training module M300 may perform training optimization on the model by using the embodiment of step S300, and the result ranking module M400 may score and rank search results queried by the user in real time by using the embodiment of step S400.
The invention also provides a sorting device in internet information retrieval, which is characterized by comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the ranking method in any one of the embodiments based on internet information retrieval via execution of the executable instructions.
An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 5. The electronic device 600 shown in fig. 5 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 5, the electronic device 600 is embodied in the form of a general purpose computing device. Components of electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different system components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.
Wherein the storage unit stores program code that is executable by the processing unit 610 such that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the ranking method section of internet information retrieval described above in the present specification. For example, the processing unit 610 may perform the steps as shown in fig. 1.
The memory unit 620 may include readable media in the form of volatile memory units, such as Random Access Memory (RAM) 6201 and/or cache memory unit 6202, and may further include Read Only Memory (ROM) 6203.
The storage unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 630 may be a local bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or using any of a variety of bus architectures.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 600, and/or any device (e.g., router, modem, etc.) that enables the electronic device 600 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 650. Also, electronic device 600 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 over the bus 630. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 600, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
The embodiment of the invention also provides a computer readable storage medium for storing a program, which when executed, implements the steps of the ranking method in internet information retrieval. In some possible embodiments, the various aspects of the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the invention as described in the ranking method section of the internet information retrieval described herein above, when said program product is executed on the terminal device.
Referring to fig. 6, a program product 800 for implementing the above-described method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be executed on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable storage medium may also be any readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
In summary, by adopting the sorting method, the system, the device and the storage medium in the internet information retrieval, the self-iteration of the linear model parameters is realized by utilizing the training sample based on the simple linear model, the model is evaluated in the iteration process, the training and the optimization of the model are completed by the evaluation index, and the optimization of the model and the evaluation of the model by the sorting method are realized in parallel. The sequencing method disclosed by the invention has high interpretability while improving the self-adaptive learning capability and the sequencing effect. The parameters used for constructing the model are explicit, the defect of poor interpretability is overcome, and the logic basis of recommendation ordering can be clearly reflected. The user can easily understand the recommended principle, so that stronger recognition feeling is generated, and experience feeling and conversion rate are improved.
The foregoing is a further detailed description of the invention in connection with the preferred embodiments, and it is not intended that the invention be limited to the specific embodiments described. It will be apparent to those skilled in the art that several simple deductions or substitutions may be made without departing from the spirit of the invention, and these should be considered to be within the scope of the invention.
Claims (13)
1. A method for ranking in internet information retrieval, comprising:
obtaining a training sample, wherein the training sample comprises a feature matrix corresponding to each search result, each feature in the feature matrix is defined as a feature dimension, and the feature dimension comprises the features of a user, the features of a candidate object and the associated features of the user and the candidate object;
constructing a ranking model based on a linear model, wherein the ranking model is configured to rank each candidate object after scoring according to the input feature matrix and the weight of each feature dimension, and output the ranking result of each candidate object;
performing iterative training on the sequencing model based on the training sample, calculating an evaluation index of the sequencing model based on a predicted sequencing result and a reference sequencing result output by the sequencing model, and optimizing the weight of each feature dimension according to the evaluation index;
and acquiring a feature matrix corresponding to the search result queried by the user in real time, and inputting the feature matrix into the ranking model to obtain the score and ranking of each candidate object.
2. The method for ranking in internet information retrieval according to claim 1, wherein the iterative training of the ranking model based on the training samples comprises the steps of:
inputting a feature matrix of a training sample into the sequencing model to obtain a predicted sequencing result of each candidate object output by the sequencing model;
obtaining a first evaluation index by adopting a preset evaluation index quantization algorithm according to the prediction sequencing result;
obtaining a reference sequencing result of each candidate object according to the correlation degree between a preset user and the candidate object;
obtaining a second evaluation index by adopting a preset evaluation index quantization algorithm according to the reference sorting result;
calculating the ratio of the first evaluation index to the second evaluation index as the evaluation index of the ranking model;
and optimizing the weight of the sequencing model according to the evaluation index.
3. The ranking method in internet information retrieval according to claim 2, wherein the evaluation index quantization algorithm evaluates the ranking result by using ndcg@top K evaluation index, K being the length of a candidate object list recommended preferentially to the user;
quantitatively scoring the predicted sorting result by adopting the NDCG@Top K evaluation index to obtain the first evaluation index, and quantitatively scoring the reference sorting result by adopting the NDCG@Top K evaluation index to obtain the second evaluation index;
calculating the ratio of the first evaluation index to the second evaluation index as an NDCG value, wherein the range of the NDCG value is (0, 1), and the larger the value is, the higher the ordered evaluation indexes are.
4. A ranking method in internet information retrieval according to claim 3, characterized in that the weights of the ranking model are iterated, optimized according to the evaluation index, each iteration comprising the steps of:
calculating the average value mu of a matrix w of weight vectors under each dimension j, and calculating a corresponding covariance matrix C through the weight vector matrix w and the average value mu, wherein C is a real matrix, and C is E R d×d D is the number of feature dimensions, and m times of sampling are carried out by using a multi-element Gaussian distribution N (mu, C) with the mean value of mu and the covariance matrix of C to obtain new m weight vectors wi, wi E R d ，1<And i is less than or equal to m, score vectors of all candidate objects under m single weight vectors and corresponding m NDCG values are obtained through a sequencing model.
5. The method according to claim 4, wherein the matrix w of weight vectors corresponds to m NDCG values in total, n_best NDCG values are taken, and the average and covariance are calculated using n_best weight vectors corresponding to the NDCG values.
6. The method of claim 4, wherein the number of samples m is a positive integer in the range of [4+log (d), d//2].
7. The ranking method in internet information retrieval according to claim 4, wherein optimizing the weight of the ranking model according to the evaluation index comprises the steps of:
when the first iteration times are not greater than a preset threshold t, recording the maximum NDCG value as NDCG_best in each iteration, selecting corresponding N_best weight vectors according to the maximum N_best NDCG values, taking m/2 or other reasonable positive integers by N_best, calculating the mean value of the weight vectors and the corresponding covariance C, sampling m times from the multiple Gaussian distribution N (mu, C), and generating m weight vectors;
when the iteration times are greater than a preset threshold t, the ordering method enters a second iteration, monitors NDCG_best and updates his_NDCG=NDCG_best until a preset condition is achieved, and when the iteration is stopped, the weight vector matrix w_best corresponding to his_NDCG is output, and offline training is completed.
8. The method for ranking in internet information retrieval according to claim 7, wherein the process of implementing the preset condition comprises:
and judging whether the NDCG_best generated by each iteration is larger than the his_NDCG, if so, updating the his_NDCG=NDCG_best, if so, increasing a Patient value, and if not, taking the w_best corresponding to the his_NDCG value at the moment as the weight of the sorting model.
9. The method according to claim 1, wherein when the linear model is used to construct the ranking model, initial weights of feature dimensions in the ranking model are obtained by multi-element gaussian random sampling, and m weight vectors { w are generated 1 ,w 2 ,…,w m In each weight w i Has the dimension d, w i The superscript i in (a) denotes the i-th vector.
10. The method of ranking in internet information retrieval according to claim 1, wherein the user features include basic features of the user and query features of the user using a search engine; the basic features of the user include the identity features of the user and the behavioral features of the user.
11. A ranking system in internet information retrieval, characterized in that it is applied to the ranking method in internet information retrieval according to any one of claims 1 to 10, the system comprising:
the sample acquisition module is used for acquiring a training sample, wherein the training sample comprises a feature matrix corresponding to each search result, each feature in the feature matrix is defined as a feature dimension, and the features in the feature matrix comprise the features of a user, the features of a candidate object and the associated features of the user and the candidate object;
the model construction module is used for constructing a sorting model based on the linear model, wherein the sorting model is configured to score each candidate object according to the input feature matrix and the weight of each feature dimension and then sort the candidate objects, and output the sorting result of each candidate object;
the model training module is used for carrying out iterative training on the sequencing model based on the training sample, calculating an evaluation index of the sequencing model based on a prediction sequencing result and a reference sequencing result output by the sequencing model, and optimizing the weight of each feature dimension according to the evaluation index;
and the result ordering module is used for acquiring a feature matrix corresponding to the search result queried by the user in real time, inputting the feature matrix into the ordering model, and obtaining the score and the ordering of each candidate object.
12. A ranking apparatus in internet information retrieval, comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the ranking method in internet-based information retrieval of any one of claims 1 to 10 via execution of the executable instructions.
13. A computer-readable storage medium storing a program, characterized in that the program when executed implements the steps of the ranking method in internet-based information retrieval according to any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010773050.9A CN111881359B (en) | 2020-08-04 | 2020-08-04 | Ordering method, ordering system, ordering equipment and ordering storage medium in internet information retrieval |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010773050.9A CN111881359B (en) | 2020-08-04 | 2020-08-04 | Ordering method, ordering system, ordering equipment and ordering storage medium in internet information retrieval |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111881359A CN111881359A (en) | 2020-11-03 |
| CN111881359B true CN111881359B (en) | 2023-11-17 |
Family
ID=73210516
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010773050.9A Active CN111881359B (en) | 2020-08-04 | 2020-08-04 | Ordering method, ordering system, ordering equipment and ordering storage medium in internet information retrieval |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111881359B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112488781A (en) * | 2020-11-10 | 2021-03-12 | 北京三快在线科技有限公司 | Search recommendation method and device, electronic equipment and readable storage medium |
| CN112735094B (en) * | 2020-12-17 | 2022-08-09 | 中国地质环境监测院 | Geological disaster prediction method and device based on machine learning and electronic equipment |
| CN112395405B (en) * | 2020-12-30 | 2021-04-27 | 支付宝(杭州)信息技术有限公司 | Query document sorting method and device and electronic equipment |
| CN113077312A (en) * | 2021-04-12 | 2021-07-06 | 上海华客信息科技有限公司 | Hotel recommendation method, system, equipment and storage medium |
| CN113450010A (en) * | 2021-07-07 | 2021-09-28 | 中国工商银行股份有限公司 | Method and device for determining evaluation result of data object and server |
| CN114548442B (en) * | 2022-02-25 | 2022-10-21 | 万表名匠(广州)科技有限公司 | Wristwatch maintenance management system based on internet technology |
| CN114862447A (en) * | 2022-04-08 | 2022-08-05 | 无线生活（杭州）信息科技有限公司 | Advertisement reordering method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110046713A (en) * | 2019-04-19 | 2019-07-23 | 井冈山大学 | Robustness sequence learning method and its application based on multi-objective particle swarm optimization |
| US10568570B1 (en) * | 2019-02-14 | 2020-02-25 | Trungram Gyaltrul Sherpa | Methods and systems for providing a preferred fitness state of a user |
| CN111310039A (en) * | 2020-02-10 | 2020-06-19 | 江苏满运软件科技有限公司 | Method, system, device and storage medium for determining insertion position of recommendation information |
-
2020
- 2020-08-04 CN CN202010773050.9A patent/CN111881359B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10568570B1 (en) * | 2019-02-14 | 2020-02-25 | Trungram Gyaltrul Sherpa | Methods and systems for providing a preferred fitness state of a user |
| CN110046713A (en) * | 2019-04-19 | 2019-07-23 | 井冈山大学 | Robustness sequence learning method and its application based on multi-objective particle swarm optimization |
| CN111310039A (en) * | 2020-02-10 | 2020-06-19 | 江苏满运软件科技有限公司 | Method, system, device and storage medium for determining insertion position of recommendation information |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111881359A (en) | 2020-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111881359B (en) | Ordering method, ordering system, ordering equipment and ordering storage medium in internet information retrieval | |
| CN109087178B (en) | Commodity recommendation method and device | |
| US10592777B2 (en) | Systems and methods for slate optimization with recurrent neural networks | |
| US8775416B2 (en) | Adapting a context-independent relevance function for identifying relevant search results | |
| CN109783738B (en) | Multi-similarity-based hybrid collaborative filtering recommendation method for double-pole-limit learning machine | |
| CN111242310B (en) | Feature validity evaluation method and device, electronic equipment and storage medium | |
| US20070118546A1 (en) | User's preference prediction from collective rating data | |
| EP2860672A2 (en) | Scalable cross domain recommendation system | |
| US20080104000A1 (en) | Determining Utility Functions from Ordinal Rankings | |
| US8010535B2 (en) | Optimization of discontinuous rank metrics | |
| US20100293175A1 (en) | Feature normalization and adaptation to build a universal ranking function | |
| CN112395487B (en) | Information recommendation method and device, computer readable storage medium and electronic equipment | |
| US7895198B2 (en) | Gradient based optimization of a ranking measure | |
| CN113239071B (en) | Retrieval query method and system for scientific and technological resource subject and research topic information | |
| CN111695024A (en) | Object evaluation value prediction method and system, and recommendation method and system | |
| Ibrahim et al. | Hybrid online–offline learning to rank using simulated annealing strategy based on dependent click model | |
| CN112488301A (en) | Food inversion method based on multitask learning and attention mechanism | |
| CN110727872A (en) | Method and device for mining ambiguous selection behavior based on implicit feedback | |
| CN109977292A (en) | Searching method, calculates equipment and computer readable storage medium at device | |
| CN111209469A (en) | Personalized recommendation method and device, computer equipment and storage medium | |
| Neshatian et al. | Using genetic programming for context-sensitive feature scoring in classification problems | |
| CN113449182A (en) | Knowledge information personalized recommendation method and system | |
| CN112836794A (en) | Method, device and equipment for determining image neural architecture and storage medium | |
| CN111445280A (en) | Model generation method, restaurant ranking method, system, device and medium | |
| CN114742062B (en) | Text keyword extraction processing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |