CN107679082A - Question and answer searching method, device and electronic equipment - Google Patents
Question and answer searching method, device and electronic equipment Download PDFInfo
- Publication number
- CN107679082A CN107679082A CN201710769081.5A CN201710769081A CN107679082A CN 107679082 A CN107679082 A CN 107679082A CN 201710769081 A CN201710769081 A CN 201710769081A CN 107679082 A CN107679082 A CN 107679082A
- Authority
- CN
- China
- Prior art keywords
- user
- answer
- information
- multiple candidate
- candidate answers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This specification embodiment discloses question and answer searching method, device and electronic equipment, and the question and answer searching method includes：According to user's question sentence and the action trail information of the user, the DSSM obtained using training, answer corresponding to user's question sentence is obtained.
Description
Technical field
This specification is related to field of computer technology, more particularly to question and answer searching method, device and electronic equipment.
Background technology
With the development of artificial intelligence technology, man-machine interaction scene is more and more.In some of scenes, user can lead to
Cross natural language and directly carry out man-machine interaction.For example, user carries out ticket booking consulting, weather forecast query, doctor by voice assistant
Consulting is treated, facility is brought to user's daily life.
In the prior art, when carrying out man-machine interaction based on natural language, smart machine or software carry generally according to user
The user's question sentence gone out, string matching is carried out in default each standard question sentence, matching degree highest standard question sentence is corresponding
Model answer as answer corresponding to user's question sentence.
Based on prior art, it is desirable to be able to more accurately obtain the scheme of answer corresponding to user's question sentence.
The content of the invention
This specification embodiment provides question and answer searching method, device and electronic equipment, for solving following technical problem：
It is required to more accurately obtain the scheme of answer corresponding to user's question sentence.
In order to solve the above technical problems, what this specification embodiment was realized in：
The question and answer searching method that this specification embodiment provides, including：
Receive user's question sentence；
Obtain user behavior trace information and multiple candidate answers；
According to user behavior trace information, train what is obtained using based on user's question and answer information and user behavior trace information
Depth structure semantic model (Deep Structured Semantic Model, DSSM), calculates the multiple candidate answers
Score corresponding to respectively；
According to the text feature of the score and the multiple candidate answers, the multiple candidate answers are ranked up,
And the answer according to corresponding to the ranking results obtain user's question sentence.
The question and answer searcher that this specification embodiment provides, including：
Receiving module, receive user's question sentence；
Data obtaining module, obtain user behavior trace information and multiple candidate answers；
Grading module, according to user behavior trace information, using based on user's question and answer information and user behavior trace information
Obtained DSSM is trained, calculates score corresponding to the multiple candidate answers difference；
Reordered module, and according to the text feature of the score and the multiple candidate answers, the multiple candidate is answered
Case is ranked up, and the answer according to corresponding to the ranking results obtain user's question sentence.
The a kind of electronic equipment that this specification embodiment provides, including：
At least one processor；And
The memory being connected with least one processor communication；Wherein,
The memory storage has can be by the instruction of at least one computing device, and the instruction is by described at least one
Individual computing device, so that at least one processor can：
To receive user's question sentence；
Obtain user behavior trace information and multiple candidate answers；
According to user behavior trace information, train what is obtained using based on user's question and answer information and user behavior trace information
DSSM, calculate score corresponding to the multiple candidate answers difference；
According to the text feature of the score and the multiple candidate answers, the multiple candidate answers are ranked up,
And the answer according to corresponding to the ranking results obtain user's question sentence.
Above-mentioned at least one technical scheme that this specification embodiment uses can reach following beneficial effect：Due to based on
User behavior trace information and text feature, answer search is carried out using DSSM, so as to be advantageous to more accurately obtain user
Answer corresponding to question sentence.
Brief description of the drawings
In order to illustrate more clearly of this specification embodiment or technical scheme of the prior art, below will to embodiment or
The required accompanying drawing used is briefly described in description of the prior art, it should be apparent that, drawings in the following description are only
Some embodiments described in this specification, for those of ordinary skill in the art, do not paying creative labor
Under the premise of, other accompanying drawings can also be obtained according to these accompanying drawings.
Fig. 1 is a kind of overall architecture schematic diagram that the scheme of this specification is related under a kind of practical application scene；
Fig. 2 is a kind of schematic flow sheet for question and answer searching method that this specification embodiment provides；
Fig. 3 is a kind of schematic flow sheet for DSSM training methods that this specification embodiment provides；
Fig. 4 is a kind of structural representation for DSSM that this specification embodiment provides；
Fig. 5 is a kind of principle schematic for model that reorders that this specification embodiment provides；
Fig. 6 is the programme of work schematic diagram for the question and answer search system that this specification embodiment provides；
Fig. 7 is a kind of structural representation for question and answer searcher that this specification embodiment provides；
Fig. 8 a, 8b are the question and answer search plan that this specification is used under the practical application scene that this specification embodiment provides
Effect diagram.
Embodiment
This specification embodiment provides question and answer searching method, device and electronic equipment.
In order that those skilled in the art more fully understand the technical scheme in this specification, below in conjunction with this explanation
Accompanying drawing in book embodiment, the technical scheme in this specification embodiment is clearly and completely described, it is clear that described
Embodiment be only some embodiments of the present application, rather than whole embodiment.Based on this specification embodiment, this area
The every other embodiment that those of ordinary skill is obtained under the premise of creative work is not made, should all belong to the application
The scope of protection.
Fig. 1 is a kind of overall architecture schematic diagram that the scheme of this specification is related under a kind of practical application scene.This is whole
The workflow of body framework mainly includes：According to user's question sentence and its corresponding user behavior trace information, DSSM equipment is utilized
Multiple candidate answers are scored, obtain each candidate answers respectively corresponding to score, send to give by network and reorder
Simulator is reordered, to obtain answer corresponding to user's question sentence.
Based on the overall architecture, the scheme of this specification is described in detail below.
This specification embodiment provides a kind of question and answer searching method, and Fig. 2 is the schematic flow sheet of the question and answer searching method,
The flow may comprise steps of：
S202：Receive user's question sentence.
S204：Obtain user behavior trace information and multiple candidate answers.
User behavior trace information can refer to the operation note that user leaves when carrying out business operation.Preferably, user
Action trail information can be related with corresponding user's question sentence, is so easy to obtain the use according to the user behavior trace information
Answer corresponding to the question sentence of family.
For example, after user queried the debt of oneself, it is desirable to refund, but do not know how to carry out refund operation, then, use
Family have sent the user's question sentence how inquiry refunds；In this case, user behavior trace information can be that debt inquiry is dynamic
Recorded corresponding to making.
S206：According to user behavior trace information, trained using based on user's question and answer information and user behavior trace information
Obtained DSSM, calculate score corresponding to the multiple candidate answers difference.
DSSM is a kind of deep neural network model being modeled based on semantic similarity.
In this specification embodiment, train to obtain DSSM using user's question and answer information and user behavior trace information.
Further, when searching for answer, the corresponding user behavior according to the user's question sentence being currently received with acquisition
Trace information, the score of multiple candidate answers is calculated using DSSM, candidate answers are ranked up according to score, in order to screen
Go out be more likely accurate answer candidate answers.
, can be using user behavior trace information as the auxiliary for helping to understand user's question sentence in this specification embodiment
Information, and then more abundant foundation is provided to candidate answers scoring for DSSM, be advantageous in not clear enough the feelings of user's question sentence
Relatively accurate answer is obtained under condition.
S208：According to the text feature of the score and the multiple candidate answers, the multiple candidate answers are carried out
Sequence, and the answer according to corresponding to the ranking results obtain user's question sentence.
It has been mentioned hereinbefore that DSSM is based on semanteme, in actual applications, further can also utilize beyond semanteme
Text feature, with help more accurately obtain user's question sentence corresponding to answer.
In this specification embodiment, for step S202, except receiving user's question sentence, user's question sentence pair can also be obtained
The user answered identifies (user ID) and/or session identification (session ID), in order to inquire about user behavior rail corresponding to acquisition
Mark information.
Session identification corresponding to user's question sentence can be the session mark between the executive agent of flow in the user and Fig. 2
Session identification between knowledge or the user and other one or more operation systems.
Further, for step S204, the acquisition user behavior trace information, can specifically include：According to described
User's mark and/or session identification, obtain user behavior trace information corresponding to user's question sentence.
For example, the user behavior trace information can include the user behavior tracing point in a period of time, it is preferable that institute
It can be recent a period of time to state a period of time.Such as at the time of receiving user's question sentence before 30 minutes in etc..
User behavior tracing point can refer to：By the executive agent or the one or more operation system notes of the others
The business operation record of the user of record at a time.
Recorded for example, it can be remote procedure call protocol (Remote Procedure Call Protocol, RPC)
And/or URL (Uniform Resource Locator, URL) record etc..Pass through typically, for user
APP accesses the scene of corresponding server, can be recorded using corresponding RPC as user behavior tracing point, pass through PC for user
The scene of website corresponding to access, it can be recorded using corresponding URL and be used as user behavior tracing point.
It should be noted that according to user's mark and/or session identification, it is that one kind can to obtain user behavior trace information
The embodiment of choosing, in actual applications, other embodiment can also be used, obtain user behavior trace information.
For example, it can be inquired about to obtain the focus question sentence related to the question sentence according to user's question sentence, according to the proposition of acquisition
The action trail information of the user of the focus question sentence, as user behavior trace information corresponding to user's question sentence.
It is described to obtain multiple candidate answers for step S204 in this specification embodiment, it can specifically include：Root
According to the text feature of user's question sentence, and the text feature of the answer included in the answer set specified, specified described
Answer set in screening obtain multiple candidate answers.
The answer set specified can be screened from bigger answer set obtaining or pre-define and divide
What class obtained.
Further, the text feature according to user's question sentence, and what is included in the answer set specified are answered
The text feature of case, screened in the answer set specified and obtain multiple candidate answers, can specifically included：Using predetermined
Search engine extraction corresponding to user's question sentence the answer set specified；Using LamdaMART algorithms to described
Answer in the answer set specified is ranked up, and obtains the first ranking results；According to first ranking results, screening obtains
Multiple candidate answers.
By the search engine, initial search can be carried out based on user's question sentence and obtains the corresponding answer set specified.
Such as, it is assumed that user's question sentence is " how refunding ", then corresponding specified answer set can such as include " credit card repayment side
The multiple answers related to refund such as method ", " housing loan paying method ".
Sequencing schemes corresponding to first ranking results, can be arranged according to the degree of correlation of answer and user's question sentence height
Sequence, the bigger answer order of degree of correlation is more forward.Use the example above and illustrate, it is assumed that user is exclusive by certain credit card
APP send " how refunding " this question sentence, then it is related to the question sentence according in general experience, " credit card repayment method "
Degree is higher than the degree of correlation of " housing loan paying method " and the question sentence, therefore, in corresponding first ranking results, " credit
Card repayment method " can be sequentially located further forward compared to " housing loan paying method ".
In this specification embodiment, for step S206, user's question and answer information is including problem information and its correspondingly
Answer information (wherein, answer information here can be as the correct option of positive sample or as negative sample
Wrong answer)；User's question and answer information can be based on to train to obtain DSSM with user behavior trace information.
In order to make it easy to understand, illustrated with reference to Fig. 3, Fig. 3 is the schematic flow sheet of DSSM training methods, and the flow can be with
Comprise the following steps：
S302：According to described problem information and user behavior trace information, the first input vector is generated；And
S304：According to answer information corresponding to described problem information, the second input vector is generated；
S306：Train to obtain DSSM using first input vector and second input vector.
In the training process, problem information is combined with corresponding user behavior trace information, will combines what is obtained
Information is as the first input vector, for training DSSM.
This specification embodiment additionally provides a kind of DSSM structural representations, specific as shown in Figure 4.
DSSM in Fig. 4 is the neural network model for including three hidden layers, the number of nodes that three hidden layers include point
It is not：300th, 300,128, compared to the DSSM of standard, the DSSM in Fig. 4 can omit word Hash (Word hashing)
Layer.DSSM training data can be obtained by filtering screening in advance, the training data can be user's question sentence information and its corresponding
User behavior trace information, or the identification information (such as token ID) of these information.
By taking token ID as an example, token ID corresponding to user behavior trace information and user's question sentence information can be pressed
Being combined (for example directly spliced or carry out arithmetic operator etc.) according to certain rule obtains the first input vector, will
The input that first input vector is held as the inquiry (query) of the DSSM；
It is possible to further each answer information according to corresponding to current problem information, one second input is generated respectively
Vector, the input as the DSSM documents (document) end.
For example, showing two the second input vectors in Fig. 4, one of them corresponds to positive sample, and another corresponds to negative
Sample.
It is described to be believed according to described problem information and user behavior track for step S302 in this specification embodiment
Breath, the first input vector is generated, can specifically be included：It is retrieved as the problem of described problem information is specified message identification, Yi Jiwei
The trace information mark that user behavior trace information corresponding to described problem information is specified；According to described problem message identification and rail
Mark message identification, generate the first input vector.
In actual applications, in order to reduce interference, problem information can pass through standardization to some initial data
Obtain.The standardization can such as include the processing such as filtering, capital and small letter conversion, punctuate additions and deletions.
By taking filtering as an example, initial data can be the question sentence in some actual language materials.It may be included in these question sentences
Relatively less important word, this partial words can be removed by filtration treatment.Such as, it is assumed that initial data is：" you get well I
Need to seek advice from film ticket booking flow", the words such as " hello " therein, " I needs to seek advice from " can be filtered out, by remaining " electricity
Movie ticket booking flow" it is used as problem information.
Can be the artificial mark directly specified or extraction for problem information mark and trace information mark
Term vector corresponding to the information that goes wrong or user behavior trace information is as mark etc..It is assumed that artificial direct designated identification, tool
Body can establish respectively in advance problem information and problem information mark between mapping relations, and user behavior trace information with
Mapping relations between trace information mark, specifying for mark is realized by the foundation of the mapping relations.
In this specification embodiment, for step S304, the answer information according to corresponding to described problem information is raw
Into the second input vector, can specifically include：Extract the knowledge dot leader participle of answer information corresponding to described problem information；Root
Segmented according to the knowledge dot leader, generate the second input vector.
Wherein, knowledge dot leader participle can reflect the main semanteme of its corresponding answer information, and it is typically appreciated that
For the descriptor in answer information.
It is described using first input vector and described second defeated for step S306 in this specification embodiment
Incoming vector trains to obtain DSSM, can specifically include：First input vector is inputted into DSSM prototypes, obtains the first output knot
Fruit；And second input vector is inputted into the DSSM prototypes, obtain the second output result；Calculate first output
As a result the correlation between the second output result, and DSSM prototypes are trained according to the correlation, obtain DSSM.
DSSM prototypes can refer to predefined not start to train or have started to train but not yet train the DSSM of completion.
The correlation can have a variety of metric forms, such as, space can also be passed through by cosine computation measure
Vector distance measurement etc..
For example, specific DSSM training can be carried out according to below equation：
li=Wix；
li=f (Wili-l+bi), i=2 ..., N-1；
Y=f (WNlN-1+bN)；
Wherein, liAs representing the i-th hidden layer, WiTo represent the parameter matrix of the i-th hidden layer, biFor being biased towards for the i-th hidden layer
Amount, f (x) generally use tanh functions, y is as output.Calculated with the output at query ends and the cosine of the output at document ends
Value metric correlation.
In this specification embodiment, the text feature according to the score and the multiple candidate answers, to institute
Multiple candidate answers are stated to be ranked up, and the answer according to corresponding to the ranking results obtain user's question sentence, specifically may be used
With including：
According to the text feature of the score and the multiple candidate answers, using LamdaMART algorithms to the multiple
Candidate answers are ranked up, and obtain the second ranking results；According to first ranking results and second ranking results, to institute
State multiple candidate answers to be reordered, obtain the result that reorders；According to the result that reorders, user's question sentence pair is obtained
The answer answered.
The text feature such as can be：Descriptor type, descriptor part of speech, placeholder corresponding to descriptor etc..
Further, it is described according to first ranking results and second ranking results, the multiple candidate is answered
Case is reordered, and is obtained the result that reorders, can specifically be included：
According to first ranking results and second ranking results, the model that reorders obtained using training is to described
Multiple candidate answers are reordered, and obtain the result that reorders；Wherein, the training model that obtains reordering includes：Obtain user's point
Hit daily record；According to user's click logs and the score, using LamdaMART algorithms, training obtains the mould that reorders
Type.
User's click logs include the history click data of active user and/or other users to answer.History hits
According to can specifically include：The statistics such as the number clicked on by each user of each answer corresponding to each question sentence.Click data can
To reflect tendentiousness of the user for answer, this tendentiousness helps to differentiate whether the answer has been hit user and asked exactly
Sentence.
For example, after user sends user's question sentence, it is assumed that executive agent is corresponded to by searching for the final one or more that gives
Answer and return to user, then user is often more likely to click on correct option, rather than wrong answer, in this way, correct option
Number of clicks be typically greater than the number of clicks of wrong answer.
In actual applications, answer is determined compared to being based only upon the first ranking results, or be based only upon the second ranking results
Answer is determined, the answer accuracy rate based on determined by reordering the first ranking results and the second ranking results is often higher,
Illustrated with reference to specific embodiment.
Compared based on identical user's click logs under a kind of practical application scene, the normalization of three set of model is lost tired
Product gain (Normalized Discounted Cumulative Gain, NDCG)@1.Usually, NDCG@1 are worth bigger, represent
Corresponding modelling effect is better), as shown in table 1：
| Specification of a model | NDCG@1 |
| Dssm feature-reranker | 0.46 |
| Base feature-reranker | 0.41 |
| Base+Dssm-reranker | 0.48 |
The NDCG@1 of the set of model of table 1 three are contrasted
Wherein, DSSM feature-reranker can be the model for obtaining the second above-mentioned ranking results, Base
Feature-reranker can be the model for obtaining the first above-mentioned ranking results, and Base+Dssm-reranker can be with
It is the model for obtaining the above-mentioned result that reorders.
From table 1 it follows that the Base+Dssm-reranker values of NDCG@1 are more than Dssm feature-reranker
NDCG@1 be worth, and the NDCG@1 more than Base feature-reranker are worth, thus, it will be seen that based on the knot that reorders
This scheme that fruit obtains answer is relatively more excellent.
In order to which the above-mentioned principle to reorder is better described, this specification embodiment additionally provides the principle for the model that reorders
Schematic diagram, as shown in figure 5, the principle can specifically include：
The input data of model of reordering includes：Current DSSM features (can be specifically above-mentioned score) and answer text
Feature (can be specifically above-mentioned answer information).(can be specifically above-mentioned second row according to ranking results corresponding to DSSM features
Sequence result) and answer text feature corresponding to another ranking results (can be specifically above-mentioned first ranking results), utilize
LamdaMART algorithms, are reordered, and obtain the result that reorders.
Further, with reference to result and the first ranking results of reordering, Utilization strategies model carries out decision-making computing, obtains pair
The answer answered.
Wherein, the model that reorders can be special previously according to the DSSM features of the history as training data, answer text
Click logs of seeking peace train to obtain.
In this specification embodiment, for step S208, after answer corresponding to acquisition user's question sentence, may be used also
To perform：According to reorder result and first ranking results, decision-making is carried out using decision Tree algorithms, obtains the user
Answer corresponding to question sentence.
The decision Tree algorithms are responsible for that answer is chosen and determined between result and the first ranking results reordering.
This specification embodiment is additionally provided under a kind of practical application scene, a kind of specific works side of question and answer search system
Case schematic diagram, as shown in fig. 6, the program can include the step of being related to：
After receive user's question sentence that user sends, preliminary screening is carried out by search engine, preceding 3000 is obtained and answers
Case；
Further, above-mentioned preceding 3000 answer is slightly sorted, obtains preceding 400 answers, can be first 400 by this
Answer is as the above-mentioned answer set specified；Thick sequence can carry out text matches sequence according to word, for example, user sends out
Go out user's question sentence for " how open-minded on-line payment is", when slightly being sorted, will include simultaneously " online ", " payment ", " why
", the answer of " open-minded " this four words come earlier position, other answers are according to including four words
It is how many to be sorted successively.
Further, smart sequence is carried out according to the text feature of preceding 400 answers, obtains multiple candidate answers and its right
The first ranking results answered；According to the multiple candidate answers, and user behavior trace information corresponding to user's question sentence, it is defeated
Enter into DSSM and the plurality of candidate answers are given a mark and sorted, obtain previously described second ranking results and its corresponding
Score；Further, reordered according to score and based on first ranking results, 3 answers before acquisition can should
Preceding 3 answers are as answer corresponding to above-mentioned user's question sentence, it is of course also possible to row for preceding 3 answers and above
Sequence result further carries out decision-making, to determine answer corresponding to user's question sentence.
It should be noted that the quantity of the answer filtered out in Fig. 6 in each step is exemplary, not to the application's
Limit.
Based on identical thinking, this specification embodiment additionally provides a kind of question and answer searcher, and Fig. 7 searches for for the question and answer
The structural representation of device, the device include：
Receiving module 701, receive user's question sentence；
Data obtaining module 702, obtain user behavior trace information and multiple candidate answers；
Grading module 703, according to user behavior trace information, believe using based on user's question and answer information and user behavior track
The depth structure semantic model DSSM that breath training obtains, calculate score corresponding to the multiple candidate answers difference；
The module that reorders 704, according to the text feature of the score and the multiple candidate answers, to the multiple candidate
Answer is ranked up, and the answer according to corresponding to the ranking results obtain user's question sentence.
Due to based on user behavior trace information and text feature, answer search being carried out using DSSM, so as to be advantageous to
More accurately obtain answer corresponding to user's question sentence.
In this specification embodiment, the receiving module 701, user's question sentence is received, in addition to：By receiving module,
Receive user's mark and/or session identification corresponding to user's question sentence.
Session identification corresponding to user's question sentence can be the session mark between the executive agent of flow in the user and Fig. 2
Session identification between knowledge or the user and other one or more operation systems.
In this specification embodiment, described information acquisition module 702, user behavior trace information is obtained, specifically can be with
Including：According to user's mark and/or session identification corresponding to user's question sentence, user behavior trace information is obtained.
In this specification embodiment, the user behavior trace information includes the user behavior track in a period of time
Point, the user behavior tracing point are that remote procedure call protocol RPC is recorded and/or uniform resource position mark URL records.
User behavior tracing point refers to the operation note that user leaves when carrying out business operation.For example, user passes through APP
After access system, the RPC (Remote Procedure Call Protocol, remote procedure call protocol) accessed can be left
Record；After user is by pc access system, URL (Uniform Resource Locator, the unified resource accessed can be left
Finger URL) record.
It is excellent in actual use, it is necessary to gather the user behavior tracing point in a period of time of user's proposition question sentence
Selection of land, collection user propose question sentence at the time of and it is former 30 minutes in data as the user behavior tracing point.
In this specification embodiment, described information acquisition module 702, multiple candidate answers are obtained, can specifically included：
According to the text feature of user's question sentence, and the text feature of the answer included in the answer set specified, in the finger
Screening obtains multiple candidate answers in fixed answer set.
The answer set specified can be obtained or pre-defined and classify to obtain by screening；Base
In the answer set that this is specified, candidate answers are further obtained, efficiency is low when avoiding choosing candidate answers from many answers
The problem of, be advantageous to improve the efficiency for obtaining the candidate answers.
In this specification embodiment, the text feature according to user's question sentence, and the answer set specified
In the text feature of answer that includes, in the answer set specified screening obtain multiple candidate answers, can specifically wrap
Include：The answer set specified using predetermined search engine extraction corresponding to user's question sentence；Utilize LamdaMART
Algorithm is ranked up to the answer in the answer set specified, and obtains the first ranking results；Tied according to the described first sequence
Fruit, screening obtain multiple candidate answers.
By the search engine, initial search can be carried out based on user's question sentence and obtains the corresponding answer set specified.
In this specification embodiment, user's question and answer information includes problem information and its corresponding answer information；Base
Training to obtain DSSM in user's question and answer information and user behavior trace information includes：According to described problem information and user behavior rail
Mark information, generate the first input vector；And the answer information according to corresponding to described problem information, generate the second input vector；
Train to obtain DSSM using first input vector and second input vector.
Described problem information is combined with corresponding user behavior trace information, user behavior trace information is believed as problem
The supplement of breath, it is overall to be used as the first input vector, and then train DSSM, the DSSM carrying out answer search according to customer problem
When, be advantageous to more accurately obtain answer corresponding to user's question sentence.
It is described defeated according to described problem information and user behavior trace information, generation first in this specification embodiment
Incoming vector；The problem of described problem information is specified message identification is retrieved as, and is user behavior corresponding to described problem information
The trace information mark that trace information is specified；Identified according to described problem message identification and trace information, generation first input to
Amount.
Problem information identifies and trace information mark, can be indicated by extracting term vector or artificially
The mark of definition., it is necessary to pre-establish reflecting between problem information and problem information mark when using artificially defined mark
Relation is penetrated, and, the mapping relations between user behavior trace information and trace information mark.Further, according to above-mentioned two
Kind mark, generates the first input vector.
In this specification embodiment, the answer information according to corresponding to described problem information, generation second input to
Amount, can specifically include：Extract the knowledge dot leader participle of answer information corresponding to described problem information；According to the knowledge point
Title segments, and generates the second input vector.
The second input vector is generated using knowledge dot leader participle corresponding to answer information corresponding to problem information is passed through
Mode is trained to DSSM, and knowledge dot leader participle is to go out main information as the answer according to the contents extraction of answer
Title.
It is described to train to obtain using first input vector and second input vector in this specification embodiment
DSSM, it can specifically include：First input vector is inputted into DSSM prototypes, obtains the first output result；And by described in
Second input vector inputs the DSSM prototypes, obtains the second output result；Calculate first output result and the second output
As a result the correlation between, and DSSM prototypes are trained according to the correlation, obtain DSSM.
In the training process, problem information is combined with corresponding user behavior trace information, will combines what is obtained
Information is as the first input vector, for training DSSM.
In this specification embodiment, the text feature according to the score and the multiple candidate answers, to institute
Multiple candidate answers are stated to be ranked up, and the answer according to corresponding to the ranking results obtain user's question sentence, specifically may be used
With including：According to the text feature of the score and the multiple candidate answers, it is ranked up, is obtained using LamdaMART algorithms
Obtain the second ranking results；Reordered according to first ranking results and second ranking results, obtain the knot that reorders
Fruit；According to the result that reorders, answer corresponding to user's question sentence is obtained.
According to matrix corresponding to the score obtained by DSSM or vectorial special with the text of the multiple candidate answers
Matrix corresponding to sign or vector, using LamdaMART algorithms, obtain the second ranking results to multiple candidate answers, Ran Houke
Several answers are sent to user as correct option before choosing second ranking results, can also be by all answers according to row
Sequence is all sent to user.
It is described according to first ranking results and second ranking results in this specification embodiment, to described
Multiple candidate answers are reordered, and are obtained the result that reorders, can specifically be included：According to first ranking results and described
Second ranking results, the model that reorders obtained using training are reordered to the multiple candidate answers, reordered
As a result；Wherein, the training model that obtains reordering includes：Obtain user's click logs；According to user's click logs and described
Score, using LamdaMART algorithms, training obtains the model that reorders.
In this specification embodiment, reorder result described in the basis, obtains answer corresponding to user's question sentence,
It can specifically include：According to reorder result and first ranking results, decision-making is carried out using decision Tree algorithms, obtains institute
State answer corresponding to user's question sentence.By using decision Tree algorithms, chosen reordering between result and the first ranking results, energy
Answer is obtained in the case of being enough effectively lifted at problem information deficiency in accuracy rate.
The embodiment for the Contrast on effect that answer is obtained based on different schemes has been also provided below：
The scheme of this specification leaves action trail information using user before enquirement, to help executive agent preferably to divide
User's question sentence is analysed, so as to be advantageous to improve the accuracy rate of the corresponding answer obtained for the customer problem for being not easy accurate understanding.
Referring to the example in table 2：
First is classified as user behavior trace information, and includes ID corresponding to each action trail point；
Second is classified as user's question sentence, and the 3rd is classified as the answer that a kind of scheme beyond the scheme by this specification obtains,
4th is classified as the answer obtained by the scheme of this specification.
Table 2 is based on DSSM question and answer Contrast on effect
Several examples in above-mentioned table 2, or user's question sentence is said very simple, lack necessary information；It is sufficiently complex
Ground describes a concrete scene, there is excessive interference information.If searching for answer using modes such as text matches merely,
The answer that user really needs is hardly resulted in, still, according to the scheme of this specification, user's row can be explored by DSSM
Deep semantic association between trace information and user's question sentence and corresponding answer, so as to be advantageous to more accurately be answered
Case.
For another example when user's question sentence is " how refunding ", Fig. 8 a and 8b are to use this specification under practical application scene
Question and answer search plan effect diagram, the comparison is as follows：
The question sentence of user only has " how refunding ", and the question sentence is relatively fuzzyyer, because, it is understood that there may be multiple functional units have
Refund function, it can not determine that user wants to refund based on which functional unit, and then, it is difficult to it is right that " how refunding " is provided exactly
The answer (such as refund operating instruction) answered.
In Fig. 8 a, user sends user's question sentence " how refunding " and arrives question and answer robot, and question and answer robot finds the use
The recent user behavior trace information at family is：Access credit loan functional unit → check bill → question and answer robot；Then ask
Answering the answer that robot provides can be：The operating instruction of " how credit loan refunds ".
In figure 8b, " how refunding " also sent and arrives question and answer robot by user, and question and answer robot finds the second user
User behavior trace information be：Access loan functional unit → check bill → question and answer robot；Then question and answer robot provides
Answer can be：The operating instruction of " how loan refunds ".
It can be seen that in Fig. 8 a, Fig. 8 b, user have issued identical user's question sentence, but before this is putd question to twice
The business that user behavior trace information is related to has significant difference, the scheme based on this specification, based on user behavior trace information
Answer is searched for, can more accurately understanding user, this puts question to the true intention of difference twice, so as to give more accurately answer.
Based on identical thinking, this specification embodiment additionally provides a kind of electronic equipment, including：
At least one processor；And
The memory being connected with least one processor communication；Wherein,
The memory storage has can be by the instruction of at least one computing device, and the instruction is by described at least one
Individual computing device, so that at least one processor can：
To receive user's question sentence；
Obtain user behavior trace information and multiple candidate answers；
According to user behavior trace information, train what is obtained using based on user's question and answer information and user behavior trace information
Depth structure semantic model DSSM, calculate score corresponding to the multiple candidate answers difference；
According to the text feature of the score and the multiple candidate answers, the multiple candidate answers are ranked up,
And the answer according to corresponding to the ranking results obtain user's question sentence.
Due to based on user behavior trace information and text feature, answer search being carried out using DSSM, so as to be advantageous to
More accurately obtain answer corresponding to user's question sentence.
It is above-mentioned that this specification specific embodiment is described.Other embodiments are in the scope of the appended claims
It is interior.In some cases, the action recorded in detail in the claims or step can be come according to different from the order in embodiment
Perform and still can realize desired result.In addition, the process described in the accompanying drawings not necessarily require show it is specific suitable
Sequence or consecutive order could realize desired result.In some embodiments, multitasking and parallel processing be also can
With or be probably favourable.
Each embodiment in this specification is described by the way of progressive, identical similar portion between each embodiment
Divide mutually referring to what each embodiment stressed is the difference with other embodiment.Especially for device,
For electronic equipment, nonvolatile computer storage media embodiment, because it is substantially similar to embodiment of the method, so description
It is fairly simple, the relevent part can refer to the partial explaination of embodiments of method.
Device that this specification embodiment provides, electronic equipment, nonvolatile computer storage media with method are corresponding
, therefore, device, electronic equipment, nonvolatile computer storage media also there is the Advantageous similar with corresponding method to imitate
Fruit, due to the advantageous effects of method being described in detail above, therefore, repeat no more here corresponding intrument,
The advantageous effects of electronic equipment, nonvolatile computer storage media.
In the 1990s, the improvement for a technology can clearly distinguish be on hardware improvement (for example,
Improvement to circuit structures such as diode, transistor, switches) or software on improvement (improvement for method flow).So
And as the development of technology, the improvement of current many method flows can be considered as directly improving for hardware circuit.
Designer nearly all obtains corresponding hardware circuit by the way that improved method flow is programmed into hardware circuit.Cause
This, it cannot be said that the improvement of a method flow cannot be realized with hardware entities module.For example, PLD
(Programmable Logic Device, PLD) (such as field programmable gate array (Field Programmable Gate
Array, FPGA)) it is exactly such a integrated circuit, its logic function is determined by user to device programming.By designer
Voluntarily programming comes a digital display circuit " integrated " on a piece of PLD, without asking chip maker to design and make
Special IC chip.Moreover, nowadays, substitution manually makes IC chip, this programming is also used instead mostly " patrols
Volume compiler (logic compiler) " software realizes that software compiler used is similar when it writes with program development,
And the source code before compiling also write by handy specific programming language, this is referred to as hardware description language
(Hardware Description Language, HDL), and HDL is also not only a kind of, but have many kinds, such as ABEL
(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description
Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL
(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby
Hardware Description Language) etc., VHDL (Very-High-Speed are most generally used at present
Integrated Circuit Hardware Description Language) and Verilog.Those skilled in the art also should
This understands, it is only necessary to method flow slightly programming in logic and is programmed into integrated circuit with above-mentioned several hardware description languages,
Can is readily available the hardware circuit for realizing the logical method flow.
Controller can be implemented in any suitable manner, for example, controller can take such as microprocessor or processing
Device and storage can by the computer of the computer readable program code (such as software or firmware) of (micro-) computing device
Read medium, gate, switch, application specific integrated circuit (Application Specific Integrated Circuit,
ASIC), the form of programmable logic controller (PLC) and embedded microcontroller, the example of controller include but is not limited to following microcontroller
Device：ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, are deposited
Memory controller is also implemented as a part for the control logic of memory.It is also known in the art that except with
Pure computer readable program code mode realized beyond controller, completely can be by the way that method and step is carried out into programming in logic to make
Controller is obtained in the form of gate, switch, application specific integrated circuit, programmable logic controller (PLC) and embedded microcontroller etc. to come in fact
Existing identical function.Therefore this controller is considered a kind of hardware component, and various for realizing to including in it
The device of function can also be considered as the structure in hardware component.Or even, can be by for realizing that the device of various functions regards
For that not only can be the software module of implementation method but also can be the structure in hardware component.
System, device, module or the unit that above-described embodiment illustrates, it can specifically be realized by computer chip or entity,
Or realized by the product with certain function.One kind typically realizes that equipment is computer.Specifically, computer for example may be used
Think personal computer, laptop computer, cell phone, camera phone, smart phone, personal digital assistant, media play
It is any in device, navigation equipment, electronic mail equipment, game console, tablet PC, wearable device or these equipment
The combination of equipment.
For convenience of description, it is divided into various units during description apparatus above with function to describe respectively.Certainly, this is being implemented
The function of each unit can be realized in same or multiple softwares and/or hardware during specification one or more embodiment.
It should be understood by those skilled in the art that, this specification embodiment can be provided as method, system or computer program
Product.Therefore, this specification embodiment can use complete hardware embodiment, complete software embodiment or with reference to software and hardware
The form of the embodiment of aspect.Moreover, this specification embodiment can be can use using computer is wherein included in one or more
It is real in the computer-usable storage medium (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form for the computer program product applied.
This specification is with reference to the method, equipment (system) and computer program product according to this specification embodiment
Flow chart and/or block diagram describe.It should be understood that can be by every in computer program instructions implementation process figure and/or block diagram
One flow and/or the flow in square frame and flow chart and/or block diagram and/or the combination of square frame.These computers can be provided
Processor of the programmed instruction to all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices
To produce a machine so that produce use by the instruction of computer or the computing device of other programmable data processing devices
In the dress for realizing the function of being specified in one flow of flow chart or multiple flows and/or one square frame of block diagram or multiple square frames
Put.
These computer program instructions, which may be alternatively stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that the instruction being stored in the computer-readable memory, which produces, to be included referring to
Make the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one square frame of block diagram or
The function of being specified in multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that counted
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented processing, so as in computer or
The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in individual square frame or multiple square frames.
In a typical configuration, computing device includes one or more processors (CPU), input/output interface, net
Network interface and internal memory.
Internal memory may include computer-readable medium in volatile memory, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only storage (ROM) or flash memory (flash RAM).Internal memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer-readable instruction, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), moved
State random access memory (DRAM), other kinds of random access memory (RAM), read-only storage (ROM), electric erasable
Programmable read only memory (EEPROM), fast flash memory bank or other memory techniques, read-only optical disc read-only storage (CD-ROM),
Digital versatile disc (DVD) or other optical storages, magnetic cassette tape, the storage of tape magnetic rigid disk or other magnetic storage apparatus
Or any other non-transmission medium, the information that can be accessed by a computing device available for storage.Define, calculate according to herein
Machine computer-readable recording medium does not include temporary computer readable media (transitory media), such as data-signal and carrier wave of modulation.
It should also be noted that, term " comprising ", "comprising" or its any other variant are intended to nonexcludability
Comprising so that process, method, commodity or equipment including a series of elements not only include those key elements, but also wrapping
Include the other element being not expressly set out, or also include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that wanted including described
Other identical element also be present in the process of element, method, commodity or equipment.
This specification can be described in the general context of computer executable instructions, such as journey
Sequence module.Usually, program module include performing particular task or realize the routine of particular abstract data type, program, object,
Component, data structure etc..Specification can also be put into practice in a distributed computing environment, in these DCEs,
By performing task by communication network and connected remote processing devices.In a distributed computing environment, program module can
With in the local and remote computer-readable storage medium including storage device.
Each embodiment in this specification is described by the way of progressive, identical similar portion between each embodiment
Divide mutually referring to what each embodiment stressed is the difference with other embodiment.It is real especially for system
For applying example, because it is substantially similar to embodiment of the method, so description is fairly simple, related part is referring to embodiment of the method
Part explanation.
This specification embodiment is the foregoing is only, is not limited to the application.For those skilled in the art
For, the application can have various modifications and variations.All any modifications made within spirit herein and principle, it is equal
Replace, improve etc., it should be included within the scope of claims hereof.
Claims (27)
1. a kind of question and answer searching method, including：
Receive user's question sentence；
Obtain user behavior trace information and multiple candidate answers；
According to user behavior trace information, the depth for training to obtain based on user's question and answer information and user behavior trace information is utilized
Structuring semantic model DSSM, calculate score corresponding to the multiple candidate answers difference；
According to the text feature of the score and the multiple candidate answers, the multiple candidate answers are ranked up, and
The answer according to corresponding to the ranking results obtain user's question sentence.
2. the method as described in claim 1, reception user's question sentence, in addition to：Obtain and used corresponding to user's question sentence
Family identifies and/or session identification.
3. method as claimed in claim 2, the acquisition user behavior trace information, are specifically included：
According to user's mark and/or session identification corresponding to user's question sentence, user behavior trace information is obtained.
4. method as claimed in claim 3, the user behavior trace information includes the user behavior track in a period of time
Point, the user behavior tracing point are that remote procedure call protocol RPC is recorded and/or uniform resource position mark URL records.
5. the method as described in claim 1, described to obtain multiple candidate answers, specifically include：
According to the text feature of user's question sentence, and the text feature of the answer included in the answer set specified, in institute
State screening in the answer set specified and obtain multiple candidate answers.
6. method as claimed in claim 5, the text feature according to user's question sentence, and the answer set specified
In the text feature of answer that includes, in the answer set specified screening obtain multiple candidate answers, specifically include：
The answer set specified using predetermined search engine extraction corresponding to user's question sentence；
The answer in the answer set specified is ranked up using LamdaMART algorithms, obtains the first ranking results；
According to first ranking results, screening obtains multiple candidate answers.
7. the method as described in claim 1, user's question and answer information includes problem information and its corresponding answer information；Base
Training to obtain DSSM in user's question and answer information and user behavior trace information includes：
According to described problem information and user behavior trace information, the first input vector is generated；And
According to answer information corresponding to described problem information, the second input vector is generated；
Train to obtain DSSM using first input vector and second input vector.
8. method as claimed in claim 7, described defeated according to described problem information and user behavior trace information, generation first
Incoming vector；
The problem of described problem information is specified message identification is retrieved as, and is user behavior track corresponding to described problem information
The trace information mark that information is specified；
Identified according to described problem message identification and the trace information, generate the first input vector.
9. method as claimed in claim 7, the answer information according to corresponding to described problem information, generation second input to
Amount, is specifically included：
Extract the knowledge dot leader participle of answer information corresponding to described problem information；
Segmented according to the knowledge dot leader, generate the second input vector.
10. method as claimed in claim 7, described to be trained using first input vector and second input vector
To DSSM, specifically include：
First input vector is inputted into DSSM prototypes, obtains the first output result；And
Second input vector is inputted into the DSSM prototypes, obtains the second output result；
The correlation between first output result and the second output result is calculated, and according to the correlation to the DSSM
Prototype is trained, and obtains DSSM.
11. method as claimed in claim 6, the text feature according to the score and the multiple candidate answers is right
The multiple candidate answers are ranked up, and the answer according to corresponding to the ranking results obtain user's question sentence, specifically
Including：
According to the text feature of the score and the multiple candidate answers, using LamdaMART algorithms to the multiple candidate
Answer is ranked up, and obtains the second ranking results；
According to first ranking results and second ranking results, the multiple candidate answers are reordered, obtained
Reorder result；
According to the result that reorders, answer corresponding to user's question sentence is obtained.
12. method as claimed in claim 11, described according to first ranking results and second ranking results, to institute
State multiple candidate answers to be reordered, obtain the result that reorders, specifically include：
According to first ranking results and second ranking results, the model that reorders obtained using training is to the multiple
Candidate answers are reordered, and obtain the result that reorders；
Wherein, the training model that obtains reordering includes：
Obtain user's click logs；
According to user's click logs and the score, using LamdaMART algorithms, training obtains the model that reorders.
13. method as claimed in claim 11, reorder result described in the basis, obtains and is answered corresponding to user's question sentence
Case, specifically include：
According to reorder result and first ranking results, decision-making is carried out using decision Tree algorithms, the user is obtained and asks
Answer corresponding to sentence.
14. a kind of question and answer searcher, including：
Receiving module, receive user's question sentence；
Data obtaining module, obtain user behavior trace information and multiple candidate answers；
Grading module, according to user behavior trace information, trained using based on user's question and answer information and user behavior trace information
Obtained depth structure semantic model DSSM, calculate score corresponding to the multiple candidate answers difference；
Reorder module, and according to the text feature of the score and the multiple candidate answers, the multiple candidate answers are entered
Row sequence, and the answer according to corresponding to the ranking results obtain user's question sentence.
15. device as claimed in claim 14, the receiving module, user's question sentence is received, in addition to：Pass through the reception mould
Block, obtain user's mark and/or session identification corresponding to user's question sentence.
16. device as claimed in claim 15, described information acquisition module, user behavior trace information is obtained, is specifically included：
According to user's mark and/or session identification corresponding to user's question sentence, user behavior trace information is obtained.
17. device as claimed in claim 16, the user behavior trace information includes the user behavior rail in a period of time
Mark point, the user behavior tracing point are that remote procedure call protocol RPC is recorded and/or uniform resource position mark URL records.
18. device as claimed in claim 14, described information acquisition module, multiple candidate answers are obtained, are specifically included：
According to the text feature of user's question sentence, and the text feature of the answer included in the answer set specified, in institute
State screening in the answer set specified and obtain multiple candidate answers.
19. device as claimed in claim 18, the text feature according to user's question sentence, and the answer set specified
The text feature of the answer included in conjunction, screened in the answer set specified and obtain multiple candidate answers, specifically included：
The answer set specified using predetermined search engine extraction corresponding to user's question sentence；
The answer in the answer set specified is ranked up using LamdaMART algorithms, obtains the first ranking results；
According to first ranking results, screening obtains multiple candidate answers.
20. device as claimed in claim 14, user's question and answer information includes problem information and its corresponding answer information；
Training to obtain DSSM based on user's question and answer information and user behavior trace information includes：
According to described problem information and user behavior trace information, the first input vector is generated；And
According to answer information corresponding to described problem information, the second input vector is generated；
Train to obtain DSSM using first input vector and second input vector.
21. device as claimed in claim 20, described according to described problem information and user behavior trace information, generation first
Input vector；
The problem of described problem information is specified message identification is retrieved as, and is user behavior track corresponding to described problem information
The trace information mark that information is specified；
Identified according to described problem message identification and the trace information, generate the first input vector.
22. device as claimed in claim 20, the answer information according to corresponding to described problem information, the input of generation second
Vector, specifically include：
Extract the knowledge dot leader participle of answer information corresponding to described problem information；
Segmented according to the knowledge dot leader, generate the second input vector.
23. device as claimed in claim 20, described to be trained using first input vector and second input vector
DSSM is obtained, is specifically included：
First input vector is inputted into DSSM prototypes, obtains the first output result；And
Second input vector is inputted into the DSSM prototypes, obtains the second output result；
The correlation between first output result and the second output result is calculated, and according to the correlation to the DSSM
Prototype is trained, and obtains DSSM.
24. device as claimed in claim 20, the text feature according to the score and the multiple candidate answers is right
The multiple candidate answers are ranked up, and the answer according to corresponding to the ranking results obtain user's question sentence, specifically
Including：
According to the text feature of the score and the multiple candidate answers, using LamdaMART algorithms to the multiple candidate
Answer is ranked up, and obtains the second ranking results；
According to first ranking results and second ranking results, the multiple candidate answers are reordered, obtained
Reorder result；
According to the result that reorders, answer corresponding to user's question sentence is obtained.
25. device as claimed in claim 24, described according to first ranking results and second ranking results, to institute
State multiple candidate answers to be reordered, obtain the result that reorders, specifically include：
According to first ranking results and second ranking results, the model that reorders obtained using training is to the multiple
Candidate answers are reordered, and obtain the result that reorders；
Wherein, the training model that obtains reordering includes：
Obtain user's click logs；
According to user's click logs and the score, using LamdaMART algorithms, training obtains the model that reorders.
26. device as claimed in claim 24, reorder result described in the basis, obtains and is answered corresponding to user's question sentence
Case, specifically include：
According to reorder result and first ranking results, decision-making is carried out using decision Tree algorithms, the user is obtained and asks
Answer corresponding to sentence.
27. a kind of electronic equipment, including：
At least one processor；And
The memory being connected with least one processor communication；Wherein,
The memory storage has can be by the instruction of at least one computing device, and the instruction is by least one place
Manage device to perform, so that at least one processor can：
To receive user's question sentence；
Obtain user behavior trace information and multiple candidate answers；
According to user behavior trace information, the depth for training to obtain based on user's question and answer information and user behavior trace information is utilized
Structuring semantic model DSSM, calculate score corresponding to the multiple candidate answers difference；
According to the text feature of the score and the multiple candidate answers, the multiple candidate answers are ranked up, and
The answer according to corresponding to the ranking results obtain user's question sentence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710769081.5A CN107679082A (en) | 2017-08-31 | 2017-08-31 | Question and answer searching method, device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710769081.5A CN107679082A (en) | 2017-08-31 | 2017-08-31 | Question and answer searching method, device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107679082A true CN107679082A (en) | 2018-02-09 |
Family
ID=61135568
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710769081.5A Pending CN107679082A (en) | 2017-08-31 | 2017-08-31 | Question and answer searching method, device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107679082A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108962228A (en) * | 2018-07-16 | 2018-12-07 | 北京百度网讯科技有限公司 | model training method and device |
| CN108984666A (en) * | 2018-06-29 | 2018-12-11 | 阿里巴巴集团控股有限公司 | Data processing method, data processing equipment and server |
| CN108984658A (en) * | 2018-06-28 | 2018-12-11 | 阿里巴巴集团控股有限公司 | A kind of intelligent answer data processing method and device |
| CN108984475A (en) * | 2018-07-06 | 2018-12-11 | 北京慧闻科技发展有限公司 | Answer selection method, device and electronic equipment based on holographic neural network |
| CN109065154A (en) * | 2018-07-27 | 2018-12-21 | 清华大学 | A kind of result of decision determines method, apparatus, equipment and readable storage medium storing program for executing |
| CN109086417A (en) * | 2018-08-06 | 2018-12-25 | 清华大学 | Search for evaluation method and device |
| CN109376298A (en) * | 2018-09-14 | 2019-02-22 | 广州神马移动信息科技有限公司 | Data processing method, device, terminal device and computer storage medium |
| CN110008322A (en) * | 2019-03-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Art recommended method and device under more wheel session operational scenarios |
| CN110008320A (en) * | 2019-03-05 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Method and apparatus for answering customer problem |
| CN110059178A (en) * | 2019-02-12 | 2019-07-26 | 阿里巴巴集团控股有限公司 | Problem distributing method and device |
| CN110909145A (en) * | 2019-11-29 | 2020-03-24 | 支付宝(杭州)信息技术有限公司 | Training method and device for multi-task model |
| CN111309882A (en) * | 2020-02-13 | 2020-06-19 | 支付宝(杭州)信息技术有限公司 | Method and device for realizing intelligent customer service question answering |
| WO2020143346A1 (en) * | 2019-01-10 | 2020-07-16 | 北京三快在线科技有限公司 | Sorting |
| CN111428125A (en) * | 2019-01-10 | 2020-07-17 | 北京三快在线科技有限公司 | Sorting method and device, electronic equipment and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012162822A1 (en) * | 2011-05-27 | 2012-12-06 | International Business Machines Corporation | Automated self-service user support based on ontology analysis |
| CN105308591A (en) * | 2014-03-03 | 2016-02-03 | 微软技术许可有限责任公司 | Dynamics of tie strength from social interaction |
| CN106202159A (en) * | 2016-06-23 | 2016-12-07 | 深圳追科技有限公司 | A kind of man-machine interaction method of customer service system |
| CN106415535A (en) * | 2014-04-14 | 2017-02-15 | 微软技术许可有限责任公司 | Context-sensitive search using a deep learning model |
| CN106649786A (en) * | 2016-12-28 | 2017-05-10 | 北京百度网讯科技有限公司 | Deep question answer-based answer retrieval method and device |
| CN106663124A (en) * | 2014-08-11 | 2017-05-10 | 微软技术许可有限责任公司 | Generating and using a knowledge-enhanced model |
-
2017
- 2017-08-31 CN CN201710769081.5A patent/CN107679082A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012162822A1 (en) * | 2011-05-27 | 2012-12-06 | International Business Machines Corporation | Automated self-service user support based on ontology analysis |
| CN105308591A (en) * | 2014-03-03 | 2016-02-03 | 微软技术许可有限责任公司 | Dynamics of tie strength from social interaction |
| CN106415535A (en) * | 2014-04-14 | 2017-02-15 | 微软技术许可有限责任公司 | Context-sensitive search using a deep learning model |
| CN106663124A (en) * | 2014-08-11 | 2017-05-10 | 微软技术许可有限责任公司 | Generating and using a knowledge-enhanced model |
| CN106202159A (en) * | 2016-06-23 | 2016-12-07 | 深圳追科技有限公司 | A kind of man-machine interaction method of customer service system |
| CN106649786A (en) * | 2016-12-28 | 2017-05-10 | 北京百度网讯科技有限公司 | Deep question answer-based answer retrieval method and device |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108984658A (en) * | 2018-06-28 | 2018-12-11 | 阿里巴巴集团控股有限公司 | A kind of intelligent answer data processing method and device |
| CN108984666A (en) * | 2018-06-29 | 2018-12-11 | 阿里巴巴集团控股有限公司 | Data processing method, data processing equipment and server |
| CN108984475A (en) * | 2018-07-06 | 2018-12-11 | 北京慧闻科技发展有限公司 | Answer selection method, device and electronic equipment based on holographic neural network |
| CN108962228B (en) * | 2018-07-16 | 2022-03-15 | 北京百度网讯科技有限公司 | Model training method and device |
| CN108962228A (en) * | 2018-07-16 | 2018-12-07 | 北京百度网讯科技有限公司 | model training method and device |
| CN109065154A (en) * | 2018-07-27 | 2018-12-21 | 清华大学 | A kind of result of decision determines method, apparatus, equipment and readable storage medium storing program for executing |
| CN109086417A (en) * | 2018-08-06 | 2018-12-25 | 清华大学 | Search for evaluation method and device |
| CN109376298B (en) * | 2018-09-14 | 2022-01-25 | 阿里巴巴（中国）有限公司 | Data processing method and device, terminal equipment and computer storage medium |
| CN109376298A (en) * | 2018-09-14 | 2019-02-22 | 广州神马移动信息科技有限公司 | Data processing method, device, terminal device and computer storage medium |
| WO2020143346A1 (en) * | 2019-01-10 | 2020-07-16 | 北京三快在线科技有限公司 | Sorting |
| CN111428125A (en) * | 2019-01-10 | 2020-07-17 | 北京三快在线科技有限公司 | Sorting method and device, electronic equipment and readable storage medium |
| CN111428125B (en) * | 2019-01-10 | 2023-05-30 | 北京三快在线科技有限公司 | Ordering method, ordering device, electronic equipment and readable storage medium |
| CN110059178A (en) * | 2019-02-12 | 2019-07-26 | 阿里巴巴集团控股有限公司 | Problem distributing method and device |
| CN110008320A (en) * | 2019-03-05 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Method and apparatus for answering customer problem |
| CN110008320B (en) * | 2019-03-05 | 2023-10-27 | 创新先进技术有限公司 | Method and device for answering user questions |
| CN110008322B (en) * | 2019-03-25 | 2023-04-07 | 创新先进技术有限公司 | Method and device for recommending dialogues in multi-turn conversation scene |
| CN110008322A (en) * | 2019-03-25 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Art recommended method and device under more wheel session operational scenarios |
| CN110909145A (en) * | 2019-11-29 | 2020-03-24 | 支付宝(杭州)信息技术有限公司 | Training method and device for multi-task model |
| CN110909145B (en) * | 2019-11-29 | 2022-08-09 | 支付宝(杭州)信息技术有限公司 | Training method and device for multi-task model |
| CN111309882A (en) * | 2020-02-13 | 2020-06-19 | 支付宝(杭州)信息技术有限公司 | Method and device for realizing intelligent customer service question answering |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107679082A (en) | Question and answer searching method, device and electronic equipment | |
| CN111177569B (en) | Recommendation processing method, device and equipment based on artificial intelligence | |
| CN111615706A (en) | Analysis of spatial sparse data based on sub-manifold sparse convolutional neural network | |
| CN107797992A (en) | Name entity recognition method and device | |
| CN109325040B (en) | FAQ question-answer library generalization method, device and equipment | |
| CN113505204B (en) | Recall model training method, search recall device and computer equipment | |
| CN109299258A (en) | A kind of public sentiment event detecting method, device and equipment | |
| CN112287069B (en) | Information retrieval method and device based on voice semantics and computer equipment | |
| CN108763535A (en) | Information acquisition method and device | |
| Hong et al. | Selective residual learning for visual question answering | |
| CN109344246B (en) | Electronic questionnaire generating method, computer readable storage medium and terminal device | |
| CN106980620A (en) | A kind of method and device matched to Chinese character string | |
| CN114357117A (en) | Transaction information query method and device, computer equipment and storage medium | |
| CN108804525A (en) | A kind of intelligent Answering method and device | |
| CN108805260A (en) | A kind of figure says generation method and device | |
| CN116796857A (en) | LLM model training method, device, equipment and storage medium thereof | |
| CN109271624A (en) | A kind of target word determines method, apparatus and storage medium | |
| CN110110218A (en) | A kind of Identity Association method and terminal | |
| CN114358014A (en) | Work order intelligent diagnosis method, device, equipment and medium based on natural language | |
| CN111259975B (en) | Method and device for generating classifier and method and device for classifying text | |
| CN110019556A (en) | A kind of topic news acquisition methods, device and its equipment | |
| CN117216617A (en) | Text classification model training method, device, computer equipment and storage medium | |
| Shin | Socio-technical design of algorithms: Fairness, accountability, and transparency | |
| CN112199954B (en) | Disease entity matching method and device based on voice semantics and computer equipment | |
| CN112818212B (en) | Corpus data acquisition method, corpus data acquisition device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1248867 Country of ref document: HK |
|
| TA01 | Transfer of patent application right |
Effective date of registration: 20200927 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200927 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180209 |
|
| RJ01 | Rejection of invention patent application after publication |