CN110457445A

CN110457445A - Answer generation technique based on user's portrait and Context Reasoning

Info

Publication number: CN110457445A
Application number: CN201810414548.9A
Authority: CN
Inventors: 吴先超; 曾敏; 周力
Original assignee: Microsoft Technology Licensing LLC
Current assignee: Microsoft Technology Licensing LLC
Priority date: 2018-05-03
Filing date: 2018-05-03
Publication date: 2019-11-15
Also published as: WO2019212729A1

Abstract

Disclosed herein is a kind of answer generation techniques based on user's portrait and Context Reasoning, with the chat robots for constructing group chat orientation, and in group chat theme diversity and user query and answer can not the property estimated, provide solution.

Description

Answer generation technique based on user's portrait and Context Reasoning

Background technique

Emotion AI chat system becomes one of most important direction in the field artificial intelligence (AI) gradually in recent years.Pass through The chat of voice and/or text, chat robots are used as various product/application program interactive entrance.

Other than the scene that single user and chat robots are chatted, it is notable that passed through in social networks Many groups are commonly present, for example, based on QQ, the types of facial makeup in Beijing operas, LinkedIn (neck English), Wechat (wechat), Line, Slack etc. building Group.These groups can be interest group (for example, film, food, travelling, music etc.), work group (for example, belonging to a certain public affairs The team of one department of department, either some from different company), groups of friends, classmate group or even comprising having house The kin group of race's relationship.If chat robots can be introduced into this " group chat ", it will it plays a positive role, it can be with Chat theme and viewpoint are summarized in help, push the progress of chat, and can provide real-time online information clothes in an active manner Business.

Summary of the invention

There is provided content of the embodiment of the present invention is to further retouch in will be described in detail below with the form introduction simplified The some concepts stated.The content of present invention is not intended to the key features or essential features of mark claimed subject, also not purport In the range for limiting claimed subject.

Disclosed herein is a kind of answer generation techniques based on user's portrait and Context Reasoning, to take for constructing group chat To chat robots, and in group chat theme diversity and user query and answer can not the property estimated, provide Solution.

Above description is only the general introduction of disclosed technique scheme, in order to better understand the technological means of the disclosure, And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects, features, and advantages of the present disclosure can It is clearer and more comprehensible, below the special specific embodiment for lifting the disclosure.

Detailed description of the invention

Fig. 1 is the example frame of AI chat system；

Fig. 2 is illustrative group chat interface schematic block diagram；

Fig. 3 is the example block diagram of user's Wiki archives of illustrative famous person " George Washington "；

Fig. 4 is user's Wiki archives example block diagram that " Bill Gates " are used in LinkedIn；

Fig. 5 is the flow processing schematic diagram of the extraction process of illustrative user's Wiki archives；

Fig. 6 is the exemplary block diagram that user pushes away text；

Fig. 7 is illustrative key-value pair classification of documents device using schematic block diagram；

Fig. 8 is the structural representation for the neural network device that the illustrative life cycle state for theme is handled Figure；

Fig. 9 illustratively describes the schematic block diagram for the disaggregated model implemented by four layers of neural network；

Figure 10 is the exemplary block diagram of the knowledge mapping based on concept；

Figure 11 is the exemplary block diagram that the answer based on reasoning generates model；

Figure 12 is the structural schematic diagram of the illustratively answer generating means based on reasoning；

Figure 13 is the structural block diagram of the illustrative electronic equipment with mobility；

Figure 14 is the illustrative structural block diagram for calculating equipment.

Specific embodiment

Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawing Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure It is fully disclosed to those skilled in the art.

Herein, term " technology ", " mechanism " may refer to such as (one or more) system, (one or more) side Method, computer-readable instruction, (one or more) module, algorithm, hardware logic are (for example, field programmable gate array (FPGA)), specific integrated circuit (ASIC), Application Specific Standard Product (ASSP), system on chip (SOC), complex programmable logic are set Standby (CPLD) and/or above-mentioned context and in this document permitted (one or more) other technologies in the whole text.

General view based on application scenarios

For group chat related application scene, this paper presents a kind of for constructing the solution of the chat robots of group chat orientation Scheme.

Group chat (for example, interest group, groups of friends, family crowd, any kind of short-term or long term activity of earthquake group) such as wechat, Become to become more and more popular in social networks (SNS) application of QQ, facebook etc..For example, being deposited in this social networking application of QQ In the millions of a groups created by QQ user, the moon active users about 800,000,000 6 thousand ten thousand.

Moreover, more and more chat robots start to be designed to be used as group chat to provide following function: (1) to provide Mood message, so that the people in group be helped to link up each other in more smooth mode；And (2) by it is non-it is predictable ask it is realistic When information needed is provided.

Specifically, (1) function is more that, using " pure chat " as main target, (2) function is more Meet and online service (for example, travel reservation, Products Show etc.) is provided.

Although the two functions exist in the one-to-one communication of user and chat robots, due to following original Cause, however it remains some challenges:

Different from one-to-one chat scenario, in group chat, theme has diversity and is often parallel.Each master Topic all may be interested to the different user groups (user group) in the group chat, and each theme may also be with other masters Topic is related.If by it is one-to-one chat regard as " single thread ", i.e., the same time only exist a user query (query) and One theme；So group chat be more like " multithreading ", i.e., the same time there are it is multiple enliven theme, multiple any active ues and Possible correlation (between theme, between user, between theme and user) between them.

In addition, we also need to manage: (1) relationship theme between, for example, from " party " to " ordering restaurant ", and into One step is to " food menu ", " drinks menu ", either on how to reach " stroke " of party venue or neighbouring subway station, Next activity even after " party ", for example, KTV is gone to sing；(2) relationship between user；And (3) user and master " Percentage bound " between topic, for example, each user is for the attitude of theme and degree of correlation etc..

Particularly, being described below may need the case where coping with or task for the chat robots of group chat.

1, discrete user query from each User ID (be referred to as user's speech or user put question to).Group After user in merely sends a user query, it can be ready to reply for the related of the user query in chat robots Before, other people user query may be sent in group chat in a manner of unpredictable.In order to handle the situation, chatting machine Device people needs to manage message respectively by user and theme, forms dedicated " storage " resource of two kinds of user and theme is dedicated " storage " resource.Furthermore, it is necessary to which it is noted that the user query of a user may be to have to put question to property to another user User query " answer ".For a theme, the user query of the theme can also play the part of such as theme Different " roles " such as beginning, the extension of theme, front/reverse side comment, the execution of theme, the end of theme and comments of theme, These " roles " embody the different conditions of theme.Identify/differentiate to " role " that user query are played the part of in theme is Very important, if a user query are for user query " answer " another in group chat, we actually can It is enough used to help collect the machine learning model for being directed to group chat using " problem-answer " relationship in current group chat scene (machine learning model is used to be based on current group chat session training data, automatically detects with the user query for puing question to property And the answer for the enquirement).

2, from the continuous meassage of group chat.Different themes may also occur in a manner of unpredictable, at " active " " the potential connection " being likely that there are in theme between theme.

For theme that is this parallel but may having related incidence relation, chat robots need the correlation to theme Information is stored, and when handling some theme phase, the storage information of another theme is accessed, to obtain the pass between theme Connection relationship, for generating the answer for being better suited for current group chat session scene.

3, the perceivable information of chat robots.For example, timestamp (timestamp), User ID/chat robots ID (UserID/ChatbotID) (, user query (Qurey) etc..Chat robots are required to identify above- mentioned information, that is, identify Out when, for which user and for which theme provide corresponding answer.

In view of the above requirements, this paper presents the example frames 100 of AI chat system as shown in Figure 1.

In scene shown in FIG. 1, the user interface 101 of affiliate is for receiving user query, user interface here 101 may come from affiliate (for example, existing social networks, the types of facial makeup in Beijing operas, wechat, Line, LinkedIn (neck English), Slack) the user interface provided is also possible to user interface (user circle based on mobile terminal or the end PC of oneself exploitation Face).As shown in Fig. 2, it is illustrative group chat interface schematic block Figure 200.Left side spokesman is not limited to a user or one A chat robots can be multiple users and a chat robots or multiple users and multiple chat robots.

In group chat user interface shown in Fig. 2, there are the user of three real worlds (" Keizo ", " Mike " and " I ") and a chat robots (such as name is called " small ice ").Right side user (or " I ") is the user that group chat is added, and And user interface is seen from the visual angle of " I ".In this example, chat robots " small ice " are in a manner of very active Answer the user query of each user.Wherein, chat robots " small ice " can use different names, example in country variant Such as: CHINESE REGION use " small ice " perhaps " XiaoIce " as name Japan and Indonesia use it is " Rinna " or " cold Dish " uses " Zo " as name as name, in the U.S., uses " Ruuh " as name etc. in India.

In user input area, user is able to carry out input text, sends picture, selects emoticon and to current Picture carries out the operation such as screenshot.Further, it is also possible to AI chat system initiating speech conversation or video calling.For example, in user In interface, user is had input " small ice, how old are you ", which is transferred to user query queue as a user query 118, wherein user query queue 118 (can may include the side such as text, voice, image and video with multimedia form Formula) storage user query.AI chat system may exist some differences in terms of handling a variety of multimedia inputs.For example, right In real-time voice and video, AI chat system needs the copy of more working cells (Worker), to ensure the user query 118 user query list of queue will not be too long, if too long in user query queue 118, user may feel that longer Reply delay.For example, being directed to text, voice and image in Fig. 1, it is respectively provided with chat text processing working cell 121, language Sound handles working cell 122, image processing work unit 123, these three processing units can call and corresponding text API (application programming interfaces) 124, speech recognition API 125, image recognition API 126 are understood to execute program processing, also, These three working cells can receive the scheduling and coordination of core work unit 119.Above-mentioned working cell can according to need It is embodied as program module, example, thread, process etc..

Core work unit (Core Worker) 119 in Fig. 1 inputs user query queue 118 as it, with advanced The mode first gone out is handled the user query in the user query queue 118 to generate and reply.

Substantially, core work unit 119 can seriatim call the phase of each working cell progress relevant user inquiry Pass processing.For example, core work unit 119 can call chat text to handle working cell 121, execute for textual form The processing of user query, further, core work unit 119 may call upon 124 pairs of chat contexts of text understanding API Information is handled.In addition, chat text processing working cell 121 can also include other than multiple chat text processing functions Service unit, for example, Time Service unit, it can be raw for the user query for such as " please reminding me to go HR meeting at 6 points " At alarm clock calling, then such as weather service unit, for for such as " tomorrow can rain ", " I will fly to Beijing needs tomorrow Band umbrella " user query, the answer in terms of Weather information is provided.

The user query of speech form, which need to be identified and be decoded as text, just can be carried out subsequent processing.Speech recognition API 125 for carrying out the convert task of speech-to-text.In addition, after going out text from tone decoding, speech processes working cell Request chat text processing working cell 121 is executed the processing such as language understanding, requirement analysis and answer generation by 122.It is chatting After text-processing working cell 12 generates answer text and returns to speech processes working cell 122, speech processes work Make the conversion that unit 122 will execute Text To Speech, and generates and be able to use the voice output that family is heard.

Similarly with speech processes, image is also required to through image recognition API 126 come " reading " and " understanding ".That is, figure As needing to be decoded as text.For example, image recognition API 126 can be identified probably when seeing the dog image of user's input The type/color of dog out, and the information identified is exported in the form of text, then, image processing work unit 123 is further according to this The image recognition information of a little textual forms, generates and replies.Certainly, more mature to text machine learning model due to image, it can Directly to generate corresponding answer according to image, carry out replying generation without recalling image processing work unit 123 Processing, it can some comments are directly generated according to the image of input by image recognition API 126 or suggestion is used as and replys, example Such as, " good lovely German shepherd！You centainly like it very much ".Since image to text model can directly be trained/be used, because This, the access for image processing work unit 123 is optional.

In frame shown in Fig. 1, other works other than previously described basic working cell can also be inserted into Make unit, so that frame is with expansible.For example, position processing working cell can be inserted, work is handled based on position Unit can easily support location based service, for example, being directed to " ordering a Pizza to my office ", " when I am when family Remind me when I am close to supermarket " as user query, location information can be introduced into the generation processing of answer, thus It is capable of providing more flexible answer mode.

After core work unit 119 generates answer, it is transmitted to answer queue 120 and/or is put into caching 127.

The sequence that caching 127 and active unit (Proactive Worker) 128 are used to be ready to reply in advance, from And it can be presented to the user with predetermined time flow.For a user query, if there is be no more than two by The answer that core work unit 119 generates then needs to reply these progress time delay setting.

For example, if user says " small ice, you have had breakfast ", and AI chat system generates two answers, first A is " yes, I has eaten bread ", and second is " you still feel hungry ".We need to reply first vertical Be presented to the user, and for second reply, time delay can be set to 2 seconds, and by active unit 128 come Control will reply and be put into the opportunity for replying queue 120.Thus after presenting first answer to user after 2 seconds, by second A answer shows user.Caching 127 and active unit 128 manage the identity of these answers to be sent, user together And the time delay of each answer.

Working cell 103 is replied from one optimal answer of selection in queue 120 is replied, it is then possible to call cooperation partner The user interface 101 that the answer of generation is supplied to affiliate is presented to the user by the answer API 102 of companion.

Particularly, the core processing module of following several this paper is further related in Fig. 1:

User's Wiki generation module 109, for utilizing user's Wiki developing algorithm from existing chat log database 106 Extract user's Wiki 112.In addition, user's Wiki generation module 109 can also be from the public information of social networks related to user Middle extraction user Wiki 112.

Theme life cycle state categorization module 108, for by using theme life cycle state sorting algorithm, from Current sessions 105 detect theme life cycle state 111.

Knowledge mapping abstraction module 107, for being excavated from network knowledge 104 by using knowledge mapping extraction algorithm often With the knowledge mapping 110 in field.

Answer generation module 114 based on reasoning, for integrate user's Wiki 112, theme life cycle state 111 and often With the knowledge mapping 110 in field, in the treatment process that this three aspects Information application is generated to answer, to generate for working as pre-group The merely best answer of scene.

It should be noted that the straight of the knowledge graph map based on user's Wiki 112 and common field can also be provided herein Connect depth question and answer (QA) function.For example, one direct problem " small ice, you know the name of my pet ", and replying can To be " Joy, you told me last week ", this is the depth question and answer function based on user's Wiki, for another example another problem is that " small Whom the first president of ice, the U.S. is ", and replying is " George Washington ", this is the knowledge mapping based on common field and mentions The direct depth question and answer function of supplying.

User's Wiki generation, the classification of theme life cycle state, knowledge mapping extraction will be discussed in detail respectively below And the answer based on reasoning generates the treatment mechanism of this several respect.

User's Wiki generates

What user's Wiki was extracted aims at, and information source based on following constructs files on each of customers in the form of Wiki:

1, in one-to-one personal chat scenario user and chat robots chat log.

2, the chat log of user and chat robots in the group that chats that user is added.

3, in a manner of explicit/implicit, the social network information of the user of acquisition.

A, implicit: for example, " we come from same university " is for the known university of user A to be assigned to user B Indirect clue, it is assumed that be able to know that (1) " we " indicate user A and user B, and the university of (2) user A is to have known 's.This can be named as to " entity transition rule ", so as to use the rule to come from user's Wiki with comparatively perfect The seed user of archives extends to the relatively incomplete relational users of user's Wiki archives.

B, explicit way: for another example, in some social networking applications, for example, on LinkedIn, user's College information can explicitly disclose in the resume arranged by LinkedIn, so as to obtain user's Wiki with explicit way One example of archive information.

As shown in figure 3, it is the example block diagram 300 of illustrative famous person " George Washington " user's Wiki archives.Fig. 3's Wiki archives are arranged in the form of key-value pair information, wherein the key in key-value pair information has corresponded to involved in user property Some entities, such as birth information, residence, achieve an honor, information involved in the value in key-value pair information, for example, " on 2 22nd, 1732 ", " Washington ... ", " Congress's gold medal ".

The finish message method of these key-value pair forms is also used by several social networking applications such as LinkedIn, As shown in figure 4, being used for user's Wiki archives example block diagram 400 of " Bill Gates " in LinkedIn.

The treatment process for extracting user's Wiki archives is described below.As shown in figure 5, it is descriptive user's Wiki The flow processing schematic diagram 500 of the extraction process of archives.

In order to excavate user's Wiki information from the chat log of user and social networks, this paper presents a kind of data collections Approach and a kind of machine learning model (the key-value pair classification of documents device 505 in Fig. 5).

In data collection approach such as Fig. 5, treatment process S501~S505 shown by left side, wherein S501~S503's Processing is used to collect the training data of trained key-value pair classification of documents device 505, the collected data of S504 and the chat of user Input data of the log together as key-value pair classification of documents device 505.Herein, two classes in social networks are mainly used Information, one kind are archive informations, this category information be by social networks arrange knowledge category information (such as wikipedia) or The public information (the archive information example in such as Fig. 3 or Fig. 4) of user account, both information may be considered it is believable, It can be used as true basis.Another kind is content information, that is, the information that user issues on social networks, for example, push away text, Model etc. in facebook.

The detailed process that user's Wiki archives extract is described below:

S501: key-value pair information is obtained from the archive information of social networks.For example, from Google+, LinkedIn, facebook Some key-value pair informations are obtained in user's Wiki archives of equal social networks, these key-value pair informations can be used as basic fact.

S502: the content information of the social networks comprising the value in the key-value pair information in treatment process S501, example are obtained Such as, obtain the Bill Gates comprising " Harvard University " pushes away text.These, which push away text, can be used as positive example.

S503: it the syntax dependence of the content information of the social networks obtained in dissection process process S502 and extracts Structure feature (for example, n member dependence arc), generates the feature training data for training key-value pair classification of documents device.

S504: being trained Machine learning classifiers using the training data generated in treatment process S503, generates key Value is to classification of documents device.What is obtained in the feature training data and treatment process S501 for having treatment process S503 to generate can Based on true key-value pair information, so that it may Machine learning classifiers are trained, key-value pair classification of documents device is generated. It is subsequent using key-value pair classification of documents device carry out online processing during, can also according to online processing obtain result after It is continuous that key-value pair classification of documents device is trained, so that key-value pair classification of documents device is more accurate to the processing of data.

Above-mentioned treatment process S501~S503 completes the training process for key-value pair classification of documents device 506, and Also some keysets are obtained to close,

S505: key-value pair information related to user is obtained from the content information of the social networks of user.In the processing In the process, can be closed according to the keyset that has determined and entity transition rule, come from such as user push away text, facebook, Key-value pair information is obtained in the content information issued on LinkedIn.Entity transition rule herein: it will have determined The content of the social networks of the key-value pair information (such as Wiki archive information of above-mentioned famous person) and the user of user is believed Consistent relationship or structure feature between breath (for example, famous person publication pushes away text) etc., are applied in social networks, and carry out The extension of seed user, to extract more key-value pair informations.

S506: the chat log of the treatment process S505 key-value pair information obtained and user is input to key-value pair archives point It is handled in class device, key-value pair information is assigned to user as the accuracy of user's Wiki archives by output.

As shown in fig. 6, its example block diagram 600 for pushing away text for the user of this paper.By to the entitled Si Diwen (Steven) The analysis for pushing away text, used the expression of " I will also remove USC " in pushing away text, USC is identified as university's title, and it is used The similar expression " removing USC " for pushing away text of certain known users, and in the Wiki archive information of the known users, contain conduct Key-value pair as " university-USC " of confirmation message.It is thus possible to " entity transition rule " is based on, by the key of " university-USC " Value is assigned to user Si Diwen to information, but user's Wiki archives based on entity transition rule can not be trusted for 100% is correct, needs to assign using trained key-value pair classification of documents device by the key-value pair information of " university-USC " Accuracy to user Si Diwen as user's Wiki archives is finally confirmed whether that the key-value pair will be accepted and believed according to the accuracy again Information.

As shown in fig. 7, it applies schematic block diagram 700 for illustrative key-value pair classification of documents device.In Fig. 7, key-value pair The input data of classification of documents device 701 is the chat log 703 of key-value pair information 702 and user, and output is key-value pair information 702 are assigned to the accuracy 704 (using the key-value pair information as the accuracy of user's Wiki archives) of user, and extend Training data 705, which can further train key-value pair classification of documents device 701, to improve key-value pair shelves 701 accuracy of case classifier.

Before being input to key-value pair classification of documents device 701, feature extraction also will do it, generate feature vector, so It is input in key-value pair classification of documents device 701 and is handled again afterwards.

The input data of key-value pair classification of documents device 701, such as are as follows: <user (user information), key is (in key-value pair information Key), value (value in key-value pair information), user log (session log obtained from personal or group chatting), User social networks (social network information of user) >.These input datas pass through key-value pair classification of documents device 701 After processing, export key-value pair information<key, value>distribute to accuracy of the active user as Wiki archives, for example, such as Fruit accuracy is greater than 0.5 it is considered that can be as the Wiki archives of the user.

In addition, following feature templates can be used during carrying out feature extraction, this feature module includes following Characteristic set, each featured items can be arbitrarily combined in characteristic set, that is, select one therein or any multinomial Combination form characteristic module:

1, the frequency for key-value pair<key, value>occur in the archive database of active user；

2, key-value pair<key, value>with active user have social relationships other users social networks content The frequency occurred in information；

3, the frequency for key-value pair<key, value>be mentioned in the chat log of active user.

4, the text term vector cosine similarity score between key and the dense vector of value.

5, the key in the chat log of active user and (=mono- user query/text of distance of the average distance between value The number of word in message between key and value).

6, the key in key-value pair information；

7, the value in key-value pair information；

8, key-value pair information<key, value>；

9, active user has had key in its Wiki archives；

10, showing as the current key in the Wiki archives of value and active user in the key-value pair information of input data There is the editing distance similarity between value；

11, the existing value of key current in the Wiki archives of active user with as in the key-value pair information of input data Text term vector similarity between value.

The classification of theme life cycle state

The multilayer circulation neural network classified for the life cycle state to a theme is had trained herein (RNN).As previously mentioned, the life cycle of a theme is classified as multiple potential time-sensitive states, such as " start, growth Or degenerate, terminate or cancel, after discussion ", the life cycle of theme is exactly to be transformed into next state from a state.For example, We show these states using following two example.

Scene 1:

User: I feels lonely；

Chat robots: how a film is seen (beginning of the theme of chat robots, " seeing a film " this answer Can be according to triple " see a film → healing → lonely " Lai Shengcheng's in knowledge mapping), (theme=see a film, State=beginning)；

User: it is good, there is anything to suggest (about watching movie) (theme=see a film, state=extension)；

Chat robots: " Spider-Man " is how, and the latest one (theme=Spider-Man, state=beginning；Father's theme =see a film, state=extension)；

User: sounding good, (theme=Spider-Man, state=front are intended to；Father's theme=see a film, state=prolong It stretches)；

Chat robots: nearest cinema wants ten minutes distances and next field was started after 30 minutes, you will buy Ticket (theme=ticket, state=beginning；Father's theme=Spider-Man, state=front are intended to；Second father's theme=see a film, State=extension)；

User: certainly, (theme=ticket, state=confirmation are booked；Father's theme=Spider-Man, state=front are intended to；Second Father's theme=see a film, state=extension)；

Chat robots: it is good, order and had here the video profile of a film；

(after one day)

User: film very interesting (theme=Spider-Man, state=end and front comment；A second father's theme=see Film, state=end)；

Chat robots: I is very glad that you like this film.

Scene 2

User: I feels lonely；

Chat robots: how a film is seen (beginning of the theme of chat robots), (theme=see a film, shape State=beginning)；

User: lose interest in (for watching movie) (theme=see that a film, state=reverse side are intended to)；

Chat robots: how is that Blues (theme=Blues, state=beginning；" Blues " this answer It is can be again according to triple " Blues → collocation → lonely " Lai Shengcheng's in knowledge mapping)；

User: it sounds good (theme=Blues, state=front are intended to)；

(playing music here)

User: I likes this (theme=Blues, state=end and front comment).

Be easy from scene 1 and scene 2 it is found out that, the life cycle state of topic detection and theme can aid in guidance Next chat robots are replied and are summarized the theme how to chat for a long time to user.Moreover, it should be appreciated that It can be obtained by some knowledge mappings with the range of theme.

The number of the life cycle state of theme can according to need and set, and can be by using fine-grained feelings Analysis system is felt to execute front/reverse side comment and analysis.

The Recognition with Recurrent Neural Network (RNN) of classification is utilized herein to encode session information and incite somebody to action by Softmax function Insertion vector projection after encoding to session is into the list of candidate state.

As shown in figure 8, it is the structural representation of the neural network device handled for the life cycle state of theme Figure comprising RNN layer 803, Softmax function handle model between RNN layer 802, sentence in input layer processing module 801, sentence 804。

Input layer processing module 801 generates sentence for each word in multiple sentences in session to be carried out vectorization Term vector indicate；

RNN layers of processing module 802 in sentence handles for the term vector expression to multiple sentences, generates entire sentence The sentence vector of son indicates；

RNN layers of processing module 803 between sentence, at multiple sentence vectors expression by RNN layers of output in sentence Reason, the session vector for exporting session indicate；

Softmax function processing module 804, for indicating to determine in given life cycle state list according to session vector The corresponding state probability of each title.

As shown in figure 9, it illustratively describes the schematic block diagram for the disaggregated model implemented by four layers of neural network 900.In the figure, each rectangle representation vector, arrow indicate the function of such as matrix-vector multiplication.The vector of each layer in Fig. 9 It is to be generated by modules processing in Fig. 8, specific:

(1) input layer 901: the processing of the layer data is executed by the input layer processing module 801 in Fig. 8.In input layer 901 In, it will each word in multiple sentences (being indicated in figure with 1~m of sentence) in words carries out vectorization, generate the word of sentence to Amount indicates.

Input layer can be expressed as one group of vector Input X:

Input X=[x₁/t₁,x₂/t₂,…,x_m/t_m] ... ... ... ... ... ... ... formula (1)

Wherein, x₁~x_mIt is indicated for the corresponding term vector of each 1~m of sentence, t₁~t_mIndicate the corresponding master of each sentence Topic, theme t₁~t_mThere may be identical themes.In the corresponding term vector expression of each sentence, and contain it is multiple to Amount (or perhaps matrix of multiple vectors composition), by taking sentence 1 as an example, corresponding term vector includes vector W_1,1~W_1,n1, sentence Sub 2 corresponding term vectors include vector W_2,1~W_2,n2, the corresponding term vector of sentence m include vector W_m,1~W_m,nm。

(2) RNN layer 902 in sentence.The processing of this layer is by RNN layer 802 in the sentence in Fig. 8.The purpose of this layer is to give birth to At the insertion of entire sentence to vector.For each sentence using two-way concurrent (Bi-directional in the layer Recurrent RNN processing), formula are expressed as follows:

H_t+1=RNN (WhhH_t+W_xhX_t+b_h) ... ... ... ... ... ... formula (2)

Wherein, W_hh、W_xhAnd b_hFor the transformation parameter of RNN layer 902 in from input layer 901 to sentence, vector H_tIt can regard Calculating the context vector generated for every wheel indicates, vector X_tThe term vector in sentence that corresponding every wheel is used when calculating.

If the number for the step of t is for circulation round to be unfolded, and vector H_tIt is the place by RNN layer 902 in sentence Reason generates final vector, it should be noted that executes loop computation in two directions, that is, from left to right and from right to left.Cause This, vector H_TIt is the result that the cascade of the vector in both direction is formed:

H_t=[H^T _{left-to-right},H^T _{right-to-left}]^T... ... ... ... ... ... ... formula (3)

(3) RNN layer 903 between sentence.The processing of this layer is by RNN layers of processing module 803 between the sentence in Fig. 8.Layer processing Purpose be to obtain the dense vector of entire session and indicate.It is still to be obtained from front layer as the algorithm that preceding layer uses Vector H_tAs input, it is corresponding to generate each sentence for the RNN processing of two-way concurrent (Bi-directional Recurrent) Vector indicates G₁~G_m, then to vector G₁~G_mSpliced, generates the vector J for corresponding to entire session.Wherein, m is input The number of sentence in session.

(4) output layer 904.In output layer 904, the vector J that RNN layer 903 between sentence generates is input to Softmax letter It is handled in number, generates theme T₁To theme T_mState probability list (share m theme), in the state probability list, Each theme T corresponds to k state, and each state corresponds to a probability value p_i, therefore, the range of i arrives (m, k) from (1,1), The state probability list of Softmax function output is the pro-bability value matrices of a m × k.

Knowledge mapping

Herein, the predominantly knowledge mapping based on concept of the knowledge mapping of use, as shown in Figure 10, for herein Knowledge mapping exemplary block diagram 1000 based on concept.In figure, concept is intuitively expressed using multiple figures, for example, train, Dog, cat, mouse, form, IE etc..

Basically, a node in figure corresponds to " concept " an of real-world objects, is that can be seen that Or it is only cogitable.For example, " dog " is such concept that can be seen that and " economy " is without directly visible form, is only The concept inferred/thought deeply can be carried out theoretically.On the other hand, a side in figure is in figure between two nodes The relationship description of " predicate form " or " attribute form ".

Answer based on reasoning generates model

As shown in figure 11, the exemplary block diagram 1100 of model is generated for the answer herein based on reasoning.Wherein, Model is after receiving user query 1101, by using attention force function f₁, (external storage source is based on for storage vector M And the vector generated) iterative processing of more rounds is carried out, to generate the answer more optimized.Below by the processor to model System is described in detail.

Store vector M:

After external storage is quantified, it is expressed as storage vector M.Storage vector M is the list of term vector, M={ m_i, Middle i=1 ... n, wherein m_iIt is fixed dimension vector.For example, in the search space of inference graph, m_iIt is each word or knowledge Vector indicates.These vectors indicate to encode the vectorization of external knowledge, context etc. from two-way RNN encoder.Scheming In 11, storage vector M can come from four storage sources, and each storage source is specially to indicate related word sequence or dependence The list of the word vector of arc.

Wherein, external storage source includes: user's Wiki archives storage 1105, session storage 1106, theme life cycle shape State stores the knowledge mapping 1108 of 1107, Opening field, these storage sources were also all introduced in the preceding part of this paper.

Pay attention to force vector X_t:

Pay attention to force vector X_tIt is based on current internal state vector S_tWith storage vector M, pass through attention force function f₁It generates , specific functional relation can indicate are as follows:

X_t=f₁(S_t, M) ... ... ... ... ... ... ... ... ... ... formula (4)

Internal state vector S_t:

Usual original state S₁It is that the last word vectors of the user query of user's input indicates, the of internal state vector T time step is by S_tIt indicates.The sequence of internal state vector is modeled by RNN, and specific functional relation can indicate are as follows:

S_t+1=RNN (S_t,X_t) ... ... ... ... ... ... ... ... ... formula (5)

Wherein, X_tIt is attention force vector above-mentioned.

Terminate door:

Door is terminated according to current internal state vector S_tGenerating random variable t_t=p (f₂(S_t)), t_tIt is that diadic becomes at random Measure (t_t=0 expression state is not finished or t_t=1 expression state terminates), function f₂It is a monolayer neural networks, for judging S_tWhether the state that terminates has been arrived, and function p is sigmoid function, is equivalent to a function f₂Output result to be mapped to [0,1] general On rate section.If t_tIt is true (true), then model stops carrying out the processing of next one, and is existed by answer generation module 1104 Time step t, which executes, replies decoded processing, generates and replies；If t_tIt is false (false), then updates attention according to formula (4) Vector X_t+1, and the vector is fed to state network to update next internal state S_t+1。

Reply generation module 1104: the movement for replying generation module 1104 is the triggering when terminating door variable and being true, is passed through Function RNN decoder to generate by word as the sentence replied.Specific functional relation can indicate are as follows:

At=p (f₃(S_t)) ... ... ... ... ... ... ... ... ... formula (6)

Wherein, at represents t-th of word in replying, function f₃For RNN decoder, function p indicates Softmax letter herein Number, for exporting the probability distribution of each candidate word in answer.

In addition, two states are also calculated using two sigmoid functions herein:

Intention state 1102: intention state includes mood care or task orientation, f₄=sigmoid (W₄*S_t+b₄), In, W₄And b₄For the parameter of Linear Combination Model, wherein W₄It is mapping matrix parameter, b₄It is offset parameter.

Answer state 1103: it is yes for replying, and it is no for not replying.f₅=sigmoid (W₅*f₄+b₅), W₅And b₅For linear combination The parameter of model, wherein W₅It is mapping matrix parameter, b₅It is offset parameter.

The above-mentioned answer based on reasoning, which generates model, can be implemented as a kind of answer generating means, as shown in figure 12, For the structural schematic diagram 1200 of the answer generating means provided herein based on reasoning, which includes attention model processing mould Block 1201, internal state vector generation module 1202, logistic regression discrimination module 1203, attention model processing module and inside State vector generation module carries out at the iteration of internal state vector of more rounds under the control of logistic regression discrimination module Reason,

Attention model processing module 1201, for raw according to the internal state vector sum external storage vector of current round At the attention force vector of current round.

Internal state vector generation module 1202, for according at least to user query vector generate initial internal state to It measures, and generates the inside shape of current round according at least to the attention force vector of the current round of internal state vector sum of previous round State vector.Specifically, the processing of internal state vector generation module can be further specifically: according to user query vector, base Initial intention state vector and initial recoil state vector in the generation of user query vector, generate initial inside shape State vector, and work as front-wheel according to what the internal state vector of previous round, the internal state vector based on the previous round generated Secondary attention force vector generates the internal state vector of current round.Wherein, it is intended that state vector may include mood care and/ Or the information in terms of task orientation.It can be between attention model processing module 1201 and internal state vector generation module 1202 The loop iteration processing of the more rounds of row, until the judgement of logistic regression discrimination module 1203 meets termination condition.

Logistic regression discrimination module 1203 differentiates for the internal state vector to current round, it is determined whether eventually Only iterative processing.

Furthermore it is also possible to include reply message generation module 1204, it is used for when iterative processing terminates, according to final round Internal state vector, generate reply message.

In addition, it can include storage vector generation module 1205, for according to user's Wiki storing data, session content One in storing data, session theme storing data and Opening field knowledge store data or any multinomial combination, Generate external storage vector.

Electronic device implementation example

The electronic device of this paper can be the electronic equipment with mobility, be also possible to less movement or non-shifting Dynamic calculating equipment.The electronic device of this paper at least has processing unit and memory, and instruction is stored on memory, and processing is single Member acquisition instruction from memory, and processing is executed, so that electronic device executes movement.

In some instances, above-mentioned Fig. 1 to Figure 12 is related to one or more modules or one or more steps or One or more treatment processes can also mutually be tied by software program with hardware circuit by software program, hardware circuit The mode of conjunction is realized.For example, above-mentioned various components or module and one or more steps all can be in system on chip (SoC) it is realized in.SoC can include: IC chip, the IC chip include following one or more: processing unit (such as central processing unit (CPU), microcontroller, microprocessing unit, digital signal processing unit (DSP)), memory, one Or the firmware of multiple communication interfaces, and/or further circuit and optional insertion for executing its function.

It as shown in figure 13, is the structural block diagram of the illustrative electronic equipment 1300 with mobility.The electronics is set Standby 1300 can be portable (or mobile) electronic equipment of small form factor.Portable (or mobile) electricity of small form factor mentioned here Sub- equipment may is that for example, cellular phone, personal digital assistant (PDA), laptop, tablet computer, individual media play Device device, wireless network viewing apparatus, personal head-wearing device, dedicated unit or the mixing including any one of above functions Device.Electronic equipment 1300 includes at least: memory 1301 and processor 1302.

Memory 1301, for storing program.In addition to above procedure, memory 1301 is also configured to store other Various data are to support the operation on electronic equipment 1300.The example of these data includes for grasping on electronic equipment 1300 The instruction of any application or method of work, contact data, telephone book data, message, picture, video etc..

Memory 1301 can realize by any kind of volatibility or non-volatile memory device or their combination, Such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable is read-only Memory (EPROM), programmable read only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk Or CD.

Memory 1301 is coupled to processor 1302 and includes the instruction being stored thereon, and described instruction is by handling Device 1302 makes electronic equipment execute movement when executing, and as the embodiment of a kind of electronic equipment, which may include: realization figure Relevant treatment process performed by 1 to Figure 12 corresponding example, processing logic and program module etc..The electronic equipment 1300 can Using as in distributed system host and/or interchanger execute corresponding function logic.

For above-mentioned processing operation, detailed description has been carried out in the embodiment of method and apparatus in front, for The detailed content of above-mentioned processing operation can equally be well applied in electronic equipment 1300, it can by what is mentioned in preceding embodiment Specific processing operation is written in memory 1301 in a manner of program, and is executed by processor 1302.

Further, as shown in figure 13, electronic equipment 1300 can also include: communication component 1303, power supply module 1304, sound Other components such as frequency component 1305, display 1306, chipset 1307.Members are only schematically provided in Figure 13, and unexpectedly Taste electronic equipment 1300 only include component shown in Figure 13.

Communication component 1303 is configured to facilitate the logical of wired or wireless way between electronic equipment 1300 and other equipment Letter.Electronic equipment can access the wireless network based on communication standard, such as WiFi, 2G, 3G, 4G and 5G or their combination. In one exemplary embodiment, communication component 1303 receives the broadcast from external broadcasting management system via broadcast channel and believes Number or broadcast related information.In one exemplary embodiment, communication component 1303 further includes near-field communication (NFC) module, with Promote short range communication.For example, can be based on radio frequency identification (RFID) technology in NFC module, Infrared Data Association (IrDA) technology surpasses Broadband (UWB) technology, bluetooth (BT) technology and other technologies are realized.

Power supply module 1304 provides electric power for the various assemblies of electronic equipment.Power supply module 1304 may include power supply pipe Reason system, one or more power supplys and other with for electronic equipment generate, manage, and distribute the associated component of electric power.

Audio component 1305 is configured as output and/or input audio signal.For example, audio component 1305 includes a wheat Gram wind (MIC), when electronic equipment is in operation mode, when such as call mode, recording mode, and voice recognition mode, microphone quilt It is configured to receive external audio signal.The received audio signal can be further stored in memory 1301 or via communication Component 1303 is sent.In some embodiments, audio component 1305 further includes a loudspeaker, is used for output audio signal.

Display 1306 includes screen, and screen may include liquid crystal display (LCD) and touch panel (TP).If screen Curtain includes touch panel, and screen may be implemented as touch screen, to receive input signal from the user.Touch panel includes one A or multiple touch sensors are to sense the gesture on touch, slide, and touch panel.Touch sensor can not only sense touching It touches or the boundary of sliding action, but also detects duration and pressure relevant with touch or slide.

Above-mentioned memory 1301, processor 1302, communication component 1303, power supply module 1304, audio component 1305 with And display 1306 can be connect with chipset 1307.Chipset 1307 can be provided in processor 1302 and electronic equipment 1300 Remaining component between interface.In addition, chipset 1307 can also provide the various components in electronic equipment 1300 to storage The communication interface mutually accessed between the access interface and various components of device 1301.

In some instances, above-mentioned Fig. 1 to Figure 12 is related to one or more modules or one or more steps or One or more treatment processes can be realized by the calculating equipment with operating system and hardware configuration.

Figure 14 its be the illustrative structural block diagram for calculating equipment 1400.For example, interchanger herein and host (are sent End main frame and reception end main frame) it can be in one or more calculating similar with the calculating equipment 1400 in stationary computers embodiment It is realized in equipment, one or more features and/or alternative features including calculating equipment 1400.It is mentioned herein to computer 1400 description simply to illustrate that, be not restrictive.Embodiment can also be known to those skilled in the relevant art It is realized in other types of computer system.

As shown in figure 14, it calculates equipment 1400 and includes one or more processors 1402, system storage 1404, and will Various system components including system storage 1404 are coupled to the bus 1406 of processor 1402.If bus 1406 indicates Ganlei The one or more of any one of bus structures of type bus structures, including it is memory bus or Memory Controller, outer Enclose bus, accelerated graphics port and processor or the local bus using any one of various bus architectures.System Memory 1404 of uniting includes read-only memory (ROM) 1408 and random access memory (RAM) 1410.Basic input/output system 1412 (BIOS) of system are stored in ROM 1408.

Computer system 1400 also has one or more following drivers: the hard disk drive for reading writing harddisk 1414, for reading or writing the disc driver 1416 of moveable magnetic disc 1418 and for reading or writing such as CD ROM, DVDROM Or the CD drive 1420 of the removable CD 1422 of other optical mediums etc.Hard disk drive 1414, disc driver 1416 and CD-ROM driver 1420 driven respectively by hard disk drive interface 1424, disk drive interface 1426 and optics Dynamic device interface 1428 is connected to bus 1406.Driver and their associated computer-readable mediums provide for computer To the nonvolatile storage of computer readable instructions, data structure, program module and other data.Although describing hard disk, can Mobile disk and removable CD, but it is also possible to use such as flash card, digital video disc, random access memory (RAM), the other kinds of computer readable storage medium of read-only memory (ROM) etc. stores data.

Several program modules can be stored on hard disk, disk, CD, ROM or RAM.These programs include operating system 1430, one or more application program 1432, other programs 1434 and program data 1436.These programs may include for example with In the program for realizing relevant treatment process performed by the corresponding example of Fig. 1 to Figure 12, processing logic and program module etc..

User can be by the input equipment of such as keyboard 1438 and pointer device 1440 etc into calculating equipment 1400 Input order and information.Other input equipment (not shown) may include microphone, control-rod, game paddle, satellite antenna, scanning Instrument, touch screen and/or touch plate, the speech recognition system for receiving voice input, the gesture for receiving gesture input It is identifying system, such.These and other input equipments can be connected by being coupled to the serial port interface 1442 of bus 1406 It is connected to processor 1402, but other interfaces (such as parallel port, game port, universal serial bus (USB) ends can also be passed through Mouthful) be attached.

Display screen 1444 is connected to bus 1406 also by the interface of such as video adapter 1446 etc.Display screen 1444 It can be outside calculating equipment 1400 or included.Display screen 1444 can show information, and as receiving user command And/or the user interface of other information (for example, passing through touch, finger gesture, dummy keyboard etc.).In addition to display screen 1444 it Outside, calculating equipment 1400 may also include other peripheral output devices (not shown), such as loudspeaker and printer.

Computer 1400 is by adapter or network interface 1450, modem 1452 or for being established by network Other means of communication are connected to network 1448 (for example, internet).It can be built-in or external modem 1452 It can be connected to bus 1406 via serial port interface 1442, as shown in figure 14, or can be used including parallel interface Another interface type is connected to bus 1406.

As used herein, term " computer program medium ", " computer-readable medium " and " computer-readable storage Medium " be used to refer to medium, hard disk such as associated with hard disk drive 1414, moveable magnetic disc 1418, removable light Disk 1422, system storage 1404, flash card, digital video disc, random-access memory (RAM), read-only memory (ROM) with And other types of physics/tangible media etc..These computer readable storage mediums (do not include communicating Jie with communication media Matter) it distinguishes and is not overlapped.Communication media usually in the modulated message signals such as carrier wave load capacity calculation machine readable instruction, Data structure, program module or other data.Term " modulated message signal " refers to so that with encoded information in the signal Mode set or change the signal of one or more feature.As an example, not a limit, communication media includes such as sound , RF, the wireless medium of infrared ray and other wireless mediums and wired medium.Each embodiment is situated between also for these communications Matter.

As indicated above, computer program and module (including application program 1432 and other programs 1434) can be stored up There are on hard disk, disk, CD, ROM or RAM.Such computer program can also pass through network interface 1450, serial port Interface 1442 or any other interface type receive.These computer programs make to succeed in one's scheme when being executed by application program or being loaded Calculation machine 1400 can be realized features of embodiments discussed herein.Therefore, these computer programs indicate computer system 1400 controller.

In this way, each embodiment further relate to include be stored in any computer-usable storage medium computer instruction/ The computer program product of code.Such code/instruction makes data when executing in one or more data processing devices Processing equipment operates as described herein.It may include the computer readable storage devices of computer readable storage medium Example includes such as RAM, hard disk drive, floppy disk drive, CD ROM drive, DVD DOM driver, compression dish driving Device, tape drive, magnetic storage device driver, optical storage apparatus driver, MEM equipment, depositing based on nanotechnology Store up the storage equipment and other types of physics/tangible computer readable storage device of equipment etc..

Example clause

A: a kind of device, comprising: attention model processing module, internal state vector generation module, logistic regression differentiate The control of module, the attention model processing module and internal state vector generation module in the logistic regression discrimination module Under, the iterative processing of the internal state vector of more rounds is carried out,

The attention model processing module, for raw according to the internal state vector sum external storage vector of current round At the attention force vector of current round；

Internal state vector generation module, for generating initial internal state vector according at least to user query vector, And the internal state of current round is generated according at least to the attention force vector of the current round of internal state vector sum of previous round Vector；

Logistic regression discrimination module differentiates for the internal state vector to current round, it is determined whether termination changes Generation processing.

B: according to device described in paragraph A, wherein the internal state vector generation module is specifically used for, according to user Query vector, the initial intention state vector based on the generation of user query vector and initial recoil state vector, generate Initial internal state vector, and according to the internal state vector of previous round, based on the internal state vector of the previous round The attention force vector of the current round generated generates the internal state vector of current round.

C: according to device described in paragraph A, wherein further include: external storage information vector generation module, for according to In family Wiki storing data, session content storing data, session theme storing data and Opening field knowledge store data One or any multinomial combination, generate the external storage vector.

D: according to device described in paragraph A, wherein further include:

Reply message generation module, for according to the internal state vector of final round, generating when iterative processing terminates Reply message.

E: according to device described in paragraph B, wherein the intention state vector includes mood care and/or task orientation The information of aspect.

A kind of F: method, comprising:

At least one first key-value pair letter related to user is obtained from the first content information of the social networks of user Breath,

The chat log of first key-value pair information and user is input in key-value pair classification of documents device and is handled, Export at least one the described accuracy of the first key-value pair information as user's Wiki archives.

G: according to method described in paragraph F, wherein further include:

At least one second key-value pair information is obtained from the archive information of social networks；

Obtain the second content information of the social networks of the value comprising second key-value pair information；

It parses the syntax dependence of second content information and extracts structure feature, generate for training the key assignments To the feature training data of classification of documents device,

Machine learning classifiers are trained using the training data, generate the key-value pair classification of documents device.

H: according to method described in paragraph G, wherein acquisition and user from the first content information of the social networks of user At least one relevant first key-value pair information includes:

It is closed according to preset keyset, using entity transition rule, first key is obtained from the first content information Value is to information.

I: according to method described in paragraph F, wherein be input to the chat record of first key-value pair information and user Processing is carried out in key-value pair classification of documents device includes:

According to preset feature templates, feature extraction is carried out to the chat record of first key-value pair information and user, Feature vector is generated, and is input in the key-value pair classification of documents device and is handled, the feature templates include such as the next item down Or the combination of any multinomial featured items:

The frequency that first key-value pair information occurs in the archive database of user；

Content of first key-value pair information in the social networks of the other users with the user with social relationships The frequency occurred in information；

The frequency that first key-value pair information is mentioned in the chat log of the user；

Key in first key-value pair information and the text term vector cosine similarity score between the dense vector of value；

The key in the chat log of the user and the average distance between described value；

Key in first key-value pair information；

Value in first key-value pair information；

First key-value pair information；

Whether there is the key in first key-value pair information in the current Wiki archives of the user；

The editor between value value corresponding with the key in the Wiki archives of the user in first key-value pair information Distance conformability degree；

The text between value in the current Wiki archives of the user in the corresponding value of key and first key-value pair information This term vector similarity.

A kind of J: electronic device, comprising:

Processing unit；And

Memory is coupled to the processing unit and includes the instruction being stored thereon, and described instruction is by described Reason unit makes described device execute movement when executing, and the movement includes:

The chat log of first key-value pair information and user is input in key-value pair classification of documents device and is handled, Exporting at least one described first key-value pair information can be as the accuracy of user's Wiki archives.

K, the electronic device according to paragraph J, wherein the movement further include:

L: according to electronic device described in paragraph J, wherein from the first content information of the social networks of user obtain with At least one relevant first key-value pair information of user includes:

A kind of M: electronic device, comprising:

Processing unit；And

Generate attention model processing module, internal state vector generation module, logistic regression discrimination module, wherein institute Attention model processing module and internal state vector generation module are stated under the control of the logistic regression discrimination module, is carried out The iterative processing of the internal state vector of more rounds,

N: according to electronic device described in paragraph M, wherein the internal state vector generation module is specifically used for, according to User query vector, the initial intention state vector based on the generation of user query vector and initial recoil state vector, Initial internal state vector is generated, and according to the internal state vector of previous round, the internal state based on the previous round The attention force vector for the current round that vector generates generates the internal state vector of current round.

O: according to electronic device described in paragraph M, wherein the movement further include: it is raw to generate external storage information vector At module, which is used for according to user's Wiki storing data, session content storing data, meeting One or any multinomial combination in theme storing data and Opening field knowledge store data are talked about, the outside is generated Store vector.

P: according to electronic device described in paragraph M, wherein the movement further include: generate reply message generation module, institute Reply message generation module is stated, for according to the internal state vector of final round, generating and replying letter when iterative processing terminates Breath.

Q: according to electronic device described in paragraph N, wherein the intention state vector includes mood care and/or task It is orientated the information of aspect.

A kind of R: device, comprising:

Input layer processing module generates sentence for each word in multiple sentences in session to be carried out vectorization Term vector indicates；

RNN layers of processing module in sentence handles for the term vector expression to multiple sentences, generates entire sentence Sentence vector indicates；

RNN layers of processing module between sentence, at multiple sentence vectors expression by RNN layers of output in the sentence Reason, the session vector for exporting the session indicate；

Softmax function processing module determines given life cycle state list for indicating according to the session vector In the corresponding state probability of each title.

A kind of S: electronic device, comprising:

Processing unit；And

Memory is coupled to the processing unit and includes the instruction being stored thereon, and described instruction is by described Reason unit makes described device execute movement when executing, and the movement includes: generation input layer processing module, in sentence at RNN layers RNN layers of processing module and Softmax function processing module between module, sentence are managed,

Wherein, input layer processing module generates sentence for each word in multiple sentences in session to be carried out vectorization The term vector of son indicates；

Conclusion

It is distinguished between the hardware and software realization of many aspects of system little；Usually (but simultaneously using hardware or software Not always, because in some contexts, the selection between hardware and software can become significant) it is to indicate cost and efficiency tradeoff Design alternative.In the presence of may be implemented processing described herein and/or system and/or other technologies (for example, hardware, software, with And/or firmware) various carrying tools, and preferably carrying tool will be with disposing the processing and/or system and/or other skills The background of art and change.For example, the realization side can choose main hard if realization side determines that speed and accuracy are most important Part and/or firmware carrying tool；If flexibility is most important, which can choose main software realization；Alternatively, furthermore Again alternatively, which can choose some combinations of hardware, software and/or firmware.

Foregoing detailed description is via using block diagram, flow chart and/or example to elaborate the device and/or processing Various embodiments.Include one or more functions and/or operation as this block diagram, flow chart and/or example, It will be appreciated by those skilled in the art that each function and/or operation in this block diagram, flow chart or example can be independent Ground and/or jointly, by the hardware, software, firmware of wide scope, or actually any combination thereof is realized.In a reality It applies in mode, several parts of purport described herein can be via specific integrated circuit (ASIC), field programmable gate array (FPGA), digital signal processor (DSP) or other integrated formats are realized.However, those skilled in the art should recognize It arrives, some aspects of embodiment disclosed herein entirely or partially can be realized equally in integrated circuits, real It is now one or more computer programs for operating on one or more computers (for example, being embodied as operating in one Or more one or more programs in computer system), be embodied as running one on one or more processors A or more program (for example, being embodied as operating in one or more programs on one or more microprocessors) is real It is now firmware, or is actually embodied as any combination thereof, and according to the disclosure, designs circuit and/or write for software And/or the code of firmware is completely in the technology of those skilled in the art.In addition, it should be apparent to those skilled in the art that It is that the mechanism of theme described herein can be distributed as program product in a variety of forms, and the example of theme described herein Exemplary embodiment is applicable in, and unrelated be used to actually execute the certain types of signal bearing medium of the distribution.Signal is held Carry medium example include but is not limited to, below: recordable-type media, as floppy disk, hard disk drive (HDD), the close disk of matter (CD), Digital versatile disc (DVD), digital magnetic tape, computer storage etc.；And transmission type media, such as number and/or analogue communication medium (for example, fiber optic cable, waveguide, wired communications links, wireless communication link etc.).

It should be recognized by those skilled in the art that device and/or processing are described by mode set forth herein, and this Afterwards, it is in the art common for the device described in this way and/or processing being integrated into data processing system using engineering practice. That is, device described herein and/or at least part of processing can be integrated into data processing system via the experiment of reasonable amount In system.Skilled artisan recognize that common data processing system generally includes one of the following or more It is multiple: system unit shell, video display devices, the memory of such as volatile and non-volatile memory, such as micro process The meter of the processor of device and digital signal processor, such as operating system, driver, graphical user interface and application program One or more interactive apparatus of entity, such as touch tablet or touch screen are calculated, and/or including feedback loop and control electricity The control system of motivation is (for example, for sensing the feedback of position and/or the feedback of speed；For moving and/or adjusting component and/or number The control motor of amount).Common data processing system, which can use any suitable commercial, can obtain component to realize, such as usually in number According to those of being found in calculating/communication and/or network communication/computing system.

Theme described herein sometimes illustrates different components in different other components or coupled. It is clear that the framework described in this way is only exemplary, and indeed, it is possible to realize many for obtaining identical function Other frameworks.On conceptual sense, all effectively " it is associated with " for obtaining any arrangement of component of identical function, so that Function is wished in acquisition.Therefore, any two components combined herein to obtain specific function can be seen as " related each other Connection " wishes function so as to obtain, and unrelated with framework or intermediate module.Similarly, any two components associated in this way can be with It is considered and " is operably connected " each other, or " being operatively coupled to ", wish function to obtain, and can so associated Two components, which can also be considered, " to be operatively coupled to " each other, wish function to obtain.The specific example being operatively coupled to Including but not limited to, it can physically cooperate and/or physically interactive component and/or can wirelessly interact and/or wirelessly hand over Mutual component and/or in logic interaction and/or in logic can interactive component.

Background can be directed to for any plural number substantially used herein and/or singular references, those skilled in the art And/or application pluralizes from plural number translation singularization and/or from odd number translation at the appropriate time.For clarity, it is various odd number/ Majority displacement can be illustrated definitely herein.

It will be appreciated by those skilled in the art that in general, it is as used herein, and especially in the appended claims In the term that is used (for example, the main body of the appended claims) be generally intended as " open " wording (for example, wording " packet Include (including) " it should be construed as " including but not limited to ", wording " having (having) " should be construed as " at least having " It should be construed as " including but not limited to " etc.).Those skilled in the art are further appreciated that, if it is desired to certain amount of introduction power Benefit requires to enumerate, then this intention will be stated clearly in the claim, and in the case where no these are enumerated, no There are this intentions.For example, claims provided below may include using introductory phrase " at least one to help to understand It is a " and " one or more " introduce claim recitation.However, should not be considered as using this phrase, imply by indefinite The claim recitation that article " one (a) " or " one (an) " are introduced will include this any specific weights for introducing claim recitation Benefit requires to be limited to only comprising a this invention enumerated, even if same claim includes introductory phrase " one or more It is multiple " or "at least one" and such as " one (a) " or " one (an) " indefinite article (for example, " one (a) " or " one (an) " is logical It should often be construed as meaning "at least one" or " one or more ")；Its claim recitation is introduced for using and It is equally remained for the definite article used true.In addition, even if clearly state it is certain amount of introduce claim recitation, this Field technical staff at least means institute's recited number (for example, " two it should also be realized that this enumerate should usually be construed as It is a to enumerate " only enumerate generally mean that at least two enumerate or two or more in the case where no other modifiers It is multiple to enumerate).Moreover, in using those of the convention for being similar to " at least one of A, B and C etc. " example, it is general next It says, this syntactic structure wishes those skilled in the art in the sense and should understand that this convention (for example, " having A, B and C At least one of system " should include but is not limited to independent A, independent B, independent C, A and B together, A and C together, B With C together and/or A, B and C system together etc.).Using similar to the used of " at least one of A, B or C etc. " In those of example example, in general, this syntactic structure wishes those skilled in the art in the sense and should understand that this be used to Example (for example, " system at least one of A, B or C " should include but is not limited to independent A, independent B, independent C, A and B together, A and C together, B and C together and/or A, B and C system together etc.).Those skilled in the art should also It is realized that in fact, no matter any adversative and/phrase that two or more alternative terms are presented are (in description, right In claim, still in the accompanying drawings it) is to be understood as, it is contemplated that including any of these terms, these terms or two A possibility that a term.For example, phrase " A or B " is to be understood as, including a possibility that " A " or " B " or " A and B ".

" implementation ", " implementation ", " some implementations ", or " other realization sides are directed in this specification The reference of formula " can be it is meant that can be wrapped in conjunction with special characteristic, structure or the characteristic that one or more implementations describe It includes at least some implementations, but is not necessarily included in all implementations.Different " the realizations occurred in foregoing description Mode ", " implementation ", or " some implementations " need not be quoted all for same implementation.

Although using distinct methods and System describe and showing particular exemplary technology, those skilled in the art should Understand, in the case where not departing from claimed theme, various other modifications can be carried out, and can replace equivalent. In addition, many modify so as to adapt to for claimed can be carried out in the case where not departing from central concept described herein Theme introduction specific condition.It is therefore desirable to the theme of protection is not limited to disclosed particular example, but this requirement The theme of protection can also include all realizations fallen into the range of the appended claims and its equivalent.

Although this theme of the dedicated language description of structural features and or methods of action has been used, it is to be understood that appended power Theme defined in sharp claim is not necessarily limited to described specific feature or action.But these specific features and movement are It is disclosed as the illustrative form for realizing the claim.

Unless specifically stated otherwise, otherwise within a context be understood that and be used generally conditional statement (such as " energy ", " can ", " possibility " or " can with ") indicate that particular example includes and other examples do not include special characteristic, element and/or step. Therefore, such conditional statement is generally not intended to imply that requires feature, element for one or more examples in any way And/or step, or one or more examples necessarily include inputting or mentioning for the logic of decision, with or without user Show, whether to include or to execute these features, element and/or step in any specific embodiment.

Unless specifically stated otherwise, it should be understood that joint language (such as phrase " at least one in X, Y or Z ") indicates item, word Language etc. can be any one of X, Y or Z, or combinations thereof.

Any customary description, element or frame should be understood to potentially in flow chart described in described herein and/or attached drawing Expression include the code of one or more executable instructions for realizing logic function specific in the routine or element module, Segment or part.Replacement example is included in the range of example described herein, and wherein each element or function can be deleted, or It is inconsistently executed with sequence shown or discussed, including substantially simultaneously executes or execute in reverse order, this depends on In related function, as those skilled in the art also will be understood that.

It should be emphasized that can to above-mentioned example, many modifications may be made and modification, element therein shows as other are acceptable Example is understood that like that.All such modifications and variations are intended to include herein within the scope of this disclosure and by following right Claim protection.

Those of ordinary skill in the art will appreciate that: realize that all or part of the steps of above-mentioned each method embodiment can lead to The relevant hardware of program instruction is crossed to complete.Program above-mentioned can be stored in a computer readable storage medium.The journey When being executed, execution includes the steps that above-mentioned each method embodiment to sequence；And storage medium above-mentioned include: ROM, RAM, magnetic disk or The various media that can store program code such as person's CD.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution The range of scheme.

Claims

1. a kind of device, comprising: attention model processing module, internal state vector generation module, logistic regression discrimination module, The control of the attention model processing module and the internal state vector generation module in the logistic regression discrimination module Under, the iterative processing of the internal state vector of more rounds is carried out,

The attention model processing module, for being worked as according to the generation of the internal state vector sum external storage vector of current round The attention force vector of preceding round；

Internal state vector generation module, for generating initial internal state vector according at least to user query vector, and extremely Few internal state vector that current round is generated according to the attention force vector of the current round of internal state vector sum of previous round；

Logistic regression discrimination module differentiates for the internal state vector to current round, it is determined whether terminates at iteration Reason.

2. the apparatus according to claim 1, wherein the internal state vector generation module is specifically used for, according to user Query vector, the initial intention state vector based on the generation of user query vector and initial recoil state vector, generate Initial internal state vector, and according to the internal state vector of previous round, based on the internal state vector of the previous round The attention force vector of the current round generated, generates the internal state vector of current round.

3. the apparatus according to claim 1, wherein further include: external storage information vector generation module, for according to In family Wiki storing data, session content storing data, session theme storing data and Opening field knowledge store data One or any multinomial combination, generate the external storage vector.

4. the apparatus according to claim 1, wherein further include:

Reply message generation module, for according to the internal state vector of final round, generating and replying when iterative processing terminates Information.

5. the apparatus of claim 2, wherein the intention state vector includes mood care and/or task orientation The information of aspect.

6. a kind of method, comprising:

At least one first key-value pair information related to user is obtained from the first content information of the social networks of user,

The chat log of first key-value pair information and user is input in key-value pair classification of documents device and is handled, is exported At least one described first key-value pair information is assigned to the user as the accuracy of user's Wiki archives.

7. according to the method described in claim 6, wherein, further includes:

It parses the syntax dependence of second content information and extracts structure feature, generate for training the key-value pair shelves The feature training data of case classifier,

8. according to the method described in claim 7, wherein, obtained from the first content information of the social networks of user and user At least one relevant first key-value pair information includes:

It is closed according to preset keyset, using entity transition rule, first key-value pair is obtained from the first content information Information.

9. a kind of electronic device, comprising:

Processing unit；And

Memory is coupled to the processing unit and includes the instruction being stored thereon, and described instruction is single by the processing Member makes described device execute movement when executing, and the movement includes:

10. electronic device according to claim 9, wherein the movement further include:

11. electronic device according to claim 9, wherein obtained from the first content information of the social networks of user At least one first key-value pair information related to user includes:

12. a kind of electronic device, comprising:

Processing unit；And

Generate attention model processing module, internal state vector generation module, logistic regression discrimination module, wherein the note Power model processing modules of anticipating and internal state vector generation module carry out more wheels under the control of the logistic regression discrimination module The iterative processing of secondary internal state vector,

13. electronic device according to claim 12, wherein the internal state vector generation module is specifically used for, root The initial intention state vector and initial recoil state generated according to user query vector, based on user query vector to Amount, generate initial internal state vector, and according to the internal state vector of previous round, based on the inside shape of the previous round The attention force vector for the current round that state vector generates generates the internal state vector of current round.

14. electronic device according to claim 12, wherein the movement further include: generate external storage information vector Generation module, the external storage information vector generation module be used for according to user's Wiki storing data, session content storing data, One in session theme storing data and Opening field knowledge store data or any multinomial combination generate described outer Portion stores vector.

15. electronic device according to claim 12, wherein the movement further include: reply message generation module is generated, The reply message generation module, for according to the internal state vector of final round, generating and replying when iterative processing terminates Information.

16. electronic device according to claim 13, wherein the intention state vector includes mood care and/or appoints The information of business orientation aspect.

17. a kind of device, comprising:

Input layer processing module, for each word in multiple sentences in session to be carried out vectorization, generate the word of sentence to Amount indicates；

RNN layers of processing module in sentence handles for the term vector expression to multiple sentences, generates the sentence of entire sentence Vector indicates；

RNN layers of processing module between sentence, for multiple sentence vectors expression of RNN layers of output in the sentence to be handled, The session vector for exporting the session indicates；

Softmax function processing module, it is each in given life cycle state list for indicating to determine according to the session vector The corresponding state probability of a title.