CN115329780A

CN115329780A - Multi-turn dialogue rewriting method, device, electronic equipment and medium

Info

Publication number: CN115329780A
Application number: CN202210987865.6A
Authority: CN
Inventors: 代文; 刘岩; 陈帅
Original assignee: Beijing Xiaomi Mobile Software Co Ltd; Beijing Xiaomi Pinecone Electronic Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd; Beijing Xiaomi Pinecone Electronic Co Ltd
Priority date: 2022-08-17
Filing date: 2022-08-17
Publication date: 2022-11-11

Abstract

The disclosure relates to a method, an apparatus, an electronic device and a medium for multi-turn dialog rewriting. The method comprises the following steps: acquiring a statement to be rewritten and a historical statement of the statement to be rewritten; extracting keywords of the sentence to be rewritten and the historical sentence; inputting the sentence to be rewritten, the historical sentence and the keyword into a preset multi-turn dialogue rewriting model to obtain a target sentence; and replacing the target statement into the statement to be rewritten. Therefore, the method does not need to define the intention and the slot position system in advance, and improves the universality and the flexibility of multi-turn conversation rewriting. And the rewriting mode is not limited to deletion, replacement, retention and insertion, so that the rewriting capability is stronger, the rewriting quality is improved, the rewritten target sentence is more consistent with the real intention of the user, and the effect of multi-turn conversation is improved.

Description

Multi-turn dialogue rewriting method, device, electronic equipment and medium

Technical Field

The present disclosure relates to the field of artificial intelligence technologies, and in particular, to a method, an apparatus, an electronic device, and a medium for multi-turn dialog rewriting.

Background

The man-machine conversation is a technology for a machine to understand and use Natural Language to realize man-machine communication, is an important research work in the field of Natural Language Processing (NLP), and has been widely applied in the fields of chat robots, intelligent customer service, voice assistants, and the like. The dialog system can be divided into a single-turn dialog and a multi-turn dialog according to the number of turns of the man-machine dialog. The technology of single-turn dialog is mature, and mainly includes search-type and generation-type methods. But the dialog system is not good enough in a multi-turn dialog scenario.

In a multi-turn dialog scenario, when a user inputs through a Speech form, there may be situations such as omission, referral, ASR (Automatic Speech Recognition) Recognition error, etc., so that it is difficult for a dialog system to judge the user's intention according to the current turn of sentences, and it is necessary to understand the intention in combination with the above. For example, the first turn of sentences in the man-machine conversation is: "what the market flowers in Beijing City are", the second round sentence is "wool in that Shanghai", the user omits "market flowers" in the question asked in the second round, and the user actually intends to ask "what the market flowers in Shanghai" are. Therefore, the dialog system needs to have strong multi-turn rewriting capability, and understand the real intention of the user in combination with the context, so as to solve the problems of omission, pronouncing and ASR recognition errors existing in the question of the user. When the real intention of the user is understood through the multi-turn dialogue rewriting technology and is expressed by a sentence, the multi-turn dialogue is converted into a single-turn dialogue, and the dialogue system can provide more accurate answers for the user.

Disclosure of Invention

To overcome the problems in the related art, the present disclosure provides a new multi-turn dialog rewriting method, apparatus, electronic device, and medium.

According to a first aspect of the embodiments of the present disclosure, there is provided a multi-round dialog rewriting method, including:

acquiring a statement to be rewritten and a historical statement of the statement to be rewritten;

extracting keywords of the sentences to be rewritten and the historical sentences;

inputting preset multi-turn dialogue rewriting models according to the sentences to be rewritten, the historical sentences and the keywords to obtain target sentences;

and replacing the target statement into the statement to be rewritten.

Optionally, the obtaining a target sentence by inputting a preset multi-turn dialogue rewrite model according to the sentence to be rewritten, the historical sentence, and the keyword includes:

inputting the statements to be rewritten, the historical statements and the keywords into the multi-turn dialogue rewrite model, and acquiring a plurality of output rewrite statements and the probability of each rewrite statement;

and determining a target sentence from the plurality of rewrite sentences according to the probability of each rewrite sentence.

Optionally, the determining a target statement from the plurality of rewrite statements according to the probability of each rewrite statement includes:

determining a preset number of candidate sentences with higher probability values from the plurality of rewritten sentences according to the probability of each rewritten sentence;

for each candidate statement, scoring the candidate statement according to the fluency of the candidate statement and/or the consistency of the candidate statement and the keyword;

and determining a target sentence from the candidate sentences according to the respective scores of each candidate sentence.

and determining the rewritten sentence with the highest probability as the target sentence according to the probability of each rewritten sentence.

Optionally, the obtaining a target sentence by inputting the sentence to be rewritten, the history sentence, and the keyword into a preset multi-turn dialogue rewriting model includes:

splicing the sentence to be rewritten, the historical sentence and the keyword to obtain a spliced text;

and inputting the spliced text into a preset multi-turn dialogue rewriting model to obtain a target sentence.

Optionally, the multi-round dialogue rewrite model is a generative model, and the multi-round dialogue rewrite model is obtained by training with a sample sentence to be rewritten in a first training sample, a historical sample sentence corresponding to the sample sentence to be rewritten, and keywords of the sample sentence to be rewritten and the historical sample sentence as model input parameters, and with a first sample labeling sentence in the first training sample as model output parameters.

Optionally, the extracting the keywords of the to-be-rewritten sentence and the historical sentence includes:

inputting the sentences to be rewritten and the historical sentences into a preset sequence labeling model to obtain output keywords;

the sequence labeling model is obtained by training with sample sentences to be rewritten in a second training sample and historical sample sentences corresponding to the sample sentences to be rewritten as model input parameters and with keywords of the sample sentences to be rewritten and the historical sample sentences in the second training sample as model output parameters.

Optionally, the keywords of the to-be-rewritten sample sentence and the historical sample sentence in the second training sample are obtained by:

performing word segmentation on the sentence to be rewritten and the historical sentence in the second training sample to obtain a first word segmentation result;

acquiring a second sample marking statement corresponding to the sample statement to be rewritten in the second training sample, and performing word segmentation on the second sample marking statement and removing stop words to obtain a second word segmentation result;

and determining whether the second segmentation result comprises the segmentation vocabulary or not for each segmentation vocabulary in the first segmentation result, and determining the segmentation vocabulary as the keywords of the sample sentence to be rewritten and the historical sample sentence when the second segmentation result comprises the segmentation vocabulary.

According to a second aspect of the embodiments of the present disclosure, there is provided a multi-round dialog rewriting device including:

a first obtaining module configured to obtain a statement to be rewritten and a history statement of the statement to be rewritten;

an extraction module configured to extract keywords of the to-be-rewritten sentence and the history sentence;

a second obtaining module configured to obtain a target sentence by inputting the sentence to be rewritten, the history sentences and the keywords into a preset multi-turn dialogue rewriting model;

a replacement module configured to replace the target statement into the statement to be rewritten.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

obtaining a statement to be rewritten and a historical statement of the statement to be rewritten;

extracting keywords of the sentence to be rewritten and the historical sentence;

inputting the sentence to be rewritten, the historical sentence and the keyword into a preset multi-turn dialogue rewriting model to obtain a target sentence;

and replacing the target statement into the statement to be rewritten.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the method provided by the first aspect of the present disclosure.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

by adopting the technical scheme, after keywords are extracted from the sentences to be rewritten and the historical sentences to obtain the keywords, the sentences to be rewritten are rewritten according to the sentences to be rewritten, the historical sentences and the keywords to obtain the target sentences. Therefore, the method does not need to define intention and slot position systems in advance, and improves the universality and flexibility of multi-turn conversation rewriting. In addition, the rewriting mode is not limited to deletion, replacement, reservation and insertion, so that the rewriting capability is stronger, the rewriting quality is improved, the rewritten target statement better accords with the real intention of a user, and the effect of multi-turn conversation is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flow diagram illustrating a method for multi-pass dialog rewriting according to an example embodiment.

FIG. 2 is a diagram illustrating a sequence annotation model in accordance with an exemplary embodiment.

Fig. 3 is a flowchart illustrating a step S13 of fig. 1 according to an exemplary embodiment.

FIG. 4 is a block diagram illustrating a multi-turn dialog rewriting device, according to an example embodiment.

FIG. 5 is a block diagram illustrating an electronic device in accordance with an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

It should be noted that all actions of acquiring signals, information or data in the present application are performed under the premise of complying with the corresponding data protection regulation policy of the country of the location and obtaining the authorization given by the owner of the corresponding device.

The ways of rewriting a plurality of dialog rounds in the related art are mainly classified into two ways. The first is a DST (dialog state tracking) based multi-round dialog rewriting method, which is mainly directed to task-based dialogs. Specifically, first, a set of intents and slot systems is set for a specific field. For example, in the field of booking airline tickets, possible intentions include flight shift inquiry, flight price inquiry and the like, related slots may include dates, departure places, arrival places, flight numbers and the like, and the intentions and slots need to be clearly defined to form a complete system. And then, extracting values of relevant slot positions according to each round of conversation between the user and the conversation system, and recording the values into the DST. When the expression of the user statement is incomplete, the value of the relevant slot position can be obtained from the DST, the user statement in the current round is rewritten, and the statement information is supplemented. However, the solution is not flexible enough and has low versatility because different intentions and slot position systems need to be set for different fields. In addition, the system of intentions and slots needs to be defined in advance, which is time-consuming, labor-consuming and relatively dependent on field experience.

The second is a multi-round dialogue rewriting method based on sequence labels, mainly aiming at open field dialogue. Specifically, firstly, inputting a history statement and a current round of statements, and labeling a sequence label of each word of the current round of statements through a sequence labeling model, wherein the sequence label comprises deletion, replacement and retention. And performing corresponding processing on each word according to the labeled sequence label. In addition, the rewriting position and the rewriting content can be marked, and the rewriting content is spliced to the rewriting position to obtain the rewritten sentence. However, this solution can only rewrite the current round of sentences word by word, the rewriting method is relatively hard, the rewriting result may not be smooth enough, and the rewriting effect is poor when the default content of the sentence is many or the ASR is wrong.

In view of the above, the present disclosure provides a new method, an apparatus, an electronic device, and a medium for multi-turn dialog rewriting, so as to improve the versatility and flexibility of multi-turn dialog rewriting, and improve the quality of the rewriting result, thereby improving the effect of multi-turn dialog.

FIG. 1 is a flow diagram illustrating a method for multi-pass dialog rewriting according to an example embodiment. As shown in fig. 1, the method may include the following steps.

In step S11, a sentence to be rewritten and a history sentence of the sentence to be rewritten are acquired.

It should be understood at first that the sentence to be rewritten is a user sentence input by a user in a human-machine conversation.

Secondly, it should be understood that the sentence to be rewritten is a sentence whose user's sentence intention is ambiguous and incomplete, but the sentence whose user's true intention can be clarified in conjunction with the context. Illustratively, the first round of user statements is "what is the nickname for the guan-Yu in the three kingdom definition" and the second round of user statements is "what is the weapon for him". The second round of user sentences are ambiguous in intention and incomplete, but the first round of user sentences can tell that the true intention of the second round of user sentences is to ask "what the weapon used to turn off the feather is", and therefore the second round of user sentences "what the weapon used by him is" the sentence to be rewritten. In addition, the history statement of the statement to be rewritten refers to a statement that contains more comprehensive information and is input by a user before the statement to be rewritten is input, and the history statement may be one or more user statements. Following the above example, if the second round of user statement "what is a weapon for him" is the to-be-rewritten statement, the history statement is the first round of user statement "what is a nickname for closing a feather in the three kingdoms of the rehearsal".

In step S12, keywords of the sentence to be rewritten and the history sentence are extracted.

In the present disclosure, the keyword may be a word included in a history sentence or a word included in a sentence to be rewritten, which can reflect a real intention of the user.

In step S13, a target sentence is acquired by inputting a sentence to be rewritten, a history sentence, and a keyword into a preset multi-turn dialogue rewrite model.

In step S14, the target sentence is replaced into the sentence to be rewritten.

In the present disclosure, a target sentence is a sentence generated by rewriting a sentence to be rewritten, and the target sentence is a complete and definite sentence. Therefore, after the target sentence is determined, the target sentence is replaced into the sentence to be rewritten. The subsequent dialogue system can provide accurate response for the user according to the target sentence, and the effect of multi-turn dialogue of the user is improved.

By adopting the technical scheme, the keywords of the sentences to be rewritten and the historical sentences are extracted, the sentences to be rewritten, the historical sentences and the keywords are input into a preset multi-turn dialogue rewriting model to obtain the target sentences, and the target sentences are replaced into the sentences to be rewritten. Therefore, the method does not need to define intention and slot position systems in advance, and improves the universality and flexibility of multi-turn conversation rewriting. And the rewriting mode is not limited to deletion, replacement, retention and insertion, so that the rewriting capability is stronger, the rewriting quality is improved, the rewritten target sentence is more consistent with the real intention of the user, and the effect of multi-turn conversation is improved.

In one embodiment, the sentence to be rewritten and the keywords of the historical sentences can be extracted through a machine learning-based manner. Illustratively, keywords of the sentences to be rewritten and the historical sentences can be extracted according to the sequence labeling model. For example, the specific implementation manner of extracting the keywords of the to-be-rewritten sentence and the history sentence in step S12 in fig. 1 is as follows: and inputting the sentences to be rewritten and the historical sentences into a preset sequence labeling model, and acquiring the output keywords. The sequence annotation model can be obtained through the following method:

(1) In order to distinguish the training samples used for training the multi-turn dialogue adaptation model from the training samples used for training the multi-turn dialogue adaptation model, the training samples used for training the multi-turn dialogue adaptation model are hereinafter referred to as first training samples, the training samples used for training the multi-turn dialogue adaptation model are hereinafter referred to as second training samples, and a plurality of second training samples are required for training the multi-turn dialogue adaptation model.

Illustratively, first, a collection of session samples of human-machine conversations is obtained, each session sample consisting of multiple rounds of conversations between a user and a dialog system. And then, only the user statements are reserved for removing the system statements replied by the dialog system in the dialog samples, and only the valid dialog samples are reserved for removing the invalid dialog samples, wherein the valid dialog samples refer to the dialog samples of which the last round of user statements are incomplete and have ambiguous semantics, but the real intention of the user can be clarified by combining with other statements in the dialog samples. For example, a conversation sample including a first sample user statement Q1 "what is a downtown in beijing", a second sample user statement Q2 "what is a downtown in guangzhou", and a third sample user statement Q3 "woollen in shanghai" is a valid conversation sample. And the last round of sample user statements in the valid conversation sample are used as sample statements to be rewritten, and other user statements in the valid conversation sample are used as historical sample statements corresponding to the sample statements to be rewritten.

Then, the real intention of the last round of sample user statements in the manually labeled effective conversation sample is obtained. And taking the real intention as a second sample marking statement corresponding to the sample statement to be rewritten. For example, for the valid conversation sample described above, the second sample corresponding to Q3 is labeled with the statement "what is the market flower in shanghai".

And then, determining keywords of the sample statement to be rewritten and the historical sample statement according to the effective conversation sample and the second sample labeling statement. And constructing a second training sample according to the keyword, the sample sentence to be rewritten and the historical sample sentence corresponding to the sample sentence to be rewritten.

In one possible implementation, the keywords of the to-be-rewritten sample sentence and the historical sample sentence in the second training sample can be obtained by: performing word segmentation on a sample sentence to be rewritten and a historical sample sentence in a second training sample to obtain a first word segmentation result, obtaining a second sample labeling sentence corresponding to the sample sentence to be rewritten in the second training sample, performing word segmentation on the second sample labeling sentence, and removing stop words to obtain a second word segmentation result; and determining whether the second segmentation result comprises the segmentation vocabulary or not aiming at each segmentation vocabulary in the first segmentation result, and determining the segmentation vocabulary as the keywords of the sample sentence to be rewritten and the historical sample sentence when the second segmentation result comprises the segmentation vocabulary.

In the present disclosure, the stop words may be some words preset according to actual needs, for example, the stop words may include "what", and the like. It should be understood that the word segmentation technique is rather mature and is not limited by the present disclosure.

By way of example, assuming that Q1 "what is a flower in beijing" and Q2 "what is a flower in guangzhou" is a historical sample sentence of the second training sample, Q3 "what is the sea" is a to-be-rewritten sample sentence of the second training sample, and a second sample annotation sentence "what is a flower in the sea" corresponding to the to-be-rewritten sample sentence, the keywords determined in the above manner are "flower in market" and "sea". For example, the word corresponding to the position may be represented by a numeral 0 as a non-keyword, and the word corresponding to the position may be represented by a numeral 1 as a keyword, and accordingly, the second training sample may be as shown in table 1.

TABLE 1

North China	Jing made of Chinese medicinal materials	City (R)	Is/are as follows	City (R)	Flower (A. B. A. B. A	Is that	Sundries	Chinese character' Tao	，	All-grass of Longtube Fang	State of the year	City (R)	Is/are as follows	Market (a)	Flower (A. B. A. B. A	Is that	Sundries	Chinese character' Tao	，	That	On the upper part	Sea water	Is/are as follows	Woolen cloth
																									0	0	0	0	1	1	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	1	1	0	0

It should be understood that, for convenience of description, the above description is only described by taking one second training sample as an example. In practical applications, the plurality of second training samples may be obtained in the above manner, which is not specifically limited by the present disclosure.

(2) And training by taking the sample sentence to be rewritten in the second training sample and the historical sample sentence corresponding to the sample sentence to be rewritten as model input parameters and taking the keywords of the sample sentence to be rewritten and the historical sample sentence in the second training sample as model output parameters to obtain the sequence labeling model.

In a possible embodiment, each history sample sentence and the sample sentence to be rewritten are connected by a preset symbol, for example, by a comma, as a model input parameter, and an output parameter of the model is a set of sequences with values of 0 or 1.

Illustratively, FIG. 2 is a schematic diagram illustrating an order annotation model according to an exemplary embodiment. As shown in fig. 2, the sequence annotation model includes a cascade of BERT and CRF models. In fig. 2, a text formed by splicing a history sample sentence and a sample sentence to be rewritten is used as a model input parameter, a word in the text is marked as a token, and an input text sequence can be expressed as [ token ₁ ,token ₂ ,…,token _K ]And K represents the character length of the text spliced by the historical sample statement and the sample statement to be rewritten. Among them, in FIG. 2, [ CLS]、[SEP]And text sequences as input to the BERT model. The BERT model learns the semantics implied by each token according to the context. After BERT, a vector representing each token is obtained, denoted by o, and then the sequence obtained after the BERT model is represented by [ C, o ₁ ,o ₂ ,…,o _K ,S]. Wherein C represents [ CLS ]]The expression vector corresponding to the character, S represents [ SEP ]]The characters correspond to the representative vectors.

In the present disclosure, a sequence labeling system adopted by the sequence labeling model includes two kinds of tags, i.e., 0 tag and 1 tag, where 1 represents that the vocabulary corresponding to the position is a keyword, and 0 represents that the vocabulary corresponding to the position is a non-keyword. Sequence [ C, o ] output from BERT model ₁ ,o ₂ ,…,o _K ,S]The CRF model is input. The CRF model takes into account the transition probabilities between two adjacent labels. Thus, the loss function of the model not only considers the likelihood probability of each label, but also considers the transition probability between adjacent labels. For a sequence of labels

token _i Corresponding label

Has a likelihood probability of

Also known as the transmit probability. In addition, assuming that the transition matrix is a, a is a 2 × 2 matrix (2 corresponds to 0/1 of two labels), and each element in the matrix a represents the transition probability from one label to another label. From

To

Has a transition probability of

Then the sequence

The score of (2) can be expressed as the sum of the emission probability and the transition probability as shown in equation (1), and the loss function of the sequence annotation model as shown in equation (2):

wherein Y characterizes all possible annotation sequences generated in the CRF model in equation (2).

Parameter values of the training parameters can be preset when the sequence labeling model is trained. The training parameters may include a maximum character length max _ seq _ length, a training batch size, a learning rate learning _ rate, a training round number num _ train _ epochs, and a parameter λ. For example, in the case of a liquid, set max _ seq _ length =250, train _batch _ size =16, leaving _ rate =10 ^-5 Num _ train _ epoch =50, λ =0.7. Thus, pressAnd training by using the second training sample according to the training parameters to obtain a sequence labeling model.

After the sequence labeling model is obtained, rewriting sentences and historical sentences are input into the sequence labeling model, so that labeling sequences output by the sequence labeling model can be obtained, then sequence segments with labeling labels as 1 are selected from the labeling sequences, and original tokens corresponding to the sequence segments are marked as keywords.

Therefore, the keywords are obtained in a machine learning mode, and the intellectualization of obtaining the keywords is improved.

Similarly, in the present disclosure, the target sentence may also be obtained by rewriting the sentence to be rewritten in a machine learning manner. In one possible implementation mode, the sentence to be rewritten, the historical sentence and the keyword are input into a multi-turn dialogue rewriting model, and the target sentence output by the multi-turn dialogue rewriting model is obtained. In another possible embodiment, as shown in fig. 3, step S13 in fig. 1 may further include step S131 and step S132.

In step S131, the sentence to be rewritten, the history sentence, and the keyword are input to the multi-turn dialogue rewrite model, and the output plurality of rewritten sentences and the probability of each rewritten sentence are acquired.

For example, the sentence to be rewritten, the historical sentence, and the keyword may be spliced to obtain a spliced text, and then the target sentence may be obtained by inputting the spliced text into a preset multi-turn dialogue rewriting model. For example, the concatenated text is input into a preset multi-turn dialogue rewrite model, and a plurality of rewrite sentences output by the multi-turn dialogue rewrite model and the probability of each rewrite sentence are obtained. Wherein, the splicing mode can be as shown in formula (3):

Text＝[CLS],Q ₁ ,Q ₂ ,...,Q _z ,[SEP],c ₁ ,[SEP],c ₂ ,[SEP],...,[SEP],c _m (3)

wherein Q is ₁ ～Q _z-1 Characterizing historical statements, Q _z Characterizing statements to be rewritten, c ₁ ～c _m Characterizing the determined keyword.

In addition, in the present disclosure, the multi-turn dialogue rewrite model may be a generative model, and the multi-turn dialogue rewrite model is obtained by training with the sample sentence to be rewritten, the history sample sentence corresponding to the sample sentence to be rewritten, and the keywords of the sample sentence to be rewritten and the history sample sentence in the first training sample as model input parameters, and with the first sample annotation sentence in the first training sample as model output parameters.

Wherein, a plurality of rounds of dialogue rewrite models can be obtained by training in the following way:

first, a first training sample for training a multi-round dialog-rewrite model is obtained. The first training sample comprises a sample sentence to be rewritten, a historical sample sentence corresponding to the sample sentence to be rewritten, keywords of the sample sentence to be rewritten and the historical sample sentence, and a first sample labeling sentence. Wherein the number of the first training samples is multiple.

Illustratively, the example is still illustrated with the valid conversation sample including a first round sample user statement Q1 "what is a city in beijing city", a second round sample user statement Q2 "what is a city in guangzhou city", and a third round sample user statement Q3 "woollen No. of shanghai". As described above, the keywords are determined as "shanghai" and "market flower" in the above manner. The first sample annotation statement is the true intention of the user, i.e., the first sample annotation statement is "what is a city flower in Shanghai".

And then, taking the sample sentence to be rewritten in the first training sample, the historical sample sentence corresponding to the sample sentence to be rewritten, and the keywords of the sample sentence to be rewritten and the historical sample sentence as model input parameters, and taking the first sample labeling sentence in the first training sample as model output parameters to train the initial model to obtain the multi-round dialogue rewriting model. The initial model may be a BERT model, a GPT (genetic Pre-Training) model, or a T5 (Transfer Text-to-Text Transformer) model. Where the T5 model converts all NLP tasks into Text-to-Text tasks.

In the present disclosure, the initial model is exemplified as a T5 model. To rewrite the sample statementInputting a text obtained by splicing the historical sample sentence corresponding to the rewriting sample sentence and the keywords of the sample sentence to be rewritten and the historical sample sentence according to the formula (3) into a T5 model, and outputting a character x generated at each time step T by the T5 model _t . Denote the first sample annotation statement as y ₁ ,y ₂ ,...,y _T And f, representing the character length of the first sample labeling statement. The output of the T5 model is generated as { x } ₁ ,x ₂ ,...,x _T Y is generated at time step t _t Has a probability of P (y) _t |M _G ,x _<t ) Wherein M is _G Characterization of the T5 model, x _<t Characterization model M _G The result is generated before time step t. Defining the loss function is shown in equation (4):

parameter values of the training parameters may be set in advance when the T5 model is trained. The training parameters may include a maximum character length max _ seq _ length, a training batch size, a learning rate learning _ rate, and a training round number num _ train _ epochs. For example, max _ seq _ length =300, train _batch _size =16, left _rate =10 are set ^-5 Num _ train _ epoch =50, and thus, after training with the first training sample according to the training parameters, the multi-turn dialog rewriting model can be obtained.

Then, the spliced text obtained by splicing the sentence to be rewritten, the history sentence and the keyword is input into the multi-turn dialogue rewrite model, and the probability of each rewritten sentence and the plurality of rewritten sentences generated by the multi-turn dialogue rewrite model can be waited. For example, each rewrite statement generated by the multi-turn dialog rewrite model may be identified as a sequence { x } ₁ ,x ₂ ,...,x _U And the probability of each sequence is

Wherein, U represents the character number of the rewrite sentence, and the probability of each sequence is used for representing that the rewrite sentence corresponding to the sequence is the real character of the userProbability of intent.

In step S132, a target term is specified from among the plurality of rewrite terms based on the probability for each rewrite term.

As described above, the probability of a rewritten sentence is used to represent the probability that the rewritten sentence is the true intention of the user, that is, the higher the probability value is, the higher the probability that the corresponding rewritten sentence is the true intention of the user is, and therefore, in one possible embodiment, the rewritten sentence with the highest probability is determined as the target sentence according to the probability of each rewritten sentence. For example, the rewrite sentence with the highest probability can be quickly determined by decoding through the beam search algorithm, and the rewrite sentence with the highest probability can be determined as the target sentence.

Considering that the accuracy of the multi-turn dialogue rewrite model is limited, the probability of the output rewrite sentences may have errors, so in another possible implementation, according to the probability of each rewrite sentence, a preset number of candidate sentences with higher probability values are determined from the rewrite sentences; for each candidate statement, scoring the candidate statement according to the fluency of the candidate statement and/or the consistency of the candidate statement and the keyword; and determining the target sentence from the candidate sentences according to the respective scores of each candidate sentence.

For example, a plurality of rewritten sentences output by the multi-turn dialogue rewrite model can be decoded by the beam search algorithm to obtain N candidate sentences with larger probability values, and then the N candidate sentences are reordered to determine the target sentence. For example, two factors may be considered in reordering: fluency of the candidate sentences and consistency of the candidate sentences with the keywords. The fluency is mainly measured whether the generated candidate sentences are fluent, and the calculation mode is to calculate the confusion degree of the generated sentences by using an open source language model (such as BERT). It should be understood that fluency is worse with greater confusion, and therefore, for ease of analysis, the inverse or negative of the confusion score is typically employed as the fluency score for the candidate sentence. The consistency is mainly measured whether the candidate sentences are consistent with the keywords or not, and the calculation mode is to count the number of words of non-stop words which are not the keywords in the candidate sentences. Similarly, the more the number of non-stop word words that are not keywords, the worse the consistency, so for ease of analysis, the reciprocal or negative number of the number of non-stop word words that are not keywords in the candidate sentence is usually adopted as the consistency score of the candidate sentence. Then, for each candidate sentence, determining the sum of the fluency score and the consistency score of the candidate sentence as the candidate sentence score. And then, reordering the N candidate sentences according to the candidate sentence scores, wherein the candidate sentences with larger candidate sentence scores are ranked more in the front. Finally, the candidate sentence ranked first is determined as the target sentence.

Therefore, a plurality of rewriting sentences and the probability of each rewriting sentence are output according to the multi-turn conversation rewriting model, after a preset number of candidate sentences with a larger probability value are obtained, the target sentences are determined from the candidate sentences according to fluency and consistency, the rewritten target sentences are more natural and more accord with the real intention of a user, and the effect of multi-turn conversation is improved.

The present disclosure also provides a multi-turn dialog rewriting device. FIG. 4 is a block diagram illustrating a multi-pass dialog rewriting device in accordance with an exemplary embodiment. As shown in fig. 4, the multi-turn dialog rewriting device 400 includes:

a first obtaining module 401 configured to obtain a statement to be rewritten and a history statement of the statement to be rewritten;

an extracting module 402 configured to extract keywords of the to-be-rewritten sentence and the history sentence;

a second obtaining module 403, configured to obtain a target sentence by inputting the sentence to be rewritten, the history sentence, and the keyword into a preset multi-turn dialogue rewriting model;

a replacing module 404 configured to replace the target statement into the statement to be rewritten.

Optionally, the second obtaining module 403 includes:

a first input sub-module, configured to input the statements to be rewritten, the historical statements and the keywords into the multi-turn dialogue rewrite model, and obtain a plurality of output rewritten statements and a probability of each rewritten statement;

a first determining submodule configured to determine a target sentence from the plurality of rewrite sentences according to the probability of each rewrite sentence.

Optionally, the first determining submodule is configured to: determining a preset number of candidate sentences with higher probability values from the plurality of rewritten sentences according to the probability of each rewritten sentence;

Optionally, the first determining sub-module is configured to: and determining the rewrite sentences with the highest probability as target sentences according to the probability of each rewrite sentence.

Optionally, the second obtaining module 403 is configured to: splicing the sentences to be rewritten, the historical sentences and the keywords to obtain spliced texts;

Optionally, the extracting module 402 includes:

the second input sub-module is configured to input the sentences to be rewritten and the historical sentences into a preset sequence labeling model and acquire output keywords;

the sequence labeling model is obtained by taking a sample sentence to be rewritten in a second training sample and a historical sample sentence corresponding to the sample sentence to be rewritten as model input parameters and taking keywords of the sample sentence to be rewritten and the historical sample sentence in the second training sample as model output parameters for training.

acquiring a second sample labeled sentence corresponding to the sample sentence to be rewritten in the second training sample, performing word segmentation on the second sample labeled sentence, and removing stop words to obtain a second word segmentation result;

and determining whether the second segmentation result comprises the segmentation vocabulary or not aiming at each segmentation vocabulary in the first segmentation result, and determining the segmentation vocabulary as the keywords of the sample sentence to be rewritten and the historical sample sentence when the second segmentation result comprises the segmentation vocabulary.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

The present disclosure also provides a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the multi-round dialog-rewrite method provided by the present disclosure.

The present disclosure also provides an electronic device comprising a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

and replacing the target statement into the statement to be rewritten.

Illustratively, FIG. 5 is a block diagram of an electronic device shown in accordance with an exemplary embodiment. For example, the electronic device 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 5, electronic device 500 may include one or more of the following components: processing component 502, memory 504, power component 506, multimedia component 508, audio component 510, input/output interface 512, sensor component 514, and communication component 516.

The processing component 502 generally controls overall operation of the electronic device 500, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. Processing component 502 may include one or more processors 520 to execute instructions to perform all or a portion of the steps of the multi-pass dialog-rewrite method. Further, the processing component 502 can include one or more modules that facilitate interaction between the processing component 502 and other components. For example, the processing component 502 can include a multimedia module to facilitate interaction between the multimedia component 508 and the processing component 502.

The memory 504 is configured to store various types of data to support operations at the electronic device 500. Examples of such data include instructions for any application or method operating on the electronic device 500, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 504 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 506 provides power to the various components of the electronic device 500. The power components 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 500.

The multimedia component 508 includes a screen providing an output interface between the electronic device 500 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 500 is in an operation mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 510 is configured to output and/or input audio signals. For example, the audio component 510 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 500 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in the memory 504 or transmitted via the communication component 516. In some embodiments, audio component 510 further includes a speaker for outputting audio signals.

The input/output interface 512 provides an interface between the processing component 502 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor component 514 includes one or more sensors for providing various aspects of status assessment for the electronic device 500. For example, the sensor assembly 514 may detect an open/closed state of the electronic device 500, the relative positioning of components, such as a display and keypad of the electronic device 500, the sensor assembly 514 may detect a change in the position of the electronic device 500 or a component of the electronic device 500, the presence or absence of user contact with the electronic device 500, orientation or acceleration/deceleration of the electronic device 500, and a change in the temperature of the electronic device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 516 is configured to facilitate wired or wireless communication between the electronic device 500 and other devices. The electronic device 500 may access a wireless network based on a communication standard, such as WiFi,2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast associated information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 500 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components for performing the multi-pass dialog adaptation method.

In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 504 comprising instructions, executable by the processor 520 of the electronic device 500 to perform a multi-round conversation rewrite method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In another exemplary embodiment, a computer program product is also provided, which comprises a computer program executable by a programmable apparatus, the computer program having code portions for performing the multi-pass dialog-rewrite method described above when executed by the programmable apparatus.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A multi-session rewrite method, comprising:

and replacing the target statement into the statement to be rewritten.

2. The method of claim 1, wherein the obtaining the target sentence by inputting the sentence to be rewritten, the history sentence and the keyword into a preset multi-turn dialogue rewrite model comprises:

inputting the statements to be rewritten, the historical statements and the keywords into the multi-turn conversation rewriting model, and acquiring a plurality of output rewritten statements and the probability of each rewritten statement;

3. The method of claim 2, wherein determining the target statement from the plurality of rewrite statements based on the probability of each rewrite statement comprises:

4. The method of claim 2, wherein determining the target statement from the plurality of rewrite statements according to the probability of each rewrite statement comprises:

5. The method of claim 1, wherein the obtaining the target sentence by inputting the sentence to be rewritten, the historical sentence and the keyword into a preset multi-turn dialogue rewriting model comprises:

splicing the sentences to be rewritten, the historical sentences and the keywords to obtain spliced texts;

6. The method according to any one of claims 1 to 5, wherein the multi-round dialogue rewrite model is a generative model, and the multi-round dialogue rewrite model is obtained by training with a sample sentence to be rewritten, a historical sample sentence corresponding to the sample sentence to be rewritten, and keywords of the sample sentence to be rewritten and the historical sample sentence in a first training sample as model input parameters, and with a first sample annotation sentence in the first training sample as model output parameters.

7. The method according to any one of claims 1 to 5, wherein the extracting the keywords of the to-be-rewritten sentence and the history sentence comprises:

8. The method according to claim 7, wherein the keywords of the to-be-overwritten sample sentence and the historical sample sentence in the second training sample are obtained by:

9. A multi-turn dialog rewriting device comprising:

the system comprises a first obtaining module, a second obtaining module and a third obtaining module, wherein the first obtaining module is configured to obtain a statement to be rewritten and a historical statement of the statement to be rewritten;

the extraction module is configured to extract keywords of the to-be-rewritten sentences and the historical sentences;

a second obtaining module configured to obtain a target sentence by inputting the sentence to be rewritten, the history sentence and the keyword into a preset multi-turn dialogue rewriting model;

10. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

and replacing the target statement into the statement to be rewritten.

11. A computer-readable storage medium, on which computer program instructions are stored, which program instructions, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 8.