CN115713065B - Method for generating problem, electronic equipment and computer readable storage medium - Google Patents
Method for generating problem, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN115713065B CN115713065B CN202211393614.1A CN202211393614A CN115713065B CN 115713065 B CN115713065 B CN 115713065B CN 202211393614 A CN202211393614 A CN 202211393614A CN 115713065 B CN115713065 B CN 115713065B
- Authority
- CN
- China
- Prior art keywords
- answer
- text
- generated
- neural network
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a method for generating a problem, electronic equipment and a computer readable storage medium: providing a text of a to-be-generated question and providing an answer to the to-be-generated question; inputting the relevant information of the answer into a first neural network to extract characteristics to obtain relevant characteristics of the answer; inputting the relevant information of the answer into a second neural network to calculate the relevance, and obtaining the relevance characteristic information of the answer and the to-be-generated question; inputting the related information of the text into a third neural network to extract characteristics, and obtaining the related characteristic information of the text and the problem to be generated; inputting the answer related characteristics of the to-be-generated questions, the related characteristic information of the answers and the to-be-generated questions and the related characteristic information of the texts and the to-be-generated questions into a neural network model for generating the questions for calculation, and obtaining words and sentences in the texts as word and sentence probability values in the to-be-generated questions; and selecting a set number of words and sentences in the text based on the word and sentence in the text as the high-low order of the word and sentence probability value in the problem to be generated, so as to form the problem to be generated. The application accurately generates the questions corresponding to the answers.
Description
Technical Field
The present application relates to the field of artificial intelligence, and in particular, to a method for generating a problem, an electronic device, and a computer readable storage medium.
Background
The question-answering system based on the knowledge base can give corresponding answers to questions posed by clients. And setting a corresponding relation between the questions and the answers in a knowledge base of a knowledge base-based question-answering system, matching the questions with the set questions after the question-answering system receives the questions proposed by the clients, and obtaining the answers corresponding to the successfully matched questions from the knowledge base and providing the answers for the clients.
In a knowledge base of a knowledge base-based question-answering system, when the corresponding relation between the questions and the answers is set, how to extract the questions from the text or extract the answers from the text, so that the questions and the answers are corresponding, and the setting of the knowledge base is a question to be solved urgently. Taking the extraction of answers from text as an example, it is possible to extract answers from text in a variety of ways. One way is as follows: based on the set template, the answer is obtained from the text in a manual mode, which requires a great deal of manpower, and has the advantages of low expansibility, low generalization and poor standard consistency. Another way is: and inputting each sentence in the text and the text feature related to the sentence into the neural network model by adopting the trained neural network model to process, obtaining the similarity value of each sentence, and taking the sentence with the highest similarity value in the text as an answer, wherein the text feature related to the sentence is the position feature of the sentence in the text, the lexical feature of the sentence in the text or/and the lexical feature of the sentence in the text. However, the precondition for obtaining the answer in this way is that the default answer is strongly related to the location feature, lexical feature, and lexical feature of the answer appearing in the text, but this is not the case in real scenes, so the obtained answer is not accurate.
The above-described approach focuses on how to obtain answers from text, which are applied in the knowledge base in the question-answering system. Similarly, questions in the knowledge base of the question-answering system may also be obtained in the manner described above. However, when the method is adopted to acquire the questions in the knowledge base of the question-answering system, the characteristic information such as the expression mode and the semantics of the questions in the knowledge base directly influences the matching success rate of matching with the questions presented by the clients, so that the user experience of the clients using the question system based on the knowledge base is influenced.
Disclosure of Invention
In view of this, embodiments of the present application provide a method, an electronic device, and a computer readable storage medium for generating questions, which can accurately generate questions of corresponding answers in a knowledge base-based question-answering system, so as to improve matching rate when matching questions presented by clients subsequently.
In one embodiment of the present application, there is provided a method of generating a problem, the method including:
providing a text of a to-be-generated question and providing an answer to the to-be-generated question;
inputting the relevant information of the answer into a first neural network for feature extraction to obtain relevant features of the answer;
inputting the relevant information of the answer into a second neural network for carrying out relevance calculation to obtain the relevant characteristic information of the answer and the to-be-generated question;
inputting the related information of the text into a third neural network for feature extraction to obtain the related feature information of the text and the problem to be generated;
inputting the answer related characteristics of the to-be-generated questions, the related characteristic information of the answers and the to-be-generated questions and the related characteristic information of the texts and the to-be-generated questions into a neural network model for generating the questions to calculate the probabilities of words and sentences in the texts as the probabilities of the words and sentences in the to-be-generated questions, and obtaining the probabilities of the words and sentences in the texts as the probabilities of the words and sentences in the to-be-generated questions;
and selecting a set number of words and sentences in the text based on the word and sentence in the text as the high-low order of the word and sentence probability value in the problem to be generated, so as to form the problem to be generated.
In the above method, the first neural network is implemented by using a self-attention mechanism, and the relevant features of the answer include: each word feature, keyword feature, location feature, or/and answer semantic feature of the answer.
In the method, the second neural network is realized by adopting a supervision comparison learning neural network.
In the above method, the inputting the relevant information of the answer into a second neural network for performing relevance calculation includes:
the supervision and comparison learning neural network acquires semantic features of the answer and segment features of the answer from the relevant information of the answer, and carries out cosine similarity calculation on the semantic features of the answer and the segment features of the answer, wherein the segment features of the answer comprise a previous sentence of the answer in the text, a current sentence of the answer in the text and a subsequent sentence of the answer in the text;
taking the calculated cosine similarity value as the similarity value between the answer and the to-be-generated problem;
and the similarity value of the answer and the to-be-generated question is the correlation characteristic information of the answer and the to-be-generated question.
In the method, the third neural network is realized by adopting a relational memory neural network R-MeN.
In the above method, inputting the relevant information of the text into a third neural network for feature extraction, and obtaining the relevant feature information of the text and the to-be-generated problem includes:
after the position information and the feature information of the triplet feature in the related information of the text are input into R-MeN for input processing, the R-MeN adopts a set self-attention mechanism network for feature extraction, and the R-MeN adopts a set convolutional neural network CNN to decode and calculate the extracted feature to obtain the triplet validity score value of the related information of the text, wherein the triplet feature of the related information of the text comprises the text feature, the relation feature between the text feature and the answer feature;
and taking the triplet effectiveness score value of the related information of the text as the related characteristic information of the text and the to-be-generated problem.
In the method, the neural network model for generating the problems is realized by adopting a gate control training unit GRU architecture or a long-term and short-term memory artificial neural network LSTM.
The method further comprises the steps of:
and setting the formed questions to be generated in a knowledge base in a question-answering system, so that the knowledge base in the question-answering system sets the corresponding relation between the formed questions to be generated and the answers.
In another embodiment of the present application, there is provided an electronic apparatus including:
a processor;
a memory storing a program configured to implement the method of generating a problem of any of the above when executed by the processor.
In yet another embodiment of the present application, a non-transitory computer-readable storage medium is provided that stores instructions that, when executed by a processor, cause the processor to perform the method of generating a problem as described in any of the above.
As seen above, the embodiment of the present application adopts the following scheme: providing a text of a to-be-generated question and providing an answer to the to-be-generated question; inputting the relevant information of the answer into a first neural network for feature extraction to obtain relevant features of the answer; inputting the relevant information of the answer into a second neural network for carrying out relevance calculation to obtain the relevant characteristic information of the answer and the to-be-generated question; inputting the related information of the text into a third neural network for feature extraction to obtain the related feature information of the text and the problem to be generated; inputting the answer related characteristics of the to-be-generated questions, the related characteristic information of the answers and the to-be-generated questions and the related characteristic information of the texts and the to-be-generated questions into a neural network model for generating the questions to calculate the probabilities of words and sentences in the texts as the probabilities of the words and sentences in the to-be-generated questions, and obtaining the probabilities of the words and sentences in the texts as the probabilities of the words and sentences in the to-be-generated questions; and selecting a set number of words and sentences in the text from the high-low order based on the words and sentences in the text as the probability values of the words and sentences in the to-be-generated problem to form the to-be-generated problem. Thus, when a question is generated, the three types of text related to the generated question and feature information related to the answer can be obtained, and the question can be accurately generated by accurately calculating the feature information according to the feature information, and the generated question can be applied to a knowledge base in a question-answering system. Therefore, the embodiment of the application accurately generates the questions of the corresponding answers in the question-answering system based on the knowledge base, so that the matching rate is improved when the questions presented by the clients are matched later.
Drawings
FIG. 1 is a schematic diagram of a neural network model architecture used for setting correspondence between questions and answers in some embodiments;
FIG. 2 is a flow chart of a method for generating a problem according to an embodiment of the present application;
fig. 3 is a schematic diagram of an architecture of a third neural network according to an embodiment of the present application for processing related information of the text;
FIG. 4 is a schematic diagram of a system architecture for implementing a generation problem provided by an embodiment of the present application;
FIG. 5 is a schematic diagram of an apparatus for generating problems according to an embodiment of the present application;
fig. 6 is a schematic diagram of an electronic device according to another embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented, for example, in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those elements but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The technical scheme of the application is described in detail below by specific examples. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.
Currently, when answers or questions are set in a knowledge base of a knowledge base-based question-answering system, the answers or questions can be realized by using a neural network obtained through training. Taking the answer set as an illustration of extracting the answer from the text. Fig. 1 is a schematic diagram of a neural network model architecture used for setting answers in a knowledge base in a question-answering system according to some embodiments. As shown in the figure, the answer is included in the text, each word in the text, the position feature (Answer Position Feature) of the word in the text, the semantic Features (semantic Features) of the word in the text, and the like (including the Lexical Features of the word in the text and the Lexical Features of the word in the text) are input into the trained neural network model for processing, the similarity value of each sentence is obtained, and the sentence with the highest similarity value in the text is used as the answer and is set in the knowledge base. The neural network model obtained by training is realized by adopting an attention mechanism neural network.
The precondition for setting answers to a knowledge base using the procedure described in fig. 1 is that the default answer is strongly correlated with the location features, lexical features, and lexical features of the answers appearing in the text, but this is not the case in real scenes, so the obtained answer is not accurate. In order to solve the problem, a scheme is proposed that the relation features are extracted and then processed by adopting a neural network model obtained by training. In the scheme, for each sentence in a text, extracting from the segment characteristics of the sentence in the text and the relation characteristics between other sentences in the text and the sentence to obtain the relation characteristics of the sentence, inputting the relation characteristics and the sentence into a neural network model obtained by training, processing, outputting to obtain the similarity value of the sentence and the other sentences in the text, and setting the sentence with the highest similarity value in the text as an answer in a knowledge base. The method can improve the accuracy of the set answer to a certain extent, but because the extraction rule is artificially set, the extracted relationship feature types are limited, and an open-source relationship extraction model OpenIE is adopted when the relationship feature extraction is carried out, so that the extraction error accumulation problem exists, and the finally set answer is inaccurate.
Of course, the questions can be obtained from the text by adopting the mode and are arranged in the knowledge base of the question-answering system. In this case, there is also a problem that the set problem is inaccurate. The knowledge base of the question-answering system is constructed, so that the coverage of the question-answering system is improved, and an important link in the question-answering task of a user side is met. For example, when the question-and-answer system is applied in the field of property sales, the customer's question is basically a question caused by the knowledge of some property politics, such as "can a different house be buying a house locally? "how long an accumulation period of loan will take", etc., these questions are in national policies and documents, and the answers to these questions are also presented in text. In this case, however, it is impossible to determine what expression of the question is adopted by the customer to ask a question, and the existing data exists in the form of a "standard question-text-answer" structure. Therefore, it is necessary to generate questions satisfying the questions of the clients through the existing data in the form of a "text-answer" structure and set the questions in a knowledge base of the question-answering system. Usually, the generation is performed manually, or the process shown in fig. 1 is referred to as the process shown in fig. 1 or the modified process shown in fig. 1, but in this case, the generated problem is inaccurate, the question requirements of the clients cannot be met, and the user experience is poor.
In order to solve the above problems, the embodiment of the present application adopts the following scheme: providing a text of a to-be-generated question and providing an answer to the to-be-generated question; inputting the relevant information of the answer into a first neural network for feature extraction to obtain relevant features of the answer; inputting the relevant information of the answer into a second neural network for carrying out relevance calculation to obtain the relevant characteristic information of the answer and the to-be-generated question; inputting the related information of the text into a third neural network for feature extraction to obtain the related feature information of the text and the problem to be generated; inputting the relevant characteristics of the answer, the relevant characteristic information of the answer and the to-be-generated questions and the relevant characteristic information of the text and the to-be-generated questions into a neural network model for generating the questions to calculate the probabilities of words and sentences in the text as the words and sentences in the to-be-generated questions, and obtaining the probabilities of the words and sentences in the text as the words and sentences in the to-be-generated questions; and selecting a set number of words and sentences in the text based on the word and sentence in the text as the high-low order of the word and sentence probability value in the problem to be generated, so as to form the problem to be generated.
Thus, when a question is generated, the three types of text related to the generated question and feature information related to the answer can be obtained, and the question can be accurately generated by accurately calculating the feature information according to the three types of text related to the generated question and the feature information, and the generated question can be applied to a knowledge base in a question-answering system.
Therefore, the embodiment of the application accurately generates the questions of the corresponding answers in the question-answering system based on the knowledge base, so that the accuracy is improved when the questions presented by the clients are matched later.
The embodiment of the application combines the constructed multiple neural networks with the neural network model for generating the questions, and processes the information comprising the text and the answers to generate the questions set by the knowledge base of the question-answering system. The third neural network (R-MeN) is adopted for processing based on the key context (to the point context) information in the text, so that the correlation characteristic information of the text and the problem to be generated can be obtained, and the accuracy of the subsequent generation problem is improved; based on the answer information and the text information, a second neural network, namely a supervised comparison learning neural network is adopted to calculate similarity (to similarity), so that the relevance characteristic information of the answer and the to-be-generated question can be described, and the accuracy of the subsequent generated question is improved based on the answer information; the three types of characteristic information are integrated by adopting a neural network model with a gating circulation conception for generating problems and calculating probability values, so that the three-dimensional characteristic information is effectively utilized when the problems are generated, the accuracy of the generated problems is improved, and the quality of the generated problems is better.
Fig. 2 is a flowchart of a method for generating a problem according to an embodiment of the present application, where specific steps include:
step 201, providing a text of a to-be-generated question and providing an answer to the to-be-generated question;
step 202, inputting relevant information of the answer into a first neural network for feature extraction to obtain relevant features of the answer;
step 203, inputting the relevant information of the answer into a second neural network for carrying out relevance calculation to obtain the relevant characteristic information of the answer and the to-be-generated question;
step 204, inputting the related information of the text into a third neural network for feature extraction to obtain the related feature information of the text and the to-be-generated problem;
step 205, inputting the relevant features of the answer, the relevant feature information of the answer and the to-be-generated question, and the relevant feature information of the text and the to-be-generated question into a neural network model for generating the question, and performing probability calculation on each word and sentence in the text as a word and sentence in the to-be-generated question to obtain a word and sentence probability value of each word and sentence in the text as the word and sentence in the to-be-generated question;
and 206, selecting a set number of words and sentences in the text based on the high-low order of the word and sentence probability values in the text as the words and sentences in the to-be-generated problem to form the to-be-generated problem.
In the above method, the set number is set as needed, which is not limited here.
In the above process, when the set number is 1, extracting the word and sentence with the highest probability value from the text to form the problem to be generated.
In the embodiment of the application, the relevant characteristic information of the answer is a characteristic obtained after encoding by adopting a first neural network, the relevant characteristic information of the answer and the to-be-generated question is a characteristic obtained after encoding by adopting a second neural network, and the relevant characteristic information of the text and the to-be-generated question is a characteristic obtained after encoding by adopting a third neural network. In the three neural networks, self-attention mechanisms (self-attention) are respectively adopted, so that corresponding characteristic information can be conveniently extracted respectively.
In the embodiment of the present application, the first neural network, the second neural network, the third neural network, and the neural network model for generating the problem are all obtained by training using a training data source. Here, the training data source includes text, answers included in the text, and questions included in the text, wherein the questions included in the text are used for subsequent verification of whether the generated questions are used accurately.
In the embodiment of the application, the related information of the text is obtained by preprocessing the text of the provided problem to be generated, and the preprocessing comprises two steps: the first step, cleaning stop words and punctuation marks in a text; in the second step, since the text is long, usually about 380 characters, the text is cut, and the sentence with the set number of characters is cut back and forth with the sentence including the answer as the center, so as to ensure that the answer is in the cut sentence and has the context information.
Similarly, in the embodiment of the application, the answer can be preprocessed to obtain the relevant information of the answer, and the stop words and punctuation marks in the answer can be cleaned when the answer is preprocessed.
As a specific example. The text of the question to be generated includes: the merchant pays money in two to three months at present, and particularly whether each bank policy is tightened or not is also checked. The loan of the pure accumulation fund is relatively fast to be released for about two months. The combined returns are relatively slow, possibly three to four months. The answer to the question to be generated is: the pure accumulation fund loan is paid out for about two months. The problems to be generated are: is the duration of a pure metric loan deposit?
In this case, the text needs to be preprocessed to obtain relevant information of the text, and the answer needs to be preprocessed to obtain relevant information of the answer.
In the embodiment of the application, the first neural network is implemented by adopting a self-attention mechanism, and the relevant characteristics of the answer comprise: each word feature, keyword feature, location feature, or/and answer semantic feature of the answer. Here, the first neural network encodes the relevant information of the answer, and obtains the basic characteristics of the answer, so as to process the basic characteristics as the input of the neural network model for generating the questions.
In the embodiment of the application, the second neural network is implemented by adopting a sentence-embedded supervised contrast learning (simcse, simple Contrastive Learning of Sentence Embeddings) network. The second neural network may encode the relevant information of the answer by using an existing simcse encoding mode, so that in order to make the encoding more applicable, a fine tuning (fine) technique is used to fine tune the encoding mode of the existing simcse network and then use the same.
Specifically, inputting the relevant information of the answer into the second neural network to perform relevance calculation comprises:
the simcse network obtains the semantic features of the answer and the segment features of the answer from the relevant information of the answer, and carries out cosine similarity calculation on the semantic features of the answer and the segment features of the answer;
taking the calculated cosine similarity value as the similarity value between the answer and the to-be-generated problem;
and the similarity value of the answer and the to-be-generated question is the correlation characteristic information of the answer and the to-be-generated question.
The semantic features of the answer and the segment features of the answer are extracted by the simcse network through a self-attention mechanism, and the segment features of the answer comprise relevant features of the to-be-generated questions. Specifically, the segment features of the answer include a previous sentence of the answer in the text, a current sentence of the answer in the text, and a subsequent sentence of the answer in the text, and the segment information of the answer is extracted through the attention mechanism of the simcse network.
Here, the cosine similarity calculation between the semantic features of the answer and the segment features of the answer adopts the following formula:
wherein A is the fragment feature of the answer, namely the relevant feature of the question to be generated, Q is the semantic feature of the answer, and the two features are coded and mapped on the simcse network.
In the embodiment of the application, the third neural network is realized by adopting a relational memory neural network (R-MeN). Specifically, inputting the relevant information of the text into a third neural network for feature extraction, and obtaining the relevant feature information of the text and the to-be-generated problem includes: after the position information and the feature information of the triplet feature (s.r.o) in the related information of the text are input into R-MeN for input processing, the R-MeN adopts a set self-attention mechanism network for feature extraction, the R-MeN adopts a set Convolutional Neural Network (CNN) for decoding calculation of the extracted feature, and the triplet effectiveness score value of the related information of the text is obtained through calculation, wherein the triplet feature of the related information of the text comprises the text feature, the relation feature between the text feature and the answer feature; and taking the triplet effectiveness score value of the text as the correlation characteristic information of the text and the to-be-generated problem. The self-attention mechanism network is a multi-layer feedforward neural network.
Here, fig. 3 is a schematic diagram of an architecture of a third neural network according to an embodiment of the present application to process related information of the text. As shown in the figure, three gray circles represent feature information (enabling) of a triplet feature in the related information of the text, three white circles represent position information (positional encoding) of the triplet feature in the related information of the text, the position information and the feature information of the triplet feature (s, R, o) in the related information of the text are taken as input, and input into R-MeN for input processing by adopting the following formula:
x 1 ＝W(v s +p 1 )+b (2)
x 2 ＝W(v r +p 2 )+b
x 3 ＝W(v o +p 3 )+b
V s ，V r ，V o feature vector representation, p, referring to triplet features (s, r, o) i Refers to a position vector. The x obtained i Refers to the input vector, x of the R-MeX network 1 、x 2 X is a group 3 Representing the computed triplet characteristics (s, r, o) as input parameters to the self-attention mechanism network, respectively.
Wherein W represents the weight, b represents the set bias factor, v s ，v r ，v o Feature vectors respectively representing triplet features (s, r, o), p represents position information of each word and sentence in the related information of the text, subscript s is a text feature, subscript r is a relationship information feature between the feature and an answer feature, and subscript o is an answer feature. Finally obtain x 1 、x 2 X is a group 3 As an input parameter to the self-attention mechanism network in R-MeN.
As shown in FIG. 3, x will be obtained 1 、x 2 X is a group 3 I.e. text features, textsAfter the relation features between the features and the answer features are subjected to multi-layer feedforward (MLF) processing of a self-attention mechanism, feature extraction results shown by three stripe circles in FIG. 3 are obtained and sent to CNN for decoding processing, and the triplet effectiveness score value of the text is obtained. In CNN, the triplet validity score value for the text is obtained using the following formula:
f(s，r，o)＝max(ReLU([y 1 ，y 2 ，y 3 ]*Ω)) T w (3)
wherein y is 1 、y 2 Y 3 Is X in formula (2) 1 、X 2 X is X 3 After the attention mechanism is processed in the third neural network, the obtained vector is expressed and is based on y 1 、y 2 Y 3 And calculating a triplet effectiveness score value of the text.
In the embodiment of the application, the neural network model for generating the problem is realized by adopting a gate control training unit (GRU) architecture or a long-short-term memory artificial neural network (LSTM), w of the formula represents a weight vector, Ω represents a set of filters, and belongs to R m*3 And represents a convolution operation. The three types of feature information obtained above are effectively combined and are effectively utilized through a neural network model (RELU function implementation of formula (3)) for generating the problem, so that the neural network model for generating the problem carries out word and sentence probability calculation in the text as the word and sentence in the problem to be generated, the word and sentence in the text is obtained as the word and sentence probability value in the problem to be generated, and the word and sentence with the highest probability value is extracted from the text later to form the problem to be generated.
Here, the neural network model for generating the problem is implemented by using the following formula, which gives the probability of the nth word and sentence in the text as the nth word and sentence of the problem based on the three types of feature information obtained above, and calculates by using the following formula (4):
wherein p is v (omega) represents the probability of the relevant feature of the answer generating the question omega, p s (ω) represents a probability of the answer matching the correlation characteristic information of the question to be generated, p M (ω) represents the probability of the text to generate a question ω with the relevance feature information of the question to be generated,respectively representing the result vectors calculated by the gating device in the network structure (figure 4). The embodiment of the application aims to maximize the probability of the generated words and sentences being connected into one sentence.
Fig. 4 is a schematic diagram of a system architecture of the method shown in fig. 2, and fig. 4 is a schematic diagram of a system architecture for implementing a problem generation implementation provided by the present application. The whole process is as shown in fig. 4:
the first step, inputting the relevant information of the answer into a first neural network with a self-attention mechanism for feature extraction, and obtaining the relevant features of the answer comprises the following steps: each word feature, keyword feature, location feature, or/and answer semantic feature of the answer, the relevant features of the answer are shown in fig. 4;
the second step, inputting the relevant information of the answer into a second neural network with a self-attention mechanism for carrying out relevance calculation to obtain the relevance characteristic information of the answer and the to-be-generated question; the relevance features of answers to questions are shown in FIG. 4;
inputting the related information of the text into a third neural network for feature extraction to obtain the related feature information of the text and the to-be-generated problem; the text-to-question relevance feature is shown in FIG. 4;
and a fourth step of inputting the relevant characteristics of the answer, the relevant characteristic information of the answer and the to-be-generated questions and the relevant characteristic information of the text and the to-be-generated questions into a neural network model for generating the questions, wherein the model adopts a gating mode to calculate the probability of each word and sentence in the text as the word and sentence in the to-be-generated questions, and each word and sentence in the text is obtained as the word and sentence probability value in the to-be-generated questions.
In a fourth step, in particular, the neural network model for generating the question is implemented using the above formula (4), in fig. 4, after the probability of generating the question ω from the relevant feature of the answer (calculated in fig. 4 using self-intent of the relevant feature of the access answer, and shown as a gray line box, is a representation of the probability of the question ω), we proceed(a representation of the control calculation in a gating manner in fig. 4); after the probability of a question ω indicating that the answer matches the correlation characteristic information of the question to be generated (calculated in fig. 4 using self-attribute of the correlation characteristic of the access answer and the question, and shown as a gray line box is a representation of the probability of the question ω), proceeding->Calculation (representation of control calculation in gating mode in fig. 4), after the probability of generating a problem ω from the text and the correlation feature information of the problem to be generated (calculation of self-attribute of the correlation feature of the text and the problem in fig. 4, and representation of probability of the problem ω as shown gray line box) is performed> Calculation (representation of control calculation in a gating manner in fig. 4); after the calculated result vectors are added (shown in fig. 4 by columns with different gray scales), each term in the text is obtained as a term probability value in the problem to be generated.
The embodiment of the application can generate accurate problems by adopting the process, and improves the quality of the generated problems. The embodiment of the application can apply the generated questions to the knowledge base of the question-answering system, thereby being convenient for the automatic matching use of the parts after the clients put forth the questions later. Specifically, the method further comprises: and setting the formed questions to be generated in a knowledge base in a question-answering system, so that the knowledge base in the question-answering system sets the corresponding relation between the formed questions to be generated and the answers.
After the problems are generated and arranged in the knowledge base of the question-answering system, the generated problems can be evaluated. In the specific evaluation, two modes can be adopted, and the following is a specific description.
The first way is: automatic digest evaluating mode (ROUGE-L)
L in ROUGE-L refers to the longest common subsequence (longest common subsequence, LCS), and the calculation formula of the ROUGE-L is as follows, wherein the longest common subsequence of the machine translation C and the reference translation S is used in the calculation of the ROUGE-L:
wherein R in the formula LCS Representing the recall of the generated question, P LCS Representing the accuracy of the problem, F LCS Representing the evaluation result obtained by ROUGE-L, and C and S in the calculation represent the text generated by the machine and the reference text respectively. Beta is set to infinity in the formula, and thus F LCS In fact R LCS . After ROUGE-L evaluation, F is obtained LCS Reaching 0.73 percent.
The second way is: manual evaluation mode
In order to ensure that the online effect can be achieved, a manual flat measurement method is added after the ROUGE-L is adopted for evaluation. And randomly selecting 500 pieces of data from the generated problems, marking, evaluating the marked problems, and taking the accuracy of the problems matched with the clients as an evaluation index. After manual evaluation, the matching accuracy of the marked problems reaches 66.2%.
Therefore, as can be seen from the results obtained by evaluation, the problem is generated by adopting the embodiment of the application, the accuracy and quality of the generated problem are improved, and the efficiency of generating the problem is also improved.
Fig. 5 is a schematic structural diagram of a device for generating a problem according to an embodiment of the present application, including: an acquisition unit, a processing unit and a generation problem unit, wherein,
the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for providing a text of a to-be-generated problem and providing an answer of the to-be-generated problem;
the processing unit is used for inputting the relevant information of the answer into the first neural network for feature extraction to obtain the relevant features of the answer; inputting the relevant information of the answer into a second neural network for carrying out relevance calculation to obtain the relevant characteristic information of the answer and the to-be-generated question; inputting the related information of the text into a third neural network for feature extraction to obtain the related feature information of the text and the problem to be generated;
the question generation unit is used for inputting the answer related characteristics of the questions to be generated, the relevant characteristic information of the answers and the questions to be generated and the relevant characteristic information of the texts and the questions to be generated into a neural network model of the questions to be generated, and carrying out word and sentence probability calculation in the texts as the word and sentence probability in the questions to be generated to obtain word and sentence probability values in the texts as the word and sentence probability values in the questions to be generated; and selecting a set number of words and sentences in the text based on the word and sentence in the text as the high-low order of the word and sentence probability value in the problem to be generated, so as to form the problem to be generated.
In another embodiment of the present application, there is also provided an electronic apparatus, including an electronic apparatus including: a processor; a memory storing a program configured to implement a method of generating a problem as described above when executed by the processor.
In another embodiment of the application, a non-transitory computer readable storage medium is provided that stores instructions that, when executed by a processor, cause the processor to perform the matching text method of the previous embodiment. Fig. 6 is a schematic diagram of an electronic device according to another embodiment of the present application. As shown in fig. 6, another embodiment of the present application further provides an electronic device, which may include a processor 601, wherein the processor 601 is configured to perform the steps of the method for generating a problem as described above. As can also be seen from fig. 6, the electronic device provided by the above embodiment further comprises a non-transitory computer readable storage medium 602, on which non-transitory computer readable storage medium 702 a computer program is stored which, when executed by the processor 601, performs the steps of a method of generating a problem as described above.
In particular, the non-transitory computer readable storage medium 602 can be a general purpose storage medium such as a removable disk, hard disk, FLASH, read Only Memory (ROM), erasable programmable read only memory (EPROM or FLASH memory), or portable compact disc read only memory (CD-ROM), etc., and the computer program on the non-transitory computer readable storage medium 602, when executed by the processor 601, can cause the processor 601 to perform the steps of a method of generating a problem as described above.
In practice, the non-transitory computer readable storage medium 602 may be included in the apparatus/device/system described in the above embodiment, or may exist alone, and not be assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs that, when executed, are capable of performing the steps of a method of generating a problem as described above.
Yet another embodiment of the present application also provides a computer program product comprising a computer program or instructions which, when executed by a processor, performs the steps of a method of generating a problem as described above.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be combined in various combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the present application. In particular, the features recited in the various embodiments of the application and/or in the claims may be combined in various combinations and/or combinations without departing from the spirit and teachings of the application, all of which are within the scope of the disclosure.
The principles and embodiments of the present application have been described herein with reference to specific examples, which are intended to be included herein for purposes of illustration only and not to be limiting of the application. It will be apparent to those skilled in the art that variations can be made in the present embodiments and applications within the spirit and principles of the application, and any modifications, equivalents, improvements, etc. are intended to be included within the scope of the present application.
Claims (6)
1. A method of generating a question, the method comprising:
providing a text of a to-be-generated question and providing an answer to the to-be-generated question;
inputting the relevant information of the answer into a first neural network for feature extraction to obtain relevant features of the answer;
inputting the relevant information of the answer into a second neural network for relevance calculation to obtain the relevant characteristic information of the answer and the question to be generated, wherein the second neural network is realized by adopting a supervision and comparison learning neural network;
inputting the related information of the text into a third neural network for feature extraction to obtain the related feature information of the text and the problem to be generated, wherein the third neural network is realized by adopting a relational memory neural network R-MeN;
inputting the answer related characteristics of the to-be-generated questions, the related characteristic information of the answers and the to-be-generated questions and the related characteristic information of the texts and the to-be-generated questions into a neural network model for generating the questions to calculate the probabilities of words and sentences in the texts as the probabilities of the words and sentences in the to-be-generated questions, and obtaining the probabilities of the words and sentences in the texts as the probabilities of the words and sentences in the to-be-generated questions;
selecting a set number of words and sentences in the text based on the word and sentence in the text as the high-low order of the word and sentence probability value in the problem to be generated, and forming the problem to be generated;
the step of inputting the relevant information of the answer into a second neural network for relevance calculation comprises the following steps:
the supervision and comparison learning neural network acquires semantic features of the answer and segment features of the answer from the relevant information of the answer, and carries out cosine similarity calculation on the semantic features of the answer and the segment features of the answer, wherein the segment features of the answer comprise a previous sentence of the answer in the text, a current sentence of the answer in the text and a subsequent sentence of the answer in the text;
taking the calculated cosine similarity value as the similarity value between the answer and the to-be-generated problem;
the similarity value of the answer and the to-be-generated question is the correlation characteristic information of the answer and the to-be-generated question;
inputting the related information of the text into a third neural network for feature extraction, and obtaining the related feature information of the text and the to-be-generated problem comprises the following steps:
inputting the position information and the feature information of the triplet feature in the related information of the text into R-MeN, performing input processing, performing feature extraction by the R-MeN through a set self-attention mechanism network, and performing decoding calculation on the extracted feature by the R-MeN through a set convolutional neural network CNN to obtain a triplet effectiveness score value of the related information of the text, wherein the triplet feature of the related information of the text comprises the text feature, a relation feature between the text feature and an answer feature and the answer feature;
and taking the triplet effectiveness score value of the related information of the text as the related characteristic information of the text and the to-be-generated problem.
2. The method of claim 1, wherein the first neural network is implemented using a self-attention mechanism, and the relevant features of the answer include: each word feature, keyword feature, location feature, or/and answer semantic feature of the answer.
3. The method of claim 1, wherein the neural network model that generates the problem is implemented using a gated training cell, GRU, architecture or a long and short term memory artificial neural network, LSTM.
4. The method of claim 1, wherein the method further comprises:
and setting the formed questions to be generated in a knowledge base in a question-answering system, so that the knowledge base in the question-answering system sets the corresponding relation between the formed questions to be generated and the answers.
5. An electronic device, comprising:
a processor;
a memory storing a program configured to implement the method of generating a problem of any one of claims 1 to 4 when executed by the processor.
6. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of generating a problem of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211393614.1A CN115713065B (en) | 2022-11-08 | 2022-11-08 | Method for generating problem, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211393614.1A CN115713065B (en) | 2022-11-08 | 2022-11-08 | Method for generating problem, electronic equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115713065A CN115713065A (en) | 2023-02-24 |
| CN115713065B true CN115713065B (en) | 2023-09-15 |
Family
ID=85232602
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211393614.1A Active CN115713065B (en) | 2022-11-08 | 2022-11-08 | Method for generating problem, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115713065B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109766423A (en) * | 2018-12-29 | 2019-05-17 | 上海智臻智能网络科技股份有限公司 | Answering method and device neural network based, storage medium, terminal |
| CN110188176A (en) * | 2019-04-30 | 2019-08-30 | 深圳大学 | Deep learning neural network and training, prediction technique, system, equipment, medium |
| CN111324721A (en) * | 2020-03-16 | 2020-06-23 | 云南电网有限责任公司信息中心 | Method for constructing intelligent question-answering knowledge base |
| CN112214593A (en) * | 2020-11-05 | 2021-01-12 | 腾讯科技（深圳）有限公司 | Question and answer processing method and device, electronic equipment and storage medium |
| WO2021164284A1 (en) * | 2020-02-19 | 2021-08-26 | 平安科技（深圳）有限公司 | Method, apparatus and device for generating reading comprehension question, and storage medium |
| WO2021184311A1 (en) * | 2020-03-19 | 2021-09-23 | 中山大学 | Method and apparatus for automatically generating inference questions and answers |
| WO2021217935A1 (en) * | 2020-04-29 | 2021-11-04 | 深圳壹账通智能科技有限公司 | Method for training question generation model, question generation method, and related device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6772213B2 (en) * | 2018-04-23 | 2020-10-21 | 日本電信電話株式会社 | Question answering device, question answering method and program |
| CN111858883A (en) * | 2020-06-24 | 2020-10-30 | 北京百度网讯科技有限公司 | Method and device for generating triple sample, electronic equipment and storage medium |
-
2022
- 2022-11-08 CN CN202211393614.1A patent/CN115713065B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109766423A (en) * | 2018-12-29 | 2019-05-17 | 上海智臻智能网络科技股份有限公司 | Answering method and device neural network based, storage medium, terminal |
| CN110188176A (en) * | 2019-04-30 | 2019-08-30 | 深圳大学 | Deep learning neural network and training, prediction technique, system, equipment, medium |
| WO2021164284A1 (en) * | 2020-02-19 | 2021-08-26 | 平安科技（深圳）有限公司 | Method, apparatus and device for generating reading comprehension question, and storage medium |
| CN111324721A (en) * | 2020-03-16 | 2020-06-23 | 云南电网有限责任公司信息中心 | Method for constructing intelligent question-answering knowledge base |
| WO2021184311A1 (en) * | 2020-03-19 | 2021-09-23 | 中山大学 | Method and apparatus for automatically generating inference questions and answers |
| WO2021217935A1 (en) * | 2020-04-29 | 2021-11-04 | 深圳壹账通智能科技有限公司 | Method for training question generation model, question generation method, and related device |
| CN112214593A (en) * | 2020-11-05 | 2021-01-12 | 腾讯科技（深圳）有限公司 | Question and answer processing method and device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 基于双注意力的段落级问题生成研究;曾碧卿，裴枫华，徐马一，丁美荣;中文信息学报;第36卷(第8期);全文 * |
| 基基于产品建模的评论问题生成研究;肖康，周夏冰，王中卿，段湘煜，周国栋，张民;计算机科学;49(2);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115713065A (en) | 2023-02-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109783655B (en) | Cross-modal retrieval method and device, computer equipment and storage medium | |
| CN110147551B (en) | Multi-category entity recognition model training, entity recognition method, server and terminal | |
| CN109376222B (en) | Question-answer matching degree calculation method, question-answer automatic matching method and device | |
| CN109992664B (en) | Dispute focus label classification method and device, computer equipment and storage medium | |
| CN109697228A (en) | Intelligent answer method, apparatus, computer equipment and storage medium | |
| CN111191002B (en) | Neural code searching method and device based on hierarchical embedding | |
| CN110895553A (en) | Semantic matching model training method, semantic matching method and answer obtaining method | |
| CN112085091B (en) | Short text matching method, device, equipment and storage medium based on artificial intelligence | |
| CN111143507B (en) | Reading and understanding method based on compound problem | |
| CN117236410B (en) | Trusted electronic file large language model training and reasoning method and device | |
| CN111368096A (en) | Knowledge graph-based information analysis method, device, equipment and storage medium | |
| CN116662488A (en) | Service document retrieval method, device, equipment and storage medium | |
| CN111695335A (en) | Intelligent interviewing method and device and terminal equipment | |
| CN117076688A (en) | Knowledge question-answering method and device based on domain knowledge graph and electronic equipment | |
| CN117520503A (en) | Financial customer service dialogue generation method, device, equipment and medium based on LLM model | |
| CN111145914A (en) | Method and device for determining lung cancer clinical disease library text entity | |
| EP4030355A1 (en) | Neural reasoning path retrieval for multi-hop text comprehension | |
| CN117609444A (en) | Searching question-answering method based on large model | |
| CN113705207A (en) | Grammar error recognition method and device | |
| CN115794898B (en) | Financial information recommendation method and device, electronic equipment and storage medium | |
| CN115713065B (en) | Method for generating problem, electronic equipment and computer readable storage medium | |
| CN114757205A (en) | Text evaluation method, device, equipment and storage medium | |
| CN111400413B (en) | Method and system for determining category of knowledge points in knowledge base | |
| CN113673237A (en) | Model training method, intent recognition method, device, electronic equipment and storage medium | |
| Gabajiwala et al. | Comprehensive review of various optimization algorithms for image captioning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |