CN111428448A - Text generation method and device, computer equipment and readable storage medium - Google Patents
Text generation method and device, computer equipment and readable storage medium Download PDFInfo
- Publication number
- CN111428448A CN111428448A CN202010136551.6A CN202010136551A CN111428448A CN 111428448 A CN111428448 A CN 111428448A CN 202010136551 A CN202010136551 A CN 202010136551A CN 111428448 A CN111428448 A CN 111428448A
- Authority
- CN
- China
- Prior art keywords
- sample
- text
- data
- target
- generator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000012549 training Methods 0.000 claims abstract description 30
- 230000015654 memory Effects 0.000 claims description 30
- 230000006870 function Effects 0.000 claims description 25
- 238000004088 simulation Methods 0.000 claims description 15
- 238000000342 Monte Carlo simulation Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 8
- 230000003042 antagnostic effect Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 8
- 230000008569 process Effects 0.000 description 16
- 230000003068 static effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000010485 coping Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000007787 long-term memory Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 230000006403 short-term memory Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a text generation method and device, computer equipment and a readable storage medium, and belongs to the field of text processing. The text generation method, the text generation device, the computer equipment and the readable storage medium generate the target text data according to the target guide data through the text generation confrontation network model obtained through pre-training, the problem that discrete output is not updatable is solved, and the purpose that text sentences can be generated according to sentence start data by adopting the text generation confrontation network model is achieved.
Description
Technical Field
The present invention relates to the field of text processing, and in particular, to a text generation method, apparatus, computer device, and readable storage medium.
Background
In an intelligent interview scene, Artificial Intelligence (AI for short) needs to ask a candidate for questions according to preset questions and also needs to ask the candidate for open questions according to actual situations so as to test the actual coping ability of the candidate. The open question requires the AI to generate a quiz text using a generative model.
The existing generation model mainly adopts a generation countermeasure network (GAN), and the generation countermeasure network needs to update parameter variables based on continuous output data, so the generation model is mainly applied to image processing, various image generation tasks comprise unsupervised generation, labeled generation, super-resolution reduction, automatic coloring, street view generation and the like, and the quality of generated images is vivid until human eyes cannot distinguish true from false.
When the generated countermeasure network is applied to a text generation task, because the generation of the countermeasure network needs to output the probability distribution of the next word in a vocabulary table based on the generated text sequence in the text generation process, and then selects the word, the output result is discrete data, and the discrete data cannot realize the training and updating of the network. Therefore, the current generation countermeasure network cannot be applied to the text generation task.
Disclosure of Invention
Aiming at the problem that the existing generation countermeasure network only supports continuous output, a text generation method, a device, computer equipment and a readable storage medium for generating the countermeasure network based on a text which can be updated according to discrete data are provided.
In order to achieve the above object, the present invention provides a text generating method, comprising the steps of:
acquiring answer data generated by a business object in a question-answer scene;
extracting the answer data and acquiring target guide data;
generating a confrontation network model through a text obtained by pre-training and generating target text data according to the target guide data;
the target guide data is sentence head data of the target text data.
In one embodiment, before the step of generating a confrontation network model from the pre-trained text and generating target text data according to the target guidance data, the method includes:
acquiring a sample guide set and a sample text set, wherein the sample guide set comprises at least one sample guide datum, the sample text set comprises at least one sample text datum, and the sample guide datum is a sentence head datum of the sample text datum;
training an initial confrontation network model according to the sample guide set and the sample text set, and obtaining a text to generate a confrontation network model.
In one embodiment, the initial confrontation network model includes a generator and a discriminator, and the training of the initial confrontation network model according to the sample guidance set and the sample text set and the obtaining of the text generation confrontation network model includes:
generating, by the generator and in accordance with at least one sample guidance data in the sample guidance set, at least one sample text data;
simulating the at least one sample text data by adopting Monte Carlo simulation and acquiring a plurality of sample simulated text data;
identifying the plurality of sample simulation text data through the discriminator according to target text data in the sample text set, and updating parameter values of the generator according to an identification result;
updating the discriminator according to a loss function based on the updated generator;
and circularly updating the generator and the discriminator until the initial confrontation network model meets a preset convergence condition, and obtaining the text generation confrontation network model formed by the updated generator.
In one embodiment, the step of generating, by the generator and from sample guidance data of at least one of the sample guidance sets, at least one sample text data comprises:
calculating according to the sample guide data through the generator to obtain a first sample word with the maximum probability in a vocabulary, and adding the first sample word to the tail of the sample guide data;
and calculating according to the first sample word by the generator to obtain a second sample word with the maximum probability in the vocabulary, adding the second sample word to the tail of the first sample word, and circularly executing the steps until sample text data with preset length is obtained.
In one embodiment, the step of simulating the at least one sample text data using monte carlo simulation and obtaining a plurality of sample simulated text data comprises:
and simulating words in each sample text data one by adopting Monte Carlo simulation, and generating a plurality of sample simulated text data corresponding to the sample text data.
In one embodiment, the step of identifying, by the discriminator, the plurality of sample simulated text data according to the target text data in the sample text set, and updating the parameter value of the generator according to the identification result includes:
identifying the plurality of sample simulation text data through the discriminator according to target text data in the sample text set, and acquiring a state cost function according to an identification result;
and calculating an objective function according to the state cost function, and updating the parameter value of the generator according to the objective function.
In one embodiment, the step of generating a confrontation network model from the pre-trained text and generating target text data according to the target guidance data includes:
calculating the target guide data by adopting a generator for generating an antagonistic network model by the text to obtain a first sample word with the maximum probability in a vocabulary, and adding the first sample word to the tail of the target guide data;
and calculating the first sample word by adopting the generator, acquiring a second sample word with the maximum probability in a vocabulary, adding the second sample word to the tail of the first sample word, and circularly executing the steps until target text data with preset length is acquired.
In order to achieve the above object, the present invention further provides a text generating apparatus, including:
the acquisition unit is used for acquiring answer data generated by the business object in the question-answer scene;
an acquisition unit, configured to extract the answer data and acquire target guidance data;
the generation unit is used for generating a confrontation network model through a text obtained by pre-training and generating target text data according to the target guide data;
the target guide data is sentence head data of the target text data.
To achieve the above object, the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the steps of the above method when executing the computer program.
To achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the above method.
The beneficial effects of the above technical scheme are that:
in the technical scheme, the text generation method, the text generation device, the computer equipment and the readable storage medium generate the target text data according to the target guide data (such as sentence head data) by the text generation confrontation network model obtained through pre-training, so that the problem that discrete output is not updatable is solved, and the aim that the text sentence (such as a text problem) can be generated according to the sentence head data by adopting the text generation confrontation network model is fulfilled.
Drawings
FIG. 1 is a flowchart of a method of an embodiment of a text generation method according to the present invention;
FIG. 2 is a flow diagram of a method of one embodiment of obtaining a text-to-generate confrontation network model;
FIG. 3 is a flowchart of a method of one embodiment of training an initial confrontation network model to obtain a text-generated confrontation network model based on a sample guide set and a sample text set;
FIG. 4 is a block diagram of an embodiment of a text generation apparatus according to the present invention;
fig. 5 is a schematic diagram of a hardware architecture of an embodiment of a computer device according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The text generation method, the text generation device, the computer equipment and the readable storage medium are suitable for the business fields of insurance, finance and the like, and provide a text generation method which can automatically generate open text problems and is convenient for testing the thinking ability of candidates for loan systems, insurance systems and financial systems. The invention generates the target text data according to the target guide data (such as sentence head data) by the text generation confrontation network model obtained by pre-training, solves the problem that discrete output is not updatable, and achieves the aim that the text generation confrontation network model can generate text sentences (such as text problems) according to the sentence head data.
Example one
Referring to fig. 1, a text generating method of the present embodiment includes the following steps:
s1, collecting answer data generated by a business object in a question and answer scene;
in this step, the business object may be a consultation user for consulting business, or a buyer of the online transaction platform, or an interviewer in the interviewing process. The answer data may be collected by a collection device (e.g., an audio receiving device, a microphone or a mobile terminal with a recording function, etc.).
The text generation method in this embodiment is mainly applied to a dialog scene (at least two users), and generates a question text based on the answer information of the target object, so that the target object can answer the question text, for example: when the text generation method is applied to an interview scene, an open text problem is generated according to keywords provided by interviewers.
S2, extracting the answer data and acquiring target guide data;
in step S2, semantic analysis may be performed on the answer data to extract a keyword in the answer data, and the keyword is used as target guidance data; analyzing the answer data to extract a noun in the answer data, and taking the noun as target guide data.
It should be noted that: the target guidance data may be a keyword, or a beginning word of a sentence.
S3, generating a confrontation network model through a text obtained through pre-training and generating target text data according to the target guide data;
it should be noted that: the target guide data is sentence head data of the target text data. For example: the target boot data is: "today"; the target text data is: "how is the weather today? ". The target guide data may be two words or three words as the beginning word of the sentence, and is not limited herein.
Referring to fig. 2, before performing step S3, the step of obtaining the text generation confrontation network model may include:
s31, a sample guide set and a sample text set are obtained, wherein the sample guide set comprises at least one sample guide datum, the sample text set comprises at least one sample text datum, and the sample guide datum is a sentence head datum of the sample text datum;
in the present embodiment, the sample pilot set is a sequence of sample pilot data (beginning data); the sample text set is a sequence of real text data composed of sample text data (complete sentence). The sample leading data is the beginning data of the real text data.
And S32, training an initial confrontation network model according to the sample guide set and the sample text set, and obtaining a text to generate the confrontation network model.
At present, in the process of image processing of the generated countermeasure network, the pixel values of each point of the generated image are continuous values, so that the calculation graph of the whole network, from the weight of the generator to the output thereof, and then to the weight and the output classification of the discriminator, are differentiable (differentiable and conductive), errors can be normally propagated backwards, and the gradient and the weight can be normally updated. However, in the process of text generation, the generator actually outputs a sequence, each round outputs the probability distribution of the next word in the vocabulary based on the generated text sequence, and then selects the word with the highest probability, the "selection" process is not trivial, the generator outputs discrete tokens, and in the training process, errors are propagated to the discrete tokens in the reverse direction, and the gradient update of pixel values cannot be performed on each token like an image generation task, so that the weight values of the generator are updated. On the other hand, the discriminator may directly receive an input of a complete text sequence, and output true or false sentences, but cannot judge half of the sentences that have not been completed by the generator, so that the discriminator cannot provide supervision for training of the generator for each word in the generated text sequence.
Therefore, in the training process for generating the countermeasure network model in this embodiment, in order to solve the infinitesimal problem caused by the discrete output of the generator, in this embodiment, the generation process of the text sequence is regarded as a sequence decision process, a policy gradient (policy gradient) method in reinforcement learning is adopted, the judgment result of the discriminator is regarded as a reward (reward), a part of the text generated by the generator is regarded as a state (state), the generator is regarded as an agent (agent), a next word is predicted as an action (action), and the generator is a policy (policy) that needs to be updated, so that the infinitesimal problem of the loss function of the discrete output is solved. For the method for judging the incomplete sequence, a Monte Carlo search (Monte Carlo search) is adopted in the embodiment, the generator continues to generate the incomplete sequence based on the generated sequence until the sequence is completed, the discriminator judges the sequence for a plurality of times, and the mean value of final rewarded is used as the rewarded estimation of the current incomplete sequence.
It should be noted that: the initial confrontation network model comprises a generator and an arbiter; referring to fig. 3, in step S32, the step of training the initial confrontation network model according to the sample guidance set and the sample text set, and obtaining a text to generate the confrontation network model includes:
by way of example and not limitation, the generator may employ a long and short term memory network (L STM) of output sequences for generating text sequences from a given initial state, and the discriminator may employ a two-class long and short term memory network for receiving the output text of the generator and the actual text to determine whether the output text is true or false.
S321, generating at least one sample text data through the generator according to at least one sample guide data in the sample guide set;
further, the step in step S321 may include:
calculating according to the sample guide data through the generator to obtain a first sample word with the maximum probability in a vocabulary, and adding the first sample word to the tail of the sample guide data;
and calculating according to the first sample word by the generator to obtain a second sample word with the maximum probability in the vocabulary, adding the second sample word to the tail of the first sample word, and circularly executing the steps (and so on) until sample text data with a preset length is obtained.
In this step, the generator G is initializedθAnd a discriminator Dφ(ii) a Sample guide data is a set of real texts S ═ X1～TThe sentence length of each real text in the real text set is T, and the tail of each real text with the length less than T is filled with zeros; sample guided set into word set Y1}。
Set words { Y }1Is input to the generator GθGenerator GθThe input layer(s) maps the input word(s) to tag information (tokenization) corresponding to the corresponding word(s) in the vocabulary table, performs embedded representation, and performs embedding representation on the corresponding word(s) in the vocabulary tableWill be (y) in the actual application1,y2,…,yt-1) Sent as input to generator GθGenerator GθAccording to the input data, the softmax classifier outputs the probability of each word of the next word in the vocabulary table, and the word with the maximum probability is taken as ytRepeating the above steps until the end y of the sentenceTThus, a set of generated sample text sets { Y) with length T (length is not filled with zero) is obtained1～T}。
Wherein (y)1,y2,…,yt-1) Representing an incomplete sentence consisting of t-1 words, y1Represents the 1 st word in a sentence; y is2Represents the 2 nd word in a sentence; y ist-1Represents the t-1 word in a sentence; y isTThe T-th word in a sentence (end of the sentence).
In step, only the generator G is usedθTo transmit a word y1Generator GθEmbedding the text sequence into L STM, outputting the generated token sequence and the corresponding words in the vocabulary, and obtaining the generated text sequence (y)1,y2,…,yT)。
S322, simulating the at least one sample text data by adopting Monte Carlo simulation and acquiring a plurality of sample simulated text data;
further, the step of step S322 includes:
and simulating words in each sample text data one by adopting Monte Carlo simulation, and generating a plurality of sample simulated text data corresponding to the sample text data.
In this implementation, for the sample text set { Y }1～TEach sequence in (y)1,y2,…,yT) Sequence is an example, traversing each word y in the sequencetN Monte Carlo simulations were performed, differing from the previous selection of the most probable word as ytHere, generator G is used each timeθSampling according to the multinomial distribution of output words, and repeating until reaching the end y of the sentenceTSo as to obtain N different complete sample simulation text sets { Y1～T 1,Y1～T 2,…,Y1～T N}。
It should be noted that the simulation times of words located at different positions in a sentence in the sample text set may be the same or different.
S323, identifying the plurality of sample simulation text data through the discriminator according to target text data in the sample text set, and updating parameter values of the generator according to an identification result;
further, step S323 may include:
identifying the plurality of sample simulation text data through the discriminator according to target text data in the sample text set, and acquiring a state cost function according to an identification result;
and calculating an objective function according to the state cost function, and updating the parameter value of the generator according to the objective function.
In an embodiment, the sample obtained is modeled as a set of texts { Y }1～T 1,Y1～T 2,…,Y1～T NIs inputted to a discriminator DφPerforming secondary classification, comparing each sample simulation text with the corresponding real text, and if the sample simulation texts are consistent, indicating that the sample simulation text generated by the generator is real (mark 1); if not, the generator generated sample simulation text is shown to be false (label 0). For complete sentences, directly convert discriminator DφOutputting the result as a status value; for incomplete sentences, the discrimination results of N complete sentences obtained by Monte Carlo simulation are averaged. In summary, the state cost function can be expressed as:
where i represents the number of simulations of the monte carlo simulation.
Updating the generator G according to the state cost functionθThe objective function of the generator is to produce as realistic a spoof discriminator as possible, i.e. to maximize its on-strategyGθThe following rewards are earned:
wherein G isθ(yt|Y1～t-1) Representing the policy output, essentially viewed as a probability, output ytProbability values in the vocabulary; y is1～t-1Is all ytThe values that appear. The parameter theta is the generator GθThe weight parameter of (1); generator GθIs updated on J (θ), in other words, the gradient of the strategy comes from J (θ):
wherein, αθIs the learning rate.
S324, updating the discriminator according to a loss function based on the updated generator;
in this step, the updated generator G is usedθGenerating a set of text sequences { Y }1～TSimultaneously from the set of authentic texts S ═ X1～TSelect the same number of text sequence sets { X }1～TIs inputted to a discriminator DφIn the middle classification, the loss function is a two-classification logarithmic loss function:
Dφupdate on J (phi):
wherein, αφIs the learning rate.
And S325, circularly updating the generator and the discriminator until the initial confrontation network model meets a preset convergence condition, and obtaining the text generation confrontation network model formed by the updated generator.
In this step, the training generator n is repeated for each round of trainingGDiscriminant n for secondary and repeated trainingDAnd thirdly, until the model meets the preset convergence condition. Such as: the preset convergence condition is nD>nGTo ensure that the arbiter can correctly guide the generator update.
In step S3, the step of generating a confrontation network model from the pre-trained text and generating target text data from the target guidance data includes:
calculating the target guide data by adopting a generator for generating an antagonistic network model by the text to obtain a first sample word with the maximum probability in a vocabulary, and adding the first sample word to the tail of the target guide data;
and calculating the first sample word by adopting the generator, acquiring a second sample word with the maximum probability in a vocabulary, adding the second sample word to the tail of the first sample word, and circularly executing the steps (and so on) until target text data with a preset length is acquired. Therefore, the target text data for questioning is generated according to the answer data, the purpose of open questioning and answering based on the response of the business object is achieved, and the temporary coping capability of the business object to the open questions is conveniently tested.
In the embodiment, the text generation method is based on a countermeasure long-short term memory network and a strategy gradient, a structure of a discriminator-generator based on L STM is used, the tasks of generating a text sequence and judging the authenticity of the text can be accurately realized, parameters of the discriminator can be dynamically updated by means of countermeasure training, the recognition capability is continuously improved, appropriate guidance is provided for the generator, the quality of the generated text is more potential than that of the generated text based on other static benchmark evaluations, the sequence generation process is converted into a sequence decision process by means of a reinforced learning idea, the problem that a loss function cannot be minimized due to discrete output is solved, the training of the countermeasure network is possible, Monte Carlo search is used, a complete sequence of each step and a scoring result of the complete sequence in the discriminator are obtained by means of strategy simulation, the mean value is used as a rewarded value of the current time step, the problem that an unfinished sequence cannot be directly obtained is solved, in addition, only a generator part needs to be reserved in a training stage, and compared with other discretization inexplicated skills such as Gumbel-software, the model occupation is smaller.
Example two
As shown in fig. 4, the present invention also provides a text generating apparatus 1, including: acquisition unit 11, acquisition unit 12 and generation unit 13, wherein:
the acquisition unit 11 is used for acquiring answer data generated by the business object in the question-answer scene;
the business object can be a consultation user for business consultation, or a buyer of an online transaction platform, or an interviewer in an interviewing process. The answer data may be collected by a collection device (e.g., an audio receiving device, a microphone or a mobile terminal with a recording function, etc.).
The text generation device 1 in the present embodiment is mainly applied to a dialog scenario (at least two users), and generates a question text based on the answer information of a target object, so that the target object can answer the question text, for example: when the text generation device 1 is applied to an interview scene, an open text question is generated according to keywords provided by interviewers.
An obtaining unit 12, configured to extract the answer data and obtain target guidance data;
the adoption unit 12 can carry out semantic analysis on the answer data to extract keywords in the answer data, and the keywords are used as target guide data; analyzing the answer data to extract a noun in the answer data, and taking the noun as target guide data.
The generation unit 13 is used for generating a confrontation network model through a text obtained by pre-training and generating target text data according to the target guide data;
the target guide data is sentence head data of the target text data.
Specifically, the generating unit 13 calculates according to the target guidance data by using the generator for generating the confrontation network model from the text, obtains a first sample word with the highest probability in a vocabulary, and adds the first sample word to the end of the target guidance data;
and the generator calculates according to the first sample word, obtains a second sample word with the maximum probability in a vocabulary, adds the second sample word to the tail of the first sample word, and so on until target text data with a preset length is obtained.
In the embodiment, the text generation device 1 can accurately realize the tasks of generating a text sequence and judging the authenticity of the text by using a structure of a discriminator-generator based on L STM based on a countermeasure long-short term memory network and a strategy gradient, can dynamically update parameters of the discriminator by means of countermeasure training, continuously improve the recognition capability, provide proper guidance for the generator, have potential compared with the quality of the generated text based on other static benchmark evaluations, convert the sequence generation process into a sequence decision process by means of the idea of reinforcement learning, solve the problem that a loss function cannot be minimized due to discrete output, enable the training of the countermeasure network to be generated, use Monte Carlo search to obtain a complete sequence of each step and a grading result of the sequence in the discriminator by means of strategy simulation, use the mean value as a rewarded value of the current time step, solve the problem that an unfinished sequence cannot be directly obtained, and only need to reserve a generator part in the training stage and compared with the problems that other discrete processing such as Gumbel-software and the like, do not need extra parameters and the model occupies smaller training.
EXAMPLE III
In order to achieve the above object, the present invention further provides a computer device 2, where the computer device 2 includes a plurality of computer devices 2, components of the text generating apparatus 1 according to the second embodiment may be distributed in different computer devices 2, and the computer device 2 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server, or a rack server (including an independent server or a server cluster formed by a plurality of servers) that executes a program, or the like. The computer device 2 of the present embodiment includes at least, but is not limited to: a memory 21, a processor 23, a network interface 22, and the text generation apparatus 1 (refer to fig. 5) that can be communicatively connected to each other through a system bus. It is noted that fig. 5 only shows the computer device 2 with components, but it is to be understood that not all of the shown components are required to be implemented, and that more or less components may be implemented instead.
In this embodiment, the memory 21 includes at least one type of computer-readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the storage 21 may be an internal storage unit of the computer device 2, such as a hard disk or a memory of the computer device 2. In other embodiments, the memory 21 may also be an external storage device of the computer device 2, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like provided on the computer device 2. Of course, the memory 21 may also comprise both an internal storage unit of the computer device 2 and an external storage device thereof. In this embodiment, the memory 21 is generally used for storing an operating system installed in the computer device 2 and various types of application software, such as a program code of the text generation method in the first embodiment. Further, the memory 21 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 23 may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data Processing chip in some embodiments. The processor 23 is typically used for controlling the overall operation of the computer device 2, such as performing control and processing related to data interaction or communication with the computer device 2. In this embodiment, the processor 23 is configured to run the program code stored in the memory 21 or process data, for example, run the text generating apparatus 1.
The network interface 22 may comprise a wireless network interface or a wired network interface, and the network interface 22 is typically used to establish a communication connection between the computer device 2 and other computer devices 2. For example, the network interface 22 is used to connect the computer device 2 to an external terminal through a network, establish a data transmission channel and a communication connection between the computer device 2 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth (Bluetooth), Wi-Fi, and the like.
It is noted that fig. 5 only shows the computer device 2 with components 21-23, but it is to be understood that not all shown components are required to be implemented, and that more or less components may be implemented instead.
In this embodiment, the text generating apparatus 1 stored in the memory 21 may be further divided into one or more program modules, and the one or more program modules are stored in the memory 21 and executed by one or more processors (in this embodiment, the processor 23) to complete the present invention.
Example four:
to achieve the above objects, the present invention also provides a computer-readable storage medium including a plurality of storage media such as a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application store, etc., on which a computer program is stored, which when executed by the processor 23, implements corresponding functions. The computer-readable storage medium of the present embodiment is used for storing the text generation apparatus 1, and when being executed by the processor 23, the computer-readable storage medium implements the text generation method of the first embodiment.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A text generation method is characterized by comprising the following steps based on a question-answer scene:
acquiring answer data generated by a business object in a question-answer scene;
extracting the answer data and acquiring target guide data;
generating a confrontation network model through a text obtained by pre-training and generating target text data according to the target guide data;
the target guide data is sentence head data of the target text data.
2. The method of claim 1, wherein before the step of generating a confrontation network model from the pre-trained text and generating target text data from the target guidance data, the method comprises:
acquiring a sample guide set and a sample text set, wherein the sample guide set comprises at least one sample guide datum, the sample text set comprises at least one sample text datum, and the sample guide datum is a sentence head datum of the sample text datum;
training an initial confrontation network model according to the sample guide set and the sample text set, and obtaining a text to generate a confrontation network model.
3. The text generation method of claim 2, wherein the initial confrontation network model comprises a generator and a discriminator, and the step of training the initial confrontation network model according to the sample guide set and the sample text set and obtaining the text generation confrontation network model comprises:
generating, by the generator and in accordance with at least one sample guidance data in the sample guidance set, at least one sample text data;
simulating the at least one sample text data by adopting Monte Carlo simulation and acquiring a plurality of sample simulated text data;
identifying the plurality of sample simulation text data through the discriminator according to target text data in the sample text set, and updating parameter values of the generator according to an identification result;
updating the discriminator according to a loss function based on the updated generator;
and circularly updating the generator and the discriminator until the initial confrontation network model meets a preset convergence condition, and obtaining the text generation confrontation network model formed by the updated generator.
4. The text generation method of claim 3, wherein the step of generating, by the generator and from at least one sample guide data in the sample guide set, at least one sample text data comprises:
calculating according to the sample guide data through the generator to obtain a first sample word with the maximum probability in a vocabulary, and adding the first sample word to the tail of the sample guide data;
calculating according to the first sample word through the generator to obtain a second sample word with the maximum probability in a vocabulary, and adding the second sample word to the tail of the first sample word;
and circularly executing the steps until sample text data with preset length is obtained.
5. The text generation method of claim 3, wherein the step of simulating the at least one sample text data using a monte carlo simulation and obtaining a plurality of sample simulated text data comprises:
and simulating words in each sample text data one by adopting Monte Carlo simulation, and generating a plurality of sample simulated text data corresponding to the sample text data.
6. The method of claim 3, wherein the step of identifying, by the discriminator, the plurality of sample simulated text data based on the target text data in the sample text set, and updating the parameter values of the generator based on the identification result comprises:
identifying the plurality of sample simulation text data through the discriminator according to target text data in the sample text set, and acquiring a state cost function according to an identification result;
and calculating an objective function according to the state cost function, and updating the parameter value of the generator according to the objective function.
7. The method of claim 3, wherein the step of generating a confrontation network model from the pre-trained text and generating target text data from the target guidance data comprises:
calculating the target guide data by adopting a generator for generating an antagonistic network model by the text to obtain a first sample word with the maximum probability in a vocabulary, and adding the first sample word to the tail of the target guide data;
calculating the first sample word by adopting the generator, acquiring a second sample word with the maximum probability in a vocabulary, and adding the second sample word to the tail of the first sample word;
and circularly executing the steps until target text data with preset length is obtained.
8. A text generating apparatus, based on a question-and-answer scenario, comprising:
the acquisition unit is used for acquiring answer data generated by the business object in the question-answer scene;
an acquisition unit, configured to extract the answer data and acquire target guidance data;
the generation unit is used for generating a confrontation network model through a text obtained by pre-training and generating target text data according to the target guide data;
the target guide data is sentence head data of the target text data.
9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that: the processor, when executing the computer program, realizes the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program when executed by a processor implements the steps of the method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010136551.6A CN111428448B (en) | 2020-03-02 | 2020-03-02 | Text generation method, device, computer equipment and readable storage medium |
| PCT/CN2020/118456 WO2021174827A1 (en) | 2020-03-02 | 2020-09-28 | Text generation method and appartus, computer device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010136551.6A CN111428448B (en) | 2020-03-02 | 2020-03-02 | Text generation method, device, computer equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111428448A true CN111428448A (en) | 2020-07-17 |
| CN111428448B CN111428448B (en) | 2024-05-07 |
Family
ID=71553527
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010136551.6A Active CN111428448B (en) | 2020-03-02 | 2020-03-02 | Text generation method, device, computer equipment and readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111428448B (en) |
| WO (1) | WO2021174827A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112036544A (en) * | 2020-07-31 | 2020-12-04 | 五八有限公司 | Image generation method and device |
| CN112861179A (en) * | 2021-02-22 | 2021-05-28 | 中山大学 | Method for desensitizing personal digital spatial data based on text-generated countermeasure network |
| WO2021174827A1 (en) * | 2020-03-02 | 2021-09-10 | 平安科技（深圳）有限公司 | Text generation method and appartus, computer device and readable storage medium |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115481630A (en) * | 2022-09-27 | 2022-12-16 | 深圳先进技术研究院 | Electronic insurance letter automatic generation method and device based on sequence countermeasure and prior reasoning |
| CN116010609B (en) * | 2023-03-23 | 2023-06-09 | 山东中翰软件有限公司 | Material data classifying method and device, electronic equipment and storage medium |
| CN117933268A (en) * | 2024-03-21 | 2024-04-26 | 山东大学 | End-to-end unsupervised resistance text rewriting method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106663125A (en) * | 2014-08-21 | 2017-05-10 | 国立研究开发法人情报通信研究机构 | Question sentence generation device and computer program |
| CN109522411A (en) * | 2018-11-12 | 2019-03-26 | 南京德磐信息科技有限公司 | A kind of writing householder method neural network based |
| CN110162595A (en) * | 2019-03-29 | 2019-08-23 | 深圳市腾讯计算机***有限公司 | For generating the method, apparatus, equipment and readable storage medium storing program for executing of text snippet |
| US20190384810A1 (en) * | 2018-06-15 | 2019-12-19 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method of training a descriptive text generating model, and method and apparatus for generating descriptive text |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105654945B (en) * | 2015-10-29 | 2020-03-06 | 乐融致新电子科技(天津)有限公司 | Language model training method, device and equipment |
| CN110019732B (en) * | 2017-12-27 | 2021-10-15 | 华为技术有限公司 | Intelligent question answering method and related device |
| CN110619118B (en) * | 2019-03-28 | 2022-10-28 | 中国人民解放军战略支援部队信息工程大学 | Automatic text generation method |
| CN110196899B (en) * | 2019-06-11 | 2020-07-21 | 中央民族大学 | Low-resource language question-answer corpus generating method |
| CN111428448B (en) * | 2020-03-02 | 2024-05-07 | 平安科技（深圳）有限公司 | Text generation method, device, computer equipment and readable storage medium |
-
2020
- 2020-03-02 CN CN202010136551.6A patent/CN111428448B/en active Active
- 2020-09-28 WO PCT/CN2020/118456 patent/WO2021174827A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106663125A (en) * | 2014-08-21 | 2017-05-10 | 国立研究开发法人情报通信研究机构 | Question sentence generation device and computer program |
| US20190384810A1 (en) * | 2018-06-15 | 2019-12-19 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method of training a descriptive text generating model, and method and apparatus for generating descriptive text |
| CN109522411A (en) * | 2018-11-12 | 2019-03-26 | 南京德磐信息科技有限公司 | A kind of writing householder method neural network based |
| CN110162595A (en) * | 2019-03-29 | 2019-08-23 | 深圳市腾讯计算机***有限公司 | For generating the method, apparatus, equipment and readable storage medium storing program for executing of text snippet |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021174827A1 (en) * | 2020-03-02 | 2021-09-10 | 平安科技（深圳）有限公司 | Text generation method and appartus, computer device and readable storage medium |
| CN112036544A (en) * | 2020-07-31 | 2020-12-04 | 五八有限公司 | Image generation method and device |
| CN112861179A (en) * | 2021-02-22 | 2021-05-28 | 中山大学 | Method for desensitizing personal digital spatial data based on text-generated countermeasure network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111428448B (en) | 2024-05-07 |
| WO2021174827A1 (en) | 2021-09-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111428448B (en) | Text generation method, device, computer equipment and readable storage medium | |
| CN111078836B (en) | Machine reading understanding method, system and device based on external knowledge enhancement | |
| CN111767366B (en) | Question and answer resource mining method and device, computer equipment and storage medium | |
| CN111444340A (en) | Text classification and recommendation method, device, equipment and storage medium | |
| CN111222305A (en) | Information structuring method and device | |
| CN110717023B (en) | Method and device for classifying interview answer text, electronic equipment and storage medium | |
| CN111581966A (en) | Context feature fusion aspect level emotion classification method and device | |
| KR102265573B1 (en) | Method and system for reconstructing mathematics learning curriculum based on artificial intelligence | |
| CN106649739B (en) | Multi-round interactive information inheritance identification method and device and interactive system | |
| CN112487139A (en) | Text-based automatic question setting method and device and computer equipment | |
| CN116824278B (en) | Image content analysis method, device, equipment and medium | |
| CN110929640B (en) | Wide remote sensing description generation method based on target detection | |
| CN112069329B (en) | Text corpus processing method, device, equipment and storage medium | |
| CN114048729A (en) | Medical document evaluation method, electronic device, storage medium, and program product | |
| CN110929532B (en) | Data processing method, device, equipment and storage medium | |
| CN110852071A (en) | Knowledge point detection method, device, equipment and readable storage medium | |
| CN113569018A (en) | Question and answer pair mining method and device | |
| CN115617974B (en) | Dialogue processing method, device, equipment and storage medium | |
| CN113342932B (en) | Target word vector determining method and device, storage medium and electronic device | |
| CN115687910A (en) | Data processing method and device, computer equipment and readable storage medium | |
| CN113610080A (en) | Cross-modal perception-based sensitive image identification method, device, equipment and medium | |
| CN113569091A (en) | Video data processing method and device | |
| CN117611845B (en) | Multi-mode data association identification method, device, equipment and storage medium | |
| CN113761837B (en) | Entity relationship type determining method, device and equipment and storage medium | |
| CN114723050B (en) | Method and device for determining prompt vector of pre-training model and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |