CN113283247A - Text generation method, device, equipment and computer readable medium - Google Patents

Text generation method, device, equipment and computer readable medium Download PDF

Info

Publication number
CN113283247A
CN113283247A CN202110679716.9A CN202110679716A CN113283247A CN 113283247 A CN113283247 A CN 113283247A CN 202110679716 A CN202110679716 A CN 202110679716A CN 113283247 A CN113283247 A CN 113283247A
Authority
CN
China
Prior art keywords
original text
text
words
vocabulary
pointer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110679716.9A
Other languages
Chinese (zh)
Inventor
张美伟
李昱
王全礼
张晨
杨占栋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
Original Assignee
China Construction Bank Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp filed Critical China Construction Bank Corp
Priority to CN202110679716.9A priority Critical patent/CN113283247A/en
Publication of CN113283247A publication Critical patent/CN113283247A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/42—Data-driven translation
    • G06F40/44—Statistical methods, e.g. probability models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a text generation method, a text generation device, text generation equipment and a computer readable medium, and relates to the technical field of artificial intelligence. One embodiment of the method comprises: generating a network by inputting an original text pointer; in the pointer generation network, updating the distribution probability of the decoded words in the original text according to the weight of the words in the original text; establishing a decoding word list based on the distribution probability of the decoded words so as to generate an evolved text of the original text in the pointer generation network according to the decoding word list; and the pointer generation network outputs the evolved text of the original text. The embodiment can improve accuracy and reduce repeatability for text generated by words beyond a vocabulary.

Description

Text generation method, device, equipment and computer readable medium
Technical Field
The present invention relates to the field of artificial intelligence technologies, and in particular, to a method, an apparatus, a device, and a computer-readable medium for text generation.
Background
Text generation technology has advanced significantly in recent years. Early techniques focused on extraction methods that prioritized the importance score of each sentence from the original text. The scoring method relies primarily on frequency and stochastic topic models. The extraction method has the advantages that the original information can be more completely retained, and particularly, the consistency of each sentence is ensured.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art: Out-Of-Vocabulary words (OOV) result in poor accuracy and high repeatability Of the generated text.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, an apparatus, a device, and a computer readable medium for generating a text, which can improve accuracy and reduce repeatability for a text generated by a word exceeding a vocabulary.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a text generation method including:
generating a network by inputting an original text pointer;
in the pointer generation network, updating the distribution probability of the decoded words in the original text according to the weight of the words in the original text;
establishing a decoding word list based on the distribution probability of the decoded words so as to generate an evolved text of the original text in the pointer generation network according to the decoding word list;
and the pointer generation network outputs the evolved text of the original text.
The generating the original text input pointer into a network comprises:
and inputting the screened original text into the pointer to generate a network.
In the pointer generation network, updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text, including:
in the pointer generation network, according to the weight of the vocabulary in the original text, the pointer coefficient in the original text is adjusted to update the distribution probability of the decoded words in the original text.
In the pointer generation network, updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text, including:
in the pointer generation network, determining the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text;
and updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text.
In the pointer generation network, determining the weight of each vocabulary in the original text based on the occurrence number of each vocabulary in the original text comprises the following steps:
and in the pointer generation network, determining the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient.
In the pointer generation network, determining the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient, wherein the determining comprises the following steps:
adjusting a sampling coefficient within a preset range, and determining the preset sampling coefficient;
and in the pointer generation network, determining the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient.
The preset range includes an interval greater than or equal to 0 and less than or equal to 1.
In the preset range, adjusting a sampling coefficient and determining the preset sampling coefficient includes:
and within a preset range, adjusting the sampling coefficients according to a descending order, and determining the preset sampling coefficients.
In the pointer generation network, determining the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient, wherein the determining comprises the following steps:
in the pointer generation network, smoothing the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient, and then determining the weight of the vocabulary in the original text.
The updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text comprises:
and according to the weight of the vocabulary in the original text, reducing the distribution probability of the decoded common words in the original text and improving the distribution probability of the decoded common words in the original text.
The establishing a decoding word list based on the distribution probability of the decoded words so as to generate the evolved text of the original text in the pointer generation network according to the decoding word list comprises the following steps:
establishing a decoding word list based on the distribution probability of the decoded words;
and in the pointer generation network, generating an evolved text of the original text according to the vocabulary in the decoding vocabulary.
The decoding word list comprises the non-common words in the original text and the common words in the original text, and the distribution probability of the non-common words is different from that of the common words.
The pointer generation network outputs the evolved text of the original text, including:
and outputting the evolved text of the original text by the pointer generation network in a preset document.
The evolving text includes a summary text.
The evolving text includes keyword text.
According to a second aspect of the embodiments of the present invention, there is provided an apparatus for text generation, including:
the input module is used for generating a network by inputting an original text pointer;
the updating module is used for updating the distribution probability of the decoded words in the original text according to the weight of the words in the original text in the pointer generation network;
the generating module is used for establishing a decoding word list based on the distribution probability of the decoded words so as to generate an evolved text of the original text in the pointer generation network according to the decoding word list;
and the output module is used for outputting the evolution text of the original text by the pointer generation network.
According to a third aspect of the embodiments of the present invention, there is provided an electronic device for text generation, including:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method as described above.
According to a fourth aspect of embodiments of the present invention, there is provided a computer readable medium, on which a computer program is stored, which when executed by a processor, implements the method as described above.
One embodiment of the above invention has the following advantages or benefits: generating a network by inputting an original text pointer; in the pointer generation network, updating the distribution probability of the decoded words in the original text according to the weight of the words in the original text; establishing a decoding word list based on the distribution probability of the decoded words so as to generate an evolved text of the original text in the pointer generation network according to the decoding word list; and the pointer generation network outputs the evolved text of the original text. The distribution probability of the words is updated according to the texts generated by the words exceeding the word list, so that the accuracy can be improved and the repeatability can be reduced.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of a main flow of a method of text generation according to an embodiment of the invention;
FIG. 2 is a schematic flow chart of updating the distribution probability of a decoded word in an original text according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating a process for determining weights of words in original text according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating the relationship between the number of occurrences of words and the distribution probability according to an embodiment of the present invention;
FIG. 5 is a schematic flow diagram of the generation of an evolved text of an original text according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a main structure of an apparatus for text generation according to an embodiment of the present invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 8 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In recent years, the direct generation of text using neural networks has proven to be an effective method. With the rapid development of deep learning technology over the years, methods for generating texts are also continuously developed. Artificial neural network based techniques have been successfully applied in the field of natural language processing including, but not limited to, syntactic analysis, text summarization, and dialogue systems. The codec structure and attention mechanism for generating text are representative methods for generating text, and good effects are obtained.
Text generation has a wide application in Natural Language Processing (NLP), and the mainstream technology is a network structure for Seq2Seq and pointer generation, which completes a general text generation task. Later, the Transformer model was developed as a successor to the improved Seq2 Seq.
However, no clear measure is proposed for the sample word frequency in the training samples, and no practical consideration is given at the decoding stage of the algorithm, regardless of the Transformer model or the algorithm combining the Seq2Seq and the pointer generation network.
In order to solve the technical problems of poor accuracy and high repeatability of the generated text, the following technical scheme in the embodiment of the invention can be adopted.
Referring to fig. 1, fig. 1 is a schematic diagram of a main flow of a text generation method according to an embodiment of the present invention, and a decoding word list is established by updating a distribution probability of words, so as to generate an evolved text of an original text. As shown in fig. 1, the method specifically comprises the following steps:
and S101, generating a network by inputting the original text into a pointer.
In the embodiment of the present invention, the text to be processed is referred to as original text. That is, the original text is text that includes a plurality of characters and requires pointer generation for network processing.
In order to improve the efficiency of the pointer generation network text processing, the original text can be screened first, and then the screened original text is input into the pointer generation network. This is because the non-standard characters originally in the original text increase the processing time for processing the text. And removing the non-standard characters from the screened original text, thereby improving the text processing efficiency.
The Pointer-Generator Networks (Pointer-generators Networks) is a generation model based on seq2seq + attribute, and can solve the OOV problem to a certain extent compared with the generation model of seq2seq + attribute.
S102, in the pointer generation network, updating the distribution probability of the decoded words in the original text according to the weight of the words in the original text.
In the existing pointer generation network, words in a certain time sequence are finally decoded, not only from weights captured by adopting an Attention mechanism, but also weights of each word in the original text, so that some key words which cannot be captured in an Attention stage in the original text are captured.
It is noted that the vocabulary added to the original text presupposes that all the vocabulary has the same weight. However, in practice, it is not difficult to find that the most commonly used words provide less information than the rare ones. Therefore, it is not reasonable to consider all the words as the same weight when building the decoding table.
In the embodiment of the invention, the decoding word list is established by updating the distribution probability of the words, and the weight of each word in the decoding word list is different.
The updating of the distribution probability of the decoded words in the original text is specifically described below with reference to the accompanying drawings. The vocabulary in the original text needs to be decoded, each vocabulary has a weight, and then the distribution probability of the decoded words in the original text is updated according to the weight of the vocabulary in the original text.
In one embodiment of the invention, considering that the pointer coefficient in the original text is related to the weight of the vocabulary in the original text in the pointer generation network, the distribution probability of the decoded words in the original text can be updated based on the pointer coefficient in the original text.
In one embodiment of the invention, the pointer generation network includes a transform model. The structure of the Transformer model is also composed of an encoder and a decoder. The Transformer model is characterized in that new words can be generated according to a fixed vocabulary, and the new words can be processed according to an original text or some keywords which cannot be generated in a model algorithm can be processed.
In particular implementations, p (w) is the distribution probability of the word being decoded, since a pointer generation network is introduced, p (w) is modified to:
P(w)＝Pgen*Pvocab(w)+(1-Pgen) Σ α ti equation 1
Where Σ α ti is the attribution weight captured by the transformations model. PgenIs a pointer coefficient, and the probability distribution of the final word to be decoded is mainly controlled by the pointer coefficient, one part of the probability distribution is from the attention result, and the other part is from the original text input Pvocab(w)。
Updating the distribution probability of the decoded words in the original text, which relates to the weights of the words in the original text, the following describes the updating process in detail with reference to the drawings.
Referring to fig. 2, fig. 2 is a schematic flowchart of updating a distribution probability of a decoded word in an original text according to an embodiment of the present invention, which specifically includes the following steps:
s201, in the pointer generation network, determining the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text.
The original text includes a plurality of words, which may occur more than once. For each vocabulary, there is a corresponding weight. Weight of each vocabulary in original textjThe sum equals 1, and there are j different words in the original text. Namely:
1＝∑weightjequation 2
The weight of each vocabulary in the original text can be calculated according to formula 3.
Figure BDA0003122409660000081
Wherein, P (word)i) Is the sampling rate of the vocabulary. P (word)i) Is related to the occurrence number of the vocabulary and the preset sampling coefficient. Thus, the weight of the vocabulary in the original text is determined based on the occurrence number of each vocabulary in the original text and the preset sampling coefficient.
Referring to fig. 3, fig. 3 is a schematic flowchart of determining weights of words in an original text according to an embodiment of the present invention, which specifically includes the following steps:
s301, in a preset range, adjusting a sampling coefficient, and determining a preset sampling coefficient.
P(wordi) Is related to the occurrence number of the vocabulary and the preset sampling coefficient. Referring to equation 4, equation 4 is P (word)i) And the functional relation between the occurrence times of the vocabularies and the preset sampling coefficient.
Figure BDA0003122409660000082
Wherein, Z (word)i) Is the number of occurrences of the vocabulary, and β is a predetermined sampling coefficient. It should be noted that equation 4 is obtained through multiple practices.
According to formula 4, in order to determine the weight of the vocabulary in the original text, two parameters, namely the number of occurrences of the vocabulary and a preset sampling coefficient, need to be determined. The number of occurrences of the vocabulary can be obtained by counting each vocabulary in the original text. The preset sampling coefficient needs to be obtained after adjustment.
In the embodiment of the invention, the adjustment of the sampling coefficient is realized within a preset range. As an example, the preset range is an interval of 0 to 1. That is, the preset range includes a range of 0 or more and 1 or less. That is, the maximum value of the sampling coefficient is 1, and the minimum value of the sampling coefficient is 0.
Referring to fig. 4, fig. 4 is a schematic diagram of the relationship between the occurrence number of words and the distribution probability according to the embodiment of the invention. The horizontal axis in fig. 4 represents the number of occurrences of words, and the vertical axis represents the distribution probability. Fig. 4 includes four curves corresponding to sampling coefficients β of 0.1, 0.01,0.001, and 0.0001.
As can be seen from fig. 4, the degree of smoothing is controlled by varying β. As an example, when β is 0.001 and the number of times of occurrence of words is 0.00089 or less, the distribution probability is 1.0. This means that rarely sampled words will be sampled at a large probability, such as: name or address. The word frequency in the original text is only 0.00089, but the probability of being sampled after smoothing is close to 1.
As β decreases, the probability that the vocabulary is sampled decreases significantly. Then the input matrix of the original samples will become sparse when the original text input is computed at the pointer generation network stage, resulting in under-fitting. Conversely, as β increases, the probability that the vocabulary was adopted will rise significantly and the smoothing effect of the sampling will disappear.
Considering that the sampling coefficients need to be adjusted in order in the process of determining the sampling coefficients, in order to determine the sampling coefficients as soon as possible, the sampling coefficients may be adjusted in order from large to small to determine the preset sampling coefficients. As an example, in the case where the preset sampling coefficient is set to 0.001, the sampling rate can satisfy the requirement of text generation.
By controlling the sampling coefficient beta, the smooth degree of sampling is controlled, the method is suitable for data distribution under various scenes, and the universality of a pointer generation network in the text generation process is improved.
S302, in the pointer generation network, smoothing the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient, and then determining the weight of the vocabulary in the original text.
In the pointer generation network, the weight of the vocabulary in the original text can be smoothed through the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient. Specifically, in order to smooth out the weight of the useless high-frequency words, a preset adoption coefficient is adjusted.
After the preset sampling coefficient is determined to smooth the weight of the vocabulary in the original text, the weight of the vocabulary in the original text is determined according to formula 2, formula 3 and formula 4.
Figure BDA0003122409660000091
From equation 5, it can be seen that the original text input P can be obtained by multiplying the weight of each vocabulary in the original text by the matrix formed by each vocabulary w in the original textvocab(w)。
The probability distribution of words in the original text is further smoothed, so that the probability matrix input by the original text is more consistent with the information structure of the original text, the weight of words with partial weak information is reduced, the weight of words with partial important information is increased, the probability distribution of the words finally obtained in the decoding stage is more reasonable, and the interpretability of the word distribution is enhanced.
In the embodiment of fig. 3, the weights of the useless high-frequency words are smoothly adjusted by adjusting the sampling coefficients, so that the weights of the words in the original text are determined.
S202, updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text.
After determining the weight of the words in the original text, the distribution probability of the decoded words in the original text can be updated according to the weight of the words in the original text on the basis of formula 1.
In one embodiment of the invention, the distribution probability of the vocabulary is adjusted according to the weight of the vocabulary in the original text. Specifically, according to the weight of the vocabulary in the original text, the distribution probability of the decoded common words in the original text is reduced, and the distribution probability of the decoded common words in the original text is improved.
In the embodiment of fig. 2, the distribution probability of the vocabulary is updated by adjusting the weight of the vocabulary, thereby increasing the probability that the non-use word is selected.
S103, establishing a decoding word list based on the distribution probability of the decoded words so as to generate the evolved text of the original text in the pointer generation network according to the decoding word list.
For the construction of the decoding word list, the traditional method selects the word list based on scalar quantities such as word frequency, tfidf and the like. Such as: the corpus has N words, and K is selected to construct a decoding word list in consideration of training efficiency and avoidance of overfitting. However, if words are selected only by word frequency, some important words cannot be included in the decoded vocabulary all the time because the word frequency is too low.
Referring to fig. 5, fig. 5 is a schematic flowchart of generating an evolved text of an original text according to an embodiment of the present invention, which specifically includes:
s501, establishing a decoding word list based on the distribution probability of the decoded words.
And after determining the distribution probability of the decoded words, storing the words with the distribution probability larger than a preset probability threshold value in a decoding word list. The distribution probability of the decoded words is the adjusted probability, the distribution probability of the decoded common words in the original text is reduced, and the distribution probability of the decoded non-common words in the original text is improved, so that the words in the decoded word list can take into account the consistency of the output of the evolution text and the importance of the low-frequency words, namely the non-common words.
It is understood that the decoding vocabulary includes the non-common words in the original text and the common words in the original text, and the distribution probability of the non-common words may be different from that of the common words.
S502, generating an evolved text of the original text according to the vocabulary in the decoding vocabulary in the pointer generation network.
In the pointer generation network, the evolved text of the original text is generated on the basis of the vocabulary in the decoding vocabulary. The evolving text is text that is processed and output via the pointer-generating network. As one example, the advancement text includes summary text and/or keyword text.
In the embodiment of fig. 5, the evolved text of the original text is generated based on the decoded vocabulary. The distribution probability of the non-common words in the decoding word list is increased compared with the original probability, and the occurrence probability of the non-common words is increased in the evolution text.
Compared with the Word distribution and Word sampling rate calculated in Word2Vec, the technical scheme in FIG. 5 is adopted, and the real probability distribution of the words is better met.
And S104, generating an evolved text of the original text output by the network through the pointer.
In order to improve the acquisition speed of the evolved text, the pointer generation network outputs the evolved text of the original text in a preset document. Therefore, the text of the original text can be directly obtained in the preset document without searching.
In the above embodiment, the original text input pointer is generated into a network; in the pointer generation network, updating the distribution probability of the decoded words in the original text according to the weight of the words in the original text; establishing a decoding word list based on the distribution probability of the decoded words so as to generate an evolved text of the original text in the pointer generation network according to the decoding word list; and the pointer generation network outputs the evolved text of the original text. The distribution probability of the words is updated according to the texts generated by the words exceeding the word list, so that the accuracy can be improved and the repeatability can be reduced.
Referring to fig. 6, fig. 6 is a schematic diagram of a main structure of a text generation apparatus according to an embodiment of the present invention, where the text generation apparatus may implement a text generation method, as shown in fig. 7, the text generation apparatus specifically includes:
an input module 601, configured to input an original text pointer into a network;
an updating module 602, configured to update, in the pointer generation network, a distribution probability of a word decoded in the original text according to a weight of a word in the original text;
a generating module 603, configured to establish a decoding word list based on the distribution probability of the decoded word, so as to generate an evolved text of the original text in the pointer generation network according to the decoding word list;
an output module 604, configured to output the evolved text of the original text by the pointer generation network.
In an embodiment of the present invention, the input module 601 is specifically configured to input the filtered original text into the pointer generation network.
In an embodiment of the present invention, the updating module 602 is specifically configured to, in the pointer generation network, adjust a pointer coefficient in the original text according to a weight of a word in the original text, so as to update a distribution probability of a decoded word in the original text.
In an embodiment of the present invention, the updating module 602 is specifically configured to determine, in the pointer generation network, a weight of each vocabulary in the original text based on the occurrence number of each vocabulary in the original text;
and updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text.
In an embodiment of the present invention, the updating module 602 is specifically configured to determine, in the pointer generation network, weights of words in the original text based on the occurrence frequency of each word in the original text and a preset sampling coefficient.
In an embodiment of the present invention, the updating module 602 is specifically configured to adjust a sampling coefficient within a preset range, and determine the preset sampling coefficient;
and in the pointer generation network, determining the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient.
In an embodiment of the present invention, the preset range includes an interval greater than or equal to 0 and less than or equal to 1.
In an embodiment of the present invention, the updating module 602 is specifically configured to adjust the sampling coefficients in a preset range according to a descending order, and determine the preset sampling coefficients.
In one embodiment of the invention, the preset sampling coefficient is equal to 0.001.
In an embodiment of the present invention, the updating module 602 is specifically configured to, in the pointer generation network, determine the weight of the vocabulary in the original text after smoothing the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient.
In an embodiment of the present invention, the updating module 602 is specifically configured to reduce the distribution probability of the decoded common words in the original text and increase the distribution probability of the decoded common words in the original text according to the weights of the words in the original text.
In one embodiment of the invention, the pointer generation network comprises a Transformer model.
In an embodiment of the present invention, the generating module 603 is specifically configured to establish a decoding word list based on the distribution probability of the decoded word;
and in the pointer generation network, generating an evolved text of the original text according to the vocabulary in the decoding vocabulary.
In one embodiment of the present invention, the decoding word list includes non-common words in the original text and common words in the original text, and the distribution probability of the non-common words is different from the distribution probability of the common words.
In an embodiment of the present invention, the output module 604 is specifically configured to output the evolved text of the original text in a preset document through the pointer generation network.
In one embodiment of the invention, the evolving text comprises abstract text.
In one embodiment of the invention, the evolving text includes keyword text.
Fig. 7 illustrates an exemplary system architecture 700 to which the method of text generation or the apparatus of text generation of an embodiment of the present invention may be applied.
As shown in fig. 7, the system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 serves to provide a medium for communication links between the terminal devices 701, 702, 703 and the server 705. Network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 701, 702, 703 to interact with a server 705 over a network 704, to receive or send messages or the like. The terminal devices 701, 702, 703 may have installed thereon various communication client applications, such as a shopping-like application, a web browser application, a search-like application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only).
The terminal devices 701, 702, 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 705 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 701, 702, 703. The backend management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (for example, target push information, product information — just an example) to the terminal device.
It should be noted that the method for generating a text provided by the embodiment of the present invention is generally executed by the server 705, and accordingly, the apparatus for generating a text is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks, and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 8, shown is a block diagram of a computer system 800 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage section 808 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program executes the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 801.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes an input module, an update module, a generation module, and an output module. Where the names of these modules do not in some cases constitute a limitation on the modules themselves, for example, the input module may also be described as "for generating a network of raw text input pointers".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise:
generating a network by inputting an original text pointer;
in the pointer generation network, updating the distribution probability of the decoded words in the original text according to the weight of the words in the original text;
establishing a decoding word list based on the distribution probability of the decoded words so as to generate an evolved text of the original text in the pointer generation network according to the decoding word list;
and the pointer generation network outputs the evolved text of the original text.
According to the technical scheme of the embodiment of the invention, an original text input pointer is used for generating a network; in the pointer generation network, updating the distribution probability of the decoded words in the original text according to the weight of the words in the original text; establishing a decoding word list based on the distribution probability of the decoded words so as to generate an evolved text of the original text in the pointer generation network according to the decoding word list; and the pointer generation network outputs the evolved text of the original text. The distribution probability of the words is updated according to the texts generated by the words exceeding the word list, so that the accuracy can be improved and the repeatability can be reduced.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (18)

1. A method of text generation, comprising:
generating a network by inputting an original text pointer;
in the pointer generation network, updating the distribution probability of the decoded words in the original text according to the weight of the words in the original text;
establishing a decoding word list based on the distribution probability of the decoded words so as to generate an evolved text of the original text in the pointer generation network according to the decoding word list;
and the pointer generation network outputs the evolved text of the original text.
2. The method of text generation according to claim 1, wherein said generating the original text input pointer into a network comprises:
and inputting the screened original text into the pointer to generate a network.
3. The method of claim 1, wherein updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text in the pointer generation network comprises:
in the pointer generation network, according to the weight of the vocabulary in the original text, the pointer coefficient in the original text is adjusted to update the distribution probability of the decoded words in the original text.
4. The method of claim 1, wherein updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text in the pointer generation network comprises:
in the pointer generation network, determining the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text;
and updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text.
5. The method of claim 4, wherein determining weights of words in the original text based on the number of occurrences of each word in the original text in the pointer generation network comprises:
and in the pointer generation network, determining the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient.
6. The method of claim 5, wherein determining weights of words in the original text based on the occurrence number of each word in the original text and a preset sampling coefficient in the pointer generation network comprises:
adjusting a sampling coefficient within a preset range, and determining the preset sampling coefficient;
and in the pointer generation network, determining the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient.
7. The method of text generation according to claim 6, wherein the preset range includes an interval greater than or equal to 0 and less than or equal to 1.
8. The method for generating text according to claim 6, wherein the adjusting the sampling coefficient and determining the preset sampling coefficient within a preset range comprises:
and within a preset range, adjusting the sampling coefficients according to a descending order, and determining the preset sampling coefficients.
9. The method of claim 5, wherein determining weights of words in the original text based on the occurrence number of each word in the original text and a preset sampling coefficient in the pointer generation network comprises:
in the pointer generation network, smoothing the weight of the vocabulary in the original text based on the occurrence frequency of each vocabulary in the original text and a preset sampling coefficient, and then determining the weight of the vocabulary in the original text.
10. The method of claim 4, wherein updating the distribution probability of the decoded words in the original text according to the weights of the words in the original text comprises:
and according to the weight of the vocabulary in the original text, reducing the distribution probability of the decoded common words in the original text and improving the distribution probability of the decoded common words in the original text.
11. The method of claim 1, wherein the creating a decoded word list based on the distributed probabilities of the decoded words to generate the evolved text of the original text in the pointer generation network according to the decoded word list comprises:
establishing a decoding word list based on the distribution probability of the decoded words;
and in the pointer generation network, generating an evolved text of the original text according to the vocabulary in the decoding vocabulary.
12. The method of claim 1, wherein the decoded vocabulary comprises non-common words in the original text and common words in the original text, and wherein the distribution probability of the non-common words is different from the distribution probability of the common words.
13. The method of text generation according to claim 1, wherein the pointer generation network outputs the evolved text of the original text, comprising:
and outputting the evolved text of the original text by the pointer generation network in a preset document.
14. The method of text generation according to claim 1, wherein the evolving text includes abstract text.
15. The method of text generation according to claim 1, wherein the evolved text includes keyword text.
16. An apparatus for text generation, comprising:
the input module is used for generating a network by inputting an original text pointer;
the updating module is used for updating the distribution probability of the decoded words in the original text according to the weight of the words in the original text in the pointer generation network;
the generating module is used for establishing a decoding word list based on the distribution probability of the decoded words so as to generate an evolved text of the original text in the pointer generation network according to the decoding word list;
and the output module is used for outputting the evolution text of the original text by the pointer generation network.
17. An electronic device for text generation, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-15.
18. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-15.
CN202110679716.9A 2021-06-18 2021-06-18 Text generation method, device, equipment and computer readable medium Pending CN113283247A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110679716.9A CN113283247A (en) 2021-06-18 2021-06-18 Text generation method, device, equipment and computer readable medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110679716.9A CN113283247A (en) 2021-06-18 2021-06-18 Text generation method, device, equipment and computer readable medium

Publications (1)

Publication Number Publication Date
CN113283247A true CN113283247A (en) 2021-08-20

Family

ID=77285060

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110679716.9A Pending CN113283247A (en) 2021-06-18 2021-06-18 Text generation method, device, equipment and computer readable medium

Country Status (1)

Country Link
CN (1) CN113283247A (en)

Similar Documents

Publication Publication Date Title
CN109376234B (en) Method and device for training abstract generation model
CN107491534B (en) Information processing method and device
US10650102B2 (en) Method and apparatus for generating parallel text in same language
CN106873799B (en) Input method and device
CN110162767A (en) The method and apparatus of text error correction
JP7301922B2 (en) Semantic retrieval method, device, electronic device, storage medium and computer program
CN110390408B (en) Transaction object prediction method and device
US11586817B2 (en) Word vector retrofitting method and apparatus
WO2019174450A1 (en) Dialogue generation method and apparatus
CN114861889B (en) Deep learning model training method, target object detection method and device
EP3377983A1 (en) Generating feature embeddings from a co-occurrence matrix
CN110874532A (en) Method and device for extracting keywords of feedback information
CN111862961A (en) Method and device for recognizing voice
CN111008213B (en) Method and apparatus for generating language conversion model
CN115631261A (en) Training method of image generation model, image generation method and device
CN113076756A (en) Text generation method and device
CN113051894B (en) Text error correction method and device
CN110852057A (en) Method and device for calculating text similarity
CN113053362A (en) Method, device, equipment and computer readable medium for speech recognition
CN110895655A (en) Method and device for extracting text core phrase
CN112464654B (en) Keyword generation method and device, electronic equipment and computer readable medium
CN115860003A (en) Semantic role analysis method and device, electronic equipment and storage medium
CN113283247A (en) Text generation method, device, equipment and computer readable medium
CN110852078A (en) Method and device for generating title
CN111737572B (en) Search statement generation method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination