WO2024131111A1 - 一种智能写作方法、装置、设备及非易失性可读存储介质 - Google Patents

一种智能写作方法、装置、设备及非易失性可读存储介质 Download PDF

Info

Publication number
WO2024131111A1
WO2024131111A1 PCT/CN2023/114518 CN2023114518W WO2024131111A1 WO 2024131111 A1 WO2024131111 A1 WO 2024131111A1 CN 2023114518 W CN2023114518 W CN 2023114518W WO 2024131111 A1 WO2024131111 A1 WO 2024131111A1
Authority
WO
WIPO (PCT)
Prior art keywords
sentence
word
paragraph
topic
text
Prior art date
Application number
PCT/CN2023/114518
Other languages
English (en)
French (fr)
Inventor
李峰
刘红丽
Original Assignee
苏州元脑智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州元脑智能科技有限公司 filed Critical 苏州元脑智能科技有限公司
Publication of WO2024131111A1 publication Critical patent/WO2024131111A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/166—Editing, e.g. inserting or deleting
    • G06F40/186—Templates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/42—Data-driven translation
    • G06F40/44—Statistical methods, e.g. probability models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation

Definitions

  • the present application relates to the field of intelligent writing technology, and in particular to an intelligent writing method, device, equipment and non-volatile readable storage medium.
  • Intelligent writing can meet the demand for labor liberation in this field.
  • intelligent writing has developed from rule-based and template-based writing to intelligent creation with deep neural network models as the core, and has gradually moved from auxiliary creation to automation.
  • New pre-trained language models are constantly being proposed, such as openAI (Open Artificial Intelligence)'s GPT (Generative Pre-Training)-3, ERNIE (Enhanced Representation through kNowledge IntEgration), T5 and other models.
  • Technicians are trying to use the original generation capabilities of these large models to solve text generation problems in different scenarios.
  • the purpose of this application is to provide an intelligent writing method, device, equipment and non-volatile readable storage medium, which can reduce labor costs and improve the accuracy of parameter adjustment while solving the problem of text duplication.
  • the optional solutions are as follows:
  • an intelligent writing method comprising:
  • the article outline includes the topic sentences of all paragraphs;
  • the optimal paragraph for any topic sentence is determined based on the candidate paragraphs corresponding to the topic sentence.
  • generate an article outline including:
  • an article outline is generated based on the writing type selected by the user, the writing topic content entered, and the first keyword, including:
  • the topic sentence of each paragraph, the second keyword, and the first keyword are respectively combined into model input data according to a preset method, and the model input data is input into the pre-trained model to generate a new topic sentence corresponding to each paragraph;
  • the template file most relevant to the writing topic content is retrieved from the content knowledge base corresponding to the writing type selected by the user, including:
  • the template file corresponding to the file name is searched from the content knowledge base corresponding to the writing type selected by the user.
  • a knowledge base pool is created; wherein the knowledge base pool includes a file name knowledge base and a content knowledge base corresponding to each writing type; the content knowledge base includes each template file, and the file name knowledge base includes the file name of each template file.
  • the probability of the target word in the dictionary is penalized based on a penalty factor, including:
  • the parameters are adjusted based on the penalty factor and the probability corresponding to the target word in the dictionary is penalized by the penalty factor.
  • the probability corresponding to the target word in the dictionary is penalized based on the penalty factor adjustment parameters and the penalty factor, including:
  • Penalize the probability of the target word in the dictionary where xi is the target word and Iin is the target text.
  • the probability of the target word is penalized. Penalty is performed, n is used to indicate the number of times the target word appears in the target text or the constant 1, and P is the penalty factor.
  • generate words from the dictionary based on probability including:
  • the generated words are selected from the dictionary based on probability, where the probability is the probability after penalty.
  • the decoding rules include temperature sampling, TopK sampling strategy and TopP sampling strategy.
  • calculate diversity evaluation indicators based on the new target text including:
  • the original target text is removed from the new target text to obtain the calculated text;
  • the original target text is the target text composed of the topic sentence in the text outline and the paragraphs taken from the content knowledge base;
  • a penalty factor is updated based on the diversity evaluation index, including:
  • update the penalty factor based on the indicator difference including:
  • the method before determining the optimal paragraph of any topic sentence based on the candidate paragraphs corresponding to the topic sentence, the method further includes:
  • both the first semantic similarity and the second semantic similarity are greater than a preset similarity threshold, the sentence is retained; otherwise, the sentence and the text following the sentence are deleted to obtain a deleted text;
  • the deleted text is used as a candidate paragraph corresponding to the topic sentence; otherwise, a candidate paragraph corresponding to the topic sentence is regenerated based on the deleted text.
  • calculating the semantic similarity between each sentence in the candidate paragraph corresponding to any topic sentence and the previous sentence and the topic sentence to obtain a first semantic similarity and a second semantic similarity includes:
  • the cosine value of the angle between each sentence and the topic sentence is calculated as the second semantic similarity, where ⁇ is the angle between vector x and vector y, xi represents the component of vector x, and yi represents the component of vector y.
  • determining the optimal paragraph for any topic sentence based on candidate paragraphs corresponding to the topic sentence includes:
  • each candidate paragraph is scored based on the preset scoring criteria, and the candidate paragraph with the highest score is determined as the optimal paragraph corresponding to the topic sentence.
  • score each candidate paragraph based on preset scoring criteria including:
  • the method further includes:
  • an intelligent writing device comprising:
  • An article outline generation module is configured to generate an article outline; the article outline includes topic sentences of all paragraphs;
  • a target text construction module is configured to form a target text by combining any topic sentence in the article outline with paragraphs related to the topic sentence taken from the content knowledge base;
  • the generated word probability acquisition module is configured to input the target text into the pre-trained model to obtain the probability of each word in the dictionary being a generated word;
  • a target word probability penalty module is configured to penalize the probability corresponding to the target word in the dictionary based on a penalty factor; wherein the target word is a word existing in the target text;
  • a generated word extraction module is configured to extract generated words from a dictionary based on probability
  • a target text updating module is configured to update the target text based on the generated words to obtain a new target text
  • a diversity evaluation index calculation module is configured to calculate a diversity evaluation index based on a new target text
  • the penalty factor updating module is configured to update the penalty factor based on the diversity evaluation index. Accordingly, the device is also configured to trigger the generated word probability acquisition module to input the new target text into the pre-trained model, and continuously iterate until the generated word extracted by the generated word extraction module is a cutoff symbol, and the current target text is determined as a candidate paragraph corresponding to the topic sentence;
  • the optimal paragraph determination module is configured to determine the optimal paragraph of any topic sentence based on the candidate paragraphs corresponding to the topic sentence.
  • the present application discloses an electronic device, including a memory and a processor, wherein:
  • a memory arranged to store a computer program
  • the processor is configured to execute a computer program to implement the aforementioned intelligent writing method.
  • the present application discloses a computer non-volatile readable storage medium, which is configured to store a computer program, wherein the computer program implements the aforementioned intelligent writing method when executed by a processor.
  • the present application first generates an article outline, which includes the topic sentences of all paragraphs. Then, any topic sentence in the article outline and the paragraphs related to the topic sentence taken out from the content knowledge base are combined into a target text, and the target text is input into the pre-trained model to obtain the probability that each word in the dictionary is a generated word, and the probability corresponding to the target word in the dictionary is penalized based on the penalty factor.
  • the generated word is taken out from the dictionary based on the probability, and the target word is a word existing in the target text.
  • the target text is updated based on the generated word to obtain a new target text, and a diversity evaluation index is calculated based on the new target text.
  • the penalty factor is updated based on the diversity evaluation index, and the new target text is input into the pre-trained model. It is continuously iterated until the generated word taken out is a cutoff character, then the current target text is determined as the candidate paragraph corresponding to the topic sentence, and then the optimal paragraph of the topic sentence is determined based on the candidate paragraph corresponding to any topic sentence. That is, the present application first generates an article outline, and for any topic sentence in the article outline, generates candidate paragraphs and selects the best paragraph from the candidate paragraphs. In the process of generating candidate paragraphs, the diversity evaluation index is used to guide the adaptive adjustment of the repetition penalty factor, which can reduce labor costs and improve the accuracy of parameter adjustment while solving the problem of text repetition.
  • FIG1 is a flow chart of an intelligent writing method disclosed in the present application.
  • FIG2 is a schematic diagram of an optional knowledge base pool disclosed in the present application.
  • FIG3 is a schematic diagram of an optional optimal paragraph generation disclosed in the present application.
  • FIG4 is a schematic diagram of an optional candidate paragraph generation method disclosed in the present application.
  • FIG5 is a schematic diagram of the structure of an intelligent writing device disclosed in the present application.
  • FIG. 6 is a structural diagram of an electronic device disclosed in this application.
  • GPT Generic Pre-Training
  • ERNIE Enhanced Representation through kNowledge IntEgration
  • T5 kNowledge IntEgration
  • GPT3 is an autoregressive generative pre-trained language model that can generate text information with a certain degree of semantic relevance based on given text information, such as answering questions, continuing articles, etc.
  • GPT mainly uses the decoder part of the Transformer (i.e., converter) to perform generative one-way language model modeling, and predicts the probability distribution of the current output word based on several words that have appeared.
  • an intelligent writing method including:
  • Step S11 Generate an article outline; the article outline includes the topic sentences of all paragraphs.
  • an article outline can be generated based on the writing type selected by the user, the input writing subject content, and the first keyword.
  • a template file most relevant to the writing subject content can be retrieved from the content knowledge base corresponding to the writing type selected by the user; each paragraph in the template file is input into a pre-trained model to generate a topic sentence and a second keyword for each paragraph; the topic sentence of each paragraph, the second keyword, and the first keyword are respectively combined into model input data in a preset manner, and the model input data is input into the pre-trained model to generate a new topic sentence corresponding to each paragraph; the new topic sentences of each paragraph are combined to obtain an article outline.
  • the model input data is a prompt.
  • the embodiment of the present application can retrieve the file name most relevant to the writing subject content from the file name knowledge base corresponding to the writing type selected by the user; and search for the template file corresponding to the file name from the content knowledge base corresponding to the writing type selected by the user.
  • the bm25 algorithm can be used to retrieve the file name most relevant to the writing subject content from the file name knowledge base corresponding to the writing type selected by the user.
  • the embodiment of the present application can create a knowledge base pool, wherein the knowledge base pool includes a file name knowledge base and a content knowledge base corresponding to each writing type; the content knowledge base includes each template file, and the file name knowledge base includes the file name of each template file.
  • the template file can be a txt file.
  • the embodiment of the present application can build a multi-scenario knowledge base pool based on the latest reference materials provided by the user or obtained through the Internet.
  • the knowledge base is built using the latest reference materials, relevant knowledge is retrieved, and prompts are formed with the topic sentence, and the text is input into the pre-trained model to enable the pre-trained model to learn the latest knowledge and analyze it, thereby solving the problem that the pre-trained model cannot keep up with the latest dynamic developments.
  • Figure 2 which is a schematic diagram of an optional knowledge base pool disclosed in the embodiment of the present application.
  • the knowledge base is classified and constructed according to different application scenarios, and can include multiple writing types such as official document writing, news information, advertising and marketing.
  • Each writing type includes a txt (a text file for storing text data) file name knowledge base and a txt content knowledge base; the file name of each txt is consistent with the title of its content, and the txt file name knowledge base is constructed by all titles; the txt content knowledge base It consists of the contents of all txt files.
  • Each txt file is a latest reference article, and one paragraph in the article occupies one line in the txt file.
  • the implementation process of automatically generating an article outline can be: input the writing type selected by the user, the subject content to be written, and the keyword K new (i.e., the first keyword).
  • the subject content the most relevant one is retrieved in the txt file name knowledge base under the writing type, and the corresponding template file is found in the txt content knowledge base according to the retrieved file name.
  • the embodiment of the present application can be implemented by the get_top_n function in the BM250kapi class), and then the template file is input into the pre-trained model by paragraphs to generate the topic sentence and keyword K old (i.e., the second keyword) of each paragraph.
  • the old keyword K old , the topic sentence of each paragraph, and the new keyword K new are respectively combined into prompts in a certain way, and input into the pre-trained model to generate new topic sentences. Finally, all the paragraph topic sentences are combined together as an article outline.
  • the method of composing a prompt is:
  • the bm25 algorithm is an algorithm used to evaluate the relevance between search terms and documents. It is an algorithm based on a probabilistic retrieval model. For example, there is a query and a batch of documents Ds (Dataset). Now to calculate the relevance score between the query and each document D, first split the query to get the words, and then the word score consists of three parts: the relevance between the word and D; the relevance between the word and the query; the weight of each word. Finally, the score of each word is summed up to get the score between the query and the document.
  • Step S12 any topic sentence in the article outline and paragraphs related to the topic sentence taken from the content knowledge base are combined into a target text.
  • the embodiment of the present application processes each topic sentence in the article outline separately, and obtains the target text corresponding to each topic sentence, so as to obtain the best paragraph for each topic sentence.
  • any topic sentence in the article outline and paragraphs related to the topic sentence taken from the content knowledge base corresponding to the writing type selected by the user are combined into a target text.
  • the content knowledge base is a pre-created knowledge base that stores template files.
  • the most relevant n paragraphs can be retrieved, where n is a positive integer greater than or equal to 1, such as 1, 2, 3, 4, 5, etc.
  • Each paragraph related to the topic sentence in the n paragraphs is combined with the topic sentence to form a target text, so as to generate candidate paragraphs corresponding to each related paragraph, and obtain n candidate paragraphs.
  • the retrieval method used can be the bm25 algorithm.
  • Step S13 Input the target text into the pre-trained model to obtain the probability that each word in the dictionary is a generated word.
  • Step S14 Penalize the probability corresponding to the target word in the dictionary based on the penalty factor, and take out the generated word from the dictionary based on the probability; wherein the target word is a word existing in the target text.
  • the probability corresponding to the target word in the dictionary can be penalized based on the penalty factor adjustment parameter and the penalty factor.
  • the number of times the target word appears in the target text can be counted to obtain the penalty factor adjustment parameter corresponding to the target word.
  • a configuration parameter can be obtained as the penalty factor adjustment parameter corresponding to the target word. That is, the penalty factor adjustment parameter can be the number of times the target word appears in the target text, or a configurable constant.
  • decoding rules such as Temperature, TopK, and TopP sampling strategies
  • decoding rules can be used to select generated words from the dictionary based on probability.
  • the probability based on which the words are selected is the probability after the penalty.
  • the penalty factor can be used to reduce the probability of words that have appeared or to force the use of repeated words. Reasonable setting of the penalty factor can avoid repetition and increase innovation, but setting it too large will have a counterproductive effect when generating long texts.
  • Penalties can be imposed based on the number of times a word appears (frequency penalty). The more times a word appears, the lower the probability of the word appearing in the subsequent text, while enhancing the creativity of the subsequent text. Penalties can also be imposed based on whether the word has appeared before (response penalty). Words that have appeared are penalized to reduce their probability of appearing in the subsequent text, while enhancing the creativity of the subsequent text.
  • Randomness can be increased through sampling strategies.
  • Temperature sampling by strengthening the probability of top words, only the most likely words are sampled, so that randomness can be increased while ensuring that no general errors occur.
  • the logits output by the model can be divided by a temperature (Temperature, T) less than 1.
  • f(i) represents the initial probability of the model outputting the i-th word
  • p(i) represents the probability of the i-th word after strengthening.
  • softmax used to convert a set of real numbers into probability distribution
  • the probability of words with high probability is higher.
  • the top words are selected first according to the obtained probability, and then sampled, which directly eliminates the possibility of low-probability words appearing.
  • TopK sampling select the k tokens with the highest probability, then recalculate the probability through softmax, then sample according to the obtained probability, and then proceed to the next generation, and repeat. But there may be a problem with TopK. If the model is very sure about the current generation, for example, the probability of the token with the highest probability is 0.9, and the probabilities of the remaining tokens are very low. At this time, if only topk sampling is used, the low probability situation will still occur. Therefore, it is necessary to limit the cumulative probability of the top token, which is TopP sampling.
  • Step S15 Update the target text based on the generated words to obtain a new target text.
  • Step S16 Calculate the diversity evaluation index according to the new target text, update the penalty factor based on the diversity evaluation index, and input the new target text into the pre-training model, and iterate continuously until the generated word taken out is the cutoff symbol, then determine the current target text as the candidate paragraph corresponding to the topic sentence.
  • the original target text can be removed from the new target text to obtain a calculated text;
  • the original target text is a target text composed of a topic sentence in the text outline and a paragraph taken from a content knowledge base; and a diversity evaluation index is calculated based on the calculated text.
  • an indicator difference between the diversity evaluation indicator and the diversity evaluation indicator obtained in the previous iteration may be calculated; and a penalty factor may be updated based on the indicator difference.
  • Distinct is used to judge the diversity of machine replies.
  • the Distinct index determines whether a large number of common and repetitive replies appear.
  • the larger the Distinct(n) the higher the diversity of the generated replies.
  • the embodiment of the present application uses the diversity evaluation index to adjust the penalty factor.
  • the embodiment of the present application can calculate the semantic similarity between each sentence in the candidate paragraph corresponding to any topic sentence and the previous sentence and the topic sentence, and obtain the first semantic similarity and the second semantic similarity; if the first semantic similarity and the second semantic similarity are both greater than the preset similarity threshold, the sentence is retained, otherwise the sentence and the text after the sentence are deleted to obtain the deleted text; if the number of words in the deleted text is greater than the preset word count threshold, the deleted text is used as the candidate paragraph corresponding to the topic sentence, otherwise the candidate paragraph corresponding to the topic sentence is regenerated based on the deleted text. It is understandable that if there are multiple candidate paragraphs, the above steps are performed separately. That is, the embodiment of the present application uses semantic similarity to detect and delete text that deviates from the topic, and regenerates qualified candidate paragraphs.
  • the calculation process of semantic similarity is as follows: first, the sentence is segmented, then the corresponding vector is obtained for each segmented word, then all vectors are added and averaged to obtain the sentence vector, and finally the cosine value of the angle is calculated using the following formula. The closer the cosine value is to 1 (i.e. the smaller the angle), the higher the similarity.
  • ⁇ is the angle between vectors x and y
  • xi and yi represent the components of vectors x and y respectively.
  • the word2vec (a model for generating word vectors) model can be used to calculate the Vector, which is to vectorize all words, so that the relationship between words can be quantitatively measured and the connection between words can be explored.
  • Step S17 Determine the optimal paragraph for any topic sentence based on the candidate paragraphs corresponding to the topic sentence.
  • each candidate paragraph is scored based on a preset scoring standard, and the candidate paragraph with the highest score is determined as the optimal paragraph corresponding to the topic sentence.
  • the present application embodiment may use the PPL indicator to score each candidate paragraph.
  • PPL refers to the perplexity in the language model, which is an indicator to measure whether a sentence is fluent. It is defined as:
  • x1 , x2 , ..., xi-1 ) represents the probability of predicting the i-th word based on the previous words
  • N represents the length of the sentence.
  • PPL value the more natural the text generated by the model and the smoother the sentences.
  • Using PPL to evaluate text quality can avoid the situation where the text generated by the model is out of order or reversed.
  • the embodiment of the present application may utilize preset detection and modification logic to perform error detection and modification on the optimal paragraph.
  • FIG3 is a schematic diagram of an optional optimal paragraph generation disclosed in an embodiment of the present application.
  • the steps include:
  • step (b) judging i ⁇ N+1, if i ⁇ N+1, obtaining the current i-th topic sentence, if i is greater than or equal to N+1, going to step (g);
  • the result list is output and saved as a txt file. Finally, the result list includes the best paragraphs corresponding to all the topic sentences in the article outline.
  • the topic sentence + related paragraphs form the prompt input pre-training model, and the generated text is prone to duplication, including duplication of the generated content itself and duplication with the input.
  • the embodiment of the present application adopts adaptive adjustment of the penalty parameter to solve this problem.
  • Figure 4 is a schematic diagram of an optional candidate paragraph generation disclosed in the embodiment of the present application. It includes the following steps:
  • Input text i.e., target text
  • prompt I 0 consisting of the relevant paragraph + topic sentence
  • xi represents any word in the dictionary. If the word is in the input text I in , the probability corresponding to the word is Penalize, n represents the number of times xi appears in the input text or a constant 1 (set according to different scenarios), P is the penalty factor, and its initial value is 1;
  • step (e) also needs to solve the problem of topic deviation in long texts, and perform the following operations on all candidate paragraphs respectively: (A) obtain an alternative answer; (B) use the trained Word2vec model to calculate the semantic similarity between each sentence in the candidate answer and the topic sentence and the sentence before the sentence; (C) if both similarities are greater than the set threshold S, retain them; otherwise delete the sentence and the following text; (D) count the number of words in the candidate paragraphs, if it is greater than the set threshold T, retain the output answer; otherwise re-input the pre-trained model based on the remaining text to generate, and continue with step (B).
  • the articles generated by intelligent writing may contain typos, sensitive words, and grammatical errors.
  • the embodiments of the present application can detect and modify these errors, and finally manually review and publish the articles.
  • the pre-trained model learns the latest knowledge and can analyze it; uses text diversity evaluation indicators to guide the adaptive adjustment of repetition penalty parameters, saving the cost of manual parameter adjustment; uses semantic similarity to detect and delete text that deviates from the topic, and regenerates qualified candidate paragraphs.
  • the user selects the document writing type and enters the topic content they want to write: please write a special economic plan for XX company based on the digital economy development planning materials.
  • the BM25 algorithm is used to retrieve the most relevant file name in the file name knowledge base based on the topic content, and the corresponding txt file is input into the pre-trained model by paragraph to generate the topic sentence for each paragraph. Then, each topic sentence and keyword combination prompt are input into the pre-trained model to generate a new topic sentence.
  • Pre-trained model generation Promote the construction of cloud centers.
  • the bm25 algorithm can be used to retrieve the three paragraphs with the highest scores in the text knowledge base.
  • the top 1, top 2, and top 3 are concatenated with the topic sentence to form a prompt, which is input into the pre-trained model to output three candidate answers (i.e., candidate paragraphs).
  • the PPL index is used as the scoring standard, and the one with the highest score is selected as the optimal answer.
  • the penalty parameter adaptive adjustment method is used to solve the problem of repeated generation.
  • the main principle is that when the repetition of the generated content increases, the penalty factor is increased, that is, the model's penalty for the generated words is increased, and the model generation is biased towards words that have not appeared.
  • the penalty factor is reduced, that is, the model's penalty for the generated words is reduced to prevent the model from deviating from the topic due to excessive penalty.
  • the trained Word2vec model is used to calculate the semantic similarity between each sentence in the answer and the topic sentence, delete off-topic sentences, and supplement semantically consistent sentences.
  • this application builds a knowledge base based on the latest reference materials, retrieves knowledge related to the topic sentence through a retrieval algorithm, and forms a prompt with the topic sentence, inputs the pre-training model to generate text, and uses this scheme to enable the model to learn the latest knowledge related to the topic.
  • the text diversity evaluation index is used to guide the adaptive adjustment of the repetition penalty parameter, increase the diversity of the generated content, reduce the time and effort of manual adjustment, and ensure that the generated text is neither repetitive nor overcorrected.
  • this application matches the semantic similarity of each sentence generated with the topic sentence and the previous sentence, deletes the sentences that deviate from the topic, and generates qualified answers.
  • the overall implementation scheme and system of intelligent writing are also refined to improve the automation level of intelligent writing.
  • an embodiment of the present application provides an intelligent writing device, including:
  • the article outline generating module 11 is configured to generate an article outline; the article outline includes topic sentences of all paragraphs;
  • the target text construction module 12 is configured to form a target text by combining any topic sentence in the article outline with paragraphs related to the topic sentence taken from the content knowledge base;
  • a generated word probability acquisition module 13 is configured to input the target text into the pre-trained model to obtain the probability of each word in the dictionary being a generated word;
  • the target word probability penalty module 14 is configured to penalize the probability corresponding to the target word in the dictionary based on the penalty factor; wherein the target word is a word existing in the target text;
  • a generated word extraction module 15 is configured to extract a generated word from a dictionary based on probability
  • a target text updating module 16 is configured to update the target text based on the generated words to obtain a new target text
  • a diversity evaluation index calculation module 17 is configured to calculate a diversity evaluation index based on a new target text
  • the penalty factor updating module 18 is configured to update the penalty factor based on the diversity evaluation index. Accordingly, the device is also configured to trigger the generated word probability acquisition module to input the new target text into the pre-trained model, and continuously iterate until the generated word extracted by the generated word extraction module is a cutoff symbol, and the current target text is determined as a candidate paragraph corresponding to the topic sentence;
  • the optimal paragraph determination module 19 is configured to determine the optimal paragraph of any topic sentence based on the candidate paragraphs corresponding to the topic sentence.
  • the embodiment of the present application first generates an article outline, which includes the topic sentences of all paragraphs, and then any topic sentence in the article outline and the paragraphs related to the topic sentence taken out from the content knowledge base are combined into a target text, and the target text is input into the pre-trained model to obtain the probability that each word in the dictionary is a generated word, and the probability corresponding to the target word in the dictionary is penalized based on the penalty factor, and the generated word is taken out from the dictionary based on the probability, and the target word is a word existing in the target text, and the target text is updated based on the generated word to obtain a new target text, and a diversity evaluation index is calculated based on the new target text, and the penalty factor is updated based on the diversity evaluation index, and the new target text is input into the pre-trained model, and it is continuously iterated until the generated word taken out If the term is a cutoff character, the current target text is determined as the candidate paragraph corresponding to the topic sentence, and then the optimal paragraph of the topic
  • the embodiment of the present application first generates an article outline, and for any topic sentence in the article outline, generates candidate paragraphs and selects the best paragraph from the candidate paragraphs.
  • the diversity evaluation index is used to guide the adaptive adjustment of the penalty factor of the repetition degree, which can reduce labor costs and improve the accuracy of parameter adjustment while solving the problem of text repetition.
  • the article outline generating module 11 is configured to generate an article outline based on the writing type selected by the user, the writing subject content inputted and the first keyword.
  • the article outline generating module 11 is configured to include:
  • a template file retrieval unit is configured to retrieve a template file most relevant to the writing subject content from a content knowledge base corresponding to the writing type selected by the user;
  • a model generation unit is configured to input each paragraph in the template file into the pre-trained model to generate a topic sentence and a second keyword for each paragraph;
  • a new topic sentence generating unit is configured to respectively form model input data from the topic sentence of each paragraph, the second keyword, and the first keyword in a preset manner, and input the model input data into the pre-trained model to generate a new topic sentence corresponding to each paragraph;
  • the article outline generating unit is configured to combine the new topic sentences of each paragraph to obtain the article outline.
  • the template file retrieval unit is configured to retrieve the file name most relevant to the writing subject content from the file name knowledge base corresponding to the writing type selected by the user; and search for the template file corresponding to the file name from the content knowledge base corresponding to the writing type selected by the user.
  • the device further comprises:
  • the knowledge base pool creation module is configured to create a knowledge base pool; wherein the knowledge base pool includes a file name knowledge base and a content knowledge base corresponding to each writing type; the content knowledge base includes each template file, and the file name knowledge base includes the file name of each template file.
  • the target word probability penalty module 14 is configured to penalize the probability corresponding to the target word in the dictionary based on the penalty factor adjustment parameter and the penalty factor.
  • the target word probability penalty module 14 is further configured to count the number of times the target word appears in the target text to obtain a penalty factor adjustment parameter corresponding to the target word; or to obtain a configuration parameter as a penalty factor adjustment parameter corresponding to the target word.
  • the diversity evaluation index calculation module 17 is configured to remove the original target text from the new target text to obtain a calculated text; the original target text is a target text composed of the topic sentence in the text outline and the paragraphs taken from the content knowledge base; and the diversity evaluation index is calculated based on the calculated text.
  • the penalty factor updating module 18 is configured to calculate the index difference between the diversity evaluation index and the diversity evaluation index obtained in the previous iteration; and update the penalty factor based on the index difference.
  • the device also includes a subject offset correction unit configured to:
  • both the first semantic similarity and the second semantic similarity are greater than a preset similarity threshold, the sentence is retained; otherwise, the sentence and the text following the sentence are deleted to obtain a deleted text;
  • the deleted text is used as a candidate paragraph corresponding to the topic sentence; otherwise, a candidate paragraph corresponding to the topic sentence is regenerated based on the deleted text.
  • the optimal paragraph determination module 19 is configured to score each candidate paragraph for any topic sentence based on a preset scoring standard, and determine the candidate paragraph with the highest score as the optimal paragraph corresponding to the topic sentence.
  • the device also includes a detection and modification module, which is configured to perform error detection and modification on the optimal paragraph using a preset detection and modification logic after determining the optimal paragraph of the topic sentence based on the candidate paragraphs corresponding to any topic sentence.
  • an embodiment of the present application discloses an electronic device 20, including a processor 21 and a memory 22; wherein the memory 22 is configured to store a computer program; the processor 21 is configured to execute the computer program, and the intelligent writing method disclosed in the above embodiment.
  • the memory 22, as a carrier for storing resources may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the storage method may be temporary storage or permanent storage.
  • the electronic device 20 also includes a power supply 23, a communication interface 24, an input/output interface 25 and a communication bus 26; wherein the power supply 23 is configured to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not limited here; the input/output interface 25 is configured to obtain external input data or output data to the outside world, and its interface type can be selected according to application needs and is not limited here.
  • an embodiment of the present application further discloses a computer non-volatile readable storage medium, which is configured to store a computer program, wherein the computer program, when executed by a processor, implements the intelligent writing method disclosed in the aforementioned embodiment.
  • each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments.
  • the same or similar parts between the embodiments can be referred to each other.
  • the description is relatively simple, and the relevant parts can be referred to the method part.
  • the steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two.
  • the software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of non-volatile readable storage medium known in the art.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Machine Translation (AREA)

Abstract

本申请公开了一种智能写作方法、装置、设备及非易失性可读存储介质，包括：生成文章大纲；将文章大纲中的任意主题句与从内容知识库中取出的与该主题句相关的段落组成目标文本；将目标文本输入预训练模型，得到字典中每个词为生成词的概率；基于惩罚因子对字典中目标词对应的概率进行惩罚，基于概率从字典中取出生成词；基于生成词更新目标文本；根据新的目标文本计算多样性评价指标，并基于该多样性评价指标更新惩罚因子，并将新的目标文本输入预训练模型，不断迭代，直到取出的生成词为截止符，则将当前的目标文本确定为候选段落；基于任意主题句的候选段落确定最优段落。能够在解决文本重复问题的情况下，减少人工成本并提升参数调节的准确度。

Description

一种智能写作方法、装置、设备及非易失性可读存储介质
相关申请的交叉引用
本申请要求于2022年12月23日提交中国专利局，申请号为202211660381.7，申请名称为“一种智能写作方法、装置、设备及介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及智能写作技术领域，特别涉及一种智能写作方法、装置、设备及非易失性可读存储介质。
背景技术
在日常办公等写作领域，资料的查找、收集、分类、引用都较为繁琐，耗费大量的人力和时间。智能写作能满足这一领域对于劳动力解放的需求。近年来，智能写作已经从规则、模板写作发展到了以深度神经网络模型为核心的智能化创作，从辅助创作逐渐走向自动化，新的预训练语言模型被不断提出，如openAI(Open Artificial Intelligence，开放人工智能)的GPT(即Generative Pre-Training，生成式预训练)-3、ERNIE(Enhanced Representation through kNowledge IntEgration，基于知识增强的表示学习)、T5等模型，技术人员试图利用这些大模型的原始生成能力来解决不同场景下文本生成问题。
目前，基于预训练模型生成短文本的能力尚可，但是生成长文本(如长段落或篇章)的能力欠缺，比较常见的是文本重复问题，现有的解决文本重复的问题通常是人工进行参数调节，这种方式比较耗时，人工成本较高，难以保障准确性。
发明内容
有鉴于此，本申请的目的在于提供一种智能写作方法、装置、设备及非易失性可读存储介质，能够在解决文本重复问题的情况下，减少人工成本并提升参数调节的准确度。其可选方案如下：
根据第一方面，本申请公开了一种智能写作方法，包括：
生成文章大纲；文章大纲包括所有段落的主题句；
将文章大纲中的任意主题句与从内容知识库中取出的与该主题句相关的段落组成目标文本；
将目标文本输入预训练模型，得到字典中每个词为生成词的概率；
基于惩罚因子对字典中目标词对应的概率进行惩罚，基于概率从字典中取出生成词；其中，目标词为在目标文本中存在的词；
基于生成词更新目标文本，得到新的目标文本；
根据新的目标文本计算多样性评价指标，并基于该多样性评价指标更新惩罚因子，并将新的目标文本输入预训练模型，不断迭代，直到取出的生成词为截止符，则将当前的目标文本确定为该主题句对应的候选段落；
基于任意主题句对应的候选段落确定该主题句的最优段落。
可选的，生成文章大纲，包括：
基于用户选择的写作类型、输入的写作主题内容以及第一关键词生成文章大纲。
可选的，基于用户选择的写作类型、输入的写作主题内容以及第一关键词生成文章大纲，包括：
从用户选择的写作类型对应的内容知识库中检索与写作主题内容最相关的模板文件；
将模板文件中每个段落输入预训练模型，生成每个段落的主题句以及第二关键词；
分别将每个段落的主题句以及第二关键词、第一关键词按照预设方式组成模型输入数据，并将模型输入数据输入预训练模型，生成每个段落对应的新的主题句；
将每个段落的新的主题句组合，得到文章大纲。
可选的，从用户选择的写作类型对应的内容知识库中检索与写作主题内容最相关的模板文件，包括：
从用户选择的写作类型对应文件名知识库中检索与写作主题内容最相关的文件名；
从用户选择的写作类型对应的内容知识库中查找该文件名对应的模板文件。
可选的，还包括：
创建知识库池；其中，知识库池中包括各写作类型对应的文件名知识库和内容知识库；内容知识库包括各模板文件，文件名知识库包括各模板文件的文件名。
可选的，基于惩罚因子对字典中目标词对应的概率进行惩罚，包括：
基于惩罚因子调整参数以及惩罚因子对字典中目标词对应的概率进行惩罚。
可选的，还包括：
统计目标词在目标文本中出现的次数，得到该目标词对应的惩罚因子调整参数；
或，获取配置参数，作为目标词对应的惩罚因子调整参数。
可选的，基于惩罚因子调整参数以及惩罚因子对字典中目标词对应的概率进行惩罚，包括：
根据公式对字典中目标词对应的概率进行惩罚，其中，xi为目标词，Iin为目标文本，在目标词在目标文本中的情况下，对目标词对应的概率进行惩罚，n用于指示目标词在目标文本中出现的次数或者常数1，P为惩罚因子。
可选的，基于概率从字典中取出生成词，包括：
基于解码规则从字典中基于概率选取生成词，其中，概率为进行了惩罚后的概率，解码规则包括温度Temperature采样、TopK采样策略和TopP采样策略。
可选的，根据新的目标文本计算多样性评价指标，包括：
从新的目标文本中去除原始目标文本，得到计算文本；原始目标文本为文本大纲中的主题句与从内容知识库中取出的段落组成的目标文本；
根据计算文本计算多样性评价指标。
可选的，基于该多样性评价指标更新惩罚因子，包括：
计算该多样性评价指标以及上一次迭代得到的多样性评价指标之间的指标差值；
基于指标差值更新惩罚因子。
可选的，基于指标差值更新惩罚因子，包括：
根据公式P＝max(1，P+Δd)更新惩罚因子，其中，P为更新后的惩罚因子，Δd为多样性评价指标以及上一次迭代得到的多样性评价指标之间的指标差值。
可选的，在基于任意主题句对应的候选段落确定该主题句的最优段落之前，还包括：
计算任意主题句对应的候选段落中每句话与上一句话以及该主题句之间的语义相似度，得到第一语义相似度和第二语义相似度；
若第一语义相似度和第二语义相似度均大于预设相似度阈值，则保留该句话，否则删除该句话以及该句话后面的文本，得到删除后文本；
若删除后文本的字数大于预设字数阈值，则将该删除后文本作为该主题句对应的候选段落，否则基于删除后文本重新生成该主题句对应的候选段落。
可选的，计算任意主题句对应的候选段落中每句话与上一句话以及该主题句之间的语义相似度，得到第一语义相似度和第二语义相似度，包括：
对句子进行分词，其中，句子包括：每句话，上一句话和主题句；
获取每个句子分词后的每一个词对应的向量，并将获取到的向量相加并求平均，得到每个句子对应的句子向量；
利用公式计算每句话与上一句话之间的夹角余弦值作为第一语义相似度，并利用公式计算每句话与主题句之间的夹角余弦值作为第二语义相似度，其中，θ为向量x和向量y之间的夹角，xi表示向量x的分量，yi表示向量y的分量。
可选的，基于任意主题句对应的候选段落确定该主题句的最优段落，包括：
对于任意主题句，基于预设评分标准分别对每个候选段落打分，并将分数最高的候选段落确定为该主题句对应的最优段落。
可选的，基于预设评分标准分别对每个候选段落打分，包括：
根据公式计算每个候选段落的分值ScorePPL，其中，P(xi|x1，x2，…，xi-1)表示根据上文词语预测第i个词的概率，PPL表示语言模型中的困惑度perplexity，N表示句子长度，ScorePPL的值越小表示生成的文本越自然、语句越通顺。
可选的，基于任意主题句对应的候选段落确定该主题句的最优段落之后，还包括：
利用预设检测和修改逻辑对最优段落进行错误检测和修改。
根据第二方面，本申请公开了一种智能写作装置，包括：
文章大纲生成模块，被设置为生成文章大纲；文章大纲包括所有段落的主题句；
目标文本构建模块，被设置为将文章大纲中的任意主题句与从内容知识库中取出的与该主题句相关的段落组成目标文本；
生成词概率获取模块，被设置为将目标文本输入预训练模型，得到字典中每个词为生成词的概率；
目标词概率惩罚模块，被设置为基于惩罚因子对字典中目标词对应的概率进行惩罚；其中，目标词为在目标文本中存在的词；
生成词取出模块，被设置为基于概率从字典中取出生成词；
目标文本更新模块，被设置为基于生成词更新目标文本，得到新的目标文本；
多样性评价指标计算模块，被设置为根据新的目标文本计算多样性评价指标；
惩罚因子更新模块，被设置为基于该多样性评价指标更新惩罚因子，相应的，装置还被设置为触发生成词概率获取模块将新的目标文本输入预训练模型，不断迭代，直到生成词取出模块取出的生成词为截止符，则将当前的目标文本确定为该主题句对应的候选段落；
最优段落确定模块，被设置为基于任意主题句对应的候选段落确定该主题句的最优段落。
根据第三方面，本申请公开了一种电子设备，包括存储器和处理器，其中：
存储器，被设置为保存计算机程序；
处理器，被设置为执行计算机程序，以实现前述的智能写作方法。
根据第四方面，本申请公开了一种计算机非易失性可读存储介质，被设置为保存计算机程序，其中，计算机程序被处理器执行时实现前述的智能写作方法。
可见，本申请先生成文章大纲，文章大纲包括所有段落的主题句，之后将文章大纲中的任意主题句与从内容知识库中取出的与该主题句相关的段落组成目标文本，并将目标文本输入预训练模型，得到字典中每个词为生成词的概率，基于惩罚因子对字典中目标词对应的概率进行惩罚，基于概率从字典中取出生成词，目标词为在目标文本中存在的词，基于生成词更新目标文本，得到新的目标文本，根据新的目标文本计算多样性评价指标，并基于该多样性评价指标更新惩罚因子，并将新的目标文本输入预训练模型，不断迭代，直到取出的生成词为截止符，则将当前的目标文本确定为该主题句对应的候选段落，之后基于任意主题句对应的候选段落确定该主题句的最优段落。也即，本申请先生成文章大纲，对于文章大纲中的任意主题句，生成候选段落并从候选段落中选出最佳段落，在生成候选段落的过程中，利用多样性评价指标指导重复度的惩罚因子自适应的调整，能够在解决文本重复问题的情况下，减少人工成本并提升参数调节的准确度。
附图说明
或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本申请公开的一种智能写作方法流程图；
图2为本申请公开的一种可选的知识库池示意图；
图3为本申请公开的一种可选的最优段落生成示意图；
图4为本申请公开的一种可选的候选段落生成示意图；
图5为本申请公开的一种智能写作装置结构示意图；
图6为本申请公开的一种电子设备结构图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
随着自然语言处理技术的不断发展，人工智能从“大炼模型”逐步迈向了“炼大模型”的阶段，利用先进的算法，整合大规模的数据，汇聚大量算力，训练出巨量人工智能模型，如GPT(Generative Pre-Training，生成式预训练)-3、ERNIE(Enhanced Representation through kNowledge IntEgration，基于知识增强的表示学习)、T5等。这些预训练模型在不同的应用场景下都具有较好原始生成能力。其中，GPT3是一种自回归生成式的预训练语言模型，可以根据给定的文本信息来生成具有一定语义相关度的文本信息，例如回答问题，续写文章等。GPT主要利用了Transformer(即转换器)的解码器部分来进行生成式的单向语言模型建模，根据已经出现的若干词语来预测当前输出词的概率分布。
目前，基于预训练模型生成短文本的能力尚可，但是生成长文本(如长段落或篇章)的能力欠缺，比较常见的是文本重复问题，现有的解决文本重复的问题通常是人工进行参数调节，这种方式比较耗时，人工成本较高，难以保障准确性。为此，本申请提供了一种智能写作方案，能够在解决文本重复问题的情况下，减少人工成本并提升参数调节的准确度。
参见图1所示，本申请实施例公开了一种智能写作方法，包括：
步骤S11：生成文章大纲；文章大纲包括所有段落的主题句。
在一种实施方式中，可以基于用户选择的写作类型、输入的写作主题内容以及第一关键词生成文章大纲。可选的，可以从用户选择的写作类型对应的内容知识库中检索与写作主题内容最相关的模板文件；将模板文件中每个段落输入预训练模型，生成每个段落的主题句以及第二关键词；分别将每个段落的主题句以及第二关键词、第一关键词按照预设方式组成模型输入数据，并将模型输入数据输入预训练模型，生成每个段落对应的新的主题句；将每个段落的新的主题句组合，得到文章大纲。可以理解的是，模型输入数据为prompt(即提示)。
其中，本申请实施例可以从用户选择的写作类型对应文件名知识库中检索与写作主题内容最相关的文件名；从用户选择的写作类型对应的内容知识库中查找该文件名对应的模板文件。在一种实施方式中，可以利用bm25算法从用户选择的写作类型对应文件名知识库中检索与写作主题内容最相关的文件名。
并且，本申请实施例可以创建知识库池；其中，知识库池中包括各写作类型对应的文件名知识库和内容知识库；内容知识库包括各模板文件，文件名知识库包括各模板文件的文件名。模板文件可以为txt文件。
本申请实施例可以根据用户提供或通过网络获取的最新参考材料构建多场景知识库池，这样，利用最新参考资料构建知识库，检索相关知识，并与主题句组成prompt，输入预训练模型生成文本，使预训练模型学会最新知识并能对其进行分析，解决预训练模型不能跟进最新动态发展的问题。例如，参见图2所示，图2为本申请实施例公开的一种可选的知识库池示意图。本根据不同的应用场景对知识库分类构建，可以包括公文写作、新闻资讯、广告营销等多个写作类型，每个写作类型包括txt(一种用于存储文本数据的文本文件)文件名知识库和txt内容知识库；每个txt的文件名与其内容的标题一致，txt文件名知识库就是由所有标题构建的；txt内容知识库 由所有txt文件内容组成，每个txt是一篇最新参考文章，文章中的一段占txt中的一行。
可选的，自动生成文章大纲的实现流程可以为：输入用户选择的写作类型、想写作的主题内容以及关键词Knew(即第一关键词)。根据主题内容在写作类型下的txt文件名知识库中检索最相关的一条，根据检索到的文件名在txt内容知识库中找到对应的模板文件。(本申请实施例可以通过BM250kapi类中get_top_n函数实现)，之后将模板文件按段输入预训练模型，生成每段的主题句及关键词Kold(即第二关键词)。分别将旧关键词Kold、每段主题句与新关键词Knew按一定方式组合为prompt，输入预训练模型，生成新的主题句。最后，将所有段落主题句组合在一起作为文章大纲。例如，组成prompt的方法为：
以“旧关键词”为关键词的主题句：“旧主题句”。以“新关键词”为关键词的主题句：
需要指出的是，bm25算法是一种用来评价搜索词和文档之间相关性的算法，它是一种基于概率检索模型提出的算法，例如，有一个query(询问)和一批文档Ds(Dataset，数据集)，现在要计算query和每篇文档D之间的相关性分数，先对query进行切分，得到单词，然后单词的分数由3部分组成：单词和D之间的相关性；单词和query之间的相关性；每个单词的权重。最后对于每个单词的分数做一个求和，就得到了query和文档之间的分数。
步骤S12：将文章大纲中的任意主题句与从内容知识库中取出的与该主题句相关的段落组成目标文本。
也即，本申请实施例对文章大纲中的每个主题句分别处理，分别得到每个主题句对应的目标文本，以得到每个主题句的最佳段落。
在可选的实施方式中，将文章大纲中的任意主题句与从用户选择的写作类型对应的内容知识库中取出的与该主题句相关的段落组成目标文本。可以理解的是，内容知识库为预先创建的保存模板文件的知识库。并且，可以检索出最相关的n个段落，n为大于等于1的正整数，比如1、2、3、4、5等。n个段落中每个与该主题句相关的段落分别与该主题句组成目标文本，以生成每个相关的段落对应的候选段落，得到n个候选段落。采用的检索方法可以为bm25算法。
步骤S13：将目标文本输入预训练模型，得到字典中每个词为生成词的概率。
步骤S14：基于惩罚因子对字典中目标词对应的概率进行惩罚，基于概率从字典中取出生成词；其中，目标词为在目标文本中存在的词。
在可选的实施方式中，可以基于惩罚因子调整参数以及惩罚因子对字典中目标词对应的概率进行惩罚。并且，在一种实施方式中，可以统计目标词在目标文本中出现的次数，得到该目标词对应的惩罚因子调整参数，在另一种实施方式中，可以获取配置参数，作为目标词对应的惩罚因子调整参数。也即，惩罚因子调整参数可以为目标词在目标文本中出现的次数，也可以配置的常数。
并且，可以采用解码规则(比如Temperature(温度)、TopK、TopP采样策略)从字典中基于概率选取生成词。选取时依据的概率为惩罚后的概率。
需要指出的是，重复惩罚的目的在于解决重复问题，通过惩罚因子将出现过词的概率变小或者强制不使用重复词来解决。合理设置惩罚因子能够避免重复，增加创新性，但是设置过大，生成长文本时起反作用。可以基于词出现的次数进行惩罚(frequency penalty)，出现的次数越多，该词在后文出现的概率越低，同时增强后文的创造性。也可以基于词是否出现过进行惩罚(response penalty)，对出现过的词进行惩罚，降低其在后文出现的概率，同时增强后文的创造性。与frequency penalty原理一致，只是多了对已输出token(词)去重，只保留出现一次。也可以重复词去除，大于等于1时起作用，表示输出中不包含长度为no_repeat_ngram_size的重复词。
可选的，为了解决搜索生成缺乏多样性问题，可以通过采样策略来增加随机性。其中，Temperature采样：通过强化顶部词的概率，只对最有可能的一些词进行采样，这样就能够在增加随机性的同时，又保证不出现一般性的错误。强化顶部词概率，可以通过对模型输出的logits除以一个小于1的温度(Temperature，T)。
f(i)表示模型输出第i个词的初始概率，表示对词表中所有词都计算求和，p(i)表示强化后第i个词的概率。这样通过softmax(用于将一组实数转化为概率分布)后使得分布更加尖锐，大概率的词概率更大。之后根据获得概率对顶部词先进行挑选，然后再采样，直接杜绝了低概率词出现的可能性。
TopK采样：挑选概率最高k个token，然后重新过softmax算概率，之后根据获得概率进行采样，接着进行下一步生成，不断重复。但关于TopK有可能会出现一个问题，假如模型对当前生成非常肯定，比如概率最高的token的概率0.9，而其余的token概率都很低。这时如果只用topk采样的话，就会导致采样到低概率情况仍然发生。因此需要对顶部token的累计概率进行限制，这就是TopP采样。
TopP采样：先设置一个概率界限，比如说p＝0.9，然后从最大概率的token往下开始取，同时将概率累加起来，当取到大于等于p也就是0.9时停止。如果最大token概率就已经有0.9了，那么就只取最大的一个token。
步骤S15：基于生成词更新目标文本，得到新的目标文本。
步骤S16：根据新的目标文本计算多样性评价指标，并基于该多样性评价指标更新惩罚因子，并将新的目标文本输入预训练模型，不断迭代，直到取出的生成词为截止符，则将当前的目标文本确定为该主题句对应的候选段落。
在可选的实施方式中，可以从新的目标文本中去除原始目标文本，得到计算文本；原始目标文本为文本大纲中的主题句与从内容知识库中取出的段落组成的目标文本；根据计算文本计算多样性评价指标。
并且，可以计算该多样性评价指标以及上一次迭代得到的多样性评价指标之间的指标差值；基于指标差值更新惩罚因子。
需要指出的是，在NLP(即Natural Language Processing，自然语言处理)领域中，多样性评价指标(Distinct)用于判断机器回复的多样性，Distinct指标判断是否出现大量的通用性、重复性回复。Distinct的定义如下：Distinct(n)＝(Count(unique ngram))/Count(word)，其中，Count(unique ngram)表示回复中不重复的ngram数量，Count(word)表示回复中ngram词语的总数量。Distinct(n)越大表示生成回复的多样性越高。本申请实施例利用多样性评价指标调节惩罚因子。
可选的，本申请实施例可以计算任意主题句对应的候选段落中每句话与上一句话以及该主题句之间的语义相似度，得到第一语义相似度和第二语义相似度；若第一语义相似度和第二语义相似度均大于预设相似度阈值，则保留该句话，否则删除该句话以及该句话后面的文本，得到删除后文本；若删除后文本的字数大于预设字数阈值，则将该删除后文本作为该主题句对应的候选段落，否则基于删除后文本重新生成该主题句对应的候选段落。可以理解的是，若候选段落为多个，分别执行前述步骤。也即，本申请实施例利用语义相似度检测并删除偏离主题的文本，重新生成符合条件的候选段落。
其中，语义相似度的计算流程为：首先对句子进行分词，然后对分好的每一个词获取其对应的Vector(向量)，然后将所有Vector相加并求平均，得到句子Vector，最后再利用如下公式计算其夹角余弦值即可。余弦值越接近1(即夹角越小)表示相似度越高。
其中，θ为向量x和y之间的夹角，xi和yi分别代表向量x和y的各分量，可以利用word2vec(产生词向量的模型)模型计算Vector，是将所有的词向量化，这样词与词之间就可以定量的去度量他们之间的关系，挖掘词之间的联系。
步骤S17：基于任意主题句对应的候选段落确定该主题句的最优段落。
在可选的实施方式中，对于任意主题句，基于预设评分标准分别对每个候选段落打分，并将分数最高的候选段落确定为该主题句对应的最优段落。
在一种实施方式中，本申请实施例可以采用PPL指标分别对每个候选段落打分。PPL指的是语言模型中的perplexity(即困惑度)，是衡量一句话是否通顺的指标。定义为：
其中，P(xi|x1，x2，…，xi-1)表示根据上文词语预测第i个词的概率，N代表句子长度。PPL值越小，说明模型生成的文本越自然、语句越通顺。通过PPL来评价文本质量，可以避免模型生成的文本有乱序、前后颠倒的情形。
可选的，本申请实施例可以利用预设检测和修改逻辑对最优段落进行错误检测和修改。
例如，参见图3所示，图3为本申请实施例公开的一种可选的最优段落生成示意图。包括以下步骤：
(a)输入用户选择的写作类型、所有段落的主题句、所有段落主题句的数目N、i的初始值1；
(b)判断i＜N+1，若i＜N+1则获取当前第i个主题句，若i大于或者等于N+1转到步骤(g)；
(c)采用bm25算法在写作类型对应的txt内容知识库中检索topn(即最相关的n个)相关段落；
(d)将topn相关段落分别与主题句拼接组成prompt；
(e)将n个prompt输入预训练模型生成n个符合主题的备选段落；
(f)分别对n个备选段落基于PPL指标(1/ScorePPL)进行打分，选择最高分对应段落作为最佳段落输出至结果列表中，i+＝1，转至步骤(b)；
(g)结果列表输出保存成txt文件。最终，结果列表中包括文章大纲中所有主题句对应的最佳段落。
可选的，前述步骤(e)中主题句+相关段落组成prompt输入预训练模型，生成文本容易出现重复问题，包括生成内容本身重复和与输入重复两方面。本申请实施例采用惩罚参数自适应调整来解决该问题。参见图4所示，图4为本申请实施例公开的一种可选的候选段落生成示意图。包括以下步骤：
(1)输入文本(即目标文本)Iin，初始值为相关段落+主题句组成的prompt I0；
(2)文本输入预训练模型，输出字典中所有词的概率；
(3)对输入文本对应词进行惩罚，采用公式如下：
其中，xi表示字典中任意一个词，若该词在输入文本Iin中，则对该词对应的概率进行惩罚，n表示xi在输入文本中出现的次数或者常数1均可(根据不同场景设置)，P是惩罚因子，初始值为1；
(4)根据解码规则(可以为Temperature、TopK、TopP采样策略)从字典中选取生成词tnew；
(5)更新输入文本Iin+＝tnew，即将新生成词拼接在原输入文本后即可；根据新生成文本(即Iin中去掉初始值I0)计算distinct(即多样性)指标，dnew＝(Distinct(m)+Distinct(m+1))/2，其中，m表示ngram的大小，需要根据不同场景设置不同值；
(6)计算Δd，Δd＝dpre-dnew，其中，dpre初始值为1，dpre＝dnew；
(7)更新惩罚因子P，P＝max(1，P+Δd)；
(8)继续步骤(2)直到生成截止符<eod>。
需要指出的是，其它的如Temperature、TopK、TopP等参数也可参考惩罚因子的调整方案采用自适应调整策略。
可选的，步骤(e)还需要解决长文本主题偏移问题，分别对所有备选段落进行如下操作：(A)获取一个备选答案；(B)利用训练好的Word2vec模型计算候选答案中的每句话与主题句以及该句话前一句的语义相似度；(C)若两个相似度均大于设定阈值S则保留，否则删除该句及后面的文本；(D)统计备选段落字数，若大于设定阈值T则保留输出答案，否则基于剩余文本重新输入预训练模型生成，继续步骤(B)。
并且，智能写作生成的文章可能包含错别字、敏感词以及语法错误等，本申请实施例可以检测并修改这些错误，最后人工对文章审核、发布。
需要指出的是，现有技术中，直接使用预训练语言模型不能跟进最新动态发展，由于预训练模型参数量巨大，即使finetune(微调)需要的数据和算力成本也会比较高。并且，基于预训练模型生成短文本(如一句话)的能力尚可，但是生成长文本(如长段落或篇章)的能力欠缺，最常见的是主题偏移和文本重复问题。而本申请利用最新参考资料构建知识库，检索相关知识，并与主题句组成prompt，输入预训练模型生成文本，通过该方法使预训练模型学会最新知识并能对其进行分析；利用文本多样性评价指标指导重复度惩罚参数自适应调整，节约人工调参的成本；利用语义相似度检测并删除偏离主题的文本，重新生成符合条件的候选段落。
下面，以公文写作为例，阐述本申请提供的智能写作方案：
首先，构建公文写作知识库，以GPT3为例进行说明，GPT3于2020年发布，那么训练它的数据集是收集2020年及以前的，那么要让GPT3基于2021年到现在的最新动态生成文章，需要将相关资料其构建知识库。用户想通过智能写作自动撰写某公司的某个经济专项规划，收集最新材料。将各种形式的材料(PDF、Word或图片)转换成txt文本，并构建公文写作的文件名知识库和内容知识库。数据库的构建方法可以采用Python软件包rank_bm25中BM250kapi类实现。
可选的，自动生成文章大纲：用户选择公文写作类型，输入的想写作的主题内容：请根据数字经济发展规划材料，撰写XX公司经济专项规划。通过BM25算法根据主题内容在文件名知识库中检索最相关的一条文件名，将对应的txt文件按段输入预训练模型，生成每段的主题句。然后分别将每段主题句与关键字组合prompt输入预训练模型，生成新的主题句。
Prompt输入举例：以“智慧交通”为关键词的主题句：推进智慧交通建设。以“云中心”为关键词的主题句：
预训练模型生成：推进云中心建设。
由于自动生成的文章大纲整体与检索模板一致，而实际应用中需要的大纲与模板还是有一定差距，需要人工干预修改。
然后生成最佳段落：假设某段主题为：A。可以采用bm25算法在文本知识库中检索获取得分最高的三个段 落：top1、top2、top3。分别将top1、top2、top3与主题句拼接组成prompt，输入预训练模型，输出3个候选答案(即候选段落)。采用PPL指标作为评分标准，选取最高分者作为最优答案。在生成候选答案的过程中，采用惩罚参数自适应调整方法解决生成重复问题。其主要原理为当生成内容重复度升高时，增大惩罚因子，即加大模型对已生成词的惩罚力度，模型生成偏向未出现过的词。当生成内容重复度降低时，减小惩罚因子，即减小模型对已生成词的惩罚力度，防止模型生成因惩罚力度过大导致偏离主题。在生成候选答案的过程中，利用训练好的Word2vec模型计算答案中的每句话与主题句的语义相似度，删除跑题语句，并补充符合语义的语句。
可选的，人工审核、发布，使用GitHub开源项目pycorrector(文本纠错开源工具)作为后处理模块，实现对生成的文章的错别字、敏感词、语法错误检测和修改；
需要注意的是，本实施例以公文写作为例进行说明，但在实际应用中不仅局限于此，其它诸如新闻资讯、故事续写等领域的写作均可以此方式进行改进。
这样，为了解决基于预训练模型生成的文本内容缺乏对最新动态的分析问题，本申请基于最新参考资料构建知识库，通过检索算法检索与主题句相关知识，并与主题句组成prompt，输入预训练模型生成文本，通过此方案使模型学习与主题相关的最新知识。为了解决预训练模型生成文本容易重复但人工调参难问题，利用文本多样性评价指标指导重复度惩罚参数自适应调整，增大生成内容的多样性，减少人工调参的耗时费力，保证生成的文本既不重复又不会矫枉过正。为了解决预训练模型生成长文本易偏离主题问题，本申请将生成的每句话分别与主题句、前一句话进行语义相似度匹配，删除偏离主题语句，生成符合条件答案。可选的，还细化了智能写作的整体实现方案及***，提高智能写作的自动化程度。
参见图5所示，本申请实施例提供了一种智能写作装置，包括：
文章大纲生成模块11，被设置为生成文章大纲；文章大纲包括所有段落的主题句；
目标文本构建模块12，被设置为将文章大纲中的任意主题句与从内容知识库中取出的与该主题句相关的段落组成目标文本；
生成词概率获取模块13，被设置为将目标文本输入预训练模型，得到字典中每个词为生成词的概率；
目标词概率惩罚模块14，被设置为基于惩罚因子对字典中目标词对应的概率进行惩罚；其中，目标词为在目标文本中存在的词；
生成词取出模块15，被设置为基于概率从字典中取出生成词；
目标文本更新模块16，被设置为基于生成词更新目标文本，得到新的目标文本；
多样性评价指标计算模块17，被设置为根据新的目标文本计算多样性评价指标；
惩罚因子更新模块18，被设置为基于该多样性评价指标更新惩罚因子，相应的，装置还被设置为触发生成词概率获取模块将新的目标文本输入预训练模型，不断迭代，直到生成词取出模块取出的生成词为截止符，则将当前的目标文本确定为该主题句对应的候选段落；
最优段落确定模块19，被设置为基于任意主题句对应的候选段落确定该主题句的最优段落。
可见，本申请实施例先生成文章大纲，文章大纲包括所有段落的主题句，之后将文章大纲中的任意主题句与从内容知识库中取出的与该主题句相关的段落组成目标文本，并将目标文本输入预训练模型，得到字典中每个词为生成词的概率，基于惩罚因子对字典中目标词对应的概率进行惩罚，基于概率从字典中取出生成词，目标词为在目标文本中存在的词，基于生成词更新目标文本，得到新的目标文本，根据新的目标文本计算多样性评价指标，并基于该多样性评价指标更新惩罚因子，并将新的目标文本输入预训练模型，不断迭代，直到取出的生成词 为截止符，则将当前的目标文本确定为该主题句对应的候选段落，之后基于任意主题句对应的候选段落确定该主题句的最优段落。也即，本申请实施例先生成文章大纲，对于文章大纲中的任意主题句，生成候选段落并从候选段落中选出最佳段落，在生成候选段落的过程中，利用多样性评价指标指导重复度的惩罚因子自适应的调整，能够在解决文本重复问题的情况下，减少人工成本并提升参数调节的准确度。
文章大纲生成模块11，被设置为基于用户选择的写作类型、输入的写作主题内容以及第一关键词生成文章大纲。
可选的，文章大纲生成模块11，被设置为包括：
模板文件检索单元，被设置为从用户选择的写作类型对应的内容知识库中检索与写作主题内容最相关的模板文件；
模型生成单元，被设置为将模板文件中每个段落输入预训练模型，生成每个段落的主题句以及第二关键词；
新主题句生成单元，被设置为分别将每个段落的主题句以及第二关键词、第一关键词按照预设方式组成模型输入数据，并将模型输入数据输入预训练模型，生成每个段落对应的新的主题句；
文章大纲生成单元，被设置为将每个段落的新的主题句组合，得到文章大纲。
其中，模板文件检索单元，被设置为从用户选择的写作类型对应文件名知识库中检索与写作主题内容最相关的文件名；从用户选择的写作类型对应的内容知识库中查找该文件名对应的模板文件。
可选的，装置还包括：
知识库池创建模块，被设置为创建知识库池；其中，知识库池中包括各写作类型对应的文件名知识库和内容知识库；内容知识库包括各模板文件，文件名知识库包括各模板文件的文件名。
其中，目标词概率惩罚模块14，被设置为基于惩罚因子调整参数以及惩罚因子对字典中目标词对应的概率进行惩罚。
目标词概率惩罚模块14，还被设置为统计目标词在目标文本中出现的次数，得到该目标词对应的惩罚因子调整参数；或，获取配置参数，作为目标词对应的惩罚因子调整参数。
多样性评价指标计算模块17，被设置为从新的目标文本中去除原始目标文本，得到计算文本；原始目标文本为文本大纲中的主题句与从内容知识库中取出的段落组成的目标文本；根据计算文本计算多样性评价指标。
惩罚因子更新模块18，被设置为计算该多样性评价指标以及上一次迭代得到的多样性评价指标之间的指标差值；基于指标差值更新惩罚因子。
装置，还包括主题偏移纠正单元，被设置为：
计算任意主题句对应的候选段落中每句话与上一句话以及该主题句之间的语义相似度，得到第一语义相似度和第二语义相似度；
若第一语义相似度和第二语义相似度均大于预设相似度阈值，则保留该句话，否则删除该句话以及该句话后面的文本，得到删除后文本；
若删除后文本的字数大于预设字数阈值，则将该删除后文本作为该主题句对应的候选段落，否则基于删除后文本重新生成该主题句对应的候选段落。
最优段落确定模块19，被设置为对于任意主题句，基于预设评分标准分别对每个候选段落打分，并将分数最高的候选段落确定为该主题句对应的最优段落。
装置还包括检测和修改模块，被设置为在基于任意主题句对应的候选段落确定该主题句的最优段落之后，利用预设检测和修改逻辑对最优段落进行错误检测和修改。
参见图6所示，本申请实施例公开了一种电子设备20，包括处理器21和存储器22；其中，存储器22，被设置为保存计算机程序；处理器21，被设置为执行计算机程序，前述实施例公开的智能写作方法。
关于上述智能写作方法的过程可以参考前述实施例中公开的相应内容，在此不再进行赘述。
并且，存储器22作为资源存储的载体，可以是只读存储器、随机存储器、磁盘或者光盘等，存储方式可以是短暂存储或者永久存储。
另外，电子设备20还包括电源23、通信接口24、输入输出接口25和通信总线26；其中，电源23被设置为为电子设备20上的各硬件设备提供工作电压；通信接口24能够为电子设备20创建与外界设备之间的数据传输通道，其所遵循的通信协议是能够适用于本申请技术方案的任意通信协议，在此不对其进行限定；输入输出接口25，被设置为获取外界输入数据或向外界输出数据，其接口类型可以根据应用需要进行选取，在此不进行限定。
可选的，本申请实施例还公开了一种计算机非易失性可读存储介质，被设置为保存计算机程序，其中，计算机程序被处理器执行时实现前述实施例公开的智能写作方法。
关于上述智能写作方法的过程可以参考前述实施例中公开的相应内容，在此不再进行赘述。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似部分互相参见即可。对于实施例公开的装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的非易失性可读存储介质中。
以上对本申请所提供的一种智能写作方法、装置、设备及非易失性可读存储介质进行了详细介绍，本文中应用了个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在可选实施方式及应用范围上均会有改变之处，综上，本说明书内容不应理解为对本申请的限制。

Claims (20)

  1. 一种智能写作方法，其特征在于，包括：
    生成文章大纲；所述文章大纲包括所有段落的主题句；
    将所述文章大纲中的任意主题句与从内容知识库中取出的与该主题句相关的段落组成目标文本；
    将目标文本输入预训练模型，得到字典中每个词为生成词的概率；
    基于惩罚因子对所述字典中目标词对应的所述概率进行惩罚，基于所述概率从所述字典中取出生成词；其中，所述目标词为在目标文本中存在的词；
    基于所述生成词更新所述目标文本，得到新的目标文本；
    根据新的目标文本计算多样性评价指标，并基于该多样性评价指标更新所述惩罚因子，并将新的目标文本输入预训练模型，不断迭代，直到取出的生成词为截止符，则将当前的目标文本确定为该主题句对应的候选段落；
    基于任意主题句对应的所述候选段落确定该主题句的最优段落。
  2. 根据权利要求1所述的智能写作方法，其特征在于，所述生成文章大纲，包括：
    基于用户选择的写作类型、输入的写作主题内容以及第一关键词生成文章大纲。
  3. 根据权利要求2所述的智能写作方法，其特征在于，所述基于用户选择的写作类型、输入的写作主题内容以及第一关键词生成文章大纲，包括：
    从用户选择的写作类型对应的内容知识库中检索与写作所述主题内容最相关的模板文件；
    将所述模板文件中每个段落输入预训练模型，生成每个段落的主题句以及第二关键词；
    分别将每个段落的主题句以及第二关键词、所述第一关键词按照预设方式组成模型输入数据，并将模型输入数据输入预训练模型，生成每个段落对应的新的主题句；
    将每个段落的所述新的主题句组合，得到文章大纲。
  4. 根据权利要求3所述的智能写作方法，其特征在于，所述从用户选择的写作类型对应的内容知识库中检索与所述写作主题内容最相关的模板文件，包括：
    从用户选择的写作类型对应文件名知识库中检索与所述写作主题内容最相关的文件名；
    从用户选择的写作类型对应的内容知识库中查找该文件名对应的模板文件。
  5. 根据权利要求3所述的智能写作方法，其特征在于，还包括：
    创建知识库池；其中，所述知识库池中包括各写作类型对应的文件名知识库和内容知识库；所述内容知识库包括各模板文件，所述文件名知识库包括各模板文件的文件名。
  6. 根据权利要求1所述的智能写作方法，其特征在于，所述基于惩罚因子对所述字典中目标词对应的所述概率进行惩罚，包括：
    基于惩罚因子调整参数以及惩罚因子对所述字典中目标词对应的所述概率进行惩罚。
  7. 根据权利要求6所述的智能写作方法，其特征在于，还包括：
    统计目标词在目标文本中出现的次数，得到该目标词对应的所述惩罚因子调整参数；
    或，获取配置参数，作为目标词对应的所述惩罚因子调整参数。
  8. 根据权利要求6所述的智能写作方法，其特征在于，所述基于惩罚因子调整参数以及惩罚因子对所述字典中目标词对应的所述概率进行惩罚，包括：
    根据公式对所述字典中所述目标词对应的所述概率进行惩罚，其中，xi为所述目标词，Iin 为所述目标文本，在所述目标词在所述目标文本中的情况下，对所述目标词对应的概率进行惩罚，n用于指示所述目标词在所述目标文本中出现的次数或者常数1，P为所述惩罚因子。
  9. 根据权利要求1所述的智能写作方法，其特征在于，所述基于所述概率从所述字典中取出生成词，包括：
    基于解码规则从所述字典中基于所述概率选取所述生成词，其中，所述概率为进行了惩罚后的概率，所述解码规则包括温度Temperature采样、TopK采样策略和TopP采样策略。
  10. 根据权利要求1所述的智能写作方法，其特征在于，所述根据新的目标文本计算多样性评价指标，包括：
    从所述新的目标文本中去除原始目标文本，得到计算文本；所述原始目标文本为文本大纲中的主题句与从内容知识库中取出的段落组成的目标文本；
    根据所述计算文本计算多样性评价指标。
  11. 根据权利要求1所述的智能写作方法，其特征在于，所述基于该多样性评价指标更新所述惩罚因子，包括：
    计算该多样性评价指标以及上一次迭代得到的多样性评价指标之间的指标差值；
    基于所述指标差值更新所述惩罚因子。
  12. 根据权利要求11所述的智能写作方法，其特征在于，所述基于所述指标差值更新所述惩罚因子，包括：
    根据公式P＝max(1，P+Δd)更新所述惩罚因子，其中，P为更新后的所述惩罚因子，Δd为多样性评价指标以及上一次迭代得到的多样性评价指标之间的指标差值。
  13. 根据权利要求1所述的智能写作方法，其特征在于，在所述基于任意主题句对应的所述候选段落确定该主题句的最优段落之前，还包括：
    计算任意主题句对应的所述候选段落中每句话与上一句话以及该主题句之间的语义相似度，得到第一语义相似度和第二语义相似度；
    若所述第一语义相似度和第二语义相似度均大于预设相似度阈值，则保留该句话，否则删除该句话以及该句话后面的文本，得到删除后文本；
    若所述删除后文本的字数大于预设字数阈值，则将该删除后文本作为该主题句对应的候选段落，否则基于所述删除后文本重新生成该主题句对应的候选段落。
  14. 根据权利要求13所述的智能写作方法，其特征在于，所述计算任意主题句对应的所述候选段落中每句话与上一句话以及该主题句之间的语义相似度，得到第一语义相似度和第二语义相似度，包括：
    对句子进行分词，其中，所述句子包括：所述每句话，所述上一句话和所述主题句；
    获取每个所述句子分词后的每一个词对应的向量，并将获取到的向量相加并求平均，得到每个所述句子对应的句子向量；
    利用公式计算所述每句话与所述上一句话之间的夹角余弦值作为所述第一语义相似度，并利用公式计算所述每句话与所述主题句之间的夹角余弦值作为所述第二语义相似度，其中，θ为向量x和向量y之间的夹角，xi表示向量x的分量，yi表示向量y的分量。
  15. 根据权利要求1所述的智能写作方法，其特征在于，所述基于任意主题句对应的所述候选段落确定该主题句的最优段落，包括：
    对于任意主题句，基于预设评分标准分别对每个候选段落打分，并将分数最高的候选段落确定为该主题句对应的最优段落。
  16. 根据权利要求15所述的智能写作方法，其特征在于，所述基于预设评分标准分别对每个候选段落打分，包括：
    根据公式计算所述每个候选段落的分值ScorePPL，其中，P(xi|x1，x2，…，xi-1)表示根据上文词语预测第i个词的概率，PPL表示语言模型中的困惑度perplexity，N表示句子长度，ScorePPL的值越小表示生成的文本越自然、语句越通顺。
  17. 根据权利要求1所述的智能写作方法，其特征在于，所述基于任意主题句对应的所述候选段落确定该主题句的最优段落之后，还包括：
    利用预设检测和修改逻辑对所述最优段落进行错误检测和修改。
  18. 一种智能写作装置，其特征在于，包括：
    文章大纲生成模块，被设置为生成文章大纲；所述文章大纲包括所有段落的主题句；
    目标文本构建模块，被设置为将所述文章大纲中的任意主题句与从内容知识库中取出的与该主题句相关的段落组成目标文本；
    生成词概率获取模块，被设置为将目标文本输入预训练模型，得到字典中每个词为生成词的概率；
    目标词概率惩罚模块，被设置为基于惩罚因子对所述字典中目标词对应的所述概率进行惩罚；其中，所述目标词为在目标文本中存在的词；
    生成词取出模块，被设置为基于所述概率从所述字典中取出生成词；
    目标文本更新模块，被设置为基于所述生成词更新所述目标文本，得到新的目标文本；
    多样性评价指标计算模块，被设置为根据新的目标文本计算多样性评价指标；
    惩罚因子更新模块，被设置为基于该多样性评价指标更新所述惩罚因子，相应的，所述装置还被设置为触发生成词概率获取模块将新的目标文本输入预训练模型，不断迭代，直到生成词取出模块取出的生成词为截止符，则将当前的目标文本确定为该主题句对应的候选段落；
    最优段落确定模块，被设置为基于任意主题句对应的所述候选段落确定该主题句的最优段落。
  19. 一种电子设备，其特征在于，包括存储器和处理器，其中：
    所述存储器，被设置为保存计算机程序；
    所述处理器，被设置为执行所述计算机程序，以实现如权利要求1至17任一项所述的智能写作方法。
  20. 一种计算机非易失性可读存储介质，其特征在于，被设置为保存计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求1至17任一项所述的智能写作方法。
PCT/CN2023/114518 2022-12-23 2023-08-23 一种智能写作方法、装置、设备及非易失性可读存储介质 WO2024131111A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211660381.7A CN115630640B (zh) 2022-12-23 2022-12-23 一种智能写作方法、装置、设备及介质
CN202211660381.7 2022-12-23

Publications (1)

Publication Number Publication Date
WO2024131111A1 true WO2024131111A1 (zh) 2024-06-27

Family

ID=84909990

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/114518 WO2024131111A1 (zh) 2022-12-23 2023-08-23 一种智能写作方法、装置、设备及非易失性可读存储介质

Country Status (2)

Country Link
CN (1) CN115630640B (zh)
WO (1) WO2024131111A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115630640B (zh) * 2022-12-23 2023-03-10 苏州浪潮智能科技有限公司 一种智能写作方法、装置、设备及介质
CN115795028B (zh) * 2023-02-09 2023-07-18 山东政通科技发展有限公司 一种公文智能生成方法及***
CN116384388B (zh) * 2023-02-14 2024-02-02 上海熙瑾信息技术有限公司 反向识别ai智能写作的方法、装置、设备和介质
CN116933751B (zh) * 2023-06-30 2024-07-19 北京百度网讯科技有限公司 文章生成方法、装置、电子设备及存储介质
CN116542260B (zh) * 2023-07-05 2023-09-26 中国民用航空飞行学院 一种基于自然语言大模型的翻译文本质量评估方法及***
CN116992834B (zh) * 2023-09-27 2024-01-05 国高信息技术产业研究院(山东)集团有限公司 基于人工智能的辅助写作方法、***、介质及电子设备
CN117633214B (zh) * 2024-01-27 2024-04-19 北京澜舟科技有限公司 一种文章大纲生成方法、装置和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110287489A (zh) * 2019-06-24 2019-09-27 北京大米科技有限公司 文本生成方法、装置、存储介质和电子设备
US20210064823A1 (en) * 2018-05-15 2021-03-04 Beijing Sankuai Online Technology Co., Ltd Article generation
CN113792543A (zh) * 2021-09-14 2021-12-14 安徽咪鼠科技有限公司 一种写作方法、装置及存储介质
CN113919336A (zh) * 2021-10-20 2022-01-11 平安科技（深圳）有限公司 基于深度学习的文章生成方法、装置及相关设备
CN115630640A (zh) * 2022-12-23 2023-01-20 苏州浪潮智能科技有限公司 一种智能写作方法、装置、设备及介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111753508B (zh) * 2020-06-29 2024-07-23 网易（杭州）网络有限公司 文字作品的内容生成方法、装置和电子设备
CN111858931B (zh) * 2020-07-08 2022-05-13 华中师范大学 一种基于深度学习的文本生成方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210064823A1 (en) * 2018-05-15 2021-03-04 Beijing Sankuai Online Technology Co., Ltd Article generation
CN110287489A (zh) * 2019-06-24 2019-09-27 北京大米科技有限公司 文本生成方法、装置、存储介质和电子设备
CN113792543A (zh) * 2021-09-14 2021-12-14 安徽咪鼠科技有限公司 一种写作方法、装置及存储介质
CN113919336A (zh) * 2021-10-20 2022-01-11 平安科技（深圳）有限公司 基于深度学习的文章生成方法、装置及相关设备
CN115630640A (zh) * 2022-12-23 2023-01-20 苏州浪潮智能科技有限公司 一种智能写作方法、装置、设备及介质

Also Published As

Publication number Publication date
CN115630640A (zh) 2023-01-20
CN115630640B (zh) 2023-03-10

Similar Documents

Publication Publication Date Title
WO2024131111A1 (zh) 一种智能写作方法、装置、设备及非易失性可读存储介质
CN110442760B (zh) 一种问答检索***的同义词挖掘方法及装置
CN104636466B (zh) 一种面向开放网页的实体属性抽取方法和***
CN107436864A (zh) 一种基于Word2Vec的中文问答语义相似度计算方法
CN109299280B (zh) 短文本聚类分析方法、装置和终端设备
JP6335898B2 (ja) 製品認識に基づく情報分類
CN108846138B (zh) 一种融合答案信息的问题分类模型构建方法、装置和介质
CN116805001A (zh) 适用于垂直领域的智能问答***、方法及其应用
CN111104803B (zh) 语义理解处理方法、装置、设备及可读存储介质
CN111694927B (zh) 一种基于改进词移距离算法的文档自动评阅方法
CN110134799B (zh) 一种基于bm25算法的文本语料库的搭建和优化方法
CN109829151B (zh) 一种基于分层狄利克雷模型的文本分割方法
CN110750635A (zh) 一种基于联合深度学习模型的法条推荐方法
CN112434164B (zh) 一种兼顾话题发现和情感分析的网络舆情分析方法及***
CN111881264B (zh) 一种开放领域问答任务中长文本检索的方法和电子设备
CN103885933A (zh) 用于评价文本的情感度的方法和设备
CN114970525B (zh) 一种文本同事件识别方法、装置及可读存储介质
CN111090994A (zh) 一种面向中文网络论坛文本的事件地点归属省份识别方法
CN115238040A (zh) 一种钢铁材料学知识图谱构建方法及***
CN114722176A (zh) 一种智能答疑的方法、装置、介质及电子设备
CN117828024A (zh) 一种插件检索方法、装置、存储介质及设备
CN112257442B (zh) 一种基于扩充语料库神经网络的政策文件信息提取方法
CN108241650B (zh) 训练分类标准的训练方法和装置
CN111339258B (zh) 基于知识图谱的大学计算机基础习题推荐方法
CN113076740A (zh) 政务服务领域的同义词挖掘方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23905290

Country of ref document: EP

Kind code of ref document: A1