WO2024114659A1 - 一种摘要生成方法及其相关设备 - Google Patents

一种摘要生成方法及其相关设备 Download PDF

Info

Publication number
WO2024114659A1
WO2024114659A1 PCT/CN2023/134903 CN2023134903W WO2024114659A1 WO 2024114659 A1 WO2024114659 A1 WO 2024114659A1 CN 2023134903 W CN2023134903 W CN 2023134903W WO 2024114659 A1 WO2024114659 A1 WO 2024114659A1
Authority
WO
WIPO (PCT)
Prior art keywords
sentences
model
scores
text
lengths
Prior art date
Application number
PCT/CN2023/134903
Other languages
English (en)
French (fr)
Inventor
介任龙
孟笑君
蒋欣
刘群
钱莉
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2024114659A1 publication Critical patent/WO2024114659A1/zh

Links

Definitions

  • the embodiments of the present application relate to the field of artificial intelligence (AI) technology, and in particular to a summary generation method and related equipment.
  • AI artificial intelligence
  • Automatic summarization technology is a technology that uses computers to automatically analyze text and summarize content to automatically generate summaries, that is, to express the main content of the text in a concise form according to user requirements. Automatic summarization technology can effectively help users find content of interest from retrieved articles and improve reading speed and quality.
  • the neural network model in AI technology can be used to process texts to extract the summary of the text.
  • the neural network model can first map multiple sentences in the text to obtain representations of multiple sentences, and then process the representations of multiple sentences based on the multi-head attention mechanism to obtain scores of multiple sentences. These scores can represent the value of multiple sentences in the text. In this way, several sentences with larger scores can be combined to form a summary of the text.
  • the embodiment of the present application provides a summary generation method and related devices, which can automatically generate a summary of a target text that meets the length limit, and the generated summary of the target text is usually a high-quality summary that can meet the user's reading needs, thereby improving the user experience.
  • a first aspect of an embodiment of the present application provides a method for generating a summary, the method comprising:
  • the target text consists of N sentences, and the lengths of the N sentences are known, and N is a positive integer greater than or equal to 2.
  • the user device can input N sentences of the target text into the first model to score the N sentences of the target text through the first model, thereby obtaining the scores of the N sentences of the target text.
  • the scores of the N sentences are used to indicate the value of the N sentences in the target document.
  • the user device can use the scores of the N sentences of the target text and the lengths of the N sentences of the target text as initial values of the knapsack problem, and solve the knapsack problem.
  • the solution obtained is the M sentences in the N sentences of the target text whose sum of scores is the largest and whose sum of lengths is less than the length threshold of the summary, where M is a positive integer greater than or equal to 1, and M is less than or equal to N.
  • the user device may compose a summary of the target text from the M sentences and display it to the user for browsing and use.
  • the N sentences can be scored by the first model to obtain the scores of the N sentences, and the scores of the N sentences are used to indicate the value of the N sentences in the target text. Then, based on the scores of the N sentences and the lengths of the N sentences, M sentences with the largest sum of scores and a sum of lengths less than the length threshold can be selected from the N sentences. Finally, the selected M sentences can be used to form a summary of the target text.
  • M sentences can be further selected from the N sentences, the sum of the scores of the M sentences is the largest, and the sum of the lengths of the M sentences is less than the length threshold of the summary. It can be seen that when generating the summary of the target text, not only the value of the M sentences in the target text is considered, but also the total length of the M sentences (that is, the length of the summary of the target text) is considered. The factors considered are relatively comprehensive, so the summary of the target text composed of these M sentences is usually a high-quality summary that can meet the reading needs of users, thereby improving user experience.
  • determining the M sentences whose sum of scores is the largest and whose sum of lengths is less than the length of the summary from the N sentences includes: calculating the scores of the N sentences and the lengths of the N sentences by a target algorithm to determine the M sentences whose sum of scores is the largest and whose sum of lengths is less than the length threshold from the N sentences.
  • the user device may obtain the target algorithm, and use the scores of the N sentences of the target text and the lengths of the N sentences of the target text as initial values of the target algorithm, and use the maximum sum of scores and the sum of lengths less than the length threshold of the summary as the optimization target of the target algorithm.
  • the user device may run the target algorithm so that the target algorithm optimizes the scores of the N sentences of the target text and the lengths of the N sentences of the target text, thereby solving the M sentences whose sum of scores is the largest and whose sum of lengths is less than the length threshold of the summary from the N sentences of the target text.
  • the target algorithm is any one of the following: dynamic programming algorithm, backtracking method, branch and bound method and greedy algorithm
  • the greedy algorithm is any one of the following: value density-based greedy algorithm, value-based greedy algorithm and size-based greedy algorithm, etc.
  • determining the M sentences with the maximum sum of scores and the sum of lengths less than the summary length from the N sentences includes: processing the scores of the N sentences and the lengths of the N sentences through a second model to obtain processing results of the N sentences, and the processing results are used to determine the M sentences with the maximum sum of scores and the sum of lengths less than the length threshold from the N sentences.
  • the user device can obtain a second model that is similar to the target algorithm, and input the scores of the N sentences of the target text and the lengths of the N sentences of the target text into the second model, so as to process the scores of the N sentences of the target text and the lengths of the N sentences of the target text through the second model, thereby obtaining and outputting the processing results of the N sentences of the target text, and the processing results generally include an indicator of whether the N sentences of the target text are selected or not, so the processing results can be used to determine the M sentences with the maximum sum of scores and the sum of lengths less than the length threshold of the summary from the N sentences.
  • the scores of N sentences and the lengths of N sentences are processed by a second model
  • the processing results of the N sentences include: performing linear operations on the scores of N sentences and the lengths of N sentences by the second model to obtain a first representation of the N sentences; performing transformer-based processing on the first representation by the second model to obtain a second representation of the N sentences; performing linear operations on the second representation by the second model to obtain a third representation of the N sentences; performing nonlinear operations on the third representation by the second model to obtain the processing results of the N sentences.
  • the user device can input the scores of N sentences of the target text and the lengths of N sentences of the target text into the second model, so that the second model concatenates the scores of N sentences of the target text and the lengths of N sentences of the target text, and then performs linear operations on the concatenated results to obtain the first representation of the N sentences of the target text.
  • the second model may sequentially perform multi-head attention mechanism-based processing, feedforward network processing, residual network processing, etc. (i.e., processing based on the transformer model) on the first representations of the N sentences of the target text, thereby obtaining the second representations of the N sentences of the target text.
  • the second model may perform linear operations on the second representations of the N sentences of the target text, thereby obtaining the third representations of the N sentences of the target text.
  • the second model may perform nonlinear operations on the third representations of the N sentences of the target text, thereby obtaining and outputting the processing results of the N sentences of the target text, which generally include indicators of whether the N sentences of the target text are selected or not, so the user device may select M sentences from the N sentences based on the processing results, the sum of the scores of the selected M sentences is the largest, and the sum of the lengths of the M sentences is less than the length threshold of the summary.
  • scoring N sentences through the first model, and obtaining the scores of the N sentences includes: mapping the N sentences through the first model to obtain the fourth representation of the N sentences; processing the fourth representation based on the transformer model through the first model to obtain the scores of the N sentences.
  • the user device can input the N sentences of the target text into the first model so that the first model maps the N sentences of the target text, thereby obtaining the fourth representation of the N sentences of the target text.
  • the first model sequentially processes the fourth representation of the N sentences of the target text based on the multi-head attention mechanism, the feedforward network, and the residual network (i.e., the processing based on the transformer model), thereby obtaining and outputting the scores of the N sentences of the target text.
  • the user device can successfully obtain the scores of the N sentences of the target text.
  • the target text is input by the user into the user device, and the length threshold of the summary can be set by the user on a display interface provided by the user device.
  • a second aspect of the embodiments of the present application provides a model training method, the method comprising:
  • the training device may first obtain a batch of training data in the first training data set.
  • the training data includes a first text
  • the first text includes P first sentences
  • the lengths of the P first sentences are known
  • P is a positive integer greater than or equal to 2.
  • the training device can input the P first sentences of the first text into the first model to be trained, so as to score the P first sentences of the first text through the first model to be trained, thereby obtaining the scores of the P first sentences of the first text.
  • the scores of the P first sentences are used to indicate the value of the P first sentences in the first file.
  • the training device can also obtain the lengths of the P first sentences of the first text and a second model that is similar to the target algorithm. Then, the training device can input the scores of the P first sentences of the first text and the lengths of the P first sentences of the first text into the second model, so as to process the scores of the P first sentences of the first text and the lengths of the P first sentences of the first text through the second model, thereby obtaining and outputting the processing results of the P first sentences of the first text, which usually include an indicator of whether the P first sentences of the first text are selected or not, so the processing result can be used to determine Q first sentences with the maximum sum of scores and a sum of lengths less than the length threshold of the summary from the P first sentences, where Q is a positive integer greater than or equal to 1, and Q is less than or equal to P.
  • the training device can update the parameters of the first model to be trained based on the Q first sentences, thereby obtaining the first model.
  • the first model trained by the above method has an automatic summary function. Specifically, after obtaining a target text containing N sentences, the N sentences can be scored by the first model to obtain the scores of the N sentences, and the scores of the N sentences are used to indicate the value of the N sentences in the target text. Then, based on the scores of the N sentences and the lengths of the N sentences, M sentences with the maximum sum of scores and the sum of lengths less than the length threshold can be selected from the N sentences. Finally, the selected M sentences can be used to form the summary of the target text.
  • M sentences can be further selected from the N sentences, and the sum of the scores of the M sentences is the largest, and the sum of the lengths of the M sentences is less than the length threshold of the summary. It can be seen that when generating the summary of the target text, not only the value of the M sentences in the target text is considered, but also the total length of the M sentences (that is, the length of the summary of the target text) is considered. The factors considered are relatively comprehensive, so the summary of the target text composed of these M sentences is usually a high-quality summary that can meet the reading needs of users, thereby improving user experience.
  • the method further includes: obtaining a second text, the second text including X second sentences, X ⁇ 2; processing the true scores of the X second sentences and the lengths of the X second sentences by a second model to be trained to obtain processing results of the X second sentences, the processing results of the X second sentences being used to determine, from the X second sentences, Y second sentences whose sum of scores is the largest and whose sum of lengths is less than a length threshold, X ⁇ Y ⁇ 1; obtaining a target loss based on the true processing results of the X second sentences and the processing results of the X second sentences, the target loss being used to indicate the difference between the true processing results of the X second sentences and the processing results of the X second sentences; and updating the parameters of the second model to be trained based on the target loss until the model training conditions are met to obtain a second model.
  • the training device may first obtain a batch of training data in the second training data set, the batch of training data includes the second text, the second text includes X second sentences (X is a positive integer greater than or equal to 2), it is worth noting that the true score of each second sentence is known, the true probability of each sentence being selected is also known, that is, the true processing results of the X second sentences are known, and the length of each second sentence is also known.
  • the training device may input the true scores of the X second sentences of the second text and the lengths of the X second sentences of the second text into the second model to be trained, so as to process the true scores and lengths of the X second sentences of the second text through the second model to be trained, thereby obtaining the processing results of the X second sentences of the second text
  • the processing results usually include an indicator of whether the X second sentences of the second text are selected or not, so the processing results can be used to determine the Y second sentences (Y is a positive integer greater than or equal to 1, and Y is less than or equal to X) with the maximum sum of scores and the sum of lengths less than the length threshold from the X second sentences.
  • the training device can use a preset loss function to calculate the processing results of the X second sentences of the second text and the actual processing results of the X second sentences of the second text, thereby obtaining a target loss, which is used to indicate the difference between the processing results of the X second sentences of the second text and the actual processing results of the X second sentences of the second text.
  • the training device can use the target loss to update the parameters of the second model to be trained, and use another batch of training data in the second training data set to continue training the second model to be trained after the updated parameters until the model training conditions are met, thereby obtaining a second model.
  • the scores of the P first sentences and the lengths of the P first sentences are processed by the second model to obtain the processing results of the P first sentences, including: performing a linear operation on the scores of the P first sentences and the lengths of the P first sentences by the second model to obtain first representations of the P first sentences; performing a transformer-based processing on the first representation by the second model to obtain second representations of the P first sentences; performing a linear operation on the second representation by the second model to obtain third representations of the P first sentences; The third representation is subjected to nonlinear operation through the second model to obtain processing results of P first sentences.
  • scoring the P first sentences through the first model to be trained to obtain the scores of the P first sentences includes: mapping the P first sentences through the first model to be trained to obtain fourth representations of the P first sentences; and processing the fourth representation based on the transformer model through the first model to be trained to obtain the scores of the P first sentences.
  • the parameters of the first model to be trained are updated to obtain the first model, including: obtaining the representation of the first text through the third model to be trained; obtaining the similarity between the fourth representation of the Q first sentences and the representation of the first text through the third model to be trained; updating the parameters of the first model to be trained and the parameters of the third model to be trained based on the similarity until the model training conditions are met, and the first model and the third model are obtained respectively.
  • the training device may multiply the processing results of the P first sentences of the first text with the fourth representation of the P first sentences of the first text, thereby screening out the fourth representations of the Q first sentences with the maximum sum of scores and the sum of lengths less than the length threshold of the summary from the fourth representations of the P first sentences of the first text.
  • the training device may input the fourth representations of the Q first sentences and the first text into the third model to be trained.
  • the third model to be trained can first map the first text to obtain the representation of the first text, and then perform a series of processing on the fourth representation of the Q first sentences and the representation of the first text, so as to obtain the similarity between the fourth representation of the Q first sentences and the representation of the first text.
  • the training device can update the parameters of the first model to be trained and the parameters of the third model to be trained based on the similarity, and use another batch of training data in the first training data set to continue training the first model to be trained after the parameters are updated and the third model to be trained after the parameters are updated until the model training conditions are met, thereby obtaining the third model and the first model in the embodiment shown in Figure 4.
  • a third aspect of an embodiment of the present application provides a summary generation device, which includes: an acquisition module, used to acquire a target text, the target text includes N sentences, N ⁇ 2; a scoring module, used to score the N sentences through a first model to obtain scores of the N sentences, and the scores of the N sentences are used to indicate the value of the N sentences in the target text; a determination module, used to determine, based on the scores of the N sentences and the lengths of the N sentences, M sentences with the maximum sum of scores and a sum of lengths less than a length threshold from the N sentences, N ⁇ M ⁇ 1; and a generation module, used to generate a summary of the target text based on the M sentences.
  • the N sentences can be scored by the first model to obtain the scores of the N sentences, and the scores of the N sentences are used to indicate the value of the N sentences in the target text. Then, based on the scores of the N sentences and the lengths of the N sentences, M sentences with the maximum sum of scores and the sum of lengths less than the length threshold can be selected from the N sentences. Finally, the selected M sentences can be used to form a summary of the target text.
  • M sentences can be further selected from the N sentences, and the sum of the scores of the M sentences is the largest, and the sum of the lengths of the M sentences is less than the length threshold of the summary. It can be seen that when generating the summary of the target text, not only the value of the M sentences in the target text is considered, but also the total length of the M sentences (i.e., the length of the summary of the target text) is considered. The factors considered are relatively comprehensive, so the summary of the target text composed of these M sentences is usually a high-quality summary that can meet the reading needs of users, thereby improving user experience.
  • the determination module is used to calculate the scores of N sentences and the lengths of the N sentences by a target algorithm to determine M sentences from the N sentences whose sum of scores is the largest and whose sum of lengths is less than a length threshold.
  • the target algorithm is any one of the following: a dynamic programming algorithm, a backtracking method, a branch and bound method, and a greedy algorithm.
  • a determination module is used to process the scores of N sentences and the lengths of the N sentences through a second model to obtain processing results of the N sentences, and the processing results are used to determine, from the N sentences, M sentences whose sum of scores is the largest and whose sum of lengths is less than a length threshold.
  • a determination module is used to: perform linear operations on the scores of N sentences and the lengths of the N sentences through a second model to obtain a first representation of the N sentences; perform transformer-based processing on the first representation through the second model to obtain a second representation of the N sentences; perform linear operations on the second representation through the second model to obtain a third representation of the N sentences; and perform nonlinear operations on the third representation through the second model to obtain processing results of the N sentences.
  • the scoring module is used to: map N sentences through a first model to obtain fourth representations of the N sentences; and process the fourth representations based on a transformer model through the first model to obtain scores of the N sentences.
  • the target text is input by a user
  • the length threshold is set by the user.
  • the fourth aspect of an embodiment of the present application provides a model training device, which includes: a first acquisition module, used to acquire a first text, the first text includes P first sentences, P ⁇ 2; a scoring module, used to score the P first sentences through a first model to be trained to obtain scores of the P first sentences, and the scores of the P first sentences are used to indicate the value of the P first sentences in the first text; a first processing module, used to process the scores of the P first sentences and the lengths of the P first sentences through a second model to obtain processing results of the P first sentences, and the processing results of the P first sentences are used to determine, from the P first sentences, Q first sentences whose sum of scores is the largest and whose sum of lengths is less than a length threshold, P ⁇ Q ⁇ 1; a first updating module, used to update the parameters of the first model to be trained based on the Q first sentences to obtain a first model.
  • the first model obtained by training the above-mentioned device has an automatic summary function. Specifically, after obtaining a target text containing N sentences, the N sentences can be scored by the first model to obtain the scores of the N sentences, and the scores of the N sentences are used to indicate the value of the N sentences in the target text. Then, based on the scores of the N sentences and the lengths of the N sentences, M sentences with the maximum sum of scores and the sum of lengths less than the length threshold can be selected from the N sentences. Finally, the selected M sentences can be used to form the summary of the target text.
  • M sentences can be further selected from the N sentences, and the sum of the scores of the M sentences is the largest, and the sum of the lengths of the M sentences is less than the length threshold of the summary. It can be seen that when generating the summary of the target text, not only the value of the M sentences in the target text is considered, but also the total length of the M sentences (i.e., the length of the summary of the target text) is considered. The factors considered are relatively comprehensive, so the summary of the target text composed of these M sentences is usually a high-quality summary that can meet the reading needs of users, thereby improving user experience.
  • the device further includes: a second acquisition module, used to acquire a second text, the second text includes X second sentences, X ⁇ 2; a second processing module, used to process the true scores of the X second sentences and the lengths of the X second sentences through a second model to be trained to obtain processing results of the X second sentences, the processing results of the X second sentences are used to determine, from the X second sentences, Y second sentences with the maximum sum of scores and the sum of lengths less than a length threshold, X ⁇ Y ⁇ 1; a third acquisition module, used to acquire a target loss based on the true processing results of the X second sentences and the processing results of the X second sentences, the target loss is used to indicate the difference between the true processing results of the X second sentences and the processing results of the X second sentences; a second updating module, used to update the parameters of the second model to be trained based on the target loss until the model training condition is met to obtain the second model.
  • a second acquisition module used to acquire a second text, the second text
  • the first processing module is used to: perform linear operations on the scores of P first sentences and the lengths of the P first sentences through a second model to obtain first representations of the P first sentences; perform transformer model-based processing on the first representation through the second model to obtain second representations of the P first sentences; perform linear operations on the second representation through the second model to obtain third representations of the P first sentences; and perform nonlinear operations on the third representation through the second model to obtain processing results of the P first sentences.
  • the scoring module is used to: map P first sentences through the first model to be trained to obtain fourth representations of the P first sentences; and process the fourth representation based on the transformer model through the first model to be trained to obtain scores of the P first sentences.
  • the first update module is used to: obtain a representation of the first text through a third model to be trained; obtain the similarity between the fourth representation of Q first sentences and the representation of the first text through the third model to be trained; update the parameters of the first model to be trained and the parameters of the third model to be trained based on the similarity until the model training conditions are met, and obtain the first model and the third model respectively.
  • a fifth aspect of an embodiment of the present application provides a summary generation device, which includes a memory and a processor; the memory stores codes, and the processor is configured to execute the codes. When the codes are executed, the summary generation device performs the method described in the first aspect or any possible implementation manner of the first aspect.
  • a sixth aspect of an embodiment of the present application provides a model training device, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code.
  • the model training device executes the method described in the second aspect or any possible implementation method of the second aspect.
  • a seventh aspect of an embodiment of the present application provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute a method as described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.
  • An eighth aspect of an embodiment of the present application provides a chip system, which includes a processor for calling a computer program or computer instructions stored in a memory so that the processor executes a method as described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.
  • the processor is coupled to the memory through an interface.
  • the chip system also includes a memory, in which a computer program or computer instructions are stored.
  • a ninth aspect of an embodiment of the present application provides a computer storage medium storing a computer program, which, when executed by a computer, enables the computer to implement the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.
  • the tenth aspect of the embodiments of the present application provides a computer program product, which stores instructions, which, when executed by a computer, enable the computer to implement the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.
  • the N sentences after obtaining the target text containing N sentences, the N sentences can be scored by the first model to obtain the scores of the N sentences, and the scores of the N sentences are used to indicate the value of the N sentences in the target text. Then, based on the scores of the N sentences and the lengths of the N sentences, M sentences with the maximum sum of scores and the sum of lengths less than the length threshold can be selected from the N sentences. Finally, the selected M sentences can be composed of the summary of the target text.
  • M sentences can be further selected from the N sentences, and the sum of the scores of these M sentences is the largest, and the sum of the lengths of these M sentences is less than the length threshold of the summary. It can be seen that when generating the summary of the target text, not only the value of these M sentences in the target text is considered, but also the total length of these M sentences (i.e. the length of the summary of the target text) is considered. The factors considered are relatively comprehensive, so the summary of the target text composed of these M sentences is usually a high-quality summary that can meet the reading needs of users, thereby improving user experience.
  • FIG1 is a schematic diagram of a structure of an artificial intelligence main framework
  • FIG2a is a schematic diagram of a structure of a text processing system provided in an embodiment of the present application.
  • FIG2b is another schematic diagram of the structure of the text processing system provided in an embodiment of the present application.
  • FIG2c is a schematic diagram of a text processing related device provided in an embodiment of the present application.
  • FIG3 is a schematic diagram of the architecture of the system 100 provided in an embodiment of the present application.
  • FIG4 is a schematic diagram of a flow chart of a summary generation method provided in an embodiment of the present application.
  • FIG5 is a schematic diagram of the interaction between a user and a user device provided in an embodiment of the present application.
  • FIG6 is a schematic structural diagram of a first model provided in an embodiment of the present application.
  • FIG7 is a schematic structural diagram of a second model provided in an embodiment of the present application.
  • FIG8 is a flow chart of a model training method provided in an embodiment of the present application.
  • FIG9 is a schematic diagram of a structure of a third model to be trained provided in an embodiment of the present application.
  • FIG10 is another schematic diagram of the structure of the third model to be trained provided in an embodiment of the present application.
  • FIG11 is another schematic diagram of the structure of the third model to be trained provided in an embodiment of the present application.
  • FIG12 is another schematic diagram of the structure of the third model to be trained provided in an embodiment of the present application.
  • FIG13 is a schematic diagram of a structure of a summary generation device provided in an embodiment of the present application.
  • FIG14 is a schematic diagram of a structure of a model training device provided in an embodiment of the present application.
  • FIG15 is a schematic diagram of a structure of an execution device provided in an embodiment of the present application.
  • FIG16 is a schematic diagram of a structure of a training device provided in an embodiment of the present application.
  • FIG. 17 is a schematic diagram of the structure of a chip provided in an embodiment of the present application.
  • the embodiment of the present application provides a summary generation method and related devices, which can automatically generate a summary of a target text that meets the length limit, and the generated summary of the target text is usually a high-quality summary that can meet the user's reading needs, thereby improving the user experience.
  • Automatic summarization technology is a technology that uses computers to automatically analyze text and summarize content to automatically generate summaries, that is, to express the main content of the text in a concise form according to user requirements. Automatic summarization technology can effectively help users find content of interest from retrieved articles and improve reading speed and quality.
  • automatic summarization technology can be realized through the neural network model in AI technology, that is, the text is processed through the neural network model to extract the summary of the text.
  • the neural network model can first map multiple sentences of the text to obtain representations of multiple sentences, and then process the representations of multiple sentences based on the multi-head attention mechanism to obtain the scores of multiple sentences.
  • the score of any sentence can represent the value of the sentence in the text. In this way, several sentences with larger scores (i.e., several sentences with higher values) can be combined into a summary of the text.
  • AI technology is a technical discipline that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence. AI technology obtains the best results by sensing the environment, acquiring knowledge and using knowledge.
  • artificial intelligence technology is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence.
  • Using artificial intelligence for data processing is a common application of artificial intelligence.
  • Figure 1 is a structural diagram of the main framework of artificial intelligence.
  • the following is an explanation of the above artificial intelligence theme framework from the two dimensions of "intelligent information chain” (horizontal axis) and “IT value chain” (vertical axis).
  • the "intelligent information chain” reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, the data has undergone a condensation process of "data-information-knowledge-wisdom".
  • the "IT value chain” reflects the value that artificial intelligence brings to the information technology industry from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the industrial ecology process of the system.
  • the infrastructure provides computing power support for the artificial intelligence system, enables communication with the outside world, and is supported by the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips (CPU, NPU, GPU, ASIC, FPGA and other hardware acceleration chips); the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.
  • smart chips CPU, NPU, GPU, ASIC, FPGA and other hardware acceleration chips
  • the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc.
  • sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.
  • the data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence.
  • the data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.
  • Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.
  • machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.
  • Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.
  • Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.
  • some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.
  • Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical applications. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.
  • FIG2a is a schematic diagram of a text processing system provided in an embodiment of the present application, wherein the text processing system includes a user device and a data processing device.
  • the user device includes an intelligent terminal such as a mobile phone, a personal computer or an information processing center.
  • the user device is usually the initiator of text processing, and as the initiator of text processing, a user usually initiates a request through the user device.
  • the above-mentioned data processing device can be a device or server with data processing function such as a cloud server, a network server, an application server and a management server.
  • the data processing device receives text processing requests from the intelligent terminal through an interactive interface, and then performs text processing in the form of machine learning, deep learning, search, reasoning, decision-making, etc. through the memory for storing data and the processor for data processing.
  • the memory in the data processing device can be a general term, including local storage and databases for storing historical data.
  • the database can be on the data processing device or on other network servers.
  • the user device can receive the user's instruction. For example, the user device can obtain a target text input/selected by the user, and then initiate a processing request containing the target text to the data processing device, so that the data processing device executes the text processing application for the target text obtained by the user device, thereby obtaining a summary of the target text.
  • the user device can obtain a target text input by the user (the target text can be either a long text or a short text), and then initiate a processing request for the target text to the data processing device, so that the data processing device processes the target text based on the automatic summarization technology, thereby obtaining a summary of the target text.
  • the data processing device may execute the summary generation method according to the embodiment of the present application.
  • Figure 2b is another structural diagram of the text processing system provided in an embodiment of the present application.
  • the user device directly serves as a data processing device.
  • the user device can directly obtain input from the user and directly process it by the hardware of the user device itself.
  • the specific process is similar to that of Figure 2a. Please refer to the above description and will not be repeated here.
  • the user device can receive instructions from the user. For example, the user device can obtain a target text selected by the user in the user device (the target text can be either a long text or a short text), and then the user device itself executes a text processing application on the target text (equivalent to processing based on automatic summarization technology) to obtain a summary of the target text.
  • the target text can be either a long text or a short text
  • the user device itself executes a text processing application on the target text (equivalent to processing based on automatic summarization technology) to obtain a summary of the target text.
  • the user equipment itself can execute the summary generation method of the embodiment of the present application.
  • FIG. 2c is a schematic diagram of a text processing related device provided in an embodiment of the present application.
  • the user device in the above Figures 2a and 2b can specifically be the local device 301 or the local device 302 in Figure 2c
  • the data processing device in Figure 2a can specifically be the execution device 210 in Figure 2c
  • the data storage system 250 can store the data to be processed of the execution device 210
  • the data storage system 250 can be integrated on the execution device 210, and can also be set on the cloud or other network servers.
  • the processors in Figures 2a and 2b can perform data training/machine learning/deep learning through a neural network model or other models (for example, a model based on a support vector machine), and use the model finally trained or learned from the data to execute text processing applications on the text, thereby obtaining corresponding processing results.
  • a neural network model or other models for example, a model based on a support vector machine
  • FIG 3 is a schematic diagram of the system 100 architecture provided in an embodiment of the present application.
  • the execution device 110 is configured with an input/output (I/O) interface 112 for data interaction with an external device.
  • the user can input data to the I/O interface 112 through the client device 140.
  • the input data may include: various tasks to be scheduled, callable resources and other parameters in the embodiment of the present application.
  • the execution device 110 When the execution device 110 preprocesses the input data, or when the computing module 111 of the execution device 110 performs calculation and other related processing (such as implementing the function of the neural network in the present application), the execution device 110 can call the data, code, etc. in the data storage system 150 for the corresponding processing, and can also store the data, instructions, etc. obtained by the corresponding processing in the data storage system 150.
  • the I/O interface 112 returns the processing result to the client device 140 so as to provide it to the user.
  • the training device 120 can generate corresponding target models/rules based on different training data for different goals or tasks, and the corresponding target models/rules can be used to achieve the above goals or complete the above tasks, thereby providing the user with the desired results.
  • the training data can be stored in the database 130 and come from the training samples collected by the data collection device 160.
  • the user can manually give input data, and the manual giving can be operated through the interface provided by the I/O interface 112.
  • the client device 140 can automatically send input data to the I/O interface 112. If the client device 140 is required to automatically send input data and needs to obtain the user's authorization, the user can set the corresponding authority in the client device 140.
  • the user can view the results output by the execution device 110 on the client device 140, and the specific presentation form can be a specific method such as display, sound, action, etc.
  • the client device 140 can also be used as a data acquisition terminal to collect the input data of the input I/O interface 112 and the output results of the output I/O interface 112 as shown in the figure as new sample data, and store them in the database 130.
  • the I/O interface 112 directly stores the input data of the input I/O interface 112 and the output results of the output I/O interface 112 as new sample data in the database 130.
  • FIG3 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation.
  • the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 can also be placed in the execution device 110.
  • a neural network can be obtained by training according to the training device 120.
  • the embodiment of the present application also provides a chip, which includes a neural network processor NPU.
  • the chip can be set in the execution device 110 as shown in Figure 3 to complete the calculation work of the calculation module 111.
  • the chip can also be set in the training device 120 as shown in Figure 3 to complete the training work of the training device 120 and output the target model/rule.
  • Neural network processor NPU is mounted on the main central processing unit (CPU) (host CPU) as a coprocessor, and the main CPU assigns tasks.
  • the core part of NPU is the operation circuit, and the controller controls the operation circuit to extract data from the memory (weight memory or input memory) and perform operations.
  • the arithmetic circuit includes multiple processing units (process engines, PEs) internally.
  • the arithmetic circuit is a two-dimensional systolic array.
  • the arithmetic circuit can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition.
  • the arithmetic circuit is a general-purpose matrix processor.
  • the operation circuit takes the corresponding data of matrix B from the weight memory and caches it on each PE in the operation circuit.
  • the operation circuit takes the matrix A data from the input memory and performs matrix operations with matrix B.
  • the partial results or final results of the matrix are stored in the accumulator.
  • the vector calculation unit can further process the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc.
  • the vector calculation unit can be used for network calculations of non-convolutional/non-FC layers in neural networks, such as pooling, batch normalization, local response normalization, etc.
  • the vector computation unit can store the processed output vector to a unified buffer.
  • the vector computation unit can apply a nonlinear function to the output of the computation circuit, such as a vector of accumulated values, to generate an activation value.
  • the vector computation unit generates a normalized value, a merged value, or both.
  • the processed output vector can be used as an activation input to the computation circuit, such as for use in a subsequent layer in a neural network.
  • the unified memory is used to store input data and output data.
  • the weight data is directly transferred from the external memory to the input memory and/or the unified memory through the direct memory access controller (DMAC), the weight data in the external memory is stored in the weight memory, and the data in the unified memory is stored in the external memory.
  • DMAC direct memory access controller
  • the bus interface unit (BIU) is used to enable interaction between the main CPU, DMAC and instruction fetch memory through the bus.
  • An instruction fetch buffer connected to the controller, used to store instructions used by the controller
  • the controller is used to call the instructions cached in the memory to control the working process of the computing accelerator.
  • the unified memory, input memory, weight memory and instruction fetch memory are all on-chip memories
  • the external memory is a memory outside the NPU, which can be a double data rate synchronous dynamic random access memory (DDR SDRAM), a high bandwidth memory (HBM) or other readable and writable memory.
  • DDR SDRAM double data rate synchronous dynamic random access memory
  • HBM high bandwidth memory
  • a neural network may be composed of neural units, and a neural unit may refer to an operation unit with xs and intercept 1 as input, and the output of the operation unit may be:
  • n is a natural number greater than 1
  • Ws is the weight of xs
  • b is the bias of the neural unit.
  • f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal.
  • the output signal of the activation function can be used as the input of the next convolutional layer.
  • the activation function can be a sigmoid function.
  • a neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit.
  • the input of each neural unit can be connected to the local receptive field of the previous layer to extract the characteristics of the local receptive field.
  • the local receptive field can be an area composed of several neural units.
  • the word "space” is used here because the classified object is not a single thing, but a class of things, and space refers to the collection of all individuals of this class of things.
  • W is a weight vector, and each value in the vector represents the weight value of a neuron in the neural network of this layer.
  • the vector W determines the spatial transformation from input space to output space described above, that is, the weight W of each layer controls how to transform the space.
  • the purpose of training a neural network is to finally obtain the weight matrix of all layers of the trained neural network (the weight matrix formed by many layers of vectors W). Therefore, the training process of a neural network is essentially about learning how to control spatial transformations, or more specifically, learning the weight matrix.
  • Neural networks can use the error back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, the forward transmission of the input signal to the output will generate error loss, and the error loss information is back-propagated to update the parameters in the initial neural network model, so that the error loss converges.
  • the back propagation algorithm is a back propagation movement dominated by error loss, which aims to obtain the optimal parameters of the neural network model, such as the weight matrix.
  • the method provided in the present application is described below from the training side of the neural network and the application side of the neural network.
  • the model training method provided in the embodiment of the present application involves the processing of data sequences, and can be specifically applied to data training, machine learning, deep learning and other methods to perform symbolic and formalized intelligent information modeling, extraction, preprocessing, training, etc. on the training data (for example, the first text and the second text in the present application), and finally obtain a trained neural network (such as the first model, the second model and the third model in the present application); and the summary generation method provided in the embodiment of the present application can use the above-mentioned trained neural network to input input data (for example, the target text in the present application) into the trained neural network to obtain output data (such as the scores of multiple sentences in the target text in the present application and the processing results of these multiple sentences, etc.).
  • model training method provided in the embodiment of the present application
  • the training method and the summary generation method are inventions based on the same idea, and can also be understood as two parts in a system, or two stages of an overall process: such as the model training stage and the model application stage.
  • FIG4 is a flow chart of the summary generation method provided in the embodiment of the present application. As shown in FIG4, the method includes:
  • the user when the user needs to obtain a summary of the target text, the user can input or select the target text in the user device, so that the user device determines that a summary of the target text needs to be generated.
  • the target text is composed of N (N is a positive integer greater than or equal to 2) sentences, each sentence is composed of one or more words, that is, each sentence has a certain length, and the length of each sentence is known.
  • the user device can provide a user interface, which includes an input window and a display window.
  • the input window is used for the user to input or select the target text, that is, the user can input the target text in the input window, or the user can select the target text in the text list displayed in the input window, etc.
  • the display window is used to display the summary of the target text for the user. It is worth noting that the size of the display window usually determines the length threshold of the summary (also referred to as the word count threshold of the summary, or the target length limit of the summary).
  • the length threshold of the summary can be determined in the following ways: (1) As shown in FIG5 (FIG.
  • the user can slide and zoom the display window through gesture control. If the user zooms in (zooms in) the display window, it means that the number of words in the summary that the user needs to read is small, and the user device can set the length threshold of the summary to be small. If the user zooms out (zooms out) the display window, it means that the number of words in the summary that the user needs to read is large, and the user device can set the length threshold of the summary to be large. (2) The user device can adjust the length threshold according to the user's own information and the size of the display window. For example, during working hours, users often use their spare time to quickly read and grasp information.
  • the number of words in the summary to be presented may be relatively small, and the user device may set a smaller threshold for the length of the summary.
  • the user device may detect that the user is in a mobile state (such as the user is in a car, etc.), the number of words in the summary to be presented may be relatively small, so that the user can easily and quickly complete simple reading while in a mobile state. Therefore, the user device may set a smaller threshold for the length of the summary, etc.
  • the user device can input the N sentences of the target text into the first model (a trained neural network model, which can also be called an extractive summarization model) to score the N sentences of the target text through the first model, thereby obtaining the scores of the N sentences of the target text.
  • the first model (also referred to as the extraction model) may include a BERT module and a transformer encoder module (i.e., the aforementioned transformer model), the input end of the BERT module is the input end of the first model, the output end of the BERT module is connected to the input end of the transformer encoder module, and the output end of the transformer encoder module is the output end of the first model.
  • the user device can obtain the scores of the N sentences of the target text in the following manner:
  • the user device can input the N sentences of the target text into the BERT module in the first model, so that the BERT module maps the N sentences of the target text, thereby obtaining the fourth representation of the N sentences of the target text, and sending the fourth representation of the N sentences of the target text to the transformer encoder module.
  • the transformer encoder module After obtaining the fourth representations of the N sentences of the target text, the transformer encoder module processes the fourth representations of the N sentences of the target text in turn based on the multi-head attention mechanism, the feedforward network, and the residual network, thereby obtaining and outputting the scores of the N sentences of the target text. At this point, the user device can successfully obtain the scores of the N sentences of the target text.
  • the user device may also obtain the lengths of the N sentences of the target text.
  • the user device may compare the problem to a knapsack problem, which can be described by the following formula:
  • V is the objective function of the knapsack problem
  • S is the M sentences selected from the N sentences of the target text (M is a positive integer greater than or equal to 1, and M is less than or equal to N), that is, the set of selected sentences
  • D is the N sentences of the target text, that is, the set of selected sentences (that is, the target text)
  • lj is the length of the jth sentence selected
  • C is the length threshold of the summary
  • sj is the score of the jth sentence selected.
  • the user device can use the scores of the N sentences of the target text and the lengths of the N sentences of the target text as the initial values of the knapsack problem, and solve the knapsack problem.
  • the solution obtained is the M sentences in the N sentences of the target text whose sum of scores is the largest and whose sum of lengths is less than the length threshold of the summary.
  • the user device may select M sentences from N sentences of the target text in a variety of ways:
  • the user device may obtain a target algorithm (i.e., an algorithm for solving the knapsack problem), and use the scores of the N sentences of the target text and the lengths of the N sentences of the target text as initial values of the target algorithm, and use the maximum sum of scores and the sum of lengths being less than the length threshold of the summary as the optimization target (constraint) of the target algorithm. Then, the user device may run the target algorithm so that the target algorithm optimizes the scores of the N sentences of the target text and the lengths of the N sentences of the target text, thereby solving M sentences with the maximum sum of scores and the sum of lengths being less than the length threshold of the summary among the N sentences of the target text.
  • a target algorithm i.e., an algorithm for solving the knapsack problem
  • the target algorithm is any one of the following: dynamic programming (DP), backtracking, branch and bound, and greedy algorithm
  • the greedy algorithm is any one of the following: density-based greedy algorithm (density greedy), value-based greedy algorithm (profit greedy), size-based greedy algorithm, etc.
  • the user device can obtain a second model (a trained neural network model) that is similar to the target algorithm, and input the scores of the N sentences of the target text and the lengths of the N sentences of the target text into the second model, so as to process the scores of the N sentences of the target text and the lengths of the N sentences of the target text through the second model, thereby obtaining and outputting the processing results of the N sentences of the target text.
  • the processing results usually include indicators of whether the N sentences of the target text are selected or not, so the processing results can be used to determine, from the N sentences, the M sentences whose sum of scores is the largest and whose sum of lengths is less than the length threshold of the summary.
  • the second model (also referred to as a knapsack transformer model) may include a first linear module, a transformer encoder module, a second linear module, and a nonlinear module, wherein the input end of the first linear module is the input end of the second model, the output end of the first linear module is connected to the input end of the transformer encoder module, the output end of the transformer encoder module is connected to the input end of the second linear module, the output end of the second linear module is connected to the input end of the nonlinear module, and the output end of the nonlinear module is the output end of the second model.
  • the user device can obtain the processing results of N sentences of the target text in the following manner:
  • the user device may input the scores of the N sentences of the target text and the lengths of the N sentences of the target text into the first linear module of the second model, so that the first linear module first concatenates the scores of the N sentences of the target text and the lengths of the N sentences of the target text, and then performs a linear operation on the concatenated result to obtain a first representation of the N sentences of the target text, and sends the first representation of the N sentences of the target text to the transformer encoder module.
  • the transformer encoder module can process the first representations of the N sentences of the target text based on the multi-head attention mechanism, the feedforward network, the residual network, etc., so as to obtain the second representations of the N sentences of the target text and send them to the second linear module.
  • the second linear module can perform a linear operation on the second representations of the N sentences of the target text, thereby obtaining the third representations of the N sentences of the target text, and sending it to the nonlinear module.
  • the nonlinear module can obtain the third representation of the N sentences of the target text.
  • the representation is subjected to nonlinear operation (for example, nonlinear operation is realized by sigmoid activation function, etc.), so as to obtain and output the processing results of N sentences of the target text, which usually include indicators of whether the N sentences of the target text are selected or not (the value of the indicator is usually 0 or 1).
  • the indicator of whether the first sentence is selected or not is 0, which means that the first sentence cannot be selected
  • the indicator of whether the second sentence is selected or not is 1, which means that the second sentence can be selected, etc.
  • the user device can select M sentences from the N sentences, and the sum of the scores of the selected M sentences is the largest, and the sum of the lengths of the M sentences is less than the length threshold of the summary.
  • the user device may compose a summary of the target text from the M sentences and display it in a display window for the user to browse and use.
  • the N sentences after obtaining the target text containing N sentences, the N sentences can be scored by the first model to obtain the scores of the N sentences, and the scores of the N sentences are used to indicate the value of the N sentences in the target text. Then, based on the scores of the N sentences and the lengths of the N sentences, M sentences with the maximum sum of scores and the sum of lengths less than the length threshold can be selected from the N sentences. Finally, the selected M sentences can be composed of the summary of the target text.
  • M sentences can be further selected from the N sentences, and the sum of the scores of these M sentences is the largest, and the sum of the lengths of these M sentences is less than the length threshold of the summary. It can be seen that when generating the summary of the target text, not only the value of these M sentences in the target text is considered, but also the total length of these M sentences (i.e. the length of the summary of the target text) is considered. The factors considered are relatively comprehensive, so the summary of the target text composed of these M sentences is usually a high-quality summary that can meet the reading needs of users, thereby improving user experience.
  • the size of the length threshold of the summary can be controlled by the user's sliding operation, that is, when the user zooms in and zooms in on the display window, the length threshold of the summary can be set to be smaller, and when the user zooms in and zooms out on the display window, the length threshold of the summary can be set to be larger.
  • the neural network model provided in the embodiment of the present application can also be compared with the neural network model of the related art.
  • the second model provided in the embodiment of the present application can be compared with the neural network model of the related art. The comparison results are shown in Table 1:
  • FIG8 is a flow chart of the model training method provided in the embodiment of the present application. As shown in FIG8, the method includes:
  • the training device may first obtain a batch of training data in the first training data set, where the batch of training data includes a first text, and the first text includes P first sentences (P is a positive integer greater than or equal to 2), each of which is composed of one or more characters, that is, each first sentence has a certain length, and the length of each first sentence is known.
  • P is a positive integer greater than or equal to 2
  • Scoring the P first sentences by using the first model to be trained to obtain scores of the P first sentences, where the scores of the P first sentences are used to indicate values of the P first sentences in the first text.
  • the training device may obtain the scores of the P first sentences of the first text in the following manner: first, the training device may map the P first sentences through the first model to be trained to obtain the fourth representations of the P first sentences. Then, the training device may process the fourth representation based on the transformer model through the first model to be trained to obtain the scores of the P first sentences.
  • the processing results of the P first sentences are used to determine Q first sentences whose sum of scores is the largest and whose sum of lengths is less than a length threshold from the P first sentences, P ⁇ Q ⁇ 1.
  • the training device can also obtain the lengths of the P first sentences of the first text and a second model (a trained neural network model) that is similar to the target algorithm. Then, the training device can input the scores of the P first sentences of the first text and the lengths of the P first sentences of the first text into the second model, so as to process the scores of the P first sentences of the first text and the lengths of the P first sentences of the first text through the second model, thereby obtaining and outputting the processing results of the P first sentences of the first text, which usually include an indicator of whether the P first sentences of the first text are selected or not, so the processing result can be used to determine Q first sentences (Q is a positive integer greater than or equal to 1, and Q is less than or equal to P) with the maximum sum of scores and the sum of lengths less than the length threshold of the summary from the P first sentences.
  • Q is a positive integer greater than or equal to 1
  • Q is less than or equal to P
  • the training device can obtain the processing results of the P first sentences of the first text in the following manner: first, the training device performs a linear operation on the scores of the P first sentences and the lengths of the P first sentences through the second model to obtain the first representations of the P first sentences. Next, the training device can perform a transformer-based processing on the first representation through the second model to obtain the second representation of the P first sentences. Then, the training device can perform a linear operation on the second representation through the second model to obtain the third representation of the P first sentences. Finally, the training device can perform a nonlinear operation on the third representation through the second model to obtain the processing results of the P first sentences.
  • the training device may first complete the training for the second model to be trained, thereby obtaining the second model.
  • the process includes:
  • the training device may first obtain a batch of training data from the second training data set, where the batch of training data includes a second text, where the second text includes X second sentences (X is a positive integer greater than or equal to 2).
  • the true score of each second sentence i.e., the true importance of each second sentence in the second text
  • the true probability of each sentence being selected is also known, that is, the true processing results of the X second sentences are known (the true processing results can be obtained by the aforementioned target algorithm), and the length of each second sentence is also known.
  • the second training data set and the first training data set are usually two different data sets, and the second training data set can be sampled in the following manner: First, an original data set can be obtained. Then, the training device can determine the average length of all sentences in the original data set, and use the average length of all sentences and the maximum length that the second model to be trained can process to determine the average number of sentences processed by the second model to be trained. Then, a Poisson distribution can be constructed based on the average length of all sentences, a gamma distribution can be constructed based on the average number of sentences processed, and a uniform distribution can be constructed based on the score of each sentence in the original data set. Finally, a new data set, namely the second training data set, is manually sampled to satisfy these three distributions at the same time, and the second training data set is provided to the training device.
  • the training device may input the true scores of the X second sentences of the second text and the lengths of the X second sentences of the second text into the second model to be trained, so as to process the true scores and lengths of the X second sentences of the second text through the second model to be trained, thereby obtaining (predicted) processing results of the X second sentences of the second text.
  • the processing results generally include the (predicted) probabilities of the X second sentences of the second text being selected, so the processing results can be used to determine, from the X second sentences, Y second sentences (Y is a positive integer greater than or equal to 1, and Y is less than or equal to X) whose sum of scores is the largest and whose sum of lengths is less than a length threshold.
  • step 403 of the embodiment shown in FIG. 4 the specific process of the second to-be-trained model processing the true scores and lengths of the X second sentences in the second text can be referred to in step 403 of the embodiment shown in FIG. 4 , in which the second model processes the scores and lengths of the N sentences in the target text. The description part will not be repeated here.
  • the training device can use a preset loss function to calculate the processing results of the X second sentences of the second text and the actual processing results of the X second sentences of the second text, thereby obtaining a target loss, where the target loss is used to indicate the difference between the processing results of the X second sentences of the second text and the actual processing results of the X second sentences of the second text.
  • the training device can use the target loss to update the parameters of the second model to be trained, and use another batch of training data in the second training data set to continue training the second model to be trained with updated parameters until the model training conditions are met (for example, the target loss converges, etc.), thereby obtaining the second model in the embodiment shown in FIG. 4 .
  • the training device can update the parameters of the first model to be trained based on these Q first sentences, thereby obtaining the first model in the embodiment shown in Figure 4.
  • the training device can train the first model in the following manner:
  • the training device may multiply the processing results of the P first sentences of the first text by the fourth representations of the P first sentences of the first text, thereby screening out the fourth representations of the Q first sentences whose sum of scores is the largest and whose sum of lengths is less than the length threshold of the summary from among the fourth representations of the P first sentences of the first text.
  • the training device can input the fourth representations of the Q first sentences and the first text into the third model to be trained (another untrained neural network model, i.e., a comparative learning model). Then, the third model to be trained can first obtain the representation of the first text, and then obtain and output the similarity between the fourth representations of the Q first sentences and the representation of the first text.
  • the third model to be trained can first obtain the representation of the first text, and then obtain and output the similarity between the fourth representations of the Q first sentences and the representation of the first text.
  • the third model to be trained can have multiple structures, which will be introduced below respectively:
  • the third model to be trained may include a BERT module, a first transformer encoder module, a second transformer encoder module, a predictor module, and a similarity determination module.
  • the BERT module is used to map the first text to obtain a representation of the first text
  • the remaining modules are used to perform a series of processing on the fourth representation of the Q first sentences and the representation of the first text, so as to obtain the similarity between the fourth representation of the Q first sentences and the representation of the first text.
  • the third model to be trained may include a BERT module, a first multi-layer perceptron (MLP), a second MLP module, a multi-head attention module, and a similarity determination module.
  • the BERT module is used to map the first text to obtain a representation of the first text
  • the remaining modules are used to perform a series of processing on the fourth representation of the Q first sentences and the representation of the first text, so as to obtain the similarity between the fourth representation of the Q first sentences and the representation of the first text.
  • the third model to be trained may include a BERT module, a first multi-layer perceptron (MLP), a second MLP module, a predictor module, and a similarity determination module.
  • the BERT module is used to map the first text to obtain a representation of the first text
  • the remaining modules are used to perform a series of processing on the fourth representation of the Q first sentences and the representation of the first text, so as to obtain the similarity between the fourth representation of the Q first sentences and the representation of the first text.
  • the third model to be trained may include a first Simbert module, a second Simbert module, a first multi-layer perceptron (MLP), a second MLP module, a predictor module, and a similarity determination module.
  • the first Simbert module is used to map the first text to obtain a representation of the first text
  • the remaining modules are used to perform a series of processing on the fourth representation of the Q first sentences and the representation of the first text, so as to obtain the similarity between the fourth representation of the Q first sentences and the representation of the first text.
  • the training device can update the parameters of the first model to be trained and the parameters of the third model to be trained based on the similarity, and use another batch of training data in the first training data set to continue training the first model to be trained after the updated parameters and the third model to be trained after the updated parameters until the model training conditions are met (for example, the similarity reaches the maximum value, etc.), thereby obtaining the third model and the first model in the embodiment shown in Figure 4.
  • the training device has completed the end-to-end training of all models, and deployed the trained first model and second model in the aforementioned user device or data processing device, so the user device or data processing device can provide automatic summary services for users.
  • the first model trained in the embodiment of the present application has an automatic summarization function. Specifically, when obtaining a target text containing N sentences After that, the N sentences can be scored by the first model to obtain the scores of the N sentences, and the scores of the N sentences are used to indicate the value of the N sentences in the target text. Then, based on the scores of the N sentences and the lengths of the N sentences, M sentences with the largest sum of scores and a sum of lengths less than the length threshold can be selected from the N sentences. Finally, the selected M sentences can be used to form the summary of the target text.
  • M sentences can be further selected from the N sentences, and the sum of the scores of these M sentences is the largest, and the sum of the lengths of these M sentences is less than the length threshold of the summary. It can be seen that when generating the summary of the target text, not only the value of these M sentences in the target text is considered, but also the total length of these M sentences (that is, the length of the summary of the target text) is considered. The factors considered are relatively comprehensive, so the summary of the target text composed of these M sentences is usually a high-quality summary that can meet the reading needs of users, thereby improving user experience.
  • a second model can be trained through supervised learning.
  • the second model has a length control function of the summary and can be combined with the aforementioned first model to be trained to achieve end-to-end training, thereby obtaining a first model that is better adapted to the length control algorithm.
  • the first model can be obtained through an unsupervised learning training method.
  • the first model trained in this way can have an accurate scoring function and can accurately score each sentence of the target text, thereby accurately inferring the summary of the target text and further improving the user experience.
  • FIG13 is a schematic diagram of the structure of the summary generation device provided in the embodiment of the present application. As shown in FIG13, the device includes:
  • An acquisition module 1301 is used to acquire a target text, where the target text includes N sentences, where N ⁇ 2;
  • a scoring module 1302 is used to score the N sentences using the first model to obtain scores of the N sentences, where the scores of the N sentences are used to indicate the values of the N sentences in the target text;
  • a determination module 1303 is used to determine, based on the scores of the N sentences and the lengths of the N sentences, M sentences whose sum of scores is the largest and whose sum of lengths is less than a length threshold from the N sentences, N ⁇ M ⁇ 1;
  • the generating module 1304 is used to generate a summary of the target text based on the M sentences.
  • the N sentences after obtaining the target text containing N sentences, the N sentences can be scored by the first model to obtain the scores of the N sentences, and the scores of the N sentences are used to indicate the value of the N sentences in the target text. Then, based on the scores of the N sentences and the lengths of the N sentences, M sentences with the maximum sum of scores and the sum of lengths less than the length threshold can be selected from the N sentences. Finally, the selected M sentences can be composed of the summary of the target text.
  • M sentences can be further selected from the N sentences, and the sum of the scores of these M sentences is the largest, and the sum of the lengths of these M sentences is less than the length threshold of the summary. It can be seen that when generating the summary of the target text, not only the value of these M sentences in the target text is considered, but also the total length of these M sentences (i.e. the length of the summary of the target text) is considered. The factors considered are relatively comprehensive, so the summary of the target text composed of these M sentences is usually a high-quality summary that can meet the reading needs of users, thereby improving user experience.
  • the determination module 1303 is used to calculate the scores of N sentences and the lengths of the N sentences by using a target algorithm to determine M sentences from the N sentences whose sum of scores is the largest and whose sum of lengths is less than a length threshold.
  • the target algorithm is any one of the following: a dynamic programming algorithm, a backtracking method, a branch and bound method, and a greedy algorithm.
  • the determination module 1303 is used to process the scores of N sentences and the lengths of the N sentences through a second model to obtain processing results of the N sentences, and the processing results are used to determine, from the N sentences, M sentences whose sum of scores is the largest and whose sum of lengths is less than a length threshold.
  • the determination module 1303 is used to: perform linear operations on the scores of N sentences and the lengths of the N sentences through the second model to obtain a first representation of the N sentences; perform transformer-based processing on the first representation through the second model to obtain a second representation of the N sentences; perform linear operations on the second representation through the second model to obtain a third representation of the N sentences; and perform nonlinear operations on the third representation through the second model to obtain processing results of the N sentences.
  • the scoring module 1302 is used to: map N sentences through the first model to obtain fourth representations of the N sentences; and process the fourth representations based on the transformer model through the first model to obtain scores of the N sentences.
  • the target text is input by a user
  • the length threshold is set by the user.
  • FIG14 is a schematic diagram of a structure of a model training device provided in an embodiment of the present application. As shown in FIG14 , the device includes:
  • a first acquisition module 1401 is used to acquire a first text, where the first text includes P first sentences, where P ⁇ 2;
  • a scoring module 1402 is used to score the P first sentences using a first model to be trained to obtain scores of the P first sentences, where the scores of the P first sentences are used to indicate the values of the P first sentences in the first text;
  • a first processing module 1403 is used to process the scores of the P first sentences and the lengths of the P first sentences through a second model to obtain processing results of the P first sentences, where the processing results of the P first sentences are used to determine Q first sentences whose sum of scores is the largest and whose sum of lengths is less than a length threshold from the P first sentences, where P ⁇ Q ⁇ 1;
  • the first updating module 1404 is used to update the parameters of the first model to be trained based on the Q first sentences to obtain a first model.
  • the first model obtained by training in the embodiment of the present application has an automatic summary function. Specifically, after obtaining a target text containing N sentences, the N sentences can be scored by the first model to obtain the scores of the N sentences, and the scores of the N sentences are used to indicate the value of the N sentences in the target text. Then, based on the scores of the N sentences and the lengths of the N sentences, M sentences with the maximum sum of scores and the sum of lengths less than the length threshold can be selected from the N sentences. Finally, the selected M sentences can be composed of the summary of the target text.
  • M sentences can be further selected from the N sentences, and the sum of the scores of these M sentences is the maximum, and the sum of the lengths of these M sentences is less than the length threshold of the summary. It can be seen that when generating the summary of the target text, not only the value of these M sentences in the target text is considered, but also the total length of these M sentences (i.e. the length of the summary of the target text) is considered. The factors considered are relatively comprehensive, so the summary of the target text composed of these M sentences is usually a high-quality summary that can meet the reading needs of the user, thereby improving the user experience.
  • the device further includes: a second acquisition module, used to acquire a second text, the second text includes X second sentences, X ⁇ 2; a second processing module, used to process the true scores of the X second sentences and the lengths of the X second sentences through a second model to be trained to obtain processing results of the X second sentences, the processing results of the X second sentences are used to determine, from the X second sentences, Y second sentences with the maximum sum of scores and the sum of lengths less than a length threshold, X ⁇ Y ⁇ 1; a third acquisition module, used to acquire a target loss based on the true processing results of the X second sentences and the processing results of the X second sentences, the target loss is used to indicate the difference between the true processing results of the X second sentences and the processing results of the X second sentences; a second updating module, used to update the parameters of the second model to be trained based on the target loss until the model training condition is met to obtain the second model.
  • a second acquisition module used to acquire a second text, the second text
  • the first processing module 1403 is used to: perform linear operations on the scores of P first sentences and the lengths of the P first sentences through a second model to obtain first representations of the P first sentences; perform transformer-based processing on the first representation through the second model to obtain second representations of the P first sentences; perform linear operations on the second representation through the second model to obtain third representations of the P first sentences; and perform nonlinear operations on the third representation through the second model to obtain processing results of the P first sentences.
  • the scoring module 1402 is used to: map the P first sentences through the first model to be trained to obtain fourth representations of the P first sentences; and process the fourth representation based on the transformer model through the first model to be trained to obtain scores of the P first sentences.
  • the first update module 1404 is used to: obtain a representation of the first text through a third model to be trained; obtain the similarity between the fourth representation of Q first sentences and the representation of the first text through the third model to be trained; update the parameters of the first model to be trained and the parameters of the third model to be trained based on the similarity until the model training conditions are met, and obtain the first model and the third model respectively.
  • FIG. 15 is a schematic diagram of the structure of the execution device provided by the embodiment of the present application.
  • the execution device 1500 can be specifically manifested as a mobile phone, a tablet, a laptop computer, an intelligent wearable device, a server, etc., which is not limited here.
  • the execution device 1500 can be deployed with the summary generation device described in the embodiment corresponding to FIG. 13, which is used to implement the function of summary generation in the embodiment corresponding to FIG. 4.
  • the execution device 1500 includes: a receiver 1501, a transmitter 1502, a processor 1503 and a memory 1504 (wherein the number of processors 1503 in the execution device 1500 can be one or more, and FIG.
  • the processor 1503 may include an application processor 15031 and a communication processor 15032.
  • the receiver 1501, the transmitter 1502, the processor 1503 and the memory 1504 may be connected via a bus or other means.
  • the memory 1504 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1503. A portion of the memory 1504 may also include a non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the memory 1504 stores processors and operation instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operation instructions may include various operation instructions for implementing various operations.
  • the processor 1503 controls the operation of the execution device.
  • the various components of the execution device are coupled together through a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc.
  • the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc.
  • various buses are referred to as bus systems in the figure.
  • the method disclosed in the above embodiment of the present application can be applied to the processor 1503, or implemented by the processor 1503.
  • the processor 1503 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 1503 or the instruction in the form of software.
  • the above processor 1503 can be a general processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • the processor 1503 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present application.
  • the general processor can be a microprocessor or the processor can also be any conventional processor, etc.
  • the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor can be executed.
  • the software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc.
  • the storage medium is located in the memory 1504, and the processor 1503 reads the information in the memory 1504 and completes the steps of the above method in combination with its hardware.
  • the receiver 1501 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the execution device.
  • the transmitter 1502 can be used to output digital or character information through the first interface; the transmitter 1502 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1502 can also include a display device such as a display screen.
  • the processor 1503 is used to generate a summary of the target text through the first model and the second model (or the target algorithm) in the embodiment corresponding to Figure 4.
  • FIG. 16 is a structural schematic diagram of the training device provided by the embodiment of the present application.
  • the training device 1600 is implemented by one or more servers.
  • the training device 1600 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1614 (for example, one or more processors) and a memory 1632, and one or more storage media 1630 (for example, one or more mass storage devices) storing application programs 1642 or data 1644.
  • the memory 1632 and the storage medium 1630 can be short-term storage or permanent storage.
  • the program stored in the storage medium 1630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the training device. Furthermore, the central processor 1614 can be configured to communicate with the storage medium 1630 to execute a series of instruction operations in the storage medium 1630 on the training device 1600.
  • the training device 1600 may also include one or more power supplies 1626, one or more wired or wireless network interfaces 1650, one or more input and output interfaces 1658; or, one or more operating systems 1641, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
  • the training device can execute the model training method in the embodiment corresponding to Figure 8.
  • An embodiment of the present application also relates to a computer storage medium, in which a program for signal processing is stored.
  • the program When the program is run on a computer, the computer executes the steps executed by the aforementioned execution device, or the computer executes the steps executed by the aforementioned training device.
  • An embodiment of the present application also relates to a computer program product, which stores instructions, which, when executed by a computer, enable the computer to execute the steps executed by the aforementioned execution device, or enable the computer to execute the steps executed by the aforementioned training device.
  • the execution device, training device or terminal device provided in the embodiments of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface, a pin or a circuit, etc.
  • the processing unit may execute computer execution instructions stored in the storage unit, so that the chip in the execution device executes the data processing method described in the above embodiment, or so that the chip in the training device executes the data processing method described in the above embodiment.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc.
  • the storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types that can store static information and instructions. Static storage devices, random access memory (RAM), etc.
  • FIG. 17 is a schematic diagram of the structure of a chip provided in an embodiment of the present application.
  • the chip can be expressed as a neural network processor NPU 1700.
  • NPU 1700 is mounted on the host CPU (Host CPU) as a coprocessor, and tasks are assigned by the Host CPU.
  • the core part of the NPU is the operation circuit 1703, which is controlled by the controller 1704 to extract matrix data from the memory and perform multiplication operations.
  • the operation circuit 1703 includes multiple processing units (Process Engine, PE) inside.
  • the operation circuit 1703 is a two-dimensional systolic array.
  • the operation circuit 1703 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition.
  • the operation circuit 1703 is a general-purpose matrix processor.
  • the operation circuit takes the corresponding data of matrix B from the weight memory 1702 and caches it on each PE in the operation circuit.
  • the operation circuit takes the matrix A data from the input memory 1701 and performs matrix operation with matrix B, and the partial result or final result of the matrix is stored in the accumulator 1708.
  • Unified memory 1706 is used to store input data and output data. Weight data is directly transferred to weight memory 1702 through Direct Memory Access Controller (DMAC) 1705. Input data is also transferred to unified memory 1706 through DMAC.
  • DMAC Direct Memory Access Controller
  • BIU stands for Bus Interface Unit, that is, bus interface unit 1713, which is used for the interaction between AXI bus and DMAC and instruction fetch buffer (IFB) 1709.
  • IOB instruction fetch buffer
  • the bus interface unit 1713 (Bus Interface Unit, BIU for short) is used for the instruction fetch memory 1709 to obtain instructions from the external memory, and is also used for the storage unit access controller 1705 to obtain the original data of the input matrix A or the weight matrix B from the external memory.
  • BIU Bus Interface Unit
  • DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1706 or to transfer weight data to the weight memory 1702 or to transfer input data to the input memory 1701.
  • the vector calculation unit 1707 includes multiple operation processing units, which further process the output of the operation circuit 1703 when necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional/fully connected layer network calculations in neural networks, such as Batch Normalization, pixel-level summation, upsampling of predicted label planes, etc.
  • the vector calculation unit 1707 can store the processed output vector to the unified memory 1706.
  • the vector calculation unit 1707 can apply a linear function; or, a nonlinear function to the output of the operation circuit 1703, such as linear interpolation of the predicted label plane extracted by the convolution layer, and then, for example, a vector of accumulated values to generate an activation value.
  • the vector calculation unit 1707 generates a normalized value, a pixel-level summed value, or both.
  • the processed output vector can be used as an activation input to the operation circuit 1703, for example, for use in a subsequent layer in a neural network.
  • An instruction fetch buffer 1709 connected to the controller 1704, for storing instructions used by the controller 1704;
  • Unified memory 1706, input memory 1701, weight memory 1702 and instruction fetch memory 1709 are all on-chip memories. External memories are private to the NPU hardware architecture.
  • the processor mentioned in any of the above places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above program.
  • the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.
  • the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
  • the present application can be implemented by means of software plus necessary general-purpose hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc.
  • special hardware including special integrated circuits, special CPUs, special memories, special components, etc.
  • all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits.
  • software program implementation is a better implementation method in most cases.
  • the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art.
  • the computer software product is stored in a readable storage medium, such as a computer floppy disk, U disk, mobile hard disk, ROM, RAM, disk or CD, etc., and includes a number of instructions to enable
  • a computer device which may be a personal computer, a training device, or a network device, etc. executes the methods described in each embodiment of the present application.
  • all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof.
  • all or part of the embodiments may be implemented in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center.
  • the computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations.
  • the available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.
  • a magnetic medium e.g., a floppy disk, a hard disk, a tape
  • an optical medium e.g., a DVD
  • a semiconductor medium e.g., a solid-state drive (SSD)

Landscapes

  • Engineering & Computer Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Machine Translation (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)

Abstract

本申请公开了一种摘要生成方法及其相关设备，可自动生成满足长度限制的目标文本的摘要，且所生成的目标文本的摘要通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。本申请的方法包括：获取目标文本，目标文本包含N个句子，N≥2；通过抽取式摘要模型对N个句子进行打分，得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值；基于N个句子的分数以及N个句子的长度，从N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子，N≥M≥1；基于M个句子生成目标文本的摘要。此外，本申请还提供了一种可微的摘要长度控制模块，且该模块可结合待训练的抽取式摘要模型实现端到端的模型训练。

Description

一种摘要生成方法及其相关设备
本申请要求于2022年11月29日提交国家知识产权局、申请号为202211534548.5、发明名称为“一种摘要生成方法及其相关设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及人工智能(artificial intelligence，AI)技术领域，尤其涉及一种摘要生成方法及其相关设备。
背景技术
自动摘要技术是利用计算机自动实现文本分析和内容归纳以自动生成摘要的技术，即按用户的要求以简洁的形式表达文本的主要内容，自动摘要技术能够有效地帮助用户从检索到的文章中寻找用户感兴趣的内容，提高阅读速度和质量。
目前，可通过AI技术中的神经网络模型，来对文本进行处理，从而抽取出文本的摘要。具体地，该神经网络模型可先对文本的多个句子进行映射，从而得到多个句子的表征，再对多个句子的表征进行基于多头注意力机制的处理，从而得到多个句子的分数，这些分数可表示多个句子在文本中的价值。如此一来，可将分数较大的若干个句子，组成文本的摘要。
然而，在生成摘要的过程中，仅考虑了句子在文本中的价值，所考虑的因素较为单一，这样会导致最终生成的文本的摘要并非是质量较高的摘要，无法满足用户的阅读需求，降低了用户体验。
发明内容
本申请实施例提供了一种摘要生成方法及其相关设备，可自动生成满足长度限制的目标文本的摘要，且所生成的目标文本的摘要通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
本申请实施例的第一方面提供了一种摘要生成方法，该方法包括：
当用户需要获取目标文本的摘要时，可在用户设备中输入或选择目标文本，以使得用户设备确定需要生成目标文本的摘要。其中，目标文本由N个句子构成，且N个句子的长度是已知的，N为大于或等于2的正整数。
得到目标文本后，用户设备可将目标文本的N个句子输入至第一模型，以通过第一模型对目标文本的N个句子进行打分，从而得到目标文本的N个句子的分数，在目标文本的N个句子中，N个句子的分数用于指示N个句子在目标文件中的价值。
得到目标文本的N个句子的得分后，用户设备可将目标文本的N个句子的得分以及目标文本的N个句子的长度作为背包问题的初始值，并对背包问题进行求解，所求得的解即为目标文本的N个句子中，分数之和最大且长度之和小于摘要的长度阈值的M个句子，M为大于或等于1的正整数，且M小于或等于N。
得到分数之和最大且长度之和小于摘要的长度阈值的M个句子后，用户设备可将M个句子组成目标文本的摘要，并在展示给用户浏览和使用。
从上述方法可以看出：在获取包含N个句子的目标文本后，可通过第一模型对N个句子进行打分，从而得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值。然后，可基于N个句子的分数以及N个句子的长度，从N个句子中选择分数之和最大且长度之和小于长度阈值的M个句子。最后，可将选择的M个句子组成目标文本的摘要。前述生成目标文本的摘要的过程中，在确定目标文本中N个句子的分数后，可进一步从N个句子中挑选出M个句子，这M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。由此可见，在生成目标文本的摘要时，不仅考虑了这M个句子在目标文本中的价值，还考虑了这M个句子的总长度(即目标文本的摘要的长度)，所考虑的因素较为全面，故由这M个句子所构成的目标文本的摘要，通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
在一种可能实现的方式中，基于N个句子的分数以及N个句子的长度，从N个句子中，确定分数之和最大且长度之和小于摘要长度的M个句子包括：通过目标算法对N个句子的分数以及N个句子的长度进行计算，以从N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子。前述实现方式中，用户设备可获取目标算法，并将目标文本的N个句子的分数以及目标文本的N个句子的长度作为目标算法的初始值，将分数之和最大且长度之和小于摘要的长度阈值作为目标算法的优化目标。然后，用户设备可运行目标算法，以使得目标算法对目标文本的N个句子的分数以及目标文本的N个句子的长度进行优化计算，从而在目标文本的N个句子中，求解出分数之和最大且长度之和小于摘要的长度阈值的M个句子。
在一种可能实现的方式中，目标算法为以下任意一种：动态规划算法、回溯法、分支限界法以及贪心算法，贪心算法为以下任意一种：基于价值密度的贪心算法、基于价值的贪心算法以及基于大小的贪心算法等等。
在一种可能实现的方式中，基于N个句子的分数以及N个句子的长度，从N个句子中，确定分数之和最大且长度之和小于摘要长度的M个句子包括：通过第二模型对N个句子的分数以及N个句子的长度进行处理，得到N个句子的处理结果，处理结果用于从N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子。前述实现方式中，用户设备可获取近似于目标算法的第二模型，并将目标文本的N个句子的分数以及目标文本的N个句子的长度输入至第二模型，以通过第二模型对目标文本的N个句子的分数以及目标文本的N个句子的长度进行处理，从而得到并输出目标文本的N个句子的处理结果，该处理结果通常包含目标文本的N个句子被选择与否的指标，故该处理结果可用于从N个句子中，确定分数之和最大且长度之和小于摘要的长度阈值的M个句子。
在一种可能实现的方式中，通过第二模型对N个句子的分数以及N个句子的长度进行处理，N个句子的处理结果包括：通过第二模型对N个句子的分数以及N个句子的长度进行线性运算，得到N个句子的第一表征；通过第二模型对第一表征进行基于transformer模型的处理，得到N个句子的第二表征；通过第二模型对第二表征进行线性运算，得到N个句子的第三表征；通过第二模型对第三表征进行非线性运算，得到N个句子的处理结果。前述实现方式中，用户设备可将目标文本的N个句子的分数以及目标文本的N个句子的长度输入至第二模型，以使得第二模型将目标文本的N个句子的分数以及目标文本的N个句子的长度进行拼接，再对拼接得到的结果进行线性运算，从而得到目标文本的N个句子的第一表征。得到目标文本的N个句子的第一表征后，第二模型可对目标文本的N个句子的第一表征依次进行基于多头注意力机制的处理、前馈网络的处理以及残差网络的处理等等(即基于transformer模型的处理)，从而得到目标文本的N个句子的第二表征。得到目标文本的N个句子的第二表征后，第二模型可对目标文本的N个句子的第二表征进行线性运算，从而得到目标文本的N个句子的第三表征。得到目标文本的N个句子的第三表征后，第二模型可对得到目标文本的N个句子的第三表征进行非线性运算，从而得到并输出目标文本的N个句子的处理结果，该处理结果通常包含目标文本的N个句子被选择与否的指标，故用户设备可基于该处理结果，从N个句子中选出M个句子，所选出的M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。
在一种可能实现的方式中，通过第一模型对N个句子进行打分，得到N个句子的分数包括：通过第一模型对N个句子进行映射，得到N个句子的第四表征；通过第一模型对第四表征进行基于transformer模型的处理，得到N个句子的分数。前述实现方式中，得到目标文本后，用户设备可将目标文本的N个句子输入至第一模型，以使得第一模型对目标文本的N个句子进行映射，从而得到目标文本的N个句子的第四表征。得到目标文本的N个句子的第四表征后，第一模型对目标文本的N个句子的第四表征依次进行基于多头注意力机制的处理、前馈网络的处理以及残差网络的处理等等(即基于transformer模型的处理)，从而得到并输出目标文本的N个句子的分数。至此，用户设备可成功得到目标文本的N个句子的得分。
在一种可能实现的方式中，目标文本由用户输入至用户设备，且摘要的长度阈值可由用户在用户设备提供的显示界面上完成设置。
本申请实施例的第二方面提供了一种模型训练方法，该方法包括：
当需要对第一待训练模型进行训练时，训练设备可先获取第一训练数据集中的一批训练数据，该批 训练数据包含第一文本，第一文本包含P个第一句子，P个第一句子的长度是已知的，P为大于或等于2的正整数。
得到第一文本后，训练设备可将第一文本的P个第一句子输入至第一待训练模型，以通过第一待训练模型对第一文本的P个第一句子进行打分，从而得到第一文本的P个第一句子的分数，在第一文本的P个第一句子中，P个第一句子的分数用于指示P个第一句子在第一文件中的价值。
得到第一文本的P个第一句子的得分后，训练设备还可获取第一文本的P个第一句子的长度以及近似于目标算法的第二模型。然后，训练设备可将第一文本的P个第一句子的分数以及第一文本的P个第一句子的长度输入至第二模型，以通过第二模型对第一文本的P个第一句子的分数以及第一文本的P个第一句子的长度进行处理，从而得到并输出第一文本的P个第一句子的处理结果，该处理结果通常包含第一文本的P个第一句子被选择与否的指标，故该处理结果可用于从P个第一句子中，确定分数之和最大且长度之和小于摘要的长度阈值的Q个第一句子，Q为大于或等于1的正整数，且Q小于或等于P。
得到分数之和最大且长度之和小于摘要的长度阈值的Q个第一句子后，训练设备可基于这Q个第一句子，来更新第一待训练模型的参数，从而得到第一模型。
上述方法训练得到的第一模型，具备自动摘要功能。具体地，在获取包含N个句子的目标文本后，可通过第一模型对N个句子进行打分，从而得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值。然后，可基于N个句子的分数以及N个句子的长度，从N个句子中选择分数之和最大且长度之和小于长度阈值的M个句子。最后，可将选择的M个句子组成目标文本的摘要。前述生成目标文本的摘要的过程中，在确定目标文本中N个句子的分数后，可进一步从N个句子中挑选出M个句子，这M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。由此可见，在生成目标文本的摘要时，不仅考虑了这M个句子在目标文本中的价值，还考虑了这M个句子的总长度(即目标文本的摘要的长度)，所考虑的因素较为全面，故由这M个句子所构成的目标文本的摘要，通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
在一种可能实现的方式中，该方法还包括：获取第二文本，第二文本包含X个第二句子，X≥2；通过第二待训练模型对X个第二句子的真实分数以及X个第二句子的长度进行处理，得到X个第二句子的处理结果，X个第二句子的处理结果用于从X个第二句子中，确定分数之和最大且长度之和小于长度阈值的Y个第二句子，X≥Y≥1；基于X个第二句子的真实处理结果以及X个第二句子的处理结果，获取目标损失，目标损失用于指示X个第二句子的真实处理结果以及X个第二句子的处理结果之间的差异；基于目标损失，更新第二待训练模型的参数，直至满足模型训练条件，得到第二模型。前述实现方式中，当需要对第二待训练模型进行训练时，训练设备可先获取第二训练数据集中的一批训练数据，该批训练数据包含第二文本，第二文本包含X个第二句子(X为大于或等于2的正整数)，值得注意的是，每个第二句子的真实分数是已知的，每个句子被选择的真实概率也是已知的，即X个第二句子的真实处理结果是已知的，且每个第二句子的长度也是已知的。得到第二文本后，训练设备可将第二文本的X个第二句子的真实分数以及第二文本的X个第二句子的长度输入至第二待训练模型，以通过第二待训练模型对第二文本的X个第二句子的真实分数以及长度进行处理，从而得到第二文本的X个第二句子的处理结果，该处理结果通常包含第二文本的X个第二句子被选择与否的指标，故该处理结果可用于用于从X个第二句子中，确定分数之和最大且长度之和小于长度阈值的Y个第二句子(Y为大于或等于1的正整数，且Y小于或等于X)。得到第二文本的X个第二句子的处理结果后，由于第二文本的X个第二句子的真实处理结果已知，故训练设备可利用预置的损失函数，对第二文本的X个第二句子的处理结果以及第二文本的X个第二句子的真实处理结果进行计算，从而得到目标损失，目标损失用于指示第二文本的X个第二句子的处理结果以及第二文本的X个第二句子的真实处理结果之间的差异。得到目标损失后，训练设备可利用目标损失，来更新第二待训练模型的参数，并利用第二训练数据集中的另一批训练数据，继续对更新参数后的第二待训练模型进行训练，直至满足模型训练条件，从而得到第二模型。
在一种可能实现的方式中，通过第二模型对P个第一句子的分数以及P个第一句子的长度进行处理，得到P个第一句子的处理结果包括：通过第二模型对P个第一句子的分数以及P个第一句子的长度进行线性运算，得到P个第一句子的第一表征；通过第二模型对第一表征进行基于transformer模型的处理，得到P个第一句子的第二表征；通过第二模型对第二表征进行线性运算，得到P个第一句子的第三表征； 通过第二模型对第三表征进行非线性运算，得到P个第一句子的处理结果。
在一种可能实现的方式中，通过第一待训练模型对P个第一句子进行打分，得到P个第一句子的分数包括：通过第一待训练模型对P个第一句子进行映射，得到P个第一句子的第四表征；通过第一待训练模型对第四表征进行基于transformer模型的处理，得到P个第一句子的分数。
在一种可能实现的方式中，基于Q个第一句子，更新第一待训练模型的参数，得到第一模型包括：通过第三待训练模型获取第一文本的表征；通过第三待训练模型获取Q个第一句子的第四表征与第一文本的表征之间的相似度；基于相似度更新第一待训练模型的参数以及第三待训练模型的参数，直至满足模型训练条件，分别得到第一模型和第三模型。前述实现方式中，得到第一文本的P个第一句子的处理结果后，训练设备可将第一文本的P个第一句子的处理结果与第一文本的P个第一句子的第四表征进行相乘，从而在第一文本的P个第一句子的第四表征中，筛选出分数之和最大且长度之和小于摘要的长度阈值的Q个第一句子的第四表征。得到这Q个第一句子的第四表征后，训练设备可将这Q个第一句子的第四表征以及第一文本输入至第三待训练模型。那么，第三待训练可先对第一文本进行映射，从而得到第一文本的表征，再对这Q个第一句子的第四表征与第一文本的表征进行一系列的处理，从而得到这Q个第一句子的第四表征与第一文本的表征之间的相似度。得到这Q个第一句子的第四表征与第一文本的表征之间的相似度后，训练设备可基于该相似度，来更新第一待训练模型的参数以及第三待训练模型的参数，并利用第一训练数据集中的另一批训练数据，继续对更新参数后的第一待训练模型以及更新参数后的第三待训练模型进行训练，直至满足模型训练条件，从而得到第三模型以及图4所示实施例中的第一模型。
本申请实施例的第三方面提供了一种摘要生成装置，该装置包括：获取模块，用于获取目标文本，目标文本包含N个句子，N≥2；打分模块，用于通过第一模型对N个句子进行打分，得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值；确定模块，用于基于N个句子的分数以及N个句子的长度，从N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子，N≥M≥1；生成模块，用于基于M个句子生成目标文本的摘要。
从上述装置可以看出：在获取包含N个句子的目标文本后，可通过第一模型对N个句子进行打分，从而得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值。然后，可基于N个句子的分数以及N个句子的长度，从N个句子中选择分数之和最大且长度之和小于长度阈值的M个句子。最后，可将选择的M个句子组成目标文本的摘要。前述生成目标文本的摘要的过程中，在确定目标文本中N个句子的分数后，可进一步从N个句子中挑选出M个句子，这M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。由此可见，在生成目标文本的摘要时，不仅考虑了这M个句子在目标文本中的价值，还考虑了这M个句子的总长度(即目标文本的摘要的长度)，所考虑的因素较为全面，故由这M个句子所构成的目标文本的摘要，通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
在一种可能实现的方式中，确定模块，用于通过目标算法对N个句子的分数以及N个句子的长度进行计算，以从N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子。
在一种可能实现的方式中，目标算法为以下任意一种：动态规划算法、回溯法、分支限界法以及贪心算法。
在一种可能实现的方式中，确定模块，用于通过第二模型对N个句子的分数以及N个句子的长度进行处理，得到N个句子的处理结果，处理结果用于从N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子。
在一种可能实现的方式中，确定模块，用于：通过第二模型对N个句子的分数以及N个句子的长度进行线性运算，得到N个句子的第一表征；通过第二模型对第一表征进行基于transformer模型的处理，得到N个句子的第二表征；通过第二模型对第二表征进行线性运算，得到N个句子的第三表征；通过第二模型对第三表征进行非线性运算，得到N个句子的处理结果。
在一种可能实现的方式中，打分模块，用于：通过第一模型对N个句子进行映射，得到N个句子的第四表征；通过第一模型对第四表征进行基于transformer模型的处理，得到N个句子的分数。
在一种可能实现的方式中，目标文本由用户输入，长度阈值由用户设置。
本申请实施例的第四方面提供了一种模型训练装置，该装置包括：第一获取模块，用于获取第一文本，第一文本包含P个第一句子，P≥2；打分模块，用于通过第一待训练模型对P个第一句子进行打分，得到P个第一句子的分数，P个第一句子的分数用于指示P个第一句子在第一文本中的价值；第一处理模块，用于通过第二模型对P个第一句子的分数以及P个第一句子的长度进行处理，得到P个第一句子的处理结果，P个第一句子的处理结果用于从P个第一句子中，确定分数之和最大且长度之和小于长度阈值的Q个第一句子，P≥Q≥1；第一更新模块，用于基于Q个第一句子，更新第一待训练模型的参数，得到第一模型。
上述装置训练得到的第一模型，具备自动摘要功能。具体地，在获取包含N个句子的目标文本后，可通过第一模型对N个句子进行打分，从而得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值。然后，可基于N个句子的分数以及N个句子的长度，从N个句子中选择分数之和最大且长度之和小于长度阈值的M个句子。最后，可将选择的M个句子组成目标文本的摘要。前述生成目标文本的摘要的过程中，在确定目标文本中N个句子的分数后，可进一步从N个句子中挑选出M个句子，这M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。由此可见，在生成目标文本的摘要时，不仅考虑了这M个句子在目标文本中的价值，还考虑了这M个句子的总长度(即目标文本的摘要的长度)，所考虑的因素较为全面，故由这M个句子所构成的目标文本的摘要，通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
在一种可能实现的方式中，该装置还包括：第二获取模块，用于获取第二文本，第二文本包含X个第二句子，X≥2；第二处理模块，用于通过第二待训练模型对X个第二句子的真实分数以及X个第二句子的长度进行处理，得到X个第二句子的处理结果，X个第二句子的处理结果用于从X个第二句子中，确定分数之和最大且长度之和小于长度阈值的Y个第二句子，X≥Y≥1；第三获取模块，用于基于X个第二句子的真实处理结果以及X个第二句子的处理结果，获取目标损失，目标损失用于指示X个第二句子的真实处理结果以及X个第二句子的处理结果之间的差异；第二更新模块，用于基于目标损失，更新第二待训练模型的参数，直至满足模型训练条件，得到第二模型。
在一种可能实现的方式中，第一处理模块，用于：通过第二模型对P个第一句子的分数以及P个第一句子的长度进行线性运算，得到P个第一句子的第一表征；通过第二模型对第一表征进行基于transformer模型的处理，得到P个第一句子的第二表征；通过第二模型对第二表征进行线性运算，得到P个第一句子的第三表征；通过第二模型对第三表征进行非线性运算，得到P个第一句子的处理结果。
在一种可能实现的方式中，打分模块，用于：通过第一待训练模型对P个第一句子进行映射，得到P个第一句子的第四表征；通过第一待训练模型对第四表征进行基于transformer模型的处理，得到P个第一句子的分数。
在一种可能实现的方式中，第一更新模块，用于：通过第三待训练模型获取第一文本的表征；通过第三待训练模型获取Q个第一句子的第四表征与第一文本的表征之间的相似度；基于相似度更新第一待训练模型的参数以及第三待训练模型的参数，直至满足模型训练条件，分别得到第一模型和第三模型。
本申请实施例的第五方面提供了一种摘要生成装置，该装置包括存储器和处理器；存储器存储有代码，处理器被配置为执行代码，当代码被执行时，摘要生成装置执行如第一方面或第一方面中任意一种可能的实现方式所述的方法。
本申请实施例的第六方面提供了一种模型训练装置，该装置包括存储器和处理器；存储器存储有代码，处理器被配置为执行代码，当代码被执行时，模型训练装置执行如第二方面或第二方面中任意一种可能的实现方式所述的方法。
本申请实施例的第七方面提供了一种电路***，该电路***包括处理电路，该处理电路配置为执行如第一方面、第一方面中的任意一种可能的实现方式、第二方面或第二方面中任意一种可能的实现方式所述的方法。
本申请实施例的第八方面提供了一种芯片***，该芯片***包括处理器，用于调用存储器中存储的计算机程序或计算机指令，以使得该处理器执行如第一方面、第一方面中的任意一种可能的实现方式、第二方面或第二方面中任意一种可能的实现方式所述的方法。
在一种可能的实现方式中，该处理器通过接口与存储器耦合。
在一种可能的实现方式中，该芯片***还包括存储器，该存储器中存储有计算机程序或计算机指令。
本申请实施例的第九方面提供了一种计算机存储介质，该计算机存储介质存储有计算机程序，该程序在由计算机执行时，使得计算机实施如第一方面、第一方面中的任意一种可能的实现方式、第二方面或第二方面中任意一种可能的实现方式所述的方法。
本申请实施例的第十方面提供了一种计算机程序产品，该计算机程序产品存储有指令，该指令在由计算机执行时，使得计算机实施如第一方面、第一方面中的任意一种可能的实现方式、第二方面或第二方面中任意一种可能的实现方式所述的方法。
本申请实施例中，在获取包含N个句子的目标文本后，可通过第一模型对N个句子进行打分，从而得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值。然后，可基于N个句子的分数以及N个句子的长度，从N个句子中选择分数之和最大且长度之和小于长度阈值的M个句子。最后，可将选择的M个句子组成目标文本的摘要。前述生成目标文本的摘要的过程中，在确定目标文本中N个句子的分数后，可进一步从N个句子中挑选出M个句子，这M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。由此可见，在生成目标文本的摘要时，不仅考虑了这M个句子在目标文本中的价值，还考虑了这M个句子的总长度(即目标文本的摘要的长度)，所考虑的因素较为全面，故由这M个句子所构成的目标文本的摘要，通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
附图说明
图1为人工智能主体框架的一种结构示意图；
图2a为本申请实施例提供的文本处理***的一个结构示意图；
图2b为本申请实施例提供的文本处理***的另一结构示意图；
图2c为本申请实施例提供的文本处理的相关设备的一个示意图；
图3为本申请实施例提供的***100架构的一个示意图；
图4为本申请实施例提供的摘要生成方法的一个流程示意图；
图5为本申请实施例提供的用户与用户设备交互的一个示意图；
图6为本申请实施例提供的第一模型的一个结构示意图；
图7为本申请实施例提供的第二模型的一个结构示意图；
图8为本申请实施例提供的模型训练方法的一个流程示意图；
图9为本申请实施例提供的第三待训练模型的一个结构示意图；
图10为本申请实施例提供的第三待训练模型的另一结构示意图；
图11为本申请实施例提供的第三待训练模型的另一结构示意图；
图12为本申请实施例提供的第三待训练模型的另一结构示意图；
图13为本申请实施例提供的摘要生成装置的一个结构示意图；
图14为本申请实施例提供的模型训练装置的一个结构示意图；
图15为本申请实施例提供的执行设备的一个结构示意图；
图16为本申请实施例提供的训练设备的一个结构示意图；
图17为本申请实施例提供的芯片的一个结构示意图。
具体实施方式
本申请实施例提供了一种摘要生成方法及其相关设备，可自动生成满足长度限制的目标文本的摘要，且所生成的目标文本的摘要通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，这仅仅是描述本申请的实施例中对相同属性的对象在描述时所采用的区分方式。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，以便包含一系列单元的过程、方法、***、产品或设备不必限于那些单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它单 元。
自动摘要技术是利用计算机自动实现文本分析和内容归纳以自动生成摘要的技术，即按用户的要求以简洁的形式表达文本的主要内容，自动摘要技术能够有效地帮助用户从检索到的文章中寻找用户感兴趣的内容，提高阅读速度和质量。
目前，自动摘要技术可通过AI技术中的神经网络模型来实现，即通过神经网络模型对文本进行处理，从而抽取出文本的摘要。具体地，该神经网络模型可先对文本的多个句子进行映射，从而得到多个句子的表征，再对多个句子的表征进行基于多头注意力机制的处理，从而得到多个句子的分数，在多个句子中，任意一个句子的分数可表示该句子在文本中的价值。如此一来，可将分数较大的若干个句子(即价值较高的若干个句子)，组成文本的摘要。
然而，在上述生成摘要的过程中，仅考虑了句子在文本中的价值，所考虑的因素较为单一，这样会导致最终生成的文本的摘要并非是质量较高的摘要，无法满足用户的阅读需求，降低了用户体验。
进一步地，目前具备摘要生成功能的神经网络模型大多通过有监督学习的训练方式得到的，难以通过无监督学习的训练方式来获取此类神经网络模型。基于此，如何通过无监督学习的训练方式来获取具备摘要生成功能的神经网络模型，成为了亟待解决的问题。
更进一步地，在生成文本的摘要的过程中，用户通常仅需完成输入文本的操作，用户的参与度过低。如何令用户更多地参与到摘要生成的过程中，即令用户与终端设备更多地实现交互，从而满足用户对摘要的自定义需求，也成为了亟待解决的问题。
为了解决上述问题，本申请实施例提供了一种摘要生成方法，该方法可结合人工智能(artificial intelligence，AI)技术实现。AI技术是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能的技术学科，AI技术通过感知环境、获取知识并使用知识获得最佳结果。换句话说，人工智能技术是计算机科学的一个分支，它企图了解智能的实质，并生产出一种新的能以人类智能相似的方式做出反应的智能机器。利用人工智能进行数据处理是人工智能常见的一个应用方式。
首先对人工智能***总体工作流程进行描述，请参见图1，图1为人工智能主体框架的一种结构示意图，下面从“智能信息链”(水平轴)和“IT价值链”(垂直轴)两个维度对上述人工智能主题框架进行阐述。其中，“智能信息链”反映从数据的获取到处理的一列过程。举例来说，可以是智能信息感知、智能信息表示与形成、智能推理、智能决策、智能执行与输出的一般过程。在这个过程中，数据经历了“数据—信息—知识—智慧”的凝练过程。“IT价值链”从人智能的底层基础设施、信息(提供和处理技术实现)到***的产业生态过程，反映人工智能为信息技术产业带来的价值。
(1)基础设施
基础设施为人工智能***提供计算能力支持，实现与外部世界的沟通，并通过基础平台实现支撑。通过传感器与外部沟通；计算能力由智能芯片(CPU、NPU、GPU、ASIC、FPGA等硬件加速芯片)提供；基础平台包括分布式计算框架及网络等相关的平台保障和支持，可以包括云存储和计算、互联互通网络等。举例来说，传感器和外部沟通获取数据，这些数据提供给基础平台提供的分布式计算***中的智能芯片进行计算。
(2)数据
基础设施的上一层的数据用于表示人工智能领域的数据来源。数据涉及到图形、图像、语音、文本，还涉及到传统设备的物联网数据，包括已有***的业务数据以及力、位移、液位、温度、湿度等感知数据。
(3)数据处理
数据处理通常包括数据训练，机器学习，深度学习，搜索，推理，决策等方式。
其中，机器学习和深度学习可以对数据进行符号化和形式化的智能信息建模、抽取、预处理、训练等。
推理是指在计算机或智能***中，模拟人类的智能推理方式，依据推理控制策略，利用形式化的信息进行机器思维和求解问题的过程，典型的功能是搜索与匹配。
决策是指智能信息经过推理后进行决策的过程，通常提供分类、排序、预测等功能。
(4)通用能力
对数据经过上面提到的数据处理后，进一步基于数据处理的结果可以形成一些通用的能力，比如可以是算法或者一个通用***，例如，翻译，文本的分析，计算机视觉的处理，语音识别，图像的识别等等。
(5)智能产品及行业应用
智能产品及行业应用指人工智能***在各领域的产品和应用，是对人工智能整体解决方案的封装，将智能信息决策产品化、实现落地应用，其应用领域主要包括：智能终端、智能交通、智能医疗、自动驾驶、智慧城市等。
接下来介绍几种本申请的应用场景。
图2a为本申请实施例提供的文本处理***的一个结构示意图，该文本处理***包括用户设备以及数据处理设备。其中，用户设备包括手机、个人电脑或者信息处理中心等智能终端。用户设备通常为文本处理的发起端，作为文本处理的发起方，通常由用户通过用户设备发起请求。
上述数据处理设备可以是云服务器、网络服务器、应用服务器以及管理服务器等具有数据处理功能的设备或服务器。数据处理设备通过交互接口接收来自智能终端的文本处理请求，再通过存储数据的存储器以及数据处理的处理器环节进行机器学习，深度学习，搜索，推理，决策等方式的文本处理。数据处理设备中的存储器可以是一个统称，包括本地存储以及存储历史数据的数据库，数据库可以在数据处理设备上，也可以在其它网络服务器上。
在图2a所示的文本处理***中，用户设备可以接收用户的指令，例如，用户设备可以获取用户输入/选择的一个目标文本，然后向数据处理设备发起包含目标文本的处理请求，以使得数据处理设备针对用户设备得到的目标文本执行文本处理应用，从而得到目标文本的摘要。示例性的，用户设备可以获取用户输入的一个目标文本(目标文本既可以是长文本，也可以是短文本)，然后向数据处理设备发起针对目标文本的处理请求，使得数据处理设备对目标文本进行基于自动摘要技术的处理，从而得到目标文本的摘要。
在图2a中，数据处理设备可以执行本申请实施例的摘要生成方法。
图2b为本申请实施例提供的文本处理***的另一结构示意图，在图2b中，用户设备直接作为数据处理设备，该用户设备能够直接获取来自用户的输入并直接由用户设备本身的硬件进行处理，具体过程与图2a相似，可参考上面的描述，在此不再赘述。
在图2b所示的文本处理***中，用户设备可以接收用户的指令，例如，用户设备可以获取用户在用户设备中所选择的一个目标文本(目标文本既可以是长文本，也可以是短文本)，然后再由用户设备自身针对目标文本执行文本处理应用(相当于进行基于自动摘要技术的处理)，从而得到目标文本的摘要。
在图2b中，用户设备自身就可以执行本申请实施例的摘要生成方法。
图2c为本申请实施例提供的文本处理的相关设备的一个示意图。
上述图2a和图2b中的用户设备具体可以是图2c中的本地设备301或者本地设备302，图2a中的数据处理设备具体可以是图2c中的执行设备210，其中，数据存储***250可以存储执行设备210的待处理数据，数据存储***250可以集成在执行设备210上，也可以设置在云上或其它网络服务器上。
图2a和图2b中的处理器可以通过神经网络模型或者其它模型(例如，基于支持向量机的模型)进行数据训练/机器学习/深度学习，并利用数据最终训练或者学习得到的模型针对文本执行文本处理应用，从而得到相应的处理结果。
图3为本申请实施例提供的***100架构的一个示意图，在图3中，执行设备110配置输入/输出(input/output，I/O)接口112，用于与外部设备进行数据交互，用户可以通过客户设备140向I/O接口112输入数据，所述输入数据在本申请实施例中可以包括：各个待调度任务、可调用资源以及其他参数。
在执行设备110对输入数据进行预处理，或者在执行设备110的计算模块111执行计算等相关的处理(比如进行本申请中神经网络的功能实现)过程中，执行设备110可以调用数据存储***150中的数据、代码等以用于相应的处理，也可以将相应处理得到的数据、指令等存入数据存储***150中。
最后，I/O接口112将处理结果返回给客户设备140，从而提供给用户。
值得说明的是，训练设备120可以针对不同的目标或称不同的任务，基于不同的训练数据生成相应的目标模型/规则，该相应的目标模型/规则即可以用于实现上述目标或完成上述任务，从而为用户提供所需的结果。其中，训练数据可以存储在数据库130中，且来自于数据采集设备160采集的训练样本。
在图3中所示情况下，用户可以手动给定输入数据，该手动给定可以通过I/O接口112提供的界面进行操作。另一种情况下，客户设备140可以自动地向I/O接口112发送输入数据，如果要求客户设备140自动发送输入数据需要获得用户的授权，则用户可以在客户设备140中设置相应权限。用户可以在客户设备140查看执行设备110输出的结果，具体的呈现形式可以是显示、声音、动作等具体方式。客户设备140也可以作为数据采集端，采集如图所示输入I/O接口112的输入数据及输出I/O接口112的输出结果作为新的样本数据，并存入数据库130。当然，也可以不经过客户设备140进行采集，而是由I/O接口112直接将如图所示输入I/O接口112的输入数据及输出I/O接口112的输出结果，作为新的样本数据存入数据库130。
值得注意的是，图3仅是本申请实施例提供的一种***架构的示意图，图中所示设备、器件、模块等之间的位置关系不构成任何限制，例如，在图3中，数据存储***150相对执行设备110是外部存储器，在其它情况下，也可以将数据存储***150置于执行设备110中。如图3所示，可以根据训练设备120训练得到神经网络。
本申请实施例还提供的一种芯片，该芯片包括神经网络处理器NPU。该芯片可以被设置在如图3所示的执行设备110中，用以完成计算模块111的计算工作。该芯片也可以被设置在如图3所示的训练设备120中，用以完成训练设备120的训练工作并输出目标模型/规则。
神经网络处理器NPU，NPU作为协处理器挂载到主中央处理器(central processing unit，CPU)(host CPU)上，由主CPU分配任务。NPU的核心部分为运算电路，控制器控制运算电路提取存储器(权重存储器或输入存储器)中的数据并进行运算。
在一些实现中，运算电路内部包括多个处理单元(process engine,PE)。在一些实现中，运算电路是二维脉动阵列。运算电路还可以是一维脉动阵列或者能够执行例如乘法和加法这样的数学运算的其它电子线路。在一些实现中，运算电路是通用的矩阵处理器。
举例来说，假设有输入矩阵A，权重矩阵B，输出矩阵C。运算电路从权重存储器中取矩阵B相应的数据，并缓存在运算电路中每一个PE上。运算电路从输入存储器中取矩阵A数据与矩阵B进行矩阵运算，得到的矩阵的部分结果或最终结果，保存在累加器(accumulator)中。
向量计算单元可以对运算电路的输出做进一步处理，如向量乘，向量加，指数运算，对数运算，大小比较等等。例如，向量计算单元可以用于神经网络中非卷积/非FC层的网络计算，如池化(pooling)，批归一化(batch normalization)，局部响应归一化(local response normalization)等。
在一些实现种，向量计算单元能将经处理的输出的向量存储到统一缓存器。例如，向量计算单元可以将非线性函数应用到运算电路的输出，例如累加值的向量，用以生成激活值。在一些实现中，向量计算单元生成归一化的值、合并值，或二者均有。在一些实现中，处理过的输出的向量能够用作到运算电路的激活输入，例如用于在神经网络中的后续层中的使用。
统一存储器用于存放输入数据以及输出数据。
权重数据直接通过存储单元访问控制器(direct memory access controller，DMAC)将外部存储器中的输入数据搬运到输入存储器和/或统一存储器、将外部存储器中的权重数据存入权重存储器，以及将统一存储器中的数据存入外部存储器。
总线接口单元(bus interface unit，BIU)，用于通过总线实现主CPU、DMAC和取指存储器之间进行交互。
与控制器连接的取指存储器(instruction fetch buffer)，用于存储控制器使用的指令；
控制器，用于调用指存储器中缓存的指令，实现控制该运算加速器的工作过程。
一般地，统一存储器，输入存储器，权重存储器以及取指存储器均为片上(On-Chip)存储器，外部存储器为该NPU外部的存储器，该外部存储器可以为双倍数据率同步动态随机存储器(double data rate synchronous dynamic random access memory，DDR SDRAM)、高带宽存储器(high bandwidth memory，HBM)或其他可读可写的存储器。
由于本申请实施例涉及大量神经网络的应用，为了便于理解，下面先对本申请实施例涉及的相关术语及神经网络等相关概念进行介绍。
(1)神经网络
神经网络可以是由神经单元组成的，神经单元可以是指以xs和截距1为输入的运算单元，该运算单元的输出可以为：
其中，s＝1、2、……n，n为大于1的自然数，Ws为xs的权重，b为神经单元的偏置。f为神经单元的激活函数(activation functions)，用于将非线性特性引入神经网络中，来将神经单元中的输入信号转换为输出信号。该激活函数的输出信号可以作为下一层卷积层的输入。激活函数可以是sigmoid函数。神经网络是将许多个上述单一的神经单元联结在一起形成的网络，即一个神经单元的输出可以是另一个神经单元的输入。每个神经单元的输入可以与前一层的局部接受域相连，来提取局部接受域的特征，局部接受域可以是由若干个神经单元组成的区域。
神经网络中的每一层的工作可以用数学表达式y＝a(Wx+b)来描述：从物理层面神经网络中的每一层的工作可以理解为通过五种对输入空间(输入向量的集合)的操作，完成输入空间到输出空间的变换(即矩阵的行空间到列空间)，这五种操作包括：1、升维/降维；2、放大/缩小；3、旋转；4、平移；5、“弯曲”。其中1、2、3的操作由Wx完成，4的操作由+b完成，5的操作则由a()来实现。这里之所以用“空间”二字来表述是因为被分类的对象并不是单个事物，而是一类事物，空间是指这类事物所有个体的集合。其中，W是权重向量，该向量中的每一个值表示该层神经网络中的一个神经元的权重值。该向量W决定着上文所述的输入空间到输出空间的空间变换，即每一层的权重W控制着如何变换空间。训练神经网络的目的，也就是最终得到训练好的神经网络的所有层的权重矩阵(由很多层的向量W形成的权重矩阵)。因此，神经网络的训练过程本质上就是学习控制空间变换的方式，更具体的就是学习权重矩阵。
因为希望神经网络的输出尽可能的接近真正想要预测的值，所以可以通过比较当前网络的预测值和真正想要的目标值，再根据两者之间的差异情况来更新每一层神经网络的权重向量(当然，在第一次更新之前通常会有初始化的过程，即为神经网络中的各层预先配置参数)，比如，如果网络的预测值高了，就调整权重向量让它预测低一些，不断的调整，直到神经网络能够预测出真正想要的目标值。因此，就需要预先定义“如何比较预测值和目标值之间的差异”，这便是损失函数(loss function)或目标函数(objective function)，它们是用于衡量预测值和目标值的差异的重要方程。其中，以损失函数举例，损失函数的输出值(loss)越高表示差异越大，那么神经网络的训练就变成了尽可能缩小这个loss的过程。
(2)反向传播算法
神经网络可以采用误差反向传播(back propagation，BP)算法在训练过程中修正初始的神经网络模型中参数的大小，使得神经网络模型的重建误差损失越来越小。具体地，前向传递输入信号直至输出会产生误差损失，通过反向传播误差损失信息来更新初始的神经网络模型中参数，从而使误差损失收敛。反向传播算法是以误差损失为主导的反向传播运动，旨在得到最优的神经网络模型的参数，例如权重矩阵。
下面从神经网络的训练侧和神经网络的应用侧对本申请提供的方法进行描述。
本申请实施例提供的模型训练方法，涉及数据序列的处理，具体可以应用于数据训练、机器学习、深度学习等方法，对训练数据(例如，将本申请中的第一文本以及第二文本)进行符号化和形式化的智能信息建模、抽取、预处理、训练等，最终得到训练好的神经网络(如本申请中的第一模型、第二模型以及第三模型)；并且，本申请实施例提供的摘要生成方法可以运用上述训练好的神经网络，将输入数据(例如，将本申请中的目标文本)输入到所述训练好的神经网络中，以得到输出数据(如本申请中目标文本中多个句子的分数以及这多个句子的处理结果等等)。需要说明的是，本申请实施例提供的模型训 练方法和摘要生成方法是基于同一个构思产生的发明，也可以理解为一个***中的两个部分，或一个整体流程的两个阶段：如模型训练阶段和模型应用阶段。
为了方便说明，下文以本申请实施例提供的摘要生成方法由用户设备来实现进行示意性介绍。图4为本申请实施例提供的摘要生成方法的一个流程示意图，如图4所示，该方法包括：
401、获取目标文本，目标文本包含N个句子，N≥2。
本实施例中，当用户需要获取目标文本的摘要时，可在用户设备中输入或选择目标文本，以使得用户设备确定需要生成目标文本的摘要。其中，目标文本由N(N为大于或等于2的正整数)个句子构成，每个句子由一个或多个字构成，也就是说，每个句子具备一定的长度，且每个句子的长度是已知的。
需要说明的是，用户设备可为用户提供一个用户界面，该用户界面包含输入窗口和展示窗口。其中，输入窗口用于供用户输入或选择目标文本，即用户可在输入窗口中输入目标文本，或者，用户可在输入窗口展示的文本列表中选择目标文本等等。展示窗口用于为用户显示目标文本的摘要，值得注意的是，展示窗口的大小通常决定摘要的长度阈值(也可以称为摘要的字数阈值，还可以称为摘要的目标长度限制)的大小，摘要的长度阈值的大小可通过以下方式来确定：(1)如图5所示(图5为本申请实施例提供的用户与用户设备交互的一个示意图)，用户可通过手势控制来滑动缩放展示窗口，若用户缩小拉近(zoom in)展示窗口，说明用户所需阅读的摘要的字数较少，用户设备可将摘要的长度阈值设置得较小，若用户放大拉远(zoom out)展示窗口，说明用户所需阅读的摘要的字数较多，用户设备可将摘要的长度阈值设置得较大。(2)用户设备可根据用户自身的信息以及展示窗口的大小，来自行调整长度阈值的大小。例如，在上班时间，用户经常利用零碎时间来快速阅读掌握资讯，此时所需呈现的摘要的字数可以较少，用户设备可将摘要的长度阈值设置得较小。又如，用户设备检测到用户处于移动状态(比如用户在车内等等)，所需呈现的摘要的字数可以更少，以便于用户在移动状态下也可以轻松快速完成简单的阅读，故用户设备可将摘要的长度阈值设置得更小等等。
402、通过第一模型对N个句子进行打分，得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值。
得到目标文本后，用户设备可将目标文本的N个句子输入至第一模型(已训练的神经网络模型，也可以称为抽取式摘要模型)，以通过第一模型对目标文本的N个句子进行打分，从而得到目标文本的N个句子的分数，在目标文本的N个句子中，第i个句子的分数用于指示第i个句子在目标文件中的价值(也可以称为重要程度)，i＝1，...，N。
具体地，如图6所示(图6为本申请实施例提供的第一模型的一个结构示意图)，第一模型(也可以称为抽取模型)可包含bert模块以及transformer encoder模块(即前述的transformer模型)，bert模块的输入端为第一模型的输入端，bert模块的输出端和transformer encoder模块的输入端连接，transformer encoder模块的输出端为第一模型的输出端。那么，用户设备可通过以下方式来获取目标文本的N个句子的分数：
(1)得到目标文本后，用户设备可将目标文本的N个句子输入至第一模型中的bert模块，以使得bert模块对目标文本的N个句子进行映射，从而得到目标文本的N个句子的第四表征，并把目标文本的N个句子的第四表征发送至transformer encoder模块。
(2)得到目标文本的N个句子的第四表征后，transformer encoder模块对目标文本的N个句子的第四表征依次进行基于多头注意力机制的处理、前馈网络的处理以及残差网络的处理等等，从而得到并输出目标文本的N个句子的分数。至此，用户设备可成功得到目标文本的N个句子的得分。
403、基于N个句子的分数以及N个句子的长度，从N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子，N≥M≥1。
得到目标文本的N个句子的得分后，用户设备还可获取目标文本的N个句子的长度，为了在目标文本的N个句子中，选出价值最高且满足字数限制的句子组合，用户设备可将该问题类比为背包问题，背包问题可通过以下公式进行描述：
上式中，V为背包问题的目标函数，S为在目标文本的N个句子中选择出的M个句子(M为大于或等于1的正整数，且M小于或等于N)，也就是所选句子的集合，D为目标文本的N个句子，即可选句子的集合(即目标文本)，lj为选择出的第j个句子的长度，C为摘要的长度阈值，sj为选择出的第j个句子的分数。
基于此，用户设备可将目标文本的N个句子的得分以及目标文本的N个句子的长度作为背包问题的初始值，并对背包问题进行求解，所求得的解即为目标文本的N个句子中，分数之和最大且长度之和小于摘要的长度阈值的M个句子。
具体地，用户设备可通过多种方式来从目标文本的N个句子中挑选出M个句子：
(1)用户设备可获取目标算法(即用于求解背包问题的算法)，并将目标文本的N个句子的分数以及目标文本的N个句子的长度作为目标算法的初始值，将分数之和最大且长度之和小于摘要的长度阈值作为目标算法的优化目标(限制条件)。然后，用户设备可运行目标算法，以使得目标算法对目标文本的N个句子的分数以及目标文本的N个句子的长度进行优化计算，从而在目标文本的N个句子中，求解出分数之和最大且长度之和小于摘要的长度阈值的M个句子。
进一步地，目标算法为以下任意一种：动态规划算法(dynamic programming，DP)、回溯法、分支限界法以及贪心算法，贪心算法为以下任意一种：基于价值密度的贪心算法(density greedy)、基于价值的贪心算法(profit greedy)以及基于大小的贪心算法等等。
(2)用户设备可获取近似于目标算法的第二模型(已训练的神经网络模型)，并将目标文本的N个句子的分数以及目标文本的N个句子的长度输入至第二模型，以通过第二模型对目标文本的N个句子的分数以及目标文本的N个句子的长度进行处理，从而得到并输出目标文本的N个句子的处理结果，该处理结果通常包含目标文本的N个句子被选择与否的指标，故该处理结果可用于从N个句子中，确定分数之和最大且长度之和小于摘要的长度阈值的M个句子。
进一步地，如图7所示(图7为本申请实施例提供的第二模型的一个结构示意图)，第二模型(也可以称为knapsack transformer模型)可包含第一线性模块、transformer encoder模块、第二线性模块以及非线性模块，其中，第一线性模块的输入端为第二模型的输入端，第一线性模块的输出端与transformer encoder模块的输入端连接，transformer encoder模块的输出端与第二线性模块的输入端连接，第二线性模块的输出端与非线性模块的输入端连接，非线性模块的输出端为第二模型的输出端。那么，用户设备可通过以下方式来获取目标文本的N个句子的处理结果：
(2.1)用户设备可将目标文本的N个句子的分数以及目标文本的N个句子的长度输入至第二模型的第一线性模块，以使得第一线性模块先将目标文本的N个句子的分数以及目标文本的N个句子的长度进行拼接，再对拼接得到的结果进行线性运算，从而得到目标文本的N个句子的第一表征，并把目标文本的N个句子的第一表征发送至transformer encoder模块。
(2.2)得到目标文本的N个句子的第一表征后，transformer encoder模块可对目标文本的N个句子的第一表征依次进行基于多头注意力机制的处理、前馈网络的处理以及残差网络的处理等等，从而得到目标文本的N个句子的第二表征，并发送至第二线性模块。
(2.3)得到目标文本的N个句子的第二表征后，第二线性模块可对目标文本的N个句子的第二表征进行线性运算，从而得到目标文本的N个句子的第三表征，并发送至非线性模块。
(2.4)得到目标文本的N个句子的第三表征后，非线性模块可对得到目标文本的N个句子的第三 表征进行非线性运算(例如，通过sigmoid激活函数来实现非线性运算等等)，从而得到并输出目标文本的N个句子的处理结果，该处理结果通常包含目标文本的N个句子被选择与否的指标(该指标的取值通常为0或1)，例如，在目标文本的N个句子中，第1个句子被选择与否的指标为0，则表示第1个句子不可被选中，第2个句子被选择与否的指标为1，则表示第2个句子可被选中等等。那么，用户设备可基于该处理结果，从N个句子中选出M个句子，所选出的M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。
404、基于M个句子生成目标文本的摘要。
得到分数之和最大且长度之和小于摘要的长度阈值的M个句子后，用户设备可将M个句子组成目标文本的摘要，并在展示窗口中展示，以供用户浏览和使用。
本申请实施例中，在获取包含N个句子的目标文本后，可通过第一模型对N个句子进行打分，从而得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值。然后，可基于N个句子的分数以及N个句子的长度，从N个句子中选择分数之和最大且长度之和小于长度阈值的M个句子。最后，可将选择的M个句子组成目标文本的摘要。前述生成目标文本的摘要的过程中，在确定目标文本中N个句子的分数后，可进一步从N个句子中挑选出M个句子，这M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。由此可见，在生成目标文本的摘要时，不仅考虑了这M个句子在目标文本中的价值，还考虑了这M个句子的总长度(即目标文本的摘要的长度)，所考虑的因素较为全面，故由这M个句子所构成的目标文本的摘要，通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
进一步地，本申请实施例中，摘要的长度阈值的大小可由用户的滑动操作来控制，即用户缩小拉近展示窗口时，可将摘要的长度阈值设置得较小，用户放大拉远展示窗口时，可将摘要的长度阈值设置得较大。由此可见，在生成目标文本的摘要的过程中，用户不仅完成输入目标文本的操作，还完成了设置摘要的长度阈值的操作，可提高用户的参与度，令用户更多地参与到摘要生成的过程中，从而满足用户对摘要的自定义需求。
此外，还可将本申请实施例提供的神经网络模型与相关技术的神经网络模型进行比较。首先，可先用本申请实施例提供的第二模型与相关技术的神经网络模型进行比较，比较结果如表一所示：
表一

基于表一可知，本申请实施例提供的第二模型在数据集二和数据集三上，均取得了较好的效果，具备一定的优势。
其次，还可将本申请实施例提供的整体模型和相关技术的整体模型进行比较，比较结果如表二所示：
表二
基于表二可知，相较于相关技术提供的模型，本申请实施例提供的整体模型在多个评估指标上提升明显，具更加优良的性能。
以上是对本申请实施例提供的摘要生成方法所进行的详细说明，以下将对本申请实施例提供的模型训练方法进行介绍。图8为本申请实施例提供的模型训练方法的一个流程示意图，如图8所示，该方法包括：
801、获取第一文本，第一文本包含P个第一句子，P≥2。
本实施例中，当需要对第一待训练模型(即某个未训练的神经网络模型)进行训练时，训练设备可先获取第一训练数据集中的一批训练数据，该批训练数据包含第一文本，第一文本包含P个第一句子(P为大于或等于2的正整数)，每个第一句子由一个或多个字构成，也就是说，每个第一句子具备一定的长度，且每个第一句子的长度是已知的。
802、通过第一待训练模型对P个第一句子进行打分，得到P个第一句子的分数，P个第一句子的分数用于指示P个第一句子在第一文本中的价值。
得到第一文本后，训练设备可将第一文本的P个第一句子输入至第一待训练模型，以通过第一待训 练模型对第一文本的P个第一句子进行打分，从而得到第一文本的P个第一句子的分数，在第一文本的P个第一句子中，第i个第一句子的分数用于指示第i个第一句子在第一文件中的价值(也可以称为重要程度)，i＝1，...，P。
具体地，训练设备可通过以下方式获取第一文本的P个第一句子的分数：首先，训练设备可通过第一待训练模型对P个第一句子进行映射，得到P个第一句子的第四表征。然后，训练设备可通过第一待训练模型对第四表征进行基于transformer模型的处理，得到P个第一句子的分数。
需要说明的是，第一待训练模型处理第一文本中P个第一句子的具体过程，可参考图4所示实施例的步骤402中，第一模型处理目标文本中N个句子的具体过程的相关说明部分，此处不再赘述。
803、通过第二模型对P个第一句子的分数以及P个第一句子的长度进行处理，得到P个第一句子的处理结果，P个第一句子的处理结果用于从P个第一句子中，确定分数之和最大且长度之和小于长度阈值的Q个第一句子，P≥Q≥1。
得到第一文本的P个第一句子的得分后，训练设备还可获取第一文本的P个第一句子的长度以及近似于目标算法的第二模型(已训练的神经网络模型)。然后，训练设备可将第一文本的P个第一句子的分数以及第一文本的P个第一句子的长度输入至第二模型，以通过第二模型对第一文本的P个第一句子的分数以及第一文本的P个第一句子的长度进行处理，从而得到并输出第一文本的P个第一句子的处理结果，该处理结果通常包含第一文本的P个第一句子被选择与否的指标，故该处理结果可用于从P个第一句子中，确定分数之和最大且长度之和小于摘要的长度阈值的Q个第一句子(Q为大于或等于1的正整数，且Q小于或等于P)。
具体地，训练设备可通过以下方式获取第一文本的P个第一句子的处理结果：首先，训练设备通过第二模型对P个第一句子的分数以及P个第一句子的长度进行线性运算，得到P个第一句子的第一表征。接着，训练设备可通过第二模型对第一表征进行基于transformer模型的处理，得到P个第一句子的第二表征。然后，训练设备可通过第二模型对第二表征进行线性运算，得到P个第一句子的第三表征。最后，训练设备可通过第二模型对第三表征进行非线性运算，得到P个第一句子的处理结果。
需要说明的是，第二模型处理第一文本中P个第一句子的分数以及长度的具体过程，可参考图4所示实施例的步骤403中，第二模型处理目标文本中N个句子的分数以及长度的具体过程的相关说明部分，此处不再赘述。
更具体地，在对第一待训练模型进行训练之前(也就是步骤801之前)，训练设备可先完成针对第二待训练模型的训练，从而得到第二模型，该过程包括：
(1)当需要对第二待训练模型(即另一个未训练的神经网络模型)进行训练时，训练设备可先获取第二训练数据集中的一批训练数据，该批训练数据包含第二文本，第二文本包含X个第二句子(X为大于或等于2的正整数)，值得注意的是，每个第二句子的真实分数(即每个第二句子在第二文本中的真实重要程度)是已知的，每个句子被选择的真实概率也是已知的，即X个第二句子的真实处理结果是已知的(该真实处理结果可由前述的目标算法来获取)，且每个第二句子的长度也是已知的。
需要说明的是，第二训练数据集和第一训练数据集通常为两个不同的数据集，且第二训练数据集可通过以下方式采样获得：首先，可获取一个原始数据集。接着，训练设备可确定原始数据集中所有句子的平均长度，利用所有句子的平均长度以及第二待训练模型所能处理的最大长度确定第二待训练模型平均处理的句子数量，那么，基于所有句子的平均长度可构建一个泊松分布，基于平均处理的句子数量可构建一个伽马分布，基于原始数据集中的每个句子的分数可构建一个均匀分布，最后人工采样一个新的数据集，即第二训练数据集，同时满足这三个分布，并将第二训练数据集提供给训练设备。
(2)得到第二文本后，训练设备可将第二文本的X个第二句子的真实分数以及第二文本的X个第二句子的长度输入至第二待训练模型，以通过第二待训练模型对第二文本的X个第二句子的真实分数以及长度进行处理，从而得到第二文本的X个第二句子的(预测)处理结果，该处理结果通常包含第二文本的X个第二句子被选择的(预测)概率，故该处理结果可用于用于从X个第二句子中，确定分数之和最大且长度之和小于长度阈值的Y个第二句子(Y为大于或等于1的正整数，且Y小于或等于X)。
需要说明的是，第二待训练模型处理第二文本中X个第二句子的真实分数以及长度的具体过程，可参考图4所示实施例的步骤403中，第二模型处理目标文本中N个句子的分数以及长度的具体过程的相 关说明部分，此处不再赘述。
(3)得到第二文本的X个第二句子的处理结果后，由于第二文本的X个第二句子的真实处理结果已知，故训练设备可利用预置的损失函数，对第二文本的X个第二句子的处理结果以及第二文本的X个第二句子的真实处理结果进行计算，从而得到目标损失，目标损失用于指示第二文本的X个第二句子的处理结果以及第二文本的X个第二句子的真实处理结果之间的差异。
(4)得到目标损失后，训练设备可利用目标损失，来更新第二待训练模型的参数，并利用第二训练数据集中的另一批训练数据，继续对更新参数后的第二待训练模型进行训练，直至满足模型训练条件(例如，目标损失收敛等等)，从而得到图4所示实施例中的第二模型。
804、基于Q个第一句子，更新第一待训练模型的参数，得到第一模型。
得到分数之和最大且长度之和小于摘要的长度阈值的Q个第一句子后，训练设备可基于这Q个第一句子，来更新第一待训练模型的参数，从而得到图4所示实施例中的第一模型。
具体地，训练设备可通过以下方式来训练得到第一模型：
(1)得到第一文本的P个第一句子的处理结果后，训练设备可将第一文本的P个第一句子的处理结果与第一文本的P个第一句子的第四表征进行相乘，从而在第一文本的P个第一句子的第四表征中，筛选出分数之和最大且长度之和小于摘要的长度阈值的Q个第一句子的第四表征。
(2)得到这Q个第一句子的第四表征后，训练设备可将这Q个第一句子的第四表征以及第一文本输入至第三待训练模型(又一个未训练的神经网络模型，即一个对比学习模型)。那么，第三待训练可先获取第一文本的表征，再获取并输出这Q个第一句子的第四表征与第一文本的表征之间的相似度。
需要说明的是，第三待训练模型可具备多种结构，以下将分别进行介绍：
(2.1)如图9所示(图9为本申请实施例提供的第三待训练模型的一个结构示意图)，第三待训练模型可包含bert模块、第一transformer encoder模块、第二transformer encoder模块、预测器(predictor)模块以及相似度确定模块。其中，bert模块用于对第一文本进行映射，从而得到第一文本的表征，而其余模块则用于对这Q个第一句子的第四表征与第一文本的表征进行一系列的处理，从而得到这Q个第一句子的第四表征与第一文本的表征之间的相似度。
(2.2)如图10所示(图10为本申请实施例提供的第三待训练模型的另一结构示意图)，第三待训练模型可包含bert模块、第一多层感知器(multi-layer perceptron，MLP)、第二MLP模块、多头注意力(multi-head attention)模块以及相似度确定模块。其中，bert模块用于对第一文本进行映射，从而得到第一文本的表征，而其余模块则用于对这Q个第一句子的第四表征与第一文本的表征进行一系列的处理，从而得到这Q个第一句子的第四表征与第一文本的表征之间的相似度。
(2.3)如图11所示(图11为本申请实施例提供的第三待训练模型的另一结构示意图)，第三待训练模型可包含bert模块、第一多层感知器(multi-layer perceptron，MLP)、第二MLP模块、预测器模块以及相似度确定模块。其中，bert模块用于对第一文本进行映射，从而得到第一文本的表征，而其余模块则用于对这Q个第一句子的第四表征与第一文本的表征进行一系列的处理，从而得到这Q个第一句子的第四表征与第一文本的表征之间的相似度。
(2.4)如图12所示(图12为本申请实施例提供的第三待训练模型的另一结构示意图)，第三待训练模型可包含第一simbert模块、第二simbert模块、第一多层感知器(multi-layer perceptron，MLP)、第二MLP模块、预测器模块以及相似度确定模块。其中，第一simbert模块用于对第一文本进行映射，从而得到第一文本的表征，而其余模块则用于对这Q个第一句子的第四表征与第一文本的表征进行一系列的处理，从而得到这Q个第一句子的第四表征与第一文本的表征之间的相似度。
(3)得到这Q个第一句子的第四表征与第一文本的表征之间的相似度后，训练设备可基于该相似度，来更新第一待训练模型的参数以及第三待训练模型的参数，并利用第一训练数据集中的另一批训练数据，继续对更新参数后的第一待训练模型以及更新参数后的第三待训练模型进行训练，直至满足模型训练条件(例如，该相似度达到最大值等等)，从而得到第三模型以及图4所示实施例中的第一模型。
至此，训练设备完成了所有模型的端到端训练，并将训练得到的第一模型以及第二模型部署在前述的用户设备或数据处理设备中，故用户设备或数据处理设备可为用户实现自动摘要服务。
本申请实施例训练得到的第一模型，具备自动摘要功能。具体地，在获取包含N个句子的目标文本 后，可通过第一模型对N个句子进行打分，从而得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值。然后，可基于N个句子的分数以及N个句子的长度，从N个句子中选择分数之和最大且长度之和小于长度阈值的M个句子。最后，可将选择的M个句子组成目标文本的摘要。前述生成目标文本的摘要的过程中，在确定目标文本中N个句子的分数后，可进一步从N个句子中挑选出M个句子，这M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。由此可见，在生成目标文本的摘要时，不仅考虑了这M个句子在目标文本中的价值，还考虑了这M个句子的总长度(即目标文本的摘要的长度)，所考虑的因素较为全面，故由这M个句子所构成的目标文本的摘要，通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
进一步地，本申请实施例中，还可以通过有监督学习的方式训练得到第二模型，第二模型具备摘要的长度控制功能，可以与前述的第一待训练模型结合，以实现端到端的训练，从而得到更好地适应于长度控制算法的第一模型。
进一步地，本申请实施例中，第一模型可通过无监督学习的训练方式来获取，通过此种方式训练得到的第一模型，可具备精准的打分功能，能够为目标文本的各个句子实现准确的打分，从而准确推断出目标文本的摘要，进一步地提高用户体验。
以上是对本申请实施例提供的模型训练方法所进行的详细说明，以下将对本申请实施例的摘要生成装置以及模型训练装置进行介绍。图13为本申请实施例提供的摘要生成装置的一个结构示意图，如图13所示，该装置包括：
获取模块1301，用于获取目标文本，目标文本包含N个句子，N≥2；
打分模块1302，用于通过第一模型对N个句子进行打分，得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值；
确定模块1303，用于基于N个句子的分数以及N个句子的长度，从N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子，N≥M≥1；
生成模块1304，用于基于M个句子生成目标文本的摘要。
本申请实施例中，在获取包含N个句子的目标文本后，可通过第一模型对N个句子进行打分，从而得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值。然后，可基于N个句子的分数以及N个句子的长度，从N个句子中选择分数之和最大且长度之和小于长度阈值的M个句子。最后，可将选择的M个句子组成目标文本的摘要。前述生成目标文本的摘要的过程中，在确定目标文本中N个句子的分数后，可进一步从N个句子中挑选出M个句子，这M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。由此可见，在生成目标文本的摘要时，不仅考虑了这M个句子在目标文本中的价值，还考虑了这M个句子的总长度(即目标文本的摘要的长度)，所考虑的因素较为全面，故由这M个句子所构成的目标文本的摘要，通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
在一种可能实现的方式中，确定模块1303，用于通过目标算法对N个句子的分数以及N个句子的长度进行计算，以从N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子。
在一种可能实现的方式中，目标算法为以下任意一种：动态规划算法、回溯法、分支限界法以及贪心算法。
在一种可能实现的方式中，确定模块1303，用于通过第二模型对N个句子的分数以及N个句子的长度进行处理，得到N个句子的处理结果，处理结果用于从N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子。
在一种可能实现的方式中，确定模块1303，用于：通过第二模型对N个句子的分数以及N个句子的长度进行线性运算，得到N个句子的第一表征；通过第二模型对第一表征进行基于transformer模型的处理，得到N个句子的第二表征；通过第二模型对第二表征进行线性运算，得到N个句子的第三表征；通过第二模型对第三表征进行非线性运算，得到N个句子的处理结果。
在一种可能实现的方式中，打分模块1302，用于：通过第一模型对N个句子进行映射，得到N个句子的第四表征；通过第一模型对第四表征进行基于transformer模型的处理，得到N个句子的分数。
在一种可能实现的方式中，目标文本由用户输入，长度阈值由用户设置。
图14为本申请实施例提供的模型训练装置的一个结构示意图，如图14所示，该装置包括：
第一获取模块1401，用于获取第一文本，第一文本包含P个第一句子，P≥2；
打分模块1402，用于通过第一待训练模型对P个第一句子进行打分，得到P个第一句子的分数，P个第一句子的分数用于指示P个第一句子在第一文本中的价值；
第一处理模块1403，用于通过第二模型对P个第一句子的分数以及P个第一句子的长度进行处理，得到P个第一句子的处理结果，P个第一句子的处理结果用于从P个第一句子中，确定分数之和最大且长度之和小于长度阈值的Q个第一句子，P≥Q≥1；
第一更新模块1404，用于基于Q个第一句子，更新第一待训练模型的参数，得到第一模型。
本申请实施例训练得到的第一模型，具备自动摘要功能。具体地，在获取包含N个句子的目标文本后，可通过第一模型对N个句子进行打分，从而得到N个句子的分数，N个句子的分数用于指示N个句子在目标文本中的价值。然后，可基于N个句子的分数以及N个句子的长度，从N个句子中选择分数之和最大且长度之和小于长度阈值的M个句子。最后，可将选择的M个句子组成目标文本的摘要。前述生成目标文本的摘要的过程中，在确定目标文本中N个句子的分数后，可进一步从N个句子中挑选出M个句子，这M个句子的分数之和最大，且这M个句子的长度之和小于摘要的长度阈值。由此可见，在生成目标文本的摘要时，不仅考虑了这M个句子在目标文本中的价值，还考虑了这M个句子的总长度(即目标文本的摘要的长度)，所考虑的因素较为全面，故由这M个句子所构成的目标文本的摘要，通常是质量较高的摘要，可以满足用户的阅读需求，从而提高用户体验。
在一种可能实现的方式中，该装置还包括：第二获取模块，用于获取第二文本，第二文本包含X个第二句子，X≥2；第二处理模块，用于通过第二待训练模型对X个第二句子的真实分数以及X个第二句子的长度进行处理，得到X个第二句子的处理结果，X个第二句子的处理结果用于从X个第二句子中，确定分数之和最大且长度之和小于长度阈值的Y个第二句子，X≥Y≥1；第三获取模块，用于基于X个第二句子的真实处理结果以及X个第二句子的处理结果，获取目标损失，目标损失用于指示X个第二句子的真实处理结果以及X个第二句子的处理结果之间的差异；第二更新模块，用于基于目标损失，更新第二待训练模型的参数，直至满足模型训练条件，得到第二模型。
在一种可能实现的方式中，第一处理模块1403，用于：通过第二模型对P个第一句子的分数以及P个第一句子的长度进行线性运算，得到P个第一句子的第一表征；通过第二模型对第一表征进行基于transformer模型的处理，得到P个第一句子的第二表征；通过第二模型对第二表征进行线性运算，得到P个第一句子的第三表征；通过第二模型对第三表征进行非线性运算，得到P个第一句子的处理结果。
在一种可能实现的方式中，打分模块1402，用于：通过第一待训练模型对P个第一句子进行映射，得到P个第一句子的第四表征；通过第一待训练模型对第四表征进行基于transformer模型的处理，得到P个第一句子的分数。
在一种可能实现的方式中，第一更新模块1404，用于：通过第三待训练模型获取第一文本的表征；通过第三待训练模型获取Q个第一句子的第四表征与第一文本的表征之间的相似度；基于相似度更新第一待训练模型的参数以及第三待训练模型的参数，直至满足模型训练条件，分别得到第一模型和第三模型。
需要说明的是，上述装置各模块/单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其带来的技术效果与本申请方法实施例相同，具体内容可参考本申请实施例前述所示的方法实施例中的叙述，此处不再赘述。
本申请实施例还涉及一种执行设备，图15为本申请实施例提供的执行设备的一个结构示意图。如图15所示，执行设备1500具体可以表现为手机、平板、笔记本电脑、智能穿戴设备、服务器等，此处不做限定。其中，执行设备1500上可部署有图13对应实施例中所描述的摘要生成装置，用于实现图4对应实施例中摘要生成的功能。具体的，执行设备1500包括：接收器1501、发射器1502、处理器1503和存储器1504(其中执行设备1500中的处理器1503的数量可以一个或多个，图15中以一个处理器为例)，其中，处理器1503可以包括应用处理器15031和通信处理器15032。在本申请的一些实施例中，接收器1501、发射器1502、处理器1503和存储器1504可通过总线或其它方式连接。
存储器1504可以包括只读存储器和随机存取存储器，并向处理器1503提供指令和数据。存储器1504 的一部分还可以包括非易失性随机存取存储器(non-volatile random access memory，NVRAM)。存储器1504存储有处理器和操作指令、可执行模块或者数据结构，或者它们的子集，或者它们的扩展集，其中，操作指令可包括各种操作指令，用于实现各种操作。
处理器1503控制执行设备的操作。具体的应用中，执行设备的各个组件通过总线***耦合在一起，其中总线***除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都称为总线***。
上述本申请实施例揭示的方法可以应用于处理器1503中，或者由处理器1503实现。处理器1503可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器1503中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1503可以是通用处理器、数字信号处理器(digital signal processing，DSP)、微处理器或微控制器，还可进一步包括专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field-programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。该处理器1503可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1504，处理器1503读取存储器1504中的信息，结合其硬件完成上述方法的步骤。
接收器1501可用于接收输入的数字或字符信息，以及产生与执行设备的相关设置以及功能控制有关的信号输入。发射器1502可用于通过第一接口输出数字或字符信息；发射器1502还可用于通过第一接口向磁盘组发送指令，以修改磁盘组中的数据；发射器1502还可以包括显示屏等显示设备。
本申请实施例中，在一种情况下，处理器1503，用于通过图4对应实施例中的第一模型以及第二模型(或目标算法)，生成目标文本的摘要。
本申请实施例还涉及一种训练设备，图16为本申请实施例提供的训练设备的一个结构示意图。如图16所示，训练设备1600由一个或多个服务器实现，训练设备1600可因配置或性能不同而产生比较大的差异，可以包括一个或一个以***处理器(central processing units，CPU)1614(例如，一个或一个以上处理器)和存储器1632，一个或一个以上存储应用程序1642或数据1644的存储介质1630(例如一个或一个以上海量存储设备)。其中，存储器1632和存储介质1630可以是短暂存储或持久存储。存储在存储介质1630的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对训练设备中的一系列指令操作。更进一步地，中央处理器1614可以设置为与存储介质1630通信，在训练设备1600上执行存储介质1630中的一系列指令操作。
训练设备1600还可以包括一个或一个以上电源1626，一个或一个以上有线或无线网络接口1650，一个或一个以上输入输出接口1658；或，一个或一个以上操作***1641，例如Windows ServerTM，Mac OS XTM，UnixTM,LinuxTM，FreeBSDTM等等。
具体的，训练设备可以执行图8对应实施例中的模型训练方法。
本申请实施例还涉及一种计算机存储介质，该计算机可读存储介质中存储有用于进行信号处理的程序，当其在计算机上运行时，使得计算机执行如前述执行设备所执行的步骤，或者，使得计算机执行如前述训练设备所执行的步骤。
本申请实施例还涉及一种计算机程序产品，该计算机程序产品存储有指令，该指令在由计算机执行时使得计算机执行如前述执行设备所执行的步骤，或者，使得计算机执行如前述训练设备所执行的步骤。
本申请实施例提供的执行设备、训练设备或终端设备具体可以为芯片，芯片包括：处理单元和通信单元，所述处理单元例如可以是处理器，所述通信单元例如可以是输入/输出接口、管脚或电路等。该处理单元可执行存储单元存储的计算机执行指令，以使执行设备内的芯片执行上述实施例描述的数据处理方法，或者，以使训练设备内的芯片执行上述实施例描述的数据处理方法。可选地，所述存储单元为所述芯片内的存储单元，如寄存器、缓存等，所述存储单元还可以是所述无线接入设备端内的位于所述芯片外部的存储单元，如只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的 静态存储设备，随机存取存储器(random access memory，RAM)等。
具体的，请参阅图17，图17为本申请实施例提供的芯片的一个结构示意图，所述芯片可以表现为神经网络处理器NPU 1700，NPU 1700作为协处理器挂载到主CPU(Host CPU)上，由Host CPU分配任务。NPU的核心部分为运算电路1703，通过控制器1704控制运算电路1703提取存储器中的矩阵数据并进行乘法运算。
在一些实现中，运算电路1703内部包括多个处理单元(Process Engine,PE)。在一些实现中，运算电路1703是二维脉动阵列。运算电路1703还可以是一维脉动阵列或者能够执行例如乘法和加法这样的数学运算的其它电子线路。在一些实现中，运算电路1703是通用的矩阵处理器。
举例来说，假设有输入矩阵A，权重矩阵B，输出矩阵C。运算电路从权重存储器1702中取矩阵B相应的数据，并缓存在运算电路中每一个PE上。运算电路从输入存储器1701中取矩阵A数据与矩阵B进行矩阵运算，得到的矩阵的部分结果或最终结果，保存在累加器(accumulator)1708中。
统一存储器1706用于存放输入数据以及输出数据。权重数据直接通过存储单元访问控制器(Direct Memory Access Controller，DMAC)1705，DMAC被搬运到权重存储器1702中。输入数据也通过DMAC被搬运到统一存储器1706中。
BIU为Bus Interface Unit即，总线接口单元1713，用于AXI总线与DMAC和取指存储器(Instruction Fetch Buffer，IFB)1709的交互。
总线接口单元1713(Bus Interface Unit，简称BIU)，用于取指存储器1709从外部存储器获取指令，还用于存储单元访问控制器1705从外部存储器获取输入矩阵A或者权重矩阵B的原数据。
DMAC主要用于将外部存储器DDR中的输入数据搬运到统一存储器1706或将权重数据搬运到权重存储器1702中或将输入数据数据搬运到输入存储器1701中。
向量计算单元1707包括多个运算处理单元，在需要的情况下，对运算电路1703的输出做进一步处理，如向量乘，向量加，指数运算，对数运算，大小比较等等。主要用于神经网络中非卷积/全连接层网络计算，如Batch Normalization(批归一化)，像素级求和，对预测标签平面进行上采样等。
在一些实现中，向量计算单元1707能将经处理的输出的向量存储到统一存储器1706。例如，向量计算单元1707可以将线性函数；或，非线性函数应用到运算电路1703的输出，例如对卷积层提取的预测标签平面进行线性插值，再例如累加值的向量，用以生成激活值。在一些实现中，向量计算单元1707生成归一化的值、像素级求和的值，或二者均有。在一些实现中，处理过的输出的向量能够用作到运算电路1703的激活输入，例如用于在神经网络中的后续层中的使用。
控制器1704连接的取指存储器(instruction fetch buffer)1709，用于存储控制器1704使用的指令；
统一存储器1706，输入存储器1701，权重存储器1702以及取指存储器1709均为On-Chip存储器。外部存储器私有于该NPU硬件架构。
其中，上述任一处提到的处理器，可以是一个通用中央处理器，微处理器，ASIC，或一个或多个用于控制上述程序执行的集成电路。
另外需说明的是，以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外，本申请提供的装置实施例附图中，模块之间的连接关系表示它们之间具有通信连接，具体可以实现为一条或多条通信总线或信号线。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现，当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器、专用元器件等来实现。一般情况下，凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现，而且，用来实现同一功能的具体硬件结构也可以是多种多样的，例如模拟电路、数字电路或专用电路等。但是，对本申请而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在可读取的存储介质中，如计算机的软盘、U盘、移动硬盘、ROM、RAM、磁碟或者光盘等，包括若干指令用以使得 一台计算机设备(可以是个人计算机，训练设备，或者网络设备等)执行本申请各个实施例所述的方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。
所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、训练设备或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、训练设备或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的训练设备、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(Solid State Disk，SSD))等。

Claims (17)

  1. 一种摘要生成方法，其特征在于，所述方法包括：
    获取目标文本，所述目标文本包含N个句子，N≥2；
    通过第一模型对所述N个句子进行打分，得到所述N个句子的分数，所述N个句子的分数用于指示所述N个句子在所述目标文本中的价值；
    基于所述N个句子的分数以及所述N个句子的长度，从所述N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子，N≥M≥1；
    基于所述M个句子生成所述目标文本的摘要。
  2. 根据权利要求1所述的方法，其特征在于，所述基于所述N个句子的分数以及所述N个句子的长度，从所述N个句子中，确定分数之和最大且长度之和小于摘要长度的M个句子包括：
    通过目标算法对所述N个句子的分数以及所述N个句子的长度进行计算，以从所述N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子。
  3. 根据权利要求2所述的方法，其特征在于，所述目标算法为以下任意一种：动态规划算法、回溯法、分支限界法以及贪心算法。
  4. 根据权利要求1所述的方法，其特征在于，所述基于所述N个句子的分数以及所述N个句子的长度，从所述N个句子中，确定分数之和最大且长度之和小于摘要长度的M个句子包括：
    通过第二模型对所述N个句子的分数以及所述N个句子的长度进行处理，得到所述N个句子的处理结果，所述处理结果用于从所述N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子。
  5. 根据权利要求4所述的方法，其特征在于，所述通过第二模型对所述N个句子的分数以及所述N个句子的长度进行处理，所述N个句子的处理结果包括：
    通过第二模型对所述N个句子的分数以及所述N个句子的长度进行线性运算，得到所述N个句子的第一表征；
    通过所述第二模型对所述第一表征进行基于transformer模型的处理，得到所述N个句子的第二表征；
    通过所述第二模型对所述第二表征进行线性运算，得到所述N个句子的第三表征；
    通过所述第二模型对所述第三表征进行非线性运算，得到所述N个句子的处理结果。
  6. 根据权利要求1至5任意一项所述的方法，其特征在于，所述通过第一模型对所述N个句子进行打分，得到所述N个句子的分数包括：
    通过第一模型对所述N个句子进行映射，得到所述N个句子的第四表征；
    通过第一模型对所述第四表征进行基于transformer模型的处理，得到所述N个句子的分数。
  7. 根据权利要求1至6任意一项所述的方法，其特征在于，所述目标文本由用户输入，所述长度阈值由所述用户设置。
  8. 一种模型训练方法，其特征在于，所述方法包括：
    获取第一文本，所述第一文本包含P个第一句子，P≥2；
    通过第一待训练模型对所述P个第一句子进行打分，得到所述P个第一句子的分数，所述P个第一句子的分数用于指示所述P个第一句子在所述第一文本中的价值；
    通过第二模型对所述P个第一句子的分数以及所述P个第一句子的长度进行处理，得到所述P个第一句子的处理结果，所述P个第一句子的处理结果用于从所述P个第一句子中，确定分数之和最大且长度之和小于长度阈值的Q个第一句子，P≥Q≥1；
    基于所述Q个第一句子，更新所述第一待训练模型的参数，得到第一模型。
  9. 根据权利要求8所述的方法，其特征在于，所述方法还包括：
    获取第二文本，所述第二文本包含X个第二句子，X≥2；
    通过第二待训练模型对所述X个第二句子的真实分数以及所述X个第二句子的长度进行处理，得到所述X个第二句子的处理结果，所述X个第二句子的处理结果用于从所述X个第二句子中，确定分数之和最大且长度之和小于长度阈值的Y个第二句子，X≥Y≥1；
    基于所述X个第二句子的真实处理结果以及所述X个第二句子的处理结果，获取目标损失，所述目标损失用于指示所述X个第二句子的真实处理结果以及所述X个第二句子的处理结果之间的差异；
    基于所述目标损失，更新所述第二待训练模型的参数，直至满足模型训练条件，得到所述第二模型。
  10. 根据权利要求8或9所述的方法，其特征在于，所述通过第二模型对所述P个第一句子的分数以及所述P个第一句子的长度进行处理，得到所述P个第一句子的处理结果包括：
    通过第二模型对所述P个第一句子的分数以及所述P个第一句子的长度进行线性运算，得到所述P个第一句子的第一表征；
    通过所述第二模型对所述第一表征进行基于transformer模型的处理，得到所述P个第一句子的第二表征；
    通过所述第二模型对所述第二表征进行线性运算，得到所述P个第一句子的第三表征；
    通过所述第二模型对所述第三表征进行非线性运算，得到所述P个第一句子的处理结果。
  11. 根据权利要求8至10任意一项所述的方法，其特征在于，所述通过第一待训练模型对所述P个第一句子进行打分，得到所述P个第一句子的分数包括：
    通过第一待训练模型对所述P个第一句子进行映射，得到所述P个第一句子的第四表征；
    通过第一待训练模型对所述第四表征进行基于transformer模型的处理，得到所述P个第一句子的分数。
  12. 根据权利要求11所述的方法，其特征在于，所述基于所述Q个第一句子，更新所述第一待训练模型的参数，得到第一模型包括：
    通过第三待训练模型获取所述第一文本的表征；
    通过所述第三待训练模型获取所述Q个第一句子的第四表征与所述第一文本的表征之间的相似度；
    基于所述相似度更新所述第一待训练模型的参数以及所述第三待训练模型的参数，直至满足模型训练条件，分别得到第一模型和第三模型。
  13. 一种摘要生成装置，其特征在于，所述装置包括：
    获取模块，用于获取目标文本，所述目标文本包含N个句子，N≥2；
    打分模块，用于通过第一模型对所述N个句子进行打分，得到所述N个句子的分数，所述N个句子的分数用于指示所述N个句子在所述目标文本中的价值；
    确定模块，用于基于所述N个句子的分数以及所述N个句子的长度，从所述N个句子中，确定分数之和最大且长度之和小于长度阈值的M个句子，N≥M≥1；
    生成模块，用于基于所述M个句子生成所述目标文本的摘要。
  14. 一种模型训练装置，其特征在于，所述装置包括：
    第一获取模块，用于获取第一文本，所述第一文本包含P个第一句子，P≥2；
    打分模块，用于通过第一待训练模型对所述P个第一句子进行打分，得到所述P个第一句子的分数，所述P个第一句子的分数用于指示所述P个第一句子在所述第一文本中的价值；
    第一处理模块，用于通过第二模型对所述P个第一句子的分数以及所述P个第一句子的长度进行处理，得到所述P个第一句子的处理结果，所述P个第一句子的处理结果用于从所述P个第一句子中，确定分数之和最大且长度之和小于长度阈值的Q个第一句子，P≥Q≥1；
    第一更新模块，用于基于所述Q个第一句子，更新所述第一待训练模型的参数，得到第一模型。
  15. 一种摘要生成装置，其特征在于，所述装置包括存储器和处理器；所述存储器存储有代码，所述处理器被配置为执行所述代码，当所述代码被执行时，所述摘要生成装置执行如权利要求1至12任意一项所述的方法。
  16. 一种计算机存储介质，其特征在于，所述计算机存储介质存储有一个或多个指令，所述指令在由一个或多个计算机执行时使得所述一个或多个计算机实施权利要求1至12任一所述的方法。
  17. 一种计算机程序产品，其特征在于，所述计算机程序产品存储有指令，所述指令在由计算机执行时，使得所述计算机实施权利要求1至12任意一项所述的方法。
PCT/CN2023/134903 2022-11-29 2023-11-29 一种摘要生成方法及其相关设备 WO2024114659A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211534548.5 2022-11-29
CN202211534548.5A CN115795025A (zh) 2022-11-29 2022-11-29 一种摘要生成方法及其相关设备

Publications (1)

Publication Number Publication Date
WO2024114659A1 true WO2024114659A1 (zh) 2024-06-06

Family

ID=85444740

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/134903 WO2024114659A1 (zh) 2022-11-29 2023-11-29 一种摘要生成方法及其相关设备

Country Status (2)

Country Link
CN (1) CN115795025A (zh)
WO (1) WO2024114659A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115795025A (zh) * 2022-11-29 2023-03-14 华为技术有限公司 一种摘要生成方法及其相关设备

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013171330A (ja) * 2012-02-17 2013-09-02 Nippon Telegr & Teleph Corp <Ntt> テキスト要約装置、方法、及びプログラム
CN108228541A (zh) * 2016-12-22 2018-06-29 深圳市北科瑞声科技股份有限公司 生成文档摘要的方法和装置
CN108628833A (zh) * 2018-05-11 2018-10-09 北京三快在线科技有限公司 原创内容摘要确定方法及装置，原创内容推荐方法及装置
CN109657054A (zh) * 2018-12-13 2019-04-19 北京百度网讯科技有限公司 摘要生成方法、装置、服务器及存储介质
US20200134091A1 (en) * 2018-10-29 2020-04-30 International Business Machines Corporation Query-based extractive summarization
CN111241267A (zh) * 2020-01-10 2020-06-05 科大讯飞股份有限公司 摘要提取和摘要抽取模型训练方法及相关装置、存储介质
CN114138936A (zh) * 2021-12-01 2022-03-04 完美世界控股集团有限公司 一种文本摘要的生成方法、装置、电子设备和存储介质
CN115795025A (zh) * 2022-11-29 2023-03-14 华为技术有限公司 一种摘要生成方法及其相关设备

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013171330A (ja) * 2012-02-17 2013-09-02 Nippon Telegr & Teleph Corp <Ntt> テキスト要約装置、方法、及びプログラム
CN108228541A (zh) * 2016-12-22 2018-06-29 深圳市北科瑞声科技股份有限公司 生成文档摘要的方法和装置
CN108628833A (zh) * 2018-05-11 2018-10-09 北京三快在线科技有限公司 原创内容摘要确定方法及装置，原创内容推荐方法及装置
US20200134091A1 (en) * 2018-10-29 2020-04-30 International Business Machines Corporation Query-based extractive summarization
CN109657054A (zh) * 2018-12-13 2019-04-19 北京百度网讯科技有限公司 摘要生成方法、装置、服务器及存储介质
CN111241267A (zh) * 2020-01-10 2020-06-05 科大讯飞股份有限公司 摘要提取和摘要抽取模型训练方法及相关装置、存储介质
CN114138936A (zh) * 2021-12-01 2022-03-04 完美世界控股集团有限公司 一种文本摘要的生成方法、装置、电子设备和存储介质
CN115795025A (zh) * 2022-11-29 2023-03-14 华为技术有限公司 一种摘要生成方法及其相关设备

Also Published As

Publication number Publication date
CN115795025A (zh) 2023-03-14

Similar Documents

Publication Publication Date Title
WO2022042002A1 (zh) 一种半监督学习模型的训练方法、图像处理方法及设备
WO2022007823A1 (zh) 一种文本数据处理方法及装置
WO2021159714A1 (zh) 一种数据处理方法及相关设备
US20230153615A1 (en) Neural network distillation method and apparatus
WO2023221928A1 (zh) 一种推荐方法、训练方法以及装置
WO2021057884A1 (zh) 语句复述方法、训练语句复述模型的方法及其装置
WO2022001724A1 (zh) 一种数据处理方法及装置
WO2024114659A1 (zh) 一种摘要生成方法及其相关设备
WO2022111387A1 (zh) 一种数据处理方法及相关装置
WO2024001806A1 (zh) 一种基于联邦学习的数据价值评估方法及其相关设备
WO2024067373A1 (zh) 一种数据处理方法及相关装置
WO2023284716A1 (zh) 一种神经网络搜索方法及相关设备
WO2024083121A1 (zh) 一种数据处理方法及其装置
WO2023050143A1 (zh) 一种推荐模型训练方法及装置
WO2024109907A1 (zh) 一种量化方法、推荐方法以及装置
WO2024120504A1 (zh) 一种数据处理方法及相关设备
WO2024067113A1 (zh) 一种动作预测方法及其相关设备
WO2024067779A1 (zh) 一种数据处理方法及相关装置
WO2024046144A1 (zh) 一种视频处理方法及其相关设备
WO2023246735A1 (zh) 一种项目推荐方法及其相关设备
WO2024012360A1 (zh) 一种数据处理方法及相关装置
WO2023197910A1 (zh) 一种用户行为预测方法及其相关设备
WO2023197857A1 (zh) 一种模型切分方法及其相关设备
WO2023185541A1 (zh) 一种模型训练方法及其相关设备
WO2023045949A1 (zh) 一种模型训练方法及其相关设备