CN113990356B - Book generation method, book generation device and storage medium - Google Patents
Book generation method, book generation device and storage medium Download PDFInfo
- Publication number
- CN113990356B CN113990356B CN202010670728.0A CN202010670728A CN113990356B CN 113990356 B CN113990356 B CN 113990356B CN 202010670728 A CN202010670728 A CN 202010670728A CN 113990356 B CN113990356 B CN 113990356B
- Authority
- CN
- China
- Prior art keywords
- character
- information
- story
- book
- emotion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Machine Translation (AREA)
Abstract
The invention provides a book generation method, book generation equipment and storage medium, wherein a story segment set and a role information set contained in text information are determined through the text information contained in a target file; determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the character features in the character information set and the story segment contained in the story segment set; wherein the emotion label set comprises emotion labels of each character in each story segment; and obtaining the book file corresponding to the target file according to the story segment set, the role information set and the emotion label set. The invention uses language processing technology and image generation technology to convert the prior story file into a book file, thereby overcoming the problem of lack of book files containing drawings.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a book generating method, a book generating device, and a storage medium.
Background
At present, books are daily habits of people, the most popular image types in many books are books containing pictorial representations, some of the books are comics which are compiled by referring to cartoon skills and styles, and some of the books are scenic pictures for expressing scenes, and the books are popular among readers because of interesting stories and beautiful pictures. The production of pictures in the books containing pictures is often drawn manually by an authoring worker, the authoring efficiency is low, and the authoring and painting works are required to have stronger painting skills, so that the book resources containing the painting works in the books at present are deficient, and the demands of readers can not be met far.
Accordingly, there is a need for further improvements in the art.
Disclosure of Invention
In view of the shortcomings of the prior art, the invention aims to provide a book generating method, book generating equipment and storage medium, which can quickly convert the existing book works only containing text information into book works containing drawing works.
The technical scheme of the invention is as follows:
in a first aspect, this embodiment provides a book generating method, including:
determining a story segment set and a role information set contained in the text information according to the text information contained in the target file; wherein the story segment set contains at least one story segment, and the character information set contains descriptive text information of character characteristics of at least one character;
Determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the character features in the character information set and the story segment contained in the story segment set; wherein the emotion label set comprises emotion labels of each character in each story segment;
and obtaining the book file corresponding to the target file according to the story segment set, the role information set and the emotion label set.
Optionally, the step of determining the story segment set corresponding to the target file according to the text information includes:
inputting the text information into a trained story information extraction model to obtain a story segment set output by the story information extraction model; the story information extraction model is trained based on the corresponding relation between a text sample information set and a plurality of story fragments contained in the text sample information set.
Optionally, the story information extraction model includes: the device comprises an intermediate vector extraction module, a classifier module and a fragment collection module;
the step of inputting the text information into the trained story information extraction model to obtain a story segment set output by the story information extraction model comprises the following steps:
Inputting the text information into the intermediate vector extraction module to obtain intermediate vector information output by the intermediate vector extraction module;
inputting the intermediate vector information into the classifier module to obtain story quantity values corresponding to the text information output by the classifier module;
and inputting the text information, the intermediate vector information and the story quantity value into the segment collection module to obtain a story segment set corresponding to the text information output by the segment collection module.
Optionally, the step of determining the character information set contained in the text information according to the text information contained in the target file includes:
inputting the character information into a trained character information extraction model to obtain a character information set output by the character information extraction model; the character information extraction model is trained based on the corresponding relation between a character sample information set and descriptive character information of a plurality of character features contained in the character sample information set.
Optionally, the character information extraction model includes: a time sequence module and a feature extraction module;
the step of inputting the text information into a trained character information extraction model to obtain a character information set output by the character information extraction model comprises the following steps:
Inputting the text information into the time sequence module to obtain the sequenced text output by the time sequence module;
and inputting the ordered text into the character feature extraction module to obtain a character information set corresponding to the text information output by the character feature extraction module.
Optionally, the step of determining the emotion label set corresponding to the character in the story segment according to the descriptive text information of the character feature in the character information set and the story segment contained in the story segment set includes:
inputting the story segment set and the role information set into a trained emotion information extraction model, and obtaining an emotion label set of a role through the emotion information extraction model, wherein the emotion information extraction model is obtained based on the corresponding relation training among the story segment sample set, the role information sample set and the role emotion label sample set, and the role emotion label sample set is generated according to story segment samples contained in the story segment sample set and role feature description information contained in the role information sample set.
Optionally, the step of obtaining the book file corresponding to the target file according to the story segment set, the role information set and the emotion label set includes:
Generating character portraits of each character under different emotion labels according to the character information sets and the emotion labels corresponding to each character;
and fusing the character portraits, the emotion labels corresponding to the characters and the story segment sets to obtain book files corresponding to the target files.
Optionally, the step of generating the character portraits of the characters under different emotion labels according to the character information sets and the emotion labels corresponding to the characters comprises the following steps:
inputting the character information set and emotion labels corresponding to all the characters into a trained portrait generation model to obtain character portraits of all the characters under different emotion labels output by the portrait generation model, wherein the portrait generation model is trained based on the corresponding relation among the character information sample set, the emotion label sample set and the character portrait sample set.
Optionally, the step of fusing the character portrait, the emotion labels corresponding to the characters and the story segment set to obtain the book file corresponding to the target file includes:
inputting the character portraits, emotion labels corresponding to all the characters and the story segment sets into a trained information fusion model to obtain a book file corresponding to the target file, wherein the book file is output by the information fusion model, the information fusion model is obtained by training corresponding relations among a character portrait sample set containing a plurality of character image samples, a character emotion label sample set containing a plurality of character emotion label samples, a story segment set containing a plurality of story segment samples and a book sample, and the book sample is generated according to the character portrait sample set, the character emotion label sample set and the story segment set.
Optionally, after the step of inputting the text information to the trained story information extraction model to obtain the story segment set output by the story information extraction model, the method further includes:
and inputting each story segment contained in the story segment set into a trained story refinement model to obtain story segment essences output by the story refinement model, and replacing each story segment with its corresponding story segment essences.
In a second aspect, the present embodiment further provides a book generating apparatus, including a processor, a storage medium communicatively coupled to the processor, the storage medium adapted to store a plurality of instructions; the processor is adapted to invoke instructions in the storage medium to perform steps implementing the book generation method.
In a third aspect, the present embodiment provides a computer-readable storage medium storing one or more programs executable by one or more processors to implement the steps of the book generating method.
The beneficial effects are that: the invention provides a book generation method, book generation equipment and storage medium, wherein a story segment set and a role information set contained in text information are determined through the text information contained in a target file; determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the character features in the character information set and the story segment contained in the story segment set; wherein the emotion label set comprises emotion labels of each character in each story segment; and obtaining the book file corresponding to the target file according to the story segment set, the role information set and the emotion label set. The invention uses language processing technology and image generation technology to convert the prior story file into a book file, thereby overcoming the problem of lack of book files containing drawings.
Drawings
FIG. 1 is a flowchart showing the steps of the book generating method of the present invention;
FIG. 2 is a flowchart illustrating steps for extracting a story segment set using a story information extraction model in the method of the present invention;
FIG. 3 is a flowchart illustrating steps for extracting a character information set using a character information extraction model in the method of the present invention;
FIG. 4 is a flowchart illustrating steps for extracting a set of emotion tags for a character using an emotion information extraction model in a method of the present invention;
FIG. 5 is a flowchart illustrating the steps of creating a book file using a deep network in the method of the present invention;
fig. 6 is a schematic diagram of the structure of the book generating apparatus in the embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clear and clear, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.
It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The inventor finds that the cartoon book works in the prior art are deeply favored by readers, especially the book works containing cartoon pictures are favored by teenagers, but the resources of the conventional domestic cartoon book works are deficient and can not meet the demands of readers, and in the manufacturing process of the cartoon book works, cartoon roles are often drawn manually by cartoon designers, and then processed by cartoon roles, so that the manufacturing efficiency is low.
In order to overcome the problems, and based on the development of language processing technology and image generation technology, it becomes possible to extract story information in a literary work, generate corresponding character images according to character images in the story, and combine the extracted story with the character images to generate a cartoon book work having the same content as the literary work. Since the culture of China is rich, a great deal of literature is suitable for being converted into book works containing drawings, such as: the conversion into the cartoon favorite by teenagers is achieved through a language processing technology and an image generation technology, so that software is automatically realized, the efficiency is high, and the drawing books can be enriched, for example: cartoon resources.
The invention provides a book generation method, which comprises the steps of extracting text information contained in a literary work, and determining a story segment set and a role information set contained in the text information according to the extracted text information; for example: the literary work contains 6 story segments, 6 different stories are respectively told, and the 6 story segments form a story segment set. Since the character images are involved in each story segment, such as the characters a and B in the story segment 1 and the characters B and C in the story segment 2, the descriptive information for each character feature in each story segment constitutes a character information set. And determining the emotion label set of the characters according to the characteristic description text information corresponding to each character in each story segment, for example: in story segment 1, character a is happy, character B is wounded, etc., in story segment 2, the emotion tags corresponding to the respective characters in the respective story segments are grouped into an emotion tag set of characters. And obtaining the book file corresponding to the target file according to the story segment set, the role information set and the emotion label set. A book work having the same content as the existing story file is generated by extracting a story segment and character information from the existing story file and fusing the story segment and character information. And when the story segment set, the character information set and the emotion labels of the characters in the literature are extracted, generating cartoon book works according to the extracted information.
The method comprises the steps that a file of a literary work is uploaded to a computer, text information in the literary work is extracted through language processing software installed in calculation, story segments contained in the text information are obtained, descriptive information of each character feature is sequentially obtained from each story segment, emotion tag sets containing emotion tags corresponding to each character feature are generated by combining the descriptive information of each character feature with the story segments, descriptive information and the emotion tag sets corresponding to each character feature are input into image generating software, the image generating software generates character images of each character under different emotion tags according to the received descriptive information and the emotion tag sets, and then the language information processing software obtains the book work corresponding to the literary work according to the story segment sets, the emotion tag sets and the character images.
It should be noted that the above application scenario is only shown for the convenience of understanding the present invention, and embodiments of the present invention are not limited in this respect. Rather, embodiments of the invention may be applied to any scenario where applicable.
Exemplary method
The present embodiment provides a book generating method, as shown in fig. 2, including:
Step S1, determining a story segment set and a role information set contained in text information according to the text information contained in a target file; the story segment set contains at least one story segment, and the character information set contains descriptive text information of character characteristics of at least one character.
In the step, firstly, a target file is acquired, text information of the target file is extracted, and a story segment set and a role information set contained in the text information of the target file are determined according to the text information of the target file.
Specifically, in this step, the target file is preferably a literary work including a novel, a story meeting, a literature famous and the like with a certain story, and the target file is preferably a literary work including a certain story line, wherein the target file is rich in stories, has a certain role and story line, and is suitable for being converted into a book work.
In the step, character information identification can be carried out on all or part of the contents of the target file through character identification software, and characters which are stored and identified form character information. The step can use the existing text information recognition software, can also use the trained deep network model, input the content of the text information to be recognized into the deep network model, output the text information recognized by the deep network model.
After the text information contained in the target file is identified, a story segment set and a role information set contained in the text information are determined according to the text information.
Because the extracted text information can be subjected to semantic analysis by utilizing semantic analysis software, the story line contained in the text information is obtained, so that the analyzed story line can be divided into story segments, and the story segments form a story segment set. The semantic analysis software can be used for analyzing the semantic analysis result of the character information to obtain the description information of the character features contained in the character information, and the description information of the character features is summarized into a character information set.
Specifically, the step of determining the story segment set corresponding to the target file according to the text information includes:
inputting the text information into a trained story information extraction model to obtain a story segment set output by the story information extraction model; the story information extraction model is trained based on the corresponding relation between a text sample information set and a plurality of story fragments contained in the text sample information set.
In the step, a trained story information extraction model is utilized to obtain a story segment set corresponding to the text information. The story information extraction model is a deep network model, and is a story information extraction model for story information set extraction, which is obtained by inputting a text sample information set into a preset network model for multiple training.
Specifically, the text sample information contained in the text sample information set is input into a first neural network model, a story segment corresponding to each text sample information output by the first neural network model is obtained, and a plurality of story segments form a story segment set corresponding to the text sample information set. And the story segments output by the first neural network model are predicted values of the story segments obtained by extracting the story segments from the text sample information, the true values of the story segments contained in the text sample information are utilized to calculate errors of the predicted values of the story segments output by the first neural network model, the errors between the predicted values and the true values are obtained according to the error calculation, and the parameters of the first neural network model are optimized according to the errors. And repeatedly inputting the text sample information contained in the text sample information set into the first neural network model, and optimizing model parameters according to the predicted value of the story segment output by the model until the error meets the preset condition, so as to obtain the trained story information extraction model.
Specifically, the story information extraction model includes: the device comprises an intermediate vector extraction module, a classifier module and a fragment collection module;
The step of inputting the text information into the trained story information extraction model to obtain a story segment set output by the story information extraction model comprises the following steps:
and step S111, inputting the text information into the intermediate vector extraction module to obtain intermediate vector information output by the intermediate vector extraction module.
Specifically, as shown in fig. 2, the text information is input to an intermediate vector extraction module, and an intermediate vector with the text information is obtained by the intermediate vector extraction module by using the intermediate vector extraction module. The intermediate vector extraction module takes text information as input by using a transducer network structure and outputs an intermediate vector V with text, wherein the intermediate vector is analysis data obtained by carrying out language information processing on the text information by a natural language processing technology.
And step S112, inputting the intermediate vector information into the classifier module to obtain a story number value corresponding to the text information output by the classifier module.
And inputting the intermediate vector obtained in the step S111 into a classifier module to obtain the number of story fragments. Specifically, the classifier module is a classifier, and the classifier is used for analyzing the intermediate vector to obtain the number of story fragments contained in the intermediate vector, and the classifier is composed of a cnn network.
And step 113, inputting the text information, the intermediate vector information and the story quantity value into the segment collection module to obtain a story segment set corresponding to the text information output by the segment collection module.
Inputting the text information, the intermediate vector extracted in step S111 and the story number value extracted in step S112And (5) inputting the story segment set into a segment set module, and outputting the story segment set through the segment set module. In one embodiment, the fragment aggregation module is comprised of a gpt2 network. The output story segment set includes a plurality of story segments. For example: generating small story text segments of N storylines from the original article, e.g. S= { S 1 ,s 2 ,…,s N S represents a set of all story segments, S represents each story segment.
Further, the step of determining the character information set contained in the text information according to the text information contained in the target file includes:
inputting the character information into a trained character information extraction model to obtain a character information set output by the character information extraction model; the character information extraction model is trained based on the corresponding relation between a character sample information set and descriptive character information of a plurality of character features contained in the character sample information set.
In this step, the character information set contained in the character information is extracted by using the trained character information extraction model. The character information extraction model is a deep network model, and is obtained by inputting a character sample information set into a preset network model for training for multiple times, and is used for character information extraction.
Specifically, the character sample information contained in the character sample information set is input into a second neural network model, the description information of character features corresponding to the character sample information output by the second neural network model is obtained, and the description information of a plurality of character features forms a character information set corresponding to the character sample information set. The description information of the character features output by the second neural network model is a predicted value of the description information of the character features extracted from each character sample information, the predicted value of the character feature description information output by the second neural network model is calculated by utilizing the true value of the character feature description information contained in the character sample information, the error between the predicted value and the true value is obtained according to the error calculation, and the parameters of the second neural network model are optimized according to the error. And repeating the steps of inputting the text sample information contained in the text sample information set into the second neural network model and optimizing the model parameters according to the predicted value of the description information of the character features output by the model until the error meets the preset condition, and obtaining the character information extraction model after training.
In one embodiment, the character information extraction model includes: a time sequence module and a feature extraction module;
the step of inputting the text information into a trained character information extraction model to obtain a character information set output by the character information extraction model comprises the following steps:
and step 121, inputting the text information into the time sequence module to obtain the sequenced text output by the time sequence module after sequencing processing.
As shown in fig. 3, the text information extracted from the target file is input to the timing module, the information contained in the text information is ordered by the timing module, so as to ensure that the time sequence of each story contained in the text information is more accurate, specifically, in a specific application embodiment, the timing module may be implemented by adopting a rnn network (a recurrent neural network, recurrent Neural Network), and training is performed on the network by using the time sequence of the general development trend of the story, so that after the text information is input into the network, the network structure may arrange the time sequences contained in the input text information according to the memorized time sequence information, thereby outputting the ordered text.
Step S122, inputting the ordered text into the character feature extraction module to obtain a character information set corresponding to the text information output by the character feature extraction module.
And inputting the ordered text to a character feature extraction module, and obtaining a character information set corresponding to the ordered text through the character feature extraction module. Specifically, in one implementation manner, the character feature extraction module may be implemented by adopting a cnn network, and the cnn network is used to extract descriptive characters corresponding to character features in the input character information, and collect each character descriptive character as a character information set.
For example: and if M characters are involved in the text information, inputting the text information into the rnn network and the cnn network, and outputting descriptive text information with M character features. This p= { P 1 ,p 2 ,…,p M Where P is the character information set and P represents descriptive text information for individual character features.
S2, determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the character features in the role information set and the story segment contained in the story segment set; wherein the emotion label set comprises emotion labels of each character in each story segment.
In the step S1, the story segment set and the character information set are obtained, and the story segment set contains descriptive text information about characters, and the character information set also contains descriptive text information about character features of each character, and the descriptive text information about the character information set and the story segment set are consistent for the same character, so that corresponding characters can be identified from a plurality of story segments in the story segment set according to the descriptive text information about the character features of each character in the character information set, and emotion labels corresponding to each character pair can be identified from the descriptive text information about the character features in the story segment.
Further, the descriptive text information of the character features contained in the character information set is a description of the character surface image of the partial character, so that each character in the story segment can be distinguished conveniently, for example: the character A is thin, the sex is female, the child is high, the character description contained in the story segment is integrated into the story line, and the information such as the weight bias, the character image and emotion of the character appear in the story segment: when a thin and tall girl sees a certain article, the character appearing in the story segment can be identified as A from the descriptive text information, and the emotion label corresponds to the character A.
In this step, the method for determining the emotion tag set according to the information contained in the story segment set and the character information set may be implemented based on a processing manner of a neural network in one implementation manner.
Specifically, the step of determining the emotion label set corresponding to the character in the story segment according to the descriptive text information of the character feature in the character information set and the story segment contained in the story segment set includes:
inputting the story segment set and the role information set into a trained emotion information extraction model, and obtaining an emotion label set of a role through the emotion information extraction model, wherein the emotion information extraction model is obtained based on the corresponding relation training among the story segment sample set, the role information sample set and the role emotion label sample set, and the role emotion label sample set is generated according to story segment samples contained in the story segment sample set and role feature description information contained in the role information sample set.
In the step, a trained emotion information extraction model is utilized, and the extraction of emotion labels of all roles appearing in all story segment sets is realized through the emotion information extraction model. The emotion information extraction model is a deep network model, and is a trained emotion information extraction model obtained by inputting a plurality of story fragments contained in a story fragment sample set, descriptive text information of a plurality of character features contained in a character information sample set and a plurality of emotion label sample information contained in a character emotion label sample set into a preset third neural network model, and training the third neural network model for multiple times.
Specifically, a story segment sample set and a role information sample set are input into a preset third neural network model, and the third neural network model is obtained to output emotion labels of each role in each story segment. And the emotion label set formed by the emotion labels of each role in each story segment output by the third neural network model is a predicted value of the emotion labels of the roles in each story segment contained in the story segment set and the role information set, the predicted value of the emotion labels output by the third neural network model is calculated by utilizing the true value of the emotion labels contained in the role emotion label sample, the error between the predicted value and the true value is obtained according to the error calculation, and the parameters of the third neural network model are optimized according to the error. And repeatedly inputting the text sample information contained in the text sample information set into a third neural network model, and optimizing model parameters according to the predicted value of the emotion label output by the model until the error meets the preset condition, so as to obtain the trained emotion information extraction model.
In one embodiment, as shown in connection with fig. 4, the selection of the cnn network is done because the output of the emotion information extraction model is an emotion tag for a different role and therefore belongs to the classification problem. And the network input of the emotion information extraction model is that the N story fragments S extracted by the story fragment extraction model and the descriptive text information of M character features P extracted by the character information extraction model are obtained, and the emotion information extraction model outputs emotion labels of M characters in the N story fragments. E= { E 1,1 ,e 1,2 ,…,e 1,M ,e 2,1 ,…,e N,M E, where e N,M Representing the emotion of the mth character in the nth episode. If someone does not appear in a particular episode, the emotion value is 0.
And step S3, obtaining the book file corresponding to the target file according to the story segment set, the role information set and the emotion label set.
According to the story segment set, the character information set and the emotion label set extracted in the steps, descriptive text information of character features contained in the character information set and each emotion label contained in the emotion label set corresponding to the character information set are fused, character figures corresponding to each character under different emotions are obtained through an image generation method, the character figures of each character are fused into the extracted story segment set according to the emotion labels described in different story segments, and therefore the converted book file is obtained.
Specifically, the step of obtaining the book file corresponding to the target file according to the story segment set, the role information set and the emotion label set includes:
and S31, generating character portraits of each character under different emotion labels according to the character information set and the emotion labels corresponding to each character.
According to the descriptive text information corresponding to each character feature in the character information set and the emotion labels corresponding to each character, character portraits under different emotion labels can be generated by utilizing image generating software.
Specifically, in the step S31, the step of generating the character portraits of each character under different emotion tags according to the character information set and the emotion tags corresponding to each character includes:
step S311, inputting the character information set and emotion labels corresponding to each character into a trained portrait generation model to obtain character portraits of each character under different emotion labels output by the portrait generation model, wherein the portrait generation model is trained based on the corresponding relation among the character information sample set, the emotion label sample set and the character portrait sample set.
The process of generating the character portraits can be realized by using a trained portraits generation model, namely, a character information set and emotion labels corresponding to each character are input into the portraits generation model, and the character portraits of each character under different emotions are obtained through the portraits generation model. The portrait generation model is a deep network model, and is a trained portrait generation model obtained by inputting descriptive text information of a plurality of character features contained in a character information sample set and a plurality of emotion label sample information contained in a character emotion label sample set into a preset fourth neural network model and training the fourth neural network model for a plurality of times.
Specifically, emotion labels of each character under different emotions, which are contained in a character information sample set and a character emotion label sample set, are input into a preset fourth neural network model, and the fourth neural network model is obtained to output character portraits of each character under different emotions. And the character image set formed by the character images of each character under different emotions output by the fourth neural network model is a predicted value of each character image in the diagonal character image set, the predicted value of the character image output by the fourth neural network model is calculated by utilizing the true value of the character image contained in the character image sample set, the error between the predicted value and the true value is obtained according to the error calculation, and the parameter of the fourth neural network model is optimized according to the error. And repeatedly inputting the character information sample set and the character emotion label sample set into a fourth neural network model, and optimizing model parameters according to the predicted value of the character image output by the model until the error meets the preset condition, so as to obtain the trained portrait generation model.
In one implementation, as shown in fig. 5, M character information and emotion information E are input into a trained representation generation model composed of a cnn+resnet network, generating character representations of M characters in different emotions.
And S32, fusing the character portraits, the emotion labels corresponding to the characters and the story segment sets to obtain book files corresponding to the target files.
And respectively generating character portraits of each character under different emotions, emotion labels corresponding to each character and story segment sets in the steps, and therefore fusing the information to obtain book files corresponding to the target files.
Further, since each story segment in the story segment set contains different roles, the different roles present different emotion labels and role portraits in each story segment, so that according to the emotion labels presented by each role in each story segment, the role portraits corresponding to the emotion labels are matched into the corresponding story segment, and thus, the book file is generated.
Specifically, in this step, the step of fusing the character portrait, the emotion tags corresponding to each character, and the story segment set to obtain the book file corresponding to the target file includes:
inputting the character portraits, emotion labels corresponding to all the characters and the story segment sets into a trained information fusion model to obtain a book file corresponding to the target file, wherein the book file is output by the information fusion model, the information fusion model is obtained by training corresponding relations among a character portrait sample set containing a plurality of character image samples, a character emotion label sample set containing a plurality of character emotion label samples, a story segment set containing a plurality of story segment samples and a book sample, and the book sample is generated according to the character portrait sample set, the character emotion label sample set and the story segment set.
The step can be realized by utilizing a trained information fusion model, and the character portraits, emotion labels corresponding to the characters and story fragments are concentrated to be input into the information fusion model, so that a book file output by the information fusion model is obtained. The information fusion model is a deep network model, and is a trained information fusion model obtained by inputting the character portrait sample set, the character emotion label sample set and the story segment set into a preset fifth neural network model and training the fifth neural network model for multiple times.
Specifically, a character portrait sample set, a character emotion label sample set and a story segment set are input into a preset fifth neural network model, and the fifth neural network model is obtained to output a book file. And the book file information output by the fifth neural network model is a predicted value of cartoon information corresponding to the target file, the predicted value of the book file output by the fifth neural network model is calculated by utilizing the book file information true value in the book sample, the error between the predicted value and the true value is obtained according to the error calculation, and the parameters of the fifth neural network model are optimized according to the error. And repeatedly inputting the character portrait sample set, the character emotion label sample set and the story segment set into a preset fifth neural network model, and optimizing model parameters according to the predicted value of the book file content output by the model until the error meets preset conditions, so as to obtain the information fusion model after training is completed.
In one embodiment, the fifth neural network model generates an antagonism network, the generated story segment set, the portrait sample set of the character under different emotions and the emotion label sample set are input together into the generation antagonism network (GAN network), and finally, M cartoon cartoons with continuous story plots are generated, and the cartoon cartoons are provided with text expressions, and the character shows different emotions according to the plot.
In order to further reduce the content contained in the book file and improve the efficiency of outputting the book file, after the step of inputting the text information into the trained story information extraction model to obtain the story segment set output by the story information extraction model, the method further comprises:
and inputting each story segment contained in the story segment set into a trained story refinement model to obtain story segment essences output by the story refinement model, and replacing each story segment with its corresponding story segment essences.
Namely, inputting the N story fragments extracted in the step S1 into a gpt2 network, secondarily refining the story fragments, and creating shorter and more representative N story fragments 
The method provided in this embodiment is further described below with reference to fig. 5.
Firstly, extracting N story segments S, M character information P of character information contained in a target file and emotion information sets E of M character information presented in different story segments.
Secondly, inputting the emotion information set E and M pieces of character information P into a cnn+resnet network to generate character images of different characters under different emotion labels, and simultaneously sequentially inputting N story segments S into a GPT2 network to obtain refined story segments respectively corresponding to the N story segments
Thirdly, the character portraits, the emotion information sets E and the refined story fragments of different characters under different emotion labels
And inputting the information into an information fusion model (generating an countermeasure network GAN), and obtaining the generated book work through the information fusion model.
The method provided by the invention can be used for rapidly converting the existing literary works into the book works with the illustrations, so that the efficiency of generating the book works is improved, the workload of book creator is lightened, character portraits can be automatically generated, the problem of deficient resources of the book works with the illustrations at present is solved, and the method has higher practical value.
Exemplary apparatus
On the basis of the method, the embodiment also discloses book generating equipment which comprises a processor and a storage medium in communication connection with the processor, wherein the storage medium is suitable for storing a plurality of instructions; the processor is adapted to invoke instructions in the storage medium to perform steps implementing the book generation method. The book generating device can be a mobile phone, a tablet computer or an intelligent television.
Specifically, as shown in fig. 6, the book generating apparatus includes at least one processor (processor) 20 and a memory (memory) 22, and may further include a display 21, a communication interface (Communications Interface) 23, and a bus 24. Wherein the processor 20, the display 21, the memory 22 and the communication interface 23 may communicate with each other via a bus 24. The display screen 21 is configured to display a user guidance interface preset in the initial setting mode. The communication interface 23 may transmit information. The processor 20 may invoke logic instructions in the memory 22 to perform the methods of the embodiments described above.
Further, the logic instructions in the memory 22 described above may be implemented in the form of software functional units and stored in a computer readable storage medium when sold or used as a stand alone product.
The memory 22, as a computer readable storage medium, may be configured to store a software program, a computer executable program, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 30 performs the functional applications and data processing, i.e. implements the methods of the embodiments described above, by running software programs, instructions or modules stored in the memory 22.
The memory 22 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include high-speed random access memory, and may also include nonvolatile memory. For example, a plurality of media capable of storing program codes such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or a transitory storage medium may be used.
In another aspect, a computer readable storage medium stores one or more programs executable by one or more processors to implement the steps of the book generating method.
The invention provides a book generation method, book generation equipment and storage medium, wherein a story segment set and a role information set contained in text information are determined through the text information contained in a target file; determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the character features in the character information set and the story segment contained in the story segment set; the emotion label set comprises emotion labels corresponding to each role in each story segment; and obtaining the book file corresponding to the target file according to the story segment set, the role information set and the emotion label set. The invention utilizes language processing technology and image generation technology to convert the existing story file into a book file, thereby overcoming the defect of lack of the book file containing drawn pictures.
It will be understood that equivalents and modifications will occur to those skilled in the art in light of the present invention and their spirit, and all such modifications and substitutions are intended to be included within the scope of the present invention as defined in the following claims.
Claims (11)
1. A book generating method, comprising:
determining a story segment set and a role information set contained in the text information according to the text information contained in the target file; wherein the story segment set contains at least one story segment, and the character information set contains descriptive text information of character characteristics of at least one character;
determining an emotion label set corresponding to the role in the story segment according to the descriptive text information of the character features in the character information set and the story segment contained in the story segment set; wherein the emotion label set comprises emotion labels of each character in each story segment;
obtaining a book file corresponding to the target file according to the story segment set, the role information set and the emotion label set;
the step of obtaining the book file corresponding to the target file according to the story segment set, the role information set and the emotion label set comprises the following steps:
according to the descriptive text information corresponding to each character feature in the character information set and the emotion labels corresponding to each character, generating character portraits under different emotion labels by utilizing image generating software;
Fusing the character portraits, emotion labels corresponding to the characters and the story segment sets to obtain book files corresponding to the target files; the book file is a cartoon book work.
2. The book generation method of claim 1, characterized in that the step of determining a story segment set corresponding to the target file from the text information comprises:
inputting the text information into a trained story information extraction model to obtain a story segment set output by the story information extraction model; the story information extraction model is trained based on the corresponding relation between a text sample information set and a plurality of story fragments contained in the text sample information set.
3. The book generation method according to claim 2, wherein the story information extraction model includes: the device comprises an intermediate vector extraction module, a classifier module and a fragment collection module;
the step of inputting the text information into a trained story information extraction model to obtain a story segment set output by the story information extraction model comprises the following steps:
inputting the text information into the intermediate vector extraction module to obtain intermediate vector information output by the intermediate vector extraction module;
Inputting the intermediate vector information into the classifier module to obtain story quantity values corresponding to the text information output by the classifier module;
and inputting the text information, the intermediate vector information and the story quantity value into the segment collection module to obtain a story segment set corresponding to the text information output by the segment collection module.
4. The book generating method of claim 2, wherein the step of determining the character information set contained in the text information of the object file based on the text information contained in the object file comprises:
inputting the character information into a trained character information extraction model to obtain a character information set output by the character information extraction model; the character information extraction model is trained based on the corresponding relation between a character sample information set and descriptive character information of a plurality of character features contained in the character sample information set.
5. The book generation method of claim 4, wherein the character information extraction model comprises: a time sequence module and a feature extraction module;
the step of inputting the text information into a trained character information extraction model to obtain a character information set output by the character information extraction model comprises the following steps:
Inputting the text information into the time sequence module to obtain the sequenced text output by the time sequence module;
and inputting the ordered text into the character feature extraction module to obtain a character information set corresponding to the text information output by the character feature extraction module.
6. The book generating method of claim 4, wherein the step of determining a set of emotion tags corresponding to the character in the story segment based on descriptive text information of the character features in the character information set and the story segment contained in the story segment set comprises:
inputting the story segment set and the role information set into a trained emotion information extraction model, and obtaining an emotion label set of a role through the emotion information extraction model, wherein the emotion information extraction model is obtained through training based on a corresponding relation among the story segment sample set, the role information sample set and the role emotion label sample set, and the role emotion label sample set is generated according to descriptive information of story segment samples contained in the story segment sample set and role characteristics contained in the role information sample set.
7. The book generating method of claim 1, wherein the step of generating character portraits of each character under different emotion tags according to the character information set and the emotion tags corresponding to each character comprises:
inputting the character information set and emotion labels corresponding to all the characters into a trained portrait generation model to obtain character portraits of all the characters under different emotion labels output by the portrait generation model, wherein the portrait generation model is trained based on the corresponding relation among the character information sample set, the emotion label sample set and the character portrait sample set.
8. The book generation method according to claim 1, wherein the step of fusing the character portraits, the emotion tags corresponding to the respective characters, and the story segment sets to obtain a book file corresponding to the target file comprises:
inputting the character portraits, emotion labels corresponding to all the characters and the story segment sets into a trained information fusion model to obtain a book file corresponding to the target file, wherein the book file is output by the information fusion model, the information fusion model is obtained by training corresponding relations among a character portrait sample set containing a plurality of character image samples, a character emotion label sample set containing a plurality of character emotion label samples, a story segment set containing a plurality of story segment samples and a book sample, and the book sample is generated according to the character portrait sample set, the character emotion label sample set and the story segment set.
9. The book generation method according to any one of claims 2 to 6, wherein after the step of inputting the text information to the trained story information extraction model to obtain a story segment set output by the story information extraction model, further comprising:
and inputting each story segment contained in the story segment set into a trained story refinement model to obtain story segment essences output by the story refinement model, and replacing each story segment with its corresponding story segment essences.
10. A book generating apparatus comprising a processor, a storage medium in communication with the processor, the storage medium adapted to store a plurality of instructions; the processor being adapted to invoke instructions in the storage medium to perform the steps of implementing the book generation method of any of the preceding claims 1-9.
11. A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs executable by one or more processors to implement the steps of the book generation method of any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010670728.0A CN113990356B (en) | 2020-07-13 | 2020-07-13 | Book generation method, book generation device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010670728.0A CN113990356B (en) | 2020-07-13 | 2020-07-13 | Book generation method, book generation device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113990356A CN113990356A (en) | 2022-01-28 |
| CN113990356B true CN113990356B (en) | 2023-05-16 |
Family
ID=79731308
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010670728.0A Active CN113990356B (en) | 2020-07-13 | 2020-07-13 | Book generation method, book generation device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113990356B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101382937A (en) * | 2008-07-01 | 2009-03-11 | 深圳先进技术研究院 | Multimedia resource processing method based on speech recognition and on-line teaching system thereof |
| CN103873642A (en) * | 2012-12-10 | 2014-06-18 | 北京三星通信技术研究有限公司 | Method and device for recording call log |
| CN108470036A (en) * | 2018-02-06 | 2018-08-31 | 北京奇虎科技有限公司 | A kind of method and apparatus that video is generated based on story text |
| CN108764299A (en) * | 2018-05-04 | 2018-11-06 | 北京物灵智能科技有限公司 | Story model training and generation method, system, robot and storage device |
| CN109977238A (en) * | 2019-04-01 | 2019-07-05 | 杭州赛鲁班网络科技有限公司 | Generate the system for drawing this, method and apparatus |
| CN110472224A (en) * | 2019-06-24 | 2019-11-19 | 深圳追一科技有限公司 | Detection method, device, computer equipment and the storage medium of service quality |
| CN110765265A (en) * | 2019-09-06 | 2020-02-07 | 平安科技（深圳）有限公司 | Information classification extraction method and device, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101594057B1 (en) * | 2009-08-19 | 2016-02-15 | 삼성전자주식회사 | Method and apparatus for processing text data |
-
2020
- 2020-07-13 CN CN202010670728.0A patent/CN113990356B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101382937A (en) * | 2008-07-01 | 2009-03-11 | 深圳先进技术研究院 | Multimedia resource processing method based on speech recognition and on-line teaching system thereof |
| CN103873642A (en) * | 2012-12-10 | 2014-06-18 | 北京三星通信技术研究有限公司 | Method and device for recording call log |
| CN108470036A (en) * | 2018-02-06 | 2018-08-31 | 北京奇虎科技有限公司 | A kind of method and apparatus that video is generated based on story text |
| CN108764299A (en) * | 2018-05-04 | 2018-11-06 | 北京物灵智能科技有限公司 | Story model training and generation method, system, robot and storage device |
| CN109977238A (en) * | 2019-04-01 | 2019-07-05 | 杭州赛鲁班网络科技有限公司 | Generate the system for drawing this, method and apparatus |
| CN110472224A (en) * | 2019-06-24 | 2019-11-19 | 深圳追一科技有限公司 | Detection method, device, computer equipment and the storage medium of service quality |
| CN110765265A (en) * | 2019-09-06 | 2020-02-07 | 平安科技（深圳）有限公司 | Information classification extraction method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113990356A (en) | 2022-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111488931B (en) | Article quality evaluation method, article recommendation method and corresponding devices | |
| CN102193620B (en) | Input method based on facial expression recognition | |
| CN106156799B (en) | Object identification method and device of intelligent robot | |
| CN108280166B (en) | Method and device for making expression, terminal and computer readable storage medium | |
| CN104076944A (en) | Chat emoticon input method and device | |
| CN104063683A (en) | Expression input method and device based on face identification | |
| CN103618918A (en) | Method and device for controlling display of smart television | |
| CN110209789A (en) | A kind of multi-modal dialog system and method for user's attention guidance | |
| CN110719525A (en) | Bullet screen expression package generation method, electronic equipment and readable storage medium | |
| CN107862322B (en) | Method, device and system for classifying picture attributes by combining picture and text | |
| CN114827752B (en) | Video generation method, video generation system, electronic device and storage medium | |
| WO2024046189A1 (en) | Text generation method and apparatus | |
| CN110879974A (en) | Video classification method and device | |
| CN111125457A (en) | Deep cross-modal Hash retrieval method and device | |
| CN111191503A (en) | Pedestrian attribute identification method and device, storage medium and terminal | |
| CN114969282B (en) | Intelligent interaction method based on rich media knowledge graph multi-modal emotion analysis model | |
| CN113642536A (en) | Data processing method, computer device and readable storage medium | |
| CN108537109A (en) | Monocular camera sign Language Recognition Method based on OpenPose | |
| CN116702749A (en) | Multimedia content analysis method, device, equipment and storage medium | |
| CN115759293A (en) | Model training method, image retrieval device and electronic equipment | |
| CN112738555B (en) | Video processing method and device | |
| CN113920561A (en) | Facial expression recognition method and device based on zero sample learning | |
| CN113990356B (en) | Book generation method, book generation device and storage medium | |
| CN110188352B (en) | Text theme determining method and device, computing equipment and storage medium | |
| CN116543798A (en) | Emotion recognition method and device based on multiple classifiers, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |