CN114781331A - Text generation method and device, storage medium and processor - Google Patents

Text generation method and device, storage medium and processor Download PDF

Info

Publication number
CN114781331A
CN114781331A CN202210320783.6A CN202210320783A CN114781331A CN 114781331 A CN114781331 A CN 114781331A CN 202210320783 A CN202210320783 A CN 202210320783A CN 114781331 A CN114781331 A CN 114781331A
Authority
CN
China
Prior art keywords
target
data structure
vector matrix
decoding
graph
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210320783.6A
Other languages
Chinese (zh)
Inventor
耿瑞莹
李亮
黎槟华
李永彬
孙健
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Priority to CN202210320783.6A priority Critical patent/CN114781331A/en
Publication of CN114781331A publication Critical patent/CN114781331A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/12—Use of codes for handling textual entities
    • G06F40/151—Transformation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/12—Use of codes for handling textual entities
    • G06F40/126—Character encoding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/166—Editing, e.g. inserting or deleting
    • G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
    • G06F40/18—Editing, e.g. inserting or deleting of tables; using ruled lines of spreadsheets
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Machine Translation (AREA)

Abstract

The invention discloses a text generation method and device, a storage medium and a processor. Wherein, the method comprises the following steps: acquiring a target table, and converting data in the target table into a data structure diagram; encoding the data structure diagram through an encoder to obtain a target vector matrix, wherein the encoder is constructed by an encoding module and a diagram neural network; and decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, wherein the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware cross-attention layer. The invention solves the technical problem of low accuracy of converting the form data into the text caused by losing the structural information of the data information because the decoding end takes the input data information as the unordered sequence.

Description

Text generation method and device, storage medium and processor
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a text generation method and apparatus, a storage medium, and a processor.
Background
Table-to-text refers to generating corresponding text descriptions from given structured Table data, and can help people to quickly acquire key information in the structured data. The method is widely applied to scenes such as character biography generation, weather broadcasting, news event broadcasting and the like. The table-to-text method in the prior art regards input information as an unordered sequence at a decoding end, loses structural information in structured data, and has the problem of low accuracy in converting table data into texts.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a text generation method and device, a storage medium and a processor, which at least solve the technical problem of low accuracy of converting table data into a text due to the fact that a decoding end takes input data information as an unordered sequence and loses structural information of the data information.
According to an aspect of an embodiment of the present invention, a method for generating a text is provided, including: acquiring a target table, and converting data in the target table into a data structure diagram; coding the data structure chart through a coder to obtain a target vector matrix, wherein the coder is constructed by a coding module and a graph neural network; and decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, wherein the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware cross attention layer.
Optionally, the encoding processing is performed on the data structure diagram through an encoder to obtain the target vector matrix, and the method includes: inputting the data structure diagram into the encoder; processing the data structure chart through the coding module to obtain a first vector matrix; and aggregating information of adjacent nodes of each node in the data structure diagram through the graph neural network to update the first vector matrix to obtain the target vector matrix.
Optionally, the first vector matrix is updated by the graph neural network by aggregating information of neighboring nodes of each node in the data structure graph using the following formula:
Figure BDA0003571613040000011
Figure BDA0003571613040000021
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003571613040000022
gv，
Figure BDA0003571613040000023
and ev,uIs a function of the intermediate variable(s),
Figure BDA0003571613040000024
is a vector representation of the node v and,
Figure BDA0003571613040000025
is a vector representation of the node u,
Figure BDA0003571613040000026
is the attention weight of the neural network of the graph,
Figure BDA0003571613040000027
Wqand WkFor learnable parameters, m representsHidden dimensions of the graph neural network.
Optionally, decoding the target vector matrix by a decoder to obtain a target text corresponding to the data structure diagram, including: calculating to obtain an initial decoding state vector through the data structure chart; obtaining a representation of first structured data according to the target vector matrix, the initial decoding state vector and the structure-perceived attention-spanning layer; according to the representation of the first structured data, modeling processing is carried out through the decoding module and the feedforward neural network adapter to obtain a second vector matrix; decoding the second vector matrix through the decoding module to obtain a first character corresponding to the data structure chart; acquiring a decoding state vector of the current decoding from the second vector matrix, and acquiring all characters corresponding to the data structure chart according to the second vector matrix, the decoding state vector and the structure-perceived crossing attention layer; and generating the target text according to all the obtained characters.
Optionally, deriving a representation of first structured data from the target vector matrix, the initial decoding state vector, and the structure-aware cross-attention layer comprises: constructing the initial decoding state vector and the data structure diagram into a first evolutionary diagram according to the target vector matrix; and sensing the first evolutionary graph through the cross attention layer of the structure sensing to obtain a representation of first structured data.
Optionally, obtaining all the texts corresponding to the data structure diagram according to the second vector matrix, the decoding state vector, and the structure-aware attention-crossing layer includes: s1, constructing the decoding state vector and the first evolutionary graph into a second evolutionary graph according to the second vector matrix; s2, pruning the target node in the second evolutionary graph through the dynamic pruning mechanism to obtain a processed second evolutionary graph; s3, sensing the processed second evolutionary graph through the structure-sensed cross attention layer to obtain a representation of second structured data; s4, according to the representation of the second structured data, carrying out modeling processing through the decoding module and the feedforward neural network adapter to obtain a third vector matrix; s5, decoding the third vector matrix through the decoding module to obtain a second character corresponding to the data structure diagram; and repeating the steps S1 to S5 until all characters corresponding to the data structure diagram are obtained.
Optionally, pruning the target node in the second evolutionary graph through the dynamic pruning mechanism, and obtaining the processed second evolutionary graph includes: representing the gate of each node of the second evolutionary graph as:
Figure BDA0003571613040000028
Figure BDA0003571613040000029
wherein, Wg，WeAnd WdAs a learnable parameter, hvAnd htIs an intermediate variable; taking the node with the gv smaller than a preset value as the target node; and pruning the target node through the dynamic pruning mechanism to obtain the processed second evolutionary graph.
According to another aspect of the embodiments of the present invention, there is also provided a text generation method, including: receiving a target form sent by a client; converting data in the target table into a data structure diagram in a cloud server, and encoding the data structure diagram through an encoder to obtain a target vector matrix, wherein the encoder is constructed by an encoding module and a diagram neural network, and decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, and the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware cross attention layer; and returning the target text to the client.
According to another aspect of the embodiments of the present invention, there is further provided a text generating apparatus, including: the conversion unit is used for acquiring a target table and converting data in the target table into a data structure diagram; the first processing unit is used for coding the data structure diagram through a coder to obtain a target vector matrix, wherein the coder is constructed by a coding module and a graph neural network; and the second processing unit is used for decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, wherein the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-perceived attention-spanning layer.
Optionally, the first processing unit includes: an input subunit, configured to input the data structure diagram into the encoder; the first processing subunit is used for processing the data structure chart through the coding subunit to obtain a first vector matrix; and the updating subunit is used for aggregating the information of the adjacent nodes of each node in the data structure diagram through the graph neural network to update the first vector matrix so as to obtain the target vector matrix.
Optionally, the first vector matrix is updated by the graph neural network by aggregating information of neighboring nodes of each node in the data structure graph using the following formula:
Figure BDA0003571613040000031
Figure BDA0003571613040000032
wherein the content of the first and second substances,
Figure BDA0003571613040000033
gv，
Figure BDA0003571613040000034
and ev,uIs a function of the intermediate variable(s),
Figure BDA0003571613040000035
for the vector representation of the node v,
Figure BDA0003571613040000036
is a vector representation of the node u and,
Figure BDA0003571613040000037
is the attention weight of the neural network of the graph,
Figure BDA0003571613040000038
Wqand WkM represents the hidden dimension of the graph neural network, which is a learnable parameter.
Optionally, the second processing unit comprises: the calculation subunit is used for calculating to obtain an initial decoding state vector through the data structure chart; a second processing subunit, configured to obtain a representation of first structured data according to the target vector matrix, the initial decoding state vector, and the structure-aware cross attention layer; the third processing subunit is used for carrying out modeling processing through the decoding subunit and the feedforward neural network adapter according to the representation of the first structured data to obtain a second vector matrix; the fourth processing subunit is configured to decode the second vector matrix through the decoding subunit to obtain a first character corresponding to the data structure diagram; an obtaining subunit, configured to obtain a currently decoded decoding state vector from the second vector matrix, and obtain all the characters corresponding to the data structure diagram according to the second vector matrix, the decoding state vector, and the structure-aware attention-spanning layer; and the generating subunit is used for generating the target text according to all the obtained characters.
Optionally, the second processing subunit includes: the first construction module is used for constructing the initial decoding state vector and the data structure diagram into a first evolutionary diagram according to the target vector matrix; and the first perception module is used for perceiving the first evolutionary graph through the cross attention layer of the structural perception to obtain the representation of the first structural data.
Optionally, the obtaining subunit includes: the second construction module is used for constructing the decoding state vector and the first evolutionary graph into a second evolutionary graph according to the second vector matrix; the first processing module is used for pruning the target node in the second evolutionary graph through the dynamic pruning mechanism to obtain a processed second evolutionary graph; the second perception module is used for perceiving the processed second evolutionary graph through the structure-perceived attention-spanning layer to obtain a representation of second structured data; the second processing module is used for carrying out modeling processing through the decoding module and the feedforward neural network adapter according to the representation of the second structured data to obtain a third vector matrix; the third processing module is used for decoding the third vector matrix through the decoding module to obtain a second character corresponding to the data structure chart; and repeating the second construction module, the first processing module, the second sensing module, the second processing module and the third processing module until all characters corresponding to the data structure diagram are obtained.
Optionally, the first processing module includes: a first processing submodule for representing the gates of each node of the second evolutionary graph as:
Figure BDA0003571613040000041
wherein, Wg，WeAnd WdAs a learnable parameter, hvAnd htIs an intermediate variable; the determining submodule is used for taking the node with the gv smaller than a preset value as the target node; and the second processing submodule is used for carrying out pruning processing on the target node through the dynamic pruning mechanism to obtain the processed second evolutionary graph.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium storing a program, wherein when the program runs, a device on which the storage medium is located is controlled to execute the text generation method according to any one of the above items.
According to another aspect of the embodiments of the present invention, there is also provided a processor, configured to execute a program, where the program executes the text generation method described in any one of the above items when running.
In the embodiment of the invention, a mode that a coder is constructed by a coding module and a graph neural network and a decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware attention-crossing layer is adopted, the data structure diagram is coded by the coder to obtain a target vector matrix, the target vector matrix is decoded by the decoder to obtain a target text corresponding to the data structure diagram, and the aim of converting table data into the text is fulfilled.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a block diagram of a hardware configuration of a computer terminal according to an embodiment of the present invention;
FIG. 2 is a flow diagram of a method of generating text in accordance with an embodiment of the present invention;
FIG. 3 is a diagram of a data structure provided in accordance with an embodiment of the present invention;
fig. 4 is a schematic diagram of an encoder and a decoder according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a neural network provided in accordance with an embodiment of the present invention;
fig. 6 is a schematic pruning diagram of a dynamic pruning mechanism provided according to an embodiment of the present invention;
FIG. 7 is a flowchart of a text generation method according to a second embodiment of the present invention;
fig. 8 is a schematic diagram of a text generating apparatus according to a third embodiment of the present invention;
fig. 9 is a block diagram of an alternative computer terminal according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
There is also provided, in accordance with an embodiment of the present invention, an embodiment of a method for generating text, it being noted that the steps illustrated in the flowchart of the figure may be performed in a computer system such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than here.
The method provided by the first embodiment of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Fig. 1 shows a hardware configuration block diagram of a computer terminal (or mobile device) for implementing a text generation method. As shown in fig. 1, the computer terminal 10 (or mobile device 10) may include one or more processors (shown as 102a, 102b, … …, 102n in the figures) which may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, a memory 104 for storing data, and a transmission module 106 for communication functions. Besides, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial BUS (USB) port (which may be included as one of the ports of the BUS), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
It should be noted that the one or more processors and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuit may be a single stand-alone processing module, or incorporated in whole or in part into any of the other elements in the computer terminal 10 (or mobile device). As referred to in the embodiments of the application, the data processing circuit acts as a processor control (e.g. selection of a variable resistance termination path connected to the interface).
The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the text generation method in the embodiment of the present invention, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the text generation method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet via wireless.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).
Under the above operating environment, the present application provides a text generation method as shown in fig. 2. Fig. 2 is a flowchart of a text generation method according to a first embodiment of the present invention.
Step S201, obtain the target table, and convert the data in the target table into a data structure diagram.
Specifically, table data to be converted into text is first converted into a data structure diagram, and the data structure diagram is composed of nodes and edges, for example, the data structure diagram shown in fig. 3.
And S202, carrying out coding processing on the data structure diagram through a coder to obtain a target vector matrix, wherein the coder is constructed by a coding module and a graph neural network.
Specifically, the encoder is constructed by an encoding module and a graph neural network, as shown in fig. 4, a schematic diagram of an encoder and a decoder is provided according to an embodiment of the present invention. And carrying out coding processing through a coder data structure diagram to obtain a target vector matrix. A schematic diagram of a graph neural Network (Relational graphical attention Network), as shown in fig. 5, the graph neural Network can process and analyze the structural information in the data structure diagram, so that the structural information can be added to the target vector matrix.
And S203, decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, wherein the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware cross-attention layer.
Specifically, a decoder is constructed by a decoding module, a feed-forward neural network adapter (FNNAdapter), a Dynamic Pruning mechanism (Dynamic Graph Pruning DGP) and a Structure-Aware Cross Attention layer (SACA), as shown in fig. 4, which is a schematic diagram of an encoder and a decoder provided according to an embodiment of the present invention. And decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure chart.
In summary, in a manner that an encoder constructed by an encoding module and a graph neural network and a decoder constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware attention-crossing layer are used for constructing the decoder, the encoder encodes the data structure diagram to obtain a target vector matrix, and the decoder decodes the target vector matrix to obtain a target text corresponding to the data structure diagram, so that the accuracy of converting table data into the text is improved.
Optionally, in the text generating method provided in the first embodiment of the present invention, the encoding processing is performed on the data structure diagram through an encoder to obtain the target vector matrix, where the method includes: inputting the data structure chart into an encoder; processing the data structure chart through a coding module to obtain a first vector matrix; and updating the first vector matrix through information of adjacent nodes of each node in the graph neural network aggregation data structure chart to obtain a target vector matrix.
Specifically, the obtained data structure diagram is input into an encoder, an encoding module performs encoding processing on each node of the data structure diagram to obtain a first vector matrix, and then the neural network aggregates information of adjacent nodes of each node in the data structure diagram to update the first vector matrix to obtain a target vector matrix. The operation enables the target vector matrix to contain the structural information in the data structure chart, and the structural information in the data structure chart can be fully combined with the data in the data structure chart during subsequent decoding, so that the accuracy of text acquisition is improved.
Optionally, in the text generating method provided in the first embodiment of the present invention, the first vector matrix is updated by the graph neural network by aggregating information of neighboring nodes of each node in the data structure diagram by using the following formula:
Figure BDA0003571613040000081
Figure BDA0003571613040000082
Figure BDA0003571613040000083
wherein the content of the first and second substances,
Figure BDA0003571613040000084
gv，
Figure BDA0003571613040000085
and ev,uIs a function of the intermediate variable(s),
Figure BDA0003571613040000086
for the vector representation of the node v,
Figure BDA0003571613040000087
is a vector representation of the node u and,
Figure BDA0003571613040000088
is the attention weight of the neural network of the graph,
Figure BDA0003571613040000089
Wqand WkFor a learnable parameter, m represents the hidden dimension of the graph neural network.
Specifically, the graph neural network updates the first vector matrix using the following equation:
Figure BDA00035716130400000810
Figure BDA00035716130400000811
Figure BDA00035716130400000812
the formula can accurately acquire the structural information in the data structure diagram.
Optionally, in the text generating method provided in the first embodiment of the present invention, decoding the target vector matrix by a decoder to obtain the target text corresponding to the data structure diagram, where the method includes: calculating to obtain an initial decoding state vector through a data structure diagram; obtaining a representation of the first structured data according to the target vector matrix, the initial decoding state vector and the structure-perceived attention-spanning layer; according to the representation of the first structured data, modeling processing is carried out through a decoding module and a feedforward neural network adapter to obtain a second vector matrix; decoding the second vector matrix through a decoding module to obtain a first character corresponding to the data structure chart; and acquiring a current decoding state vector from the second vector matrix, acquiring all characters corresponding to the data structure diagram according to the second vector matrix, the decoding state vector and the structure-perceived attention-crossing layer, and generating a target text according to all the acquired characters.
Specifically, at the decoder side, an initial decoding state vector is obtained through calculation according to the structural data diagram, then a first evolutionary diagram is constructed according to the target vector matrix and the initial decoding state vector, and the first evolutionary diagram is perceived through a Structure-Aware Cross-Attention (SACA) layer, so that a representation of first structural data is obtained. In the prior art, the structure information of the data structure diagram is ignored by a general attention crossing layer mechanism during decoding. And this structured information plays a crucial role in the representation of the nodes. Then, according to the representation of the first structured data, modeling processing is carried out through a decoding module and a feedforward neural network adapter to obtain a second vector matrix, wherein the second vector matrix comprisesThe state vector is initially decoded. The formula for modeling the first evolutionary graph by a feed forward neural network Adapter (FNN Adapter) is:
Figure BDA0003571613040000091
Figure BDA0003571613040000092
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003571613040000093
and
Figure BDA0003571613040000094
is a function of the intermediate variable(s),
Figure BDA0003571613040000095
and
Figure BDA0003571613040000096
are parameters that can be learned. And decoding the second vector matrix through a decoding module to obtain a first character corresponding to the data structure chart. The operation process fully considers the structural information of the structural data diagram, so that the accuracy of converting the table data into the text is improved.
Optionally, in the method for generating a text according to the first embodiment of the present invention, obtaining a representation of first structured data according to the target vector matrix, the initial decoding state vector, and the structure-aware attention-crossing layer includes: constructing an initial decoding state vector and a data structure chart into a first evolutionary graph according to a target vector matrix; and sensing the first evolutionary graph through a cross attention layer of structure sensing to obtain a representation of the first structured data.
Specifically, a first evolutionary graph is constructed according to a target vector matrix and an initial decoding state vector, and the first evolutionary graph is perceived through a structure perception attention-crossing layer to obtain a representation of first structured data. The structural data representation of the first evolutionary graph can be accurately obtained through the structure-aware cross attention layer, so that the structural information of the first evolutionary graph can be fully considered in the decoding process, and the decoding is more accurate.
Optionally, in the text generating method provided in the first embodiment of the present invention, obtaining all the texts corresponding to the data structure diagram according to the second vector matrix, the decoding state vector, and the structure-aware attention-crossing layer includes: s1, constructing the decoding state vector and the first evolutionary graph into a second evolutionary graph according to the second vector matrix; s2, pruning the target node in the second evolutionary graph through the dynamic pruning mechanism to obtain a processed second evolutionary graph; s3, sensing the processed second evolutionary graph through the structure-sensed attention-crossing layer to obtain a representation of second structured data; s4, according to the representation of the second structured data, carrying out modeling processing through the decoding module and the feedforward neural network adapter to obtain a third vector matrix; s5, decoding the third vector matrix through the decoding module to obtain a second character corresponding to the data structure chart; and repeating the steps S1-S5 until all characters corresponding to the data structure diagram are obtained.
In particular, as decoding proceeds, many nodes in the subsequent second evolutionary graph may be irrelevant to subsequent generation, and these irrelevant nodes may cause overfitting and even interfere with subsequent generation. Intuitively, the decoder should dynamically prune the evolutionary graph in different decoding steps. For this purpose, redundant nodes (i.e. the above target nodes) in the evolutionary graph are dynamically deleted in the decoding process through a dynamic graph pruning mechanism. For example, as shown in fig. 6, a pruning diagram of a dynamic pruning mechanism provided according to an embodiment of the present invention is shown. The dynamic graph pruning mechanism performs pruning on the target node in the second evolutionary graph (i.e., step S2 described above). And after the second character is obtained, obtaining the latest decoding state vector through the third vector matrix, and repeating the steps S1-S5 according to the third vector matrix and the latest decoding state vector until all characters corresponding to the data structure diagram are obtained. According to the method, the redundant nodes are processed through a dynamic graph pruning mechanism, so that the subsequent modeling calculation work is reduced, and the decoding accuracy is improved.
Optionally, in the method for generating a text according to the first embodiment of the present invention, pruning a target node in the second evolutionary graph through a dynamic pruning mechanism, and obtaining a processed second evolutionary graph includes: the gate for each node of the second evolutionary graph is represented as:
Figure BDA0003571613040000101
wherein, Wg，WeAnd WdAs a learnable parameter, hvAnd htIs an intermediate variable; taking the node with the gv smaller than a preset value as a target node; and pruning the target node through a dynamic pruning mechanism to obtain a processed second evolutionary graph.
Specifically, the pruning processing of the target node in the second chemograph includes: the gate for each node on the second evolutionary graph is represented as:
Figure BDA0003571613040000102
if g isvClose to 0, it means that the association between node v and all its neighbors is very weak. That is, the node v is a redundant node (the above target node), and then the node v is pruned through a dynamic pruning mechanism. The formula can accurately acquire the relevance between each node and all the neighbors of the node, and the accuracy of a dynamic pruning mechanism is improved.
In summary, in the text generating method provided in the first embodiment of the present invention, by using the encoder constructed by the encoding module and the graph neural network and the decoder constructed by the decoding module, the feedforward neural network adapter, the dynamic pruning mechanism and the structure-aware cross-attention layer, the encoder is used for encoding the data structure chart to obtain a target vector matrix, the decoder is used for decoding the target vector matrix to obtain a target text corresponding to the data structure chart, the aim of converting table data into the text is achieved, and therefore the technical effect of improving the accuracy of converting the table data into the text is achieved, and the technical problem that the accuracy rate of converting the form data into the text is low due to the fact that the decoding end takes the input data information as an unordered sequence and the structural information of the data information is lost is solved.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the above description of the embodiments, those skilled in the art can clearly understand that the generation of the text according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention or portions thereof contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (which may be a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 2
Under the operating environment, the application provides a text generation method as shown in fig. 7. Fig. 7 is a flowchart of a text generation method according to a second embodiment of the present invention.
Step S701, receiving a target table sent by the client.
Specifically, the target table is sent to the cloud server, and the work of converting the data in the target table into the text is performed through the cloud server.
Step S702, converting data in the target table into a data structure diagram in the cloud server, and performing encoding processing on the data structure diagram through an encoder to obtain a target vector matrix, wherein the encoder is constructed by an encoding module and a graph neural network, and a decoder is used for performing decoding processing on the target vector matrix to obtain a target text corresponding to the data structure diagram, and the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware attention-crossing layer.
Specifically, in the cloud server, data in a target table is converted into a data structure diagram, and then an encoder performs encoding processing on each node of the data structure diagram to obtain a target vector matrix; and then decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure chart.
Step S703, returning the target text to the client.
The cloud server is used for converting the table data, so that the efficiency of the table data conversion is improved, and the storage pressure of the local terminal is reduced.
In the cloud server, the specific method for converting the table data into the text is the same as that in the first embodiment, and is not described herein again.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention or portions thereof contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (which may be a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 3
According to an embodiment of the present invention, there is also provided a generating apparatus for implementing the above text, as shown in fig. 8, the apparatus includes: a conversion unit 801, a first processing unit 802 and a second processing unit 803.
A conversion unit 801, configured to obtain a target table and convert data in the target table into a data structure diagram;
the first processing unit 802 is configured to perform encoding processing on the data structure diagram through an encoder to obtain a target vector matrix, where the encoder is constructed by an encoding module and a graph neural network;
the second processing unit 803 is configured to perform decoding processing on the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, where the decoder is constructed by a decoding module, a feed-forward neural network adapter, a dynamic pruning mechanism, and a structure-aware cross-attention layer.
The device for generating a text provided by the third embodiment of the present invention is configured to obtain a target table through a conversion unit 801, and convert data in the target table into a data structure diagram; the first processing unit 802 is configured to perform encoding processing on the data structure diagram through an encoder to obtain a target vector matrix, where the encoder is constructed by an encoding module and a graph neural network; the second processing unit 803 is configured to decode the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, where the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism, and a structure-aware attention-spanning layer, and in the related art, a decoding end regards input data information as a disordered sequence, and structural information of the data information is lost, which results in a problem of low accuracy in converting table data into a text. The accuracy of converting table data into text is improved by constructing a decoder through an encoder and a decoding module constructed by an encoding module and a graph neural network, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware cross-attention layer.
Optionally, in the apparatus for generating a text provided in the third embodiment of the present invention, the first processing unit 802 includes: the input subunit is used for inputting the data structure chart into the encoder; the first processing subunit is used for processing the data structure chart through the coding subunit to obtain a first vector matrix; and the updating subunit is used for updating the first vector matrix through the information of the adjacent node of each node in the graph neural network aggregation data structure chart to obtain a target vector matrix.
Optionally, in the apparatus for generating a text provided in the third embodiment of the present invention, the first vector matrix is updated by aggregating, by the graph neural network, information of neighboring nodes of each node in the data structure graph by using the following formula:
Figure BDA0003571613040000131
Figure BDA0003571613040000132
Figure BDA0003571613040000133
wherein the content of the first and second substances,
Figure BDA0003571613040000134
gv，
Figure BDA0003571613040000135
and ev,uIs the intermediate variable(s) of the variable,
Figure BDA0003571613040000136
for the vector representation of the node v,
Figure BDA0003571613040000137
is a vector representation of the node u and,
Figure BDA0003571613040000138
is the attention weight of the neural network of the graph,
Figure BDA0003571613040000139
Wqand WkFor a learnable parameter, m represents the hidden dimension of the graph neural network.
Optionally, in the apparatus for generating a text provided in the third embodiment of the present invention, the second processing unit 803 includes: the calculation subunit is used for calculating to obtain an initial decoding state vector through the data structure diagram; the second processing subunit is used for obtaining the representation of the first structured data according to the target vector matrix, the initial decoding state vector and the structure perception crossing attention layer; the third processing subunit is used for carrying out modeling processing through the decoding subunit and the feedforward neural network adapter according to the representation of the first structured data to obtain a second vector matrix; the fourth processing subunit is used for decoding the second vector matrix through the decoding subunit to obtain a first character corresponding to the data structure chart; the acquisition subunit is used for acquiring the decoding state vector of the current decoding from the second vector matrix and acquiring all characters corresponding to the data structure chart according to the second vector matrix, the decoding state vector and the structure-perceived crossing attention layer; and the generating subunit is used for generating a target text according to all the obtained characters.
Optionally, in the apparatus for generating a text provided in the third embodiment of the present invention, the second processing subunit includes: the first construction module is used for constructing the initial decoding state vector and the data structure chart into a first evolutionary graph according to the target vector matrix; and the first perception module is used for perceiving the first evolutionary graph through a structure perception crossing attention layer to obtain a representation of first structural data.
Optionally, in the apparatus for generating a text provided in the third embodiment of the present invention, the obtaining subunit includes: the second construction module is used for constructing the decoding state vector and the first evolutionary graph into a second evolutionary graph according to the second vector matrix; the first processing module is used for pruning the target node in the second evolutionary graph through a dynamic pruning mechanism to obtain a processed second evolutionary graph; the second perception module is used for perceiving the processed second evolutionary graph through a structure perception crossing attention layer to obtain a representation of second structured data; the second processing module is used for carrying out modeling processing through the decoding module and the feedforward neural network adapter according to the representation of the second structured data to obtain a third vector matrix; the third processing module is used for decoding the third vector matrix through the decoding module to obtain a second character corresponding to the data structure chart; and repeating the second construction module, the first processing module, the second perception module, the second processing module and the third processing module until all characters corresponding to the data structure diagram are obtained.
Optionally, in the apparatus for generating a text provided in the third embodiment of the present invention, the first processing module includes: a first processing submodule for representing the gates of each node of the second evolutionary graph as:
Figure BDA0003571613040000141
wherein, Wg，WeAnd WdAs a learnable parameter, hvAnd htIs an intermediate variable; the determining submodule is used for taking the node with the gv smaller than a preset value as a target node; and the second processing submodule is used for carrying out pruning processing on the target node through a dynamic pruning mechanism to obtain a processed second evolutionary graph.
It should be noted here that the above-mentioned conversion unit 801, the first processing unit 802 and the second processing unit 803 correspond to steps S201 to S203 in embodiment 1, and the two modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in the above-mentioned first embodiment. It should be noted that the above modules as part of the apparatus may be run in the computer terminal 10 provided in the first embodiment.
Example 4
The embodiment of the invention can provide a computer terminal which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced with a terminal device such as a mobile terminal.
Optionally, in this embodiment, the computer terminal may be located in at least one network device of a plurality of network devices of a computer network.
In this embodiment, the computer terminal may execute program codes of the following steps in the text generation method: acquiring a target table, and converting data in the target table into a data structure diagram; coding the data structure chart through a coder to obtain a target vector matrix, wherein the coder is constructed by a coding module and a graph neural network; and decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, wherein the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware attention-spanning layer.
The computer terminal may further execute program codes of the following steps in the text generation method: inputting the data structure diagram into an encoder; processing the data structure chart through a coding module to obtain a first vector matrix; and updating the first vector matrix through information of adjacent nodes of each node in the data structure chart of the neural network aggregation to obtain a target vector matrix.
The computer terminal may further execute the program code of the following steps in the text generation method: aggregating information of neighboring nodes of each node in the data structure graph by the graph neural network using the following formula to update the first vector matrix:
Figure BDA0003571613040000142
Figure BDA0003571613040000143
wherein the content of the first and second substances,
Figure BDA0003571613040000144
gv，
Figure BDA0003571613040000145
and ev,uIs a function of the intermediate variable(s),
Figure BDA0003571613040000146
for the vector representation of the node v,
Figure BDA0003571613040000147
is a vector representation of the node u and,
Figure BDA0003571613040000148
is the attention weight of the neural network of the graph,
Figure BDA0003571613040000151
Wqand WkFor learnable parameters, m represents the hidden dimension of the graph neural network.
The computer terminal may further execute the program code of the following steps in the text generation method: calculating to obtain an initial decoding state vector through a data structure diagram; obtaining a representation of the first structured data according to the target vector matrix, the initial decoding state vector and the structure-perceived attention-spanning layer; according to the representation of the first structured data, modeling processing is carried out through a decoding module and a feedforward neural network adapter to obtain a second vector matrix; decoding the second vector matrix through a decoding module to obtain a first character corresponding to the data structure chart; acquiring a decoding state vector of the current decoding from the second vector matrix, and acquiring all characters corresponding to the data structure chart according to the second vector matrix, the decoding state vector and a structure-perceived crossing attention layer; and generating a target text according to all the obtained characters.
The computer terminal may further execute the program code of the following steps in the text generation method: constructing an initial decoding state vector and a data structure chart into a first evolutionary graph according to a target vector matrix; and sensing the first evolutionary graph through a cross attention layer of structure sensing to obtain a representation of the first structured data.
The computer terminal may further execute program codes of the following steps in the text generation method: s1, constructing the decoding state vector and the first evolutionary graph into a second evolutionary graph according to the second vector matrix; s2, pruning the target node in the second evolutionary graph through a dynamic pruning mechanism to obtain a processed second evolutionary graph; s3, sensing the processed second evolutionary graph through a structure sensing cross attention layer to obtain a representation of second structured data; s4, according to the representation of the second structured data, modeling is carried out through a decoding module and a feedforward neural network adapter to obtain a third vector matrix; s5, decoding the third vector matrix through a decoding module to obtain a second character corresponding to the data structure chart; and repeating the steps S1-S5 until all the characters corresponding to the data structure diagram are obtained.
The computer terminal may further execute the program code of the following steps in the text generation method: the gate for each node of the second evolutionary graph is represented as:
Figure BDA0003571613040000152
wherein, Wg，WeAnd WdAs a learnable parameter, hvAnd htIs an intermediate variable; taking the node with the gv smaller than a preset value as a target node; and pruning the target node through a dynamic pruning mechanism to obtain a processed second evolutionary graph.
Alternatively, fig. 9 is a block diagram of a computer terminal according to an embodiment of the present invention. As shown in fig. 9, the computer terminal 10 may include: one or more (only one shown) processors, memory.
The memory may be configured to store software programs and modules, such as program instructions/modules corresponding to the text generation method and apparatus in the embodiments of the present invention, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, so as to implement the text generation method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memories may further include a memory located remotely from the processor, which may be connected to the terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: acquiring a target table, and converting data in the target table into a data structure diagram; coding the data structure chart through a coder to obtain a target vector matrix, wherein the coder is constructed by a coding module and a graph neural network; and decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, wherein the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware cross-attention layer.
Optionally, the processor may further execute the program code of the following steps: inputting the data structure chart into an encoder; processing the data structure chart through a coding module to obtain a first vector matrix; and updating the first vector matrix through information of adjacent nodes of each node in the graph neural network aggregation data structure chart to obtain a target vector matrix.
Optionally, the processor may further execute the program code of the following steps: aggregating information of neighboring nodes of each node in the data structure graph by the graph neural network using the following formula to update the first vector matrix:
Figure BDA0003571613040000161
Figure BDA0003571613040000162
Figure BDA0003571613040000163
wherein the content of the first and second substances,
Figure BDA0003571613040000164
gv，
Figure BDA0003571613040000165
and ev,uIs the intermediate variable(s) of the variable,
Figure BDA0003571613040000166
for the vector representation of the node v,
Figure BDA0003571613040000167
is a vector representation of the node u and,
Figure BDA0003571613040000168
is the attention weight of the neural network of the graph,
Figure BDA0003571613040000169
Wqand WkFor a learnable parameter, m represents the hidden dimension of the graph neural network.
Optionally, the processor may further execute the program code of the following steps: calculating to obtain an initial decoding state vector through a data structure diagram; obtaining a representation of first structured data according to the target vector matrix, the initial decoding state vector and a structure-aware cross-attention layer; according to the representation of the first structured data, modeling processing is carried out through a decoding module and a feedforward neural network adapter to obtain a second vector matrix; decoding the second vector matrix through a decoding module to obtain a first character corresponding to the data structure chart; and acquiring a current decoding state vector from the second vector matrix, acquiring all characters corresponding to the data structure diagram according to the second vector matrix, the decoding state vector and the structure-perceived attention-crossing layer, and generating a target text according to all the acquired characters.
Optionally, the processor may further execute the program code of the following steps: constructing an initial decoding state vector and a data structure chart into a first evolutionary graph according to a target vector matrix; and sensing the first evolutionary graph through a cross attention layer of structure sensing to obtain a representation of the first structured data.
Optionally, the processor may further execute the program code of the following steps: s1, constructing the decoding state vector and the first evolutionary graph into a second evolutionary graph according to the second vector matrix; s2, pruning the target node in the second evolutionary graph through a dynamic pruning mechanism to obtain a processed second evolutionary graph; s3, sensing the processed second evolutionary graph through a structure sensing cross attention layer to obtain a representation of second structured data; s4, according to the representation of the second structured data, modeling is carried out through a decoding module and a feedforward neural network adapter to obtain a third vector matrix; s5, decoding the third vector matrix through a decoding module to obtain a second character corresponding to the data structure chart; and repeating the steps S1-S5 until all the characters corresponding to the data structure diagram are obtained.
Optionally, the processor may further execute the program code of the following steps: the gate for each node of the second evolutionary graph is represented as:
Figure BDA0003571613040000171
wherein, Wg，WeAnd WdAs a learnable parameter, hvAnd htIs an intermediate variable; taking the node with the gv smaller than a preset value as a target node; and pruning the target node through a dynamic pruning mechanism to obtain a processed second evolutionary graph.
By adopting the embodiment of the invention, the target table is obtained, and the data in the target table is converted into the data structure chart; coding the data structure chart through a coder to obtain a target vector matrix, wherein the coder is constructed by a coding module and a graph neural network; the target vector matrix is decoded by a decoder to obtain a target text corresponding to the data structure diagram, wherein the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware attention-spanning layer, and in the related technology, the decoding end takes the input data information as a disordered sequence, so that the structural information of the data information is lost, and the problem of low accuracy of converting the table data into the text is caused. The accuracy of converting table data into text is improved by constructing a decoder through an encoder and a decoding module constructed by an encoding module and a graph neural network, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware cross-attention layer.
It should be understood by those skilled in the art that the structure shown in fig. 9 is only an illustration, and the computer terminal may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 9 is a diagram illustrating a structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 9, or have a different configuration than shown in FIG. 9.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
Embodiments of the present invention also provide a computer-readable storage medium. Optionally, in this embodiment, the storage medium may be configured to store a program code executed by the text generation method provided in the first embodiment.
Optionally, in this embodiment, the storage medium may be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring a target table, and converting data in the target table into a data structure diagram; coding the data structure chart through a coder to obtain a target vector matrix, wherein the coder is constructed by a coding module and a graph neural network; and decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, wherein the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware attention-spanning layer.
Optionally, the storage medium is further configured to store program code for performing the following steps: inputting the data structure chart into an encoder; processing the data structure chart through a coding module to obtain a first vector matrix; and updating the first vector matrix through information of adjacent nodes of each node in the graph neural network aggregation data structure chart to obtain a target vector matrix.
Optionally, the storage medium is further configured to store program code for performing the following steps: aggregating information of neighboring nodes of each node in the data structure graph by the graph neural network using the following formula to update the first vector matrix:
Figure BDA0003571613040000181
Figure BDA0003571613040000182
wherein the content of the first and second substances,
Figure BDA0003571613040000183
gv，
Figure BDA0003571613040000184
and ev,uIs a function of the intermediate variable(s),
Figure BDA0003571613040000185
is a vector representation of the node v and,
Figure BDA0003571613040000186
is a vector representation of the node u and,
Figure BDA0003571613040000187
is the attention weight of the neural network of the graph,
Figure BDA0003571613040000188
Wqand WkFor a learnable parameter, m represents the hidden dimension of the graph neural network.
Optionally, the storage medium is further configured to store program code for performing the following steps: calculating to obtain an initial decoding state vector through a data structure diagram; obtaining a representation of first structured data according to the target vector matrix, the initial decoding state vector and a structure-aware cross-attention layer; according to the representation of the first structured data, modeling processing is carried out through a decoding module and a feedforward neural network adapter to obtain a second vector matrix; decoding the second vector matrix through a decoding module to obtain a first character corresponding to the data structure chart; acquiring a decoding state vector of the current decoding from the second vector matrix, and acquiring all characters corresponding to the data structure chart according to the second vector matrix, the decoding state vector and the structure perception crossing attention layer; and generating a target text according to all the obtained characters.
Optionally, the storage medium is further configured to store program code for performing the following steps: constructing an initial decoding state vector and a data structure chart into a first evolutionary graph according to a target vector matrix; and sensing the first evolutionary graph through a cross attention layer of structural sensing to obtain a representation of the first structured data.
Optionally, the storage medium is further configured to store program code for performing the following steps: s1, constructing the decoding state vector and the first evolutionary graph into a second evolutionary graph according to the second vector matrix; s2, pruning the target node in the second evolutionary graph through a dynamic pruning mechanism to obtain a processed second evolutionary graph; s3, sensing the processed second evolutionary graph through a structure sensing cross attention layer to obtain a representation of second structured data; s4, according to the representation of the second structured data, modeling is carried out through a decoding module and a feedforward neural network adapter to obtain a third vector matrix; s5, decoding the third vector matrix through a decoding module to obtain a second character corresponding to the data structure chart; and repeating the steps S1-S5 until all the characters corresponding to the data structure diagram are obtained.
Optionally, the storage medium is further configured to store program code for performing the following steps: the gate for each node of the second evolutionary graph is represented as:
Figure BDA0003571613040000191
wherein, Wg，WeAnd WdAs a learnable parameter, hvAnd htIs an intermediate variable; taking the nodes with the gv smaller than a preset numerical value as target nodes; and (4) pruning the target node through a dynamic pruning mechanism to obtain a processed second evolutionary graph.
The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, which can store program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims (11)

1. A method for generating text, comprising:
acquiring a target table, and converting data in the target table into a data structure diagram;
coding the data structure chart through a coder to obtain a target vector matrix, wherein the coder is constructed by a coding module and a graph neural network;
and decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, wherein the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware cross attention layer.
2. The method of claim 1, wherein the encoding the data structure diagram by an encoder to obtain the target vector matrix comprises:
inputting the data structure diagram into the encoder;
processing the data structure chart through the coding module to obtain a first vector matrix;
and aggregating information of adjacent nodes of each node in the data structure diagram through the graph neural network to update the first vector matrix to obtain the target vector matrix.
3. The method of claim 2, wherein the first vector matrix is updated by the graph neural network by aggregating information of neighboring nodes of each node in the data structure graph using the following formula:
Figure FDA0003571613030000011
Figure FDA0003571613030000012
Figure FDA0003571613030000013
Figure FDA0003571613030000014
Figure FDA0003571613030000015
wherein the content of the first and second substances,
Figure FDA0003571613030000016
gv，
Figure FDA0003571613030000017
and ev,uIs the intermediate variable(s) of the variable,
Figure FDA0003571613030000018
for the vector representation of the node v,
Figure FDA0003571613030000019
is a vector representation of the node u,
Figure FDA00035716130300000110
is the attention weight of the neural network of the graph,
Figure FDA00035716130300000111
Wqand WkFor learnable parameters, m represents the hidden dimension of the graph neural network.
4. The method of claim 1, wherein decoding the target vector matrix by a decoder to obtain a target text corresponding to the data structure diagram, comprises:
calculating to obtain an initial decoding state vector through the data structure chart;
obtaining a representation of first structured data according to the target vector matrix, the initial decoding state vector and the structure-perceived attention-spanning layer;
according to the representation of the first structured data, modeling processing is carried out through the decoding module and the feedforward neural network adapter to obtain a second vector matrix;
decoding the second vector matrix through the decoding module to obtain a first character corresponding to the data structure chart;
acquiring a decoding state vector of current decoding from the second vector matrix, and acquiring all characters corresponding to the data structure chart according to the second vector matrix, the decoding state vector and the structure perception crossing attention layer;
and generating the target text according to all the obtained characters.
5. The method of claim 4, wherein deriving a representation of structured data from the target vector matrix, the initial decoding state vector, and the structure-aware cross attention layer comprises:
constructing the initial decoding state vector and the data structure diagram into a first evolutionary diagram according to the target vector matrix;
and sensing the first evolutionary graph through the cross attention layer of the structural sensing to obtain a representation of the first structural data.
6. The method of claim 5, wherein obtaining all the words corresponding to the data structure diagram according to the second vector matrix, the decoding state vector and the structure-aware attention-crossing layer comprises:
s1, constructing the decoding state vector and the first evolutionary graph into a second evolutionary graph according to the second vector matrix;
s2, pruning the target node in the second evolutionary graph through the dynamic pruning mechanism to obtain a processed second evolutionary graph;
s3, sensing the processed second evolutionary graph through the structure-sensed attention-crossing layer to obtain a representation of second structured data;
s4, according to the representation of the second structured data, carrying out modeling processing through the decoding module and the feedforward neural network adapter to obtain a third vector matrix;
s5, decoding the third vector matrix through the decoding module to obtain a second character corresponding to the data structure chart;
and repeating the steps S1-S5 until all characters corresponding to the data structure diagram are obtained.
7. The method of claim 6, wherein pruning the target node in the second evolutionary graph through the dynamic pruning mechanism to obtain a processed second evolutionary graph comprises:
representing the gate of each node of the second evolutionary graph as:
Figure FDA0003571613030000031
wherein, Wg，WeAnd WdAs a learnable parameter, hvAnd htIs an intermediate variable;
g is prepared fromvTaking the nodes smaller than a preset value as the target nodes;
and pruning the target node through the dynamic pruning mechanism to obtain the processed second evolutionary graph.
8. A method for generating text, comprising:
receiving a target form sent by a client;
converting data in the target table into a data structure diagram in a cloud server, and encoding the data structure diagram through an encoder to obtain a target vector matrix, wherein the encoder is constructed by an encoding module and a diagram neural network, and decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, and the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-aware cross-attention layer;
and returning the target text to the client.
9. An apparatus for generating a text, comprising:
the conversion unit is used for acquiring a target table and converting data in the target table into a data structure chart;
the first processing unit is used for encoding the data structure diagram through an encoder to obtain a target vector matrix, wherein the encoder is constructed by an encoding module and a diagram neural network;
and the second processing unit is used for decoding the target vector matrix through a decoder to obtain a target text corresponding to the data structure diagram, wherein the decoder is constructed by a decoding module, a feedforward neural network adapter, a dynamic pruning mechanism and a structure-perceived attention-spanning layer.
10. A computer-readable storage medium, comprising a stored program, wherein when the program runs, the apparatus on which the storage medium is located is controlled to execute the text generation method according to any one of claims 1 to 8.
11. A processor, characterized in that the processor is configured to run a program, wherein the program when running performs the method for generating a text according to any one of claims 1 to 8.
CN202210320783.6A 2022-03-29 2022-03-29 Text generation method and device, storage medium and processor Pending CN114781331A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210320783.6A CN114781331A (en) 2022-03-29 2022-03-29 Text generation method and device, storage medium and processor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210320783.6A CN114781331A (en) 2022-03-29 2022-03-29 Text generation method and device, storage medium and processor

Publications (1)

Publication Number Publication Date
CN114781331A true CN114781331A (en) 2022-07-22

Family

ID=82425317

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210320783.6A Pending CN114781331A (en) 2022-03-29 2022-03-29 Text generation method and device, storage medium and processor

Country Status (1)

Country Link
CN (1) CN114781331A (en)

Similar Documents

Publication Publication Date Title
CN106657197B (en) File uploading method and device
CN111310440A (en) Text error correction method, device and system
WO2021130754A1 (en) Systems and methods of data compression
CN112749569A (en) Text translation method and device
CN113536856A (en) Image recognition method and system, and data processing method
CN106227881A (en) A kind of information processing method and server
CN110545106A (en) Method and device for coding time series data
CN113407851A (en) Method, device, equipment and medium for determining recommendation information based on double-tower model
CN113467777A (en) Path identification method, device and system
EP3474494A1 (en) Data processing method and device
CN112417874A (en) Named entity recognition method and device, storage medium and electronic device
CN114781331A (en) Text generation method and device, storage medium and processor
CN110502236B (en) Front-end code generation method, system and equipment based on multi-scale feature decoding
CN111079854A (en) Information identification method, device and storage medium
CN113761917A (en) Named entity identification method and device
CN115208881B (en) Block chain consensus method, equipment and storage medium
CN114299073A (en) Image segmentation method, image segmentation device, storage medium, and computer program
CN110929866B (en) Training method, device and system of neural network model
CN112232025B (en) Character string storage method and device and electronic equipment
CN111914540A (en) Statement identification method and device, storage medium and processor
CN114035822A (en) File updating method and equipment
CN111625628A (en) Information processing method and device, storage medium and processor
CN115017242A (en) Geo-fence generation method and device, storage medium and processor
CN111836051B (en) Desktop image encoding and decoding method and related device
CN111898387A (en) Translation method and device, storage medium and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination