CN113705168A - Cross-level attention mechanism-based chapter neural machine translation method and system - Google Patents
Cross-level attention mechanism-based chapter neural machine translation method and system Download PDFInfo
- Publication number
- CN113705168A CN113705168A CN202111016267.6A CN202111016267A CN113705168A CN 113705168 A CN113705168 A CN 113705168A CN 202111016267 A CN202111016267 A CN 202111016267A CN 113705168 A CN113705168 A CN 113705168A
- Authority
- CN
- China
- Prior art keywords
- sentence
- cross
- chapter
- machine translation
- level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/126—Character encoding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a cross-level attention mechanism-based discourse neural machine translation method, which comprises the following steps: generating a training corpus containing document structure information from the unprocessed corpus; training a discourse neural machine translation model of the basic structure by using the training corpus; acquiring a source end document sentence vector set with sentence boundary information by using a translation model; the sentence vector set is used as the input of a context capturer based on cross level attention, a cross attention mechanism is used for obtaining a dependency weight matrix between words and sentences, and an independent overall context at chapter level is obtained for each word in the current sentence; combining the global context with a translation model to obtain a discourse neural machine translation model based on a cross-attention mechanism; and carrying out neural network training on the obtained object. According to the invention, the global context information of the chapters is introduced in the training, the chapters to be translated are regarded as a whole for translation, and the error or missing of the translation caused by neglecting the context information can be avoided while the structure information of the chapters is maintained.
Description
Technical Field
The invention relates to the technical field of neural machine translation, in particular to a chapter neural machine translation method and system based on a cross-level attention mechanism.
Background
The existing common neural machine translation method is a neural machine translation method based on parallel sentence pairs, and model training is carried out by utilizing large-scale bilingual parallel linguistic data through a coder-decoder neural network framework. During training, a word-level attention mechanism is used for calculating the dependency relationship among all words in a sentence, and effective information of other words in the sentence where the words are located is obtained for the words.
However, the word-level attention mechanism can only acquire the dependency relationship between words in a single sentence, and neglects the relation and structural relationship between sentences in the document. The context-based benefits to the translation quality of documents have been supported by the research efforts of many researchers. Meanwhile, ignoring chapter context information also causes cumulative propagation of translation errors, which causes problems such as inconsistent reference.
Disclosure of Invention
Therefore, the technical problem to be solved by the present invention is to overcome the deficiencies in the prior art, and provide a method, an apparatus and a system for discourse neural machine translation based on a cross-level attention mechanism, which automatically sense the document boundary through a model, and provide a context capturer to obtain the context from the whole document for each word in the current sentence, thereby improving the translation quality and the translation fluency, and solving the problems such as inconsistent reference.
In order to solve the technical problem, the invention provides a cross-level attention mechanism-based discourse neural machine translation method, which comprises the following steps:
s1: generating a training corpus containing document structure information from the unprocessed corpus;
s2: training a discourse neural machine translation model of the basic structure by using the training corpus;
s3: acquiring a source end document sentence vector set with sentence boundary information by using the translation model;
s4: the sentence vector set is used as the input of a context capturer based on cross level attention, a cross attention mechanism is used for obtaining a dependency weight matrix between words and sentences, and an independent overall context at chapter level is obtained for each word in the current sentence;
s5: combining the overall context of the chapter level obtained in the S4 with the chapter neural machine translation model obtained in the S2 to obtain a chapter neural machine translation model based on a cross-attention mechanism;
s6: and carrying out neural network training on the discourse neural machine translation model based on the cross attention mechanism.
In one embodiment of the present invention, the corpus includes a source corpus and a target corpus.
In an embodiment of the present invention, the generating the corpus in the step S1 includes the following steps:
s1.1: processing the document into a single long sentence, and marking the document sentence boundary by using a special symbol;
s1.2: in the processing process, the long document is segmented according to the length to obtain sub-chapters with moderate length, for the segmented long document, the middle sub-chapter keeps partial sentences of the adjacent front and rear sub-chapters as a connection context, and if the number of the last sub-chapter sentences of the segmented sub-chapter set is too small, the last sub-chapter sentence is attached to the previous sub-chapter.
In one embodiment of the present invention, the ability of the discourse neural machine translation model of the infrastructure in step S2 includes:
the method has the capability of performing attention mechanism operation by taking chapters as units;
the method has the capability of translating the document by taking chapters as units;
the method has the necessary capacity in the process of carrying out translation model training by taking chapters as units;
the ability to maintain sentence alignment during the above operations.
In one embodiment of the present invention, the source end document sentence vector set having sentence boundary information in the step of S3 includes:
and generating a word-level hidden state which takes discourse as a unit and still keeps sentence independence by utilizing a neural network, and converting the word-level hidden state into a sentence vector set which takes sentences as a unit by utilizing a sentence embedding layer.
In one embodiment of the present invention, the step of S4, the dependency weight matrix between words and sentences and the obtaining of the independent overall context at chapter level for each word includes:
the word-level hidden state of the current sentence is obtained from a source material, a sentence vector set of the whole chapter is obtained to serve as a context source, the sentence vector representation and the word vector representation are operated through a cross-level attention mechanism to obtain a dependency relationship weight matrix of each word in the current sentence and all sentences in the chapter, and the whole context from the whole chapter is obtained for the words in the target sentence respectively.
In an embodiment of the present invention, the training of the translation model in the step S6 includes:
loading the parameters of the discourse neural machine translation model of the basic structure obtained in the step S2 into the discourse neural machine translation model based on the cross-attention mechanism;
freezing parameters shared by the discourse neural machine translation model of the basic structure in the discourse neural machine translation model based on the cross-attention mechanism;
training a discourse neural machine translation model based on a cross-attention mechanism.
In addition, the invention also provides a chapter neural machine translation system based on a cross-level attention mechanism, which comprises:
the corpus preprocessing module is used for generating a training corpus containing document structure information from the unprocessed corpus;
the basic translation model training module is used for training a discourse neural machine translation model of a basic structure by using training corpora;
a sentence vector acquisition module for acquiring a set of source end document sentence vectors having sentence boundary information by using the translation model;
the global context module is used for cooperating the sentence vector set as the input of the context capturer based on cross level attention, acquiring a dependency weight matrix between words and sentences by using a cross attention mechanism, and acquiring an independent overall context at a chapter level for each word in the current sentence;
the neural machine translation model obtaining module is used for combining the overall context at the chapter level with the chapter neural machine translation model to obtain a chapter neural machine translation model based on a cross-attention mechanism;
and the neural machine translation model training module is used for carrying out neural network training on the discourse neural machine translation model based on the cross-attention mechanism.
In an embodiment of the present invention, the corpus preprocessing module includes:
a corpus tagging unit for processing the document into a single long sentence, tagging document sentence boundaries with special symbols;
and the corpus segmentation unit is used for segmenting the long document according to the length in the processing process to obtain sub-chapters with moderate length, wherein for the segmented long document, the middle sub-chapter keeps partial sentences of the adjacent front and rear sub-chapters as a connection context, and if the number of the last sub-chapter sentences of the segmented sub-chapter set is too small, the last sub-chapter sentence is attached to the previous sub-chapter.
In one embodiment of the invention, the global context module comprises:
a word-level expression form encoding unit for encoding a context at a word level;
a sentence vector conversion unit for converting the word-level expression form of the context into a sentence-level expression form;
the cross level attention mechanism unit is used for acquiring a word/sentence level dependency weight relation matrix of a cross level;
and the global context allocation unit is used for allocating the acquired context to each word of the current sentence according to needs.
Compared with the prior art, the technical scheme of the invention has the following advantages:
according to the invention, the global context information from the whole chapter is introduced in the training, the chapter to be translated is regarded as a whole for translation, and the situation of wrong or missed translation caused by neglected context information can be avoided while the structure information of the chapter is maintained in the translation process.
Drawings
FIG. 1 is a flow chart of a chapter neural machine translation method based on a cross-level attention mechanism according to the present invention.
FIG. 2 is a schematic diagram of a method for generating a corpus containing document structure information from unprocessed corpus according to the present invention.
FIG. 3 is a diagram illustrating a method for utilizing a cross-attention mechanism global context according to the present invention.
FIG. 4 is a diagram illustrating a method for integrating global context with a discourse neural machine translation model according to the present invention.
FIG. 5 is a schematic diagram of the neural network training method for the discourse neural machine translation model based on the cross-attention mechanism according to the present invention.
FIG. 6 is a schematic structural diagram of a chapter neural machine translation system based on a cross-level attention mechanism according to the present invention.
FIG. 7 is a schematic diagram of the corpus pre-processing module according to the present invention.
FIG. 8 is a block diagram of the global context module of the present invention.
The reference numbers in the figures illustrate: 10. a corpus preprocessing module; 101. a corpus tagging unit; 102. a corpus segmentation unit; 20. a basic translation model training module; 30. a sentence vector acquisition module; 40. a global context module; 401. a word-level expression form encoding unit; 402. a sentence vector conversion unit; 403. a cross-level attention mechanism unit; 404. a global context allocation unit; 50. a neural machine translation model obtaining module; 60. and a neural machine translation model training module.
Detailed Description
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
Example one
Referring to fig. 1 to 8, an embodiment of the present invention provides a discourse neural machine translation method based on a cross-level attention mechanism, including:
s1: generating a training corpus containing document structure information from the unprocessed corpus;
s2: training a discourse neural machine translation model of the basic structure by using the training corpus;
s3: acquiring a source end document sentence vector set with sentence boundary information by using the translation model;
s4: the sentence vector set is used as the input of a context capturer based on cross level attention, a cross attention mechanism is used for obtaining a dependency weight matrix between words and sentences, and an independent overall context at chapter level is obtained for each word in the current sentence;
s5: combining the overall context of the chapter level obtained in the S4 with the chapter neural machine translation model obtained in the S2 to obtain a chapter neural machine translation model based on a cross-attention mechanism;
s6: and carrying out neural network training on the discourse neural machine translation model based on the cross attention mechanism.
The working principle is as follows: the dependency relationship between the sentence from the whole chapter and the characters in the current sentence is directly calculated, so that the corresponding characters can obtain the specific global context and are added into the neural training process together, and the final translation model can enable the global context from the chapter to directly participate in the translation process. Context information from the overall situation of the chapters is introduced in training, sentence sets in the documents are used as candidate contexts and are distributed to words in the current sentences, the document mechanism of the contexts can be fully utilized in the translation process, and each word can fully acquire the overall context.
Wherein, the step of generating the corpus in the step of S1 includes the following steps:
s1.1: processing the document into a single long sentence, and marking the document sentence boundary by using a special symbol;
s1.2: in the processing process, the long document is segmented according to the length to obtain sub-chapters with moderate length, for the segmented long document, the middle sub-chapter keeps partial sentences of the adjacent front and rear sub-chapters as a connection context, and if the number of the last sub-chapter sentences of the segmented sub-chapter set is too small, the last sub-chapter sentence is attached to the previous sub-chapter. The method can greatly reduce the number of sentences in a single chapter, reduce the calculated amount, shorten the training time, reduce the model parameters and ensure more reliable technology.
Wherein, the capability of the discourse neural machine translation model of the basic structure in the step of S2 includes: the method has the capability of performing attention mechanism operation by taking chapters as units; the method has the capability of translating the document by taking chapters as units; the method has the necessary capacity in the process of carrying out translation model training by taking chapters as units; the ability to maintain sentence alignment during the above operations.
The source-end document sentence vector set with sentence boundary information in the step S3 includes generating word-level hidden states in units of chapters and still maintaining sentence independence using a neural network, and converting the word-level hidden states into a sentence vector set in units of sentences using a sentence embedding layer.
In the step S4, obtaining a dependency weight matrix between words and sentences by using a cross-attention mechanism, and obtaining an independent overall context at a chapter level for each word in the current sentence includes: encoding a sentence into a vector expression form at a character level by using an encoder; converting a document into an expression form of a sentence vector set by using a sentence vector embedding layer, namely, each sentence is represented by one vector, and each chapter is represented by a set consisting of a plurality of sentence vectors; acquiring a dependency relationship weight matrix between each word in the current sentence and all sentences in the document through a cross hierarchy attention mechanism; and distributing the context according to the dependency relationship weight matrix so as to obtain a unique global context for each character in the current sentence.
The working principle is as follows: the dependency relationship weight matrix obtained by calculation represents the dependency relationship of each character in the current sentence on other sentences in the full chapter, namely the degree of closeness of the relation between the two; the sentence embedding is characterized in that a vector expression form of all characters in a sentence is used, the weight sum is calculated according to the importance degree between the characters in the sentence, so that the aim of obtaining a sentence vector is fulfilled, the model calculation amount can be reduced by using a single vector to express the sentence, and meanwhile, the weight sum is calculated by utilizing the importance of the characters, and the information carried by the important characters in the original sentence can be reserved as much as possible; and assigning the sentence vectors to each character in the current sentence according to the importance of the weight matrix obtained by calculation, so that the characters obtain unique global context from the full text. It provides a specific method for obtaining the overall context with the overall summary information of full chapters from the current sentence. By combining the sentence vectors and the relation weight matrix, the global context can be accurately and efficiently selected and obtained, and meanwhile, the calculation time and the calculation power overhead in the process are greatly reduced.
As shown in fig. 4, combining the global context at chapter level with the chapter neural machine translation model to obtain the chapter neural machine translation model based on the cross-attention mechanism includes: segmenting the corpus according to a natural document structure; obtaining context for source language chapters; combining the context with the source language chapters; and inputting the chapters combined with the chapter context as the source end of the translation model.
As shown in fig. 5, the neural network training of the cross-attention mechanism-based discourse neural machine translation model at least includes the following contents: training a translation model with a conventional coder-decoder structure; loading the parameters of the trained model and the model multiplexing into the discourse neural machine translation model based on the cross attention mechanism; freezing the trained multiplexing parameters; training a complete chapter time machine translation model based on the cross-attention mechanism provided by the invention.
The working principle is as follows: the translation model trained in the two-step training mode has better performance, and the parameters of the model trained in the first step are used as the initialization parameters of the complete large model, so that the training speed is high, and the calculation cost is relatively low. The method can greatly reduce the training time and improve the performance of the translation system.
Example two
The embodiment of the invention provides a chapter neural machine translation system based on a cross-level attention mechanism, which comprises:
the corpus preprocessing module 10 is used for generating a training corpus containing document structure information from unprocessed corpus;
a basic translation model training module 20 for training a discourse neural machine translation model of a basic structure by using training corpus;
a sentence vector obtaining module 30, configured to obtain a set of sentence vectors of a source-end document having sentence boundary information by using the translation model;
a global context module 40, configured to cooperate the sentence vector set as an input of a cross-level attention-based context capturer, obtain a dependency weight matrix between words and sentences by using a cross-attention mechanism, and obtain an independent overall context at a chapter level for each word in a current sentence;
a neural machine translation model obtaining module 50, configured to combine the overall context at the chapter level with the chapter neural machine translation model to obtain a chapter neural machine translation model based on a cross-attention mechanism;
and a neural machine translation model training module 60 for performing neural network training on the discourse neural machine translation model based on the cross-attention mechanism.
Wherein, the corpus preprocessing module 10 includes:
a corpus tagging unit 101 for processing a document into a single long sentence, tagging document sentence boundaries with special symbols;
the corpus segmentation unit 102 is configured to segment the long document according to the length during the processing process to obtain sub-chapters with an appropriate length, wherein for the segmented long document, a middle sub-chapter retains partial sentences of adjacent front and rear sub-chapters as a connection context, and if the number of last sub-chapter sentences of the segmented sub-chapter set is too small, the middle sub-chapter is attached to the previous sub-chapter.
Wherein the global context module 40 comprises:
a word-level expression form encoding unit 401 for encoding a context at a word level;
a sentence vector conversion unit 402 for converting the word-level expression form of the context into a sentence-level expression form
An intersection level attention mechanism unit 403, configured to obtain a word/sentence level dependency weight relationship matrix of an intersection level;
a global context assigning unit 404 for assigning the obtained context to each word of the current sentence as required.
The above embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.
Claims (10)
1. A discourse neural machine translation method based on a cross-level attention mechanism is characterized by comprising the following steps:
s1: generating a training corpus containing document structure information from the unprocessed corpus;
s2: training a discourse neural machine translation model of the basic structure by using the training corpus;
s3: acquiring a source end document sentence vector set with sentence boundary information by using the translation model;
s4: the sentence vector set is used as the input of a context capturer based on cross level attention, a cross attention mechanism is used for obtaining a dependency weight matrix between words and sentences, and an independent overall context at chapter level is obtained for each word in the current sentence;
s5: combining the overall context of the chapter level obtained in the S4 with the chapter neural machine translation model obtained in the S2 to obtain a chapter neural machine translation model based on a cross-attention mechanism;
s6: and carrying out neural network training on the discourse neural machine translation model based on the cross attention mechanism.
2. The discourse neural machine translation method based on the cross-level attention mechanism as claimed in claim 1, wherein: the corpus comprises a source corpus and a target corpus.
3. The discourse neural machine translation method based on the cross-level attention mechanism as claimed in claim 2, wherein: the step of generating the corpus in the step of S1 includes the following steps:
s1.1: processing the document into a single long sentence, and marking the document sentence boundary by using a special symbol;
s1.2: in the processing process, the long document is segmented according to the length to obtain sub-chapters with moderate length, for the segmented long document, the middle sub-chapter keeps partial sentences of the adjacent front and rear sub-chapters as a connection context, and if the number of the last sub-chapter sentences of the segmented sub-chapter set is too small, the last sub-chapter sentence is attached to the previous sub-chapter.
4. The discourse neural machine translation method based on the cross-level attention mechanism as claimed in claim 1, wherein: the capabilities of the discourse neural machine translation model of the infrastructure in step S2 include:
the method has the capability of performing attention mechanism operation by taking chapters as units;
the method has the capability of translating the document by taking chapters as units;
the method has the necessary capacity in the process of carrying out translation model training by taking chapters as units;
the ability to maintain sentence alignment during the above operations.
5. The discourse neural machine translation method based on the cross-level attention mechanism as claimed in claim 1, wherein: the set of source-side document sentence vectors having sentence boundary information in step S3 includes:
and generating a word-level hidden state which takes discourse as a unit and still keeps sentence independence by utilizing a neural network, and converting the word-level hidden state into a sentence vector set which takes sentences as a unit by utilizing a sentence embedding layer.
6. The discourse neural machine translation method based on the cross-level attention mechanism as claimed in claim 1, wherein: the step S4, the dependency weight matrix between words and sentences and the global context of each word at independent chapter level includes:
the word-level hidden state of the current sentence is obtained from a source material, a sentence vector set of the whole chapter is obtained to serve as a context source, the sentence vector representation and the word vector representation are operated through a cross-level attention mechanism to obtain a dependency relationship weight matrix of each word in the current sentence and all sentences in the chapter, and the whole context from the whole chapter is obtained for the words in the target sentence respectively.
7. The discourse neural machine translation method based on the cross-level attention mechanism as claimed in claim 1, wherein: the training of the translation model in the step S6 includes:
loading the parameters of the discourse neural machine translation model of the basic structure obtained in the step S2 into the discourse neural machine translation model based on the cross-attention mechanism;
freezing parameters shared by the discourse neural machine translation model of the basic structure in the discourse neural machine translation model based on the cross-attention mechanism;
training a discourse neural machine translation model based on a cross-attention mechanism.
8. A discourse neural machine translation system based on a cross-level attention mechanism is characterized by comprising:
the corpus preprocessing module is used for generating a training corpus containing document structure information from the unprocessed corpus;
the basic translation model training module is used for training a discourse neural machine translation model of a basic structure by using training corpora;
a sentence vector acquisition module for acquiring a set of source end document sentence vectors having sentence boundary information by using the translation model;
the global context module is used for cooperating the sentence vector set as the input of the context capturer based on cross level attention, acquiring a dependency weight matrix between words and sentences by using a cross attention mechanism, and acquiring an independent overall context at a chapter level for each word in the current sentence;
the neural machine translation model obtaining module is used for combining the overall context at the chapter level with the chapter neural machine translation model to obtain a chapter neural machine translation model based on a cross-attention mechanism;
and the neural machine translation model training module is used for carrying out neural network training on the discourse neural machine translation model based on the cross-attention mechanism.
9. The discourse neural machine translation system based on the cross-level attention mechanism as claimed in claim 8, wherein: the corpus preprocessing module comprises:
a corpus tagging unit for processing the document into a single long sentence, tagging document sentence boundaries with special symbols;
and the corpus segmentation unit is used for segmenting the long document according to the length in the processing process to obtain sub-chapters with moderate length, wherein for the segmented long document, the middle sub-chapter keeps partial sentences of the adjacent front and rear sub-chapters as a connection context, and if the number of the last sub-chapter sentences of the segmented sub-chapter set is too small, the last sub-chapter sentence is attached to the previous sub-chapter.
10. The discourse neural machine translation system based on the cross-level attention mechanism as claimed in claim 8, wherein: the global context module comprises:
a word-level expression form encoding unit for encoding a context at a word level;
a sentence vector conversion unit for converting the word-level expression form of the context into a sentence-level expression form;
the cross level attention mechanism unit is used for acquiring a word/sentence level dependency weight relation matrix of a cross level;
and the global context allocation unit is used for allocating the acquired context to each word of the current sentence according to needs.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111016267.6A CN113705168B (en) | 2021-08-31 | 2021-08-31 | Chapter neural machine translation method and system based on cross-level attention mechanism |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111016267.6A CN113705168B (en) | 2021-08-31 | 2021-08-31 | Chapter neural machine translation method and system based on cross-level attention mechanism |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113705168A true CN113705168A (en) | 2021-11-26 |
CN113705168B CN113705168B (en) | 2023-04-07 |
Family
ID=78658346
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111016267.6A Active CN113705168B (en) | 2021-08-31 | 2021-08-31 | Chapter neural machine translation method and system based on cross-level attention mechanism |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113705168B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114580439A (en) * | 2022-02-22 | 2022-06-03 | 北京百度网讯科技有限公司 | Translation model training method, translation device, translation equipment and storage medium |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190129947A1 (en) * | 2017-10-26 | 2019-05-02 | Electronics And Telecommunications Research Institute | Neural machine translation method and apparatus |
CN109783809A (en) * | 2018-12-22 | 2019-05-21 | 昆明理工大学 | A method of alignment sentence is extracted from Laos-Chinese chapter grade alignment corpus |
-
2021
- 2021-08-31 CN CN202111016267.6A patent/CN113705168B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190129947A1 (en) * | 2017-10-26 | 2019-05-02 | Electronics And Telecommunications Research Institute | Neural machine translation method and apparatus |
CN109783809A (en) * | 2018-12-22 | 2019-05-21 | 昆明理工大学 | A method of alignment sentence is extracted from Laos-Chinese chapter grade alignment corpus |
Non-Patent Citations (2)
Title |
---|
LINQING CHEN等: "Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training", 《PROCEEDINGS OF THE 59TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS AND THE 11TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING》 * |
陈林卿等: "层次化结构全局上下文增强的篇章级神经机器翻译", 《PROCEEDINGS OF THE 19TH CHINESE NATIONAL CONFERENCE ON COMPUTATIONAL LINGUISTICS》 * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114580439A (en) * | 2022-02-22 | 2022-06-03 | 北京百度网讯科技有限公司 | Translation model training method, translation device, translation equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN113705168B (en) | 2023-04-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109684648B (en) | Multi-feature fusion automatic translation method for ancient and modern Chinese | |
CN109359291A (en) | A kind of name entity recognition method | |
US10902218B2 (en) | System and method for adaptive quality estimation for machine translation post-editing | |
CN110162789B (en) | Word representation method and device based on Chinese pinyin | |
CN115270826B (en) | Multilingual translation model construction method, translation method and computer storage medium | |
CN112446224A (en) | Parallel corpus processing method, device and equipment and computer readable storage medium | |
CN112926344B (en) | Word vector replacement data enhancement-based machine translation model training method and device, electronic equipment and storage medium | |
CN112417897B (en) | Method, system, device and medium for training word alignment model and processing text | |
CN113705168B (en) | Chapter neural machine translation method and system based on cross-level attention mechanism | |
CN112966529A (en) | Neural network machine translation training method, system, medium, equipment and application | |
CN111814493B (en) | Machine translation method, device, electronic equipment and storage medium | |
CN115935914A (en) | Admission record missing text supplementing method | |
Samudravijaya | Indian language speech label (ILSL): a de facto national standard | |
Sun et al. | Alternative input signals ease transfer in multilingual machine translation | |
CN113204978B (en) | Machine translation enhancement training method and system | |
CN112397050B (en) | Prosody prediction method, training device, electronic equipment and medium | |
Iranzo-Sánchez et al. | From simultaneous to streaming machine translation by leveraging streaming history | |
Che et al. | A word segmentation method of ancient Chinese based on word alignment | |
CN115130481A (en) | Model training method, model training device, model translation device, model training equipment and model translation equipment, and storage medium | |
CN106896935A (en) | Input method | |
Minyun | Machine Translation Based on Neural Network: Challenge or Chance? | |
Aldón Mínguez et al. | Neural machine translation using bitmap fonts | |
JP2019159743A (en) | Correspondence generation program, correspondence generation device, correspondence generation method, and translation program | |
CN116306607B (en) | Multilingual mixed corpus generation method, device, equipment and storage medium | |
CN115204197A (en) | Machine translation model training method, device, equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |