WO2019114695A1 - 基于翻译模型的训练方法、翻译方法、计算机设备及存储介质 - Google Patents
基于翻译模型的训练方法、翻译方法、计算机设备及存储介质 Download PDFInfo
- Publication number
- WO2019114695A1 WO2019114695A1 PCT/CN2018/120293 CN2018120293W WO2019114695A1 WO 2019114695 A1 WO2019114695 A1 WO 2019114695A1 CN 2018120293 W CN2018120293 W CN 2018120293W WO 2019114695 A1 WO2019114695 A1 WO 2019114695A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- sentence
- model
- statement
- source
- target
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/126—Character encoding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
Definitions
- the present application relates to machine translation (MT) technology, and in particular to a training method based on a translation model, a translation method, a computer device, and a storage medium.
- MT machine translation
- NMT Neural Machine Translation
- the neural network machine translation system is built on the encoder-decoder framework.
- the decoder has multiple tasks, such as recording the current translated content, and the content that needs to be translated, and recording the translation. Information about fluency, etc. Due to the complexity of the decoder task, it can not balance all tasks, resulting in the current translation accuracy of the neural network machine translation system is not high, the phenomenon of missed and repeated translation is serious.
- a training method based on a translation model, a translation method, a computer device, and a storage medium are provided.
- a training method based on a translation model including:
- the computer device inputs a source statement to the translation model, and obtains a target statement output by the translation model;
- the computer device determines a loyalty of the target statement to the source statement
- the computer device uses the target sentence and the reference sentence as input of the discriminator model, and uses the fidelity as an output of the discriminator model, and trains the discriminator model to calculate the target sentence and the reference sentence. Similarity performance
- the computer device uses the discriminator model to output a similarity between the target sentence and the reference sentence;
- the computer device uses the source sentence as an input of the translation model, the target sentence as an output of the translation model, and uses the similarity as a weight coefficient to train the translation model according to an input source sentence Output the performance of the corresponding target statement.
- a translation method based on a translation model including:
- the computer device acquires a translation model trained in the confrontation network
- the translation model is obtained by using a source sentence as a translation model input, a target sentence as a translation model, and a similarity of the target sentence and the reference sentence as a weight coefficient in the confrontation network, the similarity being obtained. Calculated for using the discriminator model of the confrontation network;
- the computer device inputs a statement to be translated to the training model obtained by the training;
- the computer device calculates a semantic vector corresponding to the statement to be translated using the trained translation model
- the computer device outputs the translation result statement corresponding to the statement to be translated with the semantic vector as an input of the training model obtained by the training.
- a computer device comprising:
- a memory for storing computer readable instructions
- a translation model-based training method as described above is implemented when one or more processors are used to execute computer readable instructions stored in the memory.
- the one or more storage media storing computer readable instructions that, when executed by one or more processors, implement the translation model based training method described above.
- a computer device comprising:
- a memory for storing computer readable instructions
- the translation model-based translation method described above is implemented when one or more processors are used to execute computer readable instructions stored in the memory.
- the one or more storage media storing computer readable instructions that, when executed by the processor, implement the translation model based translation method described above.
- FIG. 1 is a schematic diagram of an encoder model-decoder model framework introducing an attention model according to an embodiment of the present application
- FIG. 2 is a block diagram of a translation model provided by an embodiment of the present application.
- FIG. 3 is a schematic diagram of a working principle of generating a discriminator model in an anti-network model according to an embodiment of the present application
- FIG. 4 is a schematic flowchart of a training method based on a translation model according to an embodiment of the present application
- FIG. 5 is a schematic diagram of a training system architecture based on a translation model according to an embodiment of the present application.
- FIG. 6 is a schematic diagram of a translation process based on a translation model according to an embodiment of the present application.
- FIG. 7 is a schematic diagram of an example of word alignment in a sentence provided by an embodiment of the present application.
- FIG. 8 is a block diagram of a training apparatus based on a translation model according to an embodiment of the present application.
- FIG. 9 is a block diagram of a training apparatus based on a translation model according to another embodiment of the present application.
- FIG. 10 is a schematic diagram of an application scenario of a translation model according to an embodiment of the present application.
- FIG. 11 is a schematic diagram of an application scenario of a translation model according to another embodiment of the present disclosure.
- FIG. 12 is a block diagram of a translation model-based translation apparatus according to an embodiment of the present application.
- a method or apparatus comprising a plurality of elements includes not only the Elements, but also other elements not explicitly listed, or elements that are inherent to the implementation of the method or device.
- an element defined by the phrase “comprising a " does not exclude the presence of additional related elements in the method or device including the element (eg, a step in the method or a unit in the device)
- a unit may be a partial circuit, a partial processor, a partial program or software, or the like.
- the translation model-based training method and the translation model-based translation method provided by the embodiments of the present application include a series of steps, but the translation model-based training method and the translation model-based translation method provided by the embodiments of the present application are not limited.
- the training device of the translation model provided by the embodiment of the present application includes a series of units, but the apparatus provided in the embodiment of the present application is not limited to including the unit explicitly described, and may further include Or the unit that needs to be set when processing based on information.
- the source statement which is a statement corresponding to a natural language of the translation model before the language conversion.
- Reference statement a preset reference standard corresponding to a certain natural language after language conversion of the source sentence.
- Coverage a parameter between 0 and 1 that represents the proportion of the content of the target statement that covers the content of the source statement.
- Translation a statement that converts a natural language statement into another natural language.
- FIG. 1 is a schematic diagram of an encoder model-decoder model framework introducing an attention model.
- the source statement is input into the translation model, and the target statement corresponding to the translation result of the source sentence is output; wherein the source statement and the target statement are sentences of different natural languages.
- the target statement output by the translation model can be obtained by: using the encoder model of the translation model to calculate the semantic vector of the corresponding source sentence; based on the semantic vector of the decoder model in the input translation model, and the decoder when inputting the semantic vector
- the state of the model, the last output of the decoder model, the alignment probability of the candidate words and the words in the source sentence; the candidate words with the highest alignment probability are selected as the output words; the corresponding output of each word for the source sentence is connected Words form the target statement.
- the semantic vector of the corresponding source sentence can be calculated by: using the encoder model of the translation model to generate a vector sequence corresponding to the source sentence; using the attention model to assign the attention probability to the word vector in the vector sequence of the source sentence; The word vector in the vector sequence of the statement is weighted and summed with the assigned attention probability to obtain the semantic vector of the corresponding source sentence.
- FIG. 2 is a schematic diagram of a translation model based architecture according to an embodiment of the present application.
- the source statement is “we are good friends” as an example, and the process of outputting a target sentence by the translation model in the embodiment of the present application is described.
- the source statement is input into the encoder model in the translation model.
- the encoder model encodes the input source statement to obtain a vector sequence corresponding to the source sentence.
- the encoded vector sequence is represented as h j
- h j is the jth in the source statement.
- the semantic coding corresponding to the words is as shown in the label 1 part of FIG. 2 (ie, a sequence of multiple circles in a rectangular frame, where " ⁇ eos>" is a sentence terminator).
- the attention model uses the attention model to read the vector sequence of the source sentence and the state of the decoder model in the translation model (s t-1 in Fig. 2), and assign the attention probability to each word vector in the vector sequence of the source sentence.
- the assigned attention probability is expressed as a ij
- a ij indicates the attention probability of the j-th word in the source sentence when the decoder model of the translation model outputs the i-th word, as shown in FIG. 2 , 0.1, 0.6, 0.1, 0.1”, the higher the value of a ij , the more attention is expressed that the i-th output is allocated on the j-th input, and the j-th input is affected when the i-th output is generated. The bigger it is.
- the word vector in the vector sequence of the source sentence is weighted and summed with the assigned attention probability to obtain a semantic vector corresponding to the source sentence, and the semantic vector is represented as c i .
- the obtained semantic vector corresponding to the source sentence is input into the decoder model, and the decoder model is read into the semantic vector c i (representing the word to be translated at the current moment), the previous time decoder model state s t-1 , decoding
- the word y t-1 output last time of the model outputs the decoder model state s t at the current time.
- the decoder model then calculates the alignment probability of the candidate words and the words in the source sentence based on the input semantic vector c i , the current decoder model state s t , and the last output word y t-1 of the decoder model, such as when translating words " When the "good”, the alignment probabilities corresponding to the candidate words "we”, “good”, and “friends” are 0.1, 0.6, and 0.1, respectively, and the "good” with the highest alignment probability is selected as the output of the decoder model, and the sequential connections are for the source. Each word of the statement corresponds to the output word, forming a target statement.
- the decoder model takes the decoder model state, the last output word, etc. as input, and outputs the words corresponding to the words in the source sentence. Therefore, it is necessary to record a lot of information, such as : The content to be translated, the content of the current translation, the fluency of the overall translation, etc., that is, the decoder model has multiple tasks, which leads to the serious translation and repeated translation of the above translation model in practical applications. phenomenon.
- the translation model can be trained by generating a GAN (Generative Adversarial Network) model, and the translation model is used as a generator model in the generation of the confrontation network model, and the convolutional neural network is used.
- the model is used as a discriminator model in the generated confrontation network model, and a generated confrontation network model composed of a generator model-discriminator model is constructed.
- the discriminator model is a two-classifier, which can rely on the probability to determine whether the input sentence is translated by humans. Its function is to distinguish the real-translated statement from the statement translated by the generator model (translation model) as much as possible. Referring to FIG. 3, FIG. 3, FIG.
- FIG. 3 is a schematic diagram of a working principle of generating a discriminator model in an anti-network model according to an embodiment of the present application, and inputting real sample data into a discriminator model, and the discriminator model is calculated to make the probability close to 1 (true) ), and input the noise obeying a certain distribution into the generator model, the generator model is calculated to generate a generated sample, and then the generated sample is input into the discriminator model, and the discriminator model is calculated to make the output probability as 0 (
- the purpose of the generator is to simulate human-translated statements (real translations) as much as possible, so that the discriminator model cannot determine whether the input statement is translated by humans or generators. Model) translated.
- the generator captures the distribution of the sample data, and generates a sample similar to the real training data by obeying the noise of a certain distribution (uniform distribution, Gaussian distribution, etc.), that is, generating a target statement similar to the reference sentence.
- the discriminator model combines the word vectors corresponding to each word in the source statement and the target sentence to form a three-dimensional "image", and inputs the three-dimensional "image” into the discriminator model, and outputs the probability that the target sentence is a real translation.
- the probability size includes two types, namely 0 and 1, respectively, when the probability is 0, the target statement is false, that is, non-human (machine) translation, and when the probability is 1, the target statement is true. That is translated by humans.
- ⁇ G is the parameter of the generator
- the generator For the gradient the parameter x is the input of the generator (source statement)
- y' is the output of the generator (target statement)
- D(x, y') is y' is the probability of true translation of x.
- the generator parameters are updated by the following formula:
- ⁇ ' is the parameter of the updated generator, and ⁇ is the learning rate; it can be seen that when D(x, y') is larger, the corresponding The larger the value, the larger the parameters that the generator updates.
- the use of the generated confrontation network model to train the translation model can partially improve the accuracy of the translation model translation.
- the objective function of the discriminator model is not accurate enough during the training of the translation model, all machine translation sentences are corresponding.
- the probability identifier is 0, and the probability corresponding to all human translated sentences is marked as 1.
- the quality of the machine translated statements is jagged, not all machine translated statements are not desirable, and the machine translation preferred statements correspond.
- the probability of being identified as 0 is bound to result in inaccurate parameter updates during model training.
- the training scheme of the translation model of the embodiment of the present application is proposed.
- the training scheme of the translation model of the embodiment of the present application is based on generating a confrontation network model.
- the translation model is used as a generator in the generated confrontation network model, and the discriminator model is added to implement the training of the translation model.
- the training scheme of the translation model in the embodiment of the present application will be described in detail with reference to specific embodiments.
- FIG. 4 is an optional schematic flowchart of a training method based on a translation model according to an embodiment of the present disclosure.
- FIG. 5 is an optional schematic diagram of a training system architecture based on a translation model according to an embodiment of the present application. As shown in FIG. 5, the translation model-based training method of the embodiment of the present application includes:
- Step 101 Input a source statement to the translation model to obtain a target statement output by the translation model.
- FIG. 6 is a schematic diagram of a translation process of a translation model according to an embodiment of the present application.
- a translation process of a translation model (translating a source language of a natural language into a target statement of another natural language) ) mainly includes the following three steps:
- Step 1 Calculate the semantic vector of the corresponding source statement using the encoder model of the translation model, that is, generate the source vector representation c i .
- the source end corresponding to the encoder model end mentioned in the embodiment of the present application corresponds to the decoder model end
- the source statement inputs the encoder model
- the encoder model encodes the source sentence to obtain a vector sequence composed of at least one word vector.
- each word vector corresponds to a word in the source sentence.
- the source sentence is “I am from China”
- the code vector is encoded by the encoder model to obtain the word vector h 1 corresponding to “I”, “From”, “China”
- the source vector sequence consisting of h 2 and h 3 then the attention model is read into the vector sequence of the encoder model and the state of the decoder model before, assigning a value between 0 and 1 for each word in the source statement.
- the attention probability such as "I”, “From”, “China”, the corresponding attention probability is 1, 0, 0, respectively, the higher the value, the more relevant the word and the target word to be generated, the attention probability And the vector sequence weighted summation generates the source end vector representation c i of the current time, that is, obtains the semantic vector c i of the corresponding source sentence,
- L x represents the length of the word of the source sentence
- a ij represents the attention probability of the jth word in the source sentence when the decoder model of the translation model outputs the i-th word
- h j is the j-th word in the source sentence Corresponding semantic coding.
- Step 2 Update the decoder model status.
- the decoder model reads the source vector representation (ie, the content of the source statement to be translated at the current time), the previous time decoder model state s t-1 , the previous word y t-1 output by the decoder model, and outputs the current The decoder model state s t at the moment.
- Step 3 The decoder model generates the target words at the current moment.
- the decoder model calculates the alignment probability of the candidate words and the words in the source sentence according to the decoder state s t at the current moment, the source vector representation c t , and the previous generated word y t-1 , and selects the highest alignment probability (similarity).
- the decoder model After the decoder model obtains the target words corresponding to the words in the source sentence, the words corresponding to the output of the words corresponding to the source sentence are connected to form a target sentence.
- Step 102 Determine the loyalty of the target statement to the source statement.
- the coverage of the target sentence can be used to represent the loyalty of the target sentence, that is, the coverage ratio is used as the standard for the translation accuracy of the source sentence, and the larger the value, the higher the accuracy of the translation.
- FIG. 5 is a system architecture for performing translation model training with the loyalty as the coverage rate.
- the source sentence and the reference sentence are input into the translation model, and the first alignment matrix of the target sentence and the source sentence is obtained through the translation model translation, and Force decoding to obtain the second alignment matrix of the reference statement and the source statement, and then input the first alignment matrix and the second alignment matrix into the coverage calculation module, and the coverage calculation module calculates the coverage of the target sentence (ie, the coverage dispersion) Enter the discriminator model.
- the coverage of the target statement can be determined by:
- the first number of words of the target statement overlay source statement may be obtained by:
- the vector of each word in the target sentence is aligned with the vector of the corresponding word in the source sentence to form a first alignment matrix; and the target is obtained from the first alignment matrix.
- the words in the statement override the first number of words in the source statement.
- the target statement and the source statement can be realized through the attention model. Align, get the alignment matrix, and then count the number of words that the target statement overwrites the source statement. Referring to FIG. 7, FIG.
- FIG. 7 is a schematic diagram of an example of word alignment in a sentence according to an embodiment of the present application.
- a 26-year-old male driver is trapped inside a car and rescued by a firefighter at the scene.
- the portion indicated by the reference numeral 1 indicates that the words in the target sentence are aligned with the words in the source sentence, that is, the words that are overwritten to the source sentence.
- the first number of words covering the source sentence in the target sentence in FIG. 7 is 4.
- the second coverage number of the words of the source statement may be obtained by the words of the reference sentence obtained by:
- the alignment information of the reference sentence can be obtained by forced decoding.
- the portion indicated by the reference numeral 2 indicates that the words in the reference sentence are aligned with the words in the source sentence, that is, the words covering the source sentence are expressed. According to statistics, the reference sentence in FIG. 7 covers the source sentence. The first number of words covered is 7.
- the ratio of the difference between the first coverage quantity and the second coverage quantity to the second coverage quantity may be calculated by using the following formula as the coverage CDR of the target sentence (y'
- C ref is the number of words in the source statement covered by the reference statement
- the current alignment information accuracy rate is not completely correct (such as aligning to a source word, several consecutive words centered on the source word are covered), so it can be regarded as the word coverage of the target sentence and The word coverage of the reference statement is close to a statement with high loyalty.
- the BLUS (billingual evaluation understudy) value may be used to characterize the loyalty of the target sentence.
- the BLEU value of the target statement may be determined by:
- the ratio of the number of the same words calculated to the number of words in the target sentence is taken as the BLEU value, that is, The BLEU value is used as the standard for the accuracy of translation of the source sentence. The larger the value, the higher the accuracy of the translation.
- Step 103 The target sentence and the reference sentence are used as input of the discriminator model, and the loyalty is used as the output of the discriminator model, and the discriminator model is trained to calculate the performance of the similarity between the target sentence and the reference sentence.
- the loyalty of the target statement is the coverage rate of the target statement.
- the target sentence and the reference sentence are taken as input samples, the coverage rate is output samples, the training sample set is constructed, the discriminator model is trained, and the discriminator model is trained. Determine the performance of the similarity between the target statement and the reference statement, and establish the association between the target statement and the reference statement.
- Step 104 Output a similarity between the target statement and the reference statement using the discriminator model.
- the discriminator model may be composed of three neural network models, and the discriminator model may calculate the similarity between the target sentence and the reference sentence by:
- first neural network model in the discriminator model to encode the target sentence to obtain a first coding vector
- second neural network model in the discriminator model to encode the reference sentence to obtain a second coding vector
- the vector and the second coding vector are input to the third neural network model in the discriminator model to perform similarity calculation, and the similarity between the target sentence and the reference sentence is obtained.
- the first neural network model and the second neural network model in the discriminator model may both be convolutional neural network models, and in other embodiments, the first neural network model and the first in the discriminator model
- the two neural network models can also be recurrent neural network models; in practical applications, the recurrent neural network model is used as the first neural network model and the second neural network model for the discriminator model training, compared with the convolutional neural network model.
- the first neural network model and the second neural network model for the training of the discriminator model it is equivalent to transforming the classification problem into a regression problem, and improving the training speed of the translation model (that is, the number of words processed per unit time) .
- the process of outputting the similarity between the target sentence and the reference sentence using the discriminator model includes:
- the target sentence and the reference sentence are respectively input into two recursive neural network models for encoding, and the first coding vector and the second coding vector corresponding to the target sentence and the reference sentence are output, and then the first coding vector and the second coding vector are input into the feedforward.
- the neural network model calculates the similarity between the two.
- Step 105 The source sentence is used as the input of the translation model, the target sentence is used as the output of the translation model, and the similarity is used as the weight coefficient, and the training translation model outputs the performance of the corresponding target sentence according to the input source statement.
- the translation model is trained using the similarity output of the discriminator model.
- the purpose of the training is to make the discriminator model think that the target sentence output by the translation model is similar to the reference sentence.
- the degree is very high, such as 1; and the discriminator training is based on the target sentence translated by the translation model, thus forming the linkage between the discriminator model and the translation model.
- the training of the translation model can be implemented as follows:
- the updated value of the dimension corresponding to the selected training sample in the model parameter; the updated value corresponding to each selected training sample is based on the similarity fusion of the corresponding sample; the translation model parameter based on the fusion corresponds to the updated value of the training sample set, and the translation model is updated. parameter.
- the translation model parameters may be updated by superimposing the translation model parameters corresponding to the updated values of the training sample set and the pre-update translation model parameters.
- the training of the discriminator model is based on the target sentence output by the translation model
- the training of the translation model is based on the similarity between the target sentence and the reference sentence output by the discriminator model, and the linkage between the discriminator model and the translation model is realized.
- the accuracy of the translation model obtained by the training is high, and the phenomenon of skipping and repeating translation is better avoided.
- FIG. 8 is a schematic structural diagram of a training device based on a translation model according to an embodiment of the present application, including:
- the first obtaining unit 11 is configured to input a source statement to the translation model, and obtain a target statement output by the translation model;
- a determining unit 12 configured to determine a loyalty of the target statement to the source statement
- a first training unit 13 configured to use the target sentence and the reference sentence as input of the discriminator model, and use the fidelity as an output of the discriminator model to train the discriminator model to calculate the target sentence and the The performance of the similarity of the reference statement;
- a first output unit 14 configured to output a similarity between the target sentence and the reference sentence by using the discriminator model
- a second training unit 15 for training the translation model according to the source sentence as an input of the translation model, using the target sentence as an output of the translation model, and using the similarity as a weight coefficient
- the input source statement outputs the performance of the corresponding target statement.
- the determining unit 12 is further configured to: in response to the loyalty being a coverage rate, acquiring a first coverage quantity of a word of the target sentence that covers the source sentence,
- a ratio of the difference to the second number of covers is calculated as the coverage.
- the determining unit 12 is further configured to: in a process of obtaining a vector sequence of the target sentence output by the translation model, corresponding to a vector of each word in the target sentence The vectors of words are aligned to form a first alignment matrix;
- a first coverage quantity of words in the target sentence in the target sentence is statistically obtained from the first alignment matrix.
- the determining unit 12 is further configured to respectively acquire a vector sequence corresponding to the reference sentence and the source statement;
- the second alignment matrix is statistically obtained to obtain a second coverage number of words in the reference sentence in the reference sentence.
- the determining unit 12 is further configured to: in response to the loyalty being a bilingual evaluation of a substitute value, matching a word in the target sentence with a word in the reference sentence to obtain the target The number of identical words in the statement and the reference statement;
- the ratio of the calculated number to the number of words in the target sentence is used as the bilingual evaluation substitute value.
- the first output unit 14 is further configured to encode the target statement by using a first neural network model in the discriminator model to obtain a first encoding vector;
- the first coding vector and the second coding vector are input to a third neural network model in the discriminator model to perform similarity calculation, and the similarity between the target sentence and the reference sentence is obtained.
- the second training unit 15 is further configured to initialize an input layer, an intermediate layer, and an output layer included in the translation model
- the training sample set includes a source statement to be translated and a target statement corresponding to a translation result of the source sentence;
- the translation model parameters obtained based on the fusion correspond to updated values of the training sample set, and the translation model parameters are updated.
- the first obtaining unit 11 is further configured to calculate a semantic vector corresponding to the source statement by using an encoder model of the translation model;
- the words corresponding to the respective words of the source sentence are connected to form the target sentence.
- the first obtaining unit 11 is further configured to generate a vector sequence corresponding to the source statement by using an encoder model of the translation model;
- the word vector in the vector sequence of the source sentence is weighted and summed with the assigned attention probability to obtain a semantic vector corresponding to the source sentence.
- the translation model based training device of the embodiment of the present application may be implemented in the computer device shown in FIG. 9, which may be a terminal or a server. It can be understood that each of the above-mentioned modules included in the translation model-based training device may be implemented in whole or in part by software, hardware or a combination thereof.
- FIG. 9 is a block diagram of a computer device according to an embodiment of the present application, including: one or more processors 21, a memory 22, and at least one external communication interface 23; The one or more processors 21, the memory 22, and the external communication interface 23 are all connected by a bus 24;
- the memory 22 is configured to store computer readable instructions 221;
- One or more processors 21 for implementing the following steps by executing computer readable instructions stored in the memory:
- the processor 21 is further configured to implement the following steps by executing computer readable instructions stored in the memory:
- a ratio of the difference to the second number of covers is calculated as the coverage.
- the processor 21 is further configured to implement the following steps by executing computer readable instructions stored in the memory:
- a first coverage quantity of words in the target sentence in the target sentence is statistically obtained from the first alignment matrix.
- the processor 21 is further configured to implement the following steps by executing computer readable instructions stored in the memory:
- the second alignment matrix is statistically obtained to obtain a second coverage number of words in the reference sentence in the reference sentence.
- the processor 21 is further configured to implement the following steps by executing computer readable instructions stored in the memory:
- the ratio of the calculated number to the number of words in the target sentence is used as the bilingual evaluation substitute value.
- the processor 21 is further configured to implement the following steps by executing computer readable instructions stored in the memory:
- the first coding vector and the second coding vector are input to a third neural network model in the discriminator model to perform similarity calculation, and the similarity between the target sentence and the reference sentence is obtained.
- the processor 21 is further configured to implement the following steps by executing computer readable instructions stored in the memory:
- the training sample set includes a source statement to be translated and a target statement corresponding to a translation result of the source sentence;
- the translation model parameters obtained based on the fusion correspond to updated values of the training sample set, and update the translation model parameters.
- the processor 21 is further configured to implement the following steps by executing computer readable instructions stored in the memory:
- the words corresponding to the respective words of the source sentence are connected to form the target sentence.
- the processor 21 is further configured to implement the following steps by executing computer readable instructions stored in the memory:
- the word vector in the vector sequence of the source sentence is weighted and summed with the assigned attention probability to obtain a semantic vector corresponding to the source sentence.
- FIG. 10 is a schematic diagram of an application scenario of a translation model provided by an embodiment of the present invention.
- the terminal is provided with a translation program (APP, Application) and a translation model.
- APP translation program
- the device the terminal trains the translation model based on the training method of the above translation model, and then updates the translation APP by using the trained translation model.
- the translation model translates the source sentence and outputs the source sentence.
- the corresponding target statement is a schematic diagram of an application scenario of a translation model provided by an embodiment of the present invention.
- the terminal is provided with a translation program (APP, Application) and a translation model.
- APP Application
- FIG. 10 is a schematic diagram of an application scenario of a translation model provided by the embodiment of the present application, where a translation APP is set on the terminal, and a training device is provided on the server with a translation model.
- the server trains the translation model based on the training method of the above translation model, and then synchronizes the trained translation model to the terminal.
- the translation APP submits the source statement to the server, and then the server The translation model translates the source statement and returns the translated target statement to the terminal.
- the embodiment of the present application further provides a translation method based on a translation model, including:
- the translation model is obtained by using a source sentence as a translation model input, a target sentence as a translation model, and a similarity of the target sentence and the reference sentence as a weight coefficient in the confrontation network, the similarity being obtained. Calculated for using the discriminator model of the confrontation network;
- the semantic result vector is used as an input of the training model obtained by the training, and a translation result sentence corresponding to the statement to be translated is output.
- the discriminator model calculates the performance of the similarity by using the reference sentence and the target sentence as the input of the discriminator model in the confrontation network, and determining the loyalty of the target sentence to the source sentence.
- the output of the model is trained.
- the using the trained translation model to calculate a semantic vector corresponding to the statement to be translated includes:
- the word vector in the vector sequence of the source sentence is weighted and summed with the assigned attention probability to obtain a semantic vector corresponding to the source sentence.
- the semantic vector is used as the input of the training model, and the translation result statement corresponding to the statement to be translated is output, including:
- FIG. 12 is a schematic diagram of a composition structure of a translation model-based translation apparatus according to an embodiment of the present application.
- the device includes:
- a second obtaining unit 31 configured to acquire a translation model trained in the confrontation network
- the translation model is obtained by using a source sentence as a translation model input, a target sentence as a translation model, and a similarity of the target sentence and the reference sentence as a weight coefficient in the confrontation network, the similarity being obtained. Calculated for using the discriminator model of the confrontation network;
- the input unit 32 is configured to input a statement to be translated to the training model obtained by the training;
- the calculating unit 33 is configured to calculate, by using the trained translation model, a semantic vector corresponding to the statement to be translated;
- the second output unit 34 is configured to output, by using the semantic vector as an input of the training model obtained by the training, a translation result statement corresponding to the statement to be translated.
- the calculating unit 33 is further configured to generate a vector sequence corresponding to the source statement by using an encoder model in the trained translation model;
- the word vector in the vector sequence of the source sentence is weighted and summed with the assigned attention probability to obtain a semantic vector corresponding to the source sentence.
- the second output unit 34 is further configured to use, as the semantic vector, a state of a decoder model in the trained translation model, and a word that is output last time in the decoder model. Training the input of the translated model, and calculating the alignment probability of the candidate word and the word in the sentence to be translated;
- the translation model based translation device of the embodiment of the present application can be implemented in the computer device shown in FIG. It can be understood that each of the above-mentioned modules included in the translation model-based translation device may be implemented in whole or in part by software, hardware or a combination thereof.
- the embodiment of the present application further provides one or more readable storage media, where the one or more readable storage media stores computer readable instructions;
- the computer readable instructions when executed by one or more processors, implement the following steps:
- the computer readable instructions when used by the processor, implement the following steps:
- a ratio of the difference to the second number of covers is calculated as the coverage.
- the computer readable instructions when used by the processor, implement the following steps:
- a first coverage quantity of words in the target sentence in the target sentence is statistically obtained from the first alignment matrix.
- the computer readable instructions when used by the processor, implement the following steps:
- the second alignment matrix is statistically obtained to obtain a second coverage number of words in the reference sentence in the reference sentence.
- the computer readable instructions when used by the processor, implement the following steps:
- the ratio of the calculated quantity to the number of words in the target sentence is used as the bilingual evaluation substitute value.
- the computer readable instructions when used by the processor, implement the following steps:
- the first coding vector and the second coding vector are input to a third neural network model in the discriminator model to perform similarity calculation, and the similarity between the target sentence and the reference sentence is obtained.
- the computer readable instructions when used by the processor, implement the following steps:
- the training sample set includes a source statement to be translated and a target statement corresponding to a translation result of the source sentence;
- the translation model parameters obtained based on the fusion correspond to updated values of the training sample set, and update the translation model parameters.
- the computer readable instructions when used by the processor, implement the following steps:
- the words corresponding to the respective words of the source sentence are connected to form the target sentence.
- the computer readable instructions when used by the processor, implement the following steps:
- the word vector in the vector sequence of the source sentence is weighted and summed with the assigned attention probability to obtain a semantic vector corresponding to the source sentence.
- the embodiment of the present application further provides a computer device, where the computer device includes:
- a memory for storing computer readable instructions
- the translation model-based translation method described above is implemented when one or more processors are used to execute computer readable instructions stored in the memory.
- the embodiment of the present application further provides one or more storage media storing computer readable instructions, which are implemented by one or more processors to implement the above translation model based translation method.
- the above description relates to the training device based on the translation model, which is similar to the above description of the training method, the description of the beneficial effects of the same method, no description, the description of the translation device based on the translation model, and the above translation method
- the description is similar, and the beneficial effects of the method are described without further elaboration.
- the translation model-based training device, the translation model-based translation device, the computer device, and the storage medium embodiment described in the present application refer to the description of the method embodiment of the present application.
- the storage medium includes: a removable storage device, a random access memory (RAM), a read-only memory (ROM), a magnetic disk, or an optical disk, and the like, which can store program codes.
- the above-described integrated unit of the present application may be stored in a computer readable storage medium if it is implemented in the form of a software function module and sold or used as a stand-alone product.
- the technical solution of the embodiments of the present application may be embodied in the form of a software product in essence or in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for making
- a computer device (which may be a personal computer, server, or network device, etc.) performs all or part of the methods described in various embodiments of the present application.
- the foregoing storage medium includes various media that can store program codes, such as a mobile storage device, a RAM, a ROM, a magnetic disk, or an optical disk.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Machine Translation (AREA)
Abstract
一种基于翻译模型的训练方法,包括:向翻译模型输入源语句,获得所述翻译模型输出的目标语句;确定所述目标语句对于所述源语句的忠实度;以所述目标语句和参考语句作为判别器模型的输入、以所述忠实度作为所述判别器模型的输出,训练所述判别器模型计算所述目标语句与所述参考语句的相似度的性能;使用所述判别器模型输出所述目标语句与所述参考语句的相似度;以所述源语句作为所述翻译模型的输入、以所述目标语句作为所述翻译模型的输出、并以所述相似度作为权重系数,训练所述翻译模型根据输入的源语句输出对应的目标语句的性能。
Description
本申请要求于2017年12月15日提交中国专利局,申请号为2017113545630,申请名称为“基于翻译模型的训练方法、翻译方法、装置及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及机器翻译(MT,Machine Translation)技术,尤其涉及一种基于翻译模型的训练方法、翻译方法、计算机设备及存储介质。
随着机器翻译的发展,神经网络机器翻译(NMT,Neural Machine Translation)作为新一代的翻译技术得到普通应用。神经网络机器翻译***基于编码器-解码器框架搭建,然而,在神经网络机器翻译***的翻译过程中,解码器兼具多重任务,如记录当前翻译的内容、以及需要翻译的内容,记录翻译的流畅度的相关信息等。由于解码器任务的复杂性导致其无法兼顾所有任务,致使目前的神经网络机器翻译***翻译的准确度不高,漏翻、重复翻译现象严重。
发明内容
根据本申请提供的各种实施例,提供了一种基于翻译模型的训练方法、翻译方法、计算机设备及存储介质。
一种基于翻译模型的训练方法,包括:
计算机设备向翻译模型输入源语句,获得所述翻译模型输出的目标语句;
所述计算机设备确定所述目标语句对于所述源语句的忠实度;
所述计算机设备以所述目标语句和参考语句作为判别器模型的输入、以所述忠实度作为所述判别器模型的输出,训练所述判别器模型计算所述目标语句与所述参考语句的相似度的性能;
所述计算机设备使用所述判别器模型输出所述目标语句与所述参考语句的相似度;及
所述计算机设备以所述源语句作为所述翻译模型的输入、以所述目标语句作为所述翻译模型的输出、并以所述相似度作为权重系数,训练所述翻译模型根据输入的源语句输出对应的目标语句的性能。
一种基于翻译模型的翻译方法,包括:
计算机设备获取在对抗网络中训练得到的翻译模型;
其中,所述翻译模型为在对抗网络中利用以源语句作为翻译模型输入、以目标语句作为翻译模型输出、并以所述目标语句与参考语句的相似度作为权重系数训练得到,所述相似度为利用所述对抗网络的判别器模型计算得到;
所述计算机设备输入待翻译语句至所述训练得到的翻译模型;
所述计算机设备使用所述训练得到的翻译模型计算对应所述待翻译语句的语义向量;及
所述计算机设备以所述语义向量作为所述训练得到的翻译模型的输入,输出对应所述待翻译语句的翻译结果语句。
一种计算机设备,包括:
存储器,用于存储计算机可读指令;
一个或多个处理器,用于执行所述存储器中存储的计算机可读指令时,实现如上述的基于翻译模型的训练方法。
一个或多个存储介质,存储有计算机可读指令,所述计算机可读指令被一个或多个处理器执行时,实现上述的基于翻译模型的训练方法。
一种计算机设备,包括:
存储器,用于存储计算机可读指令;
一个或多个处理器,用于执行所述存储器中存储的计算机可读指令时,实现上述的基于翻译模型的翻译方法。
一个或多个存储介质,存储有计算机可读指令,所述计算机可读指令一个或多个被处理器执行时,实现上述的基于翻译模型的翻译方法。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。基于本申请的说明书、附图以及权利要求书,本申请的其它特征、目的和优点将变 得更加明显。
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的引入注意力模型的编码器模型-解码器模型框架示意图;
图2为本申请实施例提供的翻译模型的框图;
图3为本申请实施例提供的生成对抗网络模型中判别器模型的工作原理示意图;
图4为本申请实施例提供的基于翻译模型的训练方法的流程示意图;
图5为本申请实施例提供的基于翻译模型的训练***架构的示意图;
图6为本申请实施例提供的基于翻译模型的翻译流程示意图;
图7为本申请实施例提供的语句中的词语对齐实例示意图;
图8为本申请一个实施例提供的基于翻译模型的训练装置的框图;
图9为本申请另一个实施例提供的基于翻译模型的训练装置的框图;
图10为本申请一个实施例提供的翻译模型的应用场景示意图;
图11为本申请另一个实施例提供的翻译模型的应用场景示意图;及
图12为本申请实施例提供的基于翻译模型的翻译装置的框图。
以下结合附图及实施例,对本申请进行进一步详细说明。应当理解,此处所提供的实施例仅仅用以解释本申请,并不用于限定本申请。另外,以下所提供的实施例是用于实施本申请的部分实施例,而非提供实施本申请的全部实施例,在不冲突的情况下,本申请实施例记载的技术方案可以任意组合的方式实施。
需要说明的是,在本申请实施例中,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的方法或者装置不仅包括所明确记载的要素,而且还包括没有明确列出的其他要素,或者是还包括为实施方法或者装置所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括该要素的方法或者装置中还存在另外的相关要素(例如方法中的步骤或者装置中的单元,例如的单元可以是部分电路、部分处理器、部分程序或软件等等)。
例如,本申请实施例提供的基于翻译模型的训练方法及基于翻译模型的翻译方法包含了一系列的步骤,但是本申请实施例提供的基于翻译模型的训练方法及基于翻译模型的翻译方法不限于所记载的步骤,同样地,本申请实施例提供的翻译模型的训练装置包括了一系列单元,但是本申请实施例提供的装置不限于包括所明确记载的单元,还可以包括为获取相关信息、或基于信息进行处理时所需要设置的单元。
对本申请进行进一步详细说明之前,对本申请实施例中涉及的名词和术语进行说明,本申请实施例中涉及的名词和术语适用于如下的解释。
1)源语句,在进行语言转换之前输入翻译模型的对应某种自然语言的语句。
2)目标语句,翻译模型输出的对源语句进行语言转换之后的对应某种自然语言的语句。
3)参考语句,预先设置的对源语句进行语言转换之后的对应某种自然语言的参考标准。
4)忠实度,表征目标语句的内容与源语句内容接近程度的介于0和1之间的参数,作为评价对源语句翻译准确度高低的标准,值越大表明目标语句的内容与源语句内容接近程度越高,即翻译的准确度越高。
5)覆盖率,表征目标语句的内容覆盖源语句内容的比例的介于0和1之间的参数。
6)翻译,将一种自然语言的语句转换成另一种自然语言的语句。
对本申请实施例进行进一步详细说明之前,对本申请实施例中涉及的由编码器-解码器组成的翻译模型进行说明。
本申请实施例中的翻译模型附加了注意力模型,即注意力模型附着在编码器-解码器框架下,参见图1,图1为引入注意力模型的编码器模型-解码器模型框架示意图,将源语句输入翻译模型,输出对应源语句的翻译结果的目标语句;其中,源语句与目标语句为不同种自然语言的语句。
在实际应用中,可通过如下方式获得翻译模型输出的目标语句:使用翻译模型的编码器模型计算对应源语句的语义向量;基于输入翻译模型中解码器模型的语义向量、输入语义向量时解码器模型的状态、解码器模型上一次输出的词语,计算候选词语与源语句中词语的对齐概率;选取计算得到的对齐概率最高的候选词语作为输出的词语;连接针对源语句的各词语对应输出的词语,形成目标语句。
其中,可通过如下方式计算对应源语句的语义向量:使用翻译模型的编码器模型生成对应源语句的向量序列;使用注意力模型为源语句的向量序列中的词向量分配注意力概率;将源语句的向量序列中的词向量与分配的注意力概率进行加权求和,得到对应源语句的语义向量。
图2为本申请实施例提供的基于翻译模型的架构示意图,参见图2,以源语句为“我们是好朋友”为例,对本申请实施例的翻译模型输出目标语句的过程进行说明。源语句输入翻译模型中的编码器模型,编码器模型对输入的源语句进行编码,得到对应源语句的向量序列,如将编码得到的向量序列表示为h
j,h
j为源语句中第j个词语对应的语义编码,如图2中标号1部分(即矩形框中多个圆构成的序列,其中,“<eos>”是句子终结符)所示。然后使用注意力模型读入源语句的向量序列及翻译模型中的解码器模型前一时刻状态(图2中s
t-1),为源语句的向量序列中的各个词向量分配注意力概率,如将分配的注意力概率表示为a
ij,a
ij表示在翻译模型的解码器模型输出第i个词语时,源语句中第j个词语的注意力概率,如图2中示出的“0.1、0.1、0.6、0.1、0.1”,a
ij的值越高,表示第i个输出在第j个输入上分配的注意力越多,在生成第i个输出的时候受第j个输入的影响也就越大。然后将源语句的向量 序列中的词向量与分配的所述注意力概率进行加权求和,得到对应所述源语句的语义向量,将语义向量表示为c
i,
其中,L
x表示源语句的词语长度,如对应源语句“我们是好朋友”来说,假设c
i对应的下标i就是上面的“好”,L
x即为4,h
1=f(“我们”),h
2=f(“是”),h
3=f(“好”),h
4=f(“朋友”),对应的注意力概率分别为0.1、0.1、0.6、0.1。然后将得到的对应所述源语句的语义向量输入解码器模型,解码器模型读入语义向量c
i(表示当前时刻待被翻译的词语),前一时刻解码器模型状态s
t-1,解码器模型上一次输出的词语y
t-1,输出当前时刻的解码器模型状态s
t。然后解码器模型根据输入的语义向量c
i、当前解码器模型状态s
t、解码器模型上一次输出的词语y
t-1,计算候选词语与源语句中词语的对齐概率,如当翻译词语“好”时,得到候选词“we”、“good”、“friends”对应的对齐概率分别为0.1、0.6、0.1,选取对齐概率最高的“good”作为解码器模型输出的词语,顺序连接针对源语句的各词语对应输出的词语,形成目标语句。
然而,在采用上述翻译模型进行语句翻译的过程中,解码器模型以解码器模型状态、上一次输出的词语等作为输入,输出对应源语句中词语的词语,因此,自身需要记录很多信息,如:需要翻译的内容、当前翻译的内容、整体翻译的流畅度等,也即解码器模型兼具多重任务,这也就导致了上述翻译模型在实际应用中会出现较严重的漏翻及重复翻译现象。
为解决上述问题,作为一个可选的实施例,可以通过生成对抗网络(GAN,Generative Adversarial Network)模型来训练翻译模型,将翻译模型作为生成对抗网络模型中的生成器模型,将卷积神经网络模型作为生成对抗网络模型中的判别器模型,构建由生成器模型-判别器模型组成的生成对抗网络模型。其中,判别器模型是一个二分类器,可以依靠概率判定输入的语句是否为人类所翻译的,其作用是尽可能的将真实翻译的语句和由生成器模型(翻译模型)翻译的语句区分开,参见图3,图3为本申请实施例提供的生成对抗网 络模型中判别器模型的工作原理示意图,将真实样本数据输入至判别器模型,判别器模型经过计算后尽量使得概率接近1(真),而将服从某一分布的噪音输入到生成器模型中,生成器模型经过计算产生生成样本,然后将生成样本输入到判别器模型中,判别器模型经过计算后尽量使得输出概率为0(假);而正好相反,生成器(翻译模型)的目的是尽可能的去模拟人类所翻译的语句(真实译文),从而使得判别器模型无法判定输入的语句是由人类翻译还是生成器(翻译模型)所翻译。在应用生成对抗网络模型过程中,生成器捕捉样本数据的分布,用服从某一分布(均匀分布,高斯分布等)的噪声生成一个类似真实训练数据的样本,即生成一个类似参考语句的目标语句;判别器模型将源语句和目标语句中的每个词对应的词向量两两组合构成一个三维“图像”,将这个三维“图像”输入到判别器模型中,输出目标语句是真实译文的概率;在本申请实施例中,概率大小包括两种,分别为0和1,当概率为0时表征目标语句为假,即非人类(机器)翻译,而当概率为1时表征目标语句为真,即由人类翻译。
在对判别器模型训练过程中,对于源语句有两个对应的语句,分别是人类翻译的参考语句和机器翻译的目标语句,将对应源语句的目标语句作为输入样本、对应的概率为0作为输出样本,构建一个训练样本实例,将对应源语句的参考语句作为输入样本、对应的概率为1作为输出样本,构建一个训练样本实例,对判别器模型进行训练,使其具备识别输入的语句是否为人类翻译的性能。在对生成器进行训练的过程中,由于生成器的训练实例为离散型(源语句对应的目标语句的概率为0或1),无法使用后向传播算法进行训练,在一些实施例中,可以使用策略梯度算法进行训练,公式如下:
其中,θ
G为生成器的参数,生成器的
为梯度,参数x为生成器的输入(源语句),y’为生成器的输出(目标语句),D(x,y')为y’是x真实翻译的概率,
为生成器的标准梯度,即通过最大似然估计算法得到的梯度。
生成器参数的更新通过如下公式:
可知采用生成对抗网络模型对翻译模型进行训练,虽然可以部分提高翻译模型翻译的准确度,然而,由于在对翻译模型训练过程中判别器模型的目标函数不够准确,将所有机器翻译的语句对应的概率标识为0,将所有人类翻译的语句对应的概率标识为1,然而,机器翻译的语句的质量是参差不齐的,并非所有机器翻译的语句均不可取,将机器翻译较佳的语句对应的概率标识为0势必导致模型训练过程中的参数更新不准确。
为了较好的避免漏翻、重复翻译现象,最大限度的提高翻译模型翻译的准确度,提出本申请实施例的翻译模型的训练方案,本申请实施例的翻译模型的训练方案基于生成对抗网络模型,将翻译模型作为生成对抗网络模型中的生成器,加入判别器模型,实现对翻译模型的训练,接下来结合具体实施例对本申请实施例的翻译模型的训练方案详细进行说明。
图4为本申请实施例提供的基于翻译模型的训练方法的一个可选的流程示意图,图5为本申请实施例提供的基于翻译模型的训练***架构的一个可选的示意图,结合图4、图5所示,本申请实施例的基于翻译模型的训练方法包括:
步骤101:向翻译模型输入源语句,获得翻译模型输出的目标语句。
参见图6,图6为本申请实施例提供的翻译模型的翻译流程示意图,在一实施例中,翻译模型的翻译过程(将一种自然语言的源语句转换为另一种自然语言的目标语句)主要包括以下三个步骤:
步骤1:使用翻译模型的编码器模型计算对应源语句的语义向量,也即生成源端向量表示c
i。
本申请实施例中提到的源端对应编码器模型端,目标端对应解码器模型端,源语句输入编码器模型,编码器模型对源语句进行编码,得到至少一个词向量组成的向量序列,其中,每个词向量对应源语句中一个词语,例如,源语句为“我来自中国”,经编码器模型编码后得到由“我”、“来自”、“中国”对应的词向量h
1、h
2、h
3组成的源端向量序列,然后注意力模型读入编码器模型的向量序列和解码器模型前一时刻状态,为源语句中每个词语分配一个 值介于0和1之间的注意力概率,如“我”、“来自”、“中国”对应的注意力概率分别为1、0、0,值越高,表示这个词语和待生成的目标词越相关,将注意力概率和向量序列加权求和生成当前时刻的源端向量表示c
i,即得到对应源语句的语义向量c
i,
其中,L
x表示源语句的词语长度,a
ij表示在翻译模型的解码器模型输出第i个词语时,源语句中第j个词语的注意力概率,h
j为源语句中第j个词语对应的语义编码。
步骤2:更新解码器模型状态。
解码器模型读入源端向量表示(即当前时刻待被翻译的源语句的内容),前一时刻解码器模型状态s
t-1,解码器模型输出的前一个词语y
t-1,输出当前时刻的解码器模型状态s
t。
步骤3:解码器模型生成当前时刻的目标端词语。
解码器模型根据当前时刻的解码器状态s
t,源端向量表示c
t,和前一个生成词y
t-1,计算候选词语与源语句中词语的对齐概率,选取对齐概率(相似度)最高的一个词语作为最终的输出,即作为目标词语。
解码器模型得到对应源语句中各词语的目标词语后,连接针对源语句的各词语对应输出的词语,形成目标语句。
步骤102:确定目标语句对于源语句的忠实度。
在一些实施例中,可以用目标语句的覆盖率表征目标语句的忠实度,也即采用覆盖率作为对源语句翻译准确度高低的标准,值越大表明翻译的准确度越高。参见图5,图5是以忠实度为覆盖率为例进行翻译模型训练的***架构,源语句和参考语句输入翻译模型,经翻译模型翻译得到目标语句与源语句的第一对齐矩阵,并通过强制解码得到参考语句与源语句的第二对齐矩阵,然后将第一对齐矩阵和第二对齐矩阵输入覆盖率计算模块,覆盖率计算模块计算得到目标语句的覆盖率(即覆盖率分散),以输入判别器模型。
在一实施例中,可通过如下方式实现确定目标语句的覆盖率:
获取目标语句的词语覆盖源语句的词语的第一覆盖数量,以及,获取参考语句的词语覆盖源语句的词语的第二覆盖数量;计算第一覆盖数量与第二 覆盖数量的差值;计算差值与所述第二覆盖数量的比值作为目标语句的覆盖率。
在一实施例中,可通过如下方式获取目标语句覆盖源语句的词语的第一覆盖数量:
在获得翻译模型输出的目标语句的向量序列的过程中,将目标语句中每个词语的向量与源语句中对应词语的向量进行对齐,形成第一对齐矩阵;从第一对齐矩阵中统计得到目标语句中词语覆盖源语句中词语的第一覆盖数量。在实际实施时,由于翻译模型将源语句转换为目标语句基于注意力模型,如图1所示,在在获得翻译模型输出的目标语句的过程中,可通过注意力模型实现目标语句与源语句的对齐,得到对齐矩阵,然后统计得到目标语句覆盖源语句的词语的数量。参见图7,图7为本申请实施例提供的语句中的词语对齐实例示意图,在图7中,“二十六岁男司机被困车内,由到场的消防员救出。”为源语句,标号1所示部分表示目标语句中的词语与源语句中词语实现对齐,即表示覆盖到源语句的词语,通过统计可知,图7中目标语句覆盖源语句的词语的第一覆盖数量为4。
在一实施例中,可通过如下方式获取参考语句的词语覆盖源语句的词语的第二覆盖数量:
分别获取对应参考语句及源语句的向量序列;对齐源语句的向量序列与参考语句的向量序列中相同词语对应的向量,得到第二对齐矩阵;从第二对齐矩阵统计得到参考语句中词语覆盖源语句中词语的第二覆盖数量。在实际实施时,由于参考语句可视为对源语句的标准译文,因此可通过强制解码的方式获得参考语句的对齐信息。参见图7,在图7中,标号2所示部分表示参考语句中的词语与源语句中词语实现对齐,即表示覆盖到源语句的词语,通过统计可知,图7中参考语句覆盖源语句的词语的第一覆盖数量为7。
当得到了第一覆盖数量及第二覆盖数量之后,可通过如下公式计算第一覆盖数量及第二覆盖数量的差值与第二覆盖数量的比值作为目标语句的覆盖率CDR(y'|y,x),
其中,C
ref为参考语句覆盖到的源语句的词语的数量,C
gen为目标语句覆盖到的源语句的词语的数量,以图7中实例进行计算,可得目标语句对于源语句的覆盖率为:(7-4)/7=0.43。如此,建立了目标语句与参考语句之间的联系,因为在实际应用中有些源端词(如虚词)没有实际意义,所以无须被覆盖。通过和参考语句的对比,可以排除这些词的影响。且由于当前对齐信息准确率不是完全正确的(如对齐到某个源端词可能以该源端词为中心的几个连续词都被覆盖了),所以可视为目标语句的词语覆盖情况和参考语句的词语覆盖情况接近便是忠实度高的语句。
在一些实施例中,可以用目标语句的双语评估替代(BLEU,bilingual evaluation understudy)值表征目标语句的忠实度,在一实施例中,可通过如下方式实现确定目标语句的BLEU值:
将目标语句中的词语与参考语句中的词语进行匹配,得到目标语句与参考语句中相同词语的数量;计算得到的相同词语的数量与目标语句中词语的数量的比值作为BLEU值,也即采用BLEU值作为对源语句翻译准确度高低的标准,值越大表明翻译的准确度越高。
步骤103:以目标语句和参考语句作为判别器模型的输入、以忠实度作为判别器模型的输出,训练判别器模型计算目标语句与参考语句的相似度的性能。
以目标语句的忠实度是目标语句的覆盖率为例进行说明,以目标语句和参考语句作为输入样本,以覆盖率为输出样本,构建训练样本集合,对判别器模型进行训练,训练判别器模型判别目标语句与参考语句的相似度的性能,建立目标语句和参考语句之间的关联。
步骤104:使用判别器模型输出目标语句与参考语句的相似度。
在一实施例中,判别器模型可由三个神经网络模型组成,判别器模型可通过如下方式计算目标语句与参考语句的相似度:
使用判别器模型中的第一神经网络模型对目标语句进行编码,得到第一编码向量;使用判别器模型中的第二神经网络模型对参考语句进行编码,得 到第二编码向量;将第一编码向量及所述第二编码向量输入到判别器模型中的第三神经网络模型进行相似度计算,得到目标语句与参考语句的相似度。
在一些实施例中,判别器模型中的第一神经网络模型和第二神经网络模型可以均为卷积神经网络模型,在另一些实施例中,判别器模型中的第一神经网络模型和第二神经网络模型亦可以均为递归神经网络模型;在实际应用中,采用递归神经网络模型作为第一神经网络模型和第二神经网络模型进行判别器模型的训练,相对于采用卷积神经网络模型作为第一神经网络模型和第二神经网络模型进行判别器模型的训练来说,相当于将分类问题转化为回归问题,提高了对翻译模型的训练速度(即单位时间内处理的词语的数量)。
以第一神经网络模型和第二神经网络模型是递归神经网络模型为例,使用判别器模型输出目标语句与参考语句的相似度的过程包括:
将目标语句和参考语句分别输入两个递归神经网络模型中进行编码,输出对应目标语句和参考语句的第一编码向量及第二编码向量,然后将第一编码向量及第二编码向量输入前馈神经网络模型,计算二者的相似度。
步骤105:以源语句作为翻译模型的输入、以目标语句作为翻译模型的输出、并以相似度作为权重系数,训练翻译模型根据输入的源语句输出对应的目标语句的性能。
这里,在实际实施时,对判别器模型完成一次训练后,采用判别器模型输出的相似度对翻译模型进行训练,训练的目的是让判别器模型认为翻译模型输出的目标语句与参考语句的相似度很高,如为1;而判别器的训练基于翻译模型翻译得到的目标语句,如此形成判别器模型与翻译模型的联动。
在一实施例中,可通过如下方式实现对翻译模型的训练:
初始化翻译模型包括的输入层、中间层和输出层;构建训练样本集合;训练样本集合包括待翻译的源语句及对应源语句的翻译结果的目标语句;初始化基于翻译模型输入、翻译模型输出、以及翻译模型参数构建的损失函数;以所选取训练样本的源语句作为输入、所选取训练样本的目标语句作为输出、采用最大似然估计算法,求解损失函数相对于所选取训练样本取得最小值时翻译模型参数中与所选取训练样本对应维度的更新值;将每个选取训练样本对应的更新值基于相应样本的相似度融合;基于融合得到的翻译模型参数对 应训练样本集合的更新值,更新翻译模型参数。在实际实施时,可通过叠加融合得到的翻译模型参数对应训练样本集合的更新值与更新前的翻译模型参数的方式,更新翻译模型参数。
应用本申请上述实施例,对判别器模型的训练基于翻译模型输出的目标语句,翻译模型的训练基于判别器模型输出的目标语句与参考语句的相似度,实现了判别器模型与翻译模型的联动,使得训练得到的翻译模型的准确度高,较好的避免漏翻、重复翻译现象。
本申请实施例还提供了一种基于翻译模型的训练装置100,参见图8,图8为本申请实施例提供的基于翻译模型的训练装置的组成结构示意图,包括:
第一获取单元11,用于向翻译模型输入源语句,获得所述翻译模型输出的目标语句;
确定单元12,用于确定所述目标语句对于所述源语句的忠实度;
第一训练单元13,用于以所述目标语句和参考语句作为判别器模型的输入、以所述忠实度作为所述判别器模型的输出,训练所述判别器模型计算所述目标语句与所述参考语句的相似度的性能;
第一输出单元14,用于使用所述判别器模型输出所述目标语句与所述参考语句的相似度;
第二训练单元15,用于以所述源语句作为所述翻译模型的输入、以所述目标语句作为所述翻译模型的输出、并以所述相似度作为权重系数,训练所述翻译模型根据输入的源语句输出对应的目标语句的性能。
在一些实施例中,所述确定单元12,还用于响应于所述忠实度为覆盖率,获取所述目标语句的词语覆盖所述源语句的词语的第一覆盖数量,
以及,获取所述参考语句的词语覆盖所述源语句的词语的第二覆盖数量;
计算所述第一覆盖数量与所述第二覆盖数量的差值;
计算所述差值与所述第二覆盖数量的比值作为所述覆盖率。
在一些实施例中,所述确定单元12,还用于在获得所述翻译模型输出的目标语句的向量序列的过程中,将所述目标语句中每个词语的向量与所述源语句中对应词语的向量进行对齐,形成第一对齐矩阵;
从所述第一对齐矩阵中统计得到所述目标语句中词语覆盖所述源语句中 词语的第一覆盖数量。
在一些实施例中,所述确定单元12,还用于分别获取对应所述参考语句及所述源语句的向量序列;
对齐所述源语句的向量序列与所述参考语句的向量序列中相同词语对应的向量,得到第二对齐矩阵;
从所述第二对齐矩阵统计得到所述参考语句中词语覆盖所述源语句中词语的第二覆盖数量。
在一些实施例中,所述确定单元12,还用于响应于所述忠实度为双语评估替代值,将所述目标语句中的词语与所述参考语句中的词语进行匹配,得到所述目标语句与所述参考语句中相同词语的数量;
计算得到的所述数量与所述目标语句中词语的数量的比值作为所述双语评估替代值。
在一些实施例中,所述第一输出单元14,还用于使用所述判别器模型中的第一神经网络模型对所述目标语句进行编码,得到第一编码向量;
使用所述判别器模型中的第二神经网络模型对所述参考语句进行编码,得到第二编码向量;
将所述第一编码向量及所述第二编码向量输入到所述判别器模型中的第三神经网络模型进行相似度计算,得到所述目标语句与所述参考语句的相似度。
在一些实施例中,所述第二训练单元15,还用于初始化所述翻译模型包括的输入层、中间层和输出层;
构建训练样本集合;所述训练样本集合包括待翻译的源语句及对应源语句的翻译结果的目标语句;
初始化基于翻译模型输入、翻译模型输出、以及翻译模型参数构建的损失函数;
以所选取训练样本的源语句作为输入、所选取训练样本的目标语句作为输出、求解翻译模型参数中与所选取训练样本对应维度的更新值;
将每个选取训练样本对应的更新值基于相应样本的相似度融合;
基于融合得到的所述翻译模型参数对应所述训练样本集合的更新值,更 新所述翻译模型参数。
在一些实施例中,所述第一获取单元11,还用于使用翻译模型的编码器模型计算对应所述源语句的语义向量;
基于所述输入所述翻译模型中解码器模型的所述语义向量、输入所述语义向量时所述解码器模型的状态、所述解码器模型上一次输出的词语,计算候选词语与所述源语句中词语的对齐概率;
选取计算得到的对齐概率最高的候选词语作为输出的词语;
连接针对所述源语句的各词语对应输出的词语,形成所述目标语句。
在一些实施例中,所述第一获取单元11,还用于使用所述翻译模型的编码器模型生成对应所述源语句的向量序列;
使用注意力模型为所述源语句的向量序列中的词向量分配注意力概率;
将所述源语句的向量序列中的词向量与分配的所述注意力概率进行加权求和,得到对应所述源语句的语义向量。
基于上述对基于翻译模型的训练方法及装置的说明,可以轻易实施翻译模型的训练以及应用训练得到的翻译模型。本申请实施例的基于翻译模型的训练装置可实施于图9中所示的计算机设备,该计算机设备可以是终端或服务器。可以理解,该基于翻译模型的训练装置中包括的上述各个模块中,每个模块可全部或部分通过软件、硬件或其组合来实现。
本申请实施例还提供了一种计算机设备,参见图9,图9为本申请实施例提供的计算机设备的框图,包括:一个或多个处理器21、存储器22以及至少一个外部通信接口23;所述一个或多个处理器21、存储器22以及外部通信接口23均通过总线24连接;其中,
存储器22,用于存储计算机可读指令221;
一个或多个处理器21,用于通过执行所述存储器中存储的计算机可读指令时实现以下步骤:
向翻译模型输入源语句,获得所述翻译模型输出的目标语句;
确定所述目标语句对于所述源语句的忠实度;
以所述目标语句和参考语句作为判别器模型的输入、以所述忠实度作为所述判别器模型的输出,训练所述判别器模型计算所述目标语句与所述参考 语句的相似度的性能;
使用所述判别器模型输出所述目标语句与所述参考语句的相似度;
以所述源语句作为所述翻译模型的输入、以所述目标语句作为所述翻译模型的输出、并以所述相似度作为权重系数,训练所述翻译模型根据输入的源语句输出对应的目标语句的性能。
所述处理器21,还用于通过执行所述存储器中存储的计算机可读指令时实现以下步骤:
响应于所述忠实度为覆盖率,获取所述目标语句的词语覆盖所述源语句的词语的第一覆盖数量,
以及,获取所述参考语句的词语覆盖所述源语句的词语的第二覆盖数量;
计算所述第一覆盖数量与所述第二覆盖数量的差值;
计算所述差值与所述第二覆盖数量的比值作为所述覆盖率。
所述处理器21,还用于通过执行所述存储器中存储的计算机可读指令时实现以下步骤:
在获得所述翻译模型输出的目标语句的向量序列的过程中,将所述目标语句中每个词语的向量与所述源语句中对应词语的向量进行对齐,形成第一对齐矩阵;
从所述第一对齐矩阵中统计得到所述目标语句中词语覆盖所述源语句中词语的第一覆盖数量。
所述处理器21,还用于通过执行所述存储器中存储的计算机可读指令时实现以下步骤:
分别获取对应所述参考语句及所述源语句的向量序列;
对齐所述源语句的向量序列与所述参考语句的向量序列中相同词语对应的向量,得到第二对齐矩阵;
从所述第二对齐矩阵统计得到所述参考语句中词语覆盖所述源语句中词语的第二覆盖数量。
所述处理器21,还用于通过执行所述存储器中存储的计算机可读指令时实现以下步骤:
响应于所述忠实度为双语评估替代值,将所述目标语句中的词语与所述 参考语句中的词语进行匹配,得到所述目标语句与所述参考语句中相同词语的数量;
计算得到的所述数量与所述目标语句中词语的数量的比值作为所述双语评估替代值。
所述处理器21,还用于通过执行所述存储器中存储的计算机可读指令时实现以下步骤:
使用所述判别器模型中的第一神经网络模型对所述目标语句进行编码,得到第一编码向量;
使用所述判别器模型中的第二神经网络模型对所述参考语句进行编码,得到第二编码向量;
将所述第一编码向量及所述第二编码向量输入到所述判别器模型中的第三神经网络模型进行相似度计算,得到所述目标语句与所述参考语句的相似度。
所述处理器21,还用于通过执行所述存储器中存储的计算机可读指令时实现以下步骤:
初始化所述翻译模型包括的输入层、中间层和输出层;
构建训练样本集合;所述训练样本集合包括待翻译的源语句及对应源语句的翻译结果的目标语句;
初始化基于翻译模型输入、翻译模型输出、以及翻译模型参数构建的损失函数;
以所选取训练样本的源语句作为输入、所选取训练样本的目标语句作为输出、求解所述损失函数相对于所选取训练样本取得最小值时翻译模型参数中与所选取训练样本对应维度的更新值;
将每个选取训练样本对应的更新值基于相应样本的相似度融合;
基于融合得到的所述翻译模型参数对应所述训练样本集合的更新值,并更新所述翻译模型参数。
所述处理器21,还用于通过执行所述存储器中存储的计算机可读指令时实现以下步骤:
使用翻译模型的编码器模型计算对应所述源语句的语义向量;
基于所述输入所述翻译模型中解码器模型的所述语义向量、输入所述语义向量时所述解码器模型的状态、所述解码器模型上一次输出的词语,计算候选词语与所述源语句中词语的对齐概率;
选取计算得到的对齐概率最高的候选词语作为输出的词语;
连接针对所述源语句的各词语对应输出的词语,形成所述目标语句。
所述处理器21,还用于通过执行所述存储器中存储的计算机可读指令时实现以下步骤:
使用所述翻译模型的编码器模型生成对应所述源语句的向量序列;
使用注意力模型为所述源语句的向量序列中的词向量分配注意力概率;
将所述源语句的向量序列中的词向量与分配的所述注意力概率进行加权求和,得到对应所述源语句的语义向量。
以基于翻译模型的训练装置实施于终端为例,参见图10,图10为本申请实施例提供的一个翻译模型的应用场景示意图,终端上设置有翻译程序(APP,Application)及翻译模型的训练装置,终端基于上述翻译模型的训练方法训练得到翻译模型,然后采用训练得到的翻译模型更新自身的翻译APP,当用户基于更新后的翻译APP输入源语句时,翻译模型对源语句进行翻译,输出相应的目标语句。
以基于翻译模型的训练装置实施于服务器为例,参见图11,图10为本申请实施例提供的一个翻译模型的应用场景示意图,终端上设置有翻译APP,服务器上设置有翻译模型的训练装置,服务器基于上述翻译模型的训练方法训练得到翻译模型,然后将训练得到的翻译模型同步至终端,当用户基于更新后的翻译APP输入源语句时,翻译APP向服务器提交源语句,然后服务器上的翻译模型对源语句进行翻译,并将翻译得到的目标语句返回给终端。
基于上述对翻译模型的训练方法及对训练得到的翻译模型的应用场景的介绍,接下来对训练得到的翻译模型的翻译方法进行说明。相应的,本申请实施例还提供了一种基于翻译模型的翻译方法,包括:
获取在对抗网络中训练得到的翻译模型;
其中,所述翻译模型为在对抗网络中利用以源语句作为翻译模型输入、以目标语句作为翻译模型输出、并以所述目标语句与参考语句的相似度作为 权重系数训练得到,所述相似度为利用所述对抗网络的判别器模型计算得到;
输入待翻译语句至所述训练得到的翻译模型;
使用所述训练得到的翻译模型计算对应所述待翻译语句的语义向量;
以所述语义向量作为所述训练得到的翻译模型的输入,输出对应所述待翻译语句的翻译结果语句。
在一实施例中,所述判别器模型计算所述相似度的性能,通过在对抗网络中利用参考语句和目标语句作为判别器模型的输入、以所述目标语句对于源语句的忠实度作为判别器模型的输出训练得到。
在一实施例中,所述使用所述训练得到的翻译模型计算对应所述待翻译语句的语义向量,包括:
使用所述训练得到的翻译模型中编码器模型生成对应所述源语句的向量序列;
使用注意力模型为所述源语句的向量序列中的词向量分配注意力概率;
将所述源语句的向量序列中的词向量与分配的所述注意力概率进行加权求和,得到对应所述源语句的语义向量。
在一实施例中,以所述语义向量作为所述训练得到的翻译模型的输入,输出对应所述待翻译语句的翻译结果语句,包括:
以所述语义向量、所述训练得到的翻译模型中解码器模型的状态、所述解码器模型上一次输出的词语作为所述训练得到的翻译模型的输入,计算候选词语与所述待翻译语句中词语的对齐概率;
基于计算得到的对齐概率,选取满足概率条件的的候选词语作为输出的词语;
连接针对所述待翻译语句的各词语对应输出的词语,形成对应所述待翻译语句的翻译结果语句。
相应于上述基于翻译模型的翻译方法,本申请实施例还提供了一种基于翻译模型的翻译装置,图12为本申请实施例提供的基于翻译模型的翻译装置的组成结构示意图,参见图12,所述装置包括:
第二获取单元31,用于获取在对抗网络中训练得到的翻译模型;
其中,所述翻译模型为在对抗网络中利用以源语句作为翻译模型输入、 以目标语句作为翻译模型输出、并以所述目标语句与参考语句的相似度作为权重系数训练得到,所述相似度为利用所述对抗网络的判别器模型计算得到;
输入单元32,用于输入待翻译语句至所述训练得到的翻译模型;
计算单元33,用于使用所述训练得到的翻译模型计算对应所述待翻译语句的语义向量;
第二输出单元34,用于以所述语义向量作为所述训练得到的翻译模型的输入,输出对应所述待翻译语句的翻译结果语句。
在一实施例中,所述计算单元33,还用于使用所述训练得到的翻译模型中编码器模型生成对应所述源语句的向量序列;
使用注意力模型为所述源语句的向量序列中的词向量分配注意力概率;
将所述源语句的向量序列中的词向量与分配的所述注意力概率进行加权求和,得到对应所述源语句的语义向量。
在一实施例中,所述第二输出单元34,还用于以所述语义向量、所述训练得到的翻译模型中解码器模型的状态、所述解码器模型上一次输出的词语作为所述训练得到的翻译模型的输入,计算候选词语与所述待翻译语句中词语的对齐概率;
基于计算得到的对齐概率,选取满足概率条件的的候选词语作为输出的词语;
连接针对所述待翻译语句的各词语对应输出的词语,形成对应所述待翻译语句的翻译结果语句。
本申请实施例的基于翻译模型的翻译装置可实施于图9中所示的计算机设备。可以理解,该基于翻译模型的翻译装置中包括的上述各个模块中,每个模块可全部或部分通过软件、硬件或其组合来实现。
本申请实施例还提供了一一个或多个可读存储介质,所述一个或多个可读存储介质存储有计算机可读指令;
所述计算机可读指令,用于被一个或多个处理器执行时实现以下步骤:
向翻译模型输入源语句,获得所述翻译模型输出的目标语句;
确定所述目标语句对于所述源语句的忠实度;
以所述目标语句和参考语句作为判别器模型的输入、以所述忠实度作为 所述判别器模型的输出,训练所述判别器模型计算所述目标语句与所述参考语句的相似度的性能;
使用所述判别器模型输出所述目标语句与所述参考语句的相似度;
以所述源语句作为所述翻译模型的输入、以所述目标语句作为所述翻译模型的输出、并以所述相似度作为权重系数,训练所述翻译模型根据输入的源语句输出对应的目标语句的性能。
所述计算机可读指令,还用于被处理器执行时实现以下步骤:
响应于所述忠实度为覆盖率,获取所述目标语句的词语覆盖所述源语句的词语的第一覆盖数量,
以及,获取所述参考语句的词语覆盖所述源语句的词语的第二覆盖数量;
计算所述第一覆盖数量与所述第二覆盖数量的差值;
计算所述差值与所述第二覆盖数量的比值作为所述覆盖率。
所述计算机可读指令,还用于被处理器执行时实现以下步骤:
在获得所述翻译模型输出的目标语句的向量序列的过程中,将所述目标语句中每个词语的向量与所述源语句中对应词语的向量进行对齐,形成第一对齐矩阵;
从所述第一对齐矩阵中统计得到所述目标语句中词语覆盖所述源语句中词语的第一覆盖数量。
所述计算机可读指令,还用于被处理器执行时实现以下步骤:
分别获取对应所述参考语句及所述源语句的向量序列;
对齐所述源语句的向量序列与所述参考语句的向量序列中相同词语对应的向量,得到第二对齐矩阵;
从所述第二对齐矩阵统计得到所述参考语句中词语覆盖所述源语句中词语的第二覆盖数量。
所述计算机可读指令,还用于被处理器执行时实现以下步骤:
响应于所述忠实度为双语评估替代值,将所述目标语句中的词语与所述参考语句中的词语进行匹配,得到所述目标语句与所述参考语句中相同词语的数量;
计算得到的所述数量与所述目标语句中词语的数量的比值作为所述双语 评估替代值。
所述计算机可读指令,还用于被处理器执行时实现以下步骤:
使用所述判别器模型中的第一神经网络模型对所述目标语句进行编码,得到第一编码向量;
使用所述判别器模型中的第二神经网络模型对所述参考语句进行编码,得到第二编码向量;
将所述第一编码向量及所述第二编码向量输入到所述判别器模型中的第三神经网络模型进行相似度计算,得到所述目标语句与所述参考语句的相似度。
所述计算机可读指令,还用于被处理器执行时实现以下步骤:
初始化所述翻译模型包括的输入层、中间层和输出层;
构建训练样本集合;所述训练样本集合包括待翻译的源语句及对应源语句的翻译结果的目标语句;
初始化基于翻译模型输入、翻译模型输出、以及翻译模型参数构建的损失函数;
以所选取训练样本的源语句作为输入、所选取训练样本的目标语句作为输出、求解所述损失函数相对于所选取训练样本取得最小值时翻译模型参数中与所选取训练样本对应维度的更新值;
将每个选取训练样本对应的更新值基于相应样本的相似度融合;
基于融合得到的所述翻译模型参数对应所述训练样本集合的更新值,并更新所述翻译模型参数。
所述计算机可读指令,还用于被处理器执行时实现以下步骤:
使用翻译模型的编码器模型计算对应所述源语句的语义向量;
基于所述输入所述翻译模型中解码器模型的所述语义向量、输入所述语义向量时所述解码器模型的状态、所述解码器模型上一次输出的词语,计算候选词语与所述源语句中词语的对齐概率;
选取计算得到的对齐概率最高的候选词语作为输出的词语;
连接针对所述源语句的各词语对应输出的词语,形成所述目标语句。
所述计算机可读指令,还用于被处理器执行时实现以下步骤:
使用所述翻译模型的编码器模型生成对应所述源语句的向量序列;
使用注意力模型为所述源语句的向量序列中的词向量分配注意力概率;
将所述源语句的向量序列中的词向量与分配的所述注意力概率进行加权求和,得到对应所述源语句的语义向量。
本申请实施例还提供了一种计算机设备,所述计算机设备包括:
存储器,用于存储计算机可读指令;
一个或多个处理器,用于执行所述存储器中存储的计算机可读指令时,实现上述基于翻译模型的翻译方法。
本申请实施例还提供了一个或多个存储介质,存储有计算机可读指令,所述计算机可读指令被一个或多个处理器执行时,实现上述基于翻译模型的翻译方法。
这里需要指出的是:以上涉及基于翻译模型的训练装置的描述,与上述训练方法描述是类似的,同方法的有益效果描述,不做赘述,基于翻译模型的翻译装置的描述,与上述翻译方法描述是类似的,同方法的有益效果描述,不做赘述。对于本申请所述基于翻译模型的训练装置、基于翻译模型的翻译装置、计算机设备以及存储介质实施例中未披露的技术细节,请参照本申请方法实施例的描述。
实施例的全部或部分步骤可以通过程序指令相关的硬件来完成,前述的程序可以存储于一计算机可读取存储介质中,该程序在执行时,执行包括上述方法实施例的步骤;而前述的存储介质包括:移动存储设备、随机存取存储器(RAM,Random Access Memory)、只读存储器(ROM,Read-Only Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
或者,本申请上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时,也可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请实施例的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本申请各个实施例所述方法的全部或部分。而前述的存储介质包括:移动存储设备、RAM、ROM、磁碟或者光盘等各种可以存储 程序代码的介质。
以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以所述权利要求的保护范围为准。
Claims (20)
- 一种基于翻译模型的训练方法,包括:计算机设备向翻译模型输入源语句,获得所述翻译模型输出的目标语句;所述计算机设备确定所述目标语句对于所述源语句的忠实度;所述计算机设备以所述目标语句和参考语句作为判别器模型的输入、以所述忠实度作为所述判别器模型的输出,训练所述判别器模型计算所述目标语句与所述参考语句的相似度的性能;所述计算机设备使用所述判别器模型输出所述目标语句与所述参考语句的相似度;及所述计算机设备以所述源语句作为所述翻译模型的输入、以所述目标语句作为所述翻译模型的输出、并以所述相似度作为权重系数,训练所述翻译模型根据输入的源语句输出对应的目标语句的性能。
- 如权利要求1所述的方法,其特征在于,所述计算机设备确定所述目标语句对于所述源语句的忠实度,包括:所述计算机设备响应于所述忠实度为覆盖率,获取所述目标语句的词语覆盖所述源语句的词语的第一覆盖数量;所述计算机设备获取所述参考语句的词语覆盖所述源语句的词语的第二覆盖数量;所述计算机设备计算所述第一覆盖数量与所述第二覆盖数量的差值;及所述计算机设备计算所述差值与所述第二覆盖数量的比值作为所述覆盖率。
- 如权利要求2所述的方法,其特征在于,所述计算机设备获取所述目标语句的词语覆盖所述源语句的词语的第一覆盖数量,包括:所述计算机设备在获得所述翻译模型输出的目标语句的向量序列的过程中,将所述目标语句中每个词语的向量与所述源语句中对应词语的向量进行对齐,形成第一对齐矩阵;及所述计算机设备从所述第一对齐矩阵中统计得到所述目标语句中词语覆盖所述源语句中词语的第一覆盖数量。
- 如权利要求2所述的方法,其特征在于,所述计算机设备获取所述参考语句的词语覆盖所述源语句的词语的第二覆盖数量,包括:所述计算机设备分别获取对应所述参考语句及所述源语句的向量序列;所述计算机设备对齐所述源语句的向量序列与所述参考语句的向量序列中相同词语对应的向量,得到第二对齐矩阵;及所述计算机设备从所述第二对齐矩阵统计得到所述参考语句中词语覆盖所述源语句中词语的第二覆盖数量。
- 如权利要求1所述的方法,其特征在于,所述计算机设备确定所述目标语句对于所述源语句的忠实度,包括:所述计算机设备响应于所述忠实度为双语评估替代值,将所述目标语句中的词语与所述参考语句中的词语进行匹配,得到所述目标语句与所述参考语句中相同词语的数量;及所述计算机设备计算得到的所述数量与所述目标语句中词语的数量的比值作为所述双语评估替代值。
- 如权利要求1所述的方法,其特征在于,所述计算机设备使用所述判别器模型输出所述目标语句与所述参考语句的相似度,包括:所述计算机设备使用所述判别器模型中的第一神经网络模型对所述目标语句进行编码,得到第一编码向量;所述计算机设备使用所述判别器模型中的第二神经网络模型对所述参考语句进行编码,得到第二编码向量;及所述计算机设备将所述第一编码向量及所述第二编码向量输入到所述判别器模型中的第三神经网络模型进行相似度计算,得到所述目标语句与所述参考语句的相似度。
- 如权利要求1所述的方法,其特征在于,所述计算机设备以所述源语句作为所述翻译模型的输入、以所述目标语句作为所述翻译模型的输出、并以所述相似度作为权重系数,训练所述翻译模型根据输入的源语句输出对应的目标语句的性能,包括:所述计算机设备初始化所述翻译模型包括的输入层、中间层和输出层;所述计算机设备构建训练样本集合;所述训练样本集合包括待翻译的源 语句及对应源语句的翻译结果的目标语句;所述计算机设备初始化基于翻译模型输入、翻译模型输出、以及翻译模型参数构建的损失函数;所述计算机设备以所选取训练样本的源语句作为输入、所选取训练样本的目标语句作为输出、求解翻译模型参数中与所选取训练样本对应维度的更新值;所述计算机设备将每个选取训练样本对应的更新值基于相应样本的相似度融合;及所述计算机设备基于融合得到的所述翻译模型参数对应所述训练样本集合的更新值,更新所述翻译模型参数。
- 如权利要求1所述的方法,其特征在于,所述计算机设备向翻译模型输入源语句,获得所述翻译模型输出的目标语句,包括:计算机设备使用翻译模型的编码器模型计算对应所述源语句的语义向量;所述计算机设备基于所述输入所述翻译模型中解码器模型的所述语义向量、输入所述语义向量时所述解码器模型的状态、所述解码器模型上一次输出的词语,计算候选词语与所述源语句中词语的对齐概率;所述计算机设备选取计算得到的对齐概率最高的候选词语作为输出的词语;及所述计算机设备连接针对所述源语句的各词语对应输出的词语,形成所述目标语句。
- 如权利要求8所述的方法,其特征在于,所述计算机设备使用翻译模型的编码器模型计算对应所述源语句的语义向量,包括:所述计算机设备使用所述翻译模型的编码器模型生成对应所述源语句的向量序列;所述计算机设备使用注意力模型为所述源语句的向量序列中的词向量分配注意力概率;及所述计算机设备将所述源语句的向量序列中的词向量与分配的所述注意力概率进行加权求和,得到对应所述源语句的语义向量。
- 一种基于翻译模型的翻译方法,包括:计算机设备获取在对抗网络中训练得到的翻译模型;其中,所述翻译模型为在对抗网络中利用以源语句作为翻译模型输入、以目标语句作为翻译模型输出、并以所述目标语句与参考语句的相似度作为权重系数训练得到,所述相似度为利用所述对抗网络的判别器模型计算得到;所述计算机设备输入待翻译语句至所述训练得到的翻译模型;所述计算机设备使用所述训练得到的翻译模型计算对应所述待翻译语句的语义向量;及所述计算机设备以所述语义向量作为所述训练得到的翻译模型的输入,输出对应所述待翻译语句的翻译结果语句。
- 如权利要求10所述的方法,其特征在于,所述判别器模型计算所述相似度的性能,通过在对抗网络中利用参考语句和目标语句作为判别器模型的输入、以所述目标语句对于源语句的忠实度作为判别器模型的输出训练得到。
- 如权利要求10所述的方法,其特征在于,所述计算机设备使用所述训练得到的翻译模型计算对应所述待翻译语句的语义向量,包括:所述计算机设备使用所述训练得到的翻译模型中编码器模型生成对应所述源语句的向量序列;所述计算机设备使用注意力模型为所述源语句的向量序列中的词向量分配注意力概率;及所述计算机设备将所述源语句的向量序列中的词向量与分配的所述注意力概率进行加权求和,得到对应所述源语句的语义向量。
- 如权利要求10所述的方法,其特征在于,所述计算机设备以所述语义向量作为所述训练得到的翻译模型的输入,输出对应所述待翻译语句的翻译结果语句,包括:所述计算机设备以所述语义向量、所述训练得到的翻译模型中解码器模型的状态、所述解码器模型上一次输出的词语作为所述训练得到的翻译模型的输入,计算候选词语与所述待翻译语句中词语的对齐概率;所述计算机设备基于计算得到的对齐概率,选取满足概率条件的的候选 词语作为输出的词语;及所述计算机设备连接针对所述待翻译语句的各词语对应输出的词语,形成对应所述待翻译语句的翻译结果语句。
- 一种计算机设备,包括存储器和一个或多个处理器,所述存储器中存储有计算机可读指令,所述计算机可读指令被一个或多个处理器执行时,使得所述一个或多个处理器执行如下步骤:向翻译模型输入源语句,获得所述翻译模型输出的目标语句;确定所述目标语句对于所述源语句的忠实度;以所述目标语句和参考语句作为判别器模型的输入、以所述忠实度作为所述判别器模型的输出,训练所述判别器模型计算所述目标语句与所述参考语句的相似度的性能;使用所述判别器模型输出所述目标语句与所述参考语句的相似度;及以所述源语句作为所述翻译模型的输入、以所述目标语句作为所述翻译模型的输出、并以所述相似度作为权重系数,训练所述翻译模型根据输入的源语句输出对应的目标语句的性能。
- 如权利要求14所述的计算机设备,其特征在于,所述确定所述目标语句对于所述源语句的忠实度,包括:响应于所述忠实度为覆盖率,获取所述目标语句的词语覆盖所述源语句的词语的第一覆盖数量;获取所述参考语句的词语覆盖所述源语句的词语的第二覆盖数量;计算所述第一覆盖数量与所述第二覆盖数量的差值;及计算所述差值与所述第二覆盖数量的比值作为所述覆盖率。
- 如权利要求15所述的计算机设备,其特征在于,所述获取所述目标语句的词语覆盖所述源语句的词语的第一覆盖数量,包括:在获得所述翻译模型输出的目标语句的向量序列的过程中,将所述目标语句中每个词语的向量与所述源语句中对应词语的向量进行对齐,形成第一对齐矩阵;及从所述第一对齐矩阵中统计得到所述目标语句中词语覆盖所述源语句中词语的第一覆盖数量。
- 如权利要求15所述的计算机设备,其特征在于,所述获取所述参考语句的词语覆盖所述源语句的词语的第二覆盖数量,包括:分别获取对应所述参考语句及所述源语句的向量序列;对齐所述源语句的向量序列与所述参考语句的向量序列中相同词语对应的向量,得到第二对齐矩阵;及从所述第二对齐矩阵统计得到所述参考语句中词语覆盖所述源语句中词语的第二覆盖数量。
- 一种计算机设备,包括存储器和一个或多个处理器,所述存储器中存储有计算机可读指令,所述计算机可读指令被一个或多个处理器执行时,使得所述一个或多个处理器执行如下步骤:获取在对抗网络中训练得到的翻译模型;其中,所述翻译模型为在对抗网络中利用以源语句作为翻译模型输入、以目标语句作为翻译模型输出、并以所述目标语句与参考语句的相似度作为权重系数训练得到,所述相似度为利用所述对抗网络的判别器模型计算得到;输入待翻译语句至所述训练得到的翻译模型;使用所述训练得到的翻译模型计算对应所述待翻译语句的语义向量;及以所述语义向量作为所述训练得到的翻译模型的输入,输出对应所述待翻译语句的翻译结果语句。
- 一个或多个存储有计算机可读指令的存储介质,所述计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行如下步骤:向翻译模型输入源语句,获得所述翻译模型输出的目标语句;确定所述目标语句对于所述源语句的忠实度;以所述目标语句和参考语句作为判别器模型的输入、以所述忠实度作为所述判别器模型的输出,训练所述判别器模型计算所述目标语句与所述参考语句的相似度的性能;使用所述判别器模型输出所述目标语句与所述参考语句的相似度;及以所述源语句作为所述翻译模型的输入、以所述目标语句作为所述翻译模型的输出、并以所述相似度作为权重系数,训练所述翻译模型根据输入的源语句输出对应的目标语句的性能。
- 一个或多个存储有计算机可读指令的存储介质,所述计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行如下步骤:获取在对抗网络中训练得到的翻译模型;其中,所述翻译模型为在对抗网络中利用以源语句作为翻译模型输入、以目标语句作为翻译模型输出、并以所述目标语句与参考语句的相似度作为权重系数训练得到,所述相似度为利用所述对抗网络的判别器模型计算得到;输入待翻译语句至所述训练得到的翻译模型;使用所述训练得到的翻译模型计算对应所述待翻译语句的语义向量;及以所述语义向量作为所述训练得到的翻译模型的输入,输出对应所述待翻译语句的翻译结果语句。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/686,083 US11270079B2 (en) | 2017-12-15 | 2019-11-15 | Translation model based training method and translation method, computer device, and storage medium |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711354563.0A CN108304390B (zh) | 2017-12-15 | 2017-12-15 | 基于翻译模型的训练方法、训练装置、翻译方法及存储介质 |
CN201711354563.0 | 2017-12-15 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/686,083 Continuation US11270079B2 (en) | 2017-12-15 | 2019-11-15 | Translation model based training method and translation method, computer device, and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019114695A1 true WO2019114695A1 (zh) | 2019-06-20 |
Family
ID=62870683
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2018/120293 WO2019114695A1 (zh) | 2017-12-15 | 2018-12-11 | 基于翻译模型的训练方法、翻译方法、计算机设备及存储介质 |
Country Status (3)
Country | Link |
---|---|
US (1) | US11270079B2 (zh) |
CN (1) | CN108304390B (zh) |
WO (1) | WO2019114695A1 (zh) |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111027332A (zh) * | 2019-12-11 | 2020-04-17 | 北京百度网讯科技有限公司 | 生成翻译模型的方法和装置 |
CN111339782A (zh) * | 2020-02-20 | 2020-06-26 | 合肥工业大学 | 一种基于多层次语义解析的手语翻译***及方法 |
CN111382582A (zh) * | 2020-01-21 | 2020-07-07 | 沈阳雅译网络技术有限公司 | 一种基于非自回归的神经机器翻译解码加速方法 |
CN112036195A (zh) * | 2020-09-16 | 2020-12-04 | 北京小米松果电子有限公司 | 机器翻译方法、装置及存储介质 |
CN112183119A (zh) * | 2020-09-16 | 2021-01-05 | 北京小米松果电子有限公司 | 机器翻译方法、装置及存储介质 |
CN112287656A (zh) * | 2020-10-12 | 2021-01-29 | 四川语言桥信息技术有限公司 | 文本比对方法、装置、设备和存储介质 |
CN112507081A (zh) * | 2020-12-16 | 2021-03-16 | 平安科技(深圳)有限公司 | 相似句匹配方法、装置、计算机设备及存储介质 |
CN112613327A (zh) * | 2021-01-07 | 2021-04-06 | 北京捷通华声科技股份有限公司 | 一种信息处理方法及装置 |
CN113408267A (zh) * | 2021-06-23 | 2021-09-17 | 沈阳雅译网络技术有限公司 | 一种基于预训练模型的词对齐性能提升方法 |
CN113449529A (zh) * | 2020-03-24 | 2021-09-28 | 北京金山数字娱乐科技有限公司 | 一种翻译模型的训练方法及装置、翻译方法及装置 |
CN113642338A (zh) * | 2020-05-11 | 2021-11-12 | 阿里巴巴集团控股有限公司 | 翻译模型训练方法及装置 |
CN114239608A (zh) * | 2021-11-16 | 2022-03-25 | 北京百度网讯科技有限公司 | 翻译方法、模型训练方法、装置、电子设备及存储介质 |
CN115081462A (zh) * | 2022-06-15 | 2022-09-20 | 京东科技信息技术有限公司 | 翻译模型训练、翻译方法和装置 |
Families Citing this family (46)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108304390B (zh) * | 2017-12-15 | 2020-10-16 | 腾讯科技(深圳)有限公司 | 基于翻译模型的训练方法、训练装置、翻译方法及存储介质 |
CN111712835B (zh) * | 2018-01-02 | 2023-09-01 | 诺基亚技术有限公司 | 数据传输***中的信道建模 |
CN111602144A (zh) * | 2018-02-09 | 2020-08-28 | 渊慧科技有限公司 | 生成指令序列以控制执行任务的代理的生成神经网络*** |
CN109271644A (zh) * | 2018-08-16 | 2019-01-25 | 北京紫冬认知科技有限公司 | 一种翻译模型训练方法及装置 |
CN110874535B (zh) * | 2018-08-28 | 2023-07-25 | 阿里巴巴集团控股有限公司 | 依存关系对齐组件、依存关系对齐训练方法、设备及介质 |
CN109118113B (zh) * | 2018-08-31 | 2021-08-10 | 传神语联网网络科技股份有限公司 | Etm架构及词移距离 |
CN109118109B (zh) * | 2018-08-31 | 2021-06-01 | 传神语联网网络科技股份有限公司 | 基于etm的质量评估 |
CN110874537B (zh) * | 2018-08-31 | 2023-06-27 | 阿里巴巴集团控股有限公司 | 多语言翻译模型的生成方法、翻译方法及设备 |
CN109145315B (zh) * | 2018-09-05 | 2022-03-18 | 腾讯科技(深圳)有限公司 | 文本翻译方法、装置、存储介质和计算机设备 |
US11151334B2 (en) * | 2018-09-26 | 2021-10-19 | Huawei Technologies Co., Ltd. | Systems and methods for multilingual text generation field |
CN109558597B (zh) * | 2018-12-17 | 2022-05-24 | 北京百度网讯科技有限公司 | 文本翻译方法及装置、设备及存储介质 |
CN109710953B (zh) * | 2018-12-29 | 2023-04-11 | 成都金山互动娱乐科技有限公司 | 一种翻译方法及装置、计算设备、存储介质和芯片 |
US11106873B2 (en) * | 2019-01-22 | 2021-08-31 | Sap Se | Context-based translation retrieval via multilingual space |
CN109885830A (zh) * | 2019-01-29 | 2019-06-14 | 金蝶软件(中国)有限公司 | 语句释义方法、装置、计算机设备 |
CN109858044B (zh) * | 2019-02-01 | 2023-04-18 | 成都金山互动娱乐科技有限公司 | 语言处理方法和装置、语言处理***的训练方法和装置 |
CN111597800B (zh) * | 2019-02-19 | 2023-12-12 | 百度在线网络技术(北京)有限公司 | 同义句的获取方法及装置、设备及存储介质 |
CN109902312B (zh) * | 2019-03-01 | 2023-07-11 | 北京金山数字娱乐科技有限公司 | 一种翻译方法及装置、翻译模型的训练方法及装置 |
CN110008312A (zh) * | 2019-04-10 | 2019-07-12 | 成都信息工程大学 | 一种文档写作助手实现方法、***及电子设备 |
CN110110337B (zh) * | 2019-05-08 | 2023-04-18 | 网易有道信息技术(北京)有限公司 | 翻译模型训练方法、介质、装置和计算设备 |
CN110210026B (zh) * | 2019-05-29 | 2023-05-26 | 北京百度网讯科技有限公司 | 语音翻译方法、装置、计算机设备和存储介质 |
CN110276082B (zh) * | 2019-06-06 | 2023-06-30 | 百度在线网络技术(北京)有限公司 | 基于动态窗口的翻译处理方法和装置 |
CN110442878B (zh) * | 2019-06-19 | 2023-07-21 | 腾讯科技(深圳)有限公司 | 翻译方法、机器翻译模型的训练方法、装置及存储介质 |
US11735184B2 (en) * | 2019-07-24 | 2023-08-22 | Alibaba Group Holding Limited | Translation and speech recognition method, apparatus, and device |
US11664820B2 (en) * | 2019-09-25 | 2023-05-30 | Nokia Technologies Oy | Domain adaptation |
CN110781686B (zh) * | 2019-10-30 | 2023-04-18 | 普信恒业科技发展(北京)有限公司 | 一种语句相似度计算方法、装置及计算机设备 |
CN111160043B (zh) * | 2019-12-31 | 2024-04-30 | 科大讯飞股份有限公司 | 特征编码方法、装置、电子设备及可读存储介质 |
CN113283249A (zh) * | 2020-02-19 | 2021-08-20 | 阿里巴巴集团控股有限公司 | 机器翻译方法、装置及计算机可读存储介质 |
KR20210113833A (ko) * | 2020-03-09 | 2021-09-17 | 삼성전자주식회사 | 벡터 변환을 이용한 데이터 처리 방법 및 장치 |
CN113223503B (zh) * | 2020-04-29 | 2022-06-14 | 浙江大学 | 一种基于测试反馈的核心训练语音选择方法 |
CN111859997B (zh) * | 2020-06-16 | 2024-01-26 | 北京百度网讯科技有限公司 | 机器翻译中的模型训练方法、装置、电子设备及存储介质 |
CN111598051B (zh) * | 2020-06-16 | 2023-11-14 | 腾讯科技(深圳)有限公司 | 一种脸部验证方法、装置、设备及可读存储介质 |
CN111859991B (zh) * | 2020-07-29 | 2023-04-07 | 中国平安财产保险股份有限公司 | 语言翻译处理模型训练方法和语言翻译处理方法 |
US11875131B2 (en) * | 2020-09-16 | 2024-01-16 | International Business Machines Corporation | Zero-shot cross-lingual transfer learning |
CN112257453B (zh) * | 2020-09-23 | 2022-02-22 | 昆明理工大学 | 融合关键词和语义特征的汉越文本相似度计算方法 |
CN112417895A (zh) * | 2020-12-15 | 2021-02-26 | 广州博冠信息科技有限公司 | 弹幕数据处理方法、装置、设备以及存储介质 |
CN112580822B (zh) * | 2020-12-16 | 2023-10-17 | 北京百度网讯科技有限公司 | 机器学习模型的对抗训练方法装置、电子设备和介质 |
KR20220093653A (ko) | 2020-12-28 | 2022-07-05 | 삼성전자주식회사 | 전자 장치 및 그 제어 방법 |
CN113762969B (zh) * | 2021-04-23 | 2023-08-08 | 腾讯科技(深圳)有限公司 | 信息处理方法、装置、计算机设备和存储介质 |
CN113505611B (zh) * | 2021-07-09 | 2022-04-15 | 中国人民解放军战略支援部队信息工程大学 | 在生成对抗中获得更好的语音翻译模型的训练方法和*** |
CN113408307B (zh) * | 2021-07-14 | 2022-06-14 | 北京理工大学 | 一种基于翻译模板的神经机器翻译方法 |
CN113723117B (zh) * | 2021-07-29 | 2024-04-30 | 北京搜狗科技发展有限公司 | 一种翻译模型的训练方法、装置和用于翻译模型训练的装置 |
CN113836192B (zh) * | 2021-08-13 | 2022-05-03 | 深译信息科技(横琴)有限公司 | 平行语料的挖掘方法、装置、计算机设备及存储介质 |
CN113901847B (zh) * | 2021-09-16 | 2024-05-24 | 昆明理工大学 | 基于源语言句法增强解码的神经机器翻译方法 |
CN116757223A (zh) * | 2023-08-16 | 2023-09-15 | 中国石油大学(华东) | 一种机器翻译模型高语义相似度对抗性样本生成方法 |
CN116992888A (zh) * | 2023-09-25 | 2023-11-03 | 天津华来科技股份有限公司 | 基于自然语义的数据分析方法及*** |
CN117252154B (zh) * | 2023-11-20 | 2024-01-23 | 北京语言大学 | 一种基于预训练语言模型的中文简繁字符转换方法及*** |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130110491A1 (en) * | 2011-10-28 | 2013-05-02 | Microsoft Corporation | Discriminative learning of feature functions of generative type in speech translation |
CN105183720A (zh) * | 2015-08-05 | 2015-12-23 | 百度在线网络技术(北京)有限公司 | 基于rnn模型的机器翻译方法和装置 |
CN105446958A (zh) * | 2014-07-18 | 2016-03-30 | 富士通株式会社 | 词对齐方法和词对齐设备 |
CN107368475A (zh) * | 2017-07-18 | 2017-11-21 | 中译语通科技(北京)有限公司 | 一种基于生成对抗神经网络的机器翻译方法和*** |
CN108304390A (zh) * | 2017-12-15 | 2018-07-20 | 腾讯科技(深圳)有限公司 | 基于翻译模型的训练方法、翻译方法、装置及存储介质 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8521516B2 (en) * | 2008-03-26 | 2013-08-27 | Google Inc. | Linguistic key normalization |
US8798984B2 (en) * | 2011-04-27 | 2014-08-05 | Xerox Corporation | Method and system for confidence-weighted learning of factored discriminative language models |
US9026425B2 (en) * | 2012-08-28 | 2015-05-05 | Xerox Corporation | Lexical and phrasal feature domain adaptation in statistical machine translation |
CN106484682B (zh) * | 2015-08-25 | 2019-06-25 | 阿里巴巴集团控股有限公司 | 基于统计的机器翻译方法、装置及电子设备 |
JP6727610B2 (ja) * | 2016-09-05 | 2020-07-22 | 国立研究開発法人情報通信研究機構 | 文脈解析装置及びそのためのコンピュータプログラム |
-
2017
- 2017-12-15 CN CN201711354563.0A patent/CN108304390B/zh active Active
-
2018
- 2018-12-11 WO PCT/CN2018/120293 patent/WO2019114695A1/zh active Application Filing
-
2019
- 2019-11-15 US US16/686,083 patent/US11270079B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130110491A1 (en) * | 2011-10-28 | 2013-05-02 | Microsoft Corporation | Discriminative learning of feature functions of generative type in speech translation |
CN105446958A (zh) * | 2014-07-18 | 2016-03-30 | 富士通株式会社 | 词对齐方法和词对齐设备 |
CN105183720A (zh) * | 2015-08-05 | 2015-12-23 | 百度在线网络技术(北京)有限公司 | 基于rnn模型的机器翻译方法和装置 |
CN107368475A (zh) * | 2017-07-18 | 2017-11-21 | 中译语通科技(北京)有限公司 | 一种基于生成对抗神经网络的机器翻译方法和*** |
CN108304390A (zh) * | 2017-12-15 | 2018-07-20 | 腾讯科技(深圳)有限公司 | 基于翻译模型的训练方法、翻译方法、装置及存储介质 |
Cited By (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111027332A (zh) * | 2019-12-11 | 2020-04-17 | 北京百度网讯科技有限公司 | 生成翻译模型的方法和装置 |
CN111027332B (zh) * | 2019-12-11 | 2023-06-02 | 北京百度网讯科技有限公司 | 生成翻译模型的方法和装置 |
CN111382582A (zh) * | 2020-01-21 | 2020-07-07 | 沈阳雅译网络技术有限公司 | 一种基于非自回归的神经机器翻译解码加速方法 |
CN111382582B (zh) * | 2020-01-21 | 2023-04-07 | 沈阳雅译网络技术有限公司 | 一种基于非自回归的神经机器翻译解码加速方法 |
CN111339782A (zh) * | 2020-02-20 | 2020-06-26 | 合肥工业大学 | 一种基于多层次语义解析的手语翻译***及方法 |
CN111339782B (zh) * | 2020-02-20 | 2023-03-28 | 合肥工业大学 | 一种基于多层次语义解析的手语翻译***及方法 |
CN113449529A (zh) * | 2020-03-24 | 2021-09-28 | 北京金山数字娱乐科技有限公司 | 一种翻译模型的训练方法及装置、翻译方法及装置 |
CN113642338A (zh) * | 2020-05-11 | 2021-11-12 | 阿里巴巴集团控股有限公司 | 翻译模型训练方法及装置 |
CN112036195A (zh) * | 2020-09-16 | 2020-12-04 | 北京小米松果电子有限公司 | 机器翻译方法、装置及存储介质 |
CN112183119A (zh) * | 2020-09-16 | 2021-01-05 | 北京小米松果电子有限公司 | 机器翻译方法、装置及存储介质 |
CN112287656B (zh) * | 2020-10-12 | 2024-05-28 | 四川语言桥信息技术有限公司 | 文本比对方法、装置、设备和存储介质 |
CN112287656A (zh) * | 2020-10-12 | 2021-01-29 | 四川语言桥信息技术有限公司 | 文本比对方法、装置、设备和存储介质 |
CN112507081A (zh) * | 2020-12-16 | 2021-03-16 | 平安科技(深圳)有限公司 | 相似句匹配方法、装置、计算机设备及存储介质 |
CN112613327A (zh) * | 2021-01-07 | 2021-04-06 | 北京捷通华声科技股份有限公司 | 一种信息处理方法及装置 |
CN113408267A (zh) * | 2021-06-23 | 2021-09-17 | 沈阳雅译网络技术有限公司 | 一种基于预训练模型的词对齐性能提升方法 |
CN113408267B (zh) * | 2021-06-23 | 2023-09-01 | 沈阳雅译网络技术有限公司 | 一种基于预训练模型的词对齐性能提升方法 |
CN114239608B (zh) * | 2021-11-16 | 2022-11-25 | 北京百度网讯科技有限公司 | 翻译方法、模型训练方法、装置、电子设备及存储介质 |
CN114239608A (zh) * | 2021-11-16 | 2022-03-25 | 北京百度网讯科技有限公司 | 翻译方法、模型训练方法、装置、电子设备及存储介质 |
CN115081462A (zh) * | 2022-06-15 | 2022-09-20 | 京东科技信息技术有限公司 | 翻译模型训练、翻译方法和装置 |
Also Published As
Publication number | Publication date |
---|---|
CN108304390A (zh) | 2018-07-20 |
CN108304390B (zh) | 2020-10-16 |
US11270079B2 (en) | 2022-03-08 |
US20200081982A1 (en) | 2020-03-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2019114695A1 (zh) | 基于翻译模型的训练方法、翻译方法、计算机设备及存储介质 | |
JP7179183B2 (ja) | ビデオキャプションの生成方法、装置、デバイスおよびコンピュータプログラム | |
CN110598224B (zh) | 翻译模型的训练方法、文本处理方法、装置及存储介质 | |
CN110110337B (zh) | 翻译模型训练方法、介质、装置和计算设备 | |
CN113343705B (zh) | 一种基于文本语义的细节保持图像生成方法及*** | |
EP3623957A1 (en) | Generation of point of interest copy | |
CN111966800B (zh) | 情感对话生成方法、装置及情感对话模型训练方法、装置 | |
CN109558605B (zh) | 用于翻译语句的方法和装置 | |
JP6969443B2 (ja) | 学習品質推定装置、方法、及びプログラム | |
CN111897933A (zh) | 情感对话生成方法、装置及情感对话模型训练方法、装置 | |
CN108021549B (zh) | 序列转换方法及装置 | |
US20220300718A1 (en) | Method, system, electronic device and storage medium for clarification question generation | |
EP3333723B1 (en) | Sequence conversion method and apparatus | |
CN111488742B (zh) | 用于翻译的方法和装置 | |
CN110705273B (zh) | 基于神经网络的信息处理方法及装置、介质和电子设备 | |
CN116912642A (zh) | 基于双模多粒度交互的多模态情感分析方法、设备及介质 | |
CN112837669A (zh) | 语音合成方法、装置及服务器 | |
CN116432663B (zh) | 基于要素简图的可控多样性专业文本生成方法及*** | |
CN110705611A (zh) | 眼底图像样本扩展方法、装置、介质及电子设备 | |
CN115662565A (zh) | 一种融合标签信息的医学影像报告生成方法及设备 | |
CN111310847B (zh) | 训练要素分类模型的方法和装置 | |
CN113515960A (zh) | 一种融合句法信息的翻译质量自动评估方法 | |
CN118098222B (zh) | 语音关系提取方法、装置、计算机设备及存储介质 | |
CN117251599B (zh) | 一种视频语料智能测试优化方法、装置和存储介质 | |
CN111652004B (zh) | 一种用于机器翻译***的融合方法及装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18889449 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18889449 Country of ref document: EP Kind code of ref document: A1 |