CN117933268A - End-to-end unsupervised resistance text rewriting method and device - Google Patents
End-to-end unsupervised resistance text rewriting method and device Download PDFInfo
- Publication number
- CN117933268A CN117933268A CN202410323254.0A CN202410323254A CN117933268A CN 117933268 A CN117933268 A CN 117933268A CN 202410323254 A CN202410323254 A CN 202410323254A CN 117933268 A CN117933268 A CN 117933268A
- Authority
- CN
- China
- Prior art keywords
- text
- rewrite
- resistance
- representing
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000012549 training Methods 0.000 claims abstract description 55
- 230000003042 antagnostic effect Effects 0.000 claims abstract description 27
- 230000006870 function Effects 0.000 claims description 52
- 230000008485 antagonism Effects 0.000 claims description 20
- 238000003860 storage Methods 0.000 claims description 12
- 239000003550 marker Substances 0.000 claims description 5
- 238000012986 modification Methods 0.000 claims description 5
- 230000004048 modification Effects 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 3
- 238000003058 natural language processing Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 238000013136 deep learning model Methods 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000004821 distillation Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 230000002996 emotional effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/194—Calculation of difference between files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/094—Adversarial learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Machine Translation (AREA)
Abstract
The invention belongs to the technical field of natural language processing, and particularly relates to an end-to-end unsupervised resistance text rewriting method and device. The method comprises the following steps: constructing an end-to-end resistance text rewrite model, which comprises a rewrite condition generator and a condition text rewrite device; in the training stage, inputting an original sample into an resistance text rewrite model, introducing a supervision label of the original sample and a label of the rewrite text output by the resistance text rewrite model by an resistance discriminator, determining the rewrite text inconsistent with the supervision label as the resistance sample, and training the resistance text rewrite model by using the original sample and the resistance sample; in the application stage, the original text is input into a trained antagonistic text rewrite model to generate the antagonistic text conforming to the corresponding control conditions. The invention can generate the resistance sample in an end-to-end manner, thereby improving the robustness of the downstream task model in a data enhancement manner.
Description
Technical Field
The invention belongs to the technical field of natural language processing, and particularly relates to an end-to-end unsupervised resistance text rewriting method and device.
Background
The challenge sample refers to a sample that is deliberately constructed for spoofing a deep learning model, also referred to as an attack sample. Attack sample generation methods have successfully spoofed natural language processing models for a variety of tasks, such as text classification, machine translation, and the like. With the wide application of the neural network model in different fields, the research on the resistance sample has important significance on the robustness and safety of the neural network model.
Typically, the resistant sample is constructed in such a way that it is perturbed from the original sample. Early challenge training was primarily directed to the image field by applying pixel-level noise on the original samples. For example, chinese patent document CN117173508a discloses a method, apparatus, device and storage medium for generating an attack-resistant image, which uses a contrast learning method to train a generator to generate a noise image and combine the noise image with an original image to achieve an attack effect.
In the field of natural language processing, there are many text rewrite methods, for example, chinese patent document CN116894431a discloses a text processing model training method, a text rewrite method and apparatus, and a storage medium that train a text rewrite model using keywords and source sentence codes of a given source sentence; chinese patent document CN116108830a discloses a syntax-controllable text rewrite method and apparatus, which uses vocabulary combination knowledge, uses original sentence and syntax structure information as input, trains a syntax-controllable text rewrite model. However, these above-described text rewriting methods do not take into consideration the resistance of rewriting text.
Because of the discrete nature of text, the resistance samples are typically generated by overwriting the original samples with word substitutions or the like. For example, the method proposed in document "TextBugger: GENERATING ADVERSARIAL Text AGAINST REAL-world Applications" generates a resistance sample by randomly replacing words in the sample to obtain candidate samples that may change the model prediction results. Samples with semantics similar to the original text are then selected from them as reasonable resistance samples. Because of the discrete nature of text vocabulary, the commonly used aggressive sample generation technology needs to repeatedly interact with a target model to judge the importance of the vocabulary in the sample and select the corresponding replacement vocabulary, and the end-to-end training mode cannot be performed, so that the methods are quite time-consuming and have high calculation cost; the generated antagonism sample only aims at a specific target model, and the mobility of the cross model is difficult to ensure.
Based on this, there is a need to design an end-to-end unsupervised, antagonistic semantic controllable text rewrite method to solve the above-mentioned problems.
Disclosure of Invention
The present invention aims to overcome at least one of the above-mentioned drawbacks of the prior art and to provide an end-to-end unsupervised method for writing resistant text.
The invention also provides a device for realizing the end-to-end unsupervised resistance text rewriting method.
Summary of the invention:
The invention designs an end-to-end resistance text rewrite model, based on a given original sample, the resistance text rewrite model is utilized to finally generate the resistance sample, so that not only is fine-granularity semantic control realized to generate a rewrite text, but also the resistance sample can be generated in an end-to-end mode, and the robustness of a downstream task model can be improved in a data enhancement mode.
Technical term interpretation:
resistance sample: a deliberately structured sample for spoofing a deep learning model, also referred to as an attack sample, is referred to in the present invention. With the wide application of the neural network model in different fields, the research on the resistance sample has important significance on the robustness and the safety of the neural network model.
Control conditions: the form of the control condition may include various contents such as changing the proportion of the vocabulary, sentence structure, or emotion tendencies. The invention uses the vocabulary replacement mark sequence as the condition, wherein each replacement mark bit corresponds to the original sample vocabulary one by one, and the vocabulary of the corresponding position in the original sample is replaced.
A rewrite condition generator: in the present invention, the rewrite condition generator receives the original sample as an input and generates a control condition that can guide the condition text rewriter to generate the resistance sample.
Conditional text rewriter: in the invention, the conditional text rewriter accepts the original sample and the control condition as input, and generates a rewritten text which has similar semantics to the original sample and meets the control condition, namely an resistance sample.
Antagonism discriminator: in the invention, a classifier or a regressor trained on a target task is used for predicting the label of the rewritten text generated by the conditional text rewriter. If the label of the rewritten text does not coincide with the supervision label of the original sample, the rewritten text is considered a potential resistance sample.
The detailed technical scheme of the invention is as follows:
an end-to-end unsupervised resistance text rewrite method, the method comprising:
s1, constructing an end-to-end resistance text rewrite model, wherein the resistance text rewrite model comprises a rewrite condition generator and a conditional text rewriter, and the rewrite condition generator is used for generating a rewrite condition according to an original text Generating control conditions/>The conditional text rewriter is used for writing/rewriting the original textAnd control conditions/>Generating corresponding rewritten text/>；
S2, in the training stage, a given original sample is obtainedInputting the resistance text rewrite model and introducing a resistance discriminator to obtain the original sample/>Supervision tag/>Rewritten text output by the resistive text rewrite modelTag/>Tag/>And the original sample/>Supervision tag/>Inconsistent rewritten text/>Determined as an antagonistic sampleAnd uses the original sample/>And the obtained antagonistic sample/>Training the antagonistic text rewrite model;
S3, in the application stage, the given original text is processed Inputting the trained antagonistic text rewrite model to generate a text rewrite model meeting the corresponding control conditions/>Resistant text/>。
According to a preferred embodiment of the present invention, in the step S2, the control condition is as followsUsing the form of vocabulary replacement marker sequence, vocabulary replacement marker bit and original sample/>Binary sequences corresponding to the medium words are denoted/>And use/>Representing the generated rewritten text/>Middle/>Word of individual position/>Is replaced with other words,/>Indicating that it was not replaced.
According to a preferred embodiment of the present invention, in the step S2, the overwrite condition generator uses a pass estimator as an activation function to make the generated control conditionIn discrete binarized form, i.e.:
（1）；
in the formula (1): To control the condition,/> For the original sample,/>For the rewrite condition generator,/>To activate the function.
According to a preferred embodiment of the present invention, the step S2 specifically further includes:
selecting an arbitrary original sample Control conditions/>, obtained by randomly selecting vocabulary replacement locationsThe conditional text rewriter is pre-trained with a training loss function of:
（2）；
In the formula (2): Representing a loss function,/> Represents weights and/>，/>Representing the rewritten text output by the conditional text rewriter,/>Representing the reference rewritten text,/>Representing the original sample,/>Representing control conditions,/>Representing the rewritten text/>, given the original sample and the control conditions, of the conditional text rewriter outputAnd reference rewritten text/>First/>The probabilities of words being identical for each position.
According to a preferred embodiment of the present invention, in the step S2, a given original sample is sampledInputting the resistance text rewrite model and introducing a resistance discriminator to obtain the original sample/>Supervision tag/>Rewritten text/>, output by the resistive text rewrite modelTag/>The method specifically comprises the following steps:
（3）；
In the formula (3): To control the condition,/> For the rewrite condition generator,/>Is a conditional text rewriter,/>For the original sample,/>For the generated rewritten samples,/>For the antagonism discriminator,/>To rewrite the sample/>A corresponding tag.
According to a preferred embodiment of the present invention, in the step S2, the training loss function of the resistance text rewrite model includes the following loss functions:
1) Countering loss function:
（4）；
In the formula (4): Representing challenge losses,/> Representing the correspondence of the resistance discriminator to the rewritten text/>Predicted output of/>Representing the correspondence of the antagonism discriminator to the original sample/>An output of (2);
2) Disturbance proportional loss function:
（5）；
In formula (5): representing disturbance proportional loss,/> Representing the expected modification ratio,/>For the sample length,/>Representing control conditions/>Middle/>Is a number of positions of (a);
3) Conditional constraint text overwrite loss function:
（6）；
In formula (6): Representing conditional constraint text rewrite penalty,/> Rewritten text/>, representing conditional text rewriter outputAnd original sample/>First/>Words of the individual positions are identical,/>Representing the probability;
4) Forced overwrite loss function:
（7）；
in the formula (7): Representing forced overwrite loss,/> Representing sigmod functions;
5) Semantic similarity loss function:
（8）；
In formula (8): Representing semantic similarity loss;
6) Syntax correctness loss function:
（9）；
In the formula (9): Representing loss of syntax correctness,/> Representing the original sample/>Is a dependency syntax tree structure of (1), wherein/>Representing the original sample/>Middle/>Words of individual locations and the/>Words in individual locations have a dependency relationship/>，/>Representing rewritten text/>Middle/>Words of individual locations and the/>Words in individual locations have a dependency relationship/>Is reasonable.
According to a preferred embodiment of the present invention, in the step S2, the training loss function of the resistance text rewrite model is:
（10）；
in the formula (10): training loss function representing an antagonistic text rewrite model,/> Representing challenge loss/>Weights of/>Representing disturbance proportional loss/>Weights of/>Representing conditional constraint text rewrite loss/>Weights of/>Represents forced overwrite loss/>Weights of/>Representing semantic similarity loss/>Weights of/>Representing loss of syntax correctness/>Is a weight of (2).
In another aspect of the present invention, there is provided an apparatus for implementing an end-to-end unsupervised resistance text rewrite method, the apparatus comprising:
A building module for building an end-to-end resistance text rewrite model, wherein the resistance text rewrite model comprises a rewrite condition generator and a condition text rewriter, and the rewrite condition generator is used for generating a rewrite condition according to an original text Generating control conditions/>The conditional text rewriter is used for writing/rewriting the original textAnd control conditions/>Generating corresponding rewritten text/>；
Training module for based on given original sampleInputting the original sample into the resistance text rewrite model, and introducing a resistance discriminator to obtain the original sample/>Supervision tag/>Rewritten text/>, output by the resistive text rewrite modelTag/>Tag/>And the original sample/>Supervision tag/>Inconsistent rewritten text/>Determined as resistant sample/>And uses the original sample/>And the obtained antagonistic sample/>Training the antagonistic text rewrite model;
An execution module for based on given original text Inputting the character string into the trained resistance text rewrite model to generate the character string meeting the corresponding control condition/>Resistant text/>。
In another aspect of the present invention, there is also provided an electronic apparatus including:
At least one processor; and
A memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform an end-to-end unsupervised resistance text rewrite method as described above.
In another aspect of the invention, there is also provided a machine-readable storage medium storing executable instructions that when executed cause the machine to perform an end-to-end unsupervised resistance text rewrite method as described above.
Compared with the prior art, the invention has the beneficial effects that:
(1) The end-to-end unsupervised resistance text rewriting method provided by the invention can capture the characteristic distribution of the original sample data set, adaptively generate the rewriting condition for the original sample to rewrite the resistance text, and generate the natural language resistance sample in an end-to-end mode without interaction with a specific model of a downstream task. The method is significantly superior to the existing methods in the effect of generating the resistance sample and in the efficiency of model training and model application generation.
(2) The method is an unsupervised method, and the constructed resistance text rewrite model does not need to rely on a large number of parallel corpora for training, so that the method has higher use value.
(3) The method of the invention is independent of a specific deep learning model of a downstream task when training and using, and the generated samples have model-independent reusability.
Drawings
FIG. 1 is a flow chart of an end-to-end unsupervised resistance text rewrite method according to the present invention.
FIG. 2 is a schematic diagram of the training phase of the resistance text rewrite model constructed in the present invention.
FIG. 3 is a schematic diagram of the application phase of the resistance text rewrite model constructed in the present invention.
Detailed Description
The invention is further described below with reference to the drawings and examples.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
Embodiments of the invention and features of the embodiments may be combined with each other without conflict.
In order to overcome the defects in the prior art, the invention provides an end-to-end unsupervised resistance text rewriting method, which designs an end-to-end resistance text rewriting model, and comprises a rewriting condition generator and a condition text rewriter. The method includes a model training phase and an application phase.
During the training phase, based on a given raw sampleAn antagonism discriminator is introduced for providing a target-oriented antagonism sample training signal to assist in training the antagonism text rewrite model, solving the problem of the rewrite condition generator in the model lacking the supervision data. The antagonism discriminator is a target task classifier or a regressor, and can check whether the antagonism rewrite text generated by the model causes the predictive label reversal.
In the application stage, given original text is processedIn the trained resistance text rewrite model, a rewrite condition generator is used for inputting the trained resistance text rewrite model according to the original text/>Generating a text capable of guiding the overwriting of text as a resistive text/>Suitable control conditions/>; Its conditional text rewriter can be based on the original text/>And control conditions/>Ultimately generating corresponding antagonistic text/>。
The end-to-end unsupervised resistance text rewrite method and apparatus of the invention are described in detail below with reference to specific embodiments.
Example 1,
Referring to fig. 1, the embodiment provides an end-to-end unsupervised resistance text rewrite method, which includes:
s1, constructing an end-to-end resistance text rewrite model, wherein the resistance text rewrite model comprises a rewrite condition generator and a conditional text rewriter, and the rewrite condition generator is used for generating a rewrite condition according to an original text Generating control conditions/>The conditional text rewriter is used for writing/rewriting the original textAnd control conditions/>Generating corresponding rewritten text/>。
The end-to-end resistant text rewrite model constructed in this embodiment includes a rewrite condition generator and a conditional text rewriter, and preferably, the rewrite condition generator and the conditional text rewriter can each employ a multilayer bidirectional recurrent neural network, the rewrite condition generator is configured to rewrite a text according to an original textGenerating control conditions/>The conditional text rewriter is used for writing/rewriting the original textAnd control conditions/>Generating corresponding rewritten text/>。
S2, in the training stage, a given original sample is obtainedInputting the resistance text rewrite model and introducing a resistance discriminator to obtain the original sample/>Supervision tag/>Rewritten text output by the resistive text rewrite modelTag/>Tag/>And the original sample/>Supervision tag/>Inconsistent rewritten text/>Determined as an antagonistic sampleAnd uses the original sample/>And the obtained antagonistic sample/>Training the antagonistic text rewrite model.
Referring specifically to FIG. 2, the overwrite condition generator accepts the original sampleAs input, the generation can direct the conditional text rewriter to generate an antagonistic sample/>Control conditions/>. Let/>Indicating the overwrite condition generator, there is/>。
It should be understood that the control conditionsMay include a variety of content such as changing the scale of the vocabulary, sentence structure, or emotional tendency. Referring to existing resistance sample generation methods, vocabulary replacement marker sequences are used here as control conditions/>In the form of (a). Vocabulary replacement flag bit and original sample/>Binary sequences corresponding to the medium words are denoted/>And useRepresenting the generated rewritten text/>Middle/>Word of individual position/>Is replaced with other words,/>Indicating that it was not replaced.
In this embodiment, the neural network structure of the writing condition generator is a multi-layer bidirectional cyclic neural network. The implicit vector output for each time step is mapped to a scalar. The output of the rewrite condition generator is then input to the conditional text rewriter.
In definition, control conditionsThe output value of a common activation function such as Sigmoid is a continuous value between 0 and 1. To ensure the control condition/>In use rationality and stability of the model training process, a Straight-through estimator (STE-through Estimator) is preferably employed as an activation function in this embodiment, so that the control conditions generated/>In discrete binarized form, namely:
（1）；
in the formula (1): To control the condition,/> For the original sample,/>For the rewrite condition generator,/>To activate the function.
To control the generated rewritten textTraining the conditional text rewriter of the conditional constraint. The basic goal of text rewrite is to rely on the original sample/>Generating rewritten text/>, which has similar meaning but different expression forms. To control the generated rewritten text/>A conditional text rewriter model of conditional constraints is employed herein.
For a given raw sampleAnd control condition output by the overwrite condition generator/>The rewritten text generated by the conditional text rewriter/>And original sample/>Have close semantics and meet control conditions/>Control conditions specified in (2). Order theIf the conditional text rewriter is represented, there is/>。
In this embodiment, the conditional text rewriter takes the form of a multi-layer bi-directional recurrent neural network, and by pre-training it, has basic conditional text rewriteability.
The pre-training data of the conditional text rewriter may be obtained using an automatically generated approach. The input pre-training data is any original sampleControl conditions/>, obtained by randomly selecting vocabulary replacement locations. The conditional text rewriter trains the output reference rewritten text/>The replacement words may be selected for generation in a variety of ways, such as using synonym dictionary replacement or filling by a pre-trained language model BERT.
To ensure semantic similarity and syntactic correctness of the substitution, the vocabulary substitution method proposed in vocabulary combination knowledge is used to select the vocabulary to obtain the rewritten textReference rewritten text/>, as a conditional text rewriter training output。
To ensure semantic consistency of the rewritten text, only a small portion of the vocabulary is replaced in the pre-training sample. Training the conditional text rewriter with conventional negative log likelihood losses at this time would make the conditional text rewriter more prone to directly copying the input content to reduce the losses. Thus here isAnd the loss of places requiring replacement vocabulary increases with higher weight/>The following steps are:
（2）；
In the formula (2): Representing a loss function,/> Represents weights and/>，/>Representing the rewritten text output by the conditional text rewriter,/>Representing the reference rewritten text,/>Representing the original sample,/>Representing control conditions,/>Representing the rewritten text/>, given the original sample and the control conditions, of the conditional text rewriter outputAnd reference rewritten text/>First/>The probabilities of words being identical for each position.
Further, in order to generate the resistance sample, a resistance discriminator is introduced by referencing the design of the resistance generation network. In order to train the above-mentioned resistance text rewrite model, a resistance discriminator is used in the present embodiment to provide a training signal, and is noted as. The antagonism discriminator being a classifier or regressor trained on the target task, i.eWherein/>The input samples and sample labels respectively representing the target task are the original samples and the corresponding supervision labels of the embodiment.
Theoretically, the conditional text rewriter should generate a sample with the original sampleText that is semantically close. If the antagonism discriminator/>In the generated rewritten sample/>Tag of upper prediction output/>And original sample/>Corresponding obtained supervision Label/>Inconsistent, indicate that the sample is rewritten/>Can be considered as potential challenge samples/>Otherwise, it is not an challenge sample.
Based on this, an antagonism discriminator is introduced in the present embodiment for providing a target-oriented antagonism sample training signal to assist in training the antagonism text rewrite model.
The core of the contrast text rewrite model is to generate proper control conditions. The challenge arbiter can be regarded as an attacked model in the form of a white box, so by using techniques such as gummel softmax, gradients can be transferred to the previous neural network structure, providing training signals. This also allows the entire resistance text rewrite model to be trained in an end-to-end fashion.
In the training process, the rewrite condition generator, the condition text rewriter and the antagonism discriminator are connected in series in a pipeline form.
For a given raw sampleThe whole neural network structure will give the predictive label/>, on the rewritten perturbation sample. Due to conditional text rewriter/>The output is in the form of a lexical probability sequence, where the probability is converted into a word vector form that can be directly input into the resistance discriminator using Gumbel softmax technique, namely:
（3）；
In the formula (3): To control the condition,/> For the rewrite condition generator,/>Is a conditional text rewriter,/>For the original sample,/>For the generated rewritten samples,/>For the antagonism discriminator,/>To rewrite the sample/>A corresponding tag.
During training, the parameters of the resistance discriminators are frozen and the parameters of the conditions generator and the conditions text rewriter are updated. During training, the rewrite condition generator can learn the data characteristics of the target task, identify the weak vocabulary position in the sample, and adaptively generate the control condition to guide the condition text rewrite device to rewrite. Meanwhile, parameters of the conditional text rewriter are updated during training, so that the conditional text rewriter can be more suitable for target tasks and requirements of resistance rewriting when selecting words.
In this embodiment, to train the resistive text rewrite model, the following penalty function is used:
1) Countering losses: the main goal of the resistance text rewrite model is to generate resistance samples that enable the resistance arbiter to predict the occurrence of a flip. The correlation work in terms of existing model distillation indicates the validity and simplicity of the mean square error loss (Mean squared error, MSE), which is therefore chosen here as a loss function that measures whether the tag is inverted or not:
（4）；
In the formula (4): Representing challenge losses,/> Representing the correspondence of the resistance discriminator to the rewritten text/>Predicted output of/>Representing the correspondence of the antagonism discriminator to the original sample/>Is provided.
It should be noted that a more complex function for measuring the distance between probabilities, such as KL divergence, may also be selected, and will not be further described herein.
2) Disturbance proportional loss: to ensure the rewritten text and the original sampleThe number of rewritten words in the text should be limited. Namely, control conditions generated by the overwrite condition generator are limited/>In/>Is used to determine the number of positions of the object. Let/>Representing the expected modification ratio,/>For the sample length, the disturbance proportional loss is:
（5）；
In formula (5): representing disturbance proportional loss,/> Representing the expected modification ratio,/>For the sample length,/>Representing control conditions/>Middle/>Is used to determine the number of positions of the object.
3) Conditional text rewrite penalty: the conditional text rewriter is trained by multiple penalty functions to ensure that the text it produces meets the requirements of a given condition. For control conditionsThe vocabulary that is not changed (i.e./>The position of (2) is denoted as/>) Reconstruction losses are designed. The loss is the negative log likelihood of the output probability of the corresponding vocabulary model in the original sentence, so that the text rewriter can reconstruct the original sample/>Namely, there are:
（6）；
In formula (6): Representing conditional constraint text rewrite penalty,/> Rewritten text/>, representing conditional text rewriter outputAnd original sample/>First/>Words of the individual positions are identical,/>Representing the probability.
4) Forced rewrite constraints: for control conditionsThe probability of the original vocabulary should be lower than the other vocabulary to cause the text rewriter to make changes to it. Inspired by the loss function for the negative sample in the noise contrast estimation method, the following loss function is proposed to penalize the case where the text rewriter outputs the original vocabulary. Let/>Representing control conditions/>The position set of 1 in (2) is:
（7）；
in the formula (7): Representing forced overwrite loss,/> The sigmod functions are represented.
5) Semantic similarity constraint: to ensure that the generated rewritten textAnd original sample/>The present embodiment uses a vocabulary level semantic similarity penalty. The loss is calculated by calculating the cosine similarity of the word vectors of the generated words and the original words, namely:
（8）；
In formula (8): Representing semantic similarity loss.
6) The syntactic rationality constraint of the vocabulary is rewritten: word vector similarity does not guarantee that the replacement vocabulary has the correct syntactic function. Therefore, the grammar rationality estimation based on the vocabulary combination knowledge is further adopted to carry out joint verification on vocabulary semantics and grammar rationality of the generated samples, and a training signal is provided.
Given two vocabularies and dependency syntactic relation between them, a triplet is formedVocabulary combination knowledge can estimate the rationality of the combination triplet, use/>Indicating rationality. Let/>Representing the original sample/>Is a dependency syntax tree structure of (1), wherein/>Representing the original sample/>Middle/>Words of individual locations and the/>Words in individual locations have a dependency relationship/>. The syntax correctness loss calculation method is as follows:
（9）；
In the formula (9): representing a loss of syntax correctness.
Based on the above-described loss function, the overall training loss function of the resistive text rewrite model in this embodiment is a weighted sum of the above-described loss functions:
（10）；
in the formula (10): training loss function representing an antagonistic text rewrite model,/> Representing challenge loss/>Weights of/>Representing disturbance proportional loss/>Weights of/>Representing conditional constraint text rewrite loss/>Weights of/>Represents forced overwrite loss/>Weights of/>Representing semantic similarity loss/>Weights of/>Representing loss of syntax correctness/>Is a weight of (2).
Based on the above process, training of the resistance text rewrite model is completed.
S3, in the application stage, the given original text is processedInputting the trained antagonistic text rewrite model to generate a text rewrite model meeting the corresponding control conditions/>Resistant text/>。
With specific reference to FIG. 3, i.e., during the actual application phase, a given original text may be directly renderedInputting the control conditions into a trained resistance text rewrite model, and generating corresponding control conditions/>, based on a rewrite condition generatorBased on the condition text rewriter, according to the given original text/>And control conditions generated/>Finally generates the control conditions meeting the corresponding control conditionsResistant text/>。
Application example 1,
In order to verify the technical advantages of the method in this embodiment, the disclosed natural language processing task IMDB is adopted, and the BERT model is used as a downstream classification model to verify the validity of the natural language resistance sample generated by the method in this embodiment.
The evaluation index mainly comprises the following aspects:
To evaluate the effect of the generated challenge sample on the downstream model of the attack, three indicators are calculated, wherein the original accuracy represents the accuracy of the downstream model on the original test set; the accuracy after attack represents the accuracy of the downstream model on the antagonism sample generated on the basis of the original test set; the attack effect is the difference value of the two, which indicates the attack effect of the method.
To evaluate the efficiency of execution of the present embodiment method, the time required for the present embodiment method to generate an challenge sample was recorded. The method is an A2T method proposed in literature Towards improving ADVERSARIAL TRAINING of NLP models, a mode of interacting with a downstream model is adopted, important vocabularies in the sample are distinguished through gradient information of the model after the sample is input, and the vocabularies are replaced by using a pre-training language model BERT, so that an opposite sample is obtained. The results of the verification are shown in table 1 below.
Table 1: verification result
From the above verification results, it can be seen that the challenge sample generated by the method of the present embodiment can effectively reduce the performance of the downstream model, and the generation efficiency is far higher than that of the existing comparison method A2T.
Application example 2,
One possible example of user input is raw text= "Boy is riding a yellow bike wearing a yellow shirt". Original text/>Inputting the control conditions into a trained resistance text rewrite model, and generating control conditions/>=/>Original text/>And control conditions/>Input to a conditional text rewriter to finally generate an antagonistic sample/>= "Boy is wearing a red shirt to play a blue bike".
EXAMPLE 2,
The embodiment provides a device for realizing an end-to-end unsupervised resistance text rewriting method, which comprises the following steps:
A building module for building an end-to-end resistance text rewrite model, wherein the resistance text rewrite model comprises a rewrite condition generator and a condition text rewriter, and the rewrite condition generator is used for generating a rewrite condition according to an original text Generating control conditions/>The conditional text rewriter is used for writing/rewriting the original textAnd control conditions/>Generating corresponding rewritten text/>；
Training module for based on given original sampleInputting the original sample into the resistance text rewrite model, and introducing a resistance discriminator to obtain the original sample/>Supervision tag/>Rewritten text/>, output by the resistive text rewrite modelTag/>Tag/>And the original sample/>Supervision tag/>Inconsistent rewritten text/>Determined as resistant sample/>And uses the original sample/>And the obtained antagonistic sample/>Training the antagonistic text rewrite model;
An execution module for based on given original text Will give the original text/>Inputting the trained antagonistic text rewrite model to generate a text rewrite model meeting the corresponding control conditions/>Resistant text/>。
EXAMPLE 3,
The embodiment also provides an electronic device, including:
At least one processor; and
A memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform an end-to-end unsupervised resistance text rewrite method as described above.
In this embodiment, the electronic device may include, but is not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smart phones, tablet computers, cellular phones, personal Digital Assistants (PDAs), handsets, messaging devices, wearable computing devices, consumer electronic devices, and the like.
EXAMPLE 4,
The present embodiment also provides a machine-readable storage medium storing executable instructions that, when executed, cause the machine to perform an end-to-end unsupervised method of resistance text rewrite as described above.
In particular, a system or apparatus provided with a readable storage medium having stored thereon software program code implementing the functions of any of the above embodiments may be provided, and a computer or processor of the system or apparatus may be caused to read out and execute instructions stored in the readable storage medium.
In this case, the program code itself read from the readable medium may implement the functions of any of the above embodiments, and thus the machine-readable code and the readable storage medium storing the machine-readable code form part of the present specification.
Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-R, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs), magnetic tapes, nonvolatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or cloud by a communications network.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be understood that the foregoing examples of the present invention are merely illustrative of the present invention and are not intended to limit the present invention to the specific embodiments thereof. Any modification, equivalent replacement, improvement, etc. that comes within the spirit and principle of the claims of the present invention should be included in the protection scope of the claims of the present invention.
Claims (10)
1. An end-to-end unsupervised resistance text rewrite method, said method comprising:
s1, constructing an end-to-end resistance text rewrite model, wherein the resistance text rewrite model comprises a rewrite condition generator and a conditional text rewriter, and the rewrite condition generator is used for generating a rewrite condition according to an original text Generating control conditions/>The conditional text rewriter is used for writing/rewriting the original textAnd control conditions/>Generating corresponding rewritten text/>；
S2, in the training stage, a given original sample is obtainedInputting the resistance text rewrite model and introducing a resistance discriminator to obtain the original sample/>Supervision tag/>Rewritten text/>, output by the resistive text rewrite modelTag/>Tag/>And the original sample/>Supervision tag/>Inconsistent rewritten text/>Determined as resistant sample/>And uses the original sample/>And the obtained antagonistic sample/>Training the antagonistic text rewrite model;
S3, in the application stage, the given original text is processed Inputting the trained antagonistic text rewrite model to generate a text rewrite model meeting the corresponding control conditions/>Resistant text/>。
2. The end-to-end unsupervised resistance text rewrite method according to claim 1, wherein in said step S2, said control condition is thatUsing the form of vocabulary replacement marker sequence, vocabulary replacement marker bit and original sample/>Binary sequences corresponding to the medium words are denoted/>And use/>Representing the generated rewritten text/>Middle/>Word of individual position/>Is replaced with other words,/>Indicating that it was not replaced.
3. The end-to-end unsupervised resistive text rewrite method according to claim 1, wherein in said step S2, said rewrite condition generator employs a pass-through estimator as an activation function to cause a generated control conditionIn discrete binarized form, i.e.:
（1）；
in the formula (1): To control the condition,/> For the original sample,/>For the rewrite condition generator,/>To activate the function.
4. The end-to-end unsupervised resistance text rewrite method according to claim 2, wherein said step S2 specifically further comprises:
selecting an arbitrary original sample Control conditions/>, obtained by randomly selecting vocabulary replacement locationsThe conditional text rewriter is pre-trained with a training loss function of:
（2）；
In the formula (2): Representing a loss function,/> Represents weights and/>，/>Representing the rewritten text output by the conditional text rewriter,/>Representing the reference rewritten text,/>Representing the original sample,/>Representing control conditions,/>Representing the rewritten text/>, given the original sample and the control conditions, of the conditional text rewriter outputWritten text with referenceFirst/>The probabilities of words being identical for each position.
5. The end-to-end unsupervised resistance text rewrite method according to claim 1, wherein in said step S2, a given original sample is subjected toInputting the resistance text rewrite model and introducing a resistance discriminator to obtain the original sample/>Supervision tag/>Rewritten text/>, output by the resistive text rewrite modelTag/>The method specifically comprises the following steps:
（3）；
In the formula (3): To control the condition,/> For the rewrite condition generator,/>Is a conditional text rewriter,/>For the original sample,/>For the generated rewritten samples,/>For the antagonism discriminator,/>To rewrite the sample/>A corresponding tag.
6. The end-to-end unsupervised resistance text rewrite method according to claim 2, wherein in said step S2, the training penalty function of the resistance text rewrite model includes the following penalty functions:
1) Countering loss function:
（4）；
In the formula (4): Representing challenge losses,/> Representing the correspondence of the resistance discriminator to the rewritten text/>Is provided with a prediction output of (c) for the prediction,Representing the correspondence of the antagonism discriminator to the original sample/>An output of (2);
2) Disturbance proportional loss function:
（5）；
In formula (5): representing disturbance proportional loss,/> Representing the expected modification ratio,/>For the sample length,/>Representing control conditions/>Middle/>Is a number of positions of (a);
3) Conditional constraint text overwrite loss function:
（6）；
In formula (6): Representing conditional constraint text rewrite penalty,/> Rewritten text/>, representing conditional text rewriter outputAnd original sample/>First/>Words of the individual positions are identical,/>Representing the probability;
4) Forced overwrite loss function:
（7）；
in the formula (7): Representing forced overwrite loss,/> Representing sigmod functions;
5) Semantic similarity loss function:
（8）；
In formula (8): Representing semantic similarity loss;
6) Syntax correctness loss function:
（9）；
In the formula (9): Representing loss of syntax correctness,/> Representing the original sample/>Is a dependency syntax tree structure of (1), wherein/>Representing the original sample/>Middle/>Words of individual locations and the/>Words in individual locations have a dependency relationship/>，Representing rewritten text/>Middle/>Words of individual locations and the/>Words in individual locations have a dependency relationship/>Is reasonable.
7. The end-to-end unsupervised resistance text rewrite method according to claim 6, wherein in said step S2, a training loss function of said resistance text rewrite model is:
（10）；
in the formula (10): training loss function representing an antagonistic text rewrite model,/> Representing challenge loss/>Weights of/>Representing disturbance proportional loss/>Weights of/>Representing conditional constraint text rewrite loss/>Weights of/>Represents forced overwrite loss/>Weights of/>Representing semantic similarity loss/>Weights of/>Representing loss of syntax correctness/>Is a weight of (2).
8. An apparatus for implementing an end-to-end unsupervised resistance text rewrite method, said apparatus comprising:
A building module for building an end-to-end resistance text rewrite model, wherein the resistance text rewrite model comprises a rewrite condition generator and a condition text rewriter, and the rewrite condition generator is used for generating a rewrite condition according to an original text Generating control conditions/>The conditional text rewriter is used for writing/rewriting the original textAnd control conditions/>Generating corresponding rewritten text/>；
Training module for based on given original sampleInputting the original sample into the resistance text rewrite model, and introducing a resistance discriminator to obtain the original sample/>Supervision tag/>Rewritten text/>, output by the resistive text rewrite modelTag/>Tag/>And the original sample/>Supervision tag/>Inconsistent rewritten text/>Determined as resistant sample/>And uses the original sample/>And the obtained antagonistic sample/>Training the antagonistic text rewrite model;
An execution module for based on given original text Inputting the character string into the trained resistance text rewrite model to generate the character string meeting the corresponding control condition/>Resistant text/>。
9. An electronic device, comprising:
At least one processor; and
A memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform the end-to-end unsupervised resistance text rewrite method according to any one of claims 1 to 7.
10. A machine-readable storage medium storing executable instructions that, when executed, cause the machine to perform the end-to-end unsupervised resistance text rewrite method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410323254.0A CN117933268A (en) | 2024-03-21 | 2024-03-21 | End-to-end unsupervised resistance text rewriting method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410323254.0A CN117933268A (en) | 2024-03-21 | 2024-03-21 | End-to-end unsupervised resistance text rewriting method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117933268A true CN117933268A (en) | 2024-04-26 |
Family
ID=90757843
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410323254.0A Pending CN117933268A (en) | 2024-03-21 | 2024-03-21 | End-to-end unsupervised resistance text rewriting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117933268A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112966112A (en) * | 2021-03-25 | 2021-06-15 | 支付宝(杭州)信息技术有限公司 | Text classification model training and text classification method and device based on counterstudy |
| WO2021174827A1 (en) * | 2020-03-02 | 2021-09-10 | 平安科技（深圳）有限公司 | Text generation method and appartus, computer device and readable storage medium |
| CN114004283A (en) * | 2021-10-12 | 2022-02-01 | 清华大学 | Text anti-attack method, device, equipment and storage medium |
| CN114757188A (en) * | 2022-05-20 | 2022-07-15 | 大连大学 | Standard medical text rewriting method based on generation of confrontation network |
| CN114781386A (en) * | 2022-05-17 | 2022-07-22 | 北京百度网讯科技有限公司 | Method and device for acquiring text error correction training corpus and electronic equipment |
| CN116108830A (en) * | 2023-03-30 | 2023-05-12 | 山东大学 | Syntax-controllable text rewriting method and device |
| US20230367976A1 (en) * | 2022-05-12 | 2023-11-16 | Alipay (Hangzhou) Information Technology Co., Ltd. | Model training methods and apparatuses, text processing methods and apparatuses, and computer devices |
-
2024
- 2024-03-21 CN CN202410323254.0A patent/CN117933268A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021174827A1 (en) * | 2020-03-02 | 2021-09-10 | 平安科技（深圳）有限公司 | Text generation method and appartus, computer device and readable storage medium |
| CN112966112A (en) * | 2021-03-25 | 2021-06-15 | 支付宝(杭州)信息技术有限公司 | Text classification model training and text classification method and device based on counterstudy |
| CN114004283A (en) * | 2021-10-12 | 2022-02-01 | 清华大学 | Text anti-attack method, device, equipment and storage medium |
| US20230367976A1 (en) * | 2022-05-12 | 2023-11-16 | Alipay (Hangzhou) Information Technology Co., Ltd. | Model training methods and apparatuses, text processing methods and apparatuses, and computer devices |
| CN114781386A (en) * | 2022-05-17 | 2022-07-22 | 北京百度网讯科技有限公司 | Method and device for acquiring text error correction training corpus and electronic equipment |
| CN114757188A (en) * | 2022-05-20 | 2022-07-15 | 大连大学 | Standard medical text rewriting method based on generation of confrontation network |
| CN116108830A (en) * | 2023-03-30 | 2023-05-12 | 山东大学 | Syntax-controllable text rewriting method and device |
Non-Patent Citations (3)
| Title |
|---|
| ANMOL GEOL等: "Socartic reasoning improves positive text rewriting", 《ARXIV》, 5 March 2024 (2024-03-05) * |
| TIANYUAN LIU等: "End-to-end Adversarial Sample Generation for Data Augmentation", 《FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS:EMNLP 2023》, 31 December 2023 (2023-12-31), pages 11359 - 11368 * |
| 张晓辉;于双元;王全新;徐保民;: "基于对抗训练的文本表示和分类算法", 计算机科学, no. 1, 15 June 2020 (2020-06-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Eger et al. | Neural end-to-end learning for computational argumentation mining | |
| CN112528034B (en) | Knowledge distillation-based entity relationship extraction method | |
| US20220237378A1 (en) | System and method for natural language processing with pretrained language models | |
| Rahman et al. | A neural network based intelligent support model for program code completion | |
| CN112183094B (en) | Chinese grammar debugging method and system based on multiple text features | |
| Song | Intelligent English translation system based on evolutionary multi-objective optimization algorithm | |
| Wåreus et al. | Automated CPE labeling of CVE summaries with machine learning | |
| CN115357719B (en) | Power audit text classification method and device based on improved BERT model | |
| CN111930939A (en) | Text detection method and device | |
| CN110334186A (en) | Data query method, apparatus, computer equipment and computer readable storage medium | |
| CN116661805B (en) | Code representation generation method and device, storage medium and electronic equipment | |
| Bonadiman et al. | Deep neural networks for named entity recognition in Italian | |
| CN112818698A (en) | Fine-grained user comment sentiment analysis method based on dual-channel model | |
| CN114818717A (en) | Chinese named entity recognition method and system fusing vocabulary and syntax information | |
| Wang et al. | Word vector modeling for sentiment analysis of product reviews | |
| CN113111184B (en) | Event detection method based on explicit event structure knowledge enhancement and terminal equipment | |
| Tran et al. | Can cross-domain term extraction benefit from cross-lingual transfer? | |
| Gao et al. | Chinese causal event extraction using causality‐associated graph neural network | |
| CN116187328A (en) | Method and device for generating counterfactual sample for entity and relation extraction task | |
| Huang et al. | Prompt-based self-training framework for few-shot named entity recognition | |
| CN117933268A (en) | End-to-end unsupervised resistance text rewriting method and device | |
| Li et al. | Word segmentation and morphological parsing for sanskrit | |
| Higashiyama et al. | Auxiliary lexicon word prediction for cross-domain word segmentation | |
| Priya et al. | Context aware joint modeling of domain classification, intent detection and slot filling with zero-shot intent detection approach | |
| Wu et al. | Analyzing the Application of Multimedia Technology Assisted English Grammar Teaching in Colleges |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |