CN111444346B - Word vector confrontation sample generation method and device for text classification - Google Patents
Word vector confrontation sample generation method and device for text classification Download PDFInfo
- Publication number
- CN111444346B CN111444346B CN202010248226.9A CN202010248226A CN111444346B CN 111444346 B CN111444346 B CN 111444346B CN 202010248226 A CN202010248226 A CN 202010248226A CN 111444346 B CN111444346 B CN 111444346B
- Authority
- CN
- China
- Prior art keywords
- word
- text
- neural network
- network model
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a method and a device for generating word vector confrontation samples for text classification, wherein the method comprises the following steps: initializing English texts needing text classification and embedding words, and converting the English texts into corresponding vector representations; repeatedly performing partial derivative operation on word vectors of the English text according to the loss function until the classification result output by the neural network model is wrong; based on the modified word and word vectors, selecting the word closest to the modified word vectors in the space by adopting an Euclidean distance formula, and constructing an attack alternative word set; and carrying out random replacement on the words of the English text according to the attack substitute word set to generate a confrontation sample. The invention can effectively generate the word vector countermeasure sample aiming at the text classification, makes the neural network text classifier identify errors on the premise of ensuring that the semantics are not changed, the word vector countermeasure sample is not perceived by people and the identification and classification of the text by people are not influenced, and ensures that the countermeasure sample does not generate illegal characters while reducing the neural network identification probability.
Description
Technical Field
The invention relates to the technical field of natural language processing in computer science, in particular to a word vector confrontation sample generation method and device for text classification.
Background
Natural Language Processing (NLP) is a key field of current computer science research, and text classification is an important branch of the NLP field and is mainly used for intelligently classifying a large number of unknown types of text information into correct types. With the arrival of the Internet age, the digital text information exponentially increases, classification of texts cannot only depend on manual operation, and the texts can be rapidly and automatically classified by means of a neural network-based classification method, so that the method is higher in classification accuracy than manual classification, and a large amount of manpower and material resources can be saved.
The conventional text classification method in the prior art is mainly a neural network classification method, and although the accuracy of the neural network classification method is greatly improved compared with the conventional classification method, the neural network-based classification method has some safety problems because of the inexplicability of the neural network: a small amount of modification (imperceptible to the human eye) to the original input can have serious consequences for a false output of the classifier. Therefore, the attack method aiming at the neural network immediately leads to the heat of academic research, and most of the attack methods surround how to generate the countermeasure sample quickly and effectively. Generally, a countermeasure sample slightly different from the original sample is generated through technical means, the countermeasure sample can make the classifier identification wrong, is not easy to be detected by human eyes, and does not affect the classification identification of people. At present, the existing countermeasure sample generation technology proposes that a Fast Gradient Sign Method (FGSM for short) is used to perform fine disturbance on an image to generate a countermeasure sample, so that a high-precision neural network classifier makes an error in identification. And the NLP field scholars provide an attack method applicable to the text field by taking the existing image countermeasure sample generation method as reference. The question-answering system is deceived by adding words or sentences which do not affect human comprehension to the original text, so as to achieve the aim of obtaining wrong answers.
However, in the course of research and practice on the prior art, the inventors of the present invention found that the fast gradient sign method is applied to the image field, and the attack method FGSM generates countermeasure samples using gradients, which are difficult to be perceived by human eyes because of continuity of images although noise is generated. The biggest difference between text and image is that the text is discrete, and slight changes are easily perceived by human eyes, so the FGSM method is not suitable for the field of text classification. The second technique of the above techniques is applied to the text field, but because the attack method only uses a small amount of original model information, the generation time of the countercheck sample is slow, and the algorithm efficiency is low. The attack method has a large modification degree on the original sample, is easy to be perceived by people, so that the imperceptible attack cannot be perfectly realized, and meanwhile, the attack method is mainly applied to a question-answering system, and cannot realize effective attack in a text classification application scene. Therefore, a method for generating a word vector countermeasure sample for text classification is needed, which can be applied to a large number of text classification scenarios to realize effective attacks.
Disclosure of Invention
The technical problem to be solved by the embodiments of the present invention is to provide a method and an apparatus for generating word vector countermeasure samples for text classification, so as to effectively generate word vector countermeasure samples for text classification, and make a neural network text classifier identify errors.
To solve the above problem, an embodiment of the present invention provides a method for generating word vector confrontation samples for text classification, which at least includes the following steps:
initializing an English text needing text classification, performing word embedding on the English text, and converting the English text into corresponding vector representation;
repeatedly performing partial derivative operation on word vectors in the English text according to the loss function until the classification result output by the neural network model is wrong;
based on the modified word and word vectors, selecting the word closest to the modified word vectors in the space by adopting an Euclidean distance formula, and constructing an attack alternative word set;
and randomly replacing the words of the English text according to the attack substitute word set to generate a confrontation sample.
Further, the method for generating word vector confrontation samples for text classification further comprises the following steps:
designing a neural network model for classifying texts; the neural network model comprises an input layer, a hidden layer and an output layer;
converting the training text into word vector representation, inputting the word vector representation to the neural network model for training to obtain an output result of the neural network model;
and correcting parameters of the neural network model according to the output result of the neural network model and the correct category of the current training text, and fixing the parameters of the neural network model after training is finished.
Further, the word embedding is performed on the english text, and the english text is converted into a corresponding vector representation, specifically:
word2vec word embedding is carried out on each word in the English text, and each word is converted into a word vector with a fixed length of m;
representing the English text as a two-dimensional matrix of n x m; wherein n is the total number of words in the English text, and m is a preset fixed length.
Further, the above repeatedly performing partial derivative operation on the word vectors in the english text according to the loss function until the classification result output by the neural network model is wrong, specifically:
performing partial derivative operation on the word vectors in the English text according to the loss function to obtain the forward change rate of the loss function along each dimension of the input word vectors;
modifying each dimension of the input word vector according to the forward change rate to maximize a loss function in a constraint range;
and repeating the steps to modify a plurality of words in the English text until the classification result output by the neural network model is wrong.
One embodiment of the present invention provides a word vector confrontation sample generation apparatus for text classification, including:
the word embedding module is used for initializing the English text needing text classification, embedding words in the English text and converting the words into corresponding vector representation;
the word vector modification module is used for repeatedly carrying out partial derivative operation on word vectors in the English text according to the loss function until the classification result output by the neural network model is wrong;
the attack substitute word set module is used for selecting the word closest to the modified word vector in the space by adopting an Euclidean distance formula based on the modified word vector to construct an attack substitute word set;
and the countermeasure sample module is used for randomly replacing the words of the English text according to the attack substitute word set to generate a countermeasure sample.
Further, the word vector confrontation sample generation device for text classification further includes:
the neural network model module is used for designing a neural network model for classifying texts; the neural network model comprises an input layer, a hidden layer and an output layer;
the training module is used for converting a training text into a word vector representation and inputting the word vector representation to the neural network model for training to obtain an output result of the neural network model;
and the parameter correction module is used for correcting the parameters of the neural network model according to the output result of the neural network model and the correct type of the current training text, and fixing the parameters of the neural network model after the training is finished.
Further, the word embedding module specifically includes:
word2vec word embedding is carried out on each word in the English text, and each word is converted into a word vector with a fixed length of m;
representing the English text as a two-dimensional matrix of n x m; wherein n is the total number of words in the English text, and m is a preset fixed length.
Further, the word vector modification module specifically includes:
performing partial derivative operation on word vectors in the English text according to the loss function to obtain the forward change rate of the loss function along each dimension of the input word vectors;
modifying each dimension of the input word vector according to the forward change rate to maximize a loss function in a constraint range;
and repeating the steps to modify a plurality of words in the English text until the classification result output by the neural network model is wrong.
An embodiment of the present invention further provides a terminal device for generating word vector countermeasure samples for text classification, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the word vector countermeasure sample generation method for text classification as described above.
An embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program runs, a device on which the computer-readable storage medium is located is controlled to execute the word vector confrontation sample generation method for text classification as described above.
The embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides a method and a device for generating word vector confrontation samples for text classification, wherein the method comprises the following steps: initializing an English text needing text classification, performing word embedding on the English text, and converting the English text into corresponding vector representation; repeatedly performing partial derivative operation on word vectors in the English text according to the loss function until the classification result output by the neural network model is wrong; based on the modified word vector, selecting the word closest to the modified word vector in the space by adopting an Euclidean distance formula, and constructing an attack alternative word set; and randomly replacing the words of the English text according to the attack substitute word set to generate a confrontation sample. The invention can effectively generate the word vector countermeasure sample aiming at the text classification by utilizing a small amount of knowledge of the original classification model, makes the recognition of the neural network text classifier have errors on the premise of ensuring that the semantics is not changed, the word vector countermeasure sample is not perceived by people and the recognition and classification of the text by the human are not influenced, selects the most appropriate substitute word by using the Euclidean distance approximation word vector method, and ensures that the countermeasure sample does not generate illegal characters while reducing the recognition probability of the neural network.
Drawings
Fig. 1 is a flowchart illustrating a method for generating a word vector confrontation sample for text classification according to a first embodiment of the present invention;
fig. 2 is a flowchart illustrating another method for generating a word vector confrontation sample for text classification according to a first embodiment of the present invention;
fig. 3 is a schematic structural diagram of a word vector confrontation sample generation apparatus for text classification according to a second embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
In the description of the present application, it is to be understood that the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless otherwise specified.
First, an application scenario that can be provided by the present invention is described, such as generating a word vector confrontation sample for text classification.
The first embodiment of the present invention:
please refer to fig. 1-2.
As shown in fig. 1, the present embodiment provides a method for generating word vector confrontation samples for text classification, which at least includes the following steps:
s101, initializing the English text needing text classification, embedding words in the English text, and converting the words into corresponding vector representations.
In a preferred embodiment, the word embedding is performed on the english text, and the english text is converted into a corresponding vector representation, specifically:
word2vec word embedding is carried out on each word in the English text, and each word is converted into a word vector with a fixed length of m;
representing the English text as a two-dimensional matrix of n x m; wherein n is the total number of words in the English text, and m is a preset fixed length.
Specifically, in step S101, since the text sequences are discrete special characters and cannot be directly processed by the computer, word embedding is performed on the text sequences first to convert the text sequences into vector representations. Assuming that each english text can be represented as [ X1, X2, ..., xn ], there are n words, and the word Xi is converted into a word vector with a fixed length of m through word2vec word embedding, i.e., xi = [ Xi1, xi2, ..., xim ], so each text is represented as a two-dimensional matrix of n × m. It should be noted that, in the method for generating a word vector attack sample for text classification, word embedding methods include, but are not limited to, word2vec word embedding methods.
And S102, repeatedly performing partial derivative operation on word vectors in the English text according to the loss function until the classification result output by the neural network model is wrong.
In a preferred embodiment, the repeatedly performing partial derivative operation on the word vectors in the english text according to the loss function until the classification result output by the neural network model is wrong specifically:
performing partial derivative operation on word vectors in the English text according to the loss function to obtain the forward change rate of the loss function along each dimension of the input word vectors;
modifying each dimension of the input word vector according to the forward change rate to maximize a loss function in a constraint range;
and repeating the steps to modify a plurality of words in the English text until the classification result output by the neural network model is wrong.
Specifically, for step S102, the loss function performs a partial derivative operation on the word vector, and since the partial derivative reflects a change rate of the target function (i.e., the loss function) along a positive direction of a coordinate axis, in this embodiment, the loss function is used to derive the input word Xi to obtain a positive change rate of the loss function along each dimension (Xi 1, xi2, ..., xim) of the input Xi, and each dimension of the input Xi is changed according to the change rate to maximize the loss function (within a constraint rule), and the above operation is repeatedly performed to modify the plurality of words until the classification of the neural network model is incorrect. For example, the output result of the model is (0.3, 0.7), and the english text X is classified as bad comment, and the classification of the neural network model is determined to be incorrect when the english text X is inconsistent with the original classification. By maximizing the loss function, the model is classified or identified as erroneous with the maximum probability for the modified input samples.
S103, based on the modified word and word vectors, selecting the word closest to the modified word vectors in the space by adopting an Euclidean distance formula, and constructing an attack substitute word set.
Specifically, for step S103, the two-dimensional representation of the english text X is changed through the previous step S102, the matrix value is changed, the matrix dimension is not changed, and the input word vector of each word is modified to [ X ' i1, X ' i2, \ 8230;, X ' im ]. However, the inventor of the present invention considers that [ X ' i1, X ' i2, ..., [ X ' im ] is likely not present in the mapping space of word2vec, meaning that the word vector cannot find the corresponding word in the real space, so the metric of Euclidean distance is adopted:
and selecting a word vector [ X 'i1, X' i2, \ 8230;, X 'im ] closest to the [ X' i1, X 'i2, \ 8230;, X' im ], wherein a word corresponding to the word vector is X 'i, so as to obtain the corresponding relation of Xi → X' i. Repeatedly searching a plurality of pairs Xi → X' i, thereby constructing an attack alternative word set. A distance approximation method is adopted to select legal words for replacement, so that semantic information of a text is not influenced to the maximum extent, human recognition is not influenced to the maximum extent, and generation of replacement words is not influenced, and meanwhile, the countermeasure samples are guaranteed not to have illegal characters while the recognition probability of a neural network is reduced.
It should be noted that, the distance manner used for measuring the distance between word vectors in the word vector attack sample generation method for text classification includes, but is not limited to, euclidean distance.
And S104, randomly replacing the words of the English text according to the attack surrogate word set to generate a confrontation sample.
Specifically, for step S104, for each piece of test text data, words of the text are randomly replaced according to the existing attack replacement word set, and the replaced text is the countermeasure sample.
In a specific embodiment, the countermeasure sample generation of the movie text comment data is taken as an example.
First, a movie comment text X is optionally selected, and assuming that the text is 5[ 2], [ X1, X2, X3, X4, X5], each word is converted into a word vector of length 3 by word2vec word embedding, and the text X is converted into a two-dimensional matrix: [ [0.3,0.5,0.2], [0.6,0.9,0.1], [0.5,0.4,0.6], [0.8,0.9,0.1], [0.5,0.6,0.8] ];
modifying the input word vector using a loss function, the modified word vector being represented as:
[[0.3,0.5,0.2],[0.3,0.5,0.8],[0.5,0.4,0.6],[0.7,0.8,0.5],[0.5,0.6,0.8]]；
obviously, only the embedded word vectors of the words X2 and X4 have changed. However, considering that the modified word vectors [0.3,0.5,0.8] and [0.7,0.8,0.5] may not have mapping objects in the embedding space, the modified word vectors are approximated by using the euclidean distance as a standard, and finally, the modified word vectors are selected until [0.4,0.5,0.8] (corresponding to the word X '2) is closest to [0.3,0.5,0.8], and [0.7,0.8,0.6] (corresponding to the word X' 4) is closest to [0.7,0.8,0.5], and the attack surrogate word set is collected into two pairs of replacement words.
And (4) reselecting a movie comment text Y, and replacing X2 with X'2 to generate a confrontation sample on the assumption that a word X2 exists in the text Y.
In a preferred embodiment, as shown in fig. 2, the method for generating word vector confrontation samples for text classification further includes:
designing a neural network model for classifying texts; the neural network model comprises an input layer, a hidden layer and an output layer;
converting the training text into word vector representation, inputting the word vector representation to the neural network model for training to obtain an output result of the neural network model;
and correcting parameters of the neural network model according to the output result of the neural network model and the correct category of the current training text, and fixing the parameters of the neural network model after training.
Specifically, a neural network model M is designed to classify the text. The structure of the neural network can be roughly divided into an input layer, a hidden layer and an output layer, wherein the node number of the input layer of the neural network corresponds to the dimension of an input word vector; the number of layers of the hidden layer and the number of nodes of each layer can be set arbitrarily; the number of output layer nodes corresponds to the number of categories. For example, in the movie comment data, the number of nodes in the output layer is 2, and the probability that each point of output text is recognized as a certain category is denoted by (a 1, a 2), where a1 denotes the probability of being classified as good comment and a2 denotes the probability of being classified as bad comment, and it is obvious that a1+ a2=1 can be observed.
The training text is converted into word vector representation and then placed into a neural network model for training, network parameters are corrected according to the model output result and the correct type of the current sample, an optimizer is generally used for minimizing the loss function of the neural network model, and after training is finished, the fixed neural network model parameters are recorded as a model M.
The embodiment provides a method for generating word vector confrontation samples for text classification, which includes: initializing an English text needing text classification, performing word embedding on the English text, and converting the English text into corresponding vector representation; repeatedly performing partial derivative operation on word vectors in the English text according to the loss function until the classification result output by the neural network model is wrong; based on the modified word vector, selecting the word closest to the modified word vector in the space by adopting an Euclidean distance formula, and constructing an attack alternative word set; and carrying out random replacement on the words of the English text according to the attack surrogate word set to generate a confrontation sample.
Compared with the prior art, the word vector attack sample generation method for text classification provided by the embodiment of the invention has simple thought and is easy to realize, only a replacement word set needs to be searched, and the input end is directly replaced, so that the method is suitable for application scenes of a large number of text classifications; and the generated countermeasure sample makes the recognition of the neural network text classifier go wrong on the premise of ensuring that the semantics are not changed and the recognition and classification of the text by the human are not influenced, and in addition, the perception of the human can not be caused to the maximum extent by inserting legal characters by adopting an approximation method.
Second embodiment of the invention:
please refer to fig. 3.
As shown in fig. 3, the present embodiment provides a word vector confrontation sample generation apparatus for text classification, including:
the word embedding module 100 is configured to initialize an english text that needs to be subjected to text classification, perform word embedding on the english text, and convert the english text into corresponding vector representation.
In a preferred embodiment, the word embedding module 100 specifically includes:
word2vec word embedding is carried out on each word in the English text, and each word is converted into a word vector with a fixed length of m;
representing the English text as a two-dimensional matrix of n x m; wherein n is the total number of words in the English text, and m is a preset fixed length.
Specifically, for the word embedding module 100, since the text sequences are discrete special characters and cannot be directly processed by a computer, word embedding is performed on the text sequences first to convert the text sequences into vector representations. Assuming that each english text can be represented as [ X1, X2, ..., xn ], there are n words, and the word Xi is converted into a word vector with fixed length m through word2vec word embedding, i.e. Xi = [ Xi1, xi2, ..., xim ], so each text is represented as a two-dimensional matrix of n × m. It should be noted that, in the word vector attack sample generation method for text classification, word embedding methods include, but are not limited to, word2vec word embedding methods.
And the word vector modification module 200 is configured to repeatedly perform partial derivative operation on word vectors in the english text according to the loss function until a classification result output by the neural network model is incorrect.
In a preferred embodiment, the word vector modification module 200 specifically includes:
performing partial derivative operation on the word vectors in the English text according to the loss function to obtain the forward change rate of the loss function along each dimension of the input word vectors;
modifying each dimension of the input word vector according to the forward change rate to maximize a loss function in a constraint range;
and repeating the steps to modify a plurality of words in the English text until the classification result output by the neural network model is wrong.
Specifically, for the word vector modification module 200, the partial derivative operation is performed on the word vector by the loss function, and since the partial derivative is a change rate reflecting the positive direction of the objective function (i.e., the loss function) along the coordinate axis, in this embodiment, the loss function is used to derive the input word Xi to obtain the positive change rate of the loss function along each dimension (Xi 1, xi2, ..., xim) of the input Xi, and each dimension of the input Xi is modified according to the change rate, so that the loss function is maximized (in the constraint rule), and the above operation is repeatedly performed to modify the words until the classification of the neural network model is wrong. For example, the output result of the model is (0.3, 0.7), when the english text X is classified as bad, and is inconsistent with the original classification, when the neural network model is judged to be classified incorrectly. By maximizing the loss function, the model is classified or identified as erroneous with the maximum probability for the modified input samples.
And the attack alternative word set module 300 is configured to select, based on the modified word vector, a word closest to the modified word vector in the space by using an euclidean distance formula, and construct an attack alternative word set.
Specifically, for the attack substitute word set module 300, the two-dimensional representation of the english text X is changed through the previous step S102, the matrix value is changed, the matrix dimension is not changed, and the input word vector of each word is modified to [ X ' i1, X ' i2, ..., X ' im ]. However, the inventor of the present invention considers that [ X ' i1, X ' i2, ..., X ' im ] is likely not present in the mapping space of word2vec, which means that the word vector cannot find the corresponding word in the real space, so the metric of Euclidean distance is adopted:
selecting the word vector [ X 'i1, X' i2, \ 8230;, X 'im ] with the nearest distance to [ X' i1, X 'i2, \ 8230;, X' im ], the word corresponding to the word vector is X 'i, and the corresponding relation of Xi → X' i is obtained. Repeatedly searching a plurality of pairs Xi → X' i, thereby constructing an attack alternative word set. A distance approximation method is adopted to select legal words for replacement, so that semantic information of a text is not influenced to the maximum extent, human recognition is not influenced to the maximum extent, and generation of replacement words is not influenced, and meanwhile, the countermeasure samples are guaranteed not to have illegal characters while the recognition probability of a neural network is reduced.
It should be noted that, the distance manner used for measuring the distance between word vectors in the word vector attack sample generation method for text classification includes, but is not limited to, euclidean distance.
And the countermeasure sample module 400 is configured to perform random replacement on the words of the english text according to the attack substitute word set, and generate a countermeasure sample.
Specifically, for the confrontation sample module 400, optionally a movie review text X, assuming that the text is 5[ X ] X1, X2, X3, X4, X5] in length, each word is converted into a word vector of length 3 via word2vec word embedding, and the text X is converted into a two-dimensional matrix:
[[0.3,0.5,0.2],[0.6,0.9,0.1],[0.5,0.4,0.6],[0.8,0.9,0.1],[0.5,0.6,0.8]]；
modifying the input word vector using a loss function, the modified word vector being represented as:
[[0.3,0.5,0.2],[0.3,0.5,0.8],[0.5,0.4,0.6],[0.7,0.8,0.5],[0.5,0.6,0.8]]；
obviously, only the embedded word vectors of the words X2 and X4 have changed. However, considering that the modified word vectors [0.3,0.5,0.8] and [0.7,0.8,0.5] may not have mapping objects in the embedding space, the modified word vectors are approximated by using euclidean distance as the criterion, and finally the modified word vectors are selected to be closest to [0.4,0.5,0.8] (corresponding to the word X '2) and [0.7,0.8,0.6] (corresponding to the word X' 4) to [0.7,0.8,0.5], so that the attack surrogate word set collects two pairs of replacement words.
And (4) reselecting a movie comment text Y, and replacing X2 with X'2 to generate a confrontation sample on the assumption that a word X2 exists in the text Y.
In a preferred embodiment, the word vector confrontation sample generation apparatus for text classification further includes:
the neural network model module is used for designing a neural network model for classifying texts; the neural network model comprises an input layer, a hidden layer and an output layer;
the training module is used for converting a training text into a word vector representation and inputting the word vector representation to the neural network model for training to obtain an output result of the neural network model;
and the parameter correction module is used for correcting the parameters of the neural network model according to the output result of the neural network model and the correct type of the current training text, and fixing the parameters of the neural network model after the training is finished.
Specifically, the neural network model module is mainly used for designing the neural network model M to classify the text. The structure of the neural network can be roughly divided into an input layer, a hidden layer and an output layer, wherein the node number of the input layer of the neural network corresponds to the dimension of an input word vector; the number of layers of the hidden layer and the number of nodes of each layer can be set arbitrarily; the number of output layer nodes corresponds to the number of categories.
Specifically, for the training module, the training text is mainly converted into word vector representation and then put into a neural network model for training.
Specifically, for the parameter correction module, the network parameters are corrected according to the model output result and the correct type of the current sample, an optimizer is generally used to minimize the loss function of the neural network model, and after training is finished, the fixed neural network model parameters are recorded as a model M.
The embodiment provides a word vector confrontation sample generation device for text classification, which includes: the word embedding module 100 is configured to initialize an english text that needs to be subjected to text classification, perform word embedding on the english text, and convert the english text into corresponding vector representation; the word vector modification module 200 is configured to repeatedly perform partial derivative operation on word vectors in the english text according to the loss function until a classification result output by the neural network model is incorrect; the attack substitute word set module 300 is configured to select, based on the modified word vector, a word closest to the modified word vector in the space by using an euclidean distance formula, and construct an attack substitute word set; and the countermeasure sample module 400 is configured to perform random replacement on the words of the english text according to the attack substitute word set, and generate a countermeasure sample. The embodiment can effectively generate the word vector countermeasure sample aiming at the text classification by utilizing a small amount of knowledge of the original classification model, makes the recognition of the neural network text classifier wrong on the premise of ensuring that the semantics is not changed, the recognition and the classification of the text are not influenced by human beings, selects the most appropriate substitute word by using the Euclidean distance approximation word vector method, and ensures that the countermeasure sample does not generate illegal characters while reducing the recognition probability of the neural network.
An embodiment of the present invention further provides a terminal device for generating word vector countermeasure samples for text classification, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the word vector countermeasure sample generation method for text classification as described above.
An embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program runs, the apparatus on which the computer-readable storage medium is located is controlled to execute the word vector confrontation sample generation method for text classification as described above.
In the above embodiments of the present invention, the description of each embodiment has its own emphasis, and reference may be made to the related description of other embodiments for parts that are not described in detail in a certain embodiment.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the modules may be a logical division, and in actual implementation, there may be another division, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be an indirect coupling or communication connection through some interfaces, units or modules, and may be electrical or in other forms.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The foregoing is directed to the preferred embodiment of the present invention, and it is understood that various changes and modifications may be made by one skilled in the art without departing from the spirit of the invention, and it is intended that such changes and modifications be considered as within the scope of the invention.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Claims (8)
1. A word vector confrontation sample generation method for text classification is characterized by at least comprising the following steps:
initializing an English text needing text classification, performing word embedding on the English text, and converting the English text into corresponding vector representation;
repeatedly carrying out partial derivative operation on word vectors in the English text according to the loss function until the classification result output by the neural network model is wrong, wherein the method specifically comprises the following steps: performing partial derivative operation on word vectors in the English text according to the loss function to obtain the forward change rate of the loss function along each dimension of the input word vectors; modifying each dimension of the input word vector according to the forward change rate to maximize a loss function in a constraint range; repeatedly carrying out the steps to modify a plurality of words in the English text until the classification result output by the neural network model is wrong;
based on the modified word and word vectors, selecting the word closest to the modified word vectors in the space by adopting an Euclidean distance formula, and constructing an attack alternative word set;
and carrying out random replacement on the words of the English text according to the attack surrogate word set to generate a confrontation sample.
2. The method of generating word vector confrontation samples for text classification as recited in claim 1, further comprising:
designing a neural network model for classifying texts; the neural network model comprises an input layer, a hidden layer and an output layer;
converting the training text into word vector representation, inputting the word vector representation to the neural network model for training to obtain an output result of the neural network model;
and correcting parameters of the neural network model according to the output result of the neural network model and the correct category of the current training text, and fixing the parameters of the neural network model after training.
3. The method for generating the word vector confrontation sample for text classification as claimed in claim 1, wherein the word embedding is performed on the english text, and the english text is converted into a corresponding vector representation, specifically:
word2vec word embedding is carried out on each word in the English text, and each word is converted into a word vector with a fixed length of m;
representing the English text as a two-dimensional matrix of n x m; wherein n is the total number of words in the English text, and m is a preset fixed length.
4. A word vector confrontation sample generation device for text classification, comprising:
the word embedding module is used for initializing the English text needing text classification, embedding words in the English text and converting the words into corresponding vector representation;
the word vector modification module is used for repeatedly carrying out partial derivative operation on word vectors in the English text according to the loss function until the classification result output by the neural network model is wrong, and specifically comprises the following steps: performing partial derivative operation on word vectors in the English text according to the loss function to obtain the forward change rate of the loss function along each dimension of the input word vectors; modifying each dimension of the input word vector according to the forward change rate to maximize a loss function in a constraint range; repeatedly carrying out the steps to modify a plurality of words in the English text until the classification result output by the neural network model is wrong;
the attack alternative word set module is used for selecting the word closest to the modified word vector in the space by adopting an Euclidean distance formula based on the modified word vector to construct an attack alternative word set;
and the countermeasure sample module is used for randomly replacing the words of the English text according to the attack substitute word set to generate a countermeasure sample.
5. The apparatus of claim 4, further comprising:
the neural network model module is used for designing a neural network model for classifying texts; the neural network model comprises an input layer, a hidden layer and an output layer;
the training module is used for converting a training text into a word vector representation and inputting the word vector representation to the neural network model for training to obtain an output result of the neural network model;
and the parameter correction module is used for correcting the parameters of the neural network model according to the output result of the neural network model and the correct type of the current training text, and fixing the parameters of the neural network model after the training is finished.
6. The method for generating word vector confrontation samples for text classification as claimed in claim 4, wherein the word embedding module is specifically:
word2vec word embedding is carried out on each word in the English text, and each word is converted into a word vector with a fixed length of m;
representing the English text as a two-dimensional matrix of n x m; wherein n is the total number of words in the English text, and m is a preset fixed length.
7. A terminal device for word vector countermeasure sample generation for text classification, characterized by comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the word vector countermeasure sample generation method for text classification as claimed in any one of claims 1 to 3 when executing the computer program.
8. A computer-readable storage medium, comprising a stored computer program, wherein when the computer program runs, the computer-readable storage medium is controlled by a device to execute the method for generating word vector confrontation samples for text classification according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010248226.9A CN111444346B (en) | 2020-03-31 | 2020-03-31 | Word vector confrontation sample generation method and device for text classification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010248226.9A CN111444346B (en) | 2020-03-31 | 2020-03-31 | Word vector confrontation sample generation method and device for text classification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111444346A CN111444346A (en) | 2020-07-24 |
| CN111444346B true CN111444346B (en) | 2023-04-18 |
Family
ID=71649518
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010248226.9A Active CN111444346B (en) | 2020-03-31 | 2020-03-31 | Word vector confrontation sample generation method and device for text classification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111444346B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11520829B2 (en) * | 2020-10-21 | 2022-12-06 | International Business Machines Corporation | Training a question-answer dialog sytem to avoid adversarial attacks |
| CN112380845B (en) * | 2021-01-15 | 2021-04-09 | 鹏城实验室 | Sentence noise design method, equipment and computer storage medium |
| CN113723506B (en) * | 2021-08-30 | 2022-08-05 | 南京星环智能科技有限公司 | Method and device for generating countermeasure sample and storage medium |
| CN114357166B (en) * | 2021-12-31 | 2024-05-28 | 北京工业大学 | Text classification method based on deep learning |
| CN114528827B (en) * | 2022-01-02 | 2024-07-19 | 西安电子科技大学 | Text-oriented countermeasure sample generation method, system, equipment and terminal |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109271632A (en) * | 2018-09-14 | 2019-01-25 | 重庆邂智科技有限公司 | A kind of term vector learning method of supervision |
| CN109471944A (en) * | 2018-11-12 | 2019-03-15 | 中山大学 | Training method, device and the readable storage medium storing program for executing of textual classification model |
| CN110457701A (en) * | 2019-08-08 | 2019-11-15 | 南京邮电大学 | Dual training method based on interpretation confrontation text |
-
2020
- 2020-03-31 CN CN202010248226.9A patent/CN111444346B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109271632A (en) * | 2018-09-14 | 2019-01-25 | 重庆邂智科技有限公司 | A kind of term vector learning method of supervision |
| CN109471944A (en) * | 2018-11-12 | 2019-03-15 | 中山大学 | Training method, device and the readable storage medium storing program for executing of textual classification model |
| CN110457701A (en) * | 2019-08-08 | 2019-11-15 | 南京邮电大学 | Dual training method based on interpretation confrontation text |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111444346A (en) | 2020-07-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111444346B (en) | Word vector confrontation sample generation method and device for text classification | |
| CN112287670A (en) | Text error correction method, system, computer device and readable storage medium | |
| CN109815336B (en) | Text aggregation method and system | |
| CN111737511B (en) | Image description method based on self-adaptive local concept embedding | |
| CN110502976B (en) | Training method of text recognition model and related product | |
| CN110188775B (en) | Image content description automatic generation method based on joint neural network model | |
| CN109815355A (en) | Image search method and device, storage medium, electronic equipment | |
| CN111159412B (en) | Classification method, classification device, electronic equipment and readable storage medium | |
| CN112183994A (en) | Method and device for evaluating equipment state, computer equipment and storage medium | |
| CN112818995B (en) | Image classification method, device, electronic equipment and storage medium | |
| CN109885180B (en) | Error correction method and apparatus, computer readable medium | |
| CN113590865A (en) | Training method of image search model and image search method | |
| CN112613293A (en) | Abstract generation method and device, electronic equipment and storage medium | |
| CN112200772A (en) | Pox check out test set | |
| CN114511023B (en) | Classification model training method and classification method | |
| CN117058716A (en) | Cross-domain behavior recognition method and device based on image pre-fusion | |
| CN112270334A (en) | Few-sample image classification method and system based on abnormal point exposure | |
| CN115248846B (en) | Text recognition method, device and medium | |
| CN110929013A (en) | Image question-answer implementation method based on bottom-up entry and positioning information fusion | |
| CN113610080B (en) | Cross-modal perception-based sensitive image identification method, device, equipment and medium | |
| CN112016281B (en) | Method and device for generating wrong medical text and storage medium | |
| CN111695526B (en) | Network model generation method, pedestrian re-recognition method and device | |
| EP4338395A1 (en) | Artificial intelligence based cognitive test script generation | |
| CN117611845B (en) | Multi-mode data association identification method, device, equipment and storage medium | |
| CN117851835B (en) | Deep learning internet of things recognition system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Gu Zhaoquan Inventor after: Xie Yushun Inventor after: Fang Binxing Inventor after: Fu Xiao Peng Inventor after: Zhu Bin Inventor after: Wu Danni Inventor after: Wang Le Inventor after: Qiu Jing Inventor after: Han Weihong Inventor before: Gu Zhaoquan Inventor before: Liao Xuxin Inventor before: Fang Binxing Inventor before: Wang Le Inventor before: Wang Xingang Inventor before: Zhang Chuanjing Inventor before: Wang Yuetian |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |