CN114491448A - Automatic confrontation training method and device - Google Patents

Abstract

One aspect of the present disclosure relates to an automated confrontational training method comprising obtaining an original training data set and a model trained using the original training data set, the original training data set comprising input data and labels thereof; automatically perturbing the input data by a predetermined number of steps, including for each of the predetermined number of steps: automatically searching a hyper-parameter value optimal to the disturbance of the step in a predefined attack hyper-parameter search space; automatically determining a perturbation of the step based on the optimal hyper-parameter value; and updating the input data by incorporating perturbations of this step; and further optimizing the model to combat the perturbation using an combat training data set comprising updated input data and its labels after the perturbation over the predetermined number of steps. The present disclosure also relates to other related aspects.

Description

Automatic confrontation training method and device

Technical Field

The present application relates generally to machine learning and more particularly to opponent training.

Background

Resistance training is an important way to enhance neural network robustness. As a method for defending against attacks, in the process of resisting training, a sample is mixed with a little disturbance (a change which cannot be perceived by human beings but is easy to cause misclassification of a neural network model) to generate a resisting sample and is added into a training set, and then the neural network learns and adapts to the change through training, so that the robustness of the neural network on the resisting sample is enhanced.

At the client (e.g., mobile phone), the account security of the Application (APP) user is important. The user account number may be maliciously stolen by relatives and friends, acquaintances, black births and the like, and operations such as transferring payment, stealing data, destroying data and the like are performed. Such risk may be referred to as client theft risk.

In order to detect the client theft risk, the existing neural network model can judge whether the client theft behavior occurs according to the user behavior mode. In order to improve the robustness of the model, counter training may be used to enhance the robustness of the model. However, the existing construction method of the countermeasure sample is single, so that the model obtained by the countermeasure training is only robust to specific attacks, and cannot cope with various attack means of black products in the stealing scene.

Accordingly, there is a need in the art for improved anti-training techniques to increase the robustness of neural network models that detect the risk of theft at clients.

Disclosure of Invention

One aspect of the present disclosure relates to an automated confrontational training method comprising obtaining an original training data set and a model trained using the original training data set, the original training data set comprising input data and labels thereof; automatically perturbing the input data by a predetermined number of steps, including for each of the predetermined number of steps: automatically searching a hyper-parameter value optimal to the disturbance of the step in a predefined attack hyper-parameter search space; automatically determining a perturbation of the step based on the optimal hyper-parameter value; and updating the input data by incorporating perturbations of this step; and further optimizing the model to combat the perturbation using an combat training data set comprising updated input data and its labels after the perturbation over the predetermined number of steps.

According to some exemplary embodiments, the predefined attack hyper-parameter search space comprises hyper-parameters with candidate values, and the hyper-parameters comprise at least perturbation operations and step sizes for perturbation of each step.

According to some exemplary embodiments, the candidate values for the perturbation operation include one or more of the following perturbation operators, or any combination thereof: FGM, FGSM, FGMM, gaussian noise, and identity variation.

According to some exemplary embodiments, the candidate values for the step size comprise one or more of the following steps, or any combination thereof: 0.0001,0.001,0.01,0.1,1.

According to some exemplary embodiments, automatically perturbing the input data by a predetermined number of steps comprises taking input data of the original training data set as input for a first step perturbation, taking the input data after each step perturbation as input for a next step perturbation, and taking input data after a last step perturbation as the updated input data in the antagonistic training data set.

According to some exemplary embodiments, automatically searching for a hyper-parameter value that is optimal for the perturbation of the step in the predefined attack hyper-parameter search space comprises using an attentiveness mechanism to determine a score for the hyper-parameter over each candidate value and determining the hyper-parameter value that is optimal for the perturbation of the step based on the scores.

According to some exemplary embodiments, determining the score of the hyperparameter over each candidate value using an attention mechanism comprises: determining a query based on the input data for the step; embedding each candidate value as a corresponding key; and obtaining an attention distribution value of the query on each key as a score of the hyperparameter on the corresponding candidate value by using an attention mechanism.

According to some exemplary embodiments, determining the score of the hyperparameter at each candidate value using the attention mechanism further comprises calculating its score at the respective candidate value using the corresponding attention mechanism for the perturbation operation and the step size, respectively; and wherein determining the value of the hyper-parameter optimal for the perturbation of the step based on the score comprises a normalization sampling unit using Gumbel Softmax as the score of the perturbation operation on its candidate value, and a normalization unit using Softmax as the score of the step size on its candidate value.

According to some exemplary embodiments, automatically determining the perturbation of the step based on the optimal hyper-parameter value comprises: determining the perturbation of the step based on the optimal perturbation operation value and the optimal step size value and the gradient of the loss function of the model on the input data of the step.

According to some exemplary embodiments, determining a query based on the input data of the step comprises determining a gradient of a loss function of the model over the input data of the step; and linearly projecting the gradient as the query.

According to some exemplary embodiments, further optimizing the model using the antagonistic training dataset comprising updated input data and its labels after perturbation over the predetermined number of steps comprises updating the model based on a gradient of a loss function of the model over the antagonistic training dataset.

According to some exemplary embodiments, the input data comprises a behavior pattern of the user on the APP of the client, and the model is used to determine whether a client spoofing behavior has occurred.

Other aspects of the disclosure also include corresponding apparatus, devices, and computer-readable media, among others.

Drawings

Fig. 1 illustrates a block diagram of a parameterized attacker in accordance with an aspect of the present disclosure.

FIG. 2 illustrates a block diagram of a single-step attack unit in a parameterized attacker according to an aspect of the present disclosure.

FIG. 3 illustrates a block diagram of a single-step attack unit in a parameterized attacker according to an aspect of the present disclosure.

FIG. 4 illustrates pseudo code of a parameterized attacker in accordance with an aspect of the present disclosure.

Fig. 5 illustrates a schematic diagram comparing automated confrontational training with standard training and prior art confrontational training in accordance with an aspect of the present disclosure.

FIG. 6 illustrates a flow diagram of an automated confrontational training method in accordance with an aspect of the disclosure.

FIG. 7 illustrates a flow diagram of an automatic perturbation attack submethod according to an aspect of the present disclosure.

Detailed Description

In order to detect the risk of client theft, the neural network model can judge whether the client theft behavior occurs according to the user behavior pattern.

According to some example embodiments, patterns of user behavior on the APP of the client may be tracked with SPM (super location model). SPMs are codes that can be used to track page module locations. With the SPM buried point scheme, a buried point log request is sent whenever a user enters a page. According to the SPM sequence clicked by the user, the neural network can judge whether the stealing behavior occurs.

However, the SPM coding of the client (e.g. the mobile phone) may change with the APP version update, i.e. the data distribution changes. Meanwhile, black products may also fumble in trying different behaviors to bypass the discrimination of the neural network model, which is called the disturbance from the black products. These and other reasons, among others, have led to rapid degradation of the performance of on-line neural network models. In particular, in contrast to blackjack, more attacks of the same or similar type tend to be performed more intensively once a blackjack attack is successful. This is a significant safety hazard for both the platform and the customer.

In order to improve the robustness of the neural network model, countertraining can be used, namely, samples which are easy to classify errors when the model is constructed are called countersamples as input, so that the robustness of the model can be enhanced. The construction method of the countermeasure sample directly influences the quality of the countermeasure sample, and further influences the robustness of the model.

In order to cope with various attack means of black products in a stealing scene, the disclosure aims to provide an automatic countercheck training method to resist various attacks. Specifically, the automated countermeasure training method disclosed by the invention can utilize an automated machine learning technology (i.e., based on AutoML) to search in a disturbance space integrating various attacks, construct diversified countermeasure samples and solve an optimization problem of the countermeasure samples, and thus significantly improve the robustness of a final model. The embodiments of the present disclosure are described below in detail with reference to specific examples.

In general, the optimization objective of traditional confrontational training is the following min-max problem:

where x is the natural sample, y is its associated label, δ ∈ S is the opposition perturbation, where S is the perturbation space, θ is the set of parameters for the neural network,

is a function of the loss as a function of,

is a data set of (x, y), and

is an error function representing the performance of the neural network as a classifier. As can be seen, inner max aims to find the optimal counterdisturbance δ that maximizes the loss function^*To change the model prediction results, while outer min includes the optimal countermeasure disturbance δ^*Learning the confrontation sample to make the error function

Minimized to obtain a more robust neural network model.

The number of steps of inner max is generally related to the optimization strength (i.e. finding an extremum that is as close as possible to the global extremum). However, the larger the number of inner max steps, the longer the training time. In addition, it may also be advantageous to construct a better countermeasure against disturbances if at least some of the hyper-parameters (i.e. the parameters set before the learning process is started, relative to the parameters obtained by training) can be adjusted, such as random initialization, step size, momentum, early stopping, etc.

In view of this, the present disclosure introduces an attacker characterized by a parameter α, which includes the parameters of the attacker. In particular, the present disclosure employs a parameterized attacker to search the perturbation space S for an optimal perturbation δ based on the current training model and inputs^*(i.e., inner max problem). Targeting the current training model f by the attacker α_θThe opposing perturbation generated by the sum input (x, y) is denoted as δ (x, y, θ; α). Thus, the min-max problem translates into the following two-layer optimization problem:

so that

Where δ (x, y, θ; α) is the aggressor α against the current training model f_θAnd the counterdisturbance, α, generated by the input (x, y)^*Is that make

The selection of the maximized attacker over-parameter value α. Thus, the present disclosure will aim to find the optimal countermeasure disturbance δ that maximizes the loss function^*The inner max problem translates into finding the optimal aggressor α^*。

Since the inner max training attacker consumes a large amount of computation power, the inner-max optimal solution can be approximated by a one-step gradient descent method commonly used in meta-learning, that is:

where α represents the selection of the current over-parameter value of the attacker, and Lr represents the learning rate at which the attacker optimizes.

In order to limit the disturbance space required to be searched by inner-max, the conventional practice of countermeasure training can be referred to, and the generation of the countermeasure disturbance is regarded as the combination of several step gradients, and the gradients of each step are utilized

The combination of (a) and step size ξ are selected. I.e. in the disturbance space

And searching is carried out so as to automatically find out the over-parameter value which is optimal for each step of disturbance.

According to an exemplary embodiment of the present disclosure, the hyper-parameter value optimal for each step of perturbation may comprise a combination of values of several hyper-parameters. In particular, according to some exemplary embodiments, the perturbation space (i.e. the hyper-parameter space) of the desired search may comprise one or more or any combination of the following hyper-parameters: the number of steps, the perturbation operations (e.g., perturbation operators) that make up each step of the attack unit, and the step size. Thus, the over-parameter value for each step that is optimal for perturbation may comprise a combination of the optimal value for the perturbation operation and the optimal value for the step size in that step. It should be noted that although reference is made herein to "values of excess parameters" or similar terms, those of ordinary skill in the art will appreciate that the term is not limited to "values," but rather may encompass substantially "values" in the form of values or "values" in the form of non-values, and the like, the particular type of value being determined by the nature of the corresponding parameter and/or may be represented by non-values and/or values corresponding or associated therewith, as desired, in various instances.

According to an exemplary embodiment of the present disclosure, the perturbation operation may be represented by a perturbation operator. The perturbation operator may include, for example, but is not limited to, FGM/FGSM/FGMM … …. The step size may include, for example, but is not limited to, 0.0001, 0.001, 0.01, 0.1, 1 … ….

According to some exemplary embodiments, each step of perturbation attack may be implemented by an attack unit. For each attack unit, an optimal perturbation operation value (i.e., perturbation operator) and an optimal step size value may be selected among the perturbation operation candidate values (e.g., including, but not limited to, the 5 perturbation operators FGM/FGSM/FGMM/gaussian noise/identity change) and the step size candidate values (e.g., including, but not limited to, 0.0001, 0.001, 0.01, 0.1, 1). Of course, as will be appreciated by those skilled in the art, the perturbation operators and/or step sizes are not limited to those listed above, but may include other perturbation operators and/or step sizes, or any combination thereof. In addition, the searchable hyperparameters are also not limited to perturbation operations and step sizes, but may include other additional, alternative, more, or fewer hyperparameters.

For example, according to an exemplary scenario, when the number of picking steps is 5, the following exemplary but non-limiting countermeasures against disturbances may be employed:

(i) selecting a disturbance operator as FGM by the attack unit in each step, which is equivalent to PGD for 5 steps;

(ii) the attack unit in the first step selects the disturbance operator as FGSM, and the attack units in the other subsequent steps select the disturbance operator as constant change, which is equivalent to attack degradation as FGSM;

(iii) the first step attack unit selects the disturbance operator as Gaussian noise, and the other subsequent steps attack units select the disturbance operator as FGM, which is equivalent to that different starting points are selected through random disturbance first, and then the local optimal disturbance is found through 4 steps of PGD.

Several exemplary, but non-limiting, example superparametric choices for combating perturbations are given above, but as one of ordinary skill in the art will appreciate, the disclosure is not so limited, and can encompass various perturbation spaces, various step counts, various attack unit combinations, various perturbation operators, various step sizes, various combinations of the above, and the like.

Fig. 1 illustrates a block diagram of a parameterized attacker 100 in accordance with an aspect of the present disclosure. As shown in fig. 1, the step number override parameter of the parameterized attacker 100 may be selected as K steps, i.e., serially connected K step attack units or K step attacks, where each step attack unit K (K is in the range of 1 to K) takes the output of the previous step attack unit as input. In general, K may be less than 10. According to a preferred embodiment, K may comprise 5.

Specifically, the input of attack unit 1 is x⁽⁰⁾I.e. the original sample. The attack unit 1 processes the original sample x based on the parameter α of the attack unit⁽⁰⁾(e.g. to incorporate perturbations of the first step)

) Then obtaining and outputting a first step disturbance sample x⁽¹⁾. The first step perturbs the sample x⁽¹⁾Is input to the attack unit 2.

The attack unit 2 processes the first step perturbation sample x based on the parameter alpha of the attack unit⁽¹⁾(for example,to incorporate disturbance of the second step

) Then obtaining and outputting a second-step disturbance sample x⁽²⁾. The second step perturbs the sample x⁽²⁾Is input to the attack unit 3 (not shown).

And so on, the attack unit K of the last step processes the disturbance sample x of the (K-1) th step based on the parameter alpha of the attack unit^(K-1)(e.g., to include perturbation of step K)

) Then obtaining and outputting a K-th step disturbance sample x^(K). The outputted Kth step disturbance sample x^(K)I.e., a challenge sample that incorporates a near-optimal challenge perturbation output by the parameterized attacker 100. On the basis, the attack sample can be used to train the model to make the error function by learning the confrontation sample containing the above-mentioned approximately optimal confrontation disturbance through outin

Minimized to obtain a more robust neural network model f_θ。

FIG. 2 illustrates a block diagram of a single-step attack unit 200 in a parameterized attacker according to an aspect of the present disclosure. The parameterized attacker may comprise or constitute the parameterized attacker 100 described above in connection with fig. 1. The single-step attack unit 200 of fig. 2 may include or constitute any one of the step-attack units in the parameterized attacker 100 described above in connection with fig. 1.

Single-step attack unit 200 of fig. 2 may include, but is not limited to, a gradient unit 202, a linear projection unit 204, a hyper-parameter embedding unit 206, a hyper-parameter score calculation unit 208, and a perturbation determination unit 210.

According to an exemplary embodiment, attack unit k (200) takes x^(k-1)Is an input. Inputted x^(k-1)Is provided to the gradient unit 202 to determine the model f_θThe gradient of the loss function on the input data after the perturbation in the (k-1) th step, which contains the current training modeType f_θAnd input x^(k-1)And information of its tag y.

The calculated gradient is linearly projected to a vector space by the linear projection unit 204 to obtain a query (query) of the current model.

On the other hand, the hyperparameter embedding unit 206 may embed candidate values of hyperparameters as continuous keys (keys). In the context of the present disclosure, a hyper-parameter may include, for example, a number of steps, an operation constituting a unit per step, and/or a step size. According to an exemplary embodiment, the hyper-parameters that need to be embedded in a single-step attack unit may include, for example, attack operations (e.g., perturbation operators), step sizes, and the like, or any combination thereof.

According to an example embodiment, hyperparameter embedding unit 206 may project the plurality of candidate values (e.g., the plurality of perturbation operators) of the attack operation into the consecutive embedding space as a number of corresponding keys by embedding, respectively, and project the plurality of candidate values of the step size into the consecutive embedding space as a number of corresponding keys by embedding, respectively.

The hyperparameter score calculation unit 208 acquires the query (query) of the current model and these keys (keys), and calculates the score of the candidate value of the hyperparameter by using, for example, an attention (attention) mechanism. The attention mechanism performs a similarity calculation of the query and the keys to obtain the corresponding attention distribution value of the query on each key. The corresponding attention distribution value is a value (value) or score (score) that reflects the probability of the candidate value. The value or score may be normalized (e.g., to obtain a probability of the candidate value).

The perturbation determining unit 210 determines optimal values (for example, an optimal perturbation operation and an optimal step size) of the hyper-parameters to be used by the current attack unit based on the score of each candidate value, and obtains the output x of the current attack unit by incorporating the corresponding perturbation (for example, a perturbation including the optimal perturbation operation and the optimal step size)^(k)And the model f can be iteratively trained accordingly_θAnd an attacker alpha (including, for example, the parameters of the hyper-parameter embedding unit 206 and the linear projection unit 204 of the attack unit, etc.).

FIG. 3 illustrates a block diagram of a single-step attack unit 300 in a parameterized attacker according to another aspect of the present disclosure. The parameterized attacker may comprise or constitute the parameterized attacker 100 described above in connection with fig. 1.

Single-step attack unit 300 of FIG. 3 may include, but is not limited to, a gradient unit 302, a linear projection unit 304, a perturbation block 306-A, a stepsize block 306-B, a hyper-parameter score calculation unit 308, and a perturbation determination unit 310.

According to an exemplary embodiment, attack unit k (300) takes x^(k-1)Is an input. Dependent on k, x^(k-1)May be the initial input data (e.g., k ═ 1) or the input data after the last perturbation attack (e.g., 1)<k<K). Inputted x^(k-1)Is provided to the gradient unit 302 to determine the model f_θThe gradient of the loss function on the input data after the perturbation in the (k-1) th step, which implies the current training model f_θAnd input x^(k-1)And information of its tag y. For example, according to an exemplary embodiment, the gradient g^(k-1)Can be calculated according to the following equation:

the calculated gradient g of the (k-1) th step^(k-1)Then passed by the linear projection unit 304 with a weight matrix

Is projected linearly into the hidden space as query q (query).

On the other hand, the attack unit k300 may embed a candidate value of the hyperparameter. For example, attack unit k300 may be perturbed at candidate perturbation o by perturbation block 306-A_p1，o_p2，o_p3，o_p4… … and selects and processes the corresponding step size block 306-B to obtain the candidate step size o corresponding to the selected step size_s1，o_s2，o_s3，o_s4… …, as the over-parameter value for attack unit k300 at this step.

Here, although the candidate perturbation is described as including four candidate perturbation operations and the candidate step sizes are described as correspondingly including four candidate step sizes, one of ordinary skill in the art will appreciate that the present disclosure is not so limited, and more and/or fewer candidate perturbations and candidate step sizes may be included.

The perturbation block 306-A embeds each of these candidate perturbation operations by learning (e.g., by a weight matrix)

Representation) to project a plurality of keys e corresponding to operations corresponding to the keys into a continuous embedding space_p1，e_p2，e_p3，e_p4… … are provided. Similarly, the step-size block 306-B embeds the candidate step-sizes corresponding to the candidate perturbation operations by learning (e.g., from the weight matrix W)_s ^(k-1)Representation) to project into a continuous embedding space a corresponding number of keys e corresponding to candidate step sizes_s1，e_s2，e_s3，e_s4……。

The attack unit k300 may calculate the score of the candidate value of the hyper-parameter by using, for example, an attention (attention) mechanism. For example, keys and queries may be input to the hyperparametric score calculation unit 308 to calculate scores for candidate values of hyperparameters. According to an example embodiment, the hyperparametric score calculation unit 308 may be based on an attention mechanism. For example, in an exemplary embodiment, the hyperparametric score calculation unit 308 may include two attention units and corresponding normalization units for the perturbation operation and the step size, respectively. For example, the normalization unit may be based on a Softmax unit for normalizing the scores by the attention units (e.g., to obtain a probability of the candidate value).

According to an exemplary embodiment, since the perturbation operation is a discrete value, the normalization unit for perturbation operation in the hyperparametric score calculation unit 308 may use Gumbel Softmax normalization sampling units to enable backpass gradients for training.

For example, according to an exemplary embodiment, key e represents a candidate perturbation operation_p1，e_p2，e_p3，e_p4… … are input into the attention unit for perturbation together with the query (query) to perform similarity calculation of the query and the key to obtain the corresponding attention distribution value of the query on each key as the value (value) or score (score) of the candidate perturbation operation. The corresponding attention distribution value reflects the probability of the candidate perturbation operation corresponding to the key. The probability of each candidate attack operation representing the current attack unit k300 is obtained through normalization by a normalization sampling unit (e.g., Gumbel Softmax) and sampling is carried out according to the obtained probability (i.e., the score after normalization and sampling (e.g., one-hot)). For example, according to an exemplary embodiment, if the probability of a candidate attack operation is p, then the candidate has a score that is sampled by Gumbel Softmax to, for example, a one-hot coded form, with a probability equal to p.

Likewise, key e representing a candidate step_s1，e_s2，e_s3，e_s4… … are also entered along with the query (query) into the attention unit for step size for similarity calculation to obtain the corresponding attention distribution value of the query on each key as the value (value) or score (score) of the candidate step size. The corresponding attention distribution value reflects the probability size of the candidate step corresponding to the key. The probability (i.e., the normalized score) of the current attack unit k300 in each candidate step is obtained after normalization by a normalization unit (e.g., a Softmax unit).

The output of the hyperparametric score calculation unit 308 (e.g., the normalized sampling score of the candidate perturbation operation and/or the normalized score of the candidate step size) may be provided to the perturbation determination unit 310 to determine the perturbation to be used by the current attack unit k300

For example, according to an exemplary embodiment, the perturbation determining unit 310 may decide a perturbation operation to be used by the current attack unit k300 based on the normalized sampling score of the candidate perturbation operation, and/or may decide a step size to be used by the current attack unit k300 based on the normalized score of the candidate step size.

According to some exemplary embodiments, the normalized sampling score of the candidate perturbation operation may include a one-hot code, so that the perturbation determining unit 310 may directly determine the candidate perturbation operation corresponding to the one-hot code as the optimal perturbation operation to be used by the current attack unit k 300.

According to some exemplary embodiments, the normalized score of the candidate step size may include a form of a probability vector, which the perturbation determining unit 310 may perform a weighted summation of the weight and the candidate value of the step size to determine the optimal step size to be used by the current attacking unit k 300.

Although the input provided by the hyperparameter score calculation unit 308 to the perturbation determination unit 310 is described here by taking probability as an example, one of ordinary skill in the art will recognize that the input is not limited to being in the form of probability, but may be in the form of various probability distributions reflecting candidate values of the hyperparameter.

According to an exemplary embodiment, the perturbation to be used by the current attack unit k300

May be determined based on the decided perturbation operation and step size to be used and the gradient of the loss function of the current model over the input data of the current attack unit k 300. According to at least some example embodiments, the disturbance is a disturbance

May be determined based on the product of the decided perturbation operation and step size to be used and the gradient of the loss function of the current model over the input data of the current attack unit k 300.

The disturbance determination unit 310 will determine the disturbance to be used

Acting on input x^(k-1)The updated output may be obtained, for example, by

To obtain an updated output.

In the course of model trainingFor each batch of data (X, Y) (e.g., X represents the set of samples X in the batch, and Y represents the set of labels Y corresponding thereto), the hyper-parameter selection (e.g., optimal perturbation operation, optimal step size) made by the current attack unit k300 results in the current attack unit k aiming at the current batch of data X^(k-1)And a model f_θAgainst disturbance

For example, according to an exemplary embodiment, data X for the current batch^(k-1)And a model f_θOf the current attack unit k

May be based on the attack operation and step size determined in the current attack unit k 300. For example, according to at least some example embodiments, data X is targeted to a current batch^(k-1)And a model f_θAgainst disturbance

Can be based on the value representing the optimal attack operation, the value representing the optimal step size and the model f determined in the current attack unit k300_θThe product of the gradients of the loss function on the input data after perturbation by step (k-1).

In addition, a current model f may be calculated_θIn the current batch data X^(k-1)Total loss in

Since Gumbel Softmax normalized sampling is used for perturbation operation, the choice of perturbation operation is differentiable for the attacker α, so that the total loss can be accounted for

Back propagation is performed to iteratively train model f_θWith the current weight of the aggressor alpha (e.g., a weight matrix including at least the perturbing block 306-A)

Weight matrix W for step block 306-B_s ^(k-1)And a weight matrix of the linear projection unit 304

One or more of the above).

Fig. 4 illustrates pseudo code 400 of a parameterized attacker in accordance with an aspect of the present disclosure. As shown in FIG. 4, automated countermeasure training with K attack units A²T^KThe parameterized attackers can take the training data set

(X, Y) and training step number K are input.

First, a parameter set θ and an attacker α of the neural network are initialized. According to an exemplary embodiment, N may be performed_epTraining of wheel defense models, where N_ep>1. For this N_epEach round in the round defense model training, so that the initial confrontation disturbance

Is 0 and makes the initial input a natural dataset X in the training dataset.

Then, for each of the K attack units, based on the current model f_θIn the current batch data X^(k-1)Total loss in

To calculate the gradient g^(k-1). For example, according to an exemplary embodiment, the gradient g^(k-1)May be based on the current model f_θIn the current batch data X^(k-1)Total loss in

Is calculated.

Then, according to the gradient g^(k-1)To construct the current one-step attack unit k pair X^(k-1)Disturbance of

For example, according to an exemplary embodiment, the disturbance is resisted

May be based on the determined optimal attack operation and step size in the current attack unit. For example, in accordance with at least some example embodiments, disturbances are resisted

Can be based on the determined optimal attack operation, optimal step length and model f in the current attack unit k300_θIs determined by multiplying the gradient of the input data after perturbation in step (k-1).

Subsequently, X is coupled by incorporation^(k-1)Disturbance of

To update X of the current attack unit k^(k). According to an exemplary embodiment, this may be achieved, for example, by aligning X with X^(k)Each x in (1)^(k)Such as

To obtain an updated output.

After the attack of the K attack units is completed, respectively updating the parameter set theta and the attacker alpha of the neural network by all the preferred gradients and step lengths accumulated in the total K steps, wherein the updating of the parameter set theta of the neural network can comprise model f-based analysis_θThe model is updated for the gradient of the loss function for the batch of training samples that incorporates K total steps of perturbation, and updating the attacker α may include updating the attacker α based on the gradient and the step size of the error function with the loss function maximized (e.g., achieved by the optimal perturbation and step size for K total steps). For example, according to an exemplary embodiment, the attacker α may be updated based on the product of the gradient of the error function and the step size. According to an exemplary embodiment, the parameter α of the attacker can be updated with the original attacker α plus the product of the gradient of the error function and the step size. According to at least some of the figuresIn an exemplary embodiment, the parameters α of the attacker may for example comprise at least a weight matrix for embedding the candidate values of the hyperparameters as consecutive keys and a weight matrix for linearly projecting the gradients into the hidden space as queries

One or more of (a).

The above pseudocode 400 describes an exemplary process for automated confrontational training in accordance with exemplary aspects of the present disclosure, but those of ordinary skill in the art will appreciate that the various steps may be organized and performed in different orders and levels without departing from the scope of the present disclosure.

Fig. 5 illustrates a schematic diagram 500 comparing automated confrontational training with standard training and prior art confrontational training in accordance with an aspect of the disclosure.

As shown in the figure, X, Y represent the training data set

Model representation defense model, i.e. neural network f_θThe left box of each step in the middle represents the attack operation, and the right box represents the step length over-parameter when constructing the attack.

On the left is normal standard model training: in the process of machine learning, data is input into a model without transformation (namely, the attack method is constant change, the step length is 0, so that the disturbance delta is equal to 0, and the influence on a training data set is avoided), and training is carried out

Intermediate is countertraining, which constructs attacks primarily through artificially selected hyper-parameters (e.g., FGM for the attack method, 0.007 for the attack time step size) (e.g., as with PGD)¹⁰There are 10 attacks in total and are therefore indicated by ellipsis). The input X is updated with the perturbation delta generated by each step of attack and the updated input X and its label Y are provided to the model for training.

On the right side is the automated confrontational training of the present disclosure. Automated countermeasure training according to the present disclosure treats generation of the countermeasure disturbance δ as a number of step gradients g^(k)By perturbation inA search is performed in space to determine the optimal attack (e.g., including attack operations and step sizes) for each step and accordingly the optimal perturbation.

The left box of each step in the middle represents the attack operation, and the right box represents the step length over-parameter when constructing the attack. It can be seen that there are several candidate attacks in the attack operation and correspondingly several candidate step sizes, the bolded representation being selected. That is, in the automated countermeasure training of the present disclosure, in each step of the attack, the hyper-parameters (e.g., attack operation and attack step size) are selected (as shown in bold) among several candidates in the perturbation space by the attacker, and then the perturbation of the step of the attack is regenerated

To update input X and input the updated input X along with its label Y into the model for training.

Fig. 6 illustrates a flow diagram of an automated confrontational training method 600 in accordance with an aspect of the disclosure. The automated confrontational training method 600 of fig. 6 may include, for example, at block 602, obtaining a raw training data set and a model trained using the raw training data set, the raw training data set including input data and its labels.

For example, according to an exemplary embodiment, a raw training data set including input data and its labels may be obtained and used to train a model, thereby obtaining a trained model. However, because the samples of the original training data set are limited, the model so trained may be misclassified in the face of minor perturbations.

At block 604, method 600 may include automatically perturbing the input data by a predetermined number of steps. Perturbation of a predetermined number of steps may include, for example, using automated machine learning techniques (i.e., based on AutoML) to search in a perturbation space that integrates multiple attacks, constructing diverse challenge samples to solve an optimization problem for the challenge samples.

For example, various combinations of values of a plurality of hyper-parameters may be included in the perturbation space. According to an exemplary embodiment, the hyper-parameters may include, for example, but are not limited to, the number of steps, the perturbation operations (i.e., perturbation operators) that make up each step of the attack unit, and the step size.

At block 606, the method 600 may include further optimization training the model using the updated input data after a predetermined number of steps of perturbation and its labels as a antagonistic training data set.

For example, to detect a client theft risk, the neural network model may be trained based on an original training data set including input data and its labels to discriminate whether a client theft behavior has occurred according to a user behavior pattern. In order to improve the robustness of the model, counter training may be used to enhance the robustness of the model. The existing construction method of the countermeasure sample is single, so that the model obtained by the countermeasure training is robust only for specific attacks, and cannot cope with various attack means in a stealing scene. By perturbing the input data using an automated confrontational training method according to the present disclosure, and further training the model using the updated input data and its labels after the perturbation over the predetermined number of steps as a confrontational training data set, the robustness of the model against the perturbation may be optimized and enhanced. Since the disturbance space can be predefined and the optimal solution to each step of disturbance is automatically determined, the efficiency and the effect of the confrontation training are greatly improved.

FIG. 7 illustrates a flow diagram of an automatic perturbation attack submethod 700 in accordance with an aspect of the present disclosure. The automatic perturbation attack submethod 700 of fig. 7 may be used, for example, in block 604 of fig. 6 to automatically perturb the input data by a predetermined number of steps. For example, sub-method 700 may come from block 602 as described above in connection with fig. 6.

As shown in FIG. 7, at decision block 702, a determination is made whether a predetermined number of steps have been attacked. If not, then the sub-method 700 proceeds to block 704.

At block 704, the submethod 700 may include automatically searching for a hyper-parameter value in a predefined attack hyper-parameter search space that is optimal for the current step of the perturbation. According to an exemplary embodiment, the hyper-parameters may include, for example, but are not limited to, perturbation operations (e.g., perturbation operators) and step sizes that make up each step of the attack unit. For example, automatically searching for a hyper-parameter value that is optimal for the current perturbation step may include automatically searching for a hyper-parameter value that is optimal for the current perturbation step using the previously described approach by calculating scores for candidate values of the hyper-parameter using, for example, an attention (attention) mechanism.

At block 706, the sub-method 700 may include automatically determining the perturbation for the current step based on the optimal hyper-parameter value. For example, automatically determining the perturbation for the current step based on the optimal hyper-parameter value may include constructing the perturbation from the gradient determined based on the selection of the optimal hyper-parameter value, as previously described.

At block 708, the submethod 700 may include updating the input data by incorporating the perturbations of the current step. For example, including the perturbation for the current step to update the input data may include adding the input data to the perturbation for the current step.

The sub-method 700 then returns to decision block 702 to determine whether a predetermined number of steps have been attacked. If not, then the sub-method 700 proceeds to block 704. If so, then the submethod 700 ends and may proceed to block 606, described above in connection with FIG. 6, for example.

The method 600 and sub-method 700 described above in connection with fig. 6 and/or fig. 7 may be implemented in various ways. For example, method 600 and sub-method 700 may be implemented in software, firmware, and/or hardware using automated machine learning techniques (i.e., based on AutoML). According to some example embodiments, the method 600 and the submethod 700 may be implemented by way of a processor executing a program stored in a memory. According to other exemplary embodiments, the method 600 and the sub-method 700 may be implemented in the form of processor-executable instructions stored on a computer-readable medium. Regardless of the manner in which it is implemented, method 600 and sub-method 700 may further incorporate and/or employ any of the schemes, techniques, and/or features described above

The technical scheme of the disclosure can be applied to detecting scenes such as client theft risk and the like, for example, when judging whether a client theft behavior occurs or not according to a user behavior mode through a neural network model, the robustness and performance of the model can be improved through disturbance attack. By means of the various attack modes formed by automatic combination, the defense performance of the model to different attacks can be improved, and better robustness is obtained. The construction of the countermeasure sample can be regarded as a means of data enhancement, and the performance of the model can be improved under the condition that the data quantity is insufficient.

What has been described above is merely exemplary embodiments of the present invention. The scope of the invention is not limited thereto. Any changes or substitutions that may be easily made by those skilled in the art within the technical scope of the present disclosure are intended to be included within the scope of the present disclosure.

The various illustrative logical blocks, modules, and circuits described in connection with the disclosure may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable Logic Device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any commercially available processor, controller, microcontroller or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

The steps of a method or algorithm described in connection with the disclosure may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may reside in any form of storage medium known in the art. Some examples of storage media that may be used include Random Access Memory (RAM), Read Only Memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, and so forth. A software module may comprise a single instruction, or many instructions, and may be distributed over several different code segments, among different programs, and across multiple storage media. A storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor.

The methods disclosed herein comprise one or more steps or actions for achieving the described method. The method steps and/or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and/or use of specific steps and/or actions may be modified without departing from the scope of the claims.

The processor may execute software stored on a machine-readable medium. A processor may be implemented with one or more general and/or special purpose processors. Examples include microprocessors, microcontrollers, DSP processors, and other circuitry capable of executing software. Software should be construed broadly to mean instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. By way of example, a machine-readable medium may include RAM (random access memory), flash memory, ROM (read only memory), PROM (programmable read only memory), EPROM (erasable programmable read only memory), EEPROM (electrically erasable programmable read only memory), registers, a magnetic disk, an optical disk, a hard drive, or any other suitable storage medium, or any combination thereof. The machine-readable medium may be embodied in a computer program product. The computer program product may include packaging material.

In a hardware implementation, the machine-readable medium may be a part of the processing system that is separate from the processor. However, as those skilled in the art will readily appreciate, the machine-readable medium, or any portion thereof, may be external to the processing system. By way of example, a machine-readable medium may include a transmission line, a carrier wave modulated by data, and/or a computer product separate from the wireless node, all of which may be accessed by a processor through a bus interface. Alternatively or additionally, the machine-readable medium or any portion thereof may be integrated into a processor, such as a cache and/or a general register file, as may be the case.

The processing system may be configured as a general purpose processing system having one or more microprocessors that provide processor functionality, and an external memory that provides at least a portion of the machine readable medium, all linked together with other supporting circuitry through an external bus architecture. Alternatively, the processing system may be implemented with an ASIC (application specific integrated circuit) having a processor, a bus interface, a user interface (in the case of an access terminal), support circuitry, and at least a portion of a machine readable medium integrated in a single chip, or with one or more FPGAs (field programmable gate arrays), PLDs (programmable logic devices), controllers, state machines, gated logic, discrete hardware components, or any other suitable circuitry, or any combination of circuitry that is capable of performing the various functionalities described throughout this disclosure. Depending on the particular application and the overall design constraints imposed on the overall system, those skilled in the art will recognize how to implement the functionality described with respect to the processing system.

The machine-readable medium may include several software modules. These software modules include instructions that, when executed by a device, such as a processor, cause the processing system to perform various functions. These software modules may include a transmitting module and a receiving module. Each software module may reside in a single storage device or be distributed across multiple storage devices. As an example, a software module may be loaded into RAM from a hard drive when a triggering event occurs. During execution of the software module, the processor may load some instructions into the cache to increase access speed. One or more cache lines may then be loaded into a general register file for execution by the processor. When referring to the functionality of a software module below, it will be understood that such functionality is implemented by the processor when executing instructions from the software module.

If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example and not limitationIn particular, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code in the form of instructions or data structures and which can be accessed by a computer. Any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a web site, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as Infrared (IR), radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk (disk) and disc (disc), as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, and

disks (disks) often reproduce data magnetically, while disks (disc) reproduce data optically with lasers. Thus, in some aspects, computer-readable media may comprise non-transitory computer-readable media (e.g., tangible media). Additionally, for other aspects, the computer-readable medium may comprise a transitory computer-readable medium (e.g., a signal). Combinations of the above should also be included within the scope of computer-readable media.

Accordingly, certain aspects may comprise a computer program product for performing the operations presented herein. For example, such a computer program product may include a computer-readable medium having instructions stored (and/or encoded) thereon, the instructions being executable by one or more processors to perform the operations described herein. In certain aspects, a computer program product may include packaging materials.

It is to be understood that the claims are not limited to the precise configuration and components illustrated above. Various changes, substitutions and alterations in the arrangement, operation and details of the method and apparatus described above may be made without departing from the scope of the claims.

Claims (25)

1. An automated confrontational training method comprising:

obtaining an original training data set and a model trained using the original training data set, the original training data set including input data and labels thereof;

automatically perturbing the input data by a predetermined number of steps, including for each of the predetermined number of steps:

automatically searching a hyper-parameter value optimal to the disturbance of the step in a predefined attack hyper-parameter search space;

automatically determining a perturbation of the step based on the optimal hyper-parameter value; and

updating the input data by incorporating perturbations of the step; and

further optimizing the model to combat the perturbation using a combat training data set comprising updated input data and its labels after the perturbation over the predetermined number of steps.

2. The automated countermeasure training method of claim 1, wherein the predefined attack hyper-parameter search space includes a hyper-parameter having candidate values, and the hyper-parameter includes at least a perturbation operation and a step size for each step.

3. The automated confrontational training method of claim 2, wherein the candidate values for perturbation operations comprise one or more perturbation operators, or any combination thereof, of: FGM, FGSM, FGMM, gaussian noise, and identity variation.

4. The automated confrontational training method of claim 2, wherein the candidate values for the step size comprise one or more of the following steps, or any combination thereof: 0.0001,0.001,0.01,0.1,1.

5. The automated confrontational training method of claim 1 wherein automatically perturbing the input data by a predetermined number of steps comprises entering the input data of the original training data set as a first perturbation step, entering the input data perturbed by each step as a next perturbation step, and entering the input data perturbed by the last perturbation step as the updated input data in the confrontational training data set.

6. The automated countermeasure training method of claim 2, wherein automatically searching for a hyperparameter value that is optimal for the perturbation of the step in a predefined attack hyperparameter search space comprises using an attention mechanism to determine a score for the hyperparameter over each candidate value and determining the hyperparameter value that is optimal for the perturbation of the step based on the score.

7. The automated confrontational training method of claim 6, wherein determining the score of the hyperparameter on each candidate value using an attention mechanism comprises:

determining a query based on the input data for the step;

embedding each candidate value as a corresponding key; and

and obtaining the attention distribution value of the query on each key as the score of the hyperparameter on the corresponding candidate value by using an attention mechanism.

8. The automated confrontational training method of claim 7 wherein determining the score of the hyperparameter at each candidate value using an attention mechanism further comprises calculating its score at the respective candidate value using a corresponding attention mechanism for the perturbation operation and the step size, respectively; and wherein

Determining a hyper-parameter value optimal for the perturbation of the step based on the score includes a normalization sampling unit using Gumbel Softmax as a score of the perturbation operation on its candidate value, and a normalization unit using Softmax as a score of the step size on its candidate value.

9. The automated confrontational training method of claim 8 wherein automatically determining the perturbation for the step based on the optimal hyper-parameter value comprises:

determining the perturbation of the step based on the optimal perturbation operation value and the optimal step size value and the gradient of the loss function of the model on the input data of the step.

10. The automated confrontational training method of claim 7 wherein determining a query based on the input data for this step comprises:

determining a gradient of a loss function of the model over the input data for this step; and

linearly projecting the gradient as the query.

11. The automated confrontational training method of claim 1, wherein using the confrontational training dataset including updated input data and its labels after the predetermined number of perturbations by the predetermined number of steps to further optimize the model comprises:

updating the model based on a gradient of a loss function of the model over the antagonistic training data set.

12. The automated confrontational training method of claim 1 wherein the input data includes a pattern of user behavior on a client's APP and the model is used to determine whether client misappropriation behavior has occurred.

13. An automated confrontational training apparatus comprising:

means for obtaining an original training data set and a model trained using the original training data set, the original training data set including input data and its labels;

means for automatically perturbing the input data by a predetermined number of steps, comprising for each of the predetermined number of steps:

automatically searching a hyper-parameter value optimal to the disturbance of the step in a predefined attack hyper-parameter search space;

automatically determining a perturbation of the step based on the optimal hyper-parameter value; and

updating the input data by incorporating perturbations of the step; and

means for further optimizing the model to combat the perturbation using an antagonistic training dataset comprising updated input data and its labels after the perturbation over the predetermined number of steps.

14. The automated countermeasure training apparatus of claim 13, wherein the predefined attack superparameter search space includes superparameters having candidate values, and the superparameters include at least perturbation operations and step sizes for perturbations of each step.

15. The automated confrontational training apparatus of claim 14 wherein the candidate values for perturbation operations comprise one or more perturbation operators, or any combination thereof: FGM, FGSM, FGMM, gaussian noise, and identity variation.

16. The automated confrontational training apparatus of claim 14 wherein the candidate values for the step size comprise one or more of the following steps or any combination thereof: 0.0001,0.001,0.01,0.1,1.

17. The automated confrontational training apparatus of claim 13 wherein the means for automatically perturbing the input data by a predetermined number of steps comprises means for taking input data from the original training data set as input for a first step perturbation, means for taking the input data after each step perturbation as input for a next step perturbation, and means for taking input data after a last step perturbation as the updated input data in the confrontational training data set.

18. The automated countermeasure training apparatus of claim 14, wherein the means for automatically searching in a predefined attack superparameter search space for a superparameter value that is optimal for the perturbation of the step comprises means for using an attention mechanism to determine a score for the superparameter on each candidate value and determine the superparameter value that is optimal for the perturbation of the step based on the score.

19. The automated resistance training apparatus of claim 18, wherein the means for determining the score of the hyperparameter on each candidate value using an attentiveness mechanism comprises:

means for determining a query based on the input data for the step;

means for embedding each candidate value as a corresponding key; and

means for obtaining an attention distribution value of the query on each key as a score of the hyperparameter on a corresponding candidate value using an attention mechanism.

20. The automated resistance training apparatus of claim 19, wherein the means for using an attention mechanism to determine the score of the hyperparameter on each candidate value further comprises means for using the corresponding attention mechanism to calculate its score on the respective candidate value for the perturbation operation and the stride length, respectively; and wherein

The means for determining the value of the hyperparametric value that is optimal for the perturbation of this step based on the score comprises means for using Gumbel Softmax as a normalization sampling unit of the score of the perturbation operation on its candidate values, and means for using Softmax as a normalization unit of the score of the step size on its candidate values.

21. The automated confrontational training apparatus of claim 20 wherein the means for automatically determining the perturbation of the step based on the optimal hyperparametric value comprises:

means for determining a perturbation of the step based on the optimal perturbation operation value and the optimal step size value and a gradient of a loss function of the model over input data of the step.

22. The automated confrontational training apparatus of claim 19 wherein the means for determining a query based on the input data for the step comprises:

means for determining a gradient of a loss function of the model over the input data for the step; and

means for linearly projecting the gradient as the query.

23. The automated confrontational training apparatus of claim 13 wherein the means for further optimizing the model using the confrontational training dataset comprising updated input data and its labels after the predetermined number of perturbations comprises:

means for updating the model based on a gradient of a loss function of the model over the antagonistic training data set.

24. The automated confrontational training apparatus of claim 13 wherein the input data comprises a pattern of user behavior on a client's APP and the model is used to determine whether client misappropriation behavior has occurred.

25. An automated confrontational training apparatus comprising:

a memory to store instructions; and

a processor coupled to the memory and configured to execute the instructions to:

obtaining an original training data set and a model trained using the original training data set, the original training data set including input data and labels thereof;

automatically perturbing the input data by a predetermined number of steps, including for each of the predetermined number of steps:

automatically searching a hyper-parameter value optimal to the disturbance of the step in a predefined attack hyper-parameter search space;

automatically determining a perturbation of the step based on the optimal hyper-parameter value; and

updating the input data by incorporating perturbations of the step; and

further optimizing the model to combat the perturbation using a combat training data set comprising updated input data and its labels after the perturbation over the predetermined number of steps.

Images