CN115496924A

CN115496924A - Data processing method, related equipment and storage medium

Info

Publication number: CN115496924A
Application number: CN202211199556.9A
Authority: CN
Inventors: 不公告发明人
Original assignee: Beijing Real AI Technology Co Ltd
Current assignee: Beijing Real AI Technology Co Ltd
Priority date: 2022-09-29
Filing date: 2022-09-29
Publication date: 2022-12-20

Abstract

The embodiment of the application relates to the technical field of artificial intelligence, and provides a data processing method which can be applied to a pre-training-fine-tuning model scene. The method comprises the following steps: acquiring target disturbance, wherein the target disturbance is related to a front N-layer structure of the first model, and N is a positive integer; acquiring a first antagonistic sample according to the target disturbance and the training data; and training the second model by using the first anti-pair sample, wherein the similarity between the first N-layer structure of the first model and the first N-layer structure of the second model is greater than a first preset threshold value. Because the model is in the fine tuning process, the change of the model low-layer structure before and after fine tuning is small. Or that lower level features of the model before the fine-tuning are more likely to be migrated to the model after the fine-tuning. Therefore, the target disturbance related to the first N layers of the first model is obtained, and then the second model can be trained according to the target disturbance, so that the robustness of the second model is improved.

Description

Data processing method, related equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of artificial intelligence, in particular to a data processing method and related equipment.

Background

As deep learning models are being widely used, the robustness of the models becomes a necessary issue. One factor that negatively impacts the accuracy of the model is fighting the sample. The challenge sample is a sample obtained by adding some noise to the original sample. And the countermeasure sample has no significant change in human cognition compared with the original sample (for example, the image after noise is added has no significant change in human eyes compared with the original image), but the corresponding processing result of the model is wrong.

In the pre-training-fine-tuning model scenario, an upstream vendor generates a pre-training model based on pre-training samples and uploads the pre-training model to the cloud for provision to customers. The customer trims the pre-trained model based on different tasks to arrive at a trimmed model (often using the client's local data to trim the pre-trained model). And then the fine tuning model is applied to different tasks. Under the pre-training-fine-tuning model scenario, the robustness of the fine-tuning model is worth being evaluated.

Therefore, how to generate noise data capable of effectively detecting/improving robustness of the fine tuning model is an urgent technical problem to be solved.

Disclosure of Invention

The embodiment of the application provides a data processing method, which can obtain target disturbance for evaluating/improving the robustness of a fine-tuned model.

In a first aspect, an embodiment of the present application provides a data processing method from a first device perspective, which may be applied to a pre-training-fine-tuning model scenario. The method may be performed by the first device, or may be performed by a component of the first device (e.g., a processor, a chip, or a system of chips, etc.). The method comprises the following steps: acquiring target disturbance, wherein the target disturbance is related to a front N-layer structure of the first model, and N is a positive integer; acquiring a first antagonistic sample according to the target disturbance and the training data; and training the second model by using the first anti-pair sample, wherein the similarity between the first N-layer structure of the first model and the first N-layer structure of the second model is greater than a first preset threshold value.

Optionally, in a possible implementation manner of the first aspect, the step of: acquiring a target disturbance, comprising: obtaining candidate disturbance, wherein the candidate disturbance is obtained according to historical disturbance; obtaining a first initial sample of a first model; obtaining a second antagonizing sample according to the candidate disturbance and the first initial sample; if the calculated value of the first characteristic diagram corresponding to the second antagonizing sample is smaller than or equal to a second preset threshold, updating the candidate disturbance until a preset condition is met; the first characteristic diagram is output information obtained by inputting the first N layers of structures of the first model into the second antagonizing sample; the preset conditions include at least one of the following: the number of times of updating candidate disturbance is larger than a third preset threshold, and the calculated value is larger than a second preset threshold; and determining the candidate disturbance meeting the preset condition as a target disturbance.

Alternatively, in a possible implementation manner of the first aspect, the calculated value is a negative value of a sum of squares of each element in the first feature map.

Optionally, in a possible implementation manner of the first aspect, the step of: obtaining a first initial sample of a first model, comprising: and receiving a second initial sample sent by the second device, wherein the second initial sample is part of training data of a third model, and the similarity between the first N-layer structure of the third model and the first N-layer structure of the first model is greater than a fourth preset threshold. And then acquiring a first initial sample based on the second initial sample.

Optionally, in a possible implementation manner of the first aspect, the obtaining, based on the second initial sample, the first initial sample includes: sampling from the uniform distribution to obtain a target mean value and a target variance; and performing Gaussian sampling on the second initial sample according to the target mean and the target variance to obtain a first initial sample.

Optionally, in a possible implementation manner of the first aspect, the step further includes: and sending first data to the second device, wherein the first data comprises target disturbance or a first antagonistic sample, the first data is used for detecting the robustness of the third model, and the similarity between the first N-layer structure of the third model and the first N-layer structure of the first model is greater than a fourth preset threshold.

In a second aspect, an embodiment of the present application provides a data processing method from a second device perspective, which may be applied to a pre-training-fine-tuning model scenario. The method may be performed by the second device, or may be performed by a component of the second device (e.g., a processor, a chip, or a system of chips, etc.). The method comprises the following steps: receiving first data sent by first equipment, wherein the first data comprise target disturbance or a first antagonistic sample, the target disturbance is related to a first N-layer structure of a first model, N is a positive integer, and the first antagonistic sample is obtained based on the target disturbance and training data; acquiring a second anti-target sample, wherein the second anti-target sample is obtained by combining target disturbance and input data, or the second anti-target sample is a first anti-target sample; and detecting the robustness of the third model according to the second antagonizing sample, wherein the similarity between the front N-layer structure of the third model and the front N-layer structure of the first model is greater than a fourth preset threshold value.

Optionally, in a possible implementation manner of the second aspect, the first data includes a target disturbance, the input data is an input image, and the second antagonistic sample is an antagonistic image; detecting the robustness of the third model based on the second antagonizing samples, comprising: and inputting the counterimage into the third model to obtain an output result, wherein the output result represents that the counterimage attack is successful.

Optionally, in a possible implementation manner of the second aspect, the foregoing steps further include: and sending a second initial sample to the first device, wherein the second initial sample is part of training data of the third model, and the second initial sample is used for the second device to determine the first initial sample used in the process of updating the target disturbance.

A third aspect of the embodiments of the present application provides a first device, which may be applied to a pre-trained-fine-tuned model scenario. The first device has a function of implementing the data processing method provided corresponding to the first aspect described above. The functions may be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules may be software and/or hardware.

In some embodiments, the first device comprises:

the processing module is used for acquiring target disturbance, the target disturbance is related to the front N-layer structure of the first model, and N is a positive integer; acquiring a first antagonistic sample according to the target disturbance and the training data; and training the second model by using the first anti-pair sample, wherein the similarity between the first N-layer structure of the first model and the first N-layer structure of the second model is greater than a first preset threshold value.

A fourth aspect of the embodiments of the present application provides a second device, which may be applied to a pre-training-fine-tuning model scenario. The second device has a function of implementing the data processing method provided corresponding to the second aspect described above. The functions may be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules may be software and/or hardware.

In some embodiments, the second device comprises: the receiving and sending module is used for receiving first data sent by first equipment, the first data comprises target disturbance or a first antagonistic sample, the target disturbance is related to a first N-layer structure of the first model, N is a positive integer, and the first antagonistic sample is obtained based on the target disturbance and training data; the processing module is used for acquiring a second anti-target sample, wherein the second anti-target sample is obtained by combining target disturbance and input data, or the second anti-target sample is a first anti-target sample; and the processing module is further used for detecting the robustness of a third model according to the second antagonizing sample, and the similarity between the front N-layer structure of the third model and the front N-layer structure of the first model is greater than a fourth preset threshold.

A fifth aspect of embodiments of the present application provides a computer device, including: a processor coupled to a memory for storing a program or instructions which, when executed by the processor, cause the computer device to carry out the method performed by the first device in any of the possible implementations of the first aspect or the first aspect described above, or cause the computer device to carry out the method performed by the first device in any of the possible implementations of the second aspect or the second aspect described above.

A sixth aspect of an embodiment of the present application provides a communication system, including: the first apparatus of the third aspect and/or the second apparatus of the fourth aspect.

A seventh aspect of embodiments of the present application provides a computer program product or a computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in the first aspect or the various alternative implementations of the first aspect. Or cause the computer device to perform a method as provided in the second aspect or in various alternative implementations of the second aspect.

In summary, it can be seen that, since the first N layers of the first model are similar to the first N layers of the second model, the change of the first model before fine tuning and the second model after fine tuning is small. Or that lower level features of the first model before the fine-tuning are more likely to be able to be migrated to the second model after the fine-tuning. The target perturbation associated with the first N-level structure of the first model is also associated with the first N-level structure of the second model. Therefore, the target disturbance can capture low-level features in the second model better, and then the second model can be trained better according to the first antagonistic sample obtained by the target disturbance, so that the robustness of the second model is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings used in the description of the embodiments and the prior art will be briefly described below.

Fig. 1A is a schematic diagram of an application scenario provided in an embodiment of the present application;

fig. 1B is another schematic diagram of an application scenario provided in the embodiment of the present application;

fig. 2 is a schematic flowchart of a data processing method according to an embodiment of the present application;

fig. 3 is another schematic flow chart of a data processing method according to an embodiment of the present application;

fig. 4A is another schematic flow chart of a data processing method according to an embodiment of the present disclosure;

fig. 4B is an exemplary diagram in a face recognition scene according to an embodiment of the present application;

fig. 5A is a schematic flowchart of another data processing method according to an embodiment of the present application;

fig. 5B is an exemplary diagram in a face recognition scene according to an embodiment of the present application;

fig. 6 is another schematic flow chart of a data processing method according to an embodiment of the present application;

fig. 7 is another schematic flowchart of a data processing method according to an embodiment of the present application;

fig. 8 is another schematic flow chart of a data processing method according to an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a first apparatus provided in an embodiment of the present application;

FIG. 10 is a schematic structural diagram of a second apparatus provided in an embodiment of the present application;

FIG. 11 is a schematic structural diagram of a computer device provided in an embodiment of the present application;

fig. 12 is another schematic structural diagram of a second apparatus provided in an embodiment of the present application.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. As can be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solution provided in the embodiments of the present application is also applicable to similar technical problems.

In the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of singular or plural items. The terms "first model," "second model," "third model," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are merely descriptive of the various embodiments of the application and how objects of the same nature can be distinguished. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of modules is not necessarily limited to those modules, but may include other modules not expressly listed or inherent to such process, method, article, or apparatus.

The scheme of the embodiment of the present application can be implemented based on technologies such as Artificial Intelligence (AI), natural Language Processing (NLP), machine Learning (ML), and specifically described by the following embodiments:

AI is a theory, method, technique and application system that simulates, extends and expands human intelligence, senses the environment, acquires knowledge and uses knowledge to obtain the best result by using a digital computer or a machine controlled by a digital computer. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

With the research and progress of artificial intelligence technology, the artificial intelligence technology is developed and applied in a plurality of fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, unmanned aerial vehicles, robots, smart medical care, smart customer service, and the like.

Currently, in a pre-training-fine-tuning model scenario, an upstream vendor generates a pre-training model based on pre-training samples and uploads the pre-training model to the cloud for provision to customers. The customer trims the pre-trained model based on different tasks to arrive at a trimmed model (often using the client's local data to trim the pre-trained model). And then the fine tuning model is applied to different tasks. Under the pre-training-fine-tuning model scenario, the robustness of the fine-tuning model is worth being evaluated.

Therefore, how to generate noise data capable of effectively detecting/improving robustness of the fine tuning model is a technical problem to be solved urgently.

In order to solve the above problem, an embodiment of the present application provides a data processing method: by obtaining the target disturbance related to the first N layers of the structure of the first model (namely, the pre-training model), the second model (namely, the fine-tuning model) can be trained according to the target disturbance, and therefore the robustness of the second model is improved.

The following describes the data processing method provided in the embodiment of the present application in detail. The method is applied to a pre-training-fine-tuning model scene. Specifically, the cloud-terminal interaction may be also performed, or the terminal-terminal interaction may also be also performed, and the details are not limited herein.

In one possible implementation, as shown in fig. 1A, in a cloud-terminal interaction scenario. The scene includes a cloud device 101, a terminal device 102, a terminal device 103, and a terminal device 104. The cloud device 101 stores a pre-training model, and the cloud device 101 acquires and stores first data (target disturbance or a sample on which the target disturbance is superimposed) based on the pre-training model, which will be described in the following fig. 2 in a specific process, and is not expanded here. In this way, the terminal device 102, the terminal device 103, and the terminal device 104 may download the first data from the cloud device 101 for use in training, detecting, and the like of the fine-tuning model in the corresponding downstream task. The fine tuning model is a model obtained by fine tuning the pre-training model through local data of the terminal device (for example, terminal device 102, terminal device 103, and terminal device 104). The similarity of the pre-training model and the first N layers of structures of the fine tuning model is larger than a certain threshold value. Or after the terminal device 102, the terminal device 103, and the terminal device 104 generate the first data, the first data is sent to the cloud device 101. In this way, other terminals of the downstream task may download the first data from the cloud device 101 for use in operations such as model training in the corresponding downstream task. It can be understood that, after the cloud device 101 generates the countermeasure sample based on the target disturbance, the attack effect may be simulated first. And if the attack effect is better, the target disturbance is issued to the terminal equipment 102, the terminal equipment 103 and the terminal equipment 104.

It can be understood that the cloud-terminal interaction scenario shown in fig. 1A is only exemplified by one cloud device and three terminal devices, and in practical applications, the number of the cloud devices may be multiple, and the number of the terminal devices may be at least one, and is not limited herein.

In another possible implementation, as shown in fig. 1B, in a terminal-terminal interaction scenario. The first terminal device 201 stores a pre-training model, and after acquiring and generating first data (target disturbance or a sample superimposed with the target disturbance) based on the pre-training model, the terminal device 201 may send the first data to the second terminal device 202 of the downstream task through wireless connection or wired connection, so that the second terminal device 202 executing the downstream task performs counterattack on the downstream task according to the first data. For example, the first data is used to train, detect, etc. a stored fine-tuning model at the second terminal device 102.

It can be understood that the terminal-terminal interaction scenario shown in fig. 1B is only exemplified by two terminal devices, and in practical applications, the number of the terminal devices may also be greater than 2, and is not limited herein.

The cloud device may be a server or a server cluster of the cloud, or a container or a virtual machine of the cloud. The terminal device may be a mobile phone (mobile phone), a tablet computer (pad), a computer with a wireless transceiving function, a Virtual Reality (VR) terminal, an Augmented Reality (AR) terminal, a wireless terminal in industrial control (industrial control), a wireless terminal in self driving (self driving), a wireless terminal in remote medical (remote medical), a wireless terminal in smart grid (smart grid), a wireless terminal in transportation safety (transportation safety), a wireless terminal in smart city (smart city), a wireless terminal in smart home (smart home), a wireless terminal in internet of things (IoT), and the like.

The data processing method provided by the embodiment of the application can be applied to a model scene of pre-training-fine tuning. Referring to fig. 2, a flow chart of a data processing method according to an embodiment of the present application is schematically illustrated, where the method may be executed by a first device. Or may be performed by a component of the first device, such as a processor, chip, or system of chips. The first device may be a cloud device or a terminal device. The method may comprise steps 201 to 203. Step 201 to step 203 will be described in detail below.

Step 201, obtaining target disturbance.

In the embodiment of the present application, there are various ways to obtain the target disturbance, which may be a way to generate the target disturbance (which will be described in detail later), a way to receive a transmission from another device, a way to select from a database, and the like, and the details are not limited herein.

Wherein the target disturbance is related to the first N layers of the first model, and N is a positive integer. The first model is a trained pre-trained model. The second model is a model (or called a fine tuning model) after the fine tuning of the pre-training model. I.e. the second model is trimmed from the first model. And the similarity between the front N layer structure of the first model and the front N layer structure of the second model is greater than or equal to a first preset threshold value.

The preset threshold (e.g., the first preset threshold, the second preset threshold, the third preset threshold, the fourth preset threshold, etc.) in the embodiment of the present application is specifically set according to actual needs, and is not limited herein.

In the embodiment of the present application, there are multiple ways for acquiring N, which are described below:

first, N may be a predetermined threshold, which is typically a positive integer greater than 1, but which is typically smaller.

Second, N is related to the number of structural layers included in the first model. For example, the first model includes an M-layer network structure, where N is M/5, or M/10, etc., where M is an integer greater than N.

Thirdly, N is determined for the decomposed first model. Module () function is used, for example, to decompose a first model into a plurality of first level submodules. N may be 1, that is, the output information of the first N-layer structure is the output value of the first sub-module.

It should be understood that, the above several determination manners of N are only examples, and in practical applications, N may also have other determination manners, which are not limited herein.

In addition, the first N-layer structure of the model can be understood as a shallow layer of the model. The shallow layer is used for outputting a low-level feature map of the countermeasure data. The low-level feature map may include information of the low-level features. In the neural network, the low-level semantic information described by the low-level features, for example, the data type of the input sample is used as an image, and the low-level features of the input sample may include one or more of features such as color, edge, contour, texture, and the like.

The specific functions and structures of the models (e.g., the first model, the second model, the third model, etc.) in the embodiments of the present application may be various, and are not limited herein. Illustratively, the model may be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Transformer network, or the like.

Illustratively, the model is an image processing model. At this point, the countermeasure data for the model may include one or more images with the initial noise data. Illustratively, the model may implement one or more of the following functions: and identifying, denoising, image style transferring, character extracting and the like of the image. Of course, in other examples, the model may be used to process data such as speech, text, etc.

In one possible implementation, the first device generates the target disturbance as an example. The process of generating the target disturbance by the first device comprises the following steps: firstly, candidate disturbance is obtained, and the candidate disturbance is obtained according to historical disturbance. A first initial sample of a first model is obtained. And secondly, obtaining a second antagonizing sample according to the candidate disturbance and the first initial sample of the first model. And if the output value of the first feature map corresponding to the second countermeasure sample is smaller than or equal to a second preset threshold, updating the candidate disturbance until the preset condition is met, wherein the first feature map is output information obtained by inputting the second countermeasure sample into the first N-layer structure of the first model. The preset conditions include at least one of: the number of times of updating the candidate disturbance is larger than a third preset threshold, and the calculated value is larger than a second preset threshold. And finally, determining the candidate disturbance meeting the preset condition as the target disturbance.

The initial sample may be image, text, voice, etc. data. Accordingly, the candidate perturbations may refer to initial noise data for image, text, speech, etc. data. The initial noise data may also be understood as a pre-set perturbation of the initial samples.

In the initialization stage, the candidate perturbation may be a small value relative to the initial sample, and specifically, the initial value of the candidate perturbation may be smaller than a certain preset threshold. At this time, the value of each element in the candidate perturbation may be smaller than a preset threshold, or a certain norm (e.g., infinite norm, F norm, L2 norm) of the candidate perturbation may be smaller than a certain preset threshold. In this way, the candidate perturbation can be updated and adjusted conveniently in the subsequent processing process. In addition, there are various manners of obtaining the first initial sample, which may be a manner selected from a database, a manner transmitted by the second device, or the like, and the specific details are not limited herein. For example, the first device receives a second initial sample sent by the second device, the second initial sample being part of training data of a third model at the second device. In one aspect, the second initial sample may be taken as the first initial sample. Alternatively, the target mean and target variance may be based on samples taken from the uniform distribution. And carrying out Gaussian sampling on the second initial sample according to the target mean value and the target variance to obtain a first initial sample. In this way, the small amount of downstream task data can be used to perform fine adjustment (e.g., replacement, etc.) on the attack sample, so as to achieve a better attack effect on the model of the downstream task. The similarity between the first N-layer structure of the third model and the first N-layer structure of the first model is larger than a fourth preset threshold value. Alternatively, the first model is a pre-trained model and the third model is a fine-tuned model. The computed value may be a norm (e.g., infinite norm, F norm, L2 norm). For example, the calculated value is the negative of the sum of squares of each element in the first feature map.

In another possible implementation manner, the above process of generating the target perturbation may also be understood as building a loss function of the first N-layer structure of the first model, taking the second antagonizing sample as an input of the first model, and training candidate perturbations in the second antagonizing sample to obtain the target perturbation, with a value of the loss function (i.e., the above calculated value) being smaller than a certain threshold as a target. Namely, the second antagonizing sample is input into the first model, and the output information of the first N-layer structure of the first model is obtained. And then constructing a loss function based on the output information, and training candidate disturbance in a second countermeasure sample based on the loss function to obtain target disturbance.

There are many cases of the loss function in the embodiment of the present application, and the loss function is exemplified by using L2 norm as an example. In practical applications, the loss function may also use an L1 norm, an L3 norm, and the like, which is not limited herein. Here, the loss function using the L2 norm may be understood as a negative value of the sum of squares of each element in the output information of the top N-layer structure used in the above training process.

In this case, the value of the loss function (or referred to as a loss value) may reflect the amount of valid feature information in the output information, or may be considered as the amount of invalid feature information in the output information. Wherein invalid feature information refers to information that does not contribute to the first model obtaining accurate output information about the second antagonizing sample.

And adding invalid information in the subsequent updating process of the initial disturbance through a loss function to cover the valid information, so that invalid characteristic information in the characteristics such as the low-layer characteristic diagram and the like identified by the first N layers is increased, and finally generating the target disturbance. I.e., the smaller the penalty value, the sum of the squares of each element in the output information can be maximized as much as possible, so that a large number of noise points that do not contribute to obtaining accurate processing results can be activated to mask the useful information. In this way, after the update is finished, the obtained target perturbation can activate more ineffective low-level features in the first N-level structure of the first model for realizing the attack effect of the second pair of anti-samples.

It can be appreciated that to improve the update efficiency of candidate perturbations. Gradient descent methods may also be used during training. The gradient descent method may be based on a mapped gradient (PGD), a Basic Iterative Method (BIM), and the like, and is not limited herein.

Step 202, a first countermeasure sample is obtained according to the target disturbance and the training data.

After the first device acquires the target disturbance, a first contrast sample may be acquired according to the target disturbance and the training data.

In addition, there are various ways to obtain the first antagonizing sample according to the training data and the target perturbation, for example, the training data and the target perturbation may be added or weighted to obtain the first antagonizing sample. For example, the target perturbation is fusion-superimposed with the training data to obtain a first pair of anti-samples.

The training data described above is similar to the initial samples described above, and may be image, text, voice, and the like. Accordingly, the target disturbance may refer to noise data of image, text, voice, and the like data.

In some examples, the data dimensions and data types of the training data and the target perturbation may be the same, and at this time, the training data and the target perturbation may be subjected to addition of corresponding elements, that is, matrix addition operation, to obtain a first antagonistic sample; or, when the training data is text data, the training data may be converted into a word vector (word embedding), and then matrix-added with the target perturbation in the form of a vector to obtain the first countermeasure sample.

Step 203, train a second model using the first antagonizing sample.

After the first device acquires the first antagonizing sample, the second model can be trained using the first antagonizing sample.

Specifically, a first countermeasure sample is input into the second model to obtain an output value, and the difference between the output value and the label value of the training data is reduced to train the second model. And obtaining a trained second model until the stopping condition is met. The stop condition includes at least one of: the difference between the output value of the second model to the first antagonizing sample and the label value is less than or equal to a threshold, the training times are greater than or equal to a certain threshold, and the like.

In this embodiment, since the first N-layer structure of the first model is similar to the first N-layer structure of the second model, the change of the first model before fine tuning and the second model after fine tuning is small. Or that lower level features of the first model before the fine-tuning are more likely to be able to be migrated to the second model after the fine-tuning. The target perturbation associated with the first N-level structure of the first model is also associated with the first N-level structure of the second model. Therefore, the target disturbance can capture low-level features in the second model better, and then the second model can be trained better according to the first antagonistic sample obtained by the target disturbance, so that the robustness of the second model is improved.

The first device in the embodiment shown in fig. 2 may be a cloud device or a terminal device. In the case that the first device is a cloud device, fig. 3 is another data processing method provided in the embodiment of the present application, and is applied to a pre-training-fine-tuning model scene. The cloud device generates target disturbance through the pre-training model and sends the target disturbance to the terminal device of the downstream task, so that the terminal device can detect/attack the fine-tuning model according to the target disturbance. Referring to fig. 3, another flow chart of the data processing method provided in the embodiment of the present application is schematically illustrated, and the method may be executed by a first device and a second device in combination. The first device may be a cloud device and the second device is a terminal device. The method may comprise steps 301 to 306. Step 301 to step 306 will be described in detail below.

Step 301, the first device obtains a target disturbance.

Step 302, the first device obtains a first contrast sample according to the target disturbance and the training data. This step is optional.

In step 303, the first device trains a second model using the first antagonizing sample. This step is optional.

Steps 301 to 303 in this embodiment are similar to steps 201 to 203 in the embodiment shown in fig. 2, and the specific process may refer to the foregoing description, which is not described herein again.

Step 304, the first device sends the first data to the second device.

The first device sends the first data to the second device, and correspondingly, the second device receives the first data sent by the first device. The first data includes target perturbation or first antagonizing samples, which are described below in conjunction with fig. 4A and 5A from the issuing of target perturbation and the issuing of first antagonizing samples, respectively.

In the first mode, the first device issues target disturbance to the second device.

As shown in fig. 4A, the process includes steps 401 through 403, described below.

Step 401, the first device sends a target disturbance to the second device.

And after acquiring the target disturbance, the first equipment sends the target disturbance to the second equipment. Correspondingly, the second device receives the target disturbance sent by the first device.

Step 402, the second device obtains a second impedance sample according to the target disturbance and the input sample.

And after the second device acquires the target disturbance, acquiring a second impedance sample according to the target disturbance and the input sample.

Specifically, the second device superposes and fuses the target disturbance to the input sample to obtain a second antagonizing sample.

Illustratively, the input samples are images, text, video, audio, and the like. The following description will be exemplarily described by taking the input sample as an image and the second antagonizing sample as a sample image.

In step 403, the second device detects the robustness of the third model according to the second contrast sample.

After the second device acquires the second antagonizing sample, a third model robustness is detected according to the second antagonizing sample. I.e. inputting the second countermeasure sample into the third model, and checking whether the inference result is correct (or understanding to check the attack effect of the second countermeasure sample).

Illustratively, as shown in FIG. 4B. After the first device obtains the target disturbance through the above process, the target disturbance can be used to perform simulated attack on the first model at the first device to check the attack effect. As shown in fig. B4, after the first device acquires the target disturbance, the target disturbance is superimposed on the input image of the user 1 to obtain a first contrast sample. And inputting the first anti-sample into the first model for recognition to obtain a recognition result. The recognition result is user 2. It is stated that the first model identification can be made incorrect, i.e. the simulation attack is successful, by adding the target perturbation. And after the simulation is successful, the first equipment sends target disturbance to the second equipment. The second device superimposes the target perturbation on the input image of the user 3 to obtain a second contrast sample. And inputting the second antagonizing sample into the third model for recognition to obtain a recognition result. The recognition result is the user 4. It is stated that the third model identification can be made wrong by adding target perturbations, i.e. the attack is successful. This example shows that the target perturbation can not only attack in the first model, but also be migrated to various fine tuning models (i.e. the third model), so as to realize effective attack on the fine tuning models. Namely, the migration effect of the target disturbance is good.

Therefore, after the target disturbance is obtained by the first device, the target disturbance is sent to the second device, so that the second device generates a countermeasure sample based on the target disturbance, the robustness of the third model can be detected according to the countermeasure sample, and effective attack of the third model is achieved. Namely, the migration effect of the target disturbance is better.

In the second mode, the first device issues a first countermeasure sample to the second device.

As shown in fig. 5A, the process includes steps 501 through 503, described below.

In step 501, a first device sends a first antagonizing sample to a second device.

After the first device acquires the first antagonizing sample, the first antagonizing sample is sent to the second device. Accordingly, the second device receives the first antagonizing sample transmitted by the first device.

Step 502, the second device takes the first antagonizing sample as a second antagonizing sample.

After the second device acquires the first antagonizing sample, the first antagonizing sample is used as a second antagonizing sample.

In step 503, the second device detects the robustness of the third model according to the second antagonizing sample.

This step 503 is similar to the step 403 in the embodiment shown in fig. 4A, and is not described here again.

Illustratively, as shown in FIG. 5B. After the first device obtains the target disturbance through the above process, the target disturbance can be used to perform simulated attack on the first model at the first device to check the attack effect. As shown in fig. B4, after the first device acquires the target disturbance, the target disturbance is superimposed on the input image of the user 1 to obtain a first contrast sample. And inputting the first anti-sample into the first model for recognition to obtain a recognition result. The recognition result is user 2. It is stated that the first model identification can be made incorrect, i.e. the simulation attack is successful, by adding the target perturbation. After the simulation is successful, the first device sends a first antagonizing sample to the second device. The second device takes the first antagonizing sample as a second antagonizing sample. And inputting the first anti-sample/the second anti-sample into a third model for recognition to obtain a recognition result. The recognition result is the user 4. It is stated that the first challenge sample may cause the third model to identify the error, i.e., the attack was successful. This example can see that the first countermeasure sample can not only attack in the first model, but can also be migrated to various fine tuning models (i.e., the third model), enabling effective attack on the fine tuning models. Namely, the first pair of resists with target disturbance has better migration effect.

Therefore, after the first device generates the first countermeasure sample, the first countermeasure sample is sent to the second device, so that the second device detects the robustness of the third model by using the first countermeasure sample, and effective attack of the third model is realized.

The deployment modes of the first mode and the second mode can be selected according to actual needs.

For example, when the function of the first model at the first device is different from the function of the third model at the second device or the input data at the first device is different from the input data at the second device, the recognition tasks may be different due to the different functions of the models, assuming that the function of the first model is male face recognition and the function of the third model is female face recognition. In such an example, the features of the male face may be slightly different from the female face, which makes the first pair of anti-samples of the first model inapplicable to the third model. Therefore, the deployment mode of the first mode can be selected, and the target disturbance is issued to the second equipment through the first equipment. And the second device superposes the target disturbance on the input sample suitable for the third model to obtain a second antagonizing sample. The second antagonizing sample is obtained from the target perturbation acquired by the first device and the input sample at the second device. Therefore, the second pair of anti-samples is more beneficial to improve the robustness of the third model.

Also for example, when the function of the first model at the first device approximates the function of the third model at the second device or the input data at the first device approximates the input data at the second device. For example, as shown in the foregoing example of fig. 5B, the first model and the third model both function as male face recognition. In this example, since the male face recognition has a certain commonality, the deployment mode of the second mode may be selected. That is, the first device issues the first countermeasure sample to the second device. The second device takes the first antagonizing sample as a second antagonizing sample. Since the first model and the third model both function as face recognition. And the first model is similar to the first N-layer structure of the third model. Therefore, the second device can detect/improve the robustness of the third model through the first countermeasure sample issued by the first device.

Step 305, the second device obtains a second antagonizing sample.

After the second device acquires the first data, a second antagonizing sample can be acquired based on the first data.

Optionally, if the first data is a target disturbance, the second device may combine the target disturbance with the input data to obtain a second antagonizing sample. The second antagonizing sample contains target disturbance related to the first model, so that the second antagonizing sample can detect the robustness of the third model after the first model is trimmed effectively.

Alternatively, if the first data is a first antagonizing sample, the second device may treat the first antagonizing sample as a second antagonizing sample. Under the condition, the processing of the second device can be reduced, and the detection efficiency of the subsequent third model is improved.

Step 306, the second device detects the robustness of the third model according to the second contrast sample.

After the second device acquires the second antagonizing sample, the robustness of the third model can be detected based on the second antagonizing sample. The third model is fine-tuned by the first device. Or the similarity between the first N-layer structure of the third model and the first N-layer structure of the first model is larger than a fourth preset threshold.

In one possible implementation, the second antagonizing sample is input into the third model to detect whether the inference result of the third model is correct (specifically, whether the difference between the inference result and the label value of the second antagonizing sample is too large). Or, as understood, attack the third model with the challenge sample and observe the attack results.

In another possible implementation manner, the second device may train the third model using the second antagonizing sample, thereby improving the robustness of the third model.

In this embodiment, on one hand, the target disturbance obtained by pre-training the low-level structure of the model is small in change of the low-level structure of the model before and after fine tuning. Or that lower level features of the model before the fine-tuning are more likely to be migrated to the model after the fine-tuning. Therefore, the target disturbance obtained through the low-level structure of the pre-training model is more beneficial to evaluating/improving the robustness of the model after fine tuning. On the other hand, after the pre-training attack sample (i.e., the second countermeasure sample) is obtained, if a small amount of downstream task data (i.e., part of training data) can be obtained, fine adjustment can be performed on the attack sample, so that a better attack effect is achieved.

In order to verify the beneficial effects of the data processing method provided by the embodiment of the present application, the following objective perturbation obtained by using the data processing method provided by the embodiment of the present application is performed in different scenarios such as different pre-training models, different downstream tasks, different pre-training modes, and the like,

for example, in an exemplary scenario one of the embodiments of the present application, a developer performs pre-training on the Resnet101 model by using a simcrv 2 training method, and obtains the pre-training model after the pre-training is completed. In addition, ten downstream classification task data sets can be obtained, and the pre-training model is finely adjusted on the ten downstream classification task data sets to obtain corresponding fine-adjustment models. Then, based on the above embodiment, the target disturbance is obtained through the pre-training model, and then the target disturbance and the input sample are fused to obtain the confrontation sample. The test results on the ten downstream classification tasks show that the countercheck sample can achieve a good attack effect even if the information of the downstream tasks is not known.

For another example, in an exemplary scenario two of the embodiments of the present application, a Resnet50 model obtained by pre-training using a simclv 2 training method may be selected as a pre-training model for testing, and other procedures are the same as those in the exemplary scenario one. In the testing process, the countervailing samples generated based on the target disturbance have good attack effect on each downstream task.

For another example, in the third exemplary scenario of the embodiment of the present application, a vision transformer (ViT) model pre-trained by a mask auto-coders (MAE) training method may be selected as a pre-training model for testing, and other procedures are the same as those in the first exemplary scenario. In the testing process, the countervailing samples generated based on the target disturbance have good attack effect on each downstream task.

Therefore, through the embodiment of the application, the target disturbance can be obtained by utilizing the first N-layer structure of the pre-training model, and then the target disturbance and the input sample are fused to obtain the confrontation sample. At this point, the target perturbation contained in the challenge sample may activate some invalid features at a given layer for achieving the effect of an attack against the sample. Due to the correlation between the pre-training model and the fine-tuning model, the countervailing sample can attack the corresponding fine-tuning model in various downstream tasks.

In addition, with respect to the manner of acquiring the first initial sample in the foregoing embodiments shown in fig. 2 to 5A, the following description is made with reference to fig. 6. Referring to fig. 6, another flow chart of the data processing method according to the embodiment of the present application is shown, where the method can be executed by the first device or the second device. The first device may be a cloud device and the second device is a terminal device. The following is an exemplary description of the method being performed by the first device only as an example. The method may include steps 601 to 603. The following describes steps 601 to 603 in detail.

Step 601, a second initial sample is obtained.

In the embodiment of the present application, there are various ways to obtain the second initial sample, which may be a way of collecting the second initial sample locally, a way of sending the second initial sample through receiving another device (for example, a second device), a way of selecting the second initial sample from a database, and the like, and the details are not limited herein.

Step 602, a target mean and a target variance are obtained from the uniform distribution by sampling.

In some cases, the mean and variance of the first initial sample used in the process of updating the target perturbation by the first device side may be greatly different from the mean and variance of the data of the downstream task, and at this time, if part of the training data used by the second device side to train the third model may be used as the first initial sample, the attack effect of the target perturbation trained from the first initial sample on the model (for example, the third model) of the downstream task may be improved.

In this embodiment, the first device may obtain one or more sets of mean and variance as one or more sets of target mean and target variance by means of random sampling or the like from a preset uniform distribution in advance. The upper and lower boundaries of the preset uniform distribution can be predetermined by related developers based on experience, experiments and other modes; alternatively, the determination may be based on the mean and variance of the data in the downstream task data set used to input the fine-tuning model in the downstream task for processing.

Step 603, performing gaussian sampling on the second initial sample according to the target mean and the target variance to obtain a first initial sample.

After the first device obtains the target mean and the target variance, the second initial sample may be subjected to gaussian sampling according to the target mean and the target variance to obtain a first initial sample.

Specifically, the first device may perform gaussian sampling on the second initial sample according to each group of target mean and target variance, and obtain a gaussian sample (i.e., a first initial sample) corresponding to each group of target mean and target variance. In this manner, the first initial sample includes at least one gaussian sample, each gaussian sample being obtained by gaussian sampling based on a target mean and a target variance, the target mean and the target variance being obtained by sampling from a uniform distribution.

At this time, the mean and variance of the training data may not be limited to the first device-side data set, and the training data is prevented from being over-fitted to the first device-side data set, but have a certain diversity, so that the target perturbation updated according to the first initial sample may be used to attack various downstream tasks. I.e. due to the characteristics (e.g. mean, variance) of the data at the first device side and the training data of the downstream tasks may be very different. This may reduce overfitting of the training data, thereby improving migration to other models (e.g., subsequent second and third models) as compared to gaussian sampling with fixed mean and variance.

In this embodiment, compared with the method that gaussian sampling is performed by using a fixed mean and variance, overfitting of training data can be reduced by the method of this embodiment, so that it is promoted that disturbance data generated according to the training data can be migrated to other models (e.g., subsequent fine tuning models), and further attack effect is promoted or robustness of the fine tuning models is promoted.

Fig. 7 is a case that, in step 601 in fig. 6, the first device obtains the second initial sample in a manner of receiving the second initial sample sent by the second device. Referring to fig. 7, another flow chart of the data processing method according to the embodiment of the present application is shown, where the method includes steps 701 to 703.

In step 701, the second device sends a second initial sample to the first device.

The second device sends a second initial sample to the first device. Accordingly, the first device receives a second initial sample sent by the second device. The second initial sample is part of the training data for the third model at the first device.

Before the first device obtains the target disturbance subsequently, the first device may obtain a part of training data used by the second device side to train the third model, and obtain the first initial sample according to the part of training data through subsequent steps 702 and 703, so that an attack effect of the target disturbance obtained by training according to the first initial sample on the model (for example, the third model) of the downstream task may be improved.

In step 702, the first device samples from the uniform distribution to obtain a target mean and a target variance.

And 703, the first device performs gaussian sampling on the second initial sample according to the target mean and the target variance to obtain a first initial sample.

Steps 702 and 703 in this embodiment are similar to

steps

602 and 603 in the embodiment shown in fig. 6, and are not repeated herein.

Therefore, in the embodiment of the application, the second device can acquire part of the training data of the third model at the first device by sending the second initial sample to the first device through the second device, so that the first initial sample acquired according to the part of the training data can generate target disturbance having an attack effect on the third model. Therefore, the attack effect on the model of the downstream task can be improved through the target disturbance.

Fig. 8 is another schematic flow chart of a data processing method according to an embodiment of the present application. After the target disturbance is obtained by the embodiments shown in fig. 2 to 7, the target disturbance may be sent to the downstream user 1 and the downstream user 2, respectively. The downstream user 1 superimposes the target disturbance on the fusion sample 1 to obtain the countermeasure sample 1. And inputting the confrontation sample 1 into the fine tuning model 1 for recognition to obtain a wrong recognition result 1. The downstream user 2 superimposes the target perturbation on the fusion sample 2 to obtain the confrontation sample 2. And inputting the confrontation sample 2 into the fine tuning model 2 for recognition to obtain a wrong recognition result 2. For example, sample 1 of the original tortoise is recognized as a snail after adding the target perturbation input fine tuning model 1. For example, sample 2 of original rhinoceros is recognized as a elephant after adding the target disturbance input fine tuning model 2. It can be seen that the fine tuning model 1 and the fine tuning model 2 can output corresponding error results respectively by the superposition of the target disturbance and the downstream local data.

That is, the attacker generates a target attack sample (PAP) using the first model low-level features without knowing the specific downstream task. At this time, if the user fine-tunes the same pre-training model to obtain another model (for example, a second model) and applies the model to a downstream task, an attacker can directly use the previously obtained attack sample to attack the pre-training model at this time. It is considered that the change of the parameters of the lower layer is generally small during the fine tuning of the model, and when the model is fine-tuned in the downstream task, the lower layer features are more likely to be migrated into the fine tuning model. Therefore, in this example, the output information based on the first N-layer structure is subsequently processed, so that the low-layer features in the countermeasure data finally obtained based on the target disturbance can be attacked not only in the pre-training model, but also in various fine tuning models, thereby realizing effective attack on the fine tuning models.

Any technical feature mentioned in the embodiment corresponding to any one of fig. 1A to 8 is also applicable to the embodiment corresponding to fig. 9 to 12 in the embodiment of the present application, and the details of the subsequent similarities are not repeated. The data processing method in the embodiment of the present application is described above, and the following describes the device in the embodiment of the present application, please refer to fig. 9, and a schematic structural diagram of a first device shown in fig. 9 is applicable to generating a target perturbation/first contrast sample, so as to send the target perturbation/first contrast sample to a second device, so that the target perturbation/first contrast sample can detect the third model robustness. The first device in the embodiment of the present application can implement the steps executed by the first device in the embodiment corresponding to any one of fig. 1A to 8 described above. The functions realized by the first device may be realized by hardware, or may be realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, which may be software and/or hardware. The first device may comprise a processing module 901 and a transceiver module 902. The processing module 901 and the transceiver module 902 may refer to operations executed by the first device in any embodiment corresponding to fig. 1A to 8.

In some embodiments, the processing module 901 may be configured to obtain a target perturbation, where the target perturbation is related to a first N-layer structure of the first model, and N is a positive integer;

the processing module 901 is further configured to obtain a first countermeasure sample according to the target disturbance and the training data;

the processing module 901 is further configured to train a second model using the first countermeasure sample, where a similarity between the first N-layer structure of the first model and the first N-layer structure of the second model is greater than a first preset threshold.

Optionally, the transceiver module 902 is configured to receive part of training data from a third model sent by the second device, where a similarity between a first N-layer structure of the third model and a first N-layer structure of the first model is greater than a fourth preset threshold;

optionally, the transceiver module 902 is further configured to send a target perturbation to the second device, where the target perturbation is used to detect robustness of the third model, and a similarity between a first N-layer structure of the third model and a first N-layer structure of the first model is greater than a fourth preset threshold.

In this embodiment, operations performed by the modules in the first device are similar to those described in the embodiments shown in fig. 2 to fig. 8, and are not described again here.

In this embodiment, in the fine tuning process of the model, the change of the low-layer structure of the model before and after the fine tuning is small. Or that lower level features of the model before the fine-tuning are more likely to be migrated to the model after the fine-tuning. Therefore, the processing module 901 can train the second model based on the target disturbance by obtaining the target disturbance related to the first N layers of the first model, so as to evaluate/improve the robustness of the model after fine tuning.

Referring to fig. 10, an embodiment of a second apparatus in an embodiment of the present application includes:

the receiving and sending module 1001 is configured to receive first data sent by a first device, where the first data includes a target disturbance or a first antagonistic sample, the target disturbance is related to a first N-layer structure of a first model, N is a positive integer, and the first antagonistic sample is obtained based on the target disturbance and training data;

the processing module 1002 is configured to obtain a second anti-collision sample, where the second anti-collision sample is obtained by combining the target disturbance and the input data, or the second anti-collision sample is the first anti-collision sample;

the processing module 1002 is further configured to detect robustness of a third model according to the second antagonistic sample, where a similarity between a first N-layer structure of the third model and a first N-layer structure of the first model is greater than a fourth preset threshold.

Optionally, the transceiver module 1001 is further configured to send, to the first device, a part of training data of the third model, where the part of training data is used to update the target perturbation.

In this embodiment, operations executed by the modules in the second device are similar to those described in the embodiments shown in fig. 2 to fig. 8, and are not repeated here.

In this embodiment, in the fine tuning process of the model, the change of the low-layer structure of the model before and after the fine tuning is small. Or that lower level features of the model before the fine-tuning are more likely to be migrated to the model after the fine-tuning. Therefore, the transceiver module 1001 may evaluate/improve the robustness of the post-fine-tuning model by receiving a target perturbation related to the first N-layer structure of the first model.

Referring to fig. 11, the present application provides a schematic structural diagram of a computer device, where the computer device is malicious as the first device or the second device in the foregoing embodiment. The computer device may include a processor 1101, a memory 1102, and a communication port 1103. The processor 1101, memory 1102 and communication port 1103 are interconnected by wires. Wherein program instructions and data are stored in memory 1102.

The memory 1102 stores program instructions and data corresponding to the steps performed by the first device/second device in the corresponding embodiments shown in fig. 2-8.

A processor 1101 configured to perform the steps performed by the first device/the second device according to any one of the embodiments shown in fig. 2 to fig. 8.

The communication port 1103 may be configured to receive and transmit data, and is configured to perform the steps related to the acquiring, transmitting, and receiving in any of the embodiments shown in fig. 2 to fig. 8.

In one implementation, the computer device may include more or fewer components than those shown in FIG. 11, which are merely illustrative and not limiting.

As shown in fig. 12, for convenience of illustration, only the parts related to the embodiments of the present application are shown, and details of the specific technology are not disclosed, please refer to the method part of the embodiments of the present application. The second device may be any second device including a mobile phone, a tablet computer, a Personal Digital Assistant (PDA, for short), a Point of sale (POS, for short), a vehicle-mounted computer, and the like, where the second device is a mobile phone:

fig. 12 is a block diagram illustrating a partial structure of a mobile phone related to a second device provided in an embodiment of the present application. Referring to fig. 12, the handset includes: radio Frequency (RF) circuit 1210, memory 1220, input unit 1230, display unit 1240, sensor 1280, audio circuit 1260, wireless-fidelity (Wi-Fi) module 1270, processor 1280, and power 1290. Those skilled in the art will appreciate that the handset configuration shown in fig. 12 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

The following describes each component of the mobile phone in detail with reference to fig. 12:

the RF circuit 1210 may be configured to receive and transmit signals during a message transmission or a call, and in particular, receive downlink information of a base station and then process the downlink information to the processor 1280; in addition, data for designing uplink is transmitted to the base station. In general, RF circuit 1210 includes, but is not limited to, an antenna, at least one Amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the RF circuit 1210 may also communicate with networks and other devices via wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to Global System for Mobile communication (GSM), general Packet Radio Service (GPRS), code Division Multiple Access (CDMA), wideband Code Division Multiple Access (WCDMA), long Term Evolution (LTE), e-mail, short Message Service (SMS), etc.

The memory 1220 may be used to store software programs and modules, and the processor 1280 executes various functional applications and data processing of the mobile phone by operating the software programs and modules stored in the memory 1220. The memory 1220 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 1220 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 1230 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the cellular phone. Specifically, the input unit 1230 may include a touch panel 1231 and other input devices 1232. The touch panel 1231, also referred to as a touch screen, can collect touch operations of a user (e.g., operations of the user on or near the touch panel 1231 using any suitable object or accessory such as a finger, a stylus, etc.) thereon or nearby, and drive the corresponding connection device according to a preset program. Alternatively, the touch panel 1231 may include two portions, a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, and sends the touch point coordinates to the processor 1280, and can receive and execute commands sent by the processor 1280. In addition, the touch panel 1231 may be implemented by various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. The input unit 1230 may include other input devices 1232 in addition to the touch panel 1231. In particular, other input devices 1232 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit 1240 may be used to display information input by the user or information provided to the user and various menus of the cellular phone. The Display unit 1240 may include a Display panel 1241, and optionally, the Display panel 1241 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, touch panel 1231 can overlay display panel 1241, and when touch panel 1231 detects a touch operation thereon or nearby, the touch panel 1231 can transmit the touch operation to processor 1280 to determine the type of the touch event, and then processor 1280 can provide a corresponding visual output on display panel 1241 according to the type of the touch event. Although in fig. 12, the touch panel 1231 and the display panel 1241 are implemented as two independent components to implement the input and output functions of the mobile phone, in some embodiments, the touch panel 1231 and the display panel 1241 may be integrated to implement the input and output functions of the mobile phone.

The handset can also include at least one sensor 1280, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1241 according to the brightness of ambient light, and the proximity sensor may turn off the display panel 1241 and/or the backlight when the mobile phone moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when stationary, and can be used for applications of recognizing the gesture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured on the mobile phone, the description is omitted here.

Audio circuitry 1260, speaker 1261, and microphone 1262 can provide an audio interface between a user and a cell phone. The audio circuit 1260 can transmit the received electrical signal converted from the audio data to the speaker 1261, and the audio signal is converted into a sound signal by the speaker 1261 and output; on the other hand, the microphone 1262 converts the collected sound signals into electrical signals, which are received by the audio circuit 1260 and converted into audio data, which are processed by the audio data output processor 1280, and then passed through the RF circuit 1210 to be transmitted to, for example, another cellular phone, or output to the memory 1220 for further processing.

Wi-Fi belongs to short-distance wireless transmission technology, and a mobile phone can help a user to receive and send emails, browse webpages, access streaming media and the like through a Wi-Fi module 1270, and provides wireless broadband internet access for the user. Although fig. 12 shows the Wi-Fi module 1270, it is understood that it does not belong to the essential constitution of the cellular phone, and may be omitted entirely within the scope not changing the essence of the application as needed.

The processor 1280 is a control center of the mobile phone, connects various parts of the entire mobile phone by using various interfaces and lines, and performs various functions of the mobile phone and processes data by operating or executing software programs and/or modules stored in the memory 1220 and calling data stored in the memory 1220, thereby performing overall monitoring of the mobile phone. Optionally, processor 1280 may include one or more processing units; preferably, the processor 1280 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It is to be appreciated that the modem processor described above may not be integrated into the processor 1280.

The mobile phone further includes a power supply 1290 (e.g., a battery) for supplying power to each component, and the power supply may be logically connected to the processor 1280 through a power management system, so that the power management system may manage charging, discharging, and power consumption.

Although not shown, the mobile phone may further include a camera, a bluetooth module, etc., which are not described herein.

In this embodiment, the processor 1280 included in the mobile phone further has a function of controlling and executing the method flow executed by the second device shown in fig. 12. The steps performed by the second device in the above embodiments may be based on the handset configuration shown in fig. 12. For example, the processor 1280, by invoking the instructions in the memory 1232, performs the following:

acquiring input data to be processed through the input unit 1230;

and obtaining a second antagonizing sample according to the input data acquired by the input unit 1230 and the target disturbance, and inputting the second antagonizing sample to a third model and the like.

In another embodiment of the present application, a computer-readable storage medium is further provided, in which computer-executable instructions are stored, and when the processor of the device executes the computer-executable instructions, the device executes the steps executed by the processor in fig. 11 or fig. 12.

In another embodiment of the present application, there is also provided a computer program product comprising computer executable instructions stored in a computer readable storage medium; when the processor of the device executes the computer-executable instructions, the device performs the steps performed by the processor of FIG. 11 or FIG. 12 described above.

In another embodiment of the present application, a chip system is further provided, where the chip system includes a processor, and the processor is configured to implement the steps performed by the processor of fig. 11 or fig. 12. In one possible design, the system-on-chip may further include a memory, for storing program instructions and data necessary for the means for data writing. The chip system may be formed by a chip, or may include a chip and other discrete devices.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.

It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working processes of the system, the apparatus, and the module described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of modules is merely a division of logical functions, and an actual implementation may have another division, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.

Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules are integrated into one module.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application, which are essential or part of the technical solutions contributing to the prior art, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only a specific embodiment of the embodiments of the present application, but the scope of the embodiments of the present application is not limited thereto.

Claims

1. A data processing method, applied to a first device, the method comprising:

obtaining target disturbance, wherein the target disturbance is related to a front N-layer structure of a first model, and N is a positive integer;

acquiring a first antagonistic sample according to the target disturbance and the training data;

and training a second model by using the first antagonizing sample, wherein the similarity of the first N-layer structure of the first model and the first N-layer structure of the second model is greater than a first preset threshold value.

2. The method of claim 1, wherein the obtaining a target perturbation comprises:

obtaining candidate disturbance, wherein the candidate disturbance is obtained according to historical disturbance;

obtaining a first initial sample of the first model;

obtaining a second impedance sample according to the candidate disturbance and the first initial sample;

if the calculated value of the first feature map corresponding to the second antagonizing sample is smaller than or equal to a second preset threshold, updating the candidate disturbance until a preset condition is met; the first feature map is output information obtained by inputting the first N-layer structure of the first model into the second antagonizing sample, and the preset condition includes at least one of: the number of times of updating candidate disturbance is larger than a third preset threshold, and the calculated value is larger than the second preset threshold;

and determining the candidate disturbance meeting the preset condition as the target disturbance.

3. The method of claim 2, wherein the computed value is a negative of a sum of squares of each element in the first profile.

4. The method of claim 2 or 3, wherein said obtaining a first initial sample of said first model comprises:

receiving a second initial sample sent by a second device, wherein the second initial sample is part of training data of a third model, and the similarity between the first N-layer structure of the third model and the first N-layer structure of the first model is greater than a fourth preset threshold;

obtaining the first initial sample based on the second initial sample.

5. The method of claim 4, wherein said obtaining the first initial sample based on the second initial sample comprises:

sampling from the uniform distribution to obtain a target mean value and a target variance;

and performing Gaussian sampling on the second initial sample according to the target mean and the target variance to obtain the first initial sample.

6. The method of claim 1, further comprising:

sending first data to a second device, wherein the first data comprises the target disturbance or the first antagonistic sample, the first data is used for detecting the robustness of a third model, and the similarity between the first N-layer structure of the third model and the first N-layer structure of the first model is greater than a fourth preset threshold.

7. A data processing method applied to a second device, the method comprising:

receiving first data sent by the first device, wherein the first data comprise target disturbance or a first antagonistic sample, the target disturbance is related to a first N-layer structure of a first model, N is a positive integer, and the first antagonistic sample is obtained based on the target disturbance and training data;

acquiring a second anti-target sample, wherein the second anti-target sample is obtained by combining the target disturbance with input data, or the second anti-target sample is the first anti-target sample;

and detecting the robustness of a third model according to the second antagonistic sample, wherein the similarity between the front N layer structure of the third model and the front N layer structure of the first model is greater than a fourth preset threshold value.

8. The method of claim 7, wherein the first data comprises the target perturbation, the input data is an input image, and the second antagonizing sample is an antagonizing image;

the detecting robustness of the third model according to the second antagonizing sample comprises:

and inputting the counterimage into the third model to obtain an output result, wherein the output result represents that the counterimage attack is successful.

9. The method of claim 7 or 8, further comprising:

sending a second initial sample to the first device, wherein the second initial sample is part of training data of the third model, and the second initial sample is used for the second device to determine the first initial sample used in the process of updating the target disturbance.

10. A first device, comprising:

the processing module is used for acquiring target disturbance, wherein the target disturbance is related to a front N-layer structure of the first model, and N is a positive integer;

the processing module is further used for obtaining a first antagonistic sample according to the target disturbance and the training data;

the processing module is further configured to train a second model using the first countermeasure sample, and a similarity between a first N-layer structure of the first model and a first N-layer structure of the second model is greater than a first preset threshold.

11. A second apparatus, comprising:

a transceiver module, configured to receive first data sent by the first device, where the first data includes a target disturbance or a first antagonistic sample, the target disturbance is related to a first N-layer structure of a first model, N is a positive integer, and the first antagonistic sample is obtained based on the target disturbance and training data;

the processing module is used for acquiring a second anti-target sample, wherein the second anti-target sample is obtained by combining the target disturbance and input data, or the second anti-target sample is the first anti-target sample;

the processing module is further configured to detect robustness of a third model according to the second countermeasure sample, and similarity between a first N-layer structure of the third model and a first N-layer structure of the first model is greater than a fourth preset threshold.

12. A computer storage medium comprising computer instructions which, when run on a computer device, cause the computer device to perform the method of any of claims 1 to 6 as performed by a first device or to perform the method of any of claims 7 to 9 as performed by a second device.