CN113792892A

CN113792892A - Federal learning modeling optimization method, apparatus, readable storage medium, and program product

Info

Publication number: CN113792892A
Application number: CN202111151142.4A
Authority: CN
Inventors: 康焱; 吴岳洲
Original assignee: WeBank Co Ltd
Current assignee: WeBank Co Ltd
Priority date: 2021-09-29
Filing date: 2021-09-29
Publication date: 2021-12-14

Abstract

The application discloses a federated learning modeling optimization method, equipment, a readable storage medium and a program product, which are applied to first equipment, wherein the federated learning modeling optimization method comprises the following steps: generating a first sample characteristic corresponding to the first training sample according to the characteristic extraction model, and generating a second sample characteristic corresponding to the first noise data and the first real classification label together according to the characteristic generation model; performing iterative training according to the first real classification label, the first sample characteristic and the second sample characteristic to obtain a characteristic generation model; sending the feature generation model and the classification model to second equipment so that the second equipment can construct a target global feature generation model and a target global classification model; and generating a model according to the target global features, and performing iterative optimization on the feature extraction model and the target global classification model to obtain a target feature extraction model and a target classification model. The method and the device solve the technical problem that the federated learning method has the risk of revealing the privacy of the data of the participants.

Description

Federal learning modeling optimization method, apparatus, readable storage medium, and program product

Technical Field

The present application relates to the field of artificial intelligence in financial technology (Fintech), and in particular, to a method, device, readable storage medium, and program product for optimizing federal learning modeling.

Background

With the continuous development of financial science and technology, especially internet science and technology, more and more technologies (such as distributed technology, artificial intelligence and the like) are applied to the financial field, but the financial industry also puts higher requirements on the technologies, for example, higher requirements on the distribution of backlog in the financial industry are also put forward.

With the continuous development of computer software, artificial intelligence and big data cloud service application, currently, each participant maintains a local model and a global model in a federal learning scenario. Each participant learns local unique knowledge through a local model, shares knowledge of all participants through a global model, and then each participant aggregates the local model and the global model together. However, this method has the disadvantage that the federal server can reversely deduce the original data of the participants through the global model of each participant, so that there is a risk of data leakage, that is, the existing federal learning method has a risk of revealing the privacy of the data of the participants.

Disclosure of Invention

The main purpose of the present application is to provide a federated learning modeling optimization method, device, readable storage medium, and program product, which aim to solve the technical problem that the federated learning method in the prior art has a risk of revealing privacy of participant data.

In order to achieve the above object, the present application provides a federated learning modeling optimization method, where the federated learning modeling optimization method is applied to a first device, and the federated learning modeling optimization method includes:

acquiring a trained feature extraction model and a classification model, and extracting a first training sample, first noise data and a first real classification label corresponding to the first training sample;

acquiring first sample features generated by the feature extraction model aiming at the first training sample and second sample features generated by the feature generation model to be trained aiming at the first noise data and the first real classification label;

classifying the second sample features through the classification model, performing sample distinguishing on the first sample features and the second sample features through a to-be-trained sample distinguishing model, and performing iterative optimization on the to-be-trained feature generation model under the condition that the feature extraction model and the classification model are fixed to obtain a feature generation model;

sending the feature generation model and the classification model to second equipment, so that the second equipment iteratively optimizes a global feature generation model obtained by aggregating the feature generation models and a global classification model obtained by aggregating the classification models according to the feature generation model sent by each first equipment to obtain a target global feature generation model and a target global classification model;

and receiving a target global feature generation model and a target global classification model sent by the second equipment, and performing iterative optimization on the feature extraction model and the target global classification model according to the target global feature generation model to obtain a target feature extraction model and a target classification model.

The application provides a federated learning modeling optimization method, which is applied to a second device and comprises the following steps:

receiving feature generation models and classification models sent by each first device, aggregating the feature generation models into a global feature generation model and aggregating the classification models into a global classification model;

extracting noise data and a real classification label corresponding to the noise data;

iteratively optimizing the global feature generation model and the global classification model by performing iterative training on the global feature generation model and the global classification model and performing knowledge distillation between the global feature generation model and each feature generation model according to the noise data and the real classification label to obtain a target global feature generation model and a target global classification model;

and feeding the target global feature generation model and the target global classification model back to each first device respectively, so that the first devices perform iterative optimization on a feature extraction model corresponding to the feature generation model and the target global classification model according to the target global feature generation model to obtain a target feature extraction model and a target classification model.

The application also provides a federal learning optimization device that models, federal learning optimization device that models is virtual device, just federal learning optimization device that models is applied to first equipment, federal learning optimization device that models includes:

the extraction module is used for acquiring a trained feature extraction model and a classification model, and extracting a first training sample, first noise data and a first real classification label corresponding to the first training sample;

the feature generation module is used for acquiring first sample features generated by the feature extraction model aiming at the first training sample and second sample features generated by the feature generation model to be trained aiming at the first noise data and the first real classification label;

the iterative training module is used for classifying the second sample features through the classification model, performing sample distinguishing on the first sample features and the second sample features through a to-be-trained sample distinguishing model, and performing iterative optimization on the to-be-trained feature generation model under the condition that the feature extraction model and the classification model are fixed to obtain a feature generation model;

the sending module is used for sending the feature generation models and the classification models to second equipment so that the second equipment can iteratively optimize the global feature generation models obtained by aggregating the feature generation models and the global classification models obtained by aggregating the classification models according to the feature generation models sent by the first equipment to obtain a target global feature generation model and a target global classification model;

and the iterative optimization module is used for receiving the target global feature generation model and the target global classification model sent by the second equipment, and performing iterative optimization on the feature extraction model and the target global classification model according to the target global feature generation model to obtain a target feature extraction model and a target classification model.

The application also provides a federal learning optimization device that models, federal learning optimization device that models is virtual device, just federal learning optimization device that models is applied to the second equipment, federal learning optimization device that models includes:

the model aggregation module is used for receiving the feature generation models and the classification models sent by the first devices, aggregating the feature generation models into a global feature generation model and aggregating the classification models into a global classification model;

the extraction module is used for extracting noise data and real classification labels corresponding to the noise data;

the iterative optimization module is used for iteratively optimizing the global feature generation model and the global classification model by performing iterative training on the global feature generation model and the global classification model and performing knowledge distillation between the global feature generation model and each feature generation model according to the noise data and the real classification label to obtain a target global feature generation model and a target global classification model;

and the feedback module is used for respectively feeding the target global feature generation model and the target global classification model back to each first device, so that the first devices perform iterative optimization on the feature extraction model corresponding to the feature generation model and the target global classification model according to the target global feature generation model to obtain the target feature extraction model and the target classification model.

The application also provides a federal learning modeling optimization device, the federal learning modeling optimization device is an entity device, the federal learning modeling optimization device includes: a memory, a processor, and a program of the federated learning modeling optimization method stored on the memory and executable on the processor, the program of the federated learning modeling optimization method when executed by the processor may implement the steps of the federated learning modeling optimization method as described above.

The present application also provides a readable storage medium having stored thereon a program for implementing the federal learning modeling optimization method, the program implementing the steps of the federal learning modeling optimization method as described above when executed by a processor.

The present application also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of the method of federated learning modeling optimization as described above.

Compared with the prior art, each participant maintains a local model and a global model in a federated learning scene. According to the technical means, firstly, a trained feature extraction model and a classification model are obtained, a first training sample, first noise data and a first real classification label corresponding to the first training sample are extracted, then, first sample features generated by the feature extraction model aiming at the first training sample and second sample features generated by a to-be-trained feature generation model aiming at the first noise data and the first real classification label are obtained, then, the second sample features are classified through the classification model and are subjected to sample distinguishing through a to-be-trained sample distinguishing model, performing iterative optimization on the feature generation model to be trained under the condition of fixing the feature extraction model and the classification model to obtain a feature generation model, achieving the purpose of constructing the feature generation model carrying model knowledge of the feature extraction model, simultaneously constructing the feature generation model not directly according to original sample data of a participant, further sending the feature generation model and the classification model to second equipment so that the second equipment can obtain a target global feature generation model and a target global classification model by iteratively optimizing a global feature generation model obtained by aggregating each feature generation model and a global classification model obtained by aggregating each classification model according to the feature generation model sent by each first equipment, and further constructing the feature generation model not directly according to the original sample data of the participant, and then the second device can not reversely push the original sample data of the first device, and then receives the target global feature generation model sent by the second device, and according to the target global feature generation model, the feature extraction model and the classification model are subjected to iterative optimization, so that the feature extraction model and the classification model can learn the knowledge of all participants carried by the target global feature generation model and the target global classification model, and further obtain the target feature extraction model and the target classification model, thereby realizing the purpose of prompting the local model to learn the model knowledge of the global model, namely, sharing the knowledge of all participants through the global model, and simultaneously not revealing the local original sample data, so the technical defect that the federal server can reversely push the original data of the participants through the global model of each participant, and therefore, the risk of data disclosure is overcome, the risk of revealing privacy data of the participants in federal learning is reduced.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.

FIG. 1 is a schematic flow chart diagram of a first embodiment of a federated learning modeling optimization method of the present application;

FIG. 2 is a schematic flow chart of a feature generation model constructed in the Federal learning modeling optimization method of the present application;

FIG. 3 is a schematic flow chart diagram of a second embodiment of the federated learning modeling optimization method of the present application;

fig. 4 is a schematic flow chart of the second device constructing a target global feature extraction model and a target global classification model in the federal learning modeling optimization method of the present application;

fig. 5 is a schematic device structure diagram of a hardware operating environment related to the federal learning modeling optimization method in the embodiment of the present application.

The objectives, features, and advantages of the present application will be further described with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The embodiment of the application provides a federated learning modeling optimization method, which is applied to a first device, and in the first embodiment of the federated learning modeling optimization method, referring to fig. 1, the federated learning modeling optimization method includes:

step S10, acquiring a trained feature extraction model and a classification model, and extracting a first training sample, first noise data and a first real classification label corresponding to the first training sample;

in this embodiment, it should be noted that the federal learning modeling optimization method is applied to a federal learning scenario, the federal learning scenario may be a horizontal federal learning scenario, the first device is a party involved in federal learning, the second device is a federal server involved in federal learning, the feature extraction model and the classification model are local models that are locally iteratively trained on the first device, the first noise data is used to be mixed with the first real classification label as an input of the feature generation model to be trained, and even if the second device reversely pushes the privacy data of the party according to the trained feature generation model, the obtained privacy data is also a mixture of the first noise data and the first real classification label, the first real classification label of the party cannot be obtained, and the leakage risk of the first real classification label can be reduced.

In addition, the process of local iterative training of the feature extraction model and the classification model is as follows:

extracting a local training sample and a local classification label corresponding to the local training sample, then enabling the local training sample to sequentially pass through a feature extraction model to be trained and a classification model to be trained to obtain a training output classification label, further calculating model loss according to the training output classification label and the local classification label corresponding to the local training sample, if the model loss is converged, taking the feature extraction model to be trained as a feature extraction model and taking the classification model to be trained as a classification model, if the model loss is not converged, updating the feature extraction model to be trained and the classification model to be trained according to a model gradient calculated by the model loss, and returning to the execution step: and extracting local training samples and local classification labels corresponding to the local training samples.

Step S20, obtaining a first sample feature generated by the feature extraction model for the first training sample, and a second sample feature generated by the feature generation model to be trained for the first noise data and the first real classification label;

in this embodiment, specifically, according to the feature extraction model, feature extraction is performed on the first training sample to obtain a first sample feature, and according to the feature generation model to be trained, the first noise data and the first real classification label are jointly converted into a second sample feature.

Step S30, classifying the second sample features through the classification model, performing sample distinguishing on the first sample features and the second sample features through a to-be-trained sample distinguishing model, and performing iterative optimization on the to-be-trained feature generation model under the condition that the feature extraction model and the classification model are fixed to obtain a feature generation model;

in this embodiment, specifically, the second sample feature is classified by the classification model, the feature generation model to be trained is iteratively updated with the classification model fixed, the first sample feature and the second sample feature are sample-distinguished by the sample-distinguishing model, and the feature generation model to be trained and the sample-distinguishing model to be trained are iteratively updated with the feature extraction model fixed, so as to optimize the feature generation model to be trained, and obtain the feature generation model.

Wherein the classifying the second sample feature by the classification model and the sample distinguishing the first sample feature from the second sample feature by a to-be-trained sample distinguishing model, and the iteratively optimizing the to-be-trained feature generation model while fixing the feature extraction model and the classification model to obtain a feature generation model comprises:

step S31, classifying the second sample characteristics through the classification model, performing sample discrimination on the first sample characteristics and the second sample characteristics through a first sample discrimination model to be trained, and calculating a first classification loss and a sample discrimination loss;

in this embodiment, it should be noted that the sample distinguishing model is a mixed structure sample that distinguishes whether the sample corresponding to the sample feature is a real sample or a noise data and a label.

Specifically, the second sample features are classified through the classification model to obtain a first prediction classification label, and a first classification loss is calculated according to the first prediction classification label and the first real classification label. In addition, according to a first sample distinguishing model to be trained, sample distinguishing is carried out on the first sample characteristics and the second sample characteristics respectively to obtain each first sample distinguishing prediction result, and then sample distinguishing loss is calculated according to each first sample distinguishing result, a real sample distinguishing label corresponding to the first sample characteristics and a real sample distinguishing label corresponding to the second sample characteristics.

Wherein the step of classifying the second sample feature by the classification model, and performing sample discrimination on the first sample feature and the second sample feature by a first sample discrimination model to be trained, and calculating a first classification loss and a sample discrimination loss comprises:

step S311, classifying the second sample characteristics according to the classification model to obtain a first prediction classification label;

step S312, calculating a first classification loss according to the first predicted classification label and the first real classification label;

in this embodiment, specifically, the second sample features are classified according to the classification model to obtain a first prediction classification label, and then a first classification loss is calculated through a preset loss function according to a difference between the first prediction classification label and the first real classification label, where the preset loss function may be an L2 loss function or a cross entropy loss function, and the like.

Step S313, respectively carrying out secondary classification on the first sample characteristic and the second sample characteristic according to the sample distinguishing model to obtain a secondary classification result;

in this embodiment, it should be noted that the sample distinguishing model may be a two-classification model, which is used to perform two classifications on the first sample feature and the second sample feature to realize sample distinguishing of the first sample feature and the second sample feature, for example, it is assumed that true two-classification labels are labels 0 and 1, respectively, where 0 represents that the corresponding sample feature is an output of the feature generation model, 1 represents that the corresponding sample feature is an output of the feature extraction model, and the output two-classification label of the sample distinguishing model may be represented as a probability value that the corresponding sample feature is an output of the feature extraction model.

Step S314, calculating the sample discrimination loss according to the two classification results and the positive and negative sample labels corresponding to the first sample feature and the second sample feature.

In this embodiment, it should be noted that the two classification results include a first two classification result corresponding to the first sample feature and a second two classification result corresponding to the second sample feature, and the positive and negative sample labels are set true two classification labels, for example, the true two classification label corresponding to the first sample feature may be set as a positive sample label, specifically may be 1, the true two classification label corresponding to the second sample feature may be set as a negative sample label, specifically may be 0, in addition, the true two classification label corresponding to the second sample feature may also be set as a positive sample label, specifically may be 1, and the true two classification label corresponding to the first sample feature may be set as a negative sample label, specifically may be 0.

Specifically, calculating a first secondary classification loss through a preset loss function according to the difference between the first secondary classification result and the corresponding real secondary classification labels of the first sample characteristic in the positive and negative sample labels; and calculating a second classification loss through a preset loss function according to the difference between the second classification result and the corresponding real classification label of the second sample characteristic in the positive and negative sample labels, and further aggregating the first classification loss and the second classification loss to obtain the sample discrimination loss, wherein the aggregation mode can be averaging, weighted summation and the like.

And step S32, iteratively optimizing the feature generation model to be trained according to the first classification loss and the sample classification loss to obtain the feature generation model.

In this embodiment, specifically, according to the model gradient calculated by the first classification loss and the sample distinguishing loss, the feature generation model to be trained and the sample distinguishing model to be trained are iteratively updated by using a preset model updating method to optimize the feature generation model to be trained until the feature generation model to be trained meets a preset iteration updating end condition, so as to obtain the feature generation model, where the preset iteration updating end condition may be that the model loss converges or reaches a maximum iteration time threshold.

Wherein, the step of iteratively optimizing the feature generation model to be trained according to the first classification loss and the sample discrimination loss to obtain the feature generation model comprises:

step S321, determining whether both the first classification loss and the sample discrimination loss converge;

in this embodiment, specifically, the first classification loss and the sample discrimination loss are aggregated into a total model loss, and whether the total model loss converges is determined, so as to determine whether both the first classification loss and the sample discrimination loss converge, where the total model loss converges, the first classification loss and the sample discrimination loss converge, otherwise, the first classification loss and the sample discrimination loss do not converge.

Step S322, if the feature generation models to be trained are converged, taking the feature generation models to be trained as the feature generation models;

step S323, if not, updating the to-be-trained feature generation model based on the first classification loss and the model gradient calculated by the sample discrimination loss, updating the to-be-trained first sample discrimination model based on the model gradient calculated by the sample discrimination loss, and returning to the execution step: and extracting a first training sample, first noise data and a first real classification label corresponding to the first training sample.

In this embodiment, if the feature generation models to be trained are all converged, the feature generation models to be trained are used as the feature generation models, if the feature generation models to be trained are not all converged, the feature generation models to be trained are updated according to the model gradient of the total model loss relative to the feature generation models to be trained, the first sample partition models to be trained are updated according to the model gradient of the total model loss relative to the first sample partition models to be trained, meanwhile, the feature extraction models and the classification models are kept fixed, and the method returns to the execution step: and extracting a first training sample, first noise data and a first real classification label corresponding to the first training sample so as to continue to perform the next round of iterative optimization.

Fig. 2 is a schematic flow chart illustrating a process of constructing a feature generation model in an embodiment of the present application, where x is the first training sample, i.e., data, z is the first noise data, i.e., noise, y is the first real classification label, i.e., label, f is the first sample feature, g is the second sample feature,

classifying label for the first prediction, L_taskIs the first classification loss.

Step S40, sending the feature generation models and the classification models to second equipment, so that the second equipment iteratively optimizes a global feature generation model obtained by aggregating the feature generation models and a global classification model obtained by aggregating the classification models according to the feature generation models sent by the first equipment to obtain a target global feature generation model and a target global classification model;

in this embodiment, specifically, the feature generation model and the classification model are sent to a second device, so that the second device iteratively optimizes a global feature generation model obtained by aggregating the feature generation models and a global classification model obtained by aggregating the classification models according to the feature generation model sent by each first device to obtain a target global feature generation model and a target global classification model, specifically, obtains noise data and a real classification label, aggregates the feature generation models into a global feature generation model, aggregates the classification models into a global classification model, and further iteratively trains the global feature generation model and the global classification model according to the noise data and the real classification label, and performs knowledge distillation between the feature generation models and the global feature generation model, and iteratively optimizing the global feature generation model and the global classification model to obtain a target global feature generation model and a target global classification model, wherein the knowledge distillation aims to promote the global feature generation model to learn the model knowledge of each feature generation model so as to learn the knowledge of all participants.

Additionally, it should be noted that the local sample data of each first device is usually distributed non-independently, that is, the features of each first device are not identical, furthermore, the local models constructed by the first devices are different, and if the local models are directly aggregated into a global model, the feature generation models interfere with each other in the aggregation process due to the property of non-independent and same distribution of data of the first devices, and the performance of the global model is reduced, in this embodiment, after the global model is obtained by polymerization, knowledge distillation is further carried out between the global model and each local model, the global model further learns the model knowledge of each local model, the performance of the global model is improved, and furthermore, the problem of model performance reduction caused by the non-independent and uniform distribution of sample data of all federal learning participators can be solved.

In addition, if the local sample data of each first device is independently and identically distributed, the feature generation models can be aggregated to obtain a global feature generation model which is directly used as a target global feature generation model, and the classification models can be aggregated to obtain a global classification model which is directly used as a target global classification model, wherein the aggregation mode comprises averaging, weighted summation and the like.

Step S50, receiving the target global feature generation model and the target global classification model sent by the second device, and performing iterative optimization on the feature extraction model and the target global classification model according to the target global feature generation model to obtain a target feature extraction model and a target classification model.

In this embodiment, specifically, the target global feature generation model and the target global classification model sent by the second device are received, and the classification model is replaced and updated to the target global classification model, and then by performing iterative updating on the feature extraction model and the target global classification model, and performing knowledge distillation between the target global feature generation model and a feature extraction model while fixing the target global feature generation model, performing iterative optimization on the feature extraction model and the target global classification model to obtain a target feature extraction model and a target classification model, the purpose of step S50 is to make the feature extraction model learn the knowledge of all participants through the target global feature generation model, i.e. share the knowledge of all participants, and make the target global classification model and the feature extraction model fit together.

The step of performing iterative optimization on the feature extraction model and the target global classification model according to the target global feature generation model to obtain a target feature extraction model and a target classification model comprises the following steps:

step S51, extracting a second training sample, second noise data and a second real classification label corresponding to the second training sample;

step S52, obtaining local sample features generated by the feature extraction model for the second training sample, and obtaining global sample features generated by the target global feature generation model for the second noise data and the second true class label;

in this embodiment, specifically, feature extraction is performed on the second training sample according to a feature extraction model to obtain local sample features, a model is generated according to target global features, and the second noise data and the second real classification label are jointly converted into global sample features.

Step S53, calculating a second classification loss according to the second real classification label and a second predicted classification label generated by the target global classification model for the local sample feature;

in this embodiment, specifically, the local sample features are classified according to the target global classification model to obtain a second prediction classification label, and then a second classification loss is calculated through a preset loss function according to a difference between the second prediction classification label and the second real classification label.

Step S54, calculating the feature similarity loss according to the similarity between the local sample feature and the global sample feature;

in this embodiment, it should be noted that the sample features are usually tensors, such as vectors and matrices, and further, the similarity between the local sample feature and the global sample feature can be represented by a distance between the local sample feature and the global sample feature. .

Specifically, distance loss is calculated according to the distance between the local sample feature and the global sample feature, and the distance loss is used as feature similarity loss, wherein the feature similarity loss is used for updating the feature extraction model to enable the feature extraction model to output sample features consistent with the target global feature generation model, so that the purpose of enabling the feature extraction model to learn model knowledge of the target global feature generation model is achieved, the target global feature generation model has knowledge of all participants, and the purpose of sharing knowledge of all participants is achieved.

Step S55, iteratively optimizing the feature extraction model and the target global classification model under the condition of fixing the target global feature generation model according to the second classification loss and the feature similarity loss, to obtain the target feature extraction model and the target classification model.

In this embodiment, specifically, whether the second classification loss and the feature similarity loss both converge is determined, if yes, the feature extraction model is used as a target feature extraction model, and the target global classification model is used as a target classification model, if not, the target global feature generation model is fixed and unchanged, the feature extraction model is updated according to a model gradient of a total loss, which is formed by the second classification loss and the feature similarity loss, relative to the feature extraction model, and the target global classification model is updated according to a model gradient of the second classification loss relative to the target global classification model, and the execution procedure is returned: and extracting a second training sample, second noise data and a second real classification label corresponding to the second training sample so as to perform the next iteration.

Additionally, it should be noted that, in order to protect data privacy in federal learning, federal learning may be performed in a homomorphic encryption environment at present, however, computation overhead involved in homomorphic encryption is extremely large, and data involved in the first device and the second device in the embodiment of the present application is plaintext data, so that the computation overhead is significantly reduced compared with a mode of federal learning based on homomorphic encryption, and thus, the efficiency of federal learning is improved. In addition, in order to protect data privacy in federal learning, federal learning can be performed based on multi-party safety calculation at present, however, calculation overhead and communication overhead involved in multi-party safety calculation are extremely large, and data involved in the first device and the second device in the embodiment of the application are all complete plaintext data and do not involve a secret sharing process in multi-party safety calculation, so that compared with a mode of performing federal learning based on multi-party safety calculation, calculation overhead and communication overhead are obviously reduced, and the efficiency of federal learning is improved; in addition, in order to protect data privacy in federal learning, currently federal learning can be performed based on differential privacy, but the differential privacy needs to realize privacy protection by adding noise, which affects usability and accuracy of the model, but the embodiment of the application obviously does not directly add noise in the feature extraction model and the classification model, and only inputs the feature generation model together with noise data and a tag to simulate the output of the feature extraction model, so that the accuracy and the availability of the feature extraction model and the classification model are not affected, and the usability and the accuracy of the federal learning model are improved compared with the federal learning mode based on the differential privacy.

Wherein, the step of iteratively optimizing the feature extraction model and the target global classification model under the condition of fixing the target global feature generation model according to the second classification loss and the feature similarity loss to obtain the target feature extraction model and the target classification model comprises:

step S551, if the target global feature generation model and the target global classification model meet preset federal end conditions, iteratively updating the feature extraction model and the target global classification model under the condition of fixing the target global feature generation model according to the second classification loss and the feature similarity loss to obtain the target feature extraction model and the target classification model;

in this embodiment, specifically, if receiving federal learning end notification information sent by a second device, it is determined that the target global generation model and the target global classification model satisfy a preset federal end condition, and then the feature extraction model and the target global classification model are iteratively updated under the condition that the target global feature generation model is fixed according to the second classification loss and the feature similarity loss, so as to obtain the target feature extraction model and the target classification model, where the feature extraction model and the target global classification model are iteratively updated under the condition that the target global feature generation model is fixed according to the second classification loss and the feature similarity loss, so as to obtain specific implementation processes of the target feature extraction model and the target classification model, refer to the step in the above step S55, and will not be described in detail herein.

Step S552, if the target global feature generation model and the target global classification model do not satisfy a preset federal end condition, iteratively updating the feature extraction model and the target global classification model under the condition of fixing the target global feature generation model according to the second classification loss and the feature similarity loss, and returning to the execution step: and extracting a first training sample, first noise data and a first real classification label corresponding to the first training sample.

In this embodiment, specifically, if the federal learning end notification information sent by the second device is not received, it is determined that the target global feature generation model and the target global classification model do not satisfy the preset federal end condition, and it is further determined whether the second classification loss and the feature similarity loss both converge, and if both converge, the execution step is returned: extracting a first training sample, first noise data and a first real classification label corresponding to the first training sample to perform the next round of federal learning iteration, if the two are not converged, updating the feature extraction model and the target global classification model under the condition of fixing the target global feature generation model according to the second classification loss and the feature similarity loss, and returning to the executing step: and extracting a second training sample, second noise data and a second real classification label corresponding to the second training sample so as to perform the next round of model iteration updating.

In addition, it should be noted that the federal learning modeling method can be used in the field of image processing, and the federal learning modeling optimization method further includes:

step Q10, acquiring a trained image feature extraction model and an image classification model, and extracting a first training image sample, first image noise data and a first real image classification label corresponding to the first training image sample;

step Q20, obtaining a first image sample feature generated by the image feature extraction model for the first training image sample, and a second image sample feature generated by the image feature generation model to be trained for the first image noise data and the first real image classification label;

step Q30, performing image classification on the second image sample characteristic through the image classification model, performing image sample discrimination on the first image sample characteristic and the second image sample characteristic through an image sample discrimination model to be trained, and performing iterative optimization on the image characteristic generation model to be trained under the condition of fixing the image characteristic extraction model and the image classification model to obtain an image characteristic generation model;

step Q40, sending the image feature generation model and the image classification model to a second device, so that the second device can iteratively optimize a global image feature generation model obtained by aggregating each image feature generation model and a global image classification model obtained by aggregating each image classification model according to the image feature generation model sent by each first device, and obtain a target global image feature generation model and a target global image classification model;

and step Q50, receiving the target global image feature generation model and the target global image classification model sent by the second device, and performing iterative optimization on the image feature extraction model and the target global image classification model according to the target global image feature generation model to obtain a target image feature extraction model and a target image classification model.

In this embodiment, it should be noted that the feature extraction model may be an image feature extraction model, the classification model may be an image classification model, the first training sample may be a first training image sample, the first noise data may be first image noise data, the first real classification label may be a first real image classification label, the feature generation model to be trained may be an image feature generation model to be trained, the second sample feature may be a second image sample feature, the sample differentiation model to be trained may be an image sample differentiation model to be trained, the feature generation model may be an image feature generation model, the target global feature generation model may be a target global image feature generation model, and the target global classification model may be a target global image classification model, the target feature extraction model may be a target image feature extraction model, and the target classification model may be a target image classification model. The detailed implementation of steps Q10 through Q50 can refer to the contents of steps S10 through S50, and will not be described herein again.

In the embodiment of the application, a trained image feature extraction model and an image classification model are obtained, a first training image sample, first image noise data, a first real image classification label corresponding to the first training image sample are extracted, a first image sample feature generated by the image feature extraction model for the first training image sample and a second image sample feature generated by an image feature generation model for the first image noise data and the first real image classification label are obtained, image classification is performed on the second image sample feature through the image classification model, image sample discrimination is performed on the first image sample feature and the second image sample feature through an image sample discrimination model to be trained, and iterative optimization is performed on the image feature generation model to be trained under the condition that the image feature extraction model and the image classification model are fixed, and obtaining the image feature generation model, achieving the purpose of constructing the feature generation model carrying the model knowledge of the image feature extraction model, and simultaneously constructing the feature generation model not directly according to the original sample data of the participator. And sending the image feature generation model and the image classification model to second equipment, so that the second equipment generates the models according to the image features sent by the first equipment, iteratively optimizes a global image feature generation model obtained by aggregating all the image feature generation models and a global image classification model obtained by aggregating all the image classification models, and obtains a target global image feature generation model and a target global image classification model. However, the feature generation model is not directly constructed according to the original image sample data of the participant, so that the second device cannot reversely derive the original sample data of the first device. Receiving a target global image feature generation model and a target global image classification model sent by the second equipment, generating a model according to the target global image feature, iteratively optimizing the image feature extraction model and the target global image classification model, so that the image feature extraction model and the image classification model can learn the knowledge of all the participants carried by the target global image feature generation model and the target global image classification model, thereby obtaining a target image characteristic extraction model and a target image classification model, realizing the purpose of sharing the knowledge of all participants through a global model, meanwhile, original sample data of the local participants are not disclosed, namely, the risk that the original image sample data of each participant is disclosed is reduced while the image feature extraction model and the image classification model are constructed based on federal learning. The problem of data island in the process of constructing the image feature extraction model and the image classification model is solved, and the accuracy of the image feature extraction model and the image classification model is improved while the data privacy of the original image sample data of each party is protected.

Compared with the image processing model (an image feature generation model and an image classification model) constructed in a federal learning mode based on homomorphic encryption, the calculation overhead is obviously reduced, and the efficiency of constructing the image processing model based on the federal learning is improved. Compared with the image processing model constructed in a federal learning mode based on multi-party safety calculation, the calculation overhead and the communication overhead are obviously reduced, and the efficiency of constructing the image processing model based on federal learning is improved; compared with the image processing model constructed based on the difference privacy in the federal learning mode, the image processing model constructed based on the federal learning mode is obviously improved in the embodiment of the application without directly adding noise in the image feature extraction model and the image classification model, and only the noise data and the label are input into the image feature generation model together to simulate the output of the image feature extraction model, so that the accuracy and the usable rows of the image feature extraction model and the image classification model are not affected, and the usability and the accuracy of the image processing model constructed based on the federal learning mode are improved.

The embodiment of the application provides a federated learning modeling optimization method, and compared with the prior art, each participant maintains a local model and a global model in a federated learning scene. According to the technical means that all participants learn local unique knowledge through a local model, share the knowledge of all the participants through a global model, and then aggregate the local model and the global model together, the embodiment of the application firstly obtains a trained feature extraction model and a classification model, extracts a first training sample, first noise data and a first real classification label corresponding to the first training sample, further obtains a first sample feature generated by the feature extraction model aiming at the first training sample, and a second sample feature generated by a to-be-trained feature generation model aiming at the first noise data and the first real classification label, further classifies the second sample feature through the classification model and performs sample distinguishing on the first sample feature and the second sample feature through a to-be-trained sample distinguishing model, performing iterative optimization on the feature generation model to be trained under the condition of fixing the feature extraction model and the classification model to obtain a feature generation model, achieving the purpose of constructing the feature generation model carrying model knowledge of the feature extraction model, simultaneously constructing the feature generation model not directly according to original sample data of a participant, further sending the feature generation model and the classification model to second equipment so that the second equipment can obtain a target global feature generation model and a target global classification model by iteratively optimizing a global feature generation model obtained by aggregating each feature generation model and a global classification model obtained by aggregating each classification model according to the feature generation model sent by each first equipment, and further constructing the feature generation model not directly according to the original sample data of the participant, and then the second device can not reversely push the original sample data of the first device, and then receives the target global feature generation model sent by the second device, and according to the target global feature generation model, the feature extraction model and the classification model are subjected to iterative optimization, so that the feature extraction model and the classification model can learn the knowledge of all participants carried by the target global feature generation model and the target global classification model, and further obtain the target feature extraction model and the target classification model, thereby realizing the purpose of prompting the local model to learn the model knowledge of the global model, namely, sharing the knowledge of all participants through the global model, and simultaneously not revealing the local original sample data, so the technical defect that the federal server can reversely push the original data of the participants through the global model of each participant, and therefore, the risk of data disclosure is overcome, the risk of revealing privacy data of the participants in federal learning is reduced.

Further, referring to fig. 3, in another embodiment of the present application, the federal learning modeling optimization method is applied to a second device, and the federal learning modeling optimization method includes:

step A10, receiving feature generation models and classification models sent by each first device, aggregating each feature generation model into a global feature generation model and aggregating each classification model into a global classification model;

step A20, extracting noise data and real classification labels corresponding to the noise data;

in this embodiment, it should be noted that the aggregation manner includes averaging, weighted summation, and the like. Each first device needs to maintain a feature generation model and a classification model, wherein the feature generation model is a model for simulating output features of a corresponding feature extraction model according to noise data and a real classification label, the output features of the feature extraction model are model output results obtained by performing feature extraction on a sample corresponding to the real classification label, and the output of the feature generation model and the output of the feature extraction model can be distinguished by a sample distinguishing model. And for the same sample, the corresponding noise and the label, the output of the feature generation model and the output of the corresponding feature extraction model can generate the same classification label through a classification model. The specific process of constructing the feature generation model by the first device may refer to the specific contents in the above step S10 to step S30, and is not described herein again.

Step A30, iteratively optimizing the global feature generation model and the global classification model by iteratively training the global feature generation model and the global classification model and knowledge distillation between the global feature generation model and each feature generation model according to the noise data and the real classification label to obtain a target global feature generation model and a target global classification model;

in this embodiment, specifically, the global feature generation model and the global classification model are iteratively trained according to the noise data and the real classification label to calculate a global classification loss, knowledge distillation is performed between the global feature generation model and each of the feature generation models to calculate a knowledge distillation loss, and then the global feature generation model and the global classification model are iteratively optimized according to the global classification loss and the knowledge distillation loss to obtain a target global feature generation model and a target global classification model.

Wherein the step of performing knowledge distillation between the global feature generation model and each of the feature generation models to calculate a knowledge distillation loss comprises:

and converting the noise data and the real classification labels into local generation characteristics according to the characteristic generation models, converting the noise data and the real classification labels into global generation characteristics according to the global characteristic generation models, wherein one characteristic generation model outputs one local generation characteristic, calculating characteristic distance loss values according to the distance between each local generation characteristic and the global generation characteristic, and aggregating the characteristic distance loss values into knowledge distillation loss, wherein the aggregation mode can be averaging or weighted summation.

Wherein the step of iteratively optimizing the global feature generation model and the global classification model by iteratively training the global feature generation model and the global classification model and by knowledge distillation between the global feature generation model and each of the feature generation models based on the noise data and the true classification labels to obtain a target global feature generation model and a target global classification model comprises:

step A31, converting the noise data and the real classification label into local generation features according to the feature generation models, and converting the noise data and the real classification label into global generation features according to the global feature generation models;

in this embodiment, it should be noted that each feature generation model takes noise data and a real classification label as input, and outputs a corresponding locally generated feature.

Step A32, calculating global classification loss according to the real classification label and a global prediction classification label generated by classifying the global generation characteristics through a global classification model;

in this embodiment, specifically, the normalized global generation features are classified according to a global classification model to obtain a global prediction classification label, and then a global classification loss is calculated through a preset loss function according to a difference between the real classification label and the global prediction classification label, where the preset loss function may be an L2 loss function or a cross entropy loss function, and the like.

Step A33, calculating global feature similarity loss according to the similarity between each of the locally generated features and the globally generated features;

in this embodiment, specifically, a global feature distance loss value is calculated by using a preset loss function according to a distance between each of the locally generated features and the globally generated feature, and then each of the global feature distance loss values is aggregated into a global feature similarity loss.

Wherein the step of calculating global feature similarity loss based on the similarity between each of the locally generated features and the globally generated feature comprises:

step A331, normalizing each locally generated feature to obtain each first normalized feature;

step A332, normalizing the global generated feature to obtain a second normalized feature;

in this embodiment, specifically, each of the locally generated features is normalized to a corresponding first normalized feature and the globally generated features is normalized to a corresponding second normalized feature according to a preset normalization index function, where it should be noted that, because sample data of each first device is usually in a non-independent and same distribution, and then feature generation models constructed by the first devices according to the sample data usually have a certain difference, outputs of the feature generation models usually have a certain difference, and a similarity cannot be directly calculated, and further, each of the locally generated features and the globally generated features need to be normalized.

Step a333, calculating the global feature similarity loss according to the similarity between each of the first normalized features and the second normalized features.

In this embodiment, specifically, a mean value of each first normalized feature is calculated to obtain a global mean value feature, and then a global distance loss value is calculated according to a distance between the global mean value feature and the second normalized feature, and the global distance loss value is used as the global feature similarity loss, that is, the knowledge distillation loss, so that the purpose of performing knowledge distillation between a global model and each local model when sample data of each participant is not independently and simultaneously distributed in federal learning is achieved.

Step A34, iteratively optimizing the global feature generation model and the global classification model according to the global feature similarity loss and the global classification loss to obtain the target global feature generation model and the target global classification model.

In this embodiment, specifically, it is determined whether the global feature generation model and the global classification model satisfy a preset number of iterative model updates, if yes, the global feature generation model is used as a target global feature generation model, the global classification model is used as a target global classification model, if not, the global classification model is updated according to a model gradient calculated by the global classification loss, and the global feature generation model is updated according to a model gradient calculated by both the global feature similarity loss and the global classification loss, and the execution step is returned: and extracting noise data and real classification labels corresponding to the noise data to perform next round of model updating iteration, wherein the preset model iteration updating times are preset model updating time thresholds, and once the threshold is reached, the second equipment needs to send a target global feature generation model and a target global classification model to the first equipment.

Before the step of feeding back the target global feature generation model and the target global classification model to each of the first devices, the federal learning modeling optimization method further includes:

step B10, judging whether the target global feature generation model and the target global classification model meet preset federal iteration end conditions;

step B20, if yes, notifying each first device that the federal learning modeling is finished, and executing the steps of: and feeding the target global feature generation model and the target global classification model back to each first device respectively.

In this embodiment, it should be noted that the preset federal iteration end condition may be convergence of model loss or maximum federal iteration number of iteration, etc.

Specifically, whether the target global feature generation model and the target global classification model meet a preset federal iteration end condition is judged; if the target global feature generation model and the target global classification model meet preset federal iteration end conditions, notifying each first device that the federal learning modeling is ended, and executing the following steps: feeding the target global feature generation model and the target global classification model back to each first device respectively to finish the federal learning modeling; if the target global feature generation model and the target global classification model do not meet the preset federal iteration end condition, directly executing the following steps: and feeding the target global feature generation model and the target global classification model back to each first device respectively so as to carry out the next round of federal learning iteration.

Step a40, feeding back the target global feature generation model and the target global classification model to each of the first devices, so that the first devices perform iterative optimization on the feature extraction model and the target global classification model corresponding to the feature generation model according to the target global feature generation model, thereby obtaining a target feature extraction model and a target classification model.

In this embodiment, specifically, the target global feature generation model and the target global classification model are fed back to each first device respectively, and then the first device replaces and updates the local classification model to a target global classification model, and iteratively updates the feature extraction model and the target global classification model, and performing knowledge distillation between the target global feature generation model and a feature extraction model while fixing the target global feature generation model, performing iterative optimization on the feature extraction model and the target global classification model to obtain a target feature extraction model and a target classification model, the specific process of generating the target feature extraction model and the target classification model by the first device may refer to the specific contents from step S51 to step S55, and is not described herein again.

Fig. 4 is a schematic flow chart of the second device constructing a target global feature extraction model and a target global classification model, where a client is the first device, softmax is the preset normalized exponential function, a generation model is the feature generation model, a global generation model is the global feature generation model, z is the noise data, y is the real classification label, and KL is the knowledge distillation loss, that is, the global feature similarity loss.

The embodiment of the application provides a federated learning modeling optimization method, and compared with the prior art, each participant maintains a local model and a global model in a federated learning scene. According to the technical means that all participants learn local unique knowledge through local models, share the knowledge of all the participants through global models, and then aggregate the local models and the global models together, the embodiment of the application firstly receives feature generation models and classification models sent by first equipment, aggregates the feature generation models into global feature generation models and aggregates the classification models into global classification models, further extracts noise data and real classification labels corresponding to the noise data, further iteratively trains the global feature generation models and the global classification models according to the noise data and the real classification labels, and iteratively optimizes the global feature generation models and the global classification models by performing knowledge distillation between the global feature generation models and the feature generation models, obtaining a target global feature generation model and a target global classification model, wherein the first device does not directly send the feature extraction model to the second device, but sends the feature generation model simulating the output of the feature extraction model to the second device, and because the input of the feature generation model is noise data and labels, even if the second device has the feature generation model, the sample and the label of the first device cannot be reversely pushed out, and the target global feature generation model and the target global classification model are respectively fed back to each first device, so that the first device carries out iterative optimization on the feature extraction model corresponding to the feature generation model and the target global classification model according to the target global feature generation model to obtain the target feature extraction model and the target classification model, wherein the first device can optimize the feature extraction model by using the global feature generation model, the feature generation model is an output model simulating the feature extraction model, so that the feature extraction model can be prompted to learn the knowledge of the global feature generation model in the process of optimizing the feature extraction model, the purpose of sharing the knowledge of the participants according to the global feature generation model is achieved, and meanwhile, the local data privacy cannot be revealed, so that the technical defect that the federal server can reversely deduce the original data of the participants through the global models of the participants, the data disclosure risk is caused is overcome, and the risk of disclosing the privacy data of the participants in the federal learning is reduced.

Referring to fig. 5, fig. 5 is a schematic device structure diagram of a hardware operating environment according to an embodiment of the present application.

As shown in fig. 5, the federal learning modeling optimization device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used for realizing connection communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a memory device separate from the processor 1001 described above.

Optionally, the federal learning modeling optimization device may further include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. The rectangular user interface may comprise a Display screen (Display), an input sub-module such as a Keyboard (Keyboard), and the optional rectangular user interface may also comprise a standard wired interface, a wireless interface. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface).

Those skilled in the art will appreciate that the federated learning modeling optimization facility architecture illustrated in FIG. 5 does not constitute a limitation of the federated learning modeling optimization facility, and may include more or fewer components than those illustrated, or some components in combination, or a different arrangement of components.

As shown in fig. 5, the memory 1005, which is a type of computer storage medium, may include an operating system, a network communication module, and a federal learning modeling optimization program. The operating system is a program for managing and controlling hardware and software resources of the Federal learning modeling optimization equipment and supports the operation of the Federal learning modeling optimization program and other software and/or programs. The network communication module is used for realizing communication among components in the memory 1005 and communication with other hardware and software in the federal learning modeling optimization system.

In the federated learning modeling optimization apparatus shown in fig. 5, the processor 1001 is configured to execute a federated learning modeling optimization program stored in the memory 1005 to implement the steps of any of the federated learning modeling optimization methods described above.

The specific implementation of the federal learning modeling optimization device of the application is basically the same as that of each embodiment of the federal learning modeling optimization method, and details are not repeated herein.

The embodiment of the present application further provides a federal learning modeling optimization device, which is applied to the first device, and includes:

Optionally, the iterative training module is further configured to:

classifying the second sample characteristics through the classification model, performing sample discrimination on the first sample characteristics and the second sample characteristics through a first sample discrimination model to be trained, and calculating first classification loss and sample discrimination loss;

and iteratively optimizing the feature generation model to be trained according to the first classification loss and the sample discrimination loss to obtain the feature generation model.

Optionally, the iterative training module is further configured to:

classifying the second sample characteristics according to the classification model to obtain a first prediction classification label;

calculating a first classification loss according to the first prediction classification label and the first real classification label;

according to the sample distinguishing model, performing secondary classification on the first sample characteristic and the second sample characteristic respectively to obtain a secondary classification result;

and calculating the sample distinguishing loss according to the two classification results and positive and negative sample labels which correspond to the first sample characteristic and the second sample characteristic together.

Optionally, the iterative optimization module is further configured to:

extracting a second training sample, second noise data and a second real classification label corresponding to the second training sample;

obtaining local sample features generated by the feature extraction model for the second training sample, and obtaining global sample features generated by the target global feature generation model for the second noise data and the second true class label;

calculating a second classification loss according to the second real classification label and a second prediction classification label generated by the target global classification model aiming at the local sample characteristics;

calculating feature similarity loss according to the similarity between the local sample features and the global sample features;

and according to the second classification loss and the feature similarity loss, iteratively optimizing the feature extraction model and the target global classification model under the condition of fixing the target global feature generation model to obtain the target feature extraction model and the target classification model.

Optionally, the iterative optimization module is further configured to:

if the target global feature generation model and the target global classification model meet a preset federal end condition, iteratively updating the feature extraction model and the target global classification model under the condition of fixing the target global feature generation model according to the second classification loss and the feature similarity loss to obtain the target feature extraction model and the target classification model;

if the target global feature generation model and the target global classification model do not meet preset federal end conditions, iteratively updating the feature extraction model and the target global classification model under the condition of fixing the target global feature generation model according to the second classification loss and the feature similarity loss, and returning to the execution step: and extracting a first training sample, first noise data and a first real classification label corresponding to the first training sample.

The embodiment of the present application further provides a federal learning modeling optimization device, the federal learning modeling optimization device is applied to the second device, the federal learning modeling optimization device includes:

Optionally, the iterative optimization module is further configured to:

respectively converting the noise data and the real classification labels into local generation characteristics according to the characteristic generation models, and converting the noise data and the real classification labels into global generation characteristics according to the global characteristic generation models;

calculating global classification loss according to the real classification labels and global prediction classification labels generated by classifying the global generation characteristics through a global classification model;

calculating global feature similarity loss according to the similarity between each of the locally generated features and the globally generated feature;

and iteratively optimizing the global feature generation model and the global classification model according to the global feature similarity loss and the global classification loss to obtain the target global feature generation model and the target global classification model.

Optionally, the iterative optimization module is further configured to:

normalizing each locally generated feature to obtain each first normalized feature;

normalizing the global generated feature to obtain a second normalized feature;

and calculating the global feature similarity loss according to the similarity between each first normalized feature and each second normalized feature.

Optionally, the federal learning modeling optimization device is further configured to:

judging whether the target global feature generation model and the target global classification model meet a preset federal iteration end condition or not;

if yes, notifying each first device that the federal learning modeling is finished, and executing the following steps: and feeding the target global feature generation model and the target global classification model back to each first device respectively.

The embodiment of the application provides a readable storage medium, and the readable storage medium stores one or more programs, which can be executed by one or more processors for implementing the steps of the federal learning modeling optimization method in any one of the above.

The specific implementation of the readable storage medium of the application is substantially the same as that of each embodiment of the federated learning modeling optimization method, and is not described herein again.

The present application provides a computer program product, and the computer program product includes one or more computer programs, which can also be executed by one or more processors for implementing the steps of any of the above methods for federated learning modeling optimization.

The specific implementation of the computer program product of the present application is substantially the same as the embodiments of the federated learning modeling optimization method described above, and is not described herein again.

The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings, or which are directly or indirectly applied to other related technical fields, are included in the scope of the present application.

Claims

1. The federated learning modeling optimization method is applied to first equipment and comprises the following steps:

2. The federal learning modeling optimization method as claimed in claim 1, wherein the step of classifying the second sample feature by the classification model and sample-distinguishing the first sample feature from the second sample feature by a to-be-trained sample-distinguishing model, and iteratively optimizing the to-be-trained feature generation model while fixing the feature extraction model and the classification model to obtain the feature generation model comprises:

3. The method of federated learning modeling optimization as set forth in claim 2, wherein the step of classifying the second sample feature via the classification model and sample discriminating the first sample feature and the second sample feature via a first sample discrimination model to be trained, calculating a first classification loss and a sample discrimination loss comprises:

4. The federal learning modeling optimization method as claimed in claim 1, wherein the step of generating a model based on the target global features, and iteratively optimizing the feature extraction model and the target global classification model to obtain a target feature extraction model and a target classification model comprises:

5. The federated learning modeling optimization method of claim 4, wherein the step of iteratively optimizing the feature extraction model and the target global classification model with the target global feature generation model fixed, in accordance with the second classification loss and the feature similarity loss, to obtain the target feature extraction model and the target classification model, comprises:

6. The federated learning modeling optimization method is applied to second equipment and comprises the following steps:

7. The federal learning modeling optimization method of claim 6, wherein the step of iteratively optimizing the global feature generation model and the global classification model based on the noise data and the true classification label by iteratively training the global feature generation model and the global classification model and by knowledge distillation between the global feature generation model and each of the feature generation models to obtain a target global feature generation model and a target global classification model comprises:

8. The federated learning modeling optimization method of claim 7, wherein the step of calculating a global feature similarity loss based on the similarity between each of the locally generated features and the globally generated feature comprises:

normalizing the global generated feature to obtain a second normalized feature;

9. The federal learning modeling optimization method as claimed in claim 7, wherein, prior to the step of feeding back the target global feature generation model and the target global classification model to each of the first devices, the federal learning modeling optimization method further comprises:

10. The Federal learning modeling optimization apparatus is characterized by comprising: a memory, a processor, and a program stored on the memory for implementing the federated learning modeling optimization method,

the memory is used for storing a program for realizing the Federal learning modeling optimization method;

the processor is configured to execute a program implementing the federal learning modeling optimization methodology to implement the steps of the federal learning modeling optimization methodology of any of claims 1 to 5 or 6 to 9.

11. A readable storage medium having stored thereon a program for implementing a federal learning modeling optimization method, the program being executable by a processor to perform the steps of the federal learning modeling optimization method as claimed in any one of claims 1 to 5 or 6 to 9.

12. A program product being a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of the federal learning modeling optimization method as claimed in any of claims 1 to 5 or 6 to 9.