CN115271089A

CN115271089A - Block chain-based federal learning credible training method and device

Info

Publication number: CN115271089A
Application number: CN202210672835.6A
Authority: CN
Inventors: 王济平; 黎刚; 汤克云; 周健雄; 谢晓锋
Original assignee: Jingxin Data Technology Co ltd
Current assignee: Jingxin Data Technology Co ltd
Priority date: 2022-06-15
Filing date: 2022-06-15
Publication date: 2022-11-01
Anticipated expiration: 2042-06-15
Also published as: CN115271089B

Abstract

The invention relates to a block chain-based federal learning credible training method. The method comprises the following steps: a coordinator initiates a Federal learning and training task request; the participator encrypts the uplink of the statistical information of the data set of the participator; the monitoring party acquires the statistical information of each feature of each participant data set and verifies the total distribution deviation of the statistical information of each participant data set; the participants carry out local training, and the gradient information obtained by training encrypts the uplink; the monitoring party verifies the gradient information of the participating party; the coordinator carries out safe aggregation on the gradient information of the participants; the coordinator sends the aggregated gradient information to each participant to update the local model; the participant judges whether the model converges or whether the iteration number reaches an upper limit. The method avoids the condition of training failure caused by inconsistent distribution, can identify the malicious participants through the gradient information abnormity, ensures the normal operation of the federal learning task, and also improves the participation degree and the understanding degree of the supervision party in the federal learning.

Description

Block chain-based federal learning credible training method and device

Technical Field

The invention relates to a block chain and a federal learning technology, in particular to a block chain-based federal learning credible training method and device.

Background

The traditional federal learning method achieves the effect of data privacy sharing by training a model together by a plurality of participants and a coordinator. Before training, each party negotiates a training model and a data format, an initial model parameter is distributed by a coordinator, then each participant uses a data set of the participant locally to train, after training, each participant sends the encrypted gradient parameter to the coordinator, the coordinator collects the gradient parameters of each participant, then carries out safety gradient aggregation and distributes the safety gradient aggregation to each participant to train continuously, and the process is iterated for a plurality of times until the preset iteration times are met or the model is converged.

Federal learning aims at building a federal learning model based on distributed data sets. During model training, model-related information can be exchanged (or in encrypted form) between the parties, but the respective data cannot be exchanged. This exchange does not expose any protected private data on each node. Traditional federal learning approaches focus on protecting the privacy of parties' data against curious adversary models (adversaries truthfully obey the agreement, but will also try to learn more information from the received information than the output), but suffer from several drawbacks:

(1) Attacks by malicious enemies are difficult to prevent, and the malicious enemies corrupt the whole training model by providing bad data, so that the federal learning task fails. Because many parties are designed to cooperate in the federal learning process, and all parties cannot see the data of other members due to privacy protection, the data are difficult to identify when a training task is a malicious adversary. For example, in the process of participating in computation of multi-party data, some nodes are malicious nodes (that is, nodes are supposed to participate in computation and obtain original data of the other party), and are intended to collect data by using illegal data or models. If a participant initially negotiates with the other party by using a fixed batch of data to participate in joint calculation, but if the participant has malicious behavior (the original data of the other party is supposed to be deduced by a model or other participating data), the initially negotiated data is modified, and if the other party does not know that the batch of data is modified and also performs joint calculation, the risk of data leakage can occur.

(2) Due to the fact that multiple parties are trained and designed, although certain identical intersection exists among data sets of the participants, the data sets may have larger difference in distribution, and if the difference in distribution among the data sets is larger, the training model is very difficult to converge, so that the training task fails. Traditional federal learning methods lack means to identify variability in the distribution of data sets prior to the training process to avoid this.

(3) In addition to involving the participants, the federal study also involves a supervisor many times, and the traditional federal study method lacks some means to allow the supervision unit to audit the federal study task to a certain extent, which brings certain difficulties to the federal study landing.

Disclosure of Invention

The invention aims to overcome the defects in the prior art, and provides a block chain-based federal learning credible training method and a block chain-based federal learning credible training device, which improve the participation degree of a supervisor in federal learning, and enable the supervisor to perform related audit verification so as to enable the federal learning to be performed smoothly.

In order to achieve the purpose, the invention adopts the following technical scheme:

a block chain-based federal learning credible training method comprises the following steps:

step S1, a coordinator initiates a Federal learning training task request;

s2, encrypting uplink by the participant according to the statistical information of the data set of the participant;

s3, the supervisor acquires statistical information of each feature of each participant data set and verifies the total deviation degree of the distribution of the statistical information of each participant data set;

s4, the participants carry out local training, and the gradient information obtained by training encrypts uplink;

s5, the supervisor verifies the gradient information of the participants;

s6, the coordinator carries out safe aggregation on the gradient information of the participants;

s7, the coordinator sends the aggregated gradient information to each participant to update a local model;

and S8, judging whether the model converges or the iteration frequency reaches the upper limit by the participant, if not, returning to the step S4 for iteration, and if so, ending the training.

Further, in step S1, before the coordinator issues the task, the coordinator needs to create a corresponding intelligent contract for the participant and the supervisor to call.

Further, in step S2, the statistical information of the data set at least includes a mean value, a median, a standard deviation, a WOE value, and an IV value.

Further, in step S3, after the monitoring party acquires the statistical information of each participant data set, the monitoring party compares every two deviation degrees of the same feature of the participant data sets with each other to determine whether the deviation degrees of the same feature of the participant data sets exceed a set value, traverses all the features of all the participant data sets, and finally collects the results to obtain a total distribution deviation degree.

Further, in step S3, when the monitoring party compares the deviation degrees of the same feature of the data sets of the participating parties in pairs, a random number set is generated according to the distribution rule and the statistical information of one feature of the data sets of the participating parties, and then the proportion of the random number set outside the distribution range of the corresponding feature of the other data set of the participating parties is judged, if the proportion exceeds a set value, the deviation degree is recorded as 1, otherwise, the deviation degree is recorded as 0; and after traversing all the characteristics of all the participants, finally adding the deviation degrees to obtain the total distributed deviation degree.

Further, step S4 includes:

step S41, the participator uploads the task code and the parameter information abstract to a block chain;

step S42, the participator inquires whether the parameter information abstracts of other participators of the task code are consistent with the participator, if so, the next step is carried out, otherwise, the training task is terminated;

and S43, initializing a model training environment, pulling a local data set, starting to execute a training task, and encrypting and winding the obtained gradient information.

Further, step S5 includes:

step S51, initializing a data set by using a random number as a virtual data set, wherein the record number of the virtual data set is the size of a training batch, and the characteristic number of each record of the virtual data set is the same as the characteristic number of the data records of the participant data set; simultaneously initializing a virtual tag set randomly;

step S52, adding the gradient change amount submitted by the participant and the model weight provided for the participant last time to obtain the model weight trained by the participant, and initializing a training model by using the weight;

step S53, a virtual data set is used as a data source and is transmitted into a training model, a model result is obtained through forward propagation, and then a loss function and a virtual gradient are calculated in a combined mode through the virtual data set and a virtual label set; iteratively updating the virtual data set and the virtual tag set by using a gradient descent method with the minimum difference value between the virtual gradient and the actual gradient as an optimization direction to finally obtain an optimal virtual data set;

and S54, calculating statistical information of the optimal virtual data set, comparing the statistical information with the statistical information of the data set submitted by the participant, terminating the training task if the deviation is greater than a set value, and otherwise, entering the next step.

Further, in step S54, when the deviation between the statistical information of the optimal virtual data set and the statistical information of the data set submitted by the participant is greater than a set value, the supervisor issues task termination information and a related monitoring report; and after monitoring that the supervisor sends out the task termination information, the coordinator acquires a related monitoring report, informs each participant to terminate the training task and sends the related monitoring report to each participant.

The invention also discloses a block chain-based federal learning credible training system, which comprises a coordinating party, participating parties and a supervising party, wherein the coordinating party is respectively connected with the participating parties, the coordinating party and the participating parties form a federal learning cluster, the participating parties are connected with the block chain, and the supervising party is connected with the block chain;

the coordination party is used for initiating a federal learning training task request;

the participant is used for encrypting the uplink by the statistical characteristics of the data set of the participant;

the monitoring party is used for acquiring the statistical characteristics of the data sets of all the participants and verifying the total deviation degree of the distribution of the statistical characteristics of the data sets of all the participants;

the participator is also used for carrying out local training and encrypting uplink on the gradient information obtained by training;

the monitoring party is also used for verifying the gradient information of the participants;

the coordinator is also used for carrying out security aggregation on the gradient information of the participants;

the coordinator is also used for sending the aggregated gradient information to each participant so as to update the local model;

and the participator is also used for judging whether the model converges or the iteration frequency reaches the upper limit, if not, the model is iteratively trained, and if so, the training is ended.

The invention also discloses an electronic device, comprising: a processor; and a memory having computer readable instructions stored thereon which, when executed by the processor, implement the above method.

The invention also discloses a computer-readable storage medium on which a computer program is stored, which computer program, when being executed by a processor, carries out the above method.

Compared with the prior art, the invention has the beneficial effects that: on the premise of guaranteeing data privacy, a supervisor checks the data deviation degree before training, and the condition of training failure caused by inconsistent distribution is avoided; meanwhile, through gradient cochain of the participants, the gradient information of the participants is verified by a supervisor during training, malicious participants can be identified through the gradient information abnormity, and normal operation of the federal learning task is guaranteed. Because the whole training process has the data chaining, the supervisor can effectively know the training process, the participation degree of the supervisor in the federal study is improved, the knowledge and trust of the supervisor on the federal study are improved, and the federal study can be smoothly carried out.

The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood, the present invention can be implemented in accordance with the content of the description, and in order to make the above and other objects, features, and advantages of the present invention more apparent, the following detailed description of the preferred embodiments is given as follows.

Drawings

FIG. 1 is a diagram of a training system framework of the present invention.

Fig. 2 is a general flowchart of the training method of the present invention.

FIG. 3 is an intelligent contract call graph of the present invention.

Fig. 4 is a flowchart of step S4 of the training method of the present invention.

Fig. 5 is a flowchart of step S5 of the training method of the present invention.

Fig. 6 is a data flow diagram of step S5 of the training method of the present invention.

It should be noted that, the products shown in the above views are all appropriately reduced/enlarged according to the size of the drawing and the clear view, and the size of the products shown in the views is not limited.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and a repetitive description thereof will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the embodiments of the disclosure can be practiced without one or more of the specific details, or with other methods, components, materials, devices, steps, and so forth. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.

The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in the form of software, or in one or more software-hardened modules, or in different networks and/or processor devices and/or microcontroller devices.

The embodiment is a block chain-based federal learning credible training method, and relates to a coordinator, participants and a monitoring party as shown in fig. 1, wherein the coordinator is respectively connected with the participants through network connections, and the coordinator and the participants form a federal learning cluster. Participants are federally learning tasks and data providers. The coordinator is a parameter server for assisting the federal learning task and helping the participants to perform gradient aggregation. And the participators are provided with a block chain client program and are connected to the block chain through a wired network or a wireless network. The supervisor is connected to the block chain through a wired network or a wireless network.

As shown in fig. 2, the training method includes the following steps: step S1, a coordinator initiates a Federal learning training task request; s2, encrypting and winding up the statistical information of the data set of the participant by the participant; s3, the supervisor acquires statistical information of each feature of each participant data set and verifies the total deviation degree of the distribution of the statistical information of each participant data set; s4, the participants carry out local training, and the gradient information obtained by training encrypts uplink; s5, the supervisor verifies the gradient information of the participants; s6, the coordinator carries out safe aggregation on the gradient information of the participants; s7, the coordinator sends the aggregated gradient information to each participant to update the local model; and S8, judging whether the model converges or the iteration frequency reaches the upper limit, if not, returning to the step S4 for iteration, and if so, ending the training.

On the premise of ensuring data privacy, the data deviation degree is verified by a supervisor before training begins, and the condition of training failure caused by inconsistent distribution is avoided; meanwhile, through gradient cochain of the participants, the gradient information of the participants is verified by a supervisor during training, malicious participants can be identified through the gradient information abnormity, and normal operation of the federal learning task is guaranteed. Because the whole training process has data chaining, the supervisor can effectively know the training process, the participation degree of the supervisor in the federal learning is improved, the knowledge and trust of the supervisor on the federal learning are improved, and the federal learning can be smoothly carried out.

The individual steps of the training method are described in detail below.

Step S1 is for the coordinator to initiate a federal learning training task request. There may be two ways for the coordinator to initiate the federal learning training task. In the first mode, each participant negotiates the model and the data set format privately, trains the task codes uniformly, and then the coordinator initiates the federal learning task according to the task codes and the participant information. In the second mode, the coordinator directly defines the model and the data set format, and then initiates the federal learning task and issues the federal learning task to the participants to start the federal learning task.

Before the task is issued, the coordinator needs to create corresponding intelligent contracts for the participants and the supervisor to call. The intelligent contract is a code segment containing a plurality of states and methods, and the intelligent contract can call the methods contained in the contract through the corresponding addresses of the block chain after being created. And initializing a task storage space in the intelligent contract, wherein the task storage space is used for storing information uploaded by each subsequent participant. Meanwhile, subsequent calling parties of the training task are limited to the participating party and the monitoring party of the training task, and other people are prevented from eavesdropping on the task information.

In some embodiments, as shown in FIG. 3, the intelligent contract created by the coordinator includes the following methods:

1. initializing, when initializing the intelligent contract, the address of the sender is stored in the contract.

2. Adding participants is limited to coordinator call and is limited to one call.

3. Adding a supervisor is limited to coordinator invocation and is limited to invocation once.

4. And acquiring a participant list, and only calling participants and a supervisor.

5. And uploading the initialization parameters, wherein the parameters are only called by participants and are only called once.

6. And acquiring the initialization parameters of the participants, and only calling the participants and the supervisor.

7. Uploading data set statistical information, only calling by participants, wherein the data statistical abstract comprises the total sample number of the data set, the mean value, the median and the standard deviation of each characteristic.

8. And acquiring statistical information of the uploaded data set, and only calling by a supervisor.

9. And uploading training gradient information, and only calling by the participants.

10. And inquiring the training gradient and only calling by a supervisor.

11. Terminating the training task and limiting the calling of a supervisor; due to the abnormal conditions monitored in the training process, the monitoring party can terminate the training task through the interface, and the participating party and the coordinating party monitor through uplink data so as to facilitate processing termination logic.

Step S2 is that the participating party encrypts the statistics of its own data set for uplink transmission. In step S2, the statistical information of the data set includes at least a mean value, a median, a standard deviation, a WOE value, and an IV value. These statistics are computed for a set of features of the data set. For example, the characteristics of the data set include age, height, and weight, and the height value of each record in the data set constitutes a set of characteristics for which corresponding statistical information, such as mean, median, standard deviation, and the like, can be calculated. After the statistical information is calculated, the participating party uploads the statistical information to the block chain through a corresponding interface of the intelligent contract, wherein the intelligent contract comprises an RSA and SM2 encryption algorithm, and the statistical information can be encrypted and linked up. The authority for calling the intelligent contract to inquire the statistical information is only opened to the monitoring party, and the statistical information can be decrypted by a private key party of the monitoring party.

And S3, the supervisor acquires the statistical information of each feature of each participant data set and verifies the total deviation degree of the distribution of the statistical information of each participant data set. After each participant links the statistical information, the supervision facilitates obtaining the data set statistical information of each participant from the blockchain. And then the monitoring party compares the deviation degree of the same characteristic of the data set of the participating parties in pairs with each other to determine whether the deviation degree exceeds a set value, traverses all the characteristics of all the participating parties and finally collects the characteristics to obtain the total distribution deviation degree. And if the obtained distribution total deviation degree is lower than the threshold value, the verification is passed and the next step is carried out. If the obtained total deviation degree of the distribution is not lower than the threshold value, the data distribution of each participant is relatively uneven, the situation of gradient offset is easy to occur during federal learning, and then the training task is stopped in time.

Specifically, in step S3, when the supervisors compare the deviation degrees of the same feature of the participant data sets pairwise, a random number set is generated according to the distribution rule and the statistical information of one feature of the participant data sets, and then the proportion of the random number set outside the distribution range of the corresponding feature of the other participant data set is determined. If the ratio exceeds the set value, the degree of deviation is recorded as 1, otherwise the degree of deviation is recorded as 0. And finally, after traversing all the characteristics of all the participants, adding the deviation degrees to obtain the total distributed deviation degree.

The following is a detailed illustration.

Example 1

In the party a, μ represents the mean value of the feature F, σ represents the standard deviation of the feature F, X represents any data of the feature F, and P represents the probability of the occurrence of the data X in a certain range. The statistic information of the characteristic F of the data set of the participant A comprises mu and sigma, and the characteristic F obeys normal distribution. Therefore, the data of the characteristic F is consistent with P (| X-mu | ≦ 3 σ) ≥ 99.7%, so the data with | X-mu | > 3 σ is regarded as deviating data. In the participant B, the mean value of the feature F is represented by μ ', the standard deviation of the feature F is represented by σ ', X ' represents certain data of the feature F, and P ' represents the probability that the data X ' appears in a certain range. And generating a random number set S 'according to the rule that P' (| X '-mu' | is less than or equal to 3 sigma ') is more than or equal to 99.7 percent, and then judging the proportion that the numerical value in the random number set S' falls outside the distribution range of the characteristic F of the party A, namely the proportion that the numerical value quantity of the numerical value in the random number set S 'falling in the range of | X-mu | > 3 sigma accounts for the total numerical value of the random number set S'. If the ratio exceeds a set value (for example, 5%), the degree of deviation T is recorded as 1, and if the ratio does not exceed the set value, the degree of deviation T is recorded as 0. Each is compared pairwise against feature FEach participant then traverses all other characteristics to finally obtain the total deviation degree T of distribution_{General assembly}=Σt. According to the total deviation T of the distribution_{General assembly}The result of the comparison with the threshold determines whether to terminate the training task.

Example 2

In the participant a, μ represents the mean value of the feature F, σ represents the standard deviation of the feature F, X represents any data of the feature F, and P represents the probability that the data X appears in a certain range. The statistical information of the feature F of the data set of the party a includes μ and σ, and the feature F is non-uniformly distributed. For non-uniformly distributed data, multiplying power is expressed by k, and the distribution of the data conforms to Chebyshev inequality

And adjusting the k value to obtain a data range of a preset threshold value, wherein the data beyond the range is regarded as deviated data. In the participant B, the mean value of the feature F is represented by μ ', the standard deviation of the feature F is represented by σ ', X ' represents certain data of the feature F, and P ' represents the probability that the data X ' appears in a certain range. According to the set multiplying power k and the Chebyshev inequality

And generating a random number set S ', and then judging the proportion of the numerical values in the random number set S ' falling outside the data range of the characteristic F of the participant A, namely the proportion of the numerical value quantity of the numerical values in the random number set S ' falling in the range of | X ' -mu ' | ≧ k σ ' to the total numerical value of the random number set S '. If the ratio exceeds a set value, the degree of deviation T is recorded as 1, and if the ratio does not exceed the set value, the degree of deviation T is recorded as 0. Comparing every two participants aiming at the characteristic F, then traversing all other characteristics to finally obtain the distribution total deviation degree T_{General assembly}=Σt. According to the total deviation T of distribution_{General assembly}The result of the comparison with the threshold determines whether to terminate the training task.

Through the verification in the step S3, under the condition that the private data of the participants are not leaked, the monitoring party can judge the data deviation degree condition of each participant, so that the conditions of gradient offset and incapability of convergence during training can be avoided before the iterative training with large workload begins, idle work is avoided, and the success rate of federal learning is improved.

Step S4, the participator carries out local training and encrypts the uplink of the gradient information obtained by training. After initializing the training model and the data set parameters, each participant starts to use the local private data set to perform model training, after training a plurality of batches (defined by the training parameters), the gradient information is obtained and then encrypted (the encryption masking algorithm used by the participant includes but is not limited to homomorphic encryption, differential privacy or secret sharing) and uploaded to the blockchain through an intelligent contract.

Specifically, as shown in fig. 4, step S4 includes: step S41, the participator uploads the task code and the parameter information abstract to a block chain; step S42, the participator inquires whether the parameter information abstracts of other participators of the task code are consistent with the participator, if so, the next step is carried out, otherwise, the training task is terminated; and S43, initializing a model training environment, pulling a local data set, starting to execute a training task, and carrying out encrypted uplink of the obtained gradient information. Firstly, in step S41, when each participant starts a federal learning training task, before initializing a model and model parameters, the initialization information such as a model structure and model parameters is encoded according to a system preset format, then corresponding information summaries (information summary algorithms include but are not limited to SHA-2, SM3, and the like) are obtained, and finally, an intelligent contract is called to upload task codes and parameter information summaries to a block chain. Then, in step S42, the participator inquires whether the parameter information abstracts of other participators of the task code are consistent with the participator through an intelligent contract, and if not, the training task is terminated; if so, the process proceeds to step S43. Finally, in step S43, a model training environment is initialized, a local data set of the participant is pulled, and a training task is started. The training iteration times are defined by training parameters, and after the iteration is finished, the gradient information obtained by training is encrypted and uploaded to a block chain.

Step S5 is to verify the gradient information of the participants by the supervisor. In step S5, after monitoring the gradient information uploaded by the participant through the intelligent contract interface, the supervisor verifies the gradient information: and (3) reducing the statistical information of the data set by a gradient descent method, comparing the statistical information with the statistical information of the data set of the participant submitted in the step (S2), if a large difference exists, regarding the gradient information submitted by the participant at this time as abnormal, and terminating the training task according to the condition.

As shown in fig. 5, step S5 is divided into steps S51 to S54, which will be described in detail below, while the data flow is shown with reference to fig. 6.

In step S51, a data set is initialized with a random number as a virtual data set, the number of records in the virtual data set is the training batch size, and the number of features of each record in the virtual data set is the same as the number of features of the data records in the participant data set. A set of virtual tags is then randomly initialized. The training batch size is batch size of batch training of the participants in training the model, and is a general optimization strategy of the neural network algorithm, wherein the general batches are 256, 512, 1024 and the like, and the training batch size is already specified when a training task is initialized. For the following example, assuming that a person is trained to predict whether the person is obese, the participant data set is characterized by age, height, weight, and body fat, and for the data set, the number of features is 4, the label is whether the person is obese, and the label value is 0 or 1. Assuming that the training batch size is 512, the random number initialized virtual data set has 512 records, each record contains 4 characteristics of age, height, weight and body fat, the total data volume is 512 x 4, and the label is 0 or 1, so that 512 random 0 or 1 are used as the virtual label set.

Step S52 is to restore the participant model weights and initialize the training model. Specifically, the gradient change amount submitted by the participant is added to the model weight provided for the participant last time to obtain the trained model weight of the participant, and then the trained model is initialized by the weight.

In step S53, the virtual data set is first used as a data source and transmitted into the training model, and the model result is obtained through forward propagation, and then the loss function and the virtual gradient are calculated in combination with the virtual label. And (3) iteratively updating the virtual data set and the virtual label set by using a gradient descent method by taking the minimum difference value between the virtual gradient and the actual gradient as an optimization direction, and finally obtaining the optimal virtual data set. This optimal virtual data set is closest to the participant data set, and thus the statistics of the optimal virtual data set will also be similar to the statistics of the participant data set.

Referring to fig. 6, x 'represents a virtual data set, W represents a model weight, y' represents a virtual label, F represents a model forward propagation map, a label result obtained by F (x ', W) calculation and y' are subjected to Loss calculation to obtain Loss = (F (x ', W), y'), and a virtual gradient is obtained by differentiating the Loss and W

Based on the formula, the optimal virtual data set x 'of the best fitting model is obtained'^*And optimal virtual tag set y'^*. Assuming that when x 'and y' are consistent with the participant raw data set and the tag set, the training gradient will be consistent, so the optimized direction is the virtual gradient obtained

And actual gradient

The difference is minimal, and the optimization direction is then the following equation:

continuously iterating and optimizing through the formula, synchronously and reversely updating the virtual data set and the virtual label set until iteration converges, and obtaining the optimal virtual data set x'^*。

In step S54, statistical information (such as mean, median, standard deviation, WOE value, IV value) of the optimal virtual data set is calculated and compared with the statistical information of the data set submitted by the participating party, and if the deviation between the two is greater than a set value, the training task is terminated, otherwise the next step is performed. In step S54, when the deviation between the statistical information of the optimal virtual data set and the statistical information of the data set submitted by the participant is greater than a set value, the supervision conveniently issues the termination task information and the related monitoring report on the blockchain. And after the block chain monitors that the supervisor sends out the termination task information, the coordinator acquires the related monitoring report, informs each participant to terminate the training task, and sends the related monitoring report to each participant.

And S5, simulating a virtual data set closest to the characteristics of the data set of the participant by constructing the virtual data set and reversely updating the virtual data set by using a gradient descent method under the condition that the data set of the participant cannot be obtained, so as to judge whether abnormality exists or not, timely identify malicious behaviors and avoid the failure of a federal learning task caused by corruption of the whole training model by providing bad data by malicious adversaries.

Step S6 is to securely aggregate the gradient information of the participants by the coordinator. In step S5, after the monitor verifies that all the participant gradient information passes, the monitor chains the verification success result, and after the coordinator monitors that the block chaining-out information obtains the verification success result sent by the monitor, the coordinator starts to pull the gradient information of each participant in the batch, and performs security aggregation on the encryption gradient to obtain an aggregation gradient, for example, the security aggregation may be performed by using a weighted average method based on homomorphic encryption.

And step S7, the coordinator sends the aggregated gradient information to each participant to update the local model. And after the coordinator obtains the aggregation gradient in the step S6, the aggregation gradient is sent to each participant. And after receiving the respective aggregation gradients, the participants decrypt the aggregation gradients by using the secret key and update the local models of the participants by using the decrypted gradient results.

In step S8, the participant determines whether the model converges or whether the number of iterations reaches the upper limit. After the participant updates the local model in step S7, the participant determines whether the model converges or whether the number of iterations reaches the upper limit. If not, or not reaching the upper iteration limit, returning to step S4, that is, the participant continues to perform local training. If the model is converged or the iteration upper limit is reached, the training task is ended.

The invention also discloses a block chain-based federal learning credible training system. As shown in fig. 1, a coordinating party, a participating party and a monitoring party are involved, the coordinating party is respectively connected with the participating party through a network connection, and the coordinating party and the participating party form a federal learning cluster. Participants are federally learning tasks and data providers. The coordinator is a parameter server for assisting the federal learning task and helping the participants to perform gradient aggregation. And the participators are provided with a block chain client program and are connected to the block chain through a wired network or a wireless network. The supervisor is connected to the block chain through a wired network or a wireless network.

The coordinator is used for initiating a Federal learning training task request.

The participant is configured to encrypt the statistical characteristics of its own data set for uplink transmission.

The monitoring party is used for obtaining the statistical characteristics of the data sets of all the participants and checking the total deviation degree of the distribution of the statistical characteristics of the data sets of all the participants.

The participants are also used for local training and encrypting the uplink by the gradient information obtained by training.

The monitoring party is also used for verifying the gradient information of the participants.

The coordinator is also used for carrying out security aggregation on the gradient information of the participants.

The coordinator is also used for sending the aggregated gradient information to each participant to update the local model.

In addition, in the embodiment of the invention, the electronic equipment capable of realizing the block chain-based federal learning credible training method is further provided.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

The electronic device is in the form of a general purpose computing device. Components of the electronic device may include, but are not limited to: the system comprises at least one processing unit, at least one storage unit, a bus for connecting different system components (comprising the storage unit and the processing unit), and a display unit.

Wherein the memory unit stores program code that is executable by the processing unit to cause the processing unit to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary method" of the present specification. For example, the processing unit may perform steps S1 to S8 of the block chain based federal learning trusted training method of the present invention.

The memory unit may include a readable medium in the form of a volatile memory unit, such as a random access memory unit (RAM) and/or a cache memory unit, and may further include a read only memory unit (ROM).

The storage unit may also include a program/utility having a set (at least one) of program modules including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

The bus may be any representation of one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device may also communicate with one or more external devices (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface. Also, the electronic device may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via a network adapter. As shown, the network adapter communicates with other modules of the electronic device over a bus. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

In an exemplary embodiment of the present disclosure, a computer readable storage medium is further provided, on which a program product for implementing the above-mentioned block chain-based federal learned trusted training method of the present specification is stored. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above-mentioned "exemplary methods" section of the description, when the program product is run on the terminal device.

The program product for implementing the above method according to an embodiment of the present invention may employ a portable compact disc read only memory (CD-ROM) and include program codes, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In situations involving remote computing devices, the remote computing devices may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to external computing devices (e.g., through the internet using an internet service provider).

Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed, for example, synchronously or asynchronously in multiple modules.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice in the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.

Claims

1. A block chain-based federated learning credible training method is characterized by comprising the following steps:

step S1, a coordinator initiates a Federal learning training task request;

s2, encrypting and winding up the statistical information of the data set of the participant by the participant;

s5, the supervisor verifies the gradient information of the participants;

2. The method of claim 1, wherein in step S2, the statistical information of the data set comprises at least a mean, a median, a standard deviation, a WOE value, an IV value.

3. The method as claimed in claim 1, wherein in step S3, after the monitoring party obtains the statistical information of each participant data set, the monitoring party compares pairwise with each other whether the deviation of the same feature of the participant data set exceeds a set value, traverses all the features of all the participant data sets, finally collects to obtain the total distribution deviation, if the total distribution deviation is lower than a threshold value, the verification is passed and the next step is performed, otherwise, the training task is terminated.

4. The method as claimed in claim 3, wherein in step S3, when the supervisor compares the deviation degree of the same feature of the participant data sets two by two, a random number set is generated according to the distribution rule and the statistical information of one feature of the participant data sets, then the proportion of the random number set outside the distribution range of the corresponding feature of the other participant data sets is judged, if the proportion exceeds a set value, the deviation degree is recorded as 1, otherwise, the deviation degree is recorded as 0; and finally, after traversing all the characteristics of all the participants, adding the deviation degrees to obtain the total distributed deviation degree.

5. The method of claim 1, wherein the step S4 comprises:

step S41, a participant uploads a task code and a parameter information abstract to a block chain;

step S42, the participant inquires whether the parameter information abstracts of other participants of the task code are consistent with the participant, if so, the next step is carried out, otherwise, the training task is terminated;

and S43, initializing a model training environment, pulling a local data set, starting to execute a training task, and carrying out encrypted uplink of the obtained gradient information.

6. The method of claim 1, wherein the step S5 comprises:

step S51, initializing a data set by using a random number as a virtual data set, wherein the record number of the virtual data set is the size of a training batch, and the characteristic quantity of each record of the virtual data set is the same as the characteristic quantity of the data records of the participant data set; simultaneously initializing a virtual tag set randomly;

step S53, a virtual data set is used as a data source and is transmitted into a training model, a model result is obtained through forward propagation, and then a loss function and a virtual gradient are calculated in combination with a virtual label; iteratively updating the virtual data set and the virtual tag set by using a gradient descent method with the minimum difference value between the virtual gradient and the actual gradient as an optimization direction to finally obtain an optimal virtual data set;

7. The method of claim 6, wherein in step S54, when the statistical information of the optimal virtual data set deviates from the statistical information of the data set submitted by the participant by more than a set value, the supervisor issues the termination task information and the related monitoring report; and after monitoring that the supervisor sends out the task termination information, the coordinator acquires a related monitoring report, informs each participant to terminate the training task and sends the related monitoring report to each participant.

8. A block chain-based federal learning credible training system is characterized by comprising a coordinator, participants and a supervisor, wherein the coordinator is respectively connected with the participants, the coordinator and the participants form a federal learning cluster, the participants are connected with the block chain, and the supervisor is connected with the block chain;

the coordinator is used for initiating a federal learning training task request;

the participant is used for encrypting uplink according to the statistical characteristics of the own data set;

the monitoring party is also used for verifying gradient information of the participants;

and the participator is also used for judging whether the model is converged or whether the iteration times reach the upper limit, if not, the model is iteratively trained, and if so, the training is ended.

9. An electronic device, comprising:

a processor; and

a memory having computer readable instructions stored thereon which, when executed by the processor, implement the method of any of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.