CN110969264B

CN110969264B - Model training method, distributed prediction method and system thereof

Info

Publication number: CN110969264B
Application number: CN201911272294.2A
Authority: CN
Inventors: 谭晋; 王磊; 王力
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2019-12-11
Filing date: 2019-12-11
Publication date: 2022-08-02
Anticipated expiration: 2039-12-11
Also published as: CN115409198A; CN110969264A

Abstract

Embodiments in the present specification provide a model training method, a distributed prediction method, and a system thereof. The center node performs model training in a trusted execution environment based on sample data of more than two user nodes, and splits the trained model into user models of more than two user nodes and distributes the user models to the user nodes. The one or more user nodes obtain local prediction results at least based on the user model and the local feature data, and the prediction node obtains prediction results aiming at the identification information of the object to be predicted based on the local prediction results of the one or more user nodes. Therefore, data privacy of all parties can be effectively protected.

Description

Model training method, distributed prediction method and system thereof

Technical Field

The embodiment of the specification relates to the technical field of information, in particular to a model training method, a distributed prediction method and a system thereof.

Background

With the development of artificial intelligence technology, machine learning models have been gradually applied in the fields of risk assessment, speech recognition, natural language processing, and the like. In the fields of medical treatment, finance and the like, different enterprises or institutions have different sample data, and if the sample data are jointly trained, the model precision can be effectively improved, and huge economic benefits are brought to the enterprises. However, sample data owned by different enterprises or institutions contains a lot of privacy for users or even business secrets, which can cause a series of negative effects once the sample data is leaked.

Therefore, it is desirable to provide a joint training and prediction scheme that can effectively protect the privacy of user data.

Disclosure of Invention

One of the embodiments of the present specification provides a model training method, where the model training method is performed by a central node, and the method includes: acquiring encrypted sample data from more than two user nodes; decrypting the encrypted sample data in a trusted execution environment; performing model training by using the decrypted sample data in a trusted execution environment to obtain a trained model; splitting the trained model to obtain user models of the more than two user nodes; and respectively sending the user models of the more than two user nodes to each user node.

One of the embodiments of the present specification provides a model training system, where, corresponding to a central node, the system includes: the data acquisition module is used for acquiring encrypted sample data from more than two user nodes; a decryption module for decrypting the encrypted sample data in a trusted execution environment; the training module is used for carrying out model training by using the decrypted sample data in the trusted execution environment to obtain a trained model; the model splitting module is used for splitting the trained model to obtain the user models of the more than two user nodes; and the model distribution module is used for respectively sending the user models of the more than two user nodes to each user node.

One of the embodiments of the present specification provides a model training apparatus, which includes at least one processor and at least one storage device, where the storage device is used to store instructions, and when the at least one processor executes the instructions, the model training apparatus implements the model training method as shown in any one of the embodiments of the present specification.

One of embodiments of the present specification provides a distributed prediction method, where the method is executed by a prediction node, where the prediction node is one of two or more user nodes, or a node other than the two or more user nodes and is in communication connection with the two or more user nodes, where the two or more user nodes respectively store respective user models, and each user model corresponds to a part of the prediction model; the method comprises the following steps: sending a prediction request for identification information of an object to be predicted to one or more user nodes; receiving local prediction results of one or more user nodes, wherein the local prediction results are obtained at least based on user models of corresponding user nodes and characteristic data of identification information aiming at an object to be predicted; and obtaining a prediction result aiming at the identification information of the object to be predicted based on the local prediction results of one or more user nodes.

One of embodiments of the present specification provides a distributed prediction system, where the system corresponds to a prediction node, the prediction node is one of two or more user nodes, or a node other than the two or more user nodes and is in communication connection with the two or more user nodes, the two or more user nodes respectively store respective user models, and each user model corresponds to a part of the prediction model; the system comprises: a prediction request sending module, configured to send a prediction request for identification information of an object to be predicted to one or more user nodes; the local prediction result receiving module is used for receiving local prediction results of one or more user nodes, and the local prediction results are obtained at least based on the user model of the corresponding user node and the characteristic data of the identification information of the object to be predicted; and the prediction module is used for obtaining a prediction result aiming at the identification information of the object to be predicted based on the local prediction results of one or more user nodes.

One of the embodiments of the present specification provides a distributed prediction apparatus, including at least one processor and at least one storage device, where the storage device is configured to store instructions, and when the instructions are executed by the at least one processor, the distributed prediction method performed by a prediction node is implemented as shown in any one of the embodiments of the present specification.

One of embodiments of the present specification provides a distributed prediction method, where the distributed prediction method is executed by a user node of two or more user nodes, where the two or more user nodes store respective user models, and each user model corresponds to a part of a prediction model; the method comprises the following steps: receiving a prediction request aiming at identification information of an object to be predicted, which is sent by a prediction node, wherein the prediction node comprises one of more than two user nodes or nodes except the more than two user nodes; acquiring characteristic data from local based on identification information of an object to be predicted; obtaining a local prediction result at least based on the user model and the characteristic data; and returning the local prediction result to the prediction node.

One of embodiments of the present specification provides a distributed prediction system, where the method corresponds to a certain user node of two or more user nodes, where the two or more user nodes store respective user models, and each user model corresponds to a part of a prediction model; the system comprises: the prediction request receiving module is used for receiving a prediction request aiming at the identification information of an object to be predicted, which is sent by a prediction node, wherein the prediction node comprises one of more than two user nodes or nodes except the more than two user nodes; the local prediction module is used for acquiring characteristic data from the local based on the identification information of the object to be predicted and obtaining a local prediction result at least based on the user model and the characteristic data; and the local prediction result returning module is used for returning the local prediction result to the prediction node.

One of the embodiments of the present specification provides a distributed prediction apparatus, including at least one processor and at least one storage device, where the storage device is configured to store instructions, and when the instructions are executed by the at least one processor, the distributed prediction method performed by a user node according to any one of the embodiments of the present specification is implemented.

Drawings

The present description will be further explained by way of exemplary embodiments, which will be described in detail by way of the accompanying drawings. These embodiments are not intended to be limiting, and in these embodiments like numerals are used to indicate like structures, wherein:

FIG. 1 is a schematic diagram of an application scenario of a machine learning system according to some embodiments of the present description;

FIG. 2 is an interaction diagram of a model training method in accordance with some embodiments of the present description;

FIG. 3 is an interaction diagram of a distributed prediction approach, according to some embodiments of the present description;

FIG. 4 is a schematic diagram of a user model for user node A, shown in accordance with some embodiments of the present description;

FIG. 5 is a user model diagram of a user node B in accordance with some embodiments of the present description;

FIG. 6 is an exemplary block diagram of a model training system in accordance with some embodiments of the present description;

FIG. 7 is an exemplary block diagram of a distributed prediction system, shown in accordance with some embodiments of the present description;

FIG. 8 is an exemplary block diagram of a distributed prediction system in accordance with some embodiments of the present description.

Detailed Description

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only examples or embodiments of the present description, and that for a person skilled in the art, the present description can also be applied to other similar scenarios on the basis of these drawings without inventive effort. Unless otherwise apparent from the context, or otherwise indicated, like reference numbers in the figures refer to the same structure or operation.

It should be understood that "system", "device", "unit" and/or "module" as used herein is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, other words may be substituted by other expressions if they accomplish the same purpose.

As used in this specification and the appended claims, the terms "a," "an," "the," and/or "the" are not intended to be inclusive in the singular, but rather are intended to be inclusive in the plural, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that steps and elements are included which are explicitly identified, that the steps and elements do not form an exclusive list, and that a method or apparatus may include other steps or elements.

Flow charts are used in this description to illustrate operations performed by a system according to embodiments of the present description. It should be understood that the preceding or following operations are not necessarily performed in the exact order in which they are performed. Rather, the various steps may be processed in reverse order or simultaneously. Meanwhile, other operations may be added to the processes, or a certain step or several steps of operations may be removed from the processes.

Fig. 1 is a schematic diagram of an application scenario of a machine learning system according to some embodiments of the present disclosure. As shown in fig. 1, the machine learning system 100 may include more than two user nodes 110, a central node 120, and a network 140.

User node 110 may refer to a device of a data provider that provides data needed for a training phase and a prediction phase. In some embodiments, the data provider may include one or more of an enterprise, organization, individual, and the like. During the training phase, the user node 110 may provide sample data to the central node 120. Different user nodes 110 may provide different feature items for the same sample identification information (e.g., sample ID). After the central node 120 has trained the model, each user node 110 may obtain a user model from the central node 120, where the user model of each user node 110 corresponds to a portion of the complete model. Similarly, in the prediction phase, different user nodes 110 may also provide different feature items of identification information (e.g., ID of the object to be predicted) for the same object to be predicted. For the identification information of the same object to be predicted, each user node 110 inputs the feature item held by itself into the user model and obtains the prediction result output by the user model, and finally the prediction result of the object to be predicted can be obtained based on the prediction result output by the user model of one or more user nodes 110.

The central node 120 may refer to a computing device with a trusted execution environment on the platform side, where the platform side may refer to a party that coordinates participation of more than two data providers in a joint training. In some embodiments, the platform side may refer to a government, an enterprise, an organization, and so on. The central node 120 may train out a model in the trusted execution environment based on sample data provided by each user node 110 and split the model into multiple user models. Wherein the splitting may be performed for a structure and/or parameters of the model. In turn, the central node 120 may distribute the user model to each of the user nodes 110. The trusted execution environment can effectively ensure that data is not leaked, so that the user privacy in sample data of all parties can be effectively protected by performing model training in the trusted execution environment.

In some embodiments, the models referred to herein may include any of linear regression models, decision trees, neural networks, and the like.

It should be noted that the user node 110 needs to encrypt the sample data before providing the sample data to the central node 120, so as to prevent data leakage during transmission. Further, the central node 120 may decrypt the sample data and perform model training in the trusted execution environment.

Notably, the machine learning system 100 can include a prediction node 130. The prediction node 130 may be configured to initiate a prediction request for identification information of an object to be predicted, and obtain a prediction result for the identification information of the object to be predicted. In some embodiments, any of the user nodes 110 may act as a predictive node 130. In some embodiments, the prediction node 130 may be a node other than two or more user nodes 110, such as the central node 120 or an external node of the machine learning system 100.

In some embodiments, one or more nodes mentioned in this specification may be a single device of a party or may belong to a device cluster of a party. In some embodiments, the cluster of devices may be centralized or distributed. In some embodiments, the cluster of devices may be regional or remote.

Network 140 may facilitate the exchange of data and/or information. In some embodiments, one or more components of machine learning system 100 (e.g., user node 110, central node 120, predictive node 130) may send data and/or information to other components of machine learning system 100 via network 140. In some embodiments, the network 140 may be any type of wired or wireless network. For example, network 140 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the internet, a Local Area Network (LAN), a Wide Area Network (WAN), a Wireless Local Area Network (WLAN), a Metropolitan Area Network (MAN), a Public Switched Telephone Network (PSTN), a bluetooth network, a ZigBee network, a Near Field Communication (NFC) network, the like, or any combination thereof. In some embodiments, network 140 may include one or more network access points. For example, the network 140 may comprise wired and/or wireless network access points, such as base stations and/or a plurality of internet switching points 140-1, 140-2, through which one or more components of the voting system 100 may connect to the network 140 to exchange data and/or information.

FIG. 2 is an interaction diagram of a model training method according to some embodiments of the present description. It should be noted that although fig. 2 only shows a case where two user nodes correspond, the training principles of the present specification are equally applicable to a case where three or more user nodes correspond. The method is executed by a central node and can comprise a data acquisition step, a data decryption step, a model training step, a model splitting step and a user model distribution step.

In the data acquisition step, acquiring the encrypted sample data from more than two user nodes.

As shown in fig. 2, user node a and user node B send respective encrypted sample data to the central node. It should be appreciated that encrypting the sample data may ensure the security of the sample data during transmission.

And decrypting the encrypted sample data in the trusted execution environment in the data decryption step.

Since the plaintext of the sample data is obtained after decryption, decryption can be performed in the trusted execution environment in order to prevent the decrypted sample data from being leaked.

It should be appreciated that one or more steps of the processes shown in any of the embodiments of the present description may be performed in a trusted execution environment in order to prevent the leakage of data related to user privacy. For example, as shown in fig. 2, the model training step may be performed in a trusted execution environment, and reference may be made specifically to the description of the model training step in this specification. For another example, in some embodiments, the model splitting step may be performed in a trusted execution environment, and reference may be made specifically to the description of the model splitting step herein.

In the step of model training, the decrypted sample data is utilized to carry out model training in a trusted execution environment, and a trained model is obtained.

Similarly, since sample data is used in the model training process, the model training may be performed in a trusted execution environment in order to prevent sample data from leaking in the model training process.

In the model splitting step, the trained model is split to obtain the user models of more than two user nodes.

In some embodiments, the central node may split the model parameters of the trained model bitwise based on feature items of sample data of more than two user nodes. The bitwise splitting refers to splitting model parameters (hereinafter referred to as user parameters) associated with feature items of sample data of a user node at a position to obtain a user model of the user node, wherein the user model has the split user parameters.

Taking the linear regression model as an example, for any user node, the central node may split the parameter multiplied by the feature item of the sample data of the user node to obtain the user model of the user node. That is, the user model of the user node has a product term in which the feature term of the sample data of the user node is located. In addition, in some embodiments, the constant term of the linear regression model may be split into two or more sub-constant terms as constant terms of the user models of two or more user nodes, respectively. In still other embodiments, the prediction nodes other than the two or more user nodes may also store constant values used for calculating the prediction values, and the constant terms of the user models of the two or more user nodes and the sum of the constant values may be equal to the constant term of the trained model. For example, assume that: the trained linear regression model is as

Wherein Y represents a predicted value,

a feature item representing sample data of the user node a,

feature item representing sample data of user node B, a ₁ 、a ₂ 、a ₃ 、b ₁ 、b ₂ C are model parameters of the linear regression model and c is a constant term, the user model of the user node A can be in the form of

The user model of the user node B may be shaped as

Wherein, Y _a 、Y _b Predicted values of user models, a, respectively representing user nodes A, B ₀ 、b ₀ Respectively, represent constant terms of the user model of user node A, B. In some embodiments, c ═ a may be satisfied ₀ +b ₀ . Accordingly, the prediction result Y ═ Y for the identification information of the object to be predicted _a +Y _b . In some embodiments, if the predicted node other than two user nodes holds a constant value c required for calculating the predicted value ₀ Then c ═ a can be satisfied ₀ +b ₀ +c ₀ . Accordingly, the prediction result Y ═ Y for the identification information of the object to be predicted _a +Y _b +c ₀ 。

Taking a decision tree as an example, the decision tree may include decision nodes that decide which branch (e.g., left child node/right child node) to enter and leaf nodes that decide the final decision result (i.e., predicted result). In the prediction, starting from the root node, each time a branch is selected based on only the feature item and the parameter corresponding to the current node, where the feature item and the parameter (e.g., a threshold) may constitute a determination condition for determining which branch to enter. And after the branch is selected, entering the next child node connected with the selected branch. And repeating the steps until the leaf node is reached, and obtaining a predicted value. In some embodiments, splitting the decision tree may satisfy: for any user node, the user model of the user node has a complete structure, and if the feature item corresponding to the decision node does not belong to the user node, the decision condition (including the feature item and the parameter) of the decision node is unknown to the user node. In this way, any user node may obtain a decision result of one or more decision tree nodes on a path from the root node to the leaf node, where the decision result of a local decision tree node on the path may be obtained based on its own user model and locally obtained feature data, and the decision results of other decision tree nodes on the path may be obtained by initiating a prediction request for identification information of the same object to be predicted to other user nodes, so as to obtain a prediction result based on the decision result corresponding to the one or more decision tree nodes on the path. In some embodiments, the final decision result (i.e., prediction result) corresponding to any leaf node in the user model may be known to any user node. In still other embodiments, the final decision result (i.e., predicted result) corresponding to any leaf node in the user model may be known to only one user node. With regard to splitting of the decision tree, reference may also be made to fig. 4, 5 and their associated description.

In some embodiments, the central node may perform an operation splitting on the model parameters of the trained model. The operation splitting may mean that the model parameter slices corresponding to the split user nodes and the model parameters of the model before splitting satisfy a certain operation relationship, so that the model parameters may be split based on the operation relationship.

Taking a neural network as an example, the split user models of more than two user nodes may be neural networks with the same structure, where the structure may include one or more of the number of layers, the number of nodes in each layer, an activation function of each node, a connection relationship between each node, and the like. However, the user models of more than two user nodes may have different neural network parameters, i.e., the model parameter slices of more than two user nodes split by the operation.

It should be noted that the present specification does not set any limit on how to split the model, and the model splitting rule satisfying the following conditions can be regarded as the disclosure scope of the present specification: there is a joint prediction rule matching the model splitting rule, so that the prediction result of each split user model and the joint prediction rule can obtain the same or equivalent (e.g. approximately same) prediction result of the pre-splitting model. In particular, reference may be made to the contents of this description relating to split and joint prediction of the same model.

It is worthy to note that in some embodiments, as shown in FIG. 2, the model splitting step may be performed in a trusted execution environment. Therefore, leakage of the trained model in the splitting process can be prevented, and in addition, leakage of the characteristic item of the sample data can be prevented when the model is split based on the characteristic item of the sample data of each user node.

In the model distribution step, the user models of more than two user nodes are respectively sent to each user node.

In some embodiments, the central node may encrypt each user model, and then send each encrypted user model to each user node. And after receiving the encrypted user model, the user node decrypts the user model. Therefore, the safety of the trained model in the transmission process can be ensured.

One or more steps disclosed herein relate to encryption/decryption of interactive data, e.g., encryption/decryption of sample data and encryption/decryption of user models. For this purpose, the central node and the user nodes may agree on an encryption/decryption algorithm and a key before interacting with data. In some embodiments, an asymmetric encryption/decryption algorithm may be adopted, the central node and the two or more user nodes may generate respective public and private key pairs before data interaction, and the central node and the user nodes respectively send their own public keys to each other. Based on this, either one of the central node and the user node can encrypt the data by using the public key of the other party before sending the data to the other party, and then the other party can decrypt the received encrypted data based on the private key of the other party.

After the user model is distributed to each user node, the user node can participate in joint prediction by utilizing the user model of the user node. FIG. 3 is an interaction diagram of a distributed prediction method in accordance with some embodiments of the present description. As can be seen from the foregoing, each user model corresponds to a portion of the predictive model. The prediction model may be a model trained based on sample data of more than two user nodes. With respect to the obtaining methods of the prediction model and the user model, reference may be made to the description about the model training method in the foregoing embodiments. It should be noted that although fig. 3 shows only two user node correspondences, the prediction principles of the present specification are equally applicable to three or more user node correspondences.

In some embodiments, the node that initiates the prediction request and ultimately obtains the prediction result may be referred to as the prediction node. The prediction node may be one of the user nodes, or may be another node other than the user node. When the prediction node is a node other than the user node, the prediction node does not participate in joint prediction, only needs to send a prediction request containing identification information of the object to be predicted to the related user node, receives a local prediction result returned by each user node, and obtains a prediction result aiming at the identification information of the object to be predicted based on each local prediction result. When the prediction node is one of the user nodes, the prediction node is also involved in joint prediction besides initiating a prediction request and summarizing local prediction results. Differences will be described below.

When executed by a prediction node, the distributed prediction method may include a prediction request sending step, a local prediction result receiving step, and a prediction step.

In the prediction request sending step, a prediction request for identification information of an object to be predicted is sent to one or more user nodes.

In some embodiments, the identification information of the object to be predicted may include an ID of the object to be predicted. In some embodiments, the ID may include a name, a cell phone number, an identification number, a serial number, and the like. The prediction node may send a prediction request to each user node having the characteristic data of the object to be predicted. When the prediction node does not participate in joint prediction, the prediction node needs to send prediction requests to more than two user nodes. When the prediction node participates in joint prediction, the prediction request may be sent to only one other user node, and of course, the prediction request may also be sent to a plurality of other user nodes.

In the local prediction result receiving step, local prediction results of one or more user nodes are received, and the local prediction results are obtained at least based on the user model of the corresponding user node and the characteristic data of the identification information of the object to be predicted.

With regard to the detailed steps of the user node obtaining the local prediction result, reference may be made to the following description of the local prediction step performed by the user node.

In the prediction step, a prediction result of the identification information of the object to be predicted is obtained based on the local prediction result of at least one user node.

It should be noted that one or more user nodes perform joint prediction based on the identification information of the same object to be predicted.

In some embodiments, the local prediction result of any user node corresponds to a portion of the prediction result for the identification information of the object to be predicted. Accordingly, the prediction node may obtain a prediction result for the identification information of the object to be predicted based on local prediction results of two or more user nodes.

Specifically, when the prediction node is one of two or more user nodes, the prediction node may initiate a prediction request for identification information of an object to be predicted to other user nodes. After each user node receives the prediction request, feature data corresponding to the identification information of the object to be predicted is obtained from the local, a local prediction result is obtained based on the user model and the locally obtained feature data, and the user nodes except the prediction node return the local prediction result to the prediction node. Further, the prediction node may obtain a prediction result for the identification information of the object to be predicted based on the local prediction result of itself and the local prediction results of other user nodes (i.e., local prediction results of two or more user nodes).

When the prediction node is a node other than two or more user nodes, the prediction node may initiate a prediction request for the identification information of the object to be predicted to all the user nodes. After each user node receives the prediction request, feature data corresponding to the identification information of the object to be predicted is obtained from the local, a local prediction result is obtained based on the user model and the locally obtained feature data (in the process, the local prediction is called as local prediction), and then the local prediction result is returned to the prediction node. Furthermore, the prediction node may obtain a prediction result for the identification information of the object to be predicted based on local prediction results of two or more user nodes.

For example only, the flow of joint prediction by the prediction node cooperating with multiple user nodes may be as shown in fig. 3, where the prediction node sends prediction requests for X (X represents identification information of an object to be predicted) to the user node a and the user node B, respectively. And after receiving the prediction request aiming at the X, the user node A and the user node B respectively carry out local prediction. User node A obtains local prediction result S _A And will predict the result S locally _A Sending to the prediction node, the user node B obtains the local prediction result S _B And will predict the result S locally _B And sending the data to the prediction node. Furthermore, the prediction node bases on the local prediction result S _A And local prediction result S _B A prediction result S for X is obtained. It can be understood that the above flow is directed to a scenario in which the prediction node is a node other than the user node, and when the prediction node is one of the user nodes, the prediction flow is substantially similar, except that the prediction node performs local prediction like other user nodes, and the prediction node determines a final prediction result based on the local prediction result of itself in addition to the local prediction result of other user nodes.

In some embodiments, the prediction result of the identification information for the object to be predicted may be obtained based on local prediction results of a plurality of user nodes. For example, the prediction node may add local prediction results of two or more user nodes to obtain a prediction result for the identification information of the object to be predicted, and refer to the relevant content of the linear regression model in this specification in detail. For another example, for a neural network, each user node may compute and split locally owned feature data for an object to be predicted and distribute the locally owned feature data to other user nodes, each user node may compute a segment of a final prediction result based on its own feature data segment and the obtained feature data segment, and a user model parameter by a secret sharing algorithm and send the segment to the prediction node, and the prediction node may add local prediction results (the segments) of each user node to obtain a prediction result for identification information of the object to be predicted. For another example, the prediction node may find an intersection from local prediction results of more than two user nodes to obtain a prediction result for the identification information of the object to be predicted.

Taking the decision tree as an example, the prediction result corresponding to the leaf node can be obtained by performing decision branches based on the feature data corresponding to the node on all decision nodes on the path from the root node (one of the decision nodes) to the leaf node. According to the related description of fig. 2 in this specification, any user node has a complete model structure, but some decision node information (such as feature items and/or their segmentation thresholds) is unknown. A user node may determine, based on feature data of identification information for an object to be predicted that the user node has, all leaf nodes that may be reached through a local user model (e.g., a decision tree), and use the leaf nodes as local prediction results of the user node. In some embodiments, the prediction node may send a prediction request to all other user nodes, receive local prediction results returned by all other user nodes, and intersect the local prediction results returned by each user node, thereby obtaining a final prediction result.

The following describes an example of a process in which the user node a and the user node B obtain a prediction result for identification information of an object to be predicted (e.g., a driver) in conjunction with fig. 4 and 5. For example only, the user node a has a user model as shown in fig. 4, which has three decision nodes and four leaf nodes, wherein two pieces of decision node information, namely "driving age" and "number of traffic violations" are known, and one piece of decision node information is unknown. The user node B has a user model as shown in fig. 5, which has three decision nodes and four leaf nodes, wherein the information of one decision node, i.e., "number of adventure-year-out" is known, and the information of the other two decision nodes is unknown. It will be understood that user node a and user node B each have portions of the same decision tree. The two methods need combined prediction to obtain the final prediction result.

And after receiving the identification information aiming at the object to be predicted, the user node A acquires the driving age of the driver and the traffic violation frequency information from the local. If the driving age of the driver is 2 years and the number of times of traffic violation is 0, determining that the threshold condition of the root node is not met based on the model shown in FIG. 4, and moving to the right; and the information of the next decision node is unknown, and the prediction result may go to the left or to the right, so that the user node a determines that the leaf node 3 and the leaf node 4 may arrive.

And after receiving the identification information aiming at the object to be predicted, the user node B locally acquires the number of times of the last year of the driver. If the number of times of the driver's adventure in the last year is 1, it can be determined based on the model described in fig. 5 that the information of the root node is unknown, the prediction result may go left or right, if left, the leaf nodes 1 and 2 may be reached, and if right, the leaf node 3 may be reached.

The "danger" for the driver's final predicted result as leaf node 3 may be determined by intersecting the predicted nodes with the local predicted results of user node A, B.

With continued reference to fig. 3, for a user node (e.g., user node a or user node B) participating in prediction in response to a prediction request from a prediction node, the execution thereof may include a prediction request receiving step, a local prediction step, and a local prediction result returning step.

In the prediction request receiving step, a prediction request for identification information of an object to be predicted, which is sent by a prediction node, is received.

In the local prediction step, the characteristic data is obtained from the local based on the identification information of the object to be predicted, and the local prediction result is obtained at least based on the user model and the locally obtained characteristic data.

In some embodiments, the user node inputs the locally obtained feature data into the user model, so as to obtain a local prediction result output by the user model, for example, a user model obtained by splitting a linear regression model, a user model obtained by splitting a decision tree, and the like. The process of local prediction with respect to a user model based on splitting a linear regression model or splitting a decision tree can be found elsewhere in the text.

In some embodiments, the user node may obtain local predicted results based on the user model, the feature data, and predicted intermediate results from other one or more user nodes. Taking a neural network as an example, for the same identification information of an object to be predicted, each user node can split own feature data into N (N is an integer not less than 2 and indicates the number of more than two user nodes) feature data fragments, respectively send N-1 feature data fragments to other N-1 user nodes, obtain an input fragment of a prediction model based on the remaining feature data fragments of itself and the feature data fragments from other N-1 user nodes, and cooperatively calculate an output fragment of the prediction model (i.e., a local prediction result of the user node) with the input fragments and the parameter fragments of other N-1 user nodes by using a secret sharing algorithm and using parameters of the input fragment and the user model of itself. Further, the output of the predictive model (i.e., the final prediction result) may be derived based on the output slices of the N user nodes. And for any user node, one or more intermediate calculation results received from other user nodes in the cooperative calculation process can be regarded as predicted intermediate results.

And in the local prediction result returning step, returning the local prediction result to the prediction node.

For more details of the prediction request receiving step, the local prediction step, and the local prediction result returning step, reference may also be made to a description about a part of the prediction result of the local prediction result of any user node corresponding to the identification information for the object to be predicted in the foregoing embodiment.

It should be noted that the above description of the flow is for illustration and description only and does not limit the scope of the application of the present specification. Various modifications and alterations to the flow may occur to those skilled in the art, given the benefit of this description. However, such modifications and variations are intended to be within the scope of the present description.

FIG. 6 is an exemplary block diagram of a model training system in accordance with some embodiments of the present description. The system 600 corresponds to a central node. In some embodiments, the system 600 may include a data acquisition module 610, a decryption module 620, a training module 630, a model splitting module 640, and a model distribution module 650.

In some embodiments, the data acquisition module 610 may be used to acquire encrypted sample data from more than two user nodes. With regard to specific embodiments of the data acquisition module 610, reference may be made to the associated description of the data acquisition steps. That is, the data acquisition step may be performed by the data acquisition module 610.

In some embodiments, the decryption module 620 may be used to decrypt encrypted sample data in the trusted execution environment. With regard to the specific implementation of the decryption module 620, reference may be made to the associated description of the data decryption step. That is, the data decryption step may be performed by the decryption module 620.

In some embodiments, the training module 630 may be configured to perform model training in the trusted execution environment by using the decrypted sample data, so as to obtain a trained model. In some embodiments, the model comprises at least one of a linear regression model, a decision tree, and a neural network. With respect to specific embodiments of the training module 630, reference may be made to the relevant description of the model training steps. That is, the model training step may be performed by training module 630.

In some embodiments, model splitting module 640 may be configured to split the trained model to obtain a user model for more than two user nodes. In some embodiments, the central node may split the model parameters of the trained model in bits or split the operation based on the feature items of the sample data of more than two user nodes. With respect to specific embodiments of the model split module 640, reference may be made to the relevant description of the model split step. That is, the model splitting step may be performed by model splitting module 640.

In some embodiments, model distribution module 650 may be configured to send user models for more than two user nodes to each user node separately.

With respect to specific embodiments of the model distribution module 650, reference may be made to the relevant description of the model distribution steps. That is, the model distribution step can be performed by model distribution module 650.

In some embodiments, the model training system may also include an encryption module. In some embodiments, the encryption module may be configured to encrypt each user model before the model distribution module sends the user models of the two or more user nodes to each user node, respectively.

FIG. 7 is an exemplary block diagram of a distributed prediction system in accordance with some embodiments of the present description. The system 700 corresponds to a predicted node. More than two user nodes respectively store respective user models, and each user model corresponds to a part of the prediction model. In some embodiments, the system 700 may include a prediction request sending module 710, a local prediction result receiving module 720, and a prediction module 730.

In some embodiments, the prediction request sending module 710 may be configured to send a prediction request for identification information of an object to be predicted to one or more user nodes. With regard to the specific embodiment of the prediction request transmission module 710, reference may be made to the relevant description of the prediction request transmission step. That is, the prediction request transmission step may be performed by the prediction request transmission module 710.

In some embodiments, the local prediction result receiving module 720 may be configured to receive local prediction results for one or more user nodes, the local prediction results being obtained based on at least a user model of the corresponding user node and feature data of identification information for an object to be predicted.

With regard to specific embodiments of the local prediction result receiving module 720, reference may be made to the relevant description of the local prediction result receiving step. That is, the local prediction result receiving step may be performed by the local prediction result receiving module 720.

In some embodiments, the prediction module 730 may be configured to obtain a prediction result for the identification information of the object to be predicted based on local prediction results of one or more user nodes. In some embodiments, the prediction module 730 may be further configured to obtain the prediction result based on local prediction results of more than two user nodes.

With regard to specific embodiments of the prediction module 730, reference may be made to the relevant description of the prediction step. I.e., the prediction step may be performed by the prediction module 730.

In some embodiments, the model may include at least one of a linear regression model, a decision tree, and a neural network.

FIG. 8 is an exemplary block diagram of a distributed prediction system in accordance with some embodiments of the present description. The system 800 corresponds to a user node of more than two user nodes. More than two user nodes respectively store respective user models, and each user model corresponds to a part of the prediction model. In some embodiments, the system 800 may include a prediction request receiving module 810, a local prediction module 820, and a local prediction result returning module 830.

In some embodiments, the prediction request receiving module 810 may be configured to receive a prediction request for identification information of an object to be predicted, which is sent by a prediction node, where the prediction node may include one of two or more user nodes or a node other than two or more user nodes.

With regard to specific embodiments of the prediction request receiving module 810, reference may be made to the relevant description of the prediction request receiving step. That is, the prediction request receiving step may be performed by the prediction request receiving module 810.

In some embodiments, the local prediction module 820 may be configured to locally obtain feature data based on identification information of an object to be predicted, and obtain a local prediction result based on at least a user model and the feature data. In some embodiments, the local prediction module 820 may be further configured to obtain a local prediction result based on the user model, the feature data, and the predicted intermediate results from the other one or more user nodes.

With respect to specific embodiments of the local prediction module 820, reference may be made to the relevant description of the local prediction step. That is, the local prediction step may be performed by the local prediction module 820.

In some embodiments, local prediction result return module 830 may be used to return local prediction results to the prediction node.

With respect to the specific implementation of the local prediction result returning module 830, reference may be made to the relevant description of the local prediction result returning step. That is, the local prediction result returning step may be performed by the local prediction result returning module 830.

It should be understood that the systems and modules thereof shown in FIGS. 6-8 can be implemented in a variety of ways. For example, in some embodiments, the system and its modules may be implemented in hardware, software, or a combination of software and hardware. Wherein the hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory for execution by a suitable instruction execution system, such as a microprocessor or specially designed hardware. Those skilled in the art will appreciate that the methods and systems described above may be implemented using computer executable instructions and/or embodied in processor control code, such code being provided, for example, on a carrier medium such as a diskette, CD-or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system and its modules in this specification may be implemented not only by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but also by software executed by various types of processors, for example, or by a combination of the above hardware circuits and software (e.g., firmware).

It should be noted that the above description of the system and its modules is for convenience only and should not limit the present disclosure to the illustrated embodiments. It will be appreciated by those skilled in the art that, given the teachings of the system, any combination of modules or sub-system configurations may be used to connect to other modules without departing from such teachings. For example, in some embodiments, decryption module 620 and training module 630 disclosed in fig. 6 may be different modules in a system, or may be a single module that implements the functionality of both modules. For another example, in some embodiments, the local prediction module 820 disclosed in fig. 8 may be a module, and may also be decomposed into a local feature data obtaining module and a local prediction result obtaining module, where the local feature data obtaining module is configured to obtain feature data from a local area based on the identification information of the object to be predicted, and the local prediction result obtaining module is configured to obtain a local prediction result based on at least the user model and the feature data. Such variations are within the scope of the present disclosure.

The beneficial effects that may be brought by the embodiments of the present specification include, but are not limited to: (1) the central node with the trusted execution environment performs centralized model training, so that the privacy of sample data of all parties can be protected, and higher training speed can be guaranteed; (2) each user node carries out prediction (namely distributed prediction) locally, local characteristic data of an object to be predicted do not need to be transmitted externally, and data privacy of each party can be protected. It is to be noted that different embodiments may produce different advantages, and in different embodiments, any one or combination of the above advantages may be produced, or any other advantages may be obtained.

Having thus described the basic concept, it will be apparent to those skilled in the art that the foregoing detailed disclosure is to be considered merely illustrative and not restrictive of the embodiments herein. Various modifications, improvements and adaptations to the embodiments described herein may occur to those skilled in the art, although not explicitly described herein. Such modifications, improvements and adaptations are proposed in the embodiments of the present specification and thus fall within the spirit and scope of the exemplary embodiments of the present specification.

Also, the description uses specific words to describe embodiments of the description. Reference throughout this specification to "one embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic described in connection with at least one embodiment of the specification is included. Therefore, it is emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, some features, structures, or characteristics of one or more embodiments of the specification may be combined as appropriate.

Moreover, those skilled in the art will appreciate that aspects of the embodiments of the present description may be illustrated and described in terms of several patentable species or situations, including any new and useful combination of processes, machines, manufacture, or materials, or any new and useful improvement thereof. Accordingly, aspects of embodiments of the present description may be carried out entirely by hardware, entirely by software (including firmware, resident software, micro-code, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as "data block," module, "" engine, "" unit, "" component, "or" system. Furthermore, aspects of the embodiments of the present specification may be represented as a computer product, including computer readable program code, embodied in one or more computer readable media.

The computer storage medium may comprise a propagated data signal with the computer program code embodied therewith, for example, on baseband or as part of a carrier wave. The propagated signal may take any of a variety of forms, including electromagnetic, optical, and the like, or any suitable combination. A computer storage medium may be any computer-readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code located on a computer storage medium may be propagated over any suitable medium, including radio, cable, fiber optic cable, RF, or the like, or any combination of the preceding.

Computer program code required for operation of various portions of the embodiments of the present description may be written in any one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C + +, C #, VB.NET, Python, and the like, a conventional programming language such as C, VisualBasic, Fortran2003, Perl, COBOL2002, PHP, ABAP, a dynamic programming language such as Python, Ruby, and Groovy, or other programming languages, and the like. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or processing device. In the latter scenario, the remote computer may be connected to the user's computer through any network format, such as a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet), or in a cloud computing environment, or as a service, such as a software as a service (SaaS).

In addition, unless explicitly stated in the claims, the order of processing elements and sequences, use of numbers and letters, or use of other names in the embodiments of the present specification are not intended to limit the order of the processes and methods in the embodiments of the present specification. While various presently contemplated embodiments of the invention have been discussed in the foregoing disclosure by way of example, it is to be understood that such detail is solely for that purpose and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover all modifications and equivalent arrangements that are within the spirit and scope of the embodiments herein. For example, although the system components described above may be implemented by hardware devices, they may also be implemented by software-only solutions, such as installing the described system on an existing processing device or mobile device.

Similarly, it should be noted that in the preceding description of embodiments of the specification, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more embodiments of the invention. This method of disclosure, however, is not intended to imply that more features are required than are expressly recited in the claims. Indeed, the embodiments may be characterized as having less than all of the features of a single embodiment disclosed above.

For each patent, patent application publication, and other material, such as articles, books, specifications, publications, documents, etc., cited in this specification, the entire contents of each are hereby incorporated by reference into this specification. Except where the application is inconsistent or conflicting with the present disclosure, as may be the case with the broadest limitation of the claims that follow (whether present or appended to the present specification). It is to be understood that the descriptions, definitions and/or uses of terms in the accompanying materials of this specification shall control if they are inconsistent or contrary to the descriptions and/or uses of terms in this specification.

Finally, it should be understood that the embodiments described herein are merely illustrative of the principles of the embodiments of the present disclosure. Other variations are possible within the scope of the embodiments of the present description. Thus, by way of example, and not limitation, alternative configurations of the embodiments of the specification can be considered consistent with the teachings of the specification. Accordingly, the embodiments of the present description are not limited to only those embodiments explicitly described and depicted herein.

Claims

1. A method of model training, wherein the method is performed by a central node, the method comprising:

acquiring encrypted sample data from more than two user nodes;

decrypting the encrypted sample data in a trusted execution environment;

performing model training by using the decrypted sample data in a trusted execution environment to obtain a trained model;

splitting the trained model to obtain user models of the more than two user nodes;

and respectively sending the user models of the more than two user nodes to each user node.

2. The method of claim 1, wherein before sending the user models of the two or more user nodes to the corresponding user nodes, respectively, further comprising:

each user model is encrypted.

3. The method of claim 1, wherein splitting the trained model to obtain user models for the two or more user nodes comprises:

splitting the model parameters of the trained model according to the bit based on the characteristic items of the sample data of the more than two user nodes; alternatively, the first and second electrodes may be,

and carrying out operation splitting on the model parameters of the trained model.

4. The method of claim 1, wherein the model comprises at least one of: linear regression models, decision trees, and neural networks.

5. A model training system, wherein, corresponding to a central node, the system comprises:

the data acquisition module is used for acquiring encrypted sample data from more than two user nodes;

the decryption module is used for decrypting the encrypted sample data in a trusted execution environment;

the training module is used for carrying out model training by using the decrypted sample data in the trusted execution environment to obtain a trained model;

the model splitting module is used for splitting the trained model to obtain the user models of the more than two user nodes;

and the model distribution module is used for respectively sending the user models of the more than two user nodes to each user node.

6. The system of claim 5, wherein the system further comprises:

and the encryption module is used for encrypting each user model before the model distribution module respectively sends the user models of the more than two user nodes to each user node.

7. The system of claim 5, wherein the model splitting module is further to:

8. The system of claim 5, wherein the model comprises at least one of: linear regression models, decision trees, and neural networks.

9. A model training apparatus comprising at least one processor and at least one memory device, the memory device being arranged to store instructions which, when executed by the at least one processor, carry out the method of any one of claims 1 to 4.

10. A distributed prediction method, wherein the method is executed by a prediction node, the prediction node is one of two or more user nodes, or a node other than the two or more user nodes and is in communication connection with the two or more user nodes, the two or more user nodes respectively store respective user models, each user model corresponds to a part of a prediction model, and the user models are obtained by the model training method according to any one of claims 1 to 4; the method comprises the following steps:

sending a prediction request for identification information of an object to be predicted to one or more user nodes;

receiving local prediction results of one or more user nodes, wherein the local prediction results are obtained at least based on user models of corresponding user nodes and characteristic data of identification information aiming at an object to be predicted;

and obtaining a prediction result aiming at the identification information of the object to be predicted based on the local prediction results of one or more user nodes.

11. The method of claim 10, wherein the deriving a prediction result for identification information of an object to be predicted based on a local prediction result of the at least one user node comprises:

and obtaining the prediction result based on the local prediction results of more than two user nodes.

12. The method of claim 10, wherein the model comprises at least one of: linear regression models, decision trees, and neural networks.

13. A distributed prediction system, wherein the system corresponds to a prediction node, the prediction node is one of two or more user nodes, or a node other than the two or more user nodes and is in communication connection with the two or more user nodes, the two or more user nodes respectively store respective user models, each user model corresponds to a part of a prediction model, and the user models are obtained by the model training method according to any one of claims 1 to 4; the system comprises:

a prediction request sending module, configured to send a prediction request for identification information of an object to be predicted to one or more user nodes;

the local prediction result receiving module is used for receiving local prediction results of one or more user nodes, and the local prediction results are obtained at least based on the user model of the corresponding user node and the characteristic data of the identification information of the object to be predicted;

and the prediction module is used for obtaining a prediction result aiming at the identification information of the object to be predicted based on the local prediction results of one or more user nodes.

14. The distributed prediction system of claim 13, wherein the prediction module is further to:

15. The distributed prediction system of claim 13, wherein the model comprises at least one of: linear regression models, decision trees, and neural networks.

16. A distributed prediction apparatus comprising at least one processor and at least one memory device for storing instructions which, when executed by the at least one processor, carry out the method of any one of claims 10 to 12.

17. A distributed prediction method, wherein the method is performed by a user node of two or more user nodes, the two or more user nodes holding respective user models, each user model corresponding to a portion of a prediction model, the user models being obtained by the model training method according to any one of claims 1 to 4; the method comprises the following steps:

receiving a prediction request aiming at identification information of an object to be predicted, which is sent by a prediction node, wherein the prediction node comprises one of more than two user nodes or nodes except the more than two user nodes;

acquiring characteristic data from local based on identification information of an object to be predicted;

obtaining a local prediction result at least based on the user model and the characteristic data;

and returning the local prediction result to the prediction node.

18. The method of claim 17, wherein said deriving a local prediction based on at least a user model and said feature data comprises:

obtaining the local prediction result based on a user model, feature data, and predicted intermediate results from other one or more user nodes.

19. The method of claim 17, wherein the model comprises at least one of: linear regression models, decision trees, and neural networks.

20. A distributed prediction system, wherein the method corresponds to a user node of two or more user nodes, the two or more user nodes holding respective user models, each user model corresponding to a portion of a prediction model, the user models being obtained by the model training method of any one of claims 1-4; the system comprises:

the prediction request receiving module is used for receiving a prediction request aiming at the identification information of an object to be predicted, which is sent by a prediction node, wherein the prediction node comprises one of more than two user nodes or nodes except the more than two user nodes;

the local prediction module is used for acquiring characteristic data from the local based on the identification information of the object to be predicted and obtaining a local prediction result at least based on the user model and the characteristic data;

and the local prediction result returning module is used for returning the local prediction result to the prediction node.

21. The system of claim 20, wherein the local prediction module is further to:

22. The system of claim 20, wherein the model comprises at least one of: linear regression models, decision trees, and neural networks.

23. A distributed prediction apparatus comprising at least one processor and at least one memory device for storing instructions which, when executed by the at least one processor, carry out the method of any one of claims 17 to 19.